Have your attorney send this clown a "lawyer's letter" telling him that if his comments cost you the opportunity to earn money, you will sue him for slander and ask for treble damages in court.
If you think that's too intense, then just ambush the guy one night and stuff him in a wood chipper.
