DO NOT feel guilty about throwing an abusive fan out. If you do not fix the situation, the rest of your games will go to pot. I stop the game, get the sheriff or game administrator to escort them out. They will not behave that way again. They are very embarrassed to be kicked out.
I have found that I do not call as well if I have a fan or a bad coach jawing me the whole game. I like the above comment that said to go ahead and chunk the kid on his fifth foul. That way his parents have no reason to wait on you. He can ride with his parents who just got escorted out.
Have you guys noticed that no matter how good an official is, if the game sucks itself, it is harder to call?