You just brought to mind a situation that occurred at a rec-league game that we had to miss.
Visiting team has one or two loud and obnoxious fans. At least as bad as OP. I think what sparked the following was a close call at first.
PU, finally getting enough tells a fan to shut up or he'll throw him out. Visiting coach reportedly tells the fan "Don't worry, he can't do anything do you." [Really helpful, wasn't she
]. Anyway, the fan follows her advice and keeps haranging the ump, who then tells him he has to leave. Eventually fan leaves and game resumes.