When should you 'notice' the fans?

[SC Ump]

I had a state play-off game last night, with bracket play having the visitors needing to win both. The visitors brought a number of fans who were really into it, like two guys with two signs of "D" and "+++" (a picket fence.) Toward the end of the game, some home team students showed up and started a little bit more agressive "cheering".

A coin toss on the second game had the local team batting first. I had the plate. The first pitch was taken right down the middle. When I called the strike, the local students went ballistic, "You suck!" "What game are you watching!" The usual stuff, but with a "we're here to yell at the umpire" attitude.

I just stayed in my game and concentrated on my mechanics so that I would be as distracted as little as possible.

As the top of the second was just getting ready to start, the "De-fense" guys started going. "De-fense! De-Fense!" After brushing the plate, I notice one of the local fans throw a plastic bottle at these guys. Not hard, not malaciously, but just a soft side arm toss. The visiting fan never saw where it came from. This caused some yelling and 'bowing up' between the fans.

Me, I just called, "play ball". I knew there was security at the field. (I could see him in the outfield sitting on the hood of a car.)

Fortunately things calmed down. But later, I starting thinking whether I should have halted the game at that point and contacted the security guy or perhaps the Atheletic Director. How would you folks have handled it?