I'll echo what Felix said. This is best left to the site administrator if s/he is available. If not, the home team coach is the next level.
Here's my story from a few years ago:
Working a JV match as the umpire. The site adminstrator had introduced himself when we entered the gym and told us that he would be back and forth between the volleyball game in the gym and the football game out on the field. My partner on the stand was not one of our best
VB officials.
After the match was well under way, about 12-15 of the varsity football players came in the gym to watch. They were very vocal, cheering for their team, but nothing out of line. My partner made a call that they did not agree with, so all of the football player started chanting the word "motion", however the chant sounded like the word "bullsh*t". I quickly scanned the gym for the administrator, but did not see him. I walked over to where these students were seated, said that I was glad they were here supporting their team, but profanity would not be tolerated. I got the "what profanity?" and "who, us?" type responses. I told them that I knew what they were saying, but I also knew what it sounded like and I didn't want to hear it again.
As I turned to go back to the match, one of the kids decided to be funny and chanted "motion" again just loud enough to be heard. I turned back to the group and asked them to leave the gym. They weren't happy, but they left.
After the match, the site adminstrator came over to me. I was about to let him know what had happened when he said the he saw what I did and he thought I handled it well.I asked him where he was and he told me that he was sitting on the top row of the gym watching the whole incident. I told him that he really should have stepped in and dealt with it as crowd behavior is his responsibility. He just kind of blew me off and walked away.
I did file a report with our state association - don't know if anything ever came of it, though.