Say you'll watch for it, maybe talk to both team captains/coaches if you think it's warranted - seems about all you can do.
I was working a (flag) football game once where the offensive line of one team (my co-workers
) kept coming up to me saying that a player on the opposite team (grad students in my department
) was continually trash-talking and swearing at them when she went through the line. I kept on telling them I didn't hear anything (they claimed she was doing it quietly) and that I wasn't going to flag what I couldn't hear. Wouldn't have been any different no matter who was playing.