Post Game Ejection ?

[Chess Ref]

It's a Boys frosh game. It's a game that needs to be managed . My partner, at best, is lazy and just doesn't care. He's got about 7 years experience of finding the hospitality rooms.

Visitors coach gets T'eed at halftime for swearing at us. He's pretty mellow the 2nd half. The game ends and partner disappears to the snack shack area. I wait for visitors to leave the court. Then I leave the court. Everyone uses the same corridor. As I enter the corridor the visiting coach is talking to someone and starts yelling"you're horrible" 'the worst ever"...I keep walking then he starts calling me a racist. I keep walking. Then he says" Now that the games over I can say or do anything to you that I want." Though the words weren't that bad , he definetly used a threating tone. So I chuckle and keep walking. He is literally about 1/2 my size. This little outburst of love was witnessed by a school security person and the home coach.

Question is could I have ejected him ?

Varsity ref says yes. I thought no because I have left the confines of the court, and we were in the locker room.

What say you ?