I don't understand WHAT you ejected him for. So he yelled at you. I would have just walked away.

Now, if he would have followed you and said the same thing in the same manner, that would have been justified.

But tossing a guy because he screamed his arguement towards you as you were walking away from him? I don't get it. If they did that in pro ball, every other batter would be ejected.