How do you answer this?

[Adam]

I've been thinking about this past high school season, and one game in particular has been bugging me. Soph boys, and home coach has had close calls go the other way. After I whack one of his kids for swearing at me after a foul call, he asks me if it's personal. He asked me the same thing several times the rest of the game. "Is it personal?"
I essentially ignored the question, and finally said "No" when he asked me after the game was over.
Question. Should I have firmly told him to stop asking the quesion after the first time? Or did I handle it correctly?
Or, does the fact that the game went relatively smoothly (aside from having a tired partner who kept getting caught slightly out of position) mean I handled it as well as necessary?
I just don't like the idea that he even tried to make it about him.
Thanks,

Adam