Flagrant

[mlancast]

A1 is fouled as he shoots a layup. B1 ends up on the floor. A little pushing and shoving goes on. Hard to believe, but no official sees A1 kick B1....intentionally. What is seen is B1's retaliation....Throwing an errant punch at A1 as B1 attempts to get up.
Q: As an official, do you go by the book and assess B1 a Flagrant "T" for fighting, even though you know in your mind that B1's actions were a retaliation for something you did not see??

The player in this case was tossed. But as I think about it, I wonder if this was the right thing to do from a game management standpoint. I do not believe there is a right or wrong answer, just curious as to your opinions.