Here's my cop-out answer for the week. I don't think you can take one judgment play out of context and decide if you should make a call or not. You have to put it in the context of what you had already established as foul contact during the entire game. If you had been calling this much contact all night, then it's a foul. If not, then not.
Of course, this could be your first contact situation of the game. Then - you're going to have to make a split second decision that will affect the contact calls for the rest of the game.
That's why we get the big bucks.