I let him get to me

[Mark Dexter]

I would suggest never mouthing/saying/indicating/etc. "shut up" or something similar to a fan - even if you know it's the head coach for the varsity team. If something's needed, just give him a stern look before the next inbounds pass or just stare him down during the next timeout (if the setup of the gym allows, and if you have the personality to do this).

I think the best solution (other than perhaps speaking to your assignor after the game) is to mention something to the JV coach (if you're comfortable with it), but more importantly to go out for the rest of the game, to not let his comments rattle you, and to prove that you don't belong at the JV level - you should be working his varsity games. (With my apologies for the run-on sentence.)