To answer you're original question about being "thin skinned" - I actually think your skin is a little too thick
First of all, this is the ASSISTANT coach who just popped off. When he says "Get a Clue" - I would say "Get on the bus". There is noooooo way I am letting an ASSISTANT coach talk to me and/or my partner like that. In fact, if a Head Coach screamed that, I might dump him too (although restricting the Head Coach might be another option).
The problem with amateur umpires is that they let base coaches get away with that stuff and if you don't "nip it in the bud", you're just making it harder for the next crew that has to deal with him. Think about it, the Head Coach usually gets a little more rope than an Assistant Coach - so if you allow the base coach to scream at/towards you "Get a Clue"... then what are you going to allow the HC to say?
And to the one guy who said "let it go, you can deal with him later in the game"... that is how things blow up on you and then you'll end up saying "I should have gotten rid of him earlier". Take care of business and address matters promptly.