Justified Technical?

Happened to me today, I would like your thoughts.

I had the same team twice in three hours. In the first game, team A won easily, but their coach was not very nice to me in his demeanor or words. I ignored most of the stuff he directed at me in the first game. Before the second game where I had his team again, I considered talking to him and telling him that while I was not going to let his actions "build up" or "carry over" from the first game, meaning I was going to treat the slate as fresh. I decided not too. (After I tell the rest, let me know if I should have talked to him before the second game.)

I communicated with my partner that this coach liked to talk at the officials, and off we went! The first half was ok, but during the second his comments started to pile up. He started yelling at my partner in the third quarter, and I gave him the stop sign and told him "that's enough coach!". In the fourth quarter he said sarcastically, "I guess you must be tired, what is this, like your fifth game today" -- saying that the reason for my poor (in his eyes) performance was that I was tired and not sharp enough. I warned him about staying in the coaches box, and not coming onto the floor, because he was getting in the way of my running up and down the court. He replied, "I'm on the court the whole time!!" This seemed to me like a major challenge. He also said "you are really good at the rules [aka the coaching box] that don't apply to the game". The ball was in play when he said the last one, so I waited until the next dead ball and gave him a technical. Should I have given it to him right away?

The thing is, what I have quoted him as saying may not seem that bad. However, the tone and manner in which he said these things was definitely negative and meant to demean/challenge me.

Thoughts?