Mark,
Start out by telling him that you were just making sure that he would have the opportunity to referee with you in the finals!
After he finishes saying things that would easily require a flagrant T, let him know that you actually did understand how much time was left in the game, and that you were calling the foul with nonsevere contact since his team needed to put the other team on the line to stop the clock. Remind him that if the "soft" foul is not called in this situation, then the next one will be harder to make sure that the whistle blows, and if you are not careful, here comes the fight....
Regardless, if the two of you work the finals together, make sure it is put behind you and have a great game. Congrats on the chance to work the game!!!!!