On what basis would you give him a time out?
If he's not given you any trouble all game, and he's just coaching his kids, I'd think about a second reminder. If he jumps back into his box, problem solved. If he doesn't, then the T is just that much more justified.
OTOH, if he's been a jerk, the hard part would be not smiling while you whack him.