Then I think you handled it as best as YOU could. Your partner, on the other hand, needs a few lessons in game management.
You gave your partner plenty of chances to retain his authority on the field. You didn't butt in right away. You gave him time to explain the ruling (maybe a little too much time). You didn't undermine him.
He undermined himself, and you had no choice but to nudge the game along.