I have seen a few umpires remove the mask with the right hand, only to transfer it to the left hand immediately.
I also have seen some professionals use both hands to remove it as well, probably taking their time to make sure they don't rip their hats off in front of 50,000 fans.
