If there was a ref in the stands yelling at me about my calls, I'd smack that jacka$$ upside the head with a frying pan.
Or - maybe give him one of these.
Also, I'd eject him from the gym and if he didn't leave, have him arrested for trespassing, which we can do under state law here in Oregon.