intentional swing and miss in SP

[Tap]

I had a blowout adult slowpitch league game recently and the count was 3-1 in favor of a batter on the team way ahead (the blowout status is probably irrelevant). The pitch was inside and low and hit the ground to the side of the plate, nearly between the batter's feet. The batter -- obviously wanting to hit -- intentionally "swung" and missed, which was expected as the bat was just below shoulder level during the "swing." I called it a ball. He stayed and complained that he swung. I repeated that it was a ball and it was not a swing, and I told him to go to 1B. The defense also wasn't happy with me. An unpopular call with all 20+ players on both teams.

I'm 99% sure that he started his purported "swing" before the ball hit the ground, so technically it wasn't dead before he "swung", but he waited until the ball had dropped well out of any possible hitting zone. My reasoning is that he was making a mockery of the game and had no intention to hit the ball, thus he did not swing.

How would everyone else have handled this?