[Adam]

Whether the player is dribbling or holding the ball, the defender may slap his hand provided the slap is incidental to an attempt to slap the ball. Dribbling vs holding is irrelevant.

Let's leave the punch out of it, as that's a flagrant foul regardless of advantage.
Let's leave the face out of it, as I'm calling a slap in the face unless the dribbler is in the process of blowing by the defender for a layup.

A slap on the hand is a slap on the hand, perfectly legal.

A slap on the wrist that doesn't cause any hinderance to the offensive player is incidental contact most of the time. Same as a slap on the arm or leg.

Now, in practice, we may have to get this sometimes in the open court.