Charged defensive conferences...

[Zepp]

I think the key is who removes the pitcher. If the coach removes the pitcher, then it's not a charged conference.
If the coach returns to the dugout, then it is the umpire who removes the pitcher and the EFFECT is enforced. (pitcher can't return to pitch)

The easy solution has already been mentioned... preventive umpiring.
When the coach calls time, simply ask the coach "who's coming in to pitch?"
This will make the coach make the change. The EFFECT of the charged conference rule shouldn't come into play.

There's no need to go lookin for trouble. It usually finds us anyway.