A charged defensive conference is for when a manager or team rep asks for suspension of play to deliver a message to a defensive player on the field. If he simply announces a pitching change to you, that's not a charged conference. That's, at most, a substitution.
Now, if the coach leaves the dugout to talk to the pitcher on the pitcher's plate, that might be a different story, and my instinct says that MIGHT be a charged conference just based upon how the rule is worded. However, I might be wrong, so I'll let the other guys pipe in. Honestly, I've not had this happen, nor have I been given instructions about this.
Fortunately, there's a national school coming near me sometime soon, and I'm pretty stoked about it.