[just another ref]

From the earliest ages of their players, when coaches teach man-to-man defense, they repeat various versions of the order: "Stay with your man!" "Stick with your man!" "Wherever he goes, you go with him!" "If he goes to the concession stand, you go with him!" (I think that last one was an exaggeration) Players routinely take their defensive position while the ball is dead, whether it be a place in a zone, or finding one's man. (Match up! Match up!) If the nose-to-nose posture is employed in a legal manner by a team during the game in an attempt to handle one or more players, I would find it difficult to deny them the right to maintain this position during a dead ball, regardless of the length of the pause.

Anticipating an extreme question, no I would not find it acceptable for a player to follow his opponent into the huddle during a time out.