Jim D. thanks for the rule reference.
(This got long winded as I kept typing, sorry)
This starts to get into a judgement decision (which is why we get paid the big bucks
). The key word in that is
potential blocker. What does potential mean? If you are between me and the ball are you a potential blocker?
If you are running a drag route across the middle I'm still reading you as potential blocker. The defender may or may not realize that A is running a sweep and the drag route is really A moving to block downfield. A knows what play they are running, B doesn't, so B is given a higher degree of latitude for contact - until the ball is thrown.
I think that everybody on A is a potential blocker until that ball is actually thrown. After its thrown, everyone but the player who caught it starts to block again, which means they remain a potential blocker throughout the down (with the exception of when the pass is in flight - which is when B is restricted).
It may seem like I'm ignoring 9-2-3-d, but I'm not aware of potential blocker being defined and it doesn't specify a time period during the down. So if A can return to being a blocker later in the down, they are a potential blocker throughout the down.