Potential blocker or not?

[Illini_Ref]

I have been having this argument.

A1 throws a legal forward pass to A2 who is behind the LOS. A2 is not attempting to block and has turned toward the QB to receive a pass. B1 takes out A1 (no tackle).

Should B1 be flagged for Illegal Use of Hands?

I say yes, he says no. He maintains that A1 is a potential blocker. I say that once he is obviously attempting to catch a pass, he is no longer a blocker.

I know there is a case play and I have read it. I just wonder when a person ceases being a potential blocker.

If B1 hits A2 downfield before the ball is released then it is agreed it is a foul for IUoH. Why is this different?