Its a foul.

[Mike Simonds]

I would have thrown my flag at the spot where the A player returned inbounds. Thats the spot where the foul occurred.

Bean bag or hat is dropped at the spot where he went out of bounds.

Penalty flag is dropped at the spot where he returns inbounds.

If it was an incomplete pass the defense could decline the penalty and move to the next down. Or perhaps they might want to accept the penalty to take the offense out of field goal range even though the down would be replayed.

You are giving an unfair advantage to A if you ignore this.