pass interference out of bounds

[don't move]

Had this last night: 2nd & 10 at B's 20. Deep pass to the back of the endzone. A80 and B20 are 3 yards beyond the endline (neither player was forced out) and B20 clearly interferes with A80. We did not throw a flag, (coaches, players and fans go nuts,etc) but I am not sure we got it right. The rule just says you cannot "interfere with an eligible opponent's opportunity to move toward, catch or bat the pass". While, by definition, A80 cannot "catch" the pass (because he cannot first contact the ground inbounds) he can still move toward the pass and was prevented from doing so by B20.

According to the rules, it looks like we blew it. Comments please.