[OverAndBack]

I can see that, but the scenario was that the referee, who's watching the quarterback, sees the ball thrown to an unoccupied area.

The wing on that side might concur. But it's possible that neither of them saw what someone on the other side saw, which was the receiver being held up (this might be on a timing route).

In that case, while the initial call from the referee (who's not watching the other side) might be intentional grounding, that flag could be picked up if the official on the other wing comes in and says "well it was thrown to an unoccupied area because the defensive back on my side was holding the receiver."

I doubt you'll see that, but that was the scenario. But the basic idea is that you can't have pass interference on an illegal forward pass, and I'm okay with that.