I think you have to rely on advantage/disadvantage on this one. While a similar bump in the middle of the floor may have caused no disadvantage to the ballhandler, the bump at the sideline put him at a tremendous disadvantage. Passing on this call would reward the defense for fouling...and almost certainly earn you the wrath of the players, coach, and fans.
Another situation to consider. You will see a similar case where a bump will cause the ballhandler to fumble the ball oob, rather than fall oob with it. In this case, some people suggest that you call the oob and give it back to the offended team, even if they were the last to touch it. There is no basis for this in the rules, but I see it called that way quite a bit. And it seems to be a pretty fair compromise much of the time.
That should get some discussion going