Let's take a look at NFHS 4-19-1. I see a foul defined as "illegal contact....which hinders an opponent from performing normal defensive and offensive movements".

I don't see anything that says illegal contact requires flesh-on-flesh contact. Does forcefully using the ball to push an opponent away prevent that opponent from performing normal defensive movements? Yes.