Possible Holding

[RochesterRef]

OK gents here's my first question for you all this season:

This past Thursday I was HL at JV game between two evenly matched teams. With 2 minutes left and Team A trailing 26-21, they take over on their own 30. First play from scrimmage on the drive, out of the double wing they run a sweep to my side. A severely undersized lead blocker tries to make a block on a LB, but the linebacker blew the blocker up, driving him backwards. As A is falling backwards to the ground, he still is engaged with B and they both fall to the ground. Team Bs coach is screaming at me for holding, as team A breaks a huge run down to Bs 20. I say its not holding as it was Bs momentum that took him to the ground and not As block. Moral of the story is A got stuffed on the 4 yard line and B won the game anyways, but I was just looking for input on whether or not I handled that play correctly.