Was the defensive player still in contact with the ball too, or had he let go? I will have to look, but if the defensive player removes their hands from the ball and the offensive player returns to the floor while still holding the ball, then isn't this considered a traveling violation?
I don't believe it matters:
Rule 4, Section 35, NCAA rulebook
A.R. 95. A1 jumps for a try for field goal. B1 jumps to defend against the try and (a) touches the ball before it leaves A1’s hand and A1 returns to the floor with the ball and the ball never loses contact with A1’s hand(s) or (b) the ball loses contact with A1’s hand(s), A1 retrieves the ball while in the air and returns to the floor in possession of the ball and begins to dribble or (c) after the ball touches the floor, A1 recovers the ball and begins to dribble. RULING: In (a), the official shall call a held ball. In (b) and (c), the play shall be legal. A1 has gained a new possession in both instances.
It seems that in this specific case, A1 has gained new posession. Even though his hands were not above his head, as the OP stated, he was going up for the shot.
In almost no cases would it be travelling, unless the defender did not touch the ball. Even if the shooter came down with the ball after 1) the defender had touched the ball in mid-air, 2) the ball never lost contact with the shooter's hand, and 3) regardless of whether or not the defender still had contact with the ball when the shooter came down - it would be a held ball.
As for the original post, there would have been no hurt in making the correct ruling, an inadvertant whistle, and going from POI (assuming the whistle came after the shooter recovered the ball) -- and awarding the shooter's team the ball without going AP. Especially when you were 100% that the defender made contact with the ball, and your partner agrees he might have missed it. Maybe not all coaches understand that a ref can't be 100% time all the time on the spot, and if they ride your partner for it, then WACK!