Screwed up with no time elapsed

[Rizzo21]

Boys JV last night. I'm the R (two-man) and administer the jump ball. First of all, my partner lines up almost in a Umpire three-man position instead of right by the table. Oh well, I just go ahead and toss, they both get a piece of it and Team A jumper grabs the ball. In that instant, I'm thinking that this should be a violation but 1) the whistle's not back in my mouth yet and 2) by the time I get the whistle back in my mouth I'm thinking "isn't U1 (my partner) responsible for this violation?". A moment later a kid from Team B says "he can't do that" and I'm thinking "you're right, he can't but now it's too late to blow the whistle without looking like idiots". Then, as play continues I'm thinking "we probably look like idiots anyway, what a way to start the game". Actually, I didn't hear any more protests about it but it took a good minute to shake it off in my head. Later, my partner and I discussed it and both agreed we blew the call.

I don't have my official's manual with me. Is the U1 responsible for these types of violations? Not that it exonerates the R (me, in this instance) since I can still step in and make the right call, however, oftentimes you are just trying to stay out of the way.