Team without possession calls time out during live ball

[Philz]

Team A on offense was in there front court moving towards the basket when I heard time out requested. Like a rookie I blew my whistle before I realized it was by Team B the defense that called the time out . I tried to correct it by giving the ball back to team A for a throw-in but my partner (with more experience) stopped me and said it was an inadvertant whistle and we need to go to the possession arrow which went to team B. I dont feel that was the right or fair call and couldnt find it in the book. We were playing NCAA rules. Can anyone give me a reference for this?