Interaction with crowd

[Indianaref]

Being a new official I would like to get some feed back from some some of the more seasoned refs. I was doing my second game last night(4th grade boys), when in the first quarter, A1 shoots the ball from near the baseline and hits the back of the backboard. I whistle we are going the other way. A couple possessions later, B1 hits the side of his backboard, of course no whistle. One particular father sitting near the division line rants and raves that I am only calling it one way, I don't pay any attention to him the rest of the quarter while I concentrate on the game. I'm the ref, so just prior to the start of second quarter, while waiting for my partner to rouse up the teams, I decide (in a pleasant way)to give him a quick rules lesson. Naturally, he replies he saw it different, I just let it go from there. My question to you is should I have even tried reached out to communicate with him or just let it go? I will say I did not hear a word out of him for the rest of the game. At the end of the game I was happy because I did get a lot of people telling me that we did a good job. Any feed back would be greatly appreciated.