I'm sure the official is a fine official and a fine person, but...
It's the appearance of impropriety that causes the problem. There's a reason why at the NCAA level the ECAC asks us to fill out a form to show *any* relationship you may have with any of its member colleges (alum, kids, parents, relatives who are employees, relatives who are alums, etc). There's a reason why we have scratch lists at the HS level. Don't give anyone an excuse to question your integrity.
I gave up a GV playoff game this year because it involved my alma mater (my mistake for not making the school my one and only scratch) and I'm more than a few years removed from HS. It wasn't worth the potential headache.