First off, driving together makes for a very good pregame...something many officials spend too little time on as it is...so that is a bonus.
Second, part of our jobs as veteran officials is to mentor our less experienced members, so IMO, you should be watching a large part of that JV game anyway, and they in turn should be watching yours...riding together, bonding, pre-gaming, talking about interesting plays, watching each other work, and breaking down the games on the ride home is a great way to strengthen your association.
So what were you complaining about again?