Ball out of play

[tzme415]

I'll clarify with the situation.

ASA Adult Men's Rec1 League
R1 on 1st, Batter hits grounder to F6 who flips ball to F4 who tries for double play. F3 misses the ball which rolls under a part of the fence where a gate is that allows it through. I immediately call dead ball and award BR 2nd as he had not touched first before the throw.

Captain tries to argue that BR is not entitled to advance as this is a problem with the field and the ball would not have gone out of play had the ground been properly filled by the gate. I just tell the captain that I am not responsible for field maintenance and the ball is out of play.