'course ya gotta get there just a mite before actual game time...
I have held up the start of a game to deal with obvious playing field problems, but never for an outfield fence problem. That kind of stuff (even temp fences that have blown over) is just dealt with during the ground rules.
The only holes I've ever had filled in were mud holes!