Is it fan interference?

[WinterWillie]

Adult Men's Slow Pitch Game -ASA Rules

Bottom of the seventh, runners on second and third, team at bat trails by a run.
Batter fouls the first pitch into the woods behind the grandstand down the third baseline. Umpire hands new ball to catcher who throws to the pitcher. Pitcher steps on the rubber and pitches. Batter swings and just after the crack of the bat, a fan tosses the ball over the fence (the playing is entirely enclosed by chain link fence) that was previously fouled off in the direction of F5. F5 turns his head to look at the ball coming toward the field just as the lazer shot coming from the batter's bat zooms by F5's turned head into left field. Would you have let the play stand or would you have ruled fan interference?