too late to call infield fly?

[scottk_61]

An infield fly can only apply when there are less than two outs (none or one) AND 1st and 2nd or 1st 2nd and 3rd are occupied.

Your sitch give us that.
However, the players are responsible for knowing the rules as well as you.
The infield fly happens whether you call it or not, you do have to enforce it.

You should have had batter out on the IF and every other runner can advance AT THEIR OWN RISK.
If they got put out on a tag, they are out, no tag=no out.

Yes, it is never too late to enforce the IF but that doesn't require the other runners to be returned to their bases.

Get in your rule book and read the sections one at a time until you learn them so as to not get taken in by goofy player/coach myths.