12 men on the field at the snap
 

Do you guys shut the play down or flag it and let the play go?

I am telling my guys to shut it down

All of that depends on when you notice it or if the time frame has been met. You can always have a live ball substitution foul.

Peace

Fed Rules:

It depends what you mean "on the field". If it is 12 in the formation, then yes it is a deadball foul for illegal sub. However, if you have the 12th player on defense running off the field in an unsuccessful substitution, then that foul does not occur until the snap.