False start or motion?

[CO ump]

I was the WH on an underclass game this weekend and working with an unfamiliar crew.
Halfback obviously misses the count and makes quick move to right one count before snap. I flag for false start and kill the play. LJ and I have a good discussion at halftime regarding my call. He said that since A45 is allowed to be in motion and his move was sideways this was a legal move.
I contend that there's a difference between going in motion and missing the count, one is a quick move simulating a snap the other is a smooth, more relaxed move. The LJ suggested that FS only be called in this sitch if the defense jumps, I say that regardless of whether they jump they may have leaned forward then caught themselves and were moving back at the snap giving A unfair advantage.
Regardless of which way a back moves, if it's obvious he missed the count and makes that quick start of the play move I've always killed the play.
What do you guys think?