Fed Balk ruling

[rbmartin]

I was a spectator (my son was playing) at a home varsity game yesterday. HP was a guy I had never worked with before but for the first 2 or 3 innings both he and his partner seemed to know their stuff (consistent zone, good communication with each other etc.).
Around the 3rd inning visitors get their first base runner. F1 (RHP) goes from the stretch for the first time in the game. With pivot foot on pitchers plate and the ball in his right hand behind his back, he is looking in for the sign. While doing this his glove hand is hanging down freely moving with a very slight, slow pendulum movement (probably 3 inches each way). HP cries "BALK". He tells F1 the pendulum motion was not allowed. Ball goes live again. "BALK" again. Same explanation. Coach tells F1 "If he's gonna call it you have to stop doing it! F1 complies. Game continues without incident.,
2 or 3 parents start heading my way asking me to explain the call. I am usually very hesitant to criticize one of my brethren in blue so I say something like "I guess he felt the movement constituted some sort of feint or something. It's a judgement call", but in the back of my mind I'm thinking "There's no way on earth I would call that a balk."

So my question is, would a slight pendulum motion with the free, glove hand while working from the stretch (as you look in for the sign) and before coming to the set position be considered a balk. I reread the book last night to make sure I wasn't missing something, but I just don't see it.