Stepping toward 1st from stretch - Balk?

[rinbee]

This is Junior Division baseball. Runner on 1B. RH pitcher pitching from the stretch places right foot against the pitcher's plate, feet spread apart. As he begins to bring his non-pivot foot in to go into his set position, he steps slightly toward 1B (about a 45 degree angle, about midway between HP and 1B) and then brings his non-pivot foot in next to the pivot foot as he goes into his set position. When he delivers the pitch he steps straight toward HP. Is this a balk? It is my understanding that anytime the pitcher is in contact with the P-plate and makes any movement toward 1B without delivering the ball to 1B it is a balk. Is this the case if he has not gone into his set position?