By "lateral" movement we're not talking about the defender's right to move sideways. What that means is the defender has to move in the same direction (forwards/backwards/right/left) as the dribbler. I can't remember the exact wording in the rule book, but it may also mention speed as well.
The defender in this case indeed moves sideways, but not at all in the same direction that the ball handler is moving.
