Batter abandons batters box

[Thaal]

I am the PU, 2 out.
First pitch is high inside and as I saw it, the batter pivots out of the path of the ball.
The ball gets away from the catcher, R3 scores.
The pitcher requested a replacement ball, which I gave him.
Because there was a brake in play, I indicate and called the count, 1Ball, No strikes.
After 2 more strikes the batter turned and ran to the dug out and the defensive team runs off.

My partner came over to me, and I asked him if I had missed a strike.
Yes, he said, the batter swung at the first pitch. As I had called the pitch a ball and I had not been asked to get help for the BU, my count of 1B & 2S was correct.

I believe I should have called the batter back and continued the count, I didn’t.

Is this correct, even though the opposition had taken the field.