I blew this, but it took an inning to realize it

[Bishopcolle]

2-1 count to batter (weak batter). Next pitch, purposely high so catcher can make a play (very close to an out) on the lazy runner at first. After ball hits the glove, batter chops at the high pitch in obvious strike. Catcher stands high to grab the pitch, whips ball to first--safe there--and my partner obviously didn't see the VERY late swing by the batter because it was very late and the catcher stood high. Before the next pitch, PU announces the count as 3-1. I wait but nothing happens, and it is obvious to me my partner missed the swinging strike, so I called time, we had a conference, and we changed the count to 2-2 based upon my advice to him, and not because of any appeal....at the time, it felt right because we got it right, but then I got to thinking....No appeal was made, and I was wrong....it should have been on the defense to be game-aware and make the appeal.....Right?