I can't find a rule cite but it has been custom that an inning begins at the third out of the previous half inning. The question to you would be why didn't the pitcher come out to warm up in a timely fashion? If it was a time wasting act you could have let the game go on, warn the manager to get the pitcher going or you would forfet the game for delay tactics. I assure you a threat of losing for delay would get you a pitcher in a hurry.