I don't think you "blew it" either. But, yes, it depends on the assignor.
I have an understanding assignor and I simply tell him there's no way I can get there in time (traffic in my case), and he moves right on to try to get me to fill the next opening.
My experience, limited that it is, is my assignor needs me to fill games, he knows I enjoy doing it, he knows I want to move up, and he knows I will accept a game if I can. I get the impression he would like to keep me "somewhat happy."
We're supposed to move to TheArbiter soon, so we'll see if that helps or hurts, as you can just block out times you can't make......