Aside from JRut's points, which are all valid, there is something to be said for just soldiering on. I have noticed that breaks are fickle beasts, and even the most deserving officials sometimes don't get them when they deserve them, sometimes never at all. Sometimes equally deserving, or even less deserving officials get their breaks long before we do. (Of course, sometimes they get their breaks for really good reasons that we're not ready to understand yet) And then there's the room at the top problem. In order to start taking games away from the folks who are already there, you can't just be as good, you've got to be better than they.
And none of that is really going to make you feel any better, I know.
Which brings us back to where I started: sometimes you just need to soldier on and wait patiently for your time. It would be easy to get bitter and quit, but who would that hurt? Only you. So hang in there.
My wife likes to say that the secret to patience is knowing what to do in the meantime. Perhaps in the meantime you need to find your joy from some other aspect of this game. Maybe mentoring a newcomer or two. Maybe helping out some struggling church or rec. league. Or maybe just avert your watchful stare from your not-yet-boiling pot and focus on enjoying your reffing and the games you are getting. I took part of last season off, and it did wonders for my game, my perspective, and my life balance. I'm not suggesting you take part of a season off, only that there is sometimes something wonderful to be gained by stepping back from our ambitions for a time.
Good luck. One foot in front of the other.