I'm with IREF4HIM. I didn't realize it so much until this year, though. I started because it was some easy money in college. I say "easy" because I knew the game pretty well; as a player, I was more smart than talented.
After college, I continued because it really was fun. As I started to work up slowly (it was hard to work up the ranks when I took about every other year off), I found that the better the ball I was working, the more fun it was.
Now, I've discovered it's a great stress relief. I've found that when I'm particularly stressed at my regular job; getting out to officiate really releases a lot of pent up energy. It's cathartic.
I'm confident enough in my knowledge and abilities, when I catch the inevitable grief, I can laugh.
I know I've got the best view on the court, and I probably know the rules better than all but, at most, a half-dozen other people in the building. For me, this isn't about power as much as it's about "being right." This is probably the most shallow reason I enjoy it. For those of you who have taken the "colors" personality test, I'm a green, and officiating fits my style.