How about an opinion:

[Rich]

Originally Posted by Kevin Finnerty

A peer review system is possibly even worse. The pettiness and jealousy involved in this kind of system kills any chance it has at being effective. If a superior or a small team of superiors is doing the evaluating, then it has a chance of yielding some accurate assessments and helpful results.

And that's not just in the umpire fraternity; peer reviews are similarly ineffective most everywhere else they're tried. It's the easy way out for an organization that doesn't want to take the time or make the effort at properly training and evaluating its employees.

Is it really? If you work a lot of games and you are evaluated fairly by most of the people you work with (assuming you're in the same peer group as your partner), you can dismiss the statistical outliers. If you're not in the same peer group as your partner, then the system can eliminate that evaluation completely -- and transparently, if the assignor or state association wishes.

Here, with coaches, they really aren't required to submit a rating. Some take the process seriously, others only rate when they want to "get even" with an official. I've never had a sport (other than football) in any season where more than 50% of the coaches even bothered submitting a rating.

In an association, peer evaluation can be made as a requirement (do it, or don't get paid). While some people will downgrade others to boost their own rankings and others will collude with regular partners to boost each other, those things can be easily spotted by those that look at the numbers.

It's better than a system where coach who gets ejected has the right to rate an umpire who's just doing his job.