Ratings through Arbiter

[SC Official]

Originally Posted by ilyazhito

I remember that you said in a different thread that peer evaluation is useless because most people give their peers 9s and 10s. If the system is convoluted and antiquated, what alternative would you prefer that the SCBOA and SCHSL use?

Not to derail the thread, but since you asked.

The overarching problem in South Carolina is that the state office controls all the varsity assignments for football and basketball. The reason is simply because it has always been this way, and SC is a relatively small state (~200 high schools), so one central contact point isn't too burdensome (i.e. people don't see the reason to change). We have to have a way to rank the officials so the booking office has something to go by. Obviously this system is stupid and I don't need to give reasons. The ideal solution would be for district directors to control all assignments within their districts, not just subvarsity games/holiday tournaments, like is done in surrounding states. Then directors could structure their own evaluation system to recommend their postseason officials. This is extremely unlikely to happen while I am still officiating.

With respect to the peer ratings, yes, very few people take them seriously. This inevitably leads to the statewide exam being the sole differentiator among officials. Currently, the rating system is structured such that on-court ability has nothing to do with one's assignments (which obviously is insane). We have to attend a camp every three years for 5% of our points, and that's it. We need observers to evaluate officials, but there is pushback on this because of cost and paranoid officials worrying about the objectivity of evaluators. I am all for rules knowledge and rewarding officials for getting in the book (and not rewarding officials who refuse to study the rules), but the reality is that we have too many officials working games they shouldn't be working simply because they test the best. It's unfortunate but that is the way it is. Eventually the coaches will raise enough hell that it will have to change, but that has not happened yet.

We do not have "conferences" in SC that have the power to hire/fire their own assigner; we have "regions" within each of the five classifications (1A-5A) that are realigned by the SCHSL every two years, so a college-type system where officials can work for whomever will hire them will not happen here. Most of the private schools in SC compete in a separate league that has their own way of dealing with officiating, so that's an option and many good officials that struggle under the current structure in the SCHSL/SCBOA do very well in the private school league (SCISA).

[deecee]

peer ratings are a waste of time. You either get all 9's or 10's or all crappy scores. It's hardly accurate and depends on whether you partner "likes" you or not. They should be taken with a grain of salt. Coaches reviews are even more worthless.

[JRutledge]

Originally Posted by deecee

peer ratings are a waste of time. You either get all 9's or 10's or all crappy scores. It's hardly accurate and depends on whether you partner "likes" you or not. They should be taken with a grain of salt. Coaches reviews are even more worthless.

I do not have an issue with a rating system, just take it for what it is worth. And in my state (Illinois) at least it is only one small factor amongst other things. Ratings are always going to be subjective and often flawed. We used to have a peer system involved with the coach's ratings, but a lot of officials would give high marks. Now I know I would rate what I felt about an official. And only Certified (the highest rank) officials were allowed to do those ratings.

Peace

[Raymond]

Originally Posted by deecee

peer ratings are a waste of time. You either get all 9's or 10's or all crappy scores. It's hardly accurate and depends on whether you partner "likes" you or not. They should be taken with a grain of salt. Coaches reviews are even more worthless.

All I could give a truly accurate rating on would be my partners' communication skills and hustle. I have enough to do on the court to be analyzing their mechanics, signals, positioning, and play-calling.

And I have worked with a bunch of officials who have no business trying to rate me.

[BillyMac]

Originally Posted by Raymond

... my partner ... I have enough to do on the court to be analyzing their mechanics, signals, positioning, and play-calling..

Agree. Before the most recent change in our rating system, all varsity officials would be required show up early to evaluate both junior varsity officials. Calmly observing the game from the bleachers for two, or three, periods was very conducive to a pretty accurate evaluation. As a retired teacher, who evaluated over three thousand students over the many years that I taught, I had no problems doing these evaluations.

Evaluating my varsity partner is another story, it's always been a problem for me. I've got more important things to do in a game other than observe my partner.

Originally Posted by Raymond

... bunch of officials who have no business trying to rate me.

Before the most recent change in our rating system, junior varsity officials were required to stick around for at least two varsity periods and evaluate the varsity officials. Yeah. Sure. A guy with a few years of experience can accurately evaluate a guy with dozens of years of experience working a highly charged varsity game. Tell me another fairy tale. Sure, some of the junior varsity veterans had been to a lot of rodeos, and could possibly do a decent evaluation, but overall, most junior varsity officials, in regard to evaluation, are in way over their heads.

After recent changes, we only do partner evaluations on Arbiter and these don't carry much weight in the overall evaluation process (how accurate can evaluations be when two inexperienced subvarsity guys rate each other in a middle school game, or a freshman game). It's the trained observer evaluations that carry the most weight when comes to one's overall varsity, or subvarsity, status.

I believe that my local board is making a big mistake by not requiring evaluations (on something other than Arbiter, we once had our own locally produced software that could do this) of junior varsity officials by the varsity officials who are already at the site, as we did in the past. These evaluations by veteran varsity officials can be a great resource in a rating system.

On the other hand, one great thing about the recent change is that junior varsity officials no longer evaluate varsity officials. I could never understand such foolishness, and utter nonsense.

[SRQREF]

Arbiter has an evaluation section. This requires the assigner to assign an evaluator to the game just like he/she assigns officials. Once the evaluator is assign, he/she will evaluate all the officials on the game and when the evaluations are complete the evaluations are:
1. Sent to the officials after completion.
2. Keep an electronic record of the evaluation within the Arbiter system.

Additionally, as an assigner I can review the following information regarding my evaluators:
1. Make sure my evaluators complete all the evaluations assigned
2. Review the average scores, individual scores completed by the evaluator
3. Compare evaluation scores between individual evaluators (make sure they are on the same page)
4. Evaluate the evaluators.

This is a very useful tool for official feedback and evaluator competency.