Ideas for setting up an evaluation process

[Geek]

Our basketball chapter is considering implementation of an evaluation process. Can anybody provide any poiters on what the benefits and pitfalls are?

[ChuckElias]

The benefit is that each official will know roughly where he or she stands in relation to the other officials on the board and that may give some guidance to whoever assigns your games.

The drawback is that your fellow officials will use the rating system to trash each other in the hopes of raising their own rating.

Even if that doesn't happen, members of your association will find problems with it and complain that it's unfair for one reason or another.

As a D1 assignor once said, "The only thing worse than not having a rating system is having one."

JMO

[zebraman]

Originally posted by Geek
Our basketball chapter is considering implementation of an evaluation process. Can anybody provide any poiters on what the benefits and pitfalls are?

If you mean strictly evaluation, in terms of giving officials feedback to improve, I can't think of many negatives.

If you mean a ratings system, you'll have benefits and drawbacks.

Benefits are that the officials will see where the rest of the officials see them in relation to the rest of the group. This may motivate them to go to camps, get videod, seek input from those rated higher and work harder to "get to the top." It also takes all the "heat" off of the assignor (or whomever it is that decides your rankings currently).

The drawbacks are that it will create cynicism and distrust between some officials. Even if you created the perfect system (which doesn't exist), low-rated officials would think that it's "all political" and blame their low ranking on the good old boys network. They'll think that officials above them are all "buddy-buddy" and working together to keep them down. Even top rated officials would find fault with the system when they dropped a couple spots, even if they remained near the top after a small drop.

Even with all that, having a peer ratings system sure beats a system where one person (often the assignor) or even a few people (often the board) decide ratings for an entire association.

Z

[Smitty]

A pure evaluation system is beneficial to everyone, I think. Our association requires that pre-varsity level officials get at least 10 evaluations per season, and they can be performed by any varsity level officials. There is a standard form template used for the evaluations.

Also, varsity level officials are randomly evaluated (we don't know when someone will be in the stands doing the evaluation) one or more times per season. I found this to be extremely useful.

I think it's a great way to try and get everyone on the same page, for the most part. I was told that the evaluations have nothing at all to do with assignments. There is a separate person (or committee, I'm not sure) that administers and processes the evaluations (not our assigner).

This is a great way for the newer officials to get some constructive feedback from real game situations. Once the pre-season scrimmages and clinics are done, there's really no easy way to monitor newer officials without a consistent type of evaluation.

For the veterans, it allows some constructive criticism from some senior or retired officials.

Of course, two different evaluators may have contradicting comments on certain things, but for the most part, it's a great tool.

[rainmaker]

Originally posted by Smitty
A pure evaluation system is beneficial to everyone, I think. Our association requires that pre-varsity level officials get at least 10 evaluations per season, and they can be performed by any varsity level officials. There is a standard form template used for the evaluations.

Also, varsity level officials are randomly evaluated (we don't know when someone will be in the stands doing the evaluation) one or more times per season. I found this to be extremely useful.

I think it's a great way to try and get everyone on the same page, for the most part. I was told that the evaluations have nothing at all to do with assignments. There is a separate person (or committee, I'm not sure) that administers and processes the evaluations (not our assigner).

This is a great way for the newer officials to get some constructive feedback from real game situations. Once the pre-season scrimmages and clinics are done, there's really no easy way to monitor newer officials without a consistent type of evaluation.

For the veterans, it allows some constructive criticism from some senior or retired officials.

Of course, two different evaluators may have contradicting comments on certain things, but for the most part, it's a great tool.

I agree that it's great to get evalled a couple times a year, and that it's good for the association to have a system in place. I also agree that there are some good things about our system. (Geek, Smitty and I are in the same association.) I do know for sure that not every varsity official gets evalled every year, though, and that the evals do have something to do with the assignments, at least sometimes.

[Smitty]

Originally posted by rainmaker

I agree that it's great to get evalled a couple times a year, and that it's good for the association to have a system in place. I also agree that there are some good things about our system. (Geek, Smitty and I are in the same association.) I do know for sure that not every varsity official gets evalled every year, though, and that the evals do have something to do with the assignments, at least sometimes.

I didn't believe for a second that the evals don't have anything to do with assignments, but that's what I was told. I sat with an evaluator once during a varsity game after my JV game. The guy he was evaluating was late to the game, acted like he was on drugs the whole game, and was in the stands talking to people during timeouts. It was a thing of beauty to watch this guy get evaluated.

[rainmaker]

Originally posted by Smitty

I didn't believe for a second that the evals don't have anything to do with assignments, but that's what I was told. I sat with an evaluator once during a varsity game after my JV game. The guy he was evaluating was late to the game, acted like he was on drugs the whole game, and was in the stands talking to people during timeouts. It was a thing of beauty to watch this guy get evaluated.

Feels good, doesn't it, for justice to happen? Care to e-mail details? YOu know I love to gossip.

I know that the biggest game I had last year, there was an eval in the stands, and I know if he'd said so, I'd have gotten a play-off game. Wish it hadn't been the day that my kid went into the ER vomiting his guts out, with a high fever and shaky knees. The eval on that game wasn't the greatest!

[Ron Pilo]

In the Seattle Girl's basketball group we have a formal evaluation system that consists of Varsity evaulation the JV officials that work the games before thiers.

The Varsity are evaluated by a group of hired evaluators who are former officials.

JV officials get 10-15 evals a season and the Varsity folks get 5.......


Everyone is evaluated using the same system and at the end of the year each official is ranked ordered based upon thier average score.....Varsity seperate from JV......the top 5 JV officials move to the Varsity and any Varsity official ranked in the bottom 5 of the varsity list for 2 years running are moved back to the JV list.

Varsity officials do not rate each other and neither do the JV officials.

Not a perfect system but it's works fairly well.

[Jurassic Referee]

Originally posted by ChuckElias

As a D1 assignor once said, "The only thing worse than not having a rating system is having one."

I'll guarantee you that assignor has his own rating system though. They all do. He's just not sharing it.

[ChuckElias]

Originally posted by Jurassic Referee

I'll guarantee you that assignor has his own rating system though. They all do. He's just not sharing it.

Good point and no doubt you're right. I think he was talking about being rated by peers, coaches, etc. Pretty sure he's got some idea of the quality of each of his officials. He's not pulling names out of a hat to assign to games

[Kelvin green]

WE have a fairly decent system

Each varsity person has to do 10 evals...

We have a standard format and then we put into a databse. We have kept track over time and it gives us a pretty good picture... go to http://www.uboa.org/ and you can see what we have done