Evaluations

[d1ref2b]

I am a veteran member of a FL association, that has a declining number of members. We need to honestly evaluate our officials without making them angry and causing them to quit. It the past our evaluation system has been such to not hurt feelings. This does not help anyone get better.
I would appreciate any input on what other associations are doing to honestly evaluate it officials

[rainmaker]

Originally posted by d1ref2b
I am a veteran member of a FL association, that has a declining number of members. We need to honestly evaluate our officials without making them angry and causing them to quit. It the past our evaluation system has been such to not hurt feelings. This does not help anyone get better.
I would appreciate any input on what other associations are doing to honestly evaluate it officials.

I have received a number of, shall we say, "Less Than Stellar" evaluations, and I tend to be a little emotional and touchy. Here's how my association has kept me around, and avoided hurting my feelings.

1) All evals are strictly private. Strictly. No one knows what's in them except me and the person who wrote them, and the commissioner. No scores, ratings or values are released to anyone at any time (except the voting for state tournaments).

2) Game assignment lists aren't available to most of us. In order to find out who's doing the best you have to show up at the best games and see who's there.

3) When evaluators have needed to say things that could be hard to hear, they have been careful and tried to stay non-judgmental. "I think you could have skipped that call." "You need to run a little faster, and stay ahead of the play (that was 7th grade girls!)" "Another year or two under your belt is going to make a big difference in your self-confidence." "Go over section 2.10 again -- you're still a little hazy on the fine points."

4) They let the commissioner be the fall guy. "Well, regardless of what you and I may want, Howard won't give you the best games if you don't do it his way."

5) When people complain and say they should be getting better games, an evaluator is sent out to watch and then goes over the eval afterwards so the person really knows what needs work. Remember, this is private.

6) Howard NEVER agrees with a coach, parent or fan against a ref, and everyone knows it. He may get pretty cagey about what he says, and he may be pretty firm with a ref after the complainant is gone, but he never turns on us. We KNOW we will never be belittled by anyone whose opinion matters, no matter how lousy we are. That's worth a lot in my book.

[dblref]

In my association, non-varsity officials initiate the rating process. We have a form to fill out and the person requesting the rating has the responsibility of putting down what they think they did right and what they did wrong and sending it to one or more of the varsity officials that watched some of their game. The varsity official offers their input, sends a copy back to the requestor, and sends a copy to the commissioner. We have some non-varsity officials that either won't initiate the process or aren't exactly to open about putting down things they did wrong. These are the individuals that complain about not advancing. The varsity official can and should offer comments even if the non-varsity official did not initiate the rating. If the non-varsity official is not receptive to the "advice/pointers", then so be it.

Each varsity official rates his/her partner(s) and the rating forms are turned in to the commissioner. We use an A, B, or C rating with A being top notch, B being (that's awkard) good partner, but needs to work on such & such, and C being "don't ever want to work with them again".

Like Juulie, I have received my share of "what were you watching" type of ratings. I always ask for feedback and I always, always, always listen to what the individual says whether I agree or not. You can get some really good feedback by saying something like..."I appreciate the info and I will work on my weaker points".

[Edited by dblref on Oct 2nd, 2003 at 05:33 AM]

[zebraman]

We have tried a number of things. The most effective way so far seems to be to hire a group of independent evaluators (former officials who have state-wide respect among referees) and paid them to give written evaluations to our officials. It's EXPENSIVE so we can only afford to do it every few years, but worth it. I felt as if I received input even more beneficial than what I got at camp (by the way, many of our evaluators work camps too) because it was in "real" game settings and I never knew they were there so I wasn't trying to ref for observers.

One thing about getting observed by several independents is that you tend to believe the feedback when you see the same thing mentioned by each of them. :-)

Z

[DownTownTonyBrown]

Originally posted by dblref
In my association, non-varsity officials initiate the rating process. We have a form to fill out and the person requesting the rating has the responsibility of putting down what they think they did right and what they did wrong and sending it to one or more of the varsity officials that watched some of their game. The varsity official offers their input, sends a copy back to the requestor, and sends a copy to the commissioner. We have some non-varsity officials that either won't initiate the process or aren't exactly to open about putting down things they did wrong. These are the individuals that complain about not advancing. The varsity official can and should offer comments even if the non-varsity official did not initiate the rating. If the non-varsity official is not receptive to the "advice/pointers", then so be it.

I like this. This is a good idea. Some don't want any input or are unwilling to accept it from certain (unrespected) sources. The one desiring evaluation should initiate the process of improvement - Great! I perform a great deal of evaluations concerning ski patroller performance and I nearly always begin with "How do you think you did?" It works well because people tend to be their own, strictest critics. It makes the evaluator's job much easier if he can be positive and encouraging after the rookie has already been critical of themself.

varsity official rates his/her partner(s) and the rating forms are turned in to the commissioner. We use an A, B, or C rating with A being top notch, B being (that's awkard) good partner, but needs to work on such & such, and C being "don't ever want to work with them again".

How well does the system work with only Good (A), Bad (B) and Really Ugly (C) for the available grades? Seems that most would receive B's because rarely do we feel secure enough on the ladder of advancement that we would rate someone as better than ourselves (A) or give a rating that might advance that person above ourself.

Like Juulie, I have received my share of "what were you watching" type of ratings. I always ask for feedback and I always, always, always listen to what the individual says whether I agree or not. You can get some really good feedback by saying something like..."I appreciate the info and I will work on my weaker points".

Aaaah the grocery cart method... just because I go to the grocery store doesn't mean I have to buy everything they have.... I take what I need. So it goes with evaluations - take what you can use.

Has anyone looked into the book being offered now through Referee Magazine - something about methodologies for evaluating officials? I'm planning to order this book but would be interested in anyone's review.

Our association is in need of an evaluation system. I have heard the oft quoted "The only thing worse than not having an evaluation system, is having one." And for the life of me I cannot remember his name, previous D1 official, I attended an IAABO camp that he instructed several years ago... Mickey Crowley! So if and when we put one together, it needs to be done correctly.

[Edited by DownTownTonyBrown on Oct 2nd, 2003 at 12:20 PM]

[Camron Rust]

Originally posted by dblref

Each varsity official rates his/her partner(s) and the rating forms are turned in to the commissioner. We use an A, B, or C rating with A being top notch, B being (that's awkard) good partner, but needs to work on such & such, and C being "don't ever want to work with them again".

Our association used to have this sort of system. It was changed just after I started. From all I've heard...What really happened is that buddies gave buddies the top scores and gave everyone else a lower score. It didn't really reflect ability or performance.

Now there is a committee of 8-10 people that are selected to do the evaluations. There still could be some favoritism but I think it is largely minimized by the evaluation chairman selecting good evaluators.

[dblref]

Originally posted by Camron Rust

Our association used to have this sort of system. It was changed just after I started. From all I've heard...What really happened is that buddies gave buddies the top scores and gave everyone else a lower score. It didn't really reflect ability or performance.

Now there is a committee of 8-10 people that are selected to do the evaluations. There still could be some favoritism but I think it is largely minimized by the evaluation chairman selecting good evaluators.

We also have the "good old boy" system to some extent. All ratings are turned in to the association VP (also heads up the rating committee) and he prepares a listing of the ratings according to what was reported. The official knows where s/he ranks on the list. The rating committee then breaks them down into basically the 3 categories. This list is used somewhat to determine playoff selections. Officials in the lower part of the ratings may be "invited to attend the summer session"...this is a varsity league that we use to look at officials to move from the non-varsity level to what we call the "swing list". These officials are "qualified" to do varsity level, but will work mostly frosh & JV games. Because of the number of schools we cover, a swing official (and even a varsity official) will work a freshman game (4:00 pm) at one school and then work a varsity game (7:30) at another school. I do this quite often because I can leave work as early as I need to. My boss and I have been together for years and he doesn't care as long as the report is finished on time.

[IowaMike]

I am getting ready to enter my third year of basketball officiating and I have never been formally evaluated by anyone. I belong to an association but to my knowledge they don't have a formal process for evaluating officials. Of course, I get very few games through the association, I get most of them on my own through school district assignors and athletic directors. I would actually appreciate an honest evaluation of my skills. I have worked some soph/frosh games with the occasional very experienced varsity official and have received some good advice and largely favorable comments about my performance, but how honest are they going to be when they are working with you? Whenever I am working with a much more experienced partner, I always tell them before the game to give me any feedback they have on my performance so that I can get better, so I hope that they are being honest.