paid evaluators

[rsl]

Our association has had a hard time getting good consistent evaluations. At meeting tonight it was suggested that we hire several paid evaluators to remedy the problem.

Does anyone have any experience with a good procedure for evaluating with paid evaluators?

[refprof]

it has to be better than having your partner evaluate you.

If he is looking at me, who's looking at the players?

[26 Year Gap]

Our assn is using two guys who are not calling games and paying them a game fee for each set of games [JV & varsity]. They will rate each official twice. Where I used to live had officials rate each other in games. Some guys harpooned others hoping to be better positioned for playoffs. Politics will always enter into ratings in some fashion. But having guys that are not competing for 'playoff assignments' seems to have 'good idea' written all over it.

[Back In The Saddle]

Originally Posted by 26 Year Gap

Our assn is using two guys who are not calling games and paying them a game fee for each set of games [JV & varsity]. They will rate each official twice. Where I used to live had officials rate each other in games. Some guys harpooned others hoping to be better positioned for playoffs. Politics will always enter into ratings in some fashion. But having guys that are not competing for 'playoff assignments' seems to have 'good idea' written all over it.

How can you realistically rate your partner? Unless my partner really screws up, or is calling a very different game than I am, I'm not paying much attention to him/her. Certainly not enough attention to be able to give a useful evaluation of his/her performance.

[jdw3018]

Originally Posted by Back In The Saddle

How can you realistically rate your partner? Unless my partner really screws up, or is calling a very different game than I am, I'm not paying much attention to him/her. Certainly not enough attention to be able to give a useful evaluation of his/her performance.

We've got the same process here of rating the officials we work with. In addition to the (major) problem that you mentioned, there has never been any guidance given on "how" to rate. It seems most officials I've asked about it start at 10 (we rate on 6 (I think) different categories, 1-10) in all categories, and then only mark down if you screw up.

However, there are a few officials out there who take a different approach, and consider a 7 a 'good' score. In a system that doesn't allow much differentiation because of a lack of education from the association, this can make one official's harsh grading of others pretty impactful. There are also, of course, those who will grade harshly to benefit their own ranking.

All that said, I still favor an official's rating system over a coach's rating system, which is how it was in the state I came from.

Best of all worlds, IMO, would be the paid evaluator system.

[iiicream]

I hung up my whistle 3 years ago. This year I was asked by our local association to come back and evaluate officials for which I would be compensated. I go out 2-3 nights a week to do both a JV and V game, so I get 6 evaluations in.

I have no horse in the race so I pretty much tell the officials what I see. I always find some positive in someone's game and let them know. I try to offer suggestions on positioning, rotations/switches, coverage area, reporting techniques, etc. If I have a question on judgment I usually ask the official what happend on that play. If a certain rule has blatantly been misapplied I will let the official know and suggest he/she review it.

Most of the officials are trying to move up the ladder and I want to be able to offer help on how to do that. The information I share with the official also goes to our Assigner & Board of Directors.

[jdw3018]

I'm curious, iiicream, do you have a specific rating system your association uses? Or do you just provide general feedback? Did you receive any training on how to observe/rate/provide feedback?

Just interested in how different areas are making this work...

[iiicream]

Originally Posted by jdw3018

I'm curious, iiicream, do you have a specific rating system your association uses? Or do you just provide general feedback? Did you receive any training on how to observe/rate/provide feedback?

Just interested in how different areas are making this work...

I use a general evalutaion form provided by our association. It has 9 areas to evaluate using a 1-below avg, 2-avg , 3-above avg.

The categories include: Consistency, game management, professionalism, reaction under pressure, feel for the game (judgment/knowledge), coverage, appearance, NFHS mechanics, NFHS signals.

I then check the official's overall ranking: above avg, avg, needs improvement.

I received no official training. Just using my 20+ years of officiating as a basis to rate the officials.
At the bottom I offer my comments and recommendation.

[jdw3018]

Sounds like a good process. I'm also a believer in the 1-3 type scale, rather than a 1-10 scale.

Nine areas - below, at, or above - plus your specific comments gives a lot of information. Thanks for sharing.

[amusedofficial]

I'm not a big fan of grades, since one evaluator's seven may be another's five, but a 1-2-3 based on below average, average and above average, in areas that mean something, would probably yield meaningful results, but comments should be mandatory, and evaluators should not have access to previous evaluations done by someone else, lest a comment made on an off-night become engraved in stone.

"On my ship, superior performance is standard; standard performance is substandard and substandard performance does not exist. I kid you not"

Phillip Francis Queeg prior to his retirement and second career as a girl's JV coach.