Rating Systems?

[JRutledge]

Originally Posted by ajjl22

In Illinois, each coach and is supposed to rate you for each game on a 1 to 5 scale. if you work with a certified official (highest promotion level) they must also rate you, but only once a year.

1 = Qualified to work state final contests
2 = Qualified to work sectional contests
3 = Qualified to work regional contests
4 = Qualified to work varsity contests
5 = Qualified to work underclass contests only

The ratings are averaged and then divided by number of ratings and you are given a percentile ranking. They do this to award officials who work more games.

The problem with the system is that coaches do not do their ratings. I have work 35 varsity games this year. I should have 76 ratings (worked with 6 certified officials) I have a total of 22 ratings. This makes it look like I have worked only 8 games all year.

I think the point needs to be made that the ratings of coaches and officials are only 5 points in a overall 30 point power rating system. Next year there will be another 5 points added to for attending camps. You will get more points for attending a Level 2 camp as compared to a Level 1 camp. I also think in our state we focus way too much on the ratings when the ratings are only a small percentage of what gets us into the playoffs. Also our state is very aware that schools do not rate officials often enough and it might seem to us that this is going to affect us in a negative way. The reality is they are looking for new people to fill spots all the time, especially in baseball because of the lack of numbers. This is why they go out and watch umpires, look for recommendations from associations and if they observe you directly all those ratings will not mean much of anything. I had Anthony Holman tell me directly to have our members on the Top 15 list (in another association) to send the IHSA their schedule for the season. He wanted to take the opportunity to watch several umpires and find people that are not normally getting a shot at the post season. I also know that similar things are being done in other sports as well. Trust me on this, the right recommendation can get you on the radar. Once you are on the radar, it is up to you to prove you deserve to stay there. I know officials in many sports that were rated very high and did not get any playoff games. It was not their ratings that sealed the deal.

Peace

[nickrego]

If your going to have Coaches rate Umpires...

I think only a games WINNING coach should submit a rating.

I think we all know that when a team loses, it is primarily the fault of the crappy umpires. Especially that slug behind the plate !

[JRutledge]

Originally Posted by nickrego

If your going to have Coaches rate Umpires...

I think only a games WINNING coach should submit a rating.

I think we all know that when a team loses, it is primarily the fault of the crappy umpires. Especially that slug behind the plate !

In our state this is not at all true. In order to give a rating in our state it has to be done online and the rating has to come with a lot of questions. The IHSA also records if the team loses or wins. They found that the difference between winning and losing was a matter of decimal points. So there was no big difference in a rating from a winning coach and a losing.

Now in our state everyone makes the playoffs (only football you have to qualify for the post season with regular season record). So if they are screwing good officials and umpires, it could come back to haunt them. So I do not think there is much incentive to consider the umpire as the reason you lost a game. As it relates to baseball, we see a lot of the same coaches over and over and they might not agree with a call but they have a history of what you have done over the years or season.

Now I am not suggesting that this applies the same in other systems or other states, I am just stating that they have found this is not true as it relates to our system and we have had a rating system for almost 10 years. I know I liked it better than the previous system when coaches picked umpires they liked or knew and did not have any accountability to who was good and why.

Peace

[BlueLawyer]

There should be two purposes in ratings: (1) constructive, helpful criticism that helps the rated official in particular and officiating in the area in general become better and (2) determining who does what level of game.

I work in a small state, population-wise (Arkansas). I worked for years in basketball and baseball before I gave basketball up- now I'm baseball only.

In basketball before I quit, the ONLY ratings came from coaches. I saw a grand total of three of them in ten years- none of them were particularly flattering or helpful. Our high school assignor told us that these ratings were used in determining regular-season varsity assignments. This did not a whole lot for either purpose, in my judgment, as I didn't find anything in the ratings to build on, and the appointments depended as much on who the coaches knew and liked (politics) as on who would give a quality officiating performance on the court.

In baseball around here, the coaches have no say in who comes to their games other than to get two scratches. That is not to say that there is no politics in baseball around here, just that the politics takes a side seat to who can actually call a game. Coaches still rate. The bottom line is this: unless the coach is an umpire too, (it happens around here) his actual comments about what you did or failed to do are not likely to help you become a better blue. If coaches get a voice in who officiates their contests, to include playoffs, their subjective judgments about "who did me a good job" are more important than your rotation and positioning on R1-R2 with two out from C.

I have the crazy idea that people who know umpiring should choose umpires for big games and playoffs.

Strikes and outs!

[illiniwek8]

Hello everyone, I searched for this topic on the board as I was so steamed and I was glad to find that others feel similar............I had to vent somehow....the following email was sent to the IHSA...I feel a little better after reading your posts.......I am seriously thinking about not renewing my licenses in 4 sports with the IHSA because of this "berating" system. I thought I would share my letter and appreciate everyones insight into this topic.......I am finding blind criticism hardly constructive............


Hello Judy,

Once again....it is I (Official X).....I just reviewed my ratings and once again noticed that someone went in and requested improvement in certain areas....which...as I take it....criticism with no constructive purpose.....the contrusctive part of it would be to point out what it is that was done wrong as alleged by this other official....what I don't like about this is the whole annonymity.....if someone is willing to take the time to go to the computer and imput their opinion of my officiating....then why have they not confronted me at the time of the incident....after the game? I obviously don't agree with this.....I went for promotion this year and made it....but...if you were to look at my ratings....it hardly reflects that I deserve it.....in the previous four seasons....this person...or any other official did not care to rate myself in area of needed improvements...now all of a sudden...in my fifth season, someone has decided to share their opinions of my officiating through the website and not with me personally. I just don't get how this is a useful tool. Does it make any sense to yourself or to Mr. Gannaway? I am sending this to you as I don't have Mr. Gannaways e-mail address handy and your email was at the top of the screen conveniently...if you don't mind...could you share my concerns with him? My whole point is this is hardly usefull information and I find it more discouraging than encouraging...as a fairly new official...I would like to find reasons to continue officiating...if the purpose of this rating system is to weed out officials...then it is doing it's job...as I don't want to continue if this is what I will be subject to. I'd really like to know who it is that is sending their criticism my way and to know what exactly it is that I have done wrong.....I take most offense to the areas of Attitude and Professionalism and the Reaction to Pressure....I can honestly say that I disagree with either of these criticisms...I am my harshest crittic...and I cannot even think of any incidents where this criticism was applicable.......just as I should be held accountable for my actions as an official, I think that those criticizing my performance should held accountable for their criticisms....what was it....when....why was i not approached at the time...after the game......this system....once again......is a farce................I can take criticism when I know who it is coming from........but for someone to hide behind what you have currently set up......is ridiculous....I hope you seriously reconsider or at the very least.....give me a reason why it is necessary to have it the way that it is currently.....

[JRutledge]

Originally Posted by illiniwek8

Hello everyone, I searched for this topic on the board as I was so steamed and I was glad to find that others feel similar............I had to vent somehow....the following email was sent to the IHSA...I feel a little better after reading your posts.......I am seriously thinking about not renewing my licenses in 4 sports with the IHSA because of this "berating" system. I thought I would share my letter and appreciate everyones insight into this topic.......I am finding blind criticism hardly constructive............


Hello Judy,

Once again....it is I (Official X).....I just reviewed my ratings and once again noticed that someone went in and requested improvement in certain areas....which...as I take it....criticism with no constructive purpose.....the contrusctive part of it would be to point out what it is that was done wrong as alleged by this other official....what I don't like about this is the whole annonymity.....if someone is willing to take the time to go to the computer and imput their opinion of my officiating....then why have they not confronted me at the time of the incident....after the game? I obviously don't agree with this.....I went for promotion this year and made it....but...if you were to look at my ratings....it hardly reflects that I deserve it.....in the previous four seasons....this person...or any other official did not care to rate myself in area of needed improvements...now all of a sudden...in my fifth season, someone has decided to share their opinions of my officiating through the website and not with me personally. I just don't get how this is a useful tool. Does it make any sense to yourself or to Mr. Gannaway? I am sending this to you as I don't have Mr. Gannaways e-mail address handy and your email was at the top of the screen conveniently...if you don't mind...could you share my concerns with him? My whole point is this is hardly usefull information and I find it more discouraging than encouraging...as a fairly new official...I would like to find reasons to continue officiating...if the purpose of this rating system is to weed out officials...then it is doing it's job...as I don't want to continue if this is what I will be subject to. I'd really like to know who it is that is sending their criticism my way and to know what exactly it is that I have done wrong.....I take most offense to the areas of Attitude and Professionalism and the Reaction to Pressure....I can honestly say that I disagree with either of these criticisms...I am my harshest crittic...and I cannot even think of any incidents where this criticism was applicable.......just as I should be held accountable for my actions as an official, I think that those criticizing my performance should held accountable for their criticisms....what was it....when....why was i not approached at the time...after the game......this system....once again......is a farce................I can take criticism when I know who it is coming from........but for someone to hide behind what you have currently set up......is ridiculous....I hope you seriously reconsider or at the very least.....give me a reason why it is necessary to have it the way that it is currently.....

First of all the "need to improvement" section can only be filled in by Certified Official. Coaches cannot even evaluate your umpiring. Secondly, these marks mean absolutely nothing. They only hold value when you give them value. The IHSA does not consider them from all accounts and they do not affect your over all rating. Really, you should not even care. Look at it as a footnote and move on.

Also Judy is not a decision maker in the official's department. She just follows what the policies the IHSA Official's Department has set in place. It is much better to talk to Dave Gannaway personally or to someone on the Official's Advisory Committee that can address this with actual change. You are not the first one to complain about this and nothing has changed. Just be aware for people have talked about this for years at the Officials Conference Delegate's meeting for a few years and the system is likely to stay.

Peace

[GarthB]

Originally Posted by JRutledge

In our state this is not at all true. In order to give a rating in our state it has to be done online and the rating has to come with a lot of questions. The IHSA also records if the team loses or wins. They found that the difference between winning and losing was a matter of decimal points. So there was no big difference in a rating from a winning coach and a losing.

Now in our state everyone makes the playoffs (only football you have to qualify for the post season with regular season record). So if they are screwing good officials and umpires, it could come back to haunt them. So I do not think there is much incentive to consider the umpire as the reason you lost a game. As it relates to baseball, we see a lot of the same coaches over and over and they might not agree with a call but they have a history of what you have done over the years or season.

Now I am not suggesting that this applies the same in other systems or other states, I am just stating that they have found this is not true as it relates to our system and we have had a rating system for almost 10 years. I know I liked it better than the previous system when coaches picked umpires they liked or knew and did not have any accountability to who was good and why.

Peace

In an association I belonged to in the past, coaches rated umpires. We found that umpires who had ejections or game ending close plays were rated lower than the other umpires. This occurred to the extent that some D-1 umpires were rated several points below JV umpires working their first varsity games.

Thankfully my current association is more enlightened.

[SanDiegoSteve]

Originally Posted by GarthB

In an association I belonged to in the past, coaches rated umpires. We found that umpires who had ejections or game ending close plays were rated lower than the other umpires. This occurred to the extent that some D-1 umpires were rated several points below JV umpires working their first varsity games.

Thankfully my current association is more enlightened.

This is the problem the San Diego HS association has. The coaches get to choose which umpires get to work their playoff games. If a close call, whether correct or not, goes against them, they keep that umpire off their playoff list. Or if they are a real jerk, and they get ejected by certain umpires who don't put up with their crap, they only choose the umpires that are weak and let them get away with anything. They really love the umpires who brown-nose them between innings and before the game.

Just go there and do a professional job, don't kiss their butts, and they forget all about you at playoff time. Except the teams that don't make the finals. They always seem to have you on their preferred lists.

[Steven Tyler]

Originally Posted by SanDiegoSteve

This is the problem the San Diego HS association has. The coaches get to choose which umpires get to work their playoff games. If a close call, whether correct or not, goes against them, they keep that umpire off their playoff list. Or if they are a real jerk, and they get ejected by certain umpires who don't put up with their crap, they only choose the umpires that are weak and let them get away with anything. They really love the umpires who brown-nose them between innings and before the game.

Just go there and do a professional job, don't kiss their butts, and they forget all about you at playoff time. Except the teams that don't make the finals. They always seem to have you on their preferred lists.

And you speak so highly of your former association. Sounds like sour grapes to me. Get on one side of the fence and stay on it.

[SanDiegoSteve]

Originally Posted by Steven Tyler

And you speak so highly of your former association. Sounds like sour grapes to me. Get on one side of the fence and stay on it.

I wasn't talking just about me, Junior. I've worked plenty of top level playoff games. I started working the CIF playoffs in 1989, and did not have a HS season without playoff assignments through the 2004 season. Don't speak of which you don't know.

I was saying the WAY IT IS in that association. No sour grapes, as I got plenty of great games while there. Unfortunately, this is EXACTLY the way it works here. And sure, I've been requested by many teams too, and often these teams don't make it into the top playoff games (or even make the playoffs). The point is, these games are not given out by merit, but by what a coach thinks is good umpiring (i.e. puts up with his antics).

And for the record, the HS association does not make the rules as to who chooses the upper level playoff assignments. The California Interscholastic Federation makes the rules, and they give coaches a "request list" to fill out, and trust me, many a great umpire has been left off these lists due to not kissing coaches' butts.

I just took you off the list to see what cutting remark you would make to me. It's really a shame that all you can do is insult others. Really speaks highly of your intellect.

Back on the list.

[JRutledge]

Originally Posted by GarthB

In an association I belonged to in the past, coaches rated umpires. We found that umpires who had ejections or game ending close plays were rated lower than the other umpires. This occurred to the extent that some D-1 umpires were rated several points below JV umpires working their first varsity games.

Thankfully my current association is more enlightened.

Let me make this clear, our system of ratings is not a local association issue. This is an IHSA system. The IHSA assigns all playoff games not a local assignor. The coach’s ratings are mixed in with Certified Official/Umpires to come up with the same percentage. Coaches can only rate during varsity contests and they can rate you as many times as you work a varsity game for that school. Certified Umpires can only rate you once per year. The coaches/umpire ratings only count for 5 points total (if you in the 90 percentile of all umpires in the entire state) in a 30 point Power Rating system where other things are factored in. And (you heard this here first) they are going to add another 5 points this coming year for the frequency of attending camps along with another category that gives you 5 points for attending a Level 2 (3 Man in baseball and basketball) or 6 hour clinic in basketball for example. So most umpires/officials in our state worry about 5 points which we cannot control and most of the points in the power rating are almost totally in our control (percentage of varsity games, promotion level, open book Part 1 Exam just to name a few). And it has been proven that losing coaches rate umpires/official only percentage points apart from each other which it is not that big of deal. And the Sports Administrators that do the assigning for all playoff contests can make decisions outside of the power rating if they choose to. So things like what you have done while they were watching you might play a much bigger role than what your actual power rating is. You can help yourself or hurt yourself by what you do when observed. I cannot speak for other places, but we worry waaaayyyyy too much about something that is not going to really make that much of a difference. So if you are 90 percent in your coaches/umpire ratings and you are just an "X" as an umpire, you might not get an opportunity that a "C" umpire will get and they are in the 70 percent range.

Peace

[jkumpire]

I saw some of my HS ratings posted today. It is an embarassment.

I won't share everything I saw, but this year I did one HS JV game. I got the call an hour before the game to sub for another guy, I show up, do the plate, the losing team gets beat 15-0, and I get a 2 on a scale of 1(bad)-5(good) from the losing coach. the team won like 4 games all year, and dollars to donuts he ripped every umpire in every game he lost. Last time that ever happens.

I applaud my state association for trying to make a better rating system and a more objective way to rate officials for tournament assignments. it is a hard job. The problem is unless you use objective standards like test scores and trained umpires to evaluate people, no rating system is fair to the umpires. Coaches just cannot evenhandedly evaluate umpires, or frankly officials in any sport.

[Cub42]

I find it ironic when any state association or umpire group rely on evaluations by coaches for ratings or assignments. First of all, think of the questions and comments made by HS coaches during the course of a game. " The hand is part of the bat", " But he held the ball for 3 seconds, that means it's a catch ", and I think everybody's favorite," Can I appeal that/ Get help". The point here is most HS coaches have no clue about the mechanics we use, and most are oblivious to the rules.( Example: Break up 2). Most of the successful and respected coaches, if they have a problem with a particular umpire, will call the assignor and let him address the issue. I think evaluations can be an effective tool when used properly. The college group I am in, I think is a good example of how it can be done properly. Our president retired from on the field this season. He will come out, unannounced and observe and take notes. Then after the game he holds a session with the crew, and goes over what was done well, along with areas that need to improve. Veteran umpires, and umpires who have moved up to Pro ball or D I should be utilized to improve things.

[JRutledge]

Originally Posted by jkumpire

I applaud my state association for trying to make a better rating system and a more objective way to rate officials for tournament assignments. it is a hard job. The problem is unless you use objective standards like test scores and trained umpires to evaluate people, no rating system is fair to the umpires. Coaches just cannot evenhandedly evaluate umpires, or frankly officials in any sport.

Test scores???

Test scores are one of the worst ways to evaluate umpires. I have no problem if they are factored in some way, but test scores only prove someone knows how to pass a test. I have seen guys that are really good at passing tests only to wilt like a flower in a hard rain when the heat gets on. Umpiring is more than passing a test and more than just knowing the ins and outs of a particular rule. We also have to manage people and deal with conflict. Not every umpire knows how to do that very well and it shows when a player or a coach gets on them.

Secondly there is always going to be input from coaches about the job we do. When I worked my first D1 game recently I heard of the report that the coach said to my assignor. When I worked for the first time in a D2 conference I heard about the opinions of what the coaches thought of the job I did. You might think there is not "rating system" in place but the reality there is. At least with a set rating system you might know what goes into it. When you work higher levels you might not ever know what is said behind closed doors and you could be affected more.

Peace

[jkumpire]

Yes, I agree with you, testing is not a great alternative here. It is your praxsis that matters.

But as a point of fact, how do you determine fitness for umpiring, on the FED or NCAA level?

Here is the list as far as I know, in no order:

1. Evaluation by trained evaluators
2. Coaches' Vote
3. Umpires' Vote
4. Rules/Mechanics Testing
5. Tape Evaluation
6. Reputation
7. Written Recommendations
8. Random Chance
9. Senority
10. # of games worked
11. Other (#of ejections?, player evaluations? I don't know?)

We all agree #1 is best, then possibly #5 is 2nd best. What is #3 on the list?
IMO its testing.
You tell me.