Rating Systems?

[jkumpire]

A question for those of you who do HS baseball in states where the coaches rate you.

In the state I work in, this year they started a new rating system where coaches rate your work. They also have catagories where you supposedly "need improvement". These ratings are a joke. The catagory ratings are even more of a joke.

How much stock do you put into these ratings and catagories? For example, the number of people who said I need to imporove rules knowledge (or words to that effect, I don't want to be too specific to give the state away) exactly equals the number of games that I called a balk in. I also happen to be a local rules interpreter for my local association, so I know the rules. I guess where I live, if you call a balk, you don't know the rules. The only other time I had a coach say anything to me about a call all year was when some guys didn't think I called the HPB rule right (Come on blue, he's got to get out of the way, he's gotta make an attempt, you know the conversation well enough I don't need to repeat it all for you).

Another rating was for "umpire professionalism". What in the world does that mean in this context?

One one hand, I'm upset at some of the ratings I see. On the other hand, I shake my head in wonder and laugh it off. People who have no clue about mechanics, or even where to stand between innings rate mechanics. Right. It just seems as if you make a call they don't like, they sitck it to you. I've been doing baseball games for a long time, I pride myself on how I carry myself on and off the field and knowledge of the rules. I make mistakes like everyone else, but some of these ratings are nuts.

I would give my left arm to have a guy who knows what he's doing come out and evaluate me. I would be glad to pay someone who knew how to umpire to evaluate me, since that will help me get better. All these ratings and catagories do is give you a bad reputation when it comes time for tournament assignments.

How do you guys feel about these things in your state?

[nickrego]

States that have the Coaches rate you are a JOKE.

I find that so ridiculous, I can't even think of a parallel example.

Wait, I know !

Having your Ex-Wife decide your divorce settlement, instead of a judge.

Fortunately, in all the associations I work for in California, all rating is done by senior members of each association.

[briancurtin]

Originally Posted by jkumpire

How much stock do you put into these ratings and catagories?

i can name 5 managers/coaches whose ratings i would buy into if they were given to me. if i was to be evaluated by coaches, which im not, i probably wouldnt pay all that much attention to the ratings. [[edit: my state does rate for HS, as you will see in ajjl22's post. i dont work HS so i didnt really know anything about it so i didnt cover it]] id read all of them and see what was said, but when people get screwed, they like to take others down with them. i would guess that coach evaluations end up in an inverted bell curve -- having a lot of coaches give poor ratings, not that many in the middle, and then a bunch of coaches saying you were good. those coaches giving negative ratings probably got "screwed" on balk calls, so they want to bring you down with them. coaches get "screwed" on the outside corner, so they want to bring you down with them there too. thats not the right way to go about it, obviously, but we see that it is still done.

now as far as ratings from other umpires, there are a whole lot more that i would trust. my association has an evaluation program run by about 20 guys (bob jenkins is one of them) in which you can pay $20 and get 2 evaluations done. they come out to a game that you choose, fill out their eval sheet and leave comments. it doesnt have an effect on assignments, its just between you and the evaluator so that you can get better. something like that is something that i would buy into, and ill be doing so this summer.

[BlueLawyer]

We do both- coaches rate us AND senior umpires rate us. We have crew chiefs, who are supposed to rate you one game and work with you one game. We ask coaches to evaluate us online (the ump evals are paper then entered into a database). It's like pulling teeth to get coaches to do this.

It is subjective. Our coaches get two scratches a year, too, before they turn in their schedules to our assignor. They only get to scratch us at their home places- they can still see us on the road.

Our assignor won't tell us how evaluations affect assignments, so I'm not sure it matters.

Strikes and outs!

[refman]

Coaches rate officials in the state I live in as well. Not only in baseball, but basketball, football, volleyball, softball also. It's certainly not the best system, but until someone comes up with a better one that's practical, sensible, and affordable, it'll have to do. And recently, our state has initiated an on-line rating system to get a higher percentage of schools to participate. That seems to be working rather well.
I don't put a lot of emphasis on the ratings from the coaches.....they often act on emotion vs. facts and also look at different things to rate that what many officials consider to be helpful.
Local associations are the best source for meaningful rating systems....if we can somehow find enough bodies to make it work.

[ajjl22]

In Illinois, each coach and is supposed to rate you for each game on a 1 to 5 scale. if you work with a certified official (highest promotion level) they must also rate you, but only once a year.

1 = Qualified to work state final contests
2 = Qualified to work sectional contests
3 = Qualified to work regional contests
4 = Qualified to work varsity contests
5 = Qualified to work underclass contests only

The ratings are averaged and then divided by number of ratings and you are given a percentile ranking. They do this to award officials who work more games.

The problem with the system is that coaches do not do their ratings. I have work 35 varsity games this year. I should have 76 ratings (worked with 6 certified officials) I have a total of 22 ratings. This makes it look like I have worked only 8 games all year.

[JRutledge]

Originally Posted by ajjl22

In Illinois, each coach and is supposed to rate you for each game on a 1 to 5 scale. if you work with a certified official (highest promotion level) they must also rate you, but only once a year.

1 = Qualified to work state final contests
2 = Qualified to work sectional contests
3 = Qualified to work regional contests
4 = Qualified to work varsity contests
5 = Qualified to work underclass contests only

The ratings are averaged and then divided by number of ratings and you are given a percentile ranking. They do this to award officials who work more games.

The problem with the system is that coaches do not do their ratings. I have work 35 varsity games this year. I should have 76 ratings (worked with 6 certified officials) I have a total of 22 ratings. This makes it look like I have worked only 8 games all year.

I think the point needs to be made that the ratings of coaches and officials are only 5 points in a overall 30 point power rating system. Next year there will be another 5 points added to for attending camps. You will get more points for attending a Level 2 camp as compared to a Level 1 camp. I also think in our state we focus way too much on the ratings when the ratings are only a small percentage of what gets us into the playoffs. Also our state is very aware that schools do not rate officials often enough and it might seem to us that this is going to affect us in a negative way. The reality is they are looking for new people to fill spots all the time, especially in baseball because of the lack of numbers. This is why they go out and watch umpires, look for recommendations from associations and if they observe you directly all those ratings will not mean much of anything. I had Anthony Holman tell me directly to have our members on the Top 15 list (in another association) to send the IHSA their schedule for the season. He wanted to take the opportunity to watch several umpires and find people that are not normally getting a shot at the post season. I also know that similar things are being done in other sports as well. Trust me on this, the right recommendation can get you on the radar. Once you are on the radar, it is up to you to prove you deserve to stay there. I know officials in many sports that were rated very high and did not get any playoff games. It was not their ratings that sealed the deal.

Peace

[nickrego]

If your going to have Coaches rate Umpires...

I think only a games WINNING coach should submit a rating.

I think we all know that when a team loses, it is primarily the fault of the crappy umpires. Especially that slug behind the plate !

[JRutledge]

Originally Posted by nickrego

If your going to have Coaches rate Umpires...

I think only a games WINNING coach should submit a rating.

I think we all know that when a team loses, it is primarily the fault of the crappy umpires. Especially that slug behind the plate !

In our state this is not at all true. In order to give a rating in our state it has to be done online and the rating has to come with a lot of questions. The IHSA also records if the team loses or wins. They found that the difference between winning and losing was a matter of decimal points. So there was no big difference in a rating from a winning coach and a losing.

Now in our state everyone makes the playoffs (only football you have to qualify for the post season with regular season record). So if they are screwing good officials and umpires, it could come back to haunt them. So I do not think there is much incentive to consider the umpire as the reason you lost a game. As it relates to baseball, we see a lot of the same coaches over and over and they might not agree with a call but they have a history of what you have done over the years or season.

Now I am not suggesting that this applies the same in other systems or other states, I am just stating that they have found this is not true as it relates to our system and we have had a rating system for almost 10 years. I know I liked it better than the previous system when coaches picked umpires they liked or knew and did not have any accountability to who was good and why.

Peace

[BlueLawyer]

There should be two purposes in ratings: (1) constructive, helpful criticism that helps the rated official in particular and officiating in the area in general become better and (2) determining who does what level of game.

I work in a small state, population-wise (Arkansas). I worked for years in basketball and baseball before I gave basketball up- now I'm baseball only.

In basketball before I quit, the ONLY ratings came from coaches. I saw a grand total of three of them in ten years- none of them were particularly flattering or helpful. Our high school assignor told us that these ratings were used in determining regular-season varsity assignments. This did not a whole lot for either purpose, in my judgment, as I didn't find anything in the ratings to build on, and the appointments depended as much on who the coaches knew and liked (politics) as on who would give a quality officiating performance on the court.

In baseball around here, the coaches have no say in who comes to their games other than to get two scratches. That is not to say that there is no politics in baseball around here, just that the politics takes a side seat to who can actually call a game. Coaches still rate. The bottom line is this: unless the coach is an umpire too, (it happens around here) his actual comments about what you did or failed to do are not likely to help you become a better blue. If coaches get a voice in who officiates their contests, to include playoffs, their subjective judgments about "who did me a good job" are more important than your rotation and positioning on R1-R2 with two out from C.

I have the crazy idea that people who know umpiring should choose umpires for big games and playoffs.

Strikes and outs!

[illiniwek8]

Hello everyone, I searched for this topic on the board as I was so steamed and I was glad to find that others feel similar............I had to vent somehow....the following email was sent to the IHSA...I feel a little better after reading your posts.......I am seriously thinking about not renewing my licenses in 4 sports with the IHSA because of this "berating" system. I thought I would share my letter and appreciate everyones insight into this topic.......I am finding blind criticism hardly constructive............


Hello Judy,

Once again....it is I (Official X).....I just reviewed my ratings and once again noticed that someone went in and requested improvement in certain areas....which...as I take it....criticism with no constructive purpose.....the contrusctive part of it would be to point out what it is that was done wrong as alleged by this other official....what I don't like about this is the whole annonymity.....if someone is willing to take the time to go to the computer and imput their opinion of my officiating....then why have they not confronted me at the time of the incident....after the game? I obviously don't agree with this.....I went for promotion this year and made it....but...if you were to look at my ratings....it hardly reflects that I deserve it.....in the previous four seasons....this person...or any other official did not care to rate myself in area of needed improvements...now all of a sudden...in my fifth season, someone has decided to share their opinions of my officiating through the website and not with me personally. I just don't get how this is a useful tool. Does it make any sense to yourself or to Mr. Gannaway? I am sending this to you as I don't have Mr. Gannaways e-mail address handy and your email was at the top of the screen conveniently...if you don't mind...could you share my concerns with him? My whole point is this is hardly usefull information and I find it more discouraging than encouraging...as a fairly new official...I would like to find reasons to continue officiating...if the purpose of this rating system is to weed out officials...then it is doing it's job...as I don't want to continue if this is what I will be subject to. I'd really like to know who it is that is sending their criticism my way and to know what exactly it is that I have done wrong.....I take most offense to the areas of Attitude and Professionalism and the Reaction to Pressure....I can honestly say that I disagree with either of these criticisms...I am my harshest crittic...and I cannot even think of any incidents where this criticism was applicable.......just as I should be held accountable for my actions as an official, I think that those criticizing my performance should held accountable for their criticisms....what was it....when....why was i not approached at the time...after the game......this system....once again......is a farce................I can take criticism when I know who it is coming from........but for someone to hide behind what you have currently set up......is ridiculous....I hope you seriously reconsider or at the very least.....give me a reason why it is necessary to have it the way that it is currently.....

[PeteBooth]

[QUOTE]

Originally Posted by jkumpire

A question for those of you who do HS baseball in states where the coaches rate you.

In the state I work in, this year they started a new rating system where coaches rate your work. They also have catagories where you supposedly "need improvement". These ratings are a joke. The catagory ratings are even more of a joke.

My HS rating system is also a joke. I have been told that coaches have input except we do not see this input. We get a letter in the mail that ranks each official by a numerical number. That's all we get. We do not know how that number was arrived at etc. It's simply a number.

Personally I did not get HS certified to get the BIG game(s) at the end of the HS year. I got HS certified so that I could umpire in the spring (other leagues than HS) , summer and fall because the contracts call for officials to be HS certified.

You can make a nice "piece of change" umpiring all summer long. You could umpire seven nights straight if your schedule permits. Also, every Saturday / Sunday being at the minimum 2 games a day sometimes 3.

It might be a cop-out answer etc. but I do not put any stock into the rating system. There's politics involved, the "good ole boy networK etc. The point is if you want to umpire in the summer / fall then it's well worth it to get HS certified regardless of the rating system.

Pete Booth

[JRutledge]

Originally Posted by illiniwek8

Hello everyone, I searched for this topic on the board as I was so steamed and I was glad to find that others feel similar............I had to vent somehow....the following email was sent to the IHSA...I feel a little better after reading your posts.......I am seriously thinking about not renewing my licenses in 4 sports with the IHSA because of this "berating" system. I thought I would share my letter and appreciate everyones insight into this topic.......I am finding blind criticism hardly constructive............


Hello Judy,

Once again....it is I (Official X).....I just reviewed my ratings and once again noticed that someone went in and requested improvement in certain areas....which...as I take it....criticism with no constructive purpose.....the contrusctive part of it would be to point out what it is that was done wrong as alleged by this other official....what I don't like about this is the whole annonymity.....if someone is willing to take the time to go to the computer and imput their opinion of my officiating....then why have they not confronted me at the time of the incident....after the game? I obviously don't agree with this.....I went for promotion this year and made it....but...if you were to look at my ratings....it hardly reflects that I deserve it.....in the previous four seasons....this person...or any other official did not care to rate myself in area of needed improvements...now all of a sudden...in my fifth season, someone has decided to share their opinions of my officiating through the website and not with me personally. I just don't get how this is a useful tool. Does it make any sense to yourself or to Mr. Gannaway? I am sending this to you as I don't have Mr. Gannaways e-mail address handy and your email was at the top of the screen conveniently...if you don't mind...could you share my concerns with him? My whole point is this is hardly usefull information and I find it more discouraging than encouraging...as a fairly new official...I would like to find reasons to continue officiating...if the purpose of this rating system is to weed out officials...then it is doing it's job...as I don't want to continue if this is what I will be subject to. I'd really like to know who it is that is sending their criticism my way and to know what exactly it is that I have done wrong.....I take most offense to the areas of Attitude and Professionalism and the Reaction to Pressure....I can honestly say that I disagree with either of these criticisms...I am my harshest crittic...and I cannot even think of any incidents where this criticism was applicable.......just as I should be held accountable for my actions as an official, I think that those criticizing my performance should held accountable for their criticisms....what was it....when....why was i not approached at the time...after the game......this system....once again......is a farce................I can take criticism when I know who it is coming from........but for someone to hide behind what you have currently set up......is ridiculous....I hope you seriously reconsider or at the very least.....give me a reason why it is necessary to have it the way that it is currently.....

First of all the "need to improvement" section can only be filled in by Certified Official. Coaches cannot even evaluate your umpiring. Secondly, these marks mean absolutely nothing. They only hold value when you give them value. The IHSA does not consider them from all accounts and they do not affect your over all rating. Really, you should not even care. Look at it as a footnote and move on.

Also Judy is not a decision maker in the official's department. She just follows what the policies the IHSA Official's Department has set in place. It is much better to talk to Dave Gannaway personally or to someone on the Official's Advisory Committee that can address this with actual change. You are not the first one to complain about this and nothing has changed. Just be aware for people have talked about this for years at the Officials Conference Delegate's meeting for a few years and the system is likely to stay.

Peace

[David B]

Originally Posted by jkumpire

A question for those of you who do HS baseball in states where the coaches rate you.

In the state I work in, this year they started a new rating system where coaches rate your work. They also have catagories where you supposedly "need improvement". These ratings are a joke. The catagory ratings are even more of a joke.

How much stock do you put into these ratings and catagories? For example, the number of people who said I need to imporove rules knowledge (or words to that effect, I don't want to be too specific to give the state away) exactly equals the number of games that I called a balk in. I also happen to be a local rules interpreter for my local association, so I know the rules. I guess where I live, if you call a balk, you don't know the rules. The only other time I had a coach say anything to me about a call all year was when some guys didn't think I called the HPB rule right (Come on blue, he's got to get out of the way, he's gotta make an attempt, you know the conversation well enough I don't need to repeat it all for you).

Another rating was for "umpire professionalism". What in the world does that mean in this context?

One one hand, I'm upset at some of the ratings I see. On the other hand, I shake my head in wonder and laugh it off. People who have no clue about mechanics, or even where to stand between innings rate mechanics. Right. It just seems as if you make a call they don't like, they sitck it to you. I've been doing baseball games for a long time, I pride myself on how I carry myself on and off the field and knowledge of the rules. I make mistakes like everyone else, but some of these ratings are nuts.

I would give my left arm to have a guy who knows what he's doing come out and evaluate me. I would be glad to pay someone who knew how to umpire to evaluate me, since that will help me get better. All these ratings and catagories do is give you a bad reputation when it comes time for tournament assignments.

How do you guys feel about these things in your state?

Our state has been rating the umpires for about three years now and its rediculous.

The coaches are supposed to be "required" to submit it by the state, but in basketball about half of my games were rated and in baseball only about 10%.

If they are going to let the coaches rate us, then we should have to rate the coaches also. Did they control their team, sportsmanship, facilities, security, administration present, etc.,

The sad thing is that most of the losing coaches will turn in the ratings. I laugh because one of the areas is "rules knowledge" and I always get rated less on that area which I would consider my highest.

Last season, we had a "no call" on a fly ball down the first base line. The runner ran to first, and the F1 and F2 came out to get the catch. It was in foul territory and ended up being a drop.

The coach of course said the runner interfered. There was no interference.
Later on my rating coach writes, "umpire definitely doesnt' understand interference"

So basically its all subjective and coaches can say whatever they want no matter how wrong they are.

The state uses the ratings for basketball playoff assignments, but baseball does not.

Thansk
David

[C'monBlue]

In this state, coaches rate umpires on the following criteria:

10% Appearance/Physical
10% Pre-Game
20% Attitude/Temperament
20% Rules Knowledge
20% Mechanics
20% Game/Field Presence

What is Pre-Game, you ask? According to the Arbiter, it includes being "On time for pre-game conference. On time (Enters playing area at appropriate time). Performs all assigned duties: inspections."

And miraculously, I have been marked down in this area. How is that even possible?

We are also expected to rate our partners after each game, a practice I find equally distasteful and which I refuse to do. It seems ridiculous for a rookie to be rating someone who has been doing it for 20 years. (Of course, there are those Smitties who have one year experience twenty times, but ratings would be wasted on them anyway.) Shouldn't these conversations happen in the post-game?

It would be interesting to come up with a system for rating coaches:

20% Attitude
20% Sportsmanship
20% Pulled the pitcher before giving up a grand slam
20% Chose a shortstop who could field a ball cleanly
20% Selected a line-up of players who swing at corner pitches

Have I missed anything?