Ratings through Arbiter

[bainsey]

Does anyone's organization do its officials' ratings (for coaches/other officials) through Arbiter? Can someone tell me what they need to click to take a look?

(I'm sure there's a setup on Arbiter's end. Just want to know if a demo is viewable.)

[JRutledge]

Maybe there is a feature somewhere, but I have never seen anything that allows anyone to rate directly through Arbiter. I know some notes can be left for the game by the officials and probably the schools, but no direct rating system that I can tell. That might be something that can be used by the assignor or association that controls their site.

Peace

[LRZ]

There is an "evaluations" tab, but, like many arbiter functions, you probably need an assigner to authorize its use.

Search the arbiter site for "evaluations" and you will find several "how-to" articles.

[Valley Man]

There is a utility that assigners can use to rank officials. You can then rank your games and Arbiter will only list officials of the ranking that you set for a game when you get ready to assign games.

[LRZ]

If I'm reading it correctly, the assigner may choose to have officials evaluate each other:

At https://arbitersports.zendesk.com/hc...to-Z#Officials, it says:

Lead Officials
A lead official is defined by the preference you have set. It can be one of two defaults: The official in the first listed position or the official who is highest ranked in the first listed position. Admins can choose the lead official on a game by game basis by editing the slot and marking it as the lead official. Each game is limited to a single lead official.

All Officials
This allows officials to complete evaluations on each other. Since officials can't evaluate themselves, if an evaluation only has the plate umpire selected to evaluate the plate umpire won't see an evaluation for that game (unless there are multiple evaluations). There is no permission or preference for which officials can complete evaluations. The permission to 'Select games to evaluate' is to allow evaluators to find officials to evaluate.

[BillyMac]

I don't know who set it up, or how it was set up, but we've been using Arbiter to rate our partners for several years. About three times a season, somebody "opens up" the ratings and we can see our ratings, with the names of the raters deleted.

At the end of the year, the partner ratings are combined with trained observer ratings, all being made available to a committee to determine our status as varsity, or subvarsity, with the partner ratings carrying only a little strength in the total rating process.

Also, we tried to use Arbiter to rate everyone at a site, not just partners, but those in the other game at the site as well, subvarsity, or varsity. Arbiter told us that this couldn't be done with their software.

[grunewar]

Our Association uses the Evaluation feature in Arbiter.

We get reminders not to forget to rate officials for games. Criteria on what is evaluated is decided by the Association - I'm not necessarily a fan of our criteria groupings, but, it seems to work ok.

As an official, after score are tallied, you can see your running total and evaluation score averages, and even comments made by other officials. While it is anonymous, if used properly, and directly after each game, you know what the other members of the crew rated you.

Comments are helpful if used as intended.

Personally, while not everyone rates on the same scale, I find it to be a pretty good indicator of areas I need to improve on and a good indicator of how the game went.

You still can't eliminate cronyism and the good ole boy system though.

Bottom Line - we used to have nothing, and this is better!

[BillyMac]

Originally Posted by grunewar

... see your running total and evaluation score averages, and even comments made by other officials. While it is anonymous, if used ... directly after each game, you know what the other members of the crew rated you.

That's why we can only see our ratings about three times each season (including all of your ratings at the end of the season), when the ratings are "opened up", usually for a few days (usually over a weekend) each "open" period, to maintain the anonymity.

Otherwise, you work Friday night, have Saturday off, and a new rating shows up on Monday, making it easy to guess who rated you.

With our system, we just see the ratings in progressive "clumps".

Even then, we have a few guys who "sign" their comments, "Nice job BillyMac. John Doe", or, "Merry Christmas BillyMac. John Q. Public".

Also, we are fined if we don't submit our Arbiter partner ratings in a timely manner.

Arbiter Late Rating – If submitted more than 5 days from game date
- 1 Late or Missed – No Fine
- 2 to 3 Late or Missed - $25 Total
- 4 or more Late or Missed - $50 Total

I find it odd that our board puts such importance, thus the fines, on partner (Arbiter) ratings since they carry such little weight in the overall rating process (how reliable can ratings be when two inexperienced subvarsity guys rate each other in a middle school game, or a freshman game, they barely know if the ball is inflated, or stuffed).

It's the trained observer ratings (these guys show up early to rate both junior varsity officials, and then rate their partner in the varsity game, sometimes they have to make a "nonworking" trip (with some monetary compensation) to rate those who, for whatever reason, may otherwise fall through the cracks) that carry the bulk of the weight when the rating committee determines our status as varsity, or subvarsity, at the end of the season.

We are told that the partner (Arbiter) ratings are only used to supplement the trained observer ratings, that it's mainly the trained observer ratings that determines our status as varsity, or subvarsity, for the upcoming season.

[JRutledge]

A rating system and an evaluation system in my mind are not the same things.

Peace

[BillyMac]

Originally Posted by JRutledge

A rating system and an evaluation system in my mind are not the same things.

Rating often implies a number being associated with one's performance (example, 1-10), which, in some cases, can be limiting. Evaluation, while it may be associated with a number, may involve other forms of measuring one's performance (written commentary, oral commentary, checklists, etc.).

Also, the question pops up, is the "system" used for:
1) Ranking (another number) for assignment purposes (the level and number of games).
2) Education (improvement of officials).

[JRutledge]

My understanding is that assignors have to rank officials in the Arbiter in order to give them games. I was told these rankings involve what games they can give as an official.

Peace

[Raymond]

Originally Posted by JRutledge

My understanding is that assignors have to rank officials in the Arbiter in order to give them games. I was told these rankings involve what games they can give as an official.

Peace

It's an option, but not mandatory. It is mandatory if you want Arbiter to do auto assigning. You can rate all the officials and rate all your games and then tell Arbiter to fill all the slots.

Sent from my SM-N950U using Tapatalk

[JRutledge]

Originally Posted by Raymond

It's an option, but not mandatory. It is mandatory if you want Arbiter to do auto assigning. You can rate all the officials and rate all your games and then tell Arbiter to fill all the slots.

Sent from my SM-N950U using Tapatalk

You are right, it was for some level of auto-assigning. I do not get to use that feature, but that is the only way I know it was used in Arbiter. And our rating system only involves schools and they do that through the state's website (not Arbiter). That might change at some point, but not right now.

Peace

[SC Official]

We have a convoluted, antiquated rating system in South Carolina. 25% of our rating is peer evaluations, which we do through Arbiter. There is an "evaluations" tab and an option to "evaluate officials" underneath it. There are six categories we rate our peers in, on a scale of 1-10. The average is then multiplied by 2.5.

[ilyazhito]

I remember that you said in a different thread that peer evaluation is useless because most people give their peers 9s and 10s. If the system is convoluted and antiquated, what alternative would you prefer that the SCBOA and SCHSL use?

BTW, I do receive evaluations through Arbiter for baseball. There is a standard form that asks the designated crew chief for the game (the person whose name is in bold in Arbiter) to evaluate his partners on mechanics, signals, judgement, procedures, and professionalism, and provide an overall score. The crew chief also is expected to add comments to any rating that he posts. We later get to see how our partners rated us, including an average of our scores in all of the categories, and an average overall score, with their comments.