Consistency? What's that?

[Back In The Saddle]

A comment on another thread got me thinking about just what consistency in officiating really means.

A simplistic and inexperienced view might be that: "if exactly the same play happens x times, then exactly the same call is made x times." But we know that "exactly the same play" never happens twice.

Perhaps we have to take a less precise approach: "if x number of similar plays occur, the same call will be made x times." That, of course, opens up a whole discussion about what it means for plays to be "similar". It is possible for two plays that appear similar in many respects to differ in very small but important ways. It is possible for two plays that differ in several large and unimportant ways to be similar in one or more small and important ways. And how do we decide what are important and unimportant ways?

Does context matter to consistency? How much should age, skill, size, ability, mismatch, experience, score, "temperature", time on the clock, foul count, time of the season, and other factors matter to how we call the game? Do such factors only diminish consistency? Or does a proper understanding of consistency include adjusting for those factors?

Does it matter whose perception is involved? Is it valid to consider your partner's perception when evaluating your consistency? Is it valid to consider the players' and coaches' perception when evaluating your own consistency? Your evaluator's/supervisor's/assigner's? If you strongly disagree with one of these individual's perception of your consistency, does that constitute any kind of crisis of integrity on the official's part to consider their perspective, and perhaps even change?

If we cannot easily define what it means to be consistent from play to play and game to game, how can we expect to be consistent from official to official within a crew, and from crew to crew over a season.

What do you think? How do you define consistency?

[BillyMac]

Back In The Saddle: Good topic. It should generate some good discussion. I don't have a simple answer for you, but I can give you what I go over with my partner during every pregame conference:

Consistency
Let’s see if we can call the same game. Be consistent with each other. Let’s try to remember what we’ve called earlier in the game, and what we haven’t called. Be consistent with what has already happened in the game.

Also, for your viewing pleasure, from the California Basketball Officials Association's "Official's Creed":

Consistency
The greatest virtue which an official can possess is consistency. He/she may have an incorrect interpretation of a rule; he/she may practice techniques contrary to those to which a team is accustomed; judgment on some play situations may be in conflict with the commonly accepted interpretations - but with it all, if the official’s practice and decision are exactly the same under the same or similar circumstances, players can readily adjust their play to fit the official. They may be surprised and confused momentarily, but when they discover that the official is unwavering in procedures, they can reorganize and continue with confidence. Probably the greatest inconsistencies occur in judgments on charging and blocking. The official should give this play much thought and attention, and learn to call it consistently. Have the courage to withhold a whistle despite contact if no advantage has been gained. If a highly technical call is made and then a flagrant act is passed on the players are placed in a position where they cannot establish a flow. Some officials may never be able to attain a high degree of consistency and they should be eliminated, just as incapable players are gradually cut from the squad. However, much can be done to point the way.

[AKOFL]

I agree with Billy. I would say most people are concerned with consistency on a single game basis. They probably don't remember how you called the last game. If you are, as a crew calling the same things and not calling the same things, that is consistent as we can be.
Your calls will change at different levels of ball. That's just the way it is, but if your calls are the same for the whole game, I believe you have been consistent. Hope that makes sence. It did in my head

[Adam]

Frankly, I don't concern myself with whether coaches and fans think I'm consistent. Only other officials are qualified to judge that. A coach's perspective is biased, and they often don't see the details (or even want to hear about them) that make plays completely different even if they appear the same.

[Back In The Saddle]

Originally Posted by Snaqwells

Frankly, I don't concern myself with whether coaches and fans think I'm consistent. Only other officials are qualified to judge that. A coach's perspective is biased, and they often don't see the details (or even want to hear about them) that make plays completely different even if they appear the same.

So you don't consider the coaches' perspective in any way at any time? Not even a little? Aren't you at least sometimes forced to deal with a coach's perspective on your consistency during a game? To address his concerns in some way, even if they are not valid, or risk the whole situation deteriorating? And doesn't that require at least trying to understand why he thinks you're not being consistent, in other words trying to see things from his perspective, if only briefly?

[AKOFL]

We do have to deal with coaches weather we like it or not. You can usually tell when a coach has a legit complaint or he is just trying to get a call. We do have to give them some time find out which one it is. Some times we may find ourselves in the tank for one reason or another and someone has to snap us out of it. Hopfully we talk ourselves back into the game or our partners do. It really shouldn't have to be the coach. I hate it when they are right.

[Back In The Saddle]

Here's a related follow on question: What do you do to help ensure you and your partner are calling the game consistently? How do you even know? If both, or all three, of you are watching your own area and rarely have four eyes on the same players or play...how do you know what your partner called and why?

This kind of came to a head for me during a game last season. They sent four of us to work two double-headers at a school, and we all switched partners for the second game. The way things work out, I finished my first game, grabbed my partner for the second game and we headed right back out to start it without any time to pregame. But we've both worked for many years, we're friends, we've talked a lot and should pretty much be on the same page. Apparently not. The visiting coach was unhappy, and frankly neither of us really could figure out why. I felt I was calling a very consistent game, so did my partner. After the game one of the V officials (also a respected college official) gave us very poor marks for our consistency. All I can figure is that my partner and I were calling two very different games even though we both felt like we were being consistent. Ugh!

[Adam]

Originally Posted by Back In The Saddle

So you don't consider the coaches' perspective in any way at any time? Not even a little? Aren't you at least sometimes forced to deal with a coach's perspective on your consistency during a game? To address his concerns in some way, even if they are not valid, or risk the whole situation deteriorating? And doesn't that require at least trying to understand why he thinks you're not being consistent, in other words trying to see things from his perspective, if only briefly?

Sorry, I'm a bit snippy today. Allow me to rephrase. I'll address it with a coach if he perceives something. However, all he usually gets is an "I'll watch for it, coach." I'll quickly evaluate if I've missed something (always possible) and move on. Normally, however, it's not really about me.

[wanja]

Three consistency propositions.
1. Higher quality of play, greater need for consistency.


With young players just starting out -- safety, sportsmanship, teamwork and a positive experience often (and justifiably) take a back seat to rules consistency. Player age and size, score, clock and skill are huge rules application factors that decrease as such with the quality of play. There are few circumstances that justify rule application inconsistency at the higher levels. Here's one that I consider. If a trend is developing in a game toward rougher player and player confrontation, tighter calling of borderline fouls may be appropriate.


2. Greater consistency is the inevitable trend.
More scrutiny from coach, supervisors and media demands more consistency. Video exposes.


3. Greater consistency can be mandated and promoted.
It is no accident that hand checking, displacement by offensive post players and palming have decreased over the past few seasons in NBA games. This past season, John Adams, the NCAA men's officials coordinator, introduced the concept of absolutes (e.g. 2 hands on opponent by defender is a foul) to promote consistency. He also mentioned at a camp this summer, that physical fitness and the ability to cover the floor was a primary consideration for tournament assignments. So officiating consistency extends beyond the rules. Trickle down is real. Consistency is also being promoted by camps, the NFHS and IAABO.

Originally Posted by BillyMac

Also, for your viewing pleasure, from the California Basketball Officials Association's "Official's Creed":

Consistency
The greatest virtue which an official can possess is consistency. He/she may have an incorrect interpretation of a rule; he/she may practice techniques contrary to those to which a team is accustomed; judgment on some play situations may be in conflict with the commonly accepted interpretations - but with it all, if the official’s practice and decision are exactly the same under the same or similar circumstances, players can readily adjust their play to fit the official. They may be surprised and confused momentarily, but when they discover that the official is unwavering in procedures, they can reorganize and continue with confidence. Probably the greatest inconsistencies occur in judgments on charging and blocking. The official should give this play much thought and attention, and learn to call it consistently. Have the courage to withhold a whistle despite contact if no advantage has been gained. If a highly technical call is made and then a flagrant act is passed on the players are placed in a position where they cannot establish a flow. Some officials may never be able to attain a high degree of consistency and they should be eliminated, just as incapable players are gradually cut from the squad. However, much can be done to point the way.

The CBOA creed (as excerpted) misses the mark on 2 counts. First, consistency by an individual official is not virtuous if it not consistent with crew consistency. Maybe crew consistency is mentioned elsewhere. Secondly, how often is too often to be consistent but clearly wrong on the rules?

A final note. A pet peeve of mine is that there should be more consistency in the approved mechanics and rules at all levels. FIBA does a much better job of this than the NBA, WNBA, NCAA-M, NCAA-W and NFHS. Elimination of some of the approved differences would directly help improve officiating consistency and also better allow newer officials to adapt good practices through observation of more accomplished officials.

[tomegun]

Wanja,

With all due respect to Mr. Adams, I will bet you $100 that he isn't willing to walk the walk come march and april concerning physical fitness.

[mbyron]

Frankly, I'm with Emerson.

Originally Posted by Ralph Waldo

A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines.

Consistency in itself has no value, or else a game that started with a couple bad calls would have to be called badly throughout. Sorry California: not a virtue at all, much less the highest one.

So what distinguishes a "foolish consistency" from whatever it is about consistency that people admire? Not making the same call, but making the right call in each case.

I think that rookies and sometimes those who teach them seek a simple formula for officiating. But the reason that no simple formula is forthcoming is that good officiating is making the right call in each case. And doing that requires experience, since we have to factor in that cool list BITS mentioned in his first post of the thread: "age, skill, size, ability, mismatch, [player] experience, score, 'temperature', time on the clock, foul count, time of the season, and other factors." That last one is a kicker, too.

And, given how the game is actually officiated, each of those is sometimes relevant. Judging when and how much is extraordinarily complicated and not algorithmic: a computer could never be programmed to officiate for this reason (and computational models of mind are misguided for the same reason -- but I digress!). And on top of the complexity of information needed in each judgment situation, the call must be made fast. Fast and right is a tall order.

My conclusion is that consistency in itself has little value: what's valuable is an official's capacity to make the right call. If officials can do that every time, great, but we'll take what we can get.

[rockyroad]

Originally Posted by mbyron

Frankly, I'm with Emerson.



Consistency in itself has no value, or else a game that started with a couple bad calls would have to be called badly throughout. Sorry California: not a virtue at all, much less the highest one.

So what distinguishes a "foolish consistency" from whatever it is about consistency that people admire? Not making the same call, but making the right call in each case.

I think that rookies and sometimes those who teach them seek a simple formula for officiating. But the reason that no simple formula is forthcoming is that good officiating is making the right call in each case. And doing that requires experience, since we have to factor in that cool list BITS mentioned in his first post of the thread: "age, skill, size, ability, mismatch, [player] experience, score, 'temperature', time on the clock, foul count, time of the season, and other factors." That last one is a kicker, too.

And, given how the game is actually officiated, each of those is sometimes relevant. Judging when and how much is extraordinarily complicated and not algorithmic: a computer could never be programmed to officiate for this reason (and computational models of mind are misguided for the same reason -- but I digress!). And on top of the complexity of information needed in each judgment situation, the call must be made fast. Fast and right is a tall order.

My conclusion is that consistency in itself has little value: what's valuable is an official's capacity to make the right call. If officials can do that every time, great, but we'll take what we can get.

Very well said, mbyron!

In my experience, "consistency" is a word used by 1) a coach who really has nothing else to gripe about because the game is being called well so he/she starts yelling about "being consistent", or 2) an assignor/supervisor/coordinator who really has nothing else to talk to you about because the game was called well so he/she uses "consistency" to prove that you have something you "need to work on".

Consistency is in the eys of the beholder, no matter what field you're in.

[zm1283]

Originally Posted by Ref Ump Welsch

Consistency is in the eys of the beholder, no matter what field you're in.

It's funny that you say that. In our state, coaches rate officials for every varsity game worked. In addition to the 1-5 rating (1 being the best), they also give "dings" for "areas of improvement" that they think you need to work on as an official. (Ridiculous and hilarious, right?) When our ratings came out after the season this year, every single person that I talked to had the highest number of "dings" in the "consistency" column.

The other "areas of improvement" are things like hustle, physical appearance, control, verbal communication, etc. I wish we could rate them too...it would only be fitting.

On another note pertaining to ratings. I tech'd a coach in a boys varsity game this last year toward the end of the season. (He was getting pounded by about 40 and screamed at me "Blow your whistle!!") I figured I would get a bad rating from him, no big deal. Well come to find out he also helps coach baseball, and one of my friends had the school in a baseball game this spring. The head basketball coach starts telling my friend about how I'm the only official he doesn't like, how he gave me a "5" rating, etc. The funny thing is that if a coach gives you a 5, he's supposed to write a special report to the state to accompany the rating. I haven't seen any report of any type.

[Hugh Refner]

It's been my experience that coaches define "consistency" as you being consistent in calling the game in their favor. What do you expect when you have one set of people - the coaches - totally subjective and another set of people - the officials - totally objective?

Remember - you can't "call it both ways". You can only call it one way - the correct way.