test tomorrow

[Schmidt MJ]

I agree that a T/F test is no way to test an individuals knowledge of the rules. The higher you score then the more you know, theoretically. The lower your score is, then there is a higher likelihood that you were guessing on more of the questions. The best way to test someone's knowledge is by making them explain their answers. That way you will know if they actually know what they're talking about. This is somewhat unlikely given the logistics nightmare it would be to grade so many different tests and rely on a graders' interpretation of what an official was trying to convey in his answer.

[JeffTheRef]

Your explanation re: 'very likely' is right on target - although the assumption that the distribution of any population is bell-like is . . . an assumption. In most of the populations we look at in life, experience shows that to be the case. But it is not the only way things can be distributed.

As to the 'guess-effect' in standardized testing, what choice do we have other than to be aware that it's there and, perhaps, supplement this kind of testing with a floor test, a game conditions review?

[JRutledge]

Originally posted by Nevadaref


You are saying that if we give a bunch of officials only the answer sheet and let them just guess T or F, that their average score will approach 50% as the number of officials that we do this with gets bigger and bigger. That is absolutely true.

I have no clue if your analogy is correct, nor do I really care. But you have to understand that people here are just going to disagree with you because it does not fit what they have experienced or will try to justify it because they take this test and get high marks.

Originally posted by Nevadaref

Therefore, the scoring of the test is misleading because the scores not only don't reflect your officiating ability, they don't even accurately approximate your rules knowledge!

I and many people have been saying that for years. And one of the reasons I have yet to be asked by an assignor or clinician, "what was your test score this past year?" They do not ask that. And the funny thing, I just attended a classroom session for a camp this weekend, and the guy running the camp was more concerned about who was sitting up in the front of the room. Because the preception is that the best "students" sit in the front of the class to hear, respond or to be called on to answer questions. And those that sit in the back of the room are the "slackers." What is funny about that, there have been studies that back up that claim.

Originally posted by Nevadaref

Mick, that is what I object to. I believe that if we are going to go through the process of taking the test that we should at least make it meaningful in some manner.

The funny thing about this discussion has always been, when we are out on the court, no one ever talks about these tests to us or even mentions them, but we sit around here and debate their importance or lack there of. But they are here to stay, I just think we need to keep them in perspective.

Peace

[Rick Vietti]

The true value of the NFHS tests are skewed by how the state associations use them.

NFHS does not have an enforceable procedure nation wide to verify the usage of these tests. Instead they leave it up to the state associations to administer the tests, score them and place whatever importance on these tests.

Some states require officials to take them closed book, some open book, some once every X years.

I agree you have to put the use of these tests into the perspective of where you live and how the tests are used.

Just like high school proficiency tests, the NFHS tests are for the benefit of a larger organization to say "Here is our standard, pass this and you can go officiate."

[AK ref SE]

Wow! Are we taking an English test, Math test, or officiating test?

AK ref SE

[Love2ref4Ever]

Be aware of all new rules, theese more than likely will be on the exam. And "never" add anything to the questions. Also this is a fact, once a question reads that a defender has established "legal guarding position" he/she can do no wrong!

Good Luck!

[Mark Dexter]

Originally posted by Nevadaref
How many officials do you think we need to test before someone would correctly guess all 100 questions? The answer is 2 to the 100th power. A huge number.

Not quite. (Frankly, I'm shocked that anyone with a degree in mathematics would use probability in this manner.) In a random sample of 2^100 officials all guessing, you would expect one to get a perfect score. However, reality is not the same as probability! The first official guessing may get 100%. They may all get 100%, it may so happen that no one scores 100%.

Even more shocking, however, is the "guessing penalty." Under that rule, there would be 10 qualified officials in the entire country.

[Dan_ref]

Originally posted by Nevadaref

Dan,
First, I have a math degree.

Sigh...if you wanna play dueling degrees I'm afraid you're not gonna come out too well. So let's just stick to the point, OK?

Probability does not apply to a *single* random test taker. This is a fundemental concept. A single random test taker could get all, some, or none right in a 100 question T/F test and his results can not be predicted by the laws of statistics. Also, as Mark said, for N large we should expect SOME guessers to get 100% correct. But for N large the guessers tend to converge on a score of 50% correct with a probability approaching 1.

And since you live in Nevada howzabout you put $100 on 00 for me, OK? Keep doing it until I win!

[Edited by Dan_ref on Jun 5th, 2003 at 10:23 PM]

[Nevadaref]

Dan,
You keep saying that probability theory does not apply to a single test taker. This is wrong. The laws of mathematics and probability certainly do apply to him. Just because there is a slim chance that his score may not be within our expected range does not mean that these concepts do not apply to this situation. Probability theory accounts for these unexpected results and adequately explains them. This means that he is fully covered by the theory.

You need to revise what you are writing to something more along the lines of "Although, we would expect someone who is guessing to get about half of their answers correct, there is an outside chance that any single test taker's results could be far from what is expected."
I agree with the rest of what you wrote.

To take a gambling example, could you come to Nevada and hit a jackpot on your very first pull of the slot machine handle? Yes, you could. Does this mean that the mathematics that casinos base their business upon does not apply to you when you first walk in the door? No. No one would come in the door thinking, "I can beat the odds because I am a single random gambler and probability theory doesn't apply to me." See how silly that sounds?
PS If they didn't have table limits, I would be down there playing roulette right now!
As for Mark, he is shocked, shocked I tell you, to learn that there is gambling going on here.

[Nevadaref]

Originally posted by Mark Dexter
The first official guessing may get 100%. They may all get 100%

I'll take those bets!

Originally posted by Mark Dexter

Even more shocking, however, is the "guessing penalty." Under that rule, there would be 10 qualified officials in the entire country. [/B]

I don't think so. Anyone who puts some time into studying the rules would have no trouble finding 75 or 80 that they knew. Heck, we have a coach telling us how easy the test is! As for those who aren't willing to make the effort, let them work the JV and FR games if they don't get the required score. What's wrong with that?

[ChuckElias]

Originally posted by Nevadaref
As for Mark, he is shocked, shocked I tell you, to learn that there is gambling going on here.

Not just there, but at the Univ of Washington, too!

[JRutledge]

Originally posted by Nevadaref


I don't think so. Anyone who puts some time into studying the rules would have no trouble finding 75 or 80 that they knew. Heck, we have a coach telling us how easy the test is! As for those who aren't willing to make the effort, let them work the JV and FR games if they don't get the required score. What's wrong with that?

Even if you put the effort in, take all that time to study for a test that is really only going to exam your word for word knowledge of the rules. The test still is not going to prove you can officiate.

Just yesterday I was at a camp and the official running the camp, who happens to be a D1 Official and does many professional leagues told us this yesterday. For the record I am paraphrasing a bit, "rulebook officials can be the worst officials." In his words he calls these kind of people, "rulebook officials," for trying to learn every aspect of the rules, but cannot officiate a lick. He further went on to say, "they can know all these rules, but cannot call a simple foul or violation." So if you get and 80 (required by my state) as compared to 100, the official with the 100 score is not necessarily a better official than the official that only scored an 80. So when I hear an official that does more levels than anyone here talks about doing personally, tell me how silly it is to quote rule after rule after rule, that speaks volumes to me personally. Because in his words he says, "you need to have people skills, common sense and a basic understanding of the rules to officiate any game." He further went on to say, "if you do not have people skills to deal with coaches and players, you are in big trouble." Call it what you want, but I have seen more articles in Referee Magazine about "Presence, Dealing with conflict, Are you Argumentative, First Impressions," and a few other topics taht had little or nothing to do with passing a test, I am convinced.

Peace

[AK ref SE]

JRUT-
I have seen people not just in officiating but in other areas of life. They can recite a manual for work, they can pass the written drivers test (and be the worst drivers in the world.) To succeed you need people skills, knowledge of whatever rules or regulations you are dealing with, common sense, and I am sure I am leaving out many other skill sets that are needed. So agree, if the only thing you can do is take a test you may not be a great official.

AK ref SE

[CYO Butch]

Originally posted by Dan_ref

Sigh...if you wanna play dueling degrees I'm afraid you're not gonna come out too well. So let's just stick to the point, OK?

Probability does not apply to a *single* random test taker. This is a fundemental concept. A single random test taker could get all, some, or none right in a 100 question T/F test and his results can not be predicted by the laws of statistics. Also, as Mark said, for N large we should expect SOME guessers to get 100% correct. But for N large the guessers tend to converge on a score of 50% correct with a probability approaching 1.

And since you live in Nevada howzabout you put $100 on 00 for me, OK? Keep doing it until I win!

[Edited by Dan_ref on Jun 5th, 2003 at 10:23 PM]

Gotta side with Nevada on this. The laws of probability (which is really the relevant area) do apply equally to individuals and groups. If each "guess" is an independent event with two equally probable outcomes, it makes no difference whether the event is triggered by one or many different individuals. Thus, to continue with the coin toss analogy, the probability of 100 people each tossing a head with a single toss is the same as one individual tossing 100 out of 100 heads. The only issues that matter for the topic at hand are whether each "guess" is truly a random event (i.e., not influenced by prior events) and whether the two possible outcomes (T or F) are equally probable. Theoretically one could quantify those factors as well and develop a complete probability model to take them into account. My take is that they are not germain to Nevad's point that guessing should be penalized. The bottom line is that while I can never predict the outcome of a random event, I can assign a probility to it. I can also assign a probabilty to combinations and permutations of independent random events. I can give a probability for 7 out of 7 heads in a row, 7 out of ten, seven in a row in a series of ten tosses, etc. In fact, I can determine the exact probability of ANY possible outcome in the situation being described. Once actual results have been recorded, those results will fit into some place in the computed probability distribution. If the individual repeast the same series of questions randomly guessing again, his results will again fit into one of the slots. In fact, the distribution of these predicted outcomes fit a polynomial distribution which, at the size we are dealing with, is "normal" for all practical puposes. Over time, the frequency of his results will converge on the probabilities predicted. This is where statistics (as opposed to straightforward probability theory) comes into play.

Statistics are generally used when one is trying to extrapolate (predict) population characteristics or parameters from sample statistics. Under those situations, increases in the sample population will tend to produce data points from the sample that are normally distributed about the true value of the parameter of the underlying population. This is independent as to whether the characteristic iself follows a normal distribution in the actual population. (If I determine the average age of many different classes of 9th graders in New York, those averages will tend to be normally distributed around the actual mean of all 9th graders in New York. This is true even though the actual ages of the 9th graders are not likely to be normally distributed.) This principle can be used to predict the probability that the results from any given sample accurately reflect the underlying population. For example, it could be used modify the assumption that guessers on a test are equally likely to guess True and False. It also has the same basis as the statement that the more often an individual repeats the guessing process, the more likely his results will match the predicted outcomes. However, it has nothing to do with calculating the probability of the different possible outcomes from multiple coin tosses, or multiple random guesses between two possible answers.

[Dan_ref]

Originally posted by Nevadaref

You need to revise what you are writing to something more along the lines of "Although, we would expect someone who is guessing to get about half of their answers correct, there is an outside chance that any single test taker's results could be far from what is expected."
I agree with the rest of what you wrote.

Well, maybe you should go back & reread my posts, pay attention to this:

Probability does not apply to a *single* random test taker. This is a fundemental concept. A single random test taker could get all, some, or none right in a 100 question T/F test and his results can not be predicted by the laws of statistics. Also, as Mark said, for N large we should expect SOME guessers to get 100% correct. But for N large the guessers tend to converge on a score of 50% correct with a probability approaching 1.

I would suggest you go back to YOUR original post & reword that to make it fit the mathematics that we both seem to know so well - I suggest a few sprinkles of "on average", "likely expected outcome", and like that.