Re: Question difficulty

• From: tfmichael1
• Date: Wed, 05 Apr 2000 11:55:29 -0700
• Message-ID: <000SEN311.000GTB4AL.12170551-quizbowl_at_...>

David Hayes wrote:

"I am in the process of
writing a packet and this suggested some questions about
difficulty. What do you think is the best way to keep a
packet at an easy to moderate level? Does only a
well-known answer mean that a question is easy? I think for
bonuses, yes, it does, but for tossups this is not
necessarily the case. Do the introductory clues in a tossup
determine whether it's easy? Does a tossup with very
obscure clues in the beginning about a well-known subject
make the question hard, or just of moderate level?
Give me some feedback."

The difficulties with
difficulty include that measurement of difficulty is highly
subjective prior to a question being read, and can only be
measured accurately after the question has been played.
And, as you've noted, difficulty has to be assessed in
both the answer to the question and the clues leading
to it.

The rule of thumb goal for any packet
in most formats is that 70-80% minimum of most
questions are answerable by one of the players in a match.
So assessing desired difficulty while writing
questions involves knowledge of who the audience for the
questions is, and an understanding of the typical level of
difficulty for the format the questions are written
for.

But the bottom line is that knowledge of the audience
and appreciation of a format are gained only through
subjective experience. There is no way other than subjective
opinion to measure the difficulty of a question
beforehand.

That being said, the measurement of difficulty in a
question involves three factors: is the answer one the
player can reasonably be expected to recall, does the
giveaway clue make the answer obvious to at least 80% of
the players hearing the question, and how many
further clues occur between the point where a person with
perfect knowledge of the subject would have reduced the
Universe of All Possible Answers to one answer, and the
point of the giveaway clue. Examining these three
factors enables one to determine the appropriateness of
the answer to the format, the accessibility of the
answer to the average player, and how many opportunities
are provided for a player to retrieve the answer.
Unfortunately, such an analysis of difficulty in the first two
factors can only be objectively determined post
hoc.

Tom