Re: ICT Comments

acfraud <*no_reply_at_yahoogroups.com*> · Sat, 12 Apr 2003 08:42:27 -0000

NAQTrauma? What's next, CBIAmGettingSuedForUsingThisAcronym? (No
offense intended to any representatives of the Varsity Sport of the
Mind who may be reading.)

Anyhow, I have to agree with naqtrauma. The questions at the ICT were
plagued by easy giveaway clues. 

I want to emphasize one thing that naqtrauma said:

"Never begin a science [or mathematics] tossup with a straightforward
definition of the answer."

That's right, NEVER. Whoever wrote the "continuous" question may have
thought they were giving some obscure initial clue, but "the inverse
image of an open set is open" is the definition of continuous. Period.
Maybe you think basic point-set topology is advanced, but it isn't.
This is the definition anyone who knows anything about real
mathematics uses. Epsilon-delta stuff is ok for basic analysis, but
it's not the definition of continuous in full generality.

The problem with giving a definition - which is possible in
mathematics questions to a larger degree than in other fields - in the
first clue is that, to do *anything* with a term, you need to know the
definition. This means that almost anyone with knowledge of a subject
is equally capable of answering the question at that point, and you
get a buzzer race. In practice, some may know anecdotal facts without
knowing the definition of a term, and so could potentially answer such
a question later, but in general, *if one knows anything about a
mathematical term, one knows the definition*.

The "continuous" question wasn't the only such example. The
"Mandelbrot set" question was as well, if I remember the phrasing
correctly; in any event, it was phrased in such a way that anyone who
knows more than "it's a pretty fractal picture" could answer it in the
first line. Ditto the question on the "triangle inequality": if you
know of it, it's probably in the context of metric spaces, so don't
give the definition of metric space in the first line! Use common
sense, please. Just because you don't know anything about math, don't
make the questions you write or edit imbecilic.

As has already been discussed on this group, another poorly
thought-out mathematical question (point groups) was extremely
important in determining the final standings near the top of the
tournament. It's important to realize that often mathematicians will
define terms in slightly different ways; one may take "manifold" to
include "second countable, Hausdorff" while another may take it to
include "paracompact," and so on. Some may take "the category of
topological spaces" to actually mean "the category of weak Hausdorff
spaces" (e.g. to ensure Map(X,Map(Y,Z)) = Map(X x Y, Z)). Similar
examples abound. Whether or not "crystallographic group" refers only
to point symmetries, or to the full group of symmetries of a crystal,
may be largely a matter of taste. One should check the literature when
writing such things. If you aren't an expert on mathematics, then
either stick to well-established mathematical terms in your questions,
or search the literature. It sounds like common sense to me; people
should verify their facts when writing about any subject. Granted, the
"point group" question is an issue where chemists and other scientists
use the term as well, but it is indicative of the lack of research
involved in determining alternate answers that "point symmetries" was
evidently not among the acceptable or promptable answers listed. If
upon examining references and literature, you determine that something
that sounds like a reasonable answer to a question is *not* an
acceptable answer, by all means, write "do not accept" that answer!

I think naqtrauma has nicely summarized a lot of the bad questions in
other categories that come to mind. "Self-actualization" is one I can
think of; Abraham Lincoln tends to be the canonical example of one
whom Maslow considered self-actualized, and he was mentioned almost
immediately in the question. I think self-actualization is an
overrated concept anyway, but that's an issue for another time. (I was
pleased to not have too many questions about psychoanalysts; I wish
people would learn that psychology does *not* just mean Freud et. al.
But again, that's orthogonal to my comments about the ICT.)

Another problematic example: it was late in a round, so perhaps I
missed something crucial, but from what I heard of the beginning of
the "anesthesia" question, "sleep" would have been a perfectly
reasonable buzz. (It mentioned it having a Stage 3, and being measured
by monitoring eye movements, if I remember correctly.) 

Not *all* the science questions were incompetent. From my recollection
of what I heard of them, the question on monoids was fine, as was the
"solitons" question and the "Hall coefficient" question.

My complaints are not limited solely to science, but naqtrauma has
provided nice examples of bad questions in other categories. It's
simple to fix many of these! Just make sure your question does not
start with a giveaway. If you're not sure, have others read the
question. The initial clues at the ICT should challenge the best teams
in the nation, not aim for the lowest common denominator.

I should add that my experience at the ICT was not as bad as this post
may suggest. There were a fair number of good questions. But when
there are glaring problems with questions, problems that take little
effort to correct, one cannot help but be irked. I respect NAQT as an
organization, but I am disappointed in its ability to produce
competent questions for a national-level tournament.

Also, congratulations to Chicago, i.e. Subash, for an impressive
performance, and to the other teams with high finishes.

P.S. I don't think I need to point out how ludicrous it is that, had
Berkeley lost to Michigan in their game to determine the 2nd place
team, Michigan would have advanced despite the two having *equal*
records, each having beat the other once. Or how odd it is that
Michigan dropped to 4th rather than 3rd, due to the vagaries of the
NAQT playoff system. But I'll mention it anyway. I assume the numbers
2.25 and 6.5 weren't obtained from a random number generator, but I'm
at a loss to see where they came from. I conjecture that in such a
playoff system, inexplicable and morally repugnant results are in fact
generic. I'll leave the proof as an exercise to the reader.

--- In quizbowl_at_yahoogroups.com, "naqtrauma" <naqtrauma_at_y...> wrote:
> Raj Duwalia wrote:
> 
> "Sorry, one more added comment.  While I thought the questions were
a 
> very good NAQT set, there were a number of tossups which had
lead-ins 
> which were too easy for the topic."
> 
> Is it just me, or is this comment kind of funny?  It mentions,
almost 
> as an afterthought, the most damning of the many flaws in this 
> weekend's tournament: in packet after packet, the best players in
the 
> country were treated to incompetently written tossups that began
with 
> silly giveaways.
> 
> I know I'm going to annoy some people by writing this message, but I
> hope that someone at NAQT is listening.  Looking over my notes from 
> the weekend, I see tossup after tossup that should have been 
> rewritten.  A Popul Vuh tossup that began with the phrase "Council 
> Book."  An R.U.R. question that mentions the play's two most 
> important characters in the first line.  A John Cage tossup that 
> begins with a reference to the I Ching and proceeds to tell players 
> about the prepared piano, at a bizarrely early point in the 
> question.  An Appalachian Spring question that begins with the
German 
> title "Springtime in Pennsylvania."  A Protestant Ethic question
that 
> mentions "calling" and "Puritan" in the first line.  A Mandelbrot
set 
> question that begins with an allusion to the Julia set.  A triangle 
> inequality question that can be answered almost instantly by anyone 
> who knows the definition of a metric space.  An atrociously bad 
> tossup on "The Negro Speaks of Rivers" that begins - amazingly 
> enough - by talking about rivers!  The list goes on and on...  I 
> could easily double or triple my "bad question count" if I felt
like 
> taking the time, but I think you get the idea.
> 
> I suspect that some of you are shaking your heads as you read 
> this: "But I didn't know that 'Popul Vuh' means 'Council Book'!",
you 
> might be saying.  "Is it really a big deal if the R.U.R. question 
> mentioned its two main characters early on?" you might ask.  "Does 
> anyone actually read that play anyway?"  The problem is that the 
> initial clues in ICT-level questions should (in general) be 
> answerable only by the most knowledgeable players at the country's 
> largest national championship - a test that NAQT failed again and 
> again.  Sometimes an answer was fairly easy and accessible, with a 
> complete giveaway early on (Protestant Ethic), and sometimes the 
> answer was slightly more challenging (Popul Vuh) but the initial
clue 
> was just as obvious.  (Even if a lot of the people on this group 
> don't know the etymology of the name "Popul Vuh," I'd be willing to 
> bet that a disproportionate number of the people who got this
tossup 
> buzzed in really early.)  In all these cases, however, NAQT failed
to 
> achieve something remotely resembling pyramidal structure.  The 
> questions at the ICT frequently broke two of the cardinal rules of 
> good question-writing: never begin a tossup by mentioning a book's 
> main characters, and never begin a science tossup with a 
> straightforward definition of the answer.
> 
> A lot of people at the tournament agreed with me about the
questions, 
> but some of them seem resigned to the problem.  (I think that's why 
> no one else has posted a complaint yet.)  Others even thought that 
> NAQT had written a lot of bad questions intentionally: I had 
> conversations with several people who thought that NAQT
intentionally 
> began tossups with bad clues to increase the number of powers and 
> make the game more "exciting."  I personally think that a wide 
> variety of people work for NAQT, ranging from CBI retreads to 
> competent and dedicated (but not outstanding) players to a handful
of 
> talented and capable writers and editors.  Unfortunately, the least 
> competent people have a really big impact on the company's final 
> product, and NAQT's highest priority has never been to produce a 
> fantastic ICT.
> 
> I did have fun this weekend, though.  The fun occasionally - very
> occasionally - came in the form of good questions: the tossups on 
> Oliver Otis Howard and the Dread Pirate Roberts spring to mind, 
> though there were more.  On other rare occasions, the fun came from 
> the quality of play: Subash's dominating performance was amazing to 
> watch, and I have no doubt that it would have been just as
impressive 
> on better-structured questions.  More often, however, the weekend's 
> fun resulted from witnessing the (dys)functioning of a truly
dreadful 
> playoff system, from getting the chance to mock CBIish questions
with 
> answers like "pro bono," and from revelling in the insanity that 
> surrounds everyone's favorite NAQT personality, Samer Ismail.
> 
> That last point bears repeating.  Whether he was whining (in front
of
> teams!) about the best questions in the packet he was reading, 
> informing players of why their negs were incorrect (while the clock 
> was running!), or writing his moderator statistics on the
blackboard, 
> the esteemed Mr. Ismail was a wonder to behold.  (We know you're a 
> fast reader, Samer: you don't need to keep track on the blackboard
of 
> how many tossups you've read each round, along with how many
seconds 
> were left on the  clock.)  Again and again, Samer seemed determined 
> to convince the circuit that he was the smartest person in the room 
> and the best moderator in his bracket.  His job would have been 
> easier if the science had been better edited, or if players could 
> understand what he was saying when he read.  Pomposity, thy name is 
> Ismail!
> 
> So I left this year's ICT with a mixture of amusement, bemusement, 
> disgust, and disappointment.  I think I've described the first three
> emotions pretty clearly, but I don't want to underemphasize the 
> last.  I can never understand how it is that such a talented group
of 
> people can produce such a crappy set of questions.  Partly, as I 
> wrote above, NAQT isn't a uniformly competent group: for every R. 
> Hentzel writing interesting and competent questions, there's a Matt 
> Bruce flooding the circuit with crap.  Partly, I think that NAQT is 
> isolated from the best players on the circuit, so they don't really 
> know what people think of their questions.  And partly, I think
NAQT 
> might consider its questions "good enough": if only a few people
will 
> know the first clue on that awful Popul Vuh question, then why
change 
> it?  After all, it's not as if the questions allowed a crap team to 
> sneak into the top three.  That was the format's job!