Criticisms of the NAC

This article is about an important topic in quizbowl. However, a QBWiki administrator has deemed it incomplete. Thus, this article should not yet be used as a source of information.

The criticisms leveled against the National Academic Championship are numerous, and have been mounting over the past decade or so. They can be broken down into a few basic categories:

1. Tournament format (only six preliminary games guaranteed, and single-elimination playoffs)

2. Question quality (non-pyramidal, vague, and often incorrect tossups)

3. Unprofessionalism (favoritism towards certain teams, unfair protest resolution)

4. "Game show" quality (moderators will often play more to the audience than the players, and will affect the fairness of the game by doing so)

5. Plagiarism (several documented instances of plagiarism by Questions Unlimited, which provides the questions for NAC)

Documented accusations of plagiarism

Eight documented instances of plagiarism in Questions Unlimited material:

1) The December 2000 “Twenty Questions” featured the following question:

15. The angel is the only figure in the picture to look outwards at the spectator and his pointing right hand directs the viewer's attention to the Virgin. Mary is kneeling and puts her right hand about the little St. John, who, close to her, presses forward in silent prayer to the Christ child. The child, held by the angel and seated on the ground, answers with a gesture of blessing. The whole scene is set in a lonely, secret, mountainous wilderness with occassional glimpses into the open air and the light. Identify this painting by Leonardo noted for its sfumato and chiaroscura. [1]

The Maryland packet from the 1995 Terrapin tournament included this question: T-16. The angel is the only figure in the picture to look outwards at the spectator and his pointing right hand directs the viewer's attention to the Virgin. Mary is kneeling and puts her right hand about the little St. John, who, close to her, presses forward in silent prayer to the Christ child. The child, held by the angel and seated on the ground, answers with a gesture of blessing. The whole scene is set in a lonely, secret, mountainous wilderness with occassional glimpses into the open air and the light. FTP identify this painting by Leonardo noted for its sfumato and chiaroscuro. [2]

2) A December 2000 Quiznet summary reported the use of this question:

“Do not sit beside me and complain, you two-faced rogue.” This is what Zeus said to a god wounded during the Trojan War by Diomedes. The goddess of the dawn was one of his several mistresses. Identify this divine swashbuckler, the son of Zeus and Hera. [3]

The NC State packet from the 1994 ACF Regionals contained this question:

“Do not sit beside me and complain, you two-faced rogue.” This is what Zeus said to a god wounded during theTrojan War by Diomedes. The goddess of the dawn was one of his several mistresses. FTP identify this divine swashbuckler, the son of Zeus and Hera. [4]

3) The February 2001 “Twenty Questions” contained this question:

It ends with an offstage female chorus singing a hypnotic fragment that grows softer and softer, in what appears to be the first "fade-out" ending in music history. Completed in 1917, this piece was influenced by English folk songs, Wagner, and astrology. Name this seven-movement composition, not the nine you might expect, written by Gustav Holst. [5]

NC State’s packet from the 1995 Terrapin tournament:

10. It ends with an offstage female chorus singing a hypnotic fragment that grows softer and softer, in what appears to be the first "fade-out" ending in musical history. Completed in 1917, this piece was influenced by English folk songs, Wagner, and astrology. FTP, name this work with seven movements, not the nine you might expect, written by Gustav Holst.[6]

4) From a summary of a February 2001 QuizNet match:

It was shown by Robert Koch in 1876 to be caused by a rod-shaped bacterium; Pasteur produced a vaccine for it in 1881. What disease, both the first to be conclusively shown to be caused by a microorganism and the first to be prevented by a vaccine, is fatal within 24 hours of the first symptoms in cattle, sheep, and horses? [7]

From BYU's packet for their 1994 tournament:

4. It was shown by Robert Koch in 1876 to be caused by a rod-shaped bacterium; Pasteur produced a vaccine for it in 1881. What disease, both the first to be conclusively shown to be caused by a microorganism and the first to be prevented by a vaccine, is fatal within 24 hours of the first symptoms in cattle, sheep, and horses, and which later lent its name to a rock group?[8]

5) January 2002 Twenty Questions:

2. The brightest one yet discovered is Draco HS 1946 + 7658, which shines with the light of 1.5 quadrillion suns. Found only in very young galaxies, they have at their centers igantic black holes. As objects are sucked into the black holes, a burst of energy is released that includes light, X-rays, ultraviolet radiation, and sometimes radio waves. Name these celestial bodies, the most distant objects visible from earth. [9]

From the 1998 ACF Nationals Quincy/Furman packet:

3. The brightest one yet discovered is Draco HS 1946 + 7658, which shines with the light of 1.5 quadrillion suns. Found only in very young galaxies, they have at their centers gigantic black holes. As objects are sucked into the black holes, a burst of energy is released that includes light, X-rays, ultraviolet radiation, and sometimes radio waves. FTP, name hese celestial bodies, the most distant objects visible from earth.

6) March 2002 Twenty Questions:

13. Most RBI titles, with 8; most extra-base hits in a season, with 119; most walks in a season, with 170 and in a career, with 2,056. Highest slugging average for a season, with .847 in 1920, and in a career with .690. All of these records are still held by what New York Yankee right fielder. [10]

Cardinal Classic X: Questions by UC-Davis B and Adam Kemezis

4. Most RBI titles, with 8; most extra-base hits in a season, with 119; most walks (*) in a season, with 170 and in a career, with 2,056. Highest slugging average for a season, with .847 in 1920, and in a career with .690. All of these records are still held by, FTP, what New York Yankee, whose 61 homers in 1927 was also a record until Roger Maris broke it?[11]

7) Question 15 from April 2002 Twenty Questions:

15. Dr. M. Scott Peck uses it to describe people who take too much responsibility upon themselves and who are therefore the most likely to benefit from a therapist. Give me this psychological adjective describing people with “an emotional disorder in which feelings of anxiety and obsessive thoughts dominate the personality.” (the above question was changed in the document after the plagiarism was pointed out) [12]

Question 13 from a JCV 4 packet by Bryce Avery

13. Dr. M. Scott Peck uses it to describe people who take too much responsibility upon themselves and who are therefore the most likely to benefit from a therapist. For 10 points, give this psychological adjective describing people with “an emotional disorder in which feelings of anxiety and obsessive thoughts dominate the personality.” [13]

8) In April 2008, Gonzaga High School student Dan Puma reported [[14]] that while reading through a CBI packet, he had come across a question, that he later found repeated almost word-for-word on the Quiznet portion of Questions Unlimited's website. This report included pictures of both the original CBI packet, as well as the offending Quiznet question.

The first seven cases can also be found here

In 2008, there were claims that questions that were used in earlier aspects of the tournament (or in the middle school national championship) also showed up in the Chicago phase, despite the fact that these questions could have been discussed and shared over the internet, or in some way communicated to teams playing in Chicago.

Specifically, a 60-second round on etymology (the mystery category) that came up in Chicago had players identify the origins of ten words as being "Sanskrit, Chinese, Japanese, or Arabic". The word "ninja" very certainly came up in Chicago as the first word on the list. This is an example of question recycling, and is generally not in good form unless the host takes precautions to assure that questions and answers will not be communicated to future sites.

Question recycling continues to happen often at NAC, at the 2012 NAC at least one math question was copied straight from a Questions Unlimited packet played at the Westchester "Omar Q Beckins" tournament, and 60 second rounds are infamous for their being recycled.

In addition to uncited plagiarism, there are some NAC questions which are almost like "cited plagiarism," as they are questions ripped directly from a source instead of novel material produced from a question set. This is included but not limited to identifying things based on descriptions in the Onion and identifying members of lists produced by national surveys and magazines.

Documented demonstrations of potential favoritism and/or intimidation

Some of the problems from 2000 were discussed on hsquizbowl.

In his report on the 2005 NAC, Tom Egan noted that he felt that he did not get the feel that there was a show of favoritism or intimidation, with willful malice, but that it was likely due to thoughtlessness on the part of the tournament officials, who were trying to act the roles of game show hosts. In particular, Egan singled out some specific occurrences:

(During a match between teams from Texas and Vermont) ... Throughout the match, the host twice made friendly comments toward the Texas team, referring to their PBS station after answering a question about public broadcasting.

In a later match, involving a team from downstate Illinois:

... a question came up in the category of “Women”, which prompted a note from the host that “we have two women on the panel”. The comment was inane, and only drew attention to those two players for reasons dealing strictly with their gender, rather than their abilities as players.

In the fourth match, a different moderator also made a comment that could have been construed as intimidating:

Before the match began, the host introduced the teams as a "perennial playoff contender” and “only the fourth all-female team in tournament history”.

Before a 2008 match in Chicago involving Wilmington Charter, Chip Beall introduced the team as having finished second at last weeks "junior nationals" (a comment referring to finishing second at the NAQT HSNCT two weeks earlier). The comment was taken as a derogatory and classless sleight at Wilmington's accomplishment and the HSNCT as a whole.

In another 2008 match, after a team had rang in early with "Italy" to identify the nation whose national anthem begins with the words "Roma imortale ...", a player from the second team rang in with the correct answer, "Vatican City". Chip Beall's response was "I saw you on Jeopardy!"

Later, in this same match, after the first team had answered, and the question was finished, Mr. Beall continued to add information that was apparently not in the question, as he had stopped reading for a few seconds. This can be construed as either being poor judgement in not thinking about what he was saying, or can be construed as favoring the team that has not yet attempted to answer.

In the 2012 NAC final, Chip Beall mentioned that he "could not remember the last time" High Tech from New Jersey did not appear there, while noting of the other contender, Watson Chapel, that it had only been their first or second year at the tournaments but it was impressive that they were in the playoffs.

Documented acts of unprofessional conduct

In addition to acts singled out as "thoughtless", documented in the section on favoritism and intimidation, Egan's 2005 report noted some specific instances of behavior that would almost certainly not be tolerated at other national tournaments, and likely would have had moderators forcibly excluded from future duties, if they had occurred in his home state of Illinois. One moderator, in particular was responsible for all of the conduct which Egan labeled as "unprofessional".

In the second match discussed in his report, the following was documented about the response by the game officials after a particular question:

“Name the four former presidents not buried in the United States”. (for those not immediately sure what the question was asking, the answer was “Ford, Carter, Bush, and Clinton”). I was mortified and embarrassed as a member of the quiz bowl community that such a question would be asked at any level, especially in a tournament professing to be a national tournament. The whooping by the team which answered the question correctly was uncalled for ... <sic> The team that did not answer the question looked utterly defeated. The situation was in no way helped by the snickering by the host and judge.

later in that same match:

a question (asked) for the name which the Republican Party used in the election of 1864. When no team could answer “Union”, the host responded “Our Rich Fat-Cat Friends.....I’m a Democrat and proud of it”. Irrelevant of political affiliation, these comments were unnecessary, and would not have been any less offensive if the politics were reversed.

The same moderator, in the third match Egan witnessed, went even further:

The most grossly unprofessional act that I witnessed was in this round. After a pop music question, the host joked about Whitney Houston, and then made a gesture that implied sucking on a crack pipe; further joking about Bobby Brown. It was utterly tasteless for someone who works with young students in a professional manner to make such jokes. Again, I was mortified, and noted that while most people were laughing, there were obviously some in the room who looked uncomfortable.

As noted above, before a 2008 match in Chicago involving Wilmington Charter, Chip Beall introduced the team as having finished second at last weeks "junior nationals" (a comment referring to finishing second at the NAQT HSNCT two weeks earlier). The comment was taken as a derogatory, classless,and unprofessional swipe at Wilmington's accomplishment and the HSNCT as a whole.

The perennial NAC moderator Ernie Anderson (or "Uncle Ernie" as he likes to be called) is known for making jokes throughout moderation, especially by comparing clues to "his ex-wife" and even making jokes in the middle of questions (such as a question beginning "There are 30,000 men in the Autralian military..." being interrupted by a comment of "name them all!" before the question could be finished)

Documented emphasis on trivia vs. academia

One of the boldest claims made by the NAC for many years has been: The competition focuses on academic information - "significa" rather than "trivia" - and the questions are supplied by Questions Unlimited. Our emphasis is light on popular culture and heavy on the academic.

This claim has been challenged numerous times.

From Matt Lafer's recap of the 2000 tournament:

There was actually a question that read "Name this common household appliance" and then an audio clip of that appliance (a blender) was run...

While making his 2005 observations, Egan noted the following questions:

- One question referred to a recent vote in Oregon regarding the new state fruit. After two long sentences, the question is finally revealed to be “which fruit won?”

- Twice in the match, particular questions came up that helped the Texas team (one question asked for the time zone of the town from where the team came from. A second question was about Roger Staubach).

- “What musical instrument does Woody Allen play?”

- At one point, the host misread a literature toss-up, and needed to replace, stating emphatically “I will find a replacement literature question.” The replacement asked the players to identify a component of the “Worlds Strongest Man” competition on ESPN.

- “Name the four former presidents not buried in the United States”.

- Another question asked about a herb “whose name is the reverse of the author of The Stranger".

- In the 60-second round, the two categories that were chosen were “The Wizard of Oz” (two questions earlier, the music question had been “Judy Garland”) and “Winnie the Pooh”. The other option was no better, and, again, I was mortified that this was the level of question being asked at a supposed national caliber high school varsity tournament. Surely, there could be some very legitimate questions asked about what many consider a culturally and historically important film, though only one of the nine questions heard constituted, in my opinion, anything but cursory knowledge of the film.

- After a pop music question, the host joked about Whitney Houston, and then made a gesture that implied sucking on a crack pipe; further joking about Bobby Brown.

- The first and fourth period each included a single music listening question, though only one of them was typically of the non-pop variety.

While on the one hand, this may not seem like a great many trivial questions, it should be noted that this is a non-exhaustive list, and was made over only four rounds. Assuming that these rounds were typical, it shows that, at least in 2005, the questions were very much rooted in trivia and trash, vs. any reasonable definition of "academia". Further, compared to the other national tournaments (NCT, NSC, PAC), this sampling suggests that the NAC has the highest percentage of trivia questions and the smallest percentage of academia.

In 2008, Egan did take a more careful survey of the answer selection being offered. Of the 52 questions asked, thirteen had some aspect of trivia or pop culture. No other tournament claiming to be a national tournament has a rate of 25% of their questions being in trivia or pop culture. In the second round he witnessed, Egan found 12 such questions, but arriving late, only saw 37 questions, for a slightly over 32% rate of trivia or pop culture. Such questions as trivia included (paraphrasing): - Which face of Mt. Everest was the first to be scaled? - From which language do we get the words "anime"?, "gung-ho"?, "harem"?  - Two regiments compose what larger unit in the American Civil War? - Which cardinal direction can be found in the title of an Oscar winning film for "Best Picture"? - Which Shakespearean characters are also named in the NATO alphabet?

Many examples of poor questions can be found by reading the tournament summaries on the NAC website. Examples from the 2008 jr high championship include: - Disney announced that rights of High School Musical will be released so what places can do their own performance at the local level? (Ans. High schools) - that Peyton Manning is the NFL's most marketable player as far as commercials are concerned; that what the French called La Grande Guerre, we call World War I - The Alaska Native Language Center now considers the name ‘Eskimo’ derogatory … (Ans. Inuit) Examples from the high school championship include: - There once was a lady, Eileen, / Who lived on distilled kerosene. / But she started absorbin' / A new hydrocarbon / And since then she's never ... (Ans. benzene) - identification of Hawaii as the only state name that, when spelled officially, includes a diacritical mark (Note: This was the Final Jeopardy question on February 11, 2008.) - identify three of the four basic methods of solving a quadratic equation - recognizing a cartouche from a visual clue

Instances of non-academic and trivial questions continue to abound at NAC, including a 2012 playoff round question about Lady Gaga and numerous questions about identifying award-winning movies from the 1930's to 1950's.

In 2012 a protest caused Irvington to lose a semifinal match as the protest was thrown out largely because a failure to adjudicate it properly (an initial protest was handled only by those moderators who were in the room and was thrown out, and a second, though slightly different protest regarding the flaws of a math question which should have been replaced was disregarded, largely as by the time a decision has been reached the final match was about to begin)

Explanation of the unfairness of giving different questions to different teams in the same round

The 60 second round used at the NAC is one of the sources of its greatest controversy. It is the only time in any of the three major national tournaments (HSNCT, NSC) where teams competing in the same round are subjected to different questions.

Tom Egan explains the mechanics of this part of the match in his 2005 report:

The team behind chooses from four categories (one of which is always the “mystery” category), and has one minute to answer up to 10 questions on that topic (though sometimes the topic name is somewhat misleading). Any part that was read, but not answered correctly, rebounds to the other team. If a question was partially read when time expires, only the part that was read is read to the rebounding team, though at least in one case, only two words were sufficient (with the category) to determine the answer for the rebounding team). The process then repeats for the team that was leading at the beginning of the period, choosing from the three remaining categories.

There are three critical issues that need to be examined. The first, is that the format of this part of the match permits for the very real possibility that one team to hear a relatively easy set of questions, while the other team hears a relatively difficult set of questions. Further, if one set of questions is written with questions that are more wordy than the other sets, it creates a situation where one team will have a decided advantage in hearing all of the questions, while one team may not even get that opportunity.

The second issue is that, because of the nature of this being a timed part of the competition, the moderator needs to be careful to read equally fast for both teams. If the moderator speeds up for one team, it becomes an inherently unfair competition; even if the moderator does this without the intent of being unfair.

Third, While the questions are certainly written in advance, there is no guarantee that these questions are written before the teams are determined, and after the tournament personnel has a feel for the category strengths of certain teams or players. Even if the questions are in fact written neutrally, the choice of particular categories (often including the "mystery" category") opens this aspect of the competition to accusations of fraud that are virtually indefensible, even if they are not true.

It should be noted: the use of a 60 second round, per se, is not at issue. If the tournament were to arrange for teams to hear the same questions, and were fairly certain that the moderators could read at the same speed for both teams, this could be a legitimized part of the tournament. In its current form it is obviously and grossly unfair, and not to be considered an aspect of "good quizbowl".

Comments on the overall quality of teams participating

While this has not always been the case, over time, there have been those in the quizbowl community who have called into question the overall quality of the field which participates at the NAC.

One red flag which could indicate the field's overall poor quality is the fact that a large percentage of the teams come from a relatively small number of states (compared to the roughly 35 + states represented at the NAQT HSNCT, with no more than roughly 10-12% of the field coming from any one state). Speaking from a strictly statistical standpoint, the chances of so many national caliber teams coming from one state is considered unlikely.

Take for example, Virginia; a state that has won more national titles in high school quizbowl than any other. Despite the overall success of Virginia in national competitions, only a very small number of schools has managed to place at nationals. Despite this, some several dozen schools are invited from some states to participate.

This leads to a crossover discussion involving professionalism. In other tournaments, teams are not very actively recruited to participate in the national tournaments (PAC requires a selection process of some kind; NAQT requires teams to qualify through tournament experience). The NAC does maintain a qualification process, but also actively invites teams to sign up for their tournament, despite the high expense, and the chances of advancing being very small.

While these invitations were normally a form letter or e-mail of some kind, in 2008, the NAC began cold calling teams (three Illinois teams: Loyola Academy, New Trier, and Maine South confirmed that they were cold called by an NAC representative) to sign up. In the case of Maine South, not in the midst of a great season, the team had not otherwise qualified for the NAC, but was being approached to attend anyway. One interpretation of this is that the NAC was looking for more teams to pay the entry fees, even if they had no chance to advance and win. This begs the question as to whether or not the NAC was more interested in making money, or building a field of national caliber teams.

In 2005, the NAC champion Holland Hall participated in the NAQT HSNCT and finished 20th. The 2006 champion Byram Hills attracted head-scratching from many high school quizbowl watchers, as they didn't appear to compete in any tournaments outside of Westchester before winning the NAC. In 2007, the tournament champion Harrison was rated as the 37th-best team in the country by the Bykowski computer rankings and finished 35th in the Fred Morlan national poll. In 2008, the Booker T. Washington team that won the tournament finished 24th in the Bykowski ratings and received 0 votes for Top 25 status in the Morlan poll.

Nonpyramidal, vague, or incorrect tossups

At the 1995 NAC, Brookwood came out on the losing end of what could only be called a controversial decision centered on a poor, open ended question that required fine interpretation. The details of this event and the poor question can be found here. This also highlights another problem with the NAC: while poor questions pop up with regularity, sometimes the tournament director refuses to take responsibility for his actions or those who work under him, and would rather create a situation where the blame cane be deflected to one of the teams involved.

Egan's 2005 visit to the NAC focused more on overall impressions, instead of focusing on particular questions. Nonetheless, there were a few questions that appeared in his report that qualified as poorly written:

At one point (paraphrasing the question), a question asked “While an airplane has a fixed wing, what name is given to the rotating wings on a helicopter?” The first team answered “blades”, which is a commonly referred to alternative. It was not accepted, and no protest was filed (the other team earned the toss-up with “rotor”).

The most damning evidence of the overall question quality being poor, was the one non-question that Egan noted:

One of the single worst questions asked in my years in quiz bowl was asked in this round: “Name the four former presidents not buried in the United States”. (for those not immediately sure what the question was asking, the answer was “Ford, Carter, Bush, and Clinton”). I was mortified and embarrassed as a member of the quiz bowl community that such a question would be asked at any level, especially in a tournament professing to be a national tournament.

Egan later noted that this was not a question, as much as it was a riddle, and was utterly indefensible as being appropriate in any quizzing beyond "the second grade level".

In 2008, the NAC homepage included the following defense of its actions: We Love the Pyramid, but Don’t Worship It

A Vermont school turned down our invitation last spring to join us at Nationals, saying, “We prefer tournaments that use pyramid-style questions.” It seems that a bit of misinformation has been spread about the kinds of questions we use at Nationals. Pyramid-style questions – three- or four-sentence tossups that start with the difficult or obscure and end with the easy and accessible – are our favorite kinds of questions, and always have been. Almost all of our tossups-leading-to-bonuses are pyramidal. On the other hand, to insist that all tossup questions must be pyramidal is narrow-minded and fatuous. Math calculation questions, of course, make great tossups, and can only be made pyramidal artificially. Questions starting with “why” make great tossups; e.g., “In 1174 King Henry II walked barefoot from London to Canterbury. Why?” (Ans. He was doing penance for having ordered the murder of Thomas Becket.)

Mr. Beall's statement above shows a fundamental misunderstanding as to what pyramidal style is (in the several rounds that Tom Egan witnessed, he only identified two pyramdal questions being asked at the NAC). His insistence on the "why" questions further demonstrates a disconnect from good quizbowl in that "why" questions often have numerous answers, though the "answer on paper" is often oversimplified. Further, the question Mr. Beall cites on his website is an example. While some see "penance" as an act motivated internally to gain forgiveness, many believe that (given his four year delay) that Henry performed this act to avoid the excommunication of the whole of England. It begs to question: would "to avoid the excommunication of England" have been acceptable? Based on the tournament history, this legitimate answer would likely have been refused.

As a part of Egan's return to see the 2008 NAC, he observed two matches (11 am and 11:30 am on the first day of the touranment).

One question asked about an event that took place on busses beginning in Alabama. The first team rang in and answered: "Freedom Riders". The answer was disallowed, as the correct answer was "Freedom Rides", with Beall explaining after the question "You clearly knew the answer." It is unclear as to whether other moderators were as strict in their interpretation of the answer.

In a later round, the following, an example of a "hose" came up (paraphrasing): Adolph Hitler was in prison writing his "Mein Kampf" before coming to power. What nation did Hitler specifically write about invading in order to gain "living space"?

The problem occurs in the lead in, which is specifically asking teams to ring in and answer Mein Kampf within the first few words. It is a misleading hose meant to trick a team with deeper knowledge. In fact, if this occurs, the team that did not already know the title of Hitler's book almost certainly will not guess the correct answer (the two teams guessed Czechoslovakia and Austria), but if the second team was only good enough to buzz a little slow, and not get hosed on the title, they can clean up the answer at the end, not because they were necessarily smarter, but because, by sheer luck, they were slower on the buzzer.

Questions at the NAC continue to be vague. 2012 examples of this issue include a bonus category entitled "why?" in which, like any such vague question, it is difficult to determine what counts as a correct example. Often, perfectly acceptable answers are not accepted simply because of poor construction - for example, a question using only the identify "this place" was not even prompted when a country was given, as the answer was a city within that country.

Nonpyramidal questions or "buzzer beaters" or "speed check" questions continue to be present, an example from the 2012 NAC was the question "identify the value of a which makes the following statement true for all theta: sin^2 theta + cos^2 theta = a" - a question which can only be buzzed on at the end and tests very basic knowledge, thus differentiating between teams' buzzing abilities more than their knowledge.

Criticisms of the NAC

Contents