All Things Bonus

A guide by Seth Teitler regarding improving quality of bonus questions. This guide was originally posted on hsquizbowl's forums

Text

Now that I've got even more free time, here's a ridiculously long post on making an already-pretty-good game slightly better. Enjoy.

The recent/current threads on reducing packet-writing requirements, the future of collegiate quizbowl, packet submission etiquette, and various tournament set discussions got me thinking about the rigors of editing today's packet submission tournaments. It seems to me that there's a larger population of decent-to-good question-writers now than there has ever been, which is fantastic. It also seems to me that this hasn't yet translated into a significant reduction in the amount of time editors have to spend working on a tournament set to bring it in line with today's high standards, and that's not so fantastic. In general, I think we should try to find ways to shift more of the set production burden onto the writers. In particular, I think that most decent-to-good question-writers should start by working on their bonuses.

My impression is that there are lots of people that understand the tossup-related concepts of pyramidality; hard, unique lead-ins; and (subject-appropriate, if possible) giveaways. Once a writer has those things figured out, I think the next two things to work on are bonuses and middle clues in tossups. Of those two, I think it might be more productive to talk about bonuses, because I think good bonus-writing habits are easier to acquire than the skill of finding good middle clues to tossups.

There have been a number of pretty good tournament sets this year that left various players (including some high-level players) expressing variants of the sentiment "The questions were generally good, but the hard parts on the bonuses were too hard." I think overly-hard bonuses are detrimental to the cause of good quizbowl: specifically, I think they make it less likely that the most knowledgeable team will win most of the time. There will always be statistical fluctuations/upsets in a game played on a finite number of questions, but I think it's "good quizbowl practice" to try to write questions so that more knowledgeable teams have as many opportunities as possible to take advantage of their superior knowledge, and that means that super-hard bonus parts have to go.

I'll pause for a moment to note that there's been some debate over the relative merits of the "quizbowl as a game" and "quizbowl as an opportunity to learn cool stuff" viewpoints. I subscribe to both, but I think the game aspect has to trump the learning aspect, at least at the level of bonus-writing that I'm talking about. Also, I think it should be possible to work in some amount of cool new material as clues to stuff knowledgeable teams can answer without forcing it in as answers no one (or almost no one) can give. I would argue that this makes for a more satisfying playing experience and at least as satisfying a learning experience: if you bring in some cool new thing as an answer and you can't link it to anything people recognize, many people won't be interested; if you can link it to something people recognize, why not make the recognizable thing the answer and use the new thing as one of several clues?

Moving back into bonus theory: I'm advocating the idea that the hardest parts of bonuses should be less hard, in general, than most people make them. Let's talk numbers: suppose you have a tournament with 20 teams, on the large side (I think) for a typical invitational. Suppose you write/edit a bonus with a hard part that only 1 team can answer (or 5% of the field). Well, about half the time that bonus will see a 30-point conversion; half the time, it won't, when that team's opponent gets the bonus. This seems pretty lame to me, especially when you factor in that teams routinely fail to answer things they know--because they misunderstand the question, because they're too tired to pull the answer, or whatever. If no one at a decent-sized tournament answers a bonus part, that bonus part has completely failed to distinguish any teams on the basis of knowledge; I think this should be minimized. There's a similar argument against easy bonus parts that every team can convert in their sleep, but I don't think those are a problem in today's tournaments. I would argue that the hard part of a bonus should aim for 15-20% conversion by the field. In a 20-tournament field, this translates to 3 or 4 teams knowing the hard part.

Let me put this another way: suppose you only write three part bonuses where each part is worth 10 points. Then in some sense, you're sorting teams into 4 classes: teams that will earn 0 on the bonus, or 10, or 20, or 30. If your goal is to provide the best-possible ranking of teams based on knowledge, it makes sense to write bonuses so that all 4 of these classes are populated. It doesn't make sense to write a bonus so that everyone gets clumped into the "will earn 10" and "will earn 20" classes. To my mind, 1 team out of 20 is insufficient for the population of any class. Occasionally people write a bonus with a hard part that they know is extra-hard, then try to make up for it by making the easy/medium parts easier than normal. This seems suboptimal; there's always variation from one bonus to the next, but it seems to me the appropriate thing to do when you recognize that a hard part is too hard is to rework or replace it. Also, while making the easy/medium parts easier helps preserve the average bonus conversion rate, it actually does a worse job of sorting teams by knowledge level than leaving the easy/medium parts alone.

While I'm pulling numbers out of thin air, I'll go ahead and toss out 80-90% and 40-50% as proposed conversion rates for the easy and middle parts. Note that this gives an average conversion of about 15 ppb. The specifics of the easy and middle part conversion rates and the total average ppb don't interest me much right now, but if people want to talk about them (or anything else bonus-related), go for it.

I'll assume the high-powered analysis I presented above has convinced everyone that hard bonus parts should be written so that there's a decent conversion rate, and move on to bonus practice. There are lots of ways to make hard bonus parts gettable. First off, you have to forget about answers that the tournament audience doesn't know anything about--by definition, there's no way to make these gettable using acceptable clues. Sometimes people write this type of bonus part because they're writing in a subject they know little about; it's unfortunate, but it's understandable. I'm more concerned with people that should know better, but still submit bonuses with really hard parts in topics they know well (see example 1 below). For people writing bonuses in topics with which they're not comfortable, here's a suggestion: the hard part of a bonus doesn't have to have an answer that's not commonly known--you can write a hard bonus part on a well-known answer if you use hard clues. I think this type of hard bonus part is also easier for editors to work on and use than a hard bonus part on a marginal answer, which makes it more desirable.

Another source of poorly-constructed bonuses: sometimes people come up with an idea for a bonus, then find that they can't find a good third part and, rather than abandon the bonus theme, write a bad third part (usually a ridiculously hard part). I would suggest widening the scope of the bonus (see example 2 below [also example 1 again, I guess]) until you can find an acceptable third part.

Here are two examples, both written by me for this year's Cardinal Classic.

Example 1:

Name these lazy bums of Norse myth that will survive Ragnarok, FTPE.
[10] This son of Rind and Odin refrains from washing his hands or combing his hair until he avenges Balder’s death. He accomplishes the task the day after he is born, after which he presumably can wash his hands and comb his hair all he likes.
ANSWER: Vali
[10] This long-legged god pissed off the Vanir by refusing to give them counsel after being sent as a hostage, so the Vanir naturally beheaded his wise companion Mimir and left him alone.
ANSWER: Honir or Hoenir
[10] This god is said to occupy the enviable position of second-strongest of the Aesir. When he’s not busy fulfilling his duties as the god of silence, he can usually be found putting together scraps of discarded leather to make the Thickmost Shoe, with which he will kill Fenrir at Ragnarok.
ANSWER: Vidarr

Example 2:

It united with the Carlist party during the Spanish Civil War. FTPE:
[10] Name this nationalist movement which brought Franco to power.
ANSWER: Phalange or Falange Española Tradicionalista Y De Las Juntas De Ofensiva Nacional-sindicalista or Traditionalist Spanish Phalanx of the Juntas of the National Syndicalist Offensive
[10] The Falange was founded by the elder son of this man, who took control of the Spanish government for seven years as Prime Minister under Alfonso XIII after leading a 1923 coup.
ANSWER: Miguel Primo de Rivera y Orbaneja, Marqués de Estella
[10] Miguel’s daughter Pilar Primo de Rivera formed this organization, the women’s branch of the Falange.
ANSWER: Sección Femenina or Feminine Section

What's up with these bonuses?

In the first case, I decided to knock out a Norse myth bonus for "my choice" since I like Norse myth, then I started thinking about gods that survive Ragnarok, and then...I picked three of the hardest possible answers. I'm not really sure why I did this. I think Hoenir might be too hard, even for a hard part answer. I think Vali and Vidarr should be okay, but certainly the three of them put together are not okay. I think at least one team at Cardinal Classic could 30 this bonus, maybe even two teams, but it has no distinguishable easy and medium parts. It's hard for me to imagine a team that could 10 this bonus. A couple days after submitting my team's packet, I looked back over my questions and decided the original version of the bonus was insane. I then rewrote the bonus and submitted the following version:

Answer the following related to some lazy bums of Norse myth who will survive Ragnarok, FTPE.
[10] This long-legged god pissed off the Vanir by refusing to give them counsel after being sent as a hostage, so the Vanir naturally beheaded his wise companion Mimir and left him alone.
ANSWER: Honir or Hoenir
[10] When Vidar’s not busy doing whatever it is that a god of silence does, he works very slowly on the Thickmost Shoe, with which he will kill this son of Loki at Ragnarok. This gigantic wolf ate Tyr’s hand.
ANSWER: Fenrir or Fenrisulfr
[10] Vali gets to sleep in late, wash his hands and comb his hair all he likes after accomplishing his sole task in life one day after his birth, when he kills this blind god. This poor guy was duped into killing his brother Balder.
ANSWER: Hoder or Hodur or Hodr

This version's hard part (Hoenir) may still be too hard--I'm not really sure how many teams know about him--but I'm much more confident that Fenrir and Hoder make fine easy and medium parts than Vali and Vidarr. Note that I had to widen the scope of the bonus slightly--the answers aren't all "lazy gods that survive Ragnarok" any more.

Moving on to example number 2: this is me writing a bonus in a topic I don't know well. I found some stuff on José Primo de Rivera and decided to make a bonus on the Primo de Rivera family and the Phalange. Unfortunately, I don't think I found a good hard part. Perhaps multiple teams at Cardinal Classic know about Pilar Primo de Rivera's work with the Sección Femenina, but I doubt it. In this case, the editors did a fine job of widening the scope of the bonus and adding a better hard part: they replaced the third part with a part on Sanjurjo, if I remember correctly. I don't know for a fact that multiple teams at CC know about Sanjurjo, but it certainly seems more likely.

Moving on to the Cardinal Classic discussion thread: from what I read of the indie music discussion, it sounds like there's a subpopulation of quizbowlers that follows indie music in some depth, while the majority of quizbowlers know very little about it. Given that the proper structure for an indie music bonus is the same as for a bonus on any other topic, this suggests that the hard part should be somewhat challenging for the people that follow indie music, and the easy and medium parts should be significantly easier, since there will be a sizable number of teams that don't have an indie music expert. The easy part may even have to be non-indie, just to achieve an acceptable 10 point conversion rate.

Really long story short: people that have figured out the basics of writing tossups should be able to write/edit good bonuses, especially in areas they know well. I think the most common problem with non-good bonuses is that they are too hard to 30. Suggestions for avoiding this problem include widening the scope of bonuses (e.g., including 1 or 2 non-indie music parts with a harder indie music part, rather than 3 indie music parts that most teams can't get) and using an "easy" answer with hard clues as the hard part of a bonus.