Debate experiments

Three theories and five debates I've run

I like debate. I have done for years. So I have been slowly trying to improve it. Here is a set of theories I had and things, experiments I've run so far.

Theory: Any debates are good.

Are any debates actually good at all? Should I give up?

Test: Watch different debates.

Evidence: I much prefer some debates to others.

Good debates:

• Dr. Richard Carrier andDr. Michael Licona. I like how they chat to one another.

• Destiny and Ben Shapiro. I recall liking this one. I remember them as having good chemistry.

• Jubilee’s “Surrounded” debates. I love an experimental format and these get a lot of different arguments in a short amount of time¹.

Bad debates:

• Finkelstein, Destiny and M. Rabbani & Benny Morris. Long and acrimonious. I think Lex Fridman is deeply guilty of the “I’ll just let them talk it out” school of debate. I think this is lazy.

• Most things with William Lane Craig. Craig is an excellent debater on theology. I’m not sure I recall him ever losing. But his debates always hinge on niche points or technical arguments I don’t care about.

• Anything with Jordan B. Peterson. Like trying to nail a cake to a wall.

• Presidential debates. Trump in particular can lie with no cost at all, so he does.

Unclear:

• Ezra Klein, Sam Harris. Bad that they don’t understand one another, but pretty interesting as a historical artefact to see two clever men who I like really fail to understand one another for very ~2018 culture war reasons.

• Matt Dillahunty, Matthew Adelstein (aka Bentham's Bulldog). Dillahunty is sloppy but somehow his audience think he’s making good points. Frustrating to watch.

Status: Theory survived attempted falsification².

Theory: The format is the problem.

Test: Run some different debate formats (see next).

Theory: Debates are bad because debaters focus on their own status.

They have to focus on how they appear to the audience and this stops them admitting points where they are wrong.

Test 1: Find ways to protect the status of the debaters

Evidence:

I tried running two debates like this at The Curve (Daniel Kokatajlo vs. Sayash Kapoor; Dean W. Ball vs. Gabriel Weil). I tried to moderate a bit more strongly than people tend to, ensuring that there were blocks of time where each was in control of the discussion.

The debates were okay but not great.

In both, it took us a long time to get to what felt like the meat of the discussion. I recall Ball and Weil saying they didn’t really understand one another’s position coming in.

In the the Ball vs. Weil debate, they weren’t really interested in being moderated, which to me felt like Ball therefore spent a lot more time defending his position and had less control over the discussion than I might like to see (though I think he was fine with it).

Kokatajlo and Kapoor felt solid debate, though not spectacular.

Test 2: Try and remove the status of the debaters and place it somewhere else.

Evidence: Courtly debates, Future of the Democratic Party, China discussion.

Ray Rafiq and I have had a goofy idea for a while of debates in a court style. king, knight, fool, etc. So at LessOnline I tried this out. Each debate had a king (or queen) to set the topic, two knights to argue it and a fool to ask questions. They took about 10 minutes each

I think our debaters (knights) were much less focused on their own positions than other rapid fire debates we could have run. In many ways it was a role play game. But it did feel like I partly succeeded in my aim - to pull status away from the debaters and put it somewhere else.

Later, Oliver Habryka wanted to run a session about the future of the Democratic Party. I pushed to try a new format there too, suggesting that Oliver would stand as the questioner and the dicussion would be about what interested him—whether somebody would speak, whether the audience would be able to ask questions would really be up to him and then I would serve as a meta-moderator to guard his time and attention. Habryka is a good candidate here because he's high status (CEO of Lightcone Infrastructure, who organise LessOnline) within the community and people respect his thinking.

This felt really good. There was a single questioner which provided a single viewpoint, rather than many questions from the audience or a rambling discussion from the panel. To me, this gave the event shape. Questions were answered, things were put to the side as new directions were investigated.

A couple of anecdotes:

• At one point someone in the audience put their hand up and one of the panel pointed at them, so they asked a question. The panel member was about to answer, but I interrupted and asked Oliver whether he was interested in the question. Oliver said no, so the panel didn't answer it. This felt jarring but good. We expected Habryka to have better taste than the typical audience question³.

• At another point, Oliver wanted to sit and think. Somebody asked if they could ask a question, and I said no. It's strange to have a room full of people sitting in silence, but the typical 30 seconds of a talk is pretty mediocre so it doesn’t actually seem that bad to lose. Then Habryka asked a question.

This felt like a genuine success in that we had a panel and they were being called on to answer questions that felt interesting to someone we resepected. For me a failure mode of debates is that debaters are scared of losing or trying to take turns and so what’s being discussed is not really of interest to anyone.

Next, I ran the discussion after a talk by Steve Hsu where he and Noah Smith discussed China. This was okay. At points it felt quite alive between them. But it could have been better for having somebody who was more willing to argue for US values. And perhaps someone to pin down Steve on specific facts about China, which Noah didn't really do (nor did he claim he would, professing not to be an expert⁴).

Status: This theory is doing okay. I have had a couple of good events, but it’s unclear to me what great might look like.

Current top theory: A good investigator is best

My current top theory is that it really matters who is moderating/investigating. And that if this person is willing to hold the debaters/panel and force them to answer the difficult questions or engage with them, that makes a much more interesting debate than otherwise.

I suggest that Dwarkesh Patel is a particularly good podcast host because he is so knowledgeable on AI topics and so willing to actually chase down his guests and say things like "okay, but what about the data centre built in Saudi Arabia?"

Suggested test: Future conferences, podcasts.

For the next set of conferences I run, I might like to focus on finding a good investigator for a topic and then choosing panelists afterwards and build an event around trying to understand AI, China, Ukraine war.

It's possible I'll also try the strategy for my podcast, which I haven't done episodes for in a while.

Other theories I may test later

• Debaters should discuss beforehand. It's fun for people to discuss on the day because there is something very alive about people discussing things for the first time. But it seems worth to me having a short discussion beforehand to figure out the exact areas of disagreement and to check that there won't be 20 minutes of discussion on the day that could be avoided.

• Debates are fun discussion pieces, but less good for sharing information. Debates are primarily useful as a way to set up discussions happening at a conference or to see discussion in the public sphere.

• The key thing is who the debaters are. This seems too powerful. The point of a debate is that it's a format that allows two people who disagree to produce valuable work for other people. But if one has to select very carefully the two people who disagree, then that suggests that debates are much less valuable than one might want.

• It would be better to have someone explain a field. It's possible I'm too focused on debate and that trying for a collaborative explanation or an overview of the field might be good. I struggle to think of a good format here.

One more thing..

Duncan Haldane built a home made Nielsen rating system that allowed audience members to twist a knob to display either red or green lights on their head. If they were interested, they turned to green. If they were bored, they turned to red. I didn't catch discussions where this was used, but it felt like a pretty interesting thing to do to be able to monitor people's interest in real time. And I can imagine using tools like this with a set of trusted “tastemakers” to guide an investigator on what interested some relevant group.

I'm not super interested in giving every audience member these because in general I think large groups of people can have quite poor taste⁵.

1. The main issue with Surrounded is that the circle often removes good debaters because they disagree with the specific arguments as opposed to because they are doing badly. If you don’t follow, watch one! They are really good. eg here

2. I know it's kind of retentive for me to describe theories like this, but it does seem important to me to note that theories are never validated, they just survive falsification.

There is no way to know that a theory is correct. How could you know? Instead, you have a theory and you test it until it breaks. And our best theories are the ones which have survived loads and loads of tests.

3. A better version of this would be to have an app where people could upvote questions and allow the questioner to see these in case any lines of inquiry were interesting to them.

4. To me this felt too humble. Smith is a solid commentator on geopolitical issues with a moderate knowledge of China and better than almost all of the attendees, I’d guess.

5. The median of a large group is quite accurate, but I tend to think the media they produce is not very interesting. Accurate but not tasteful. One to consider for LLMs perhaps.