Statistical nihilism and culture-war island hopping

Aug 2021

The Guadalcanal campaign was the first major offensive operation by the Allies in the Pacific theater of World War 2. This nightmarish battle ran for six months and—while an Allied victory—involved losses so high the US Navy refused to release casualty figures for years afterward.

When this campaign ended in 1943, Japan’s major forward base in the South Pacific was Rabaul, a town Japan had seized from Australia in 1942 and fortified with aircraft, ships, structural defenses, and over 100,000 troops. General MacArthur wanted a direct attack on Rabaul. However, the US joint chiefs would not shift resources away from the war in Europe and likely feared horrendous losses.

rabaul

Eventually, the Allies settled on a strategy of island-hopping: With air and naval supremacy, it was decided to simply avoid heavily fortified areas like Rabaul. Instead, the Allies took only the fewest and most weakly defended points necessary to establish supply lines towards the Japanese mainland.

So, here’s a theory: If you want to make the world better, culture-war debates are often like attacking Rabaul.

Statistical nihilism

I spend a lot of time sort of arguing that nothing means anything. For example:

  1. There are lots of ratio arguments about police bias. Some claim the number of people killed per capita proves the police are racially biased. The problem is that in current society, races vary in basically every dimension: Age, income, where they live, etc. These would produce racial discrepancies, even if race itself were invisible. Other people claim the number of people killed per violent crime arrest proves police aren’t biased. This doesn’t work either because the number of arrests could itself be biased.

  2. Some claim that standardized tests are useless for predicting success in college. The argument is that if you already know grades, test scores aren’t predictive of finishing a college degree. The problem is that this is based on a regression that conditions on if a student goes to college at all and what college they enroll in. But those colleges—at least when the data was collected—used test scores to decide which students to accept! And more selective colleges have higher completion rates. Here are two possible causal chains:

    two causal models between college prep, test scores, and college performance

    If it’s the one on the left, removing test scores from admissions would mean that more students with low scores would get into selective colleges, and thus more of those students would graduate. If it’s the one on the right, those students would still graduate at lower rates and see little benefit. Which is it? Very hard to say.

  3. The gender-equality paradox is the idea in more gender-equal countries, fewer women participate in STEM. It’s debated if this really exists, but if it did, would that mean women were “intrinsically” less interested in STEM? No. Maybe parents in poorer countries suppress their biases and guide daughters towards high-paying jobs. Maybe it’s a product of different access to other non-STEM career paths, or some complex cultural feedback loop. Beyond all that, we should remember that “interest” and “culture” aren’t separate things—culture influences what people want!

  4. Heritability of human traits is uncomfortable for people across the political spectrum. People on the right worry that if skill at high-paying jobs is heritable, that damages the case for a meritocratic society. People on the left worry that high heritability would imply that inequality is “natural”. The thing is, the technical definition of “heritability” is just a ratio. This can have weird consequences, like the heritability of hair color changing in different countries, or heritability being higher than 100%.

    Statements like “intelligence is 60% heritable” aren’t as consequential as people often think. Many people suggest that we should just “fix” the technical definition of heritability to better correspond to the everyday meaning of the word, but it’s not clear that’s possible. Like most words, the way people actually use “heritable” is probably inconsistent and impossible to formalize.

What’s the point of all this? Why spend so much effort just to point out that, whatever the subject, there’s no implications for what we actually care about? Partly, it’s a personal fixation that statistical arguments should be careful and not over-claim. That is perfectly valid! But there’s a deeper reason, too.

We argue a lot

We tend to care a lot about culture war questions. Often, we believe we know the truth, perhaps because of personal experience or strong convictions about how the world works. And surely we need to agree a problem exists if we want to fix it, right?

Well, how’s that going?

Here’s what I see. For issues without clear culture-war implications, some semblance of rational discussion is possible. Between January and July 2021, the US House and Senate overwhelmingly passed bills to speed disability benefits for people with ALS, avoid cutting Medicare, allow FDA to use a narrow definition of “active ingredient”, treat sesame as a potential allergen, temporarily allow foreign cruise ships to skip stops in Canada, fund bone marrow transplants, prohibit reverse auctions for specialized construction work, and make June 19 a national holiday.

Many perceive Congress to be totally deadlocked, but that’s not right. It’s just that with rare exceptions (June 19) bipartisan stuff passes quietly and isn’t relevant to culture war.

Even well-publicized bills can proceed with some sanity, provided they aren’t culture-war relevant. Take the debates about infrastructure spending that happened in Summer 2021. These were quite partisan and well-publicized, yet the discussion seemed to be at least somewhat grounded in the reality that it would be nice to have better roads/rail/ports/internet, but all that stuff costs money.

For issues that become culture-war signifiers, forget it. In politically diverse institutions (the US Senate) everything stops. Even if someone is convinced by an opposing argument, they can’t change their position because it’s terrible politics to give the other side a “win”. Bringing more attention doesn’t create a healthy debate, it makes debate impossible.

Or take a politically homogenous institution, like a university. If everyone is “on the same side” on an issue, that often leads people to support every possible measure regardless of effectiveness. Witness the attempted censure of Steven Pinker for misdeeds like writing an equivocal review article about the intelligence of Ashkenazi Jews or suggesting it was a bad idea to abolish the police. My best guess for what’s happening here is that once there’s a “correct” opinion, a sort of moral superiority arms race starts. Eventually, even people who agree with the general opinion can’t question any evidence or mention any downsides, lest they look heretical.

As far as I can see, you don’t “win” culture war. If you tell the people who disagree with you that they are horrible and evil, they’ll probably just dig in. And if you tell your allies that anyone who disagrees is a monster, you won’t have a very nuanced internal discussion. We aren’t finding any consensus, we’re just yelling past each other.

Culture war island-hopping

Many people say that culture war is a waste of time. But isn’t that kind of defeatist? Policing is important. Education is important. Because these topics have become polarized, are they now off-limits? What’s the plan, to sit around and watch impotently as the set of things we can talk about shrinks down to nothing?

And anyway, suppose you won the culture war. What now?

It would have some value. It might bring attention to a neglected issue. Fine, but it wouldn’t tell us what we should now do. For example, say you proved that women and men have equal intrinsic interest in STEM (however you define that). That suggests something be done to equalize things, but what?

The alternative to culture war is to focus on interventions. This has three advantages.

The first advantage of interventions is that we have good methods to arrive at truth. In the recent paper Systemic Discrimination Among Large US Employers, researchers sent out 83,000 fake resumes, which were equivalent except for having stereotypically black or white names. They found that resumes with “black” names were called back 23% of the time, compared to 25% of the time with “white” names.

This is about as culture-war of a topic as it gets but, as far as I can tell, it caused little controversy. Partly, I suspect that’s because the magnitude of the result is neither large nor small enough to make a good headline. But is it naive to think that the lack of quibbling is partly because there isn’t much to quibble about? The researchers changed names and nothing else—they intervened! This makes it hard to argue with the conclusion.

People, in general, are terrible at finding flaws in research that reinforces their views but excellent at spotting flaws in research that challenges those views. When two sides are locked in a trench-warfare “my research says A” vs “my research says B” kind of thing, that’s often because the research being cited on both sides is flawed.

A second advantage is that interventions help us focus on tradeoffs. Engineers know that changes to complex systems usually have many effects. In a car, making an engine bigger might increase acceleration but decrease fuel efficiency. In human bodies, genes for the Asian flush decrease alcoholism but make alcohol more carcinogenic for people who drink anyway.

Society is a very complex system. Creating civilian review boards might reduce police use of force. That would be good, but maybe police do their job less aggressively and crime goes up. Or maybe police don’t like being reviewed and you have to raise salaries to keep enough employed. I suspect the tradeoff is positive, but that’s an empirical matter, not something I can just assert as obvious. A conversation about “are police biased?” crowds out these questions.

A third advantage of interventions is that it’s easier to convince someone about them, and you don’t need to resolve culture-war to do it. Consider policing again. Here are some of the policy suggestions I’ve seen:

  • Increase / decrease the number of police officers.
  • Increase / decrease prison sentences for certain crimes.
  • Ban shooting at moving cars.
  • Hire more mental-health workers to work in collaboration with police.
  • Eliminate qualified immunity.
  • Make more records on use of force public.
  • Mandate implicit bias training.
  • Require officers to de-escalate or exhaust alternatives before resorting to force.

Say you have evidence that banning chokeholds reduces the number of people killed by police without causing any increase in crime. A hardcore police supporter might agree to that policy if it’s phrased as an evidence-based policy tune-up rather than a win for Team Police Are Bad.

If we can resist the urge to talk about grander themes and stick to the effects of what we can do, we’d have a better chance of good policies.

Control air and sea

As well as increasing focus on interventions, it would be good to decrease focus on intractable culture-war debates.

What happens today when someone makes a culture-war claim based on sloppy reasoning and dodgy statistics?

  1. Ideological allies make broad supportive comments.
  2. Ideological opponents are dismissive and fussy about the details.

And what happens when someone makes a culture-war claim based on careful reasoning and rigorous statistics?

  1. Ideological allies make broad supportive comments.
  2. Ideological opponents are dismissive and fussy about the details.

It’s indistinguishable. There needs to be some incentive against ideologically convenient theories built on mirages of evidence. We need more voices who focus on the strength of claims rather than how good the conclusions are.

To some degree, fact-checkers play this role, but they are supposed to check facts that are either right or wrong. If someone is making a bald-faced lie, by all means, call it a lie. But with culture war, it’s often that someone makes a subtle statistical error or overgeneralizes from an unrepresentative dataset or something. I don’t think it’s helpful to award Pinocchios in these cases.

A more gentle approach would be more effective. Treat all arguments with the same skeptical eye. Point out the flaw in the argument, explain why the broad question is so hard to resolve, and then turn the debate towards questions where there are answers: those about interventions.

Counterarguments

I see two major counterarguments. The first is that it assumes that you view policies in consequentialist terms, where what matters is effects. If you truly think that what’s important in policies is the spirit behind them, then I’ll reluctantly concede that culture war might be the right choice for you. Also, for some issues—say creating a new national holiday—symbolism is the whole point.

The second is that maybe “nature finds a way”. My assumption is that culture war is intractable because it focuses on controversial questions that are beyond the reach of evidence. The other possibility is that culture war is intractable because… we just love intractable arguments. Maybe it’s human nature to see what’s in the news and then line up for trench warfare. The drive towards tribalism and group cohesion seems stronger than the drive towards cold-blooded analysis.

This second counterargument worries me. One response is that, sure, maybe most people won’t do this, but you can, and if it’s a good idea you’ll help on the margin.

But is it a good idea? If everyone focused on the effects of interventions, would we get people across the political spectrum high-fiving in the streets and cheering the benefits of fixing particulate levels in subways? Or would we get “You don’t want the Fed to do inflation targeting—what are you, a fascist?

I think this would happen to some degree. But the discourse just isn’t big enough to support hundreds of policies. Already today, both Republicans and Democrats give idiosyncratic answers when asked about specific issues. Maybe some high-profile issues become super polarized, but I doubt it could happen everywhere.

TLDR

Broad culture war questions are irresolvable mind-killers, but we can’t simply give up rational discussion of huge realms of human life. Statistics might not be able to answer those questions. What is can do is find the effect of interventions: What happens after you do something? We should skip the broad culture-war, and ask “if we push this policy lever, what are the effects”. At the same time, we should try to deflate the culture-war by skeptically evaluating all broad claims. Studying interventions is less controversial, more grounded in evidence, and must happen eventually anyway. Directly fighting the culture war is like a frontal assault on Rabaul.