[Disclaimer: I’m not an AI policy person, the people who are have thought about these scenarios in more depth, and if they disagree with this I’ll link to their rebuttals]

Some people argue against delaying AI because it might make China (or someone else) “win” the AI “race”.

But suppose AI is “only” a normal transformative technology, no more important than electricity, automobiles, or computers.

Who “won” the electricity “race”? Maybe Thomas Edison, but that didn’t cause Edison’s descendants to rule the world as emperors, or make Menlo Park a second Rome. It didn’t even especially advantage America. Edison personally got rich, the overall balance of power didn’t change, and today all developed countries have electricity.

Who “won” the automobile race? Karl Benz? Henry Ford? There were many steps between the first halting prototype and widespread adoption. Benz and Ford both personally got rich, their companies remain influential today, and Mannheim and Detroit remain important auto manufacturing hubs. But other companies like Toyota and Tesla are equally important, the overall balance of power didn’t change, and today all developed countries have automobiles.

Who “won” the computer “race”? Charles Babbage? Alan Turing? John von Neumann? Steve Jobs? Bill Gates? Again, it was a long path of incremental improvements. Jobs and Gates got rich, and their hometowns are big tech hubs, but other people have gotten even richer, and the world chip manufacturing center is in Taiwan now for some reason. The overall balance of power didn’t change (except maybe during a brief window when the Bombes broke Enigma) and today all developed countries have computers.

The most consequential “races” have been for specific military technologies during wars; most famously, the US won the “race” for nuclear weapons. America’s enemies got nukes soon afterwards, but the brief moment of dominance was enough to win World War II. Maybe in some sense the British won a “race” for radar, although it wasn’t a “race” in the sense that the Axis knew about it and was competing to get it first. Maybe in some sense countries “race” to get better fighter jets, tanks, satellites, etc than their rivals. But ordinary mortals don’t concern themselves with such things. No part of US automobile policy is based on “winning the car race” against China, in some sense where consumer car R&D will affect tanks and our military risks being left behind.

I think some people hear transhumanists talk about an “AI race” and mindlessly repeat it, without asking what assumptions it commits them to. Transhumanists talk about winning an AI “race” for two reasons:

First, because if you believe unaligned AI could destroy humanity at some point, it’s important to align AI before it gets to that point. Companies that care about alignment might race to reach that point before companies that don’t care about alignment. Right now this is all academic, because nobody knows how to align AIs. But if someone figured that out, we would want those people to win a race.¹

Second, because some transhumanists think AI could cause a technological singularity that speedruns the next several millennia worth of advances in a few years.

In a more gradual technological singularity (sometimes called a “slow takeoff”) there’s some incentive to race. Paul Christiano defined a slow takeoff as one where AI accelerates growth so fast that GDP doubles every four years (so 20% year-on-year growth). This is faster than any country has achieved in real life, fast enough that wealth would increase 100x in the course of a generation. China is currently about 2 years behind the US in AI. If they’re still two years behind when a slow takeoff happens, the US would get a ~40% GDP advantage. That’s not enough to automatically win any conflict (Russia has a 10x GDP advantage over Ukraine; India has a 10x GDP advantage over Pakistan). It’s a big deal, but it probably still results in a multipolar world. Slow-takeoff worlds have races, but not crucial ones.

So one case in which losing an AI race is fatal is in what transhumanists call a “fast takeoff”, where AI speedruns millennia worth of usual tech progress in months, weeks, or even days. This probably only happens if superintelligent AI can figure out ways to improve its own intelligence in a critical feedback loop. I’m pretty skeptical of these scenarios in the current AI paradigm where compute is often the limiting resource, but other people disagree. In a fast takeoff, it could be that you go to sleep with China six months ahead of the US, and wake up the next morning with China having fusion, nanotech, and starships,.

We remember the race for nuclear weapons because they’re a binary technology - either you have them, or you don’t. When the US invented stealth bombers, its enemies had slightly worse planes that were slightly less stealthy. But when the US invented nukes, its enemies were stuck with normal bombs; there is no slightly-worse-nuke that can only destroy half a city. Everywhere outside the most extreme transhumanist scenarios, AI is more like the stealth bomber. You may have GPT-3, GPT-4, some future GPT-5, but a two year gap means you have slightly worse AIs, not that you have no AI at all. The only case where there’s a single critical point - where you either have the transformative AI or nothing - is in the hard-takeoff scenario where at a certain threshold AI recursively self-improves to infinity. If someone reaches this threshold before you do, then you’ve lost a race!²

Everyone I know who believes in fast takeoffs is a doomer. There’s no way you go to sleep with a normal only-slightly-above-human-level AI, wake up with the AI having godlike powers, and the AI is still doing what you want. You have no chance to debug the AI at level N and get it ready for level N+1. You skip straight from level N to level N + 1,000,000. The AI is radically rewriting its code many times in a single night. You are pretty doomed.

If you don’t believe in crazy science fiction scenarios like these, fine. But then why are you so sure that it’s crucial to “win” the AI “race”? If you’re sure these kinds of thing won’t happen, then you should treat AI like electricity, automobiles, or stealth bombers. It might tip the balance of a badly timed war, but otherwise you can just steal the tech and catch up.

I’m harping on this point because people think “we need to win the race with China” is an argument against worrying about alignment at all. I’m on board with claims like “as we worry about alignment, one thing we should consider is whether we’re losing a race against China”. But if you’re using the idea of a race to argue against alignment worries, I think you’re confused. In most scenarios, you don’t care that much about the race. And in the scenario where you do, you really want to worry about alignment.