The Hundredth EyeFR
AI & Jobs · Essay 4 / 5

The Race Nobody Can Stop

Nobody can slow the AI race. It is putting capabilities that once took the resources of a state within reach of small groups, and concentrating power in a few companies.

November 22, 20266 min readLire en français

With no brakes, the AI race brings capabilities once reserved for states within reach of small groups.

In brief
  • Washington wants to “win the AI race,” Beijing is aiming for over 90% adoption of AI agents by 2030, and nobody will slow down alone.
  • In Hong Kong, a fake video call was enough to steal 25 million dollars, and an AI ran 80 to 90% of a spying campaign.
  • Power, meanwhile, is concentrating: three groups supply two thirds of the cloud that AI runs on.
  • A promise nobody can check binds only those who believe it, and I doubt AI gets reined in before the first serious accident.

It all starts with a video call in Hong Kong, in 2024.

Someone on the finance team of a multinational gets a message that appears to come from the group’s chief financial officer in the United Kingdom, then is invited to a group video call about confidential transactions. Asked to make transfers, they authorize them: about 200 million Hong Kong dollars, or 25 million US dollars. Police say the meeting had been pre-recorded from public videos and voice recordings of the CFO. Apart from the victim, nobody on that call was real.1 If fraudsters could pull this off in 2024, what will the states leading the AI race be able to do?

Ten years ago, I think a setup like that would have taken the resources of an intelligence agency. This time, a few videos found online were the raw material. Those videos are data, and data is for sale: stolen personal data is bought and resold on criminal markets online.2 Nobody can stop the AI race, and it is pulling in two directions. What once took the resources of a state is coming within reach of small groups with bad intentions. In the other direction, the power to build the most advanced AI, and the machines that run it, is concentrating in a few companies.

Nobody can afford to stop

I don’t believe the United States or China will slow down. Washington says so in black and white: its AI plan, published in July 2025, says the country must “win the AI race.”3 A month later, Beijing set a 2030 target of over 90% adoption for new-generation smart devices and AI agents, programs that can carry out tasks on their own.4 We’re back in the prisoner’s dilemma, this time between states: whoever slows down alone falls behind. What’s at stake is each side’s economic and military lead for decades to come.

Each watches the other, nobody touches the brake, and the track ends somewhere in the mist.

Behind the scenes, the race is also fought without weapons, through dependencies. The vast majority of the world’s most advanced chips are made in Taiwan, the machine needed to etch them has a single maker, the Dutch company ASML, and more than 90% of gallium and rare earths are refined in China.5 Each side holds a piece of the puzzle the others need. In 2025, when the Netherlands put Nexperia, a chipmaker owned by a Chinese group, under state supervision, China blocked part of the company’s exports, and European carmakers feared their assembly lines would grind to a halt. Within seven weeks, both sides had backed down.6 These dependencies push everyone to the table when tensions rise. They don’t slow the race.

The means of a state, within reach of small groups

The first of these means is knowing everything about someone. States have always wanted that, and it took them agents and years. Researchers have shown that AI models can work out, from ordinary posts on a forum, where the author lives, how much they earn or how old they are, getting up to 85% right.7 That raw material is for sale: according to the US consumer protection agency, a single data broker collected more than 500 million phone identifiers tied to precise locations.8

An AI that knows everything about you also knows how to talk to you. In an experiment published in 2025, an AI model given a few personal details about the person it was debating did better than humans at changing that person’s mind.9 In the wrong hands, an AI that knows you can do real damage: impersonate you, as in Hong Kong, or push you somewhere you never meant to go.

The second of these means is acting at scale with very few people. In September 2025, according to Anthropic, a state-backed hacking group hijacked its Claude Code tool. The AI did 80 to 90% of the work in a spying campaign against about thirty targets; humans picked the targets and made the key decisions.10 That group still had a state behind it. But when the machine does most of the work, you need fewer and fewer people. Nobody has shown that a lone individual has reached the power of a state. Still, I think the bar keeps dropping, year after year.

The third, and the most serious, is doing harm in the physical world. AI labs take the biological risk seriously. In 2025, Anthropic and then OpenAI switched on stronger safeguards for their models, as a precaution, without proof that the models could really help build a biological weapon.11 The International AI Safety Report, published in February 2026, finds that these capabilities are improving, but that putting them to use in the real world is still, fortunately, hard.12 But for how long?

On the battlefield, the bar is dropping too. The American B-2 stealth bomber program cost a little over two billion dollars per aircraft, development included.13 A Shahed-type attack drone, built for far more modest missions, costs about 35,000 dollars,14 and some small combat drones about 400.15 AI is starting to guide them: according to a researcher at CSIS, an American think tank, some drones lock onto a target a human has approved using the camera image, then finish the attack on their own, even when jamming cuts the link to their operator.15

The movement also runs the other way. While some capabilities trickle down to small groups, power is concentrating in a few companies. In 2025, more than 90% of the AI models Stanford University rated as notable came from companies, not from universities or states.16 In 2024, three groups, Amazon, Microsoft and Google, supplied about two thirds of the world’s cloud services, the remote computing these models run on.16 I think their rules now count as much as the law in deciding what an AI will or won’t do.

The races we managed to slow

Still, history shows that other technology races between states were eventually reined in. After the Cuban missile crisis in 1962, Washington and Moscow set up a direct hotline, then banned nuclear tests in the atmosphere, in space and underwater. Not underground: an underground test could pass for an earthquake, and the two sides couldn’t agree on inspections on each other’s soil. The treaty came only after the world had stood at the brink of nuclear war, and it covered only what each rival could watch the other doing.17

Things turn out worse when nothing can be checked. In 1972, an international convention banned biological weapons, with no verification mechanism at all.18 The Soviet Union signed it. Seven years later, an anthrax outbreak hit the Soviet city of Sverdlovsk: according to a scientific investigation published in 1994, it came from a leak at a military microbiology site.19 A promise nobody can check binds only those who believe it.

Why AI is harder to slow

AI has more in common with biological weapons than with nuclear tests. There is something to count: scarce chips that come out of a handful of factories, and huge data centers. But counting chips doesn’t tell you what they compute. The same AI model can help find a new drug or plan a cyberattack. And researchers keep finding ways to do as much with fewer chips, so the count tells you less and less.20 We know where the machines are. We know far less about what they do.

Today’s brakes reflect that. There are laws, such as the European Union’s AI Act, in force since 2024, but none applies to the whole world.21 At that scale, there are only promises. In 2024, sixteen AI companies voluntarily committed to publishing their safety rules and spelling out the risks they would consider intolerable.22 At the UN, more than 160 countries voted for resolutions on autonomous weapons, but no binding text exists yet.23 Putting the brakes on AI as a whole seems impossible to me. Restricting some uses, perhaps, if the rivals find a way to check on each other. I doubt they will manage it before the first serious accident.

Where does a race lead when nobody knows how to slow it down? Not necessarily to the worst: history has seen falls, and then recoveries. The last piece in this series sets progress against the long sweep of human history: a rising wave, with its peaks and troughs. Where do we stand, and who would the next trough hit, if it comes?

Sources

  1. Hong Kong government, reply to the Legislative Council on deepfake fraud, June 26, 2024 (chief financial officer impersonated; meeting pre-recorded from public videos and the CFO’s voice; loss of about HK$200 million) — https://www.info.gov.hk/gia/general/202406/26/P2024062600192p.htm
  2. Europol, “Steal, deal and repeat: How cybercriminals trade and exploit your data”, IOCTA 2025, 11 Jun 2025 (data has become “a key commodity”) — https://www.europol.europa.eu/publication-events/main-reports/steal-deal-and-repeat-how-cybercriminals-trade-and-exploit-your-data
  3. White House, “White House Unveils America’s AI Action Plan,” July 23, 2025 — https://www.whitehouse.gov/releases/2025/07/white-house-unveils-americas-ai-action-plan/
  4. State Council of the People’s Republic of China, “AI+” plan, Aug. 27, 2025 (2030 target for new-generation intelligent terminals and AI agents) — https://english.www.gov.cn/policies/latestreleases/202508/27/content_WS68ae7976c6d0868f4e8f51a0.html
  5. Stanford Emerging Technology Review 2026, “Semiconductors,” Jan. 2026 (TSMC: about 90% of advanced-chip manufacturing in 2024) — https://setr.stanford.edu/sites/default/files/2026-01/SETR2026_09-Semi_web-260109.pdf · ASML, 2025 annual report (only maker of EUV lithography systems; company statement) — https://ourbrand.asml.com/m/6ea363f69344ebd4/original/asml-2025-annual-report-based-on-ifrs.pdf · IEA, “Global Critical Minerals Outlook 2026,” July 16, 2026 (China: over 90% of gallium and rare-earth refining) — https://www.iea.org/reports/global-critical-minerals-outlook-2026/executive-summary
  6. Dutch government, letter to Parliament of Nov. 19, 2025 — https://web.archive.org/web/20260324044459/https://www.government.nl/binaries/government/documenten/parliamentary-documents/2025/11/19/letter-to-the-parliament-on-the-latest-developments-concerning-nexperia/102176837%2BDevelopments%2Bconcerning%2BNexperia.pdf · ACEA, “Critical chip shortage worsens by the day,” Oct. 2025 (warnings, no confirmed stoppage) — https://www.acea.auto/press-release/critical-chip-shortage-worsens-by-the-day/
  7. Staab et al., “Beyond Memorization: Violating Privacy via Inference with Large Language Models,” ICLR 2024 (Reddit profiles; up to 85% top-1 accuracy, on that dataset) — https://openreview.net/pdf?id=kmn0BhQk7p
  8. FTC, action against Mobilewalla, Dec. 3, 2024 (regulator’s allegations; over 500 million advertising identifiers, 2018–2020) — https://web.archive.org/web/20261030172845/https://www.ftc.gov/news-events/news/press-releases/2024/12/ftc-takes-action-against-mobilewalla-collecting-selling-sensitive-location-data
  9. Salvi et al., Nature Human Behaviour, May 19, 2025 (controlled debates; GPT-4 with personal information: markedly higher odds of shifting opinion than humans) — https://doi.org/10.1038/s41562-025-02194-6
  10. Anthropic, “Disrupting the first reported AI-orchestrated cyber espionage campaign,” Nov. 13, 2025 (company estimates; humans chose the targets) — https://assets.anthropic.com/m/ec212e6566a0d47/original/Disrupting-the-first-reported-AI-orchestrated-cyber-espionage-campaign.pdf
  11. Anthropic, “Activating AI Safety Level 3 protections,” May 22, 2025 (precautionary) — https://www.anthropic.com/news/activating-asl3-protections · OpenAI, ChatGPT agent system card, July 17, 2025 (treated as “High” biological capability as a precaution) — https://openai.com/index/chatgpt-agent-system-card/
  12. International AI Safety Report 2026, Feb. 2026 — https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026 · RAND, 2024 (no significant difference in the tested setting) — https://www.rand.org/content/dam/rand/pubs/research_reports/RRA2900/RRA2977-2/RAND_RRA2977-2.pdf
  13. GAO, report NSIAD-97-181, 1997 (average B-2 program cost per aircraft, development included, December 1996 estimate for 21 aircraft) — https://www.gao.gov/assets/nsiad-97-181.pdf
  14. CSIS, “Calculating the Cost-Effectiveness of Russia’s Drone Strikes,” Feb. 2025 (estimate, depends on configuration) — https://www.csis.org/analysis/calculating-cost-effectiveness-russias-drone-strikes
  15. ETH Zurich, Center for Security Studies, Ukrainian Analytical Digest No. 11 (some FPV drones around 400 dollars) — https://css.ethz.ch/content/dam/ethz/special-interest/gess/cis/center-for-securities-studies/pdfs/ukrainiananalyticaldigest-011.pdf · CSIS, “The Russia-Ukraine Drone War: Innovation on the Frontlines and Beyond,” May 28, 2025 (a researcher’s assessment; AI-assisted visual lock after an operator approves the target; prevalence not quantified) — https://www.csis.org/analysis/russia-ukraine-drone-war-innovation-frontlines-and-beyond
  16. Stanford HAI, AI Index Report 2026 (“over 90%” of notable frontier models from industry in 2025, Stanford’s selection) — https://hai.stanford.edu/assets/files/ai_index_report_2026.pdf · International AI Safety Report 2025 (AWS, Microsoft Azure and Google: about 67% of global cloud services in 2024) — https://internationalaisafetyreport.org/publication/international-ai-safety-report-2025
  17. US State Department: hotline agreement, June 20, 1963 — https://1997-2001.state.gov/global/arms/treaties/hotline1.html · Office of the Historian, “The Limited Test Ban Treaty, 1963” (earthquakes, inspection dispute) — https://history.state.gov/milestones/1961-1968/limited-ban
  18. UNODA, Biological Weapons Convention (no verification regime) — https://geneva-s3.unoda.org/static-unoda-site/pages/templates/the-biological-weapons-convention/topics/BWS%2Bbrochure.pdf
  19. Meselson et al., “The Sverdlovsk anthrax outbreak of 1979,” Science, Nov. 18, 1994 — https://doi.org/10.1126/science.7973702
  20. CNAS, “Artificial Intelligence and Arms Control” (civilian and military uses) — https://www.cnas.org/publications/reports/artificial-intelligence-and-arms-control · Sastry, Heim et al., “Computing Power and the Governance of Artificial Intelligence,” 2024 — https://arxiv.org/abs/2402.08797
  21. European Union, Regulation (EU) 2024/1689 on artificial intelligence (AI Act), in force since Aug. 1, 2024, obligations applied in stages — https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:32024R1689
  22. UK government, Seoul commitments, May 21, 2024 (16 companies; voluntary commitment) — https://www.gov.uk/government/news/historic-first-as-companies-spanning-north-america-asia-europe-and-middle-east-agree-safety-commitments-on-development-of-ai
  23. UN Digital Library, resolutions on lethal autonomous weapons systems: 2024 (166 for, 3 against, 15 abstentions) — https://digitallibrary.un.org/record/4068497?ln=en · 2025 (164 for, 6 against, 7 abstentions) — https://digitallibrary.un.org/record/4094456?ln=en
Next essay · 05 / 05

Where Are We on the Wave of Progress?

Progress rises in waves, and we are not all at the same point on the wave.

Read on →