The Centaur's Dilemma

James Baker began drafting The Centaur’s Dilemma in the late 2010s, after stepping down from the United States Court of Appeals for the Armed Forces and returning to academic life at Syracuse, where he runs the Institute for Security Policy and Law. The book arrived in 2021, into a moment when American national security debate about artificial intelligence had moved past the question of whether the technology mattered and had begun, slowly, to take up the harder question of what to do about it. The Defense Department had stood up the Joint Artificial Intelligence Center. The National Security Commission on Artificial Intelligence, chaired by Eric Schmidt and Robert Work, had issued its sprawling final report. Project Maven, the Pentagon’s computer-vision programme for analysing drone imagery, had become a public controversy after Google employees objected to their company’s involvement. Baker himself had served on a State Department advisory board on international security and arms control, and his book reads as the product of someone who has spent decades watching how legal and policy machinery in Washington actually moves — or fails to move — when a new technology lands on the executive branch’s desk.

The book’s title comes from the chess world. After Garry Kasparov lost to Deep Blue in 1997, he popularised the format of “centaur chess” or “advanced chess”, in which a human player works with a chess engine. For a while, centaurs beat both unaided humans and unaided engines. Baker takes that image and stretches it into a frame for the national security state. The centaur is a metaphor for human-machine teaming in war, surveillance, and decision-making: a force whose effectiveness depends on getting the seam between human judgment and machine output exactly right. The dilemma is that the seam is the hardest part to design, the easiest part to neglect, and the place where law, ethics, and operational pressure all collide. Push too much weight onto the machine and the human becomes a rubber stamp on outputs they do not understand. Push too much onto the human and the machine’s speed advantage — often the entire reason for fielding it — disappears. The book’s central argument is that American national security law is nowhere near ready for that seam, and that getting it ready is a project that has to happen now, before the most consequential decisions about autonomy, accountability, and command are locked in by procurement choices and tactical doctrine.

Baker writes for a specific kind of reader — the national security lawyer, the policy adviser, the executive-branch official, the congressional staffer — and the book’s structure reflects that. An opening section explains what artificial intelligence is in terms a lawyer can use without pretending to be a computer scientist: supervised and unsupervised learning, neural networks, the distinction between narrow and general AI, the difference between machine learning and the older expert-systems tradition. He is careful with terminology, because terminology in law has consequences, and he wants his readers to be able to spot when a term of art is being smuggled across from engineering into policy without enough scrutiny. From there the book widens into the national security setting: the geopolitical race with China, the Russian doctrinal interest in autonomous systems, the rise of the Chinese surveillance state and its export of facial-recognition platforms to authoritarian governments. Baker spends time on the People’s Liberation Army’s concept of “intelligentized warfare”, on Xi Jinping’s military-civil fusion strategy, and on the Made in China 2025 industrial plan, not because the book is about China but because the urgency of his legal argument depends on the reader accepting that the technology is going to be fielded one way or another, and that American restraint without a paired legal framework simply hands the field to states with fewer scruples.

The middle of the book is a sustained tour through the specific bodies of American law that AI in national security will touch. Constitutional questions come first: the Article II authority of the president as commander in chief, the Article I authority of Congress to declare war and to regulate the armed forces, and the Fourth Amendment law of search and seizure as it applies to algorithmic surveillance. Baker walks through the war powers debate as it relates to autonomous and semi-autonomous weapons — whether the use of an autonomous system in a hostility crosses the threshold for War Powers Resolution reporting, and what “introduction of armed forces” even means when the armed force in question is a software process running on a drone. He examines the existing Department of Defense policy on autonomy in weapons systems, the 2012 directive that requires “appropriate levels of human judgment” in the use of force, and the 2017 update that re-issued the same language with extended applicability. He is sceptical of the directive’s elasticity and warns that policy documents that everyone can read in their own favoured way are precisely the documents that fail at the moment of stress.

A long section addresses surveillance, where the legal terrain is even more developed and the AI question even more pressing. Baker takes the reader through the Foreign Intelligence Surveillance Act, the bulk-collection programmes disclosed by Edward Snowden in 2013, and the subsequent reforms in the USA Freedom Act of 2015. He frames AI as a force multiplier on every authority the intelligence community already has: machine learning over signals intercepts, image-recognition systems run against satellite imagery, link-analysis algorithms run against telephony metadata, predictive analytics applied to financial transactions. The legal question, in his telling, is not whether the underlying collection is lawful — Congress and the courts have answered that, however controversially — but whether the inferences drawn by machines from lawfully collected data are themselves a kind of search, and whether the long-standing doctrine that there is no Fourth Amendment interest in information voluntarily exposed to third parties survives a world in which inference is cheap and total. The Carpenter decision of 2018, in which the Supreme Court limited that third-party doctrine for cell-site location information, sits at the centre of his discussion as a sign that the doctrinal foundations are quietly shifting beneath the intelligence community’s feet.

Cyber operations get their own treatment, with Baker drawing on his experience watching the slow legalisation of offensive cyber capabilities through the 2010s — the move from covert action authorities to Title 10 authorities, the standing up of United States Cyber Command, the policy fight over National Security Presidential Memorandum 13 in 2018 which loosened the interagency process for approving cyber operations. He treats machine-speed cyber defence as one of the genuinely hard cases for the centaur frame: a network intrusion may need to be repelled in milliseconds, far faster than a human can authorise a response, and the law of armed conflict has nothing useful to say about the timing of the human in the loop when the loop is measured in microseconds. Adjacent to this he discusses information operations, deepfakes, and the use of generative models for influence campaigns, drawing on the Internet Research Agency’s activity in 2016 and the subsequent recognition that the next generation of such campaigns would be powered by tools that did not yet exist when the 2016 playbook was written.

The book devotes substantial space to autonomous weapons in a strict sense. Baker examines the international debate at the United Nations Convention on Certain Conventional Weapons, the long-running working group on lethal autonomous weapons systems, and the political coalitions that have grown up around a possible preemptive ban — the Campaign to Stop Killer Robots and its non-governmental partners on one side, and on the other a group of major military powers, including the United States, Russia, and the United Kingdom, who have resisted a binding treaty. He is plainly uncomfortable with the idea that a ban will work, and equally uncomfortable with the idea that informal political pressure alone will be enough. He proposes instead a layered approach: clearer domestic doctrine on what “meaningful human control” actually means in procurement and operational terms, a serious investment in test and evaluation regimes that can verify the behaviour of learning systems before they are fielded, and a diplomatic posture that pursues confidence-building measures and verification protocols rather than a single grand treaty that no major military will sign.

Throughout, Baker returns to a theme that is more about institutions than about technology. Good national security decisions, in his telling, are products of good process — interagency review, legal review, presidential decision memoranda that genuinely surface dissent, and an executive branch culture that takes seriously the value of saying no. AI will stress every part of that process. It will stress it on speed, by collapsing decision timelines from hours to seconds. It will stress it on transparency, because the models themselves are not interpretable and the engineers who build them often cannot answer the question a lawyer needs answered. It will stress it on accountability, because the chain of authorship for an autonomous action runs through procurement officers, vendors, data scientists, commanders, and the system itself, none of whom is straightforwardly the responsible party in the way an officer pulling a trigger is. Baker’s recommendations therefore lean towards process: a standing AI council inside the National Security Council, dedicated AI counsel inside each of the relevant departments and agencies, a meaningful role for the inspectors general, and a continuing education programme for judges and lawyers so that the courts and the executive branch lawyers who advise on AI can keep up with what is actually being deployed.

Reception in the field has been respectful and somewhat uneven. National security lawyers and academic readers — Ashley Deeks at Virginia, Rebecca Crootof at Richmond, Matthew Waxman at Columbia — treated the book as a serious entry in a thin literature, and it has been widely cited in subsequent law-review work on autonomous weapons and machine-learning surveillance. Technologists have sometimes found Baker’s account of the underlying technology cautious to the point of conservatism, particularly on the question of how quickly capabilities will mature; readers coming from the artificial-intelligence ethics tradition have wished for more on bias, fairness, and the civil-rights dimension of domestic AI use, areas Baker covers but does not centre. The defence community’s own response has been to converge, slowly, on something close to his policy line: the Department of Defense issued its AI Ethical Principles in 2020, and the 2023 update to its autonomy directive sharpened some of the language Baker had flagged as elastic. None of this should be confused with the book having won the argument outright. The deeper questions about constitutional authority, about war powers, about the Fourth Amendment in an age of inference, remain unresolved and are unlikely to be resolved soon.

For a reader assembling a working library on AI in war, The Centaur’s Dilemma pairs naturally with Paul Scharre’s Army of None on the operational and doctrinal side and with Kenneth Payne’s I, Warbot on the strategic side. It is less concerned with the technology than Scharre and less concerned with strategy than Payne; its centre of gravity is institutional, and its great strength is the depth of its engagement with how American legal machinery actually functions. The book does not cover commercial AI regulation in any depth, does not address China’s domestic AI law on its own terms, and gives only glancing attention to the European Union’s parallel regulatory project — readers wanting those should turn elsewhere. Its weakness, depending on the reader, is that the careful procedural sensibility that gives the book its authority also gives it a certain reluctance to imagine that the existing institutions might be the wrong instruments for the problem. Baker writes as a man who has spent his career inside those institutions and believes, on the whole, that they can be made to work.

What is likely to age well is the framing. The centaur metaphor remains useful, and the call to take the seam between human and machine seriously as a legal problem rather than a tactical one will be relevant for as long as AI is fielded in security settings, which is to say indefinitely. What is already dating is the specific catalogue of programmes and authorities. Project Maven has moved through several phases; the directive on autonomy has been updated; the National Security Commission on Artificial Intelligence has dissolved into its successor organisations; the constitutional cases Baker discusses have been overtaken by newer ones. A reader coming to the book today should treat the legal architecture as a snapshot from 2020, but the snapshot is a careful one, and the questions it raises have outlived the answers it offers.