Robot Ethics

Robot Ethics appeared in 2012 as the first serious multidisciplinary anthology to tackle its subject in English, published by MIT Press and co-edited by Patrick Lin and Keith Abney of California Polytechnic State University’s Ethics and Emerging Sciences Group together with George Bekey, the USC roboticist whose textbook had helped define autonomous robotics as an engineering discipline a decade earlier. The volume grew out of a 2008 report Lin, Abney and Bekey had delivered to the US Office of Naval Research titled Autonomous Military Robotics: Risk, Ethics, and Design, a document the Navy commissioned when it became clear that armed unmanned systems were moving from the laboratory to the inventory faster than the ethical and legal frameworks surrounding them. The book expanded that work outward from the military case into a wider survey of where robots were starting to touch human life — operating rooms, nursing homes, highways, factories, bedrooms — and gathered roughly forty contributors from philosophy, computer science, law, religious studies, cognitive science and engineering to examine the resulting problems.

The editors’ central argument is that the ethics of robotics is not a question that can be deferred until the machines are fielded. By the time a self-driving car is in a crash or a surgical robot slips, or an autonomous weapon fires on the wrong target, the design decisions that produced the outcome have already been made — often years earlier, by engineers who did not frame their choices as ethical ones. The book insists that moral reasoning has to migrate upstream into design and programming, that this is a tractable technical problem as well as a philosophical one, and that the field needs its own vocabulary rather than borrowing wholesale from bioethics or just-war theory. The editors do not pretend the discipline has settled answers; they argue that the questions are now urgent enough that the lack of answers is itself a problem.

The book is organised into eight parts after an opening overview. The first sets out definitions and taxonomies — what counts as a robot, where autonomy sits on a spectrum from teleoperation to full independence, how robots differ from software agents or ordinary machines. A design and programming section follows, including Wendell Wallach and Colin Allen’s framing of top-down, bottom-up and hybrid approaches to building moral reasoning into machines, drawn from their 2008 book Moral Machines. A section on rights and ethics asks whether robots can have moral status, what duties humans might owe them, and whether anthropomorphic design confers unearned trust. There is a part on religion, with chapters by scholars from Christian, Buddhist and Jewish traditions on how established religious frameworks respond to artificial agents.

The war section is the longest and the most consequential, drawing explicitly on the editors’ earlier Navy report. Ronald Arkin of Georgia Tech contributes a chapter laying out his proposed ethical governor, a software architecture that would prevent an autonomous weapon from firing when its own analysis determined that doing so would violate the laws of armed conflict — a position Arkin had developed in his 2009 book Governing Lethal Behavior in Autonomous Robots and defended on the ground that a properly engineered machine could in principle be more law-abiding than a stressed, frightened or angry human soldier. Peter Asaro and Noel Sharkey take the opposing view: that the discrimination required by international humanitarian law cannot be mechanised, that accountability dissolves when the decision to kill is delegated to a machine, and that permitting such weapons would lower the threshold for the use of force. Robert Sparrow’s contribution turns on moral responsibility in autonomous warfare — the argument that no human can fairly be held accountable for the actions of a sufficiently autonomous machine, and that this responsibility gap is itself a violation of the laws of war. The chapters together map the fault lines along which the subsequent decade of debate at the UN Convention on Certain Conventional Weapons would develop.

A law section examines liability — when a surgical robot errs or a self-driving car crashes, who pays — and the problems products-liability doctrine will face with learning systems whose behaviour cannot be audited back to a line of code. A service and civilian applications part looks at domestic robots: the Roomba, the Paro therapeutic seal used with dementia patients, prototype home-care robots then under development in Japan and South Korea in response to demographic pressure. Chapters on elder care raise concerns Sherry Turkle had been making prominent in the same period: that giving an isolated person a machine designed to simulate companionship may relieve a symptom while deepening the condition. A psychology and sex section places David Levy’s work from Love and Sex with Robots alongside more sceptical pieces from Blay Whitby and others. A medicine section covers surgical teleoperation on the da Vinci model, rehabilitation robotics, and the early literature on autonomous diagnostic systems. A final part on specialised applications takes in environmental monitoring, mining, space exploration and nuclear decommissioning — places where machines were already working in human-hostile conditions and raising distinct safety and oversight questions.

Alongside the arguments, the book is a catalogue of hardware and software that was shaping the discussion at the time. The MQ-1 Predator and MQ-9 Reaper are discussed as the pre-autonomous baseline — remotely piloted, with a human in the loop for every weapons release. The Samsung SGR-A1 sentry gun deployed by South Korea along the demilitarised zone, with its optional autonomous engagement mode, is examined as an early instance of a lethal system a government had already fielded. The Phalanx close-in weapon system aboard US warships is treated as a case study in bounded autonomy — a defensive machine permitted to engage incoming threats faster than a human could react, within a narrow envelope the operator approved in advance. The da Vinci Surgical System, Intuitive Surgical’s teleoperated platform, grounds the medical chapters. Honda’s ASIMO and various Japanese care-robot prototypes anchor the service discussion. The Roomba supplies the mundane limit case — a machine most readers had met, cheap enough to be unremarkable, which was already changing household behaviour in ways nobody had thought to study.

The contributors are drawn from a recognisable cross-section of the field as it stood in 2012. Arkin and Sharkey represent the engineering and the critical poles of the autonomous-weapons debate. Wallach and Allen bring the philosophy-of-mind strand. John Sullins contributes a chapter on robot friendship and companionship that would be cited widely in the subsequent care-robot literature. James Hughes writes on the transhumanist stakes. Marcello Guarini and Paul Bello address machine consciousness and moral status. Legal scholars including Peter Asaro — who holds degrees in both philosophy and informatics — treat liability and the law of armed conflict. The religious-studies chapters come from scholars working at the intersection of theology and technology. The result is a book deliberately broader than the editors’ own training would have produced on its own.

The reception since publication has treated Robot Ethics as a founding reference for the field it helped establish. The Campaign to Stop Killer Robots was launched the following year in London, drawing on Sharkey’s work and on the arguments made in the book’s war section; the Convention on Certain Conventional Weapons began its series of expert meetings on lethal autonomous weapons systems in 2014, and the terms of that debate — meaningful human control, the accountability gap, the discrimination problem — are in large part the terms the book put into circulation. Wallach went on to further develop the moral-machine framework in subsequent work. Paul Scharre’s Army of None, which appeared in 2018, is in many ways a direct successor to the war chapters here, written from inside the Department of Defense’s own policy process. Shannon Vallor, Mark Coeckelbergh and others expanded the service-robot and companionship arguments in later monographs. The anthology format has itself been imitated repeatedly; a second edition, Robot Ethics 2.0, was published in 2017 under Lin, Abney and Ryan Jenkins to catch up with developments in machine learning, autonomous vehicles and medical robotics that the first edition did not anticipate.

Critics have pushed back along two main lines. The first is that the book’s treatment of autonomous weapons is too hospitable to Arkin’s position — that giving a sympathetic hearing to the ethical governor proposal lends respectability to a technology its opponents consider categorically unacceptable, and that a volume co-edited by authors of a Navy-funded report cannot be a neutral forum on the question. The editors’ response, repeated in later writing, is that excluding the engineering case from the discussion would make the discussion useless to the engineers who are going to build the systems whether philosophers approve or not. The second line of criticism, from scholars in science and technology studies, is that the book is too focused on individual machines and their users and too thin on the political economy of the industries building them — a critique Kate Crawford, Virginia Eubanks and others would develop more fully later in the decade with respect to algorithmic systems generally.

For a reader working through the literature on artificial intelligence in warfare today, Robot Ethics occupies a specific and still-useful place. It is where the vocabulary came from. A reader who wants to understand why the debate at the UN uses the particular terms it uses, why military ethicists talk about meaningful human control rather than some other formulation, why the responsibility gap keeps being invoked, will find the sources of those phrases here. It pairs with Wallach and Allen’s Moral Machines on the philosophical side, with Scharre’s Army of None on the operational side, and with Peter Singer’s Wired for War, which had appeared three years earlier and set the journalistic scene. It is less useful as a guide to current systems; the drones it describes are a generation old, large-language-model-driven decision aids were not yet imagined, and the demonstration in Ukraine of what cheap autonomous munitions can do to a conventional army lay a decade in the future.

What has aged well in the book is the framework. The taxonomy of autonomy, the distinction between moral agency and moral patienthood, the responsibility gap, the discrimination problem, the elder-care simulation worry — these are still the categories the field uses. Arkin’s engineering case for the ethical governor still has adherents, Sharkey’s categorical opposition still has adherents, and the gap between them is still roughly where the book left it. What has dated is the technical surface: the particular robots named, the particular capabilities assumed, the near-term predictions. A reader coming to it now should expect to spend most of their time with chapters that ask whether a machine can be a moral agent, how liability should be assigned when it harms someone, and whether a soldier can be replaced by software — and to skim the chapters that ask whether a Roomba understands your house.