The Runaway Tram That Was Never Meant to Be Polled

The trolley problem is usually told as a puzzle about killing, but it began as a side argument in a debate about abortion. In 1967 the British philosopher Philippa Foot published an essay on the doctrine of double effect, the old idea that harm foreseen as a side effect can be permissible when the same harm intended as a means is not. To test the doctrine she offered pairs of cases, one of them about the driver of a runaway tram who can steer from a track with five workers onto a track with one. The version in which a bystander pushes a heavy man from a footbridge came later in the work of Judith Jarvis Thomson, who is credited with the name trolley problem in a paper from 1976.

Decades later the puzzle left the seminar room. In 2018 a team at the Massachusetts Institute of Technology reported in Nature the results of the Moral Machine, an online game in which visitors judged how a driverless car should behave in unavoidable crashes. The experiment is said to have gathered roughly forty million decisions from people in more than two hundred countries and territories. Respondents on the whole preferred saving more lives to fewer, humans to animals and the young to the old, yet the weight given to each preference varied by region, in clusters that the authors linked loosely to cultural and economic differences.

Engineers and policymakers drew a quick lesson, that machines should be taught the average of human intuitions. Foot's own purpose points the other way. She did not mean the cases to measure what people would do. She meant them to expose the principles that might justify what people do, and principles are what a society must defend when a crash is reviewed in court. A majority poll reveals how people feel about the five and the one. It cannot tell whether the feeling is a moral insight or a bias.

It cannot tell whether the feeling is a moral insight or a bias.

A defender of the poll would reply that law and morality in practice depend on shared intuitions, and that a car which offends them will be neither trusted nor bought. The reply has force, since a principle that no population accepts is hard to enforce. Yet the Moral Machine data show that the population itself does not speak with one voice, so the poll cannot say whose intuitions the car should obey. Nor do respondents bear the cost of the choice, since a game has no victims.

The deeper difficulty is that the verdict a person gives may change with a detail that seems morally irrelevant, such as whether harm is done by a push or by a switch, as studies of the footbridge variant have suggested. If intuition shifts with such details, then treating it as evidence and treating it as noise lead to opposite rules for the machine. A puzzle designed to isolate a principle has become a survey of preferences, and nobody has yet shown that the principle survived the journey.