Chapter 1: The Ethical Dilemma of Algorithms

Algorithms are increasingly tasked with making ethical choices. A prominent illustration of this is a modern interpretation of the trolley problem: if a self-driving vehicle faces a scenario where it must choose between the lives of two pedestrians, how should its programming determine which one to save?

In practice, this scenario does not accurately reflect the operations of self-driving cars. However, many current and upcoming technologies will need to navigate complex ethical dilemmas. For instance, risk assessment tools utilized in the criminal justice system must balance societal safety against the rights of individual defendants, and autonomous military systems need to evaluate the lives of combatants versus non-combatants.

The challenge is that algorithms have not been created to manage such difficult ethical choices. They are designed to focus on a singular mathematical objective, such as maximizing the number of lives saved or minimizing casualties. When faced with competing objectives or trying to incorporate abstract values like "freedom" and "well-being," an adequate mathematical solution may not always be achievable.

As Peter Eckersley, research director at the Partnership on AI, notes, “We as humans want multiple incompatible things.” He recently published a paper addressing this concern, emphasizing that in high-stakes scenarios, it's often inappropriate — and potentially perilous — to enforce a singular objective function that attempts to encapsulate ethical considerations.

These dilemmas are not exclusive to algorithms; ethicists have examined them for decades, referring to them as impossibility theorems. When Eckersley recognized their relevance to artificial intelligence, he proposed a solution borrowed from ethical theory: introducing uncertainty into algorithmic decision-making.

“We often make decisions in uncertain ways,” Eckersley explains. “Our moral actions are inherently uncertain. Yet, when we apply ethical behavior to AI, it tends to be overly simplified.” Instead, he suggests that we should deliberately program our algorithms to acknowledge uncertainty in decision-making.

Eckersley outlines two mathematical approaches to embody this concept. Traditional algorithms are typically programmed with explicit rules reflecting human preferences. For instance, one might need to program the algorithm to prioritize friendly soldiers over friendly civilians and friendly civilians over enemy soldiers — even if that certainty is not always warranted.

The first method, known as partial ordering, introduces a small degree of uncertainty. This approach allows the algorithm to prefer friendly soldiers over enemy soldiers and friendly civilians over enemy soldiers, but does not dictate a clear preference between friendly soldiers and friendly civilians.

The second method, termed uncertain ordering, involves multiple lists of absolute preferences, each with an attached probability. For example, an algorithm might favor friendly soldiers over friendly civilians three-quarters of the time, but in the remaining quarter, it might prioritize friendly civilians over friendly soldiers.

This uncertainty allows the algorithm to generate multiple potential solutions and present a range of options to humans, along with their associated trade-offs. For example, if the AI system assists in medical decisions, it could offer three treatment paths: one aimed at maximizing patient longevity, another focused on minimizing suffering, and a third that prioritizes cost-effectiveness. "The system should openly acknowledge its uncertainty," Eckersley suggests, "and return the ethical decision-making to humans."

Carla Gomes, a computer science professor at Cornell University, has explored similar ideas in her research. In one project, she is developing a system to assess the ecological impacts of new hydroelectric dam projects in the Amazon River basin. While these dams provide clean energy, they also significantly disrupt local ecosystems.

“This scenario differs from the typical ethical dilemmas associated with autonomous vehicles, but it presents genuine challenges,” she notes. “There are conflicting objectives — what’s the best course of action?”

“The overall complexity is substantial,” she continues. “Addressing all these issues will require extensive research, but Peter's approach is a vital step forward.”

As our dependence on algorithmic systems increases, these challenges will only intensify. “More intricate systems will increasingly require AI oversight,” states Roman V. Yampolskiy, an associate professor of computer science at the University of Louisville. “No single individual can comprehend the intricacies of something like the entire stock market or military response protocols. Thus, we will have to relinquish some control to machines.”

Karen Hao serves as the artificial intelligence reporter for MIT Technology Review, where she focuses on the ethical and social implications of technology, as well as its beneficial applications. She also authors the A.I. newsletter, the Algorithm.

An earlier version of this article was published in the Algorithm. To receive it directly in your inbox, subscribe here for free.

Chapter 2: Practical Applications of Ethical Algorithms

This video discusses a human-centered review of algorithms in higher education, highlighting the importance of ethical considerations in decision-making processes.

In this video, experts discuss AI ethics, featuring insights from prominent thinkers like Bostrom and Yudkowsky on the moral implications of artificial intelligence.