AI safety advocates have plenty of hypothetical disaster scenarios they can evoke in order to win support for their cause. But none of those have the raw emotional appeal of killer robots.
For decades, we as a culture have been steeped in sci-fi depictions of intelligent machines going rogue and waging war against their human creators. Films like The Terminator and The Matrix come to mind, for example. A 2017 episode of the British TV series Black Mirror called “Metalhead” likewise envisions a nightmarish future in which humans are systematically hunted down and killed by autonomous four-legged robots (that look a lot like the famous “Spot” robot built by Boston Dynamics). The events in Frank Herbert’s Dune take place in the aftermath of a cataclysmic war between humanity and “thinking machines.” I could go on.
The point is that when the Secretary General of the United Nations calls upon the international community to ban “killer robots,” that phrase is going to strike a resounding chord in the public imagination—especially when that public is already becoming wary of the current course of AI development for much more immediately tangible reasons, like increasingly steep electricity bills and the unconstrained spread of online deepfakes.
On Monday, during the first-ever Global Dialogue on AI Governance in Geneva, U.N. Secretary General António Guterres called the development of AI-powered weapons “morally repugnant,” and urged the U.N.’s 193 member states to unilaterally prohibit the technology from the battlefield.
“Some decisions must remain forever human,” Guterres said in his speech, “none more than taking a human life.”
His words echoed a segment from Pope Leo XIV’s recent encyclical, Magnifica Humanitas, in which the pontiff wrote that “the development and use of AI in warfare must be subject to the most rigorous ethical constraints, to guarantee respect for human dignity and the sanctity of life and to avoid a race to develop such arms.”
Silicon Valley divided
Military use of new technology has long been the cause of a moral divide in Silicon Valley. In 2018, thousands of Google employees signed a letter addressed to CEO Sundar Pichai protesting against the company’s partnership developing technology for the Pentagon known as Project Maven. The following year, Microsoft faced a similar internal backlash over its agreement to provide HoloLens augmented reality headsets to the US Army for combat training purposes.
But AI, with its ability to identify subtle patterns from vast swathes of data and make decisions independently of any human oversight, has raised urgent new questions about the tech industry’s relationship with the military. Most famously, Anthropic became embroiled in a dispute with the U.S. Department of War earlier this year over the question of whether the company’s AI systems could be used for domestic surveillance and autonomous weapons systems. The company held firm in its stance against both use cases, leading Secretary of War Pete Hegseth to designate it a “supply chain risk,” meaning its technology was officially deemed to be a threat to national security. (It was the first time that label has been applied to an American company, and Anthropic is currently fighting in court to have it removed.)
Proponents of the use of AI-equipped weapons argue that it can aid in human decision-making in the heat of battle, and reduce the likelihood that innocent civilians will be targeted as hostile combatants. Some even invoke the moral rhetoric of the broader AI race, claiming the US has an ethical obligation to maintain a technological edge in the sophistication of its autonomous weapons program over adversaries like China, Russia, and Iran.
Trump muddies the AI weapon waters
In a national security memo published last month, President Trump accused previous administrations of implementing “undue bureaucracy” around the development of AI, including autonomous weapons. Ironically, since that memo was published, both OpenAI and Anthropic—the two leading U.S. AI developers—have become ensnared in a cobweb of federal bureaucracy which is forcing them to roll out their most powerful models on the Trump administration’s terms. “Meanwhile,” Trump went on to write in the memo, “our competitors continued to develop and deploy their own AI and sophisticated autonomous technologies for military and intelligence purposes, employing them with little regard for appropriate human oversight or civil liberties.”
Trump added in the memo that he vowed to “responsibly accelerate the use of AI across intelligence and warfighting domains in line with American values.”
U.N. Secretary General Guterres, Pope Leo, and other safety advocates, on the other hand, fear the arms technological arms race dynamic that’s already fueling AI developments in the US and China could soon produce autonomous weapons with decision-making processes that are completely opaque to the humans who built them, and which could eventually escape human control entirely.
Such a scenario could be far bleaker than even the darkest work of dystopian fiction can imagine.
Read the full article here
