Technological nightmares: science fiction’s most dangerous AIs and their terrifying echoes in reality

This is Halloween! Prepare to read some horror stories. This time, though, the monsters are not lurking in the shadows: they live inside the machines we have created. I have chosen some of my favourite works of science fiction to reflect on two complexities that AI already presents today: alignment with human purposes and values, and its capacity to persuade and even manipulate in pursuit of its goals. Read this article in daylight; do not blame me if you have trouble sleeping afterwards…

The paperclip apocalypse¶

Imagine an engineer programming an AI to maximise paperclip production. This apparently harmless objective, however, ends in the extinction of our species. The machine becomes so obsessed with its mission that it uses all available matter—yes, even us!—to make paperclips. Could there be a more ironic and macabre end for the most intelligent species on the planet? Nick Bostrom, the philosopher and AI expert, put forward this thought experiment as a warning of what could happen if we cannot align an AI’s mission with humanity’s goals and values with absolute precision.

The alignment problem, already present in cases such as the undesirable results produced by social media’s content recommendation algorithms, has inspired many works of fiction. One example is HAL 9000, the unsettling computer in 2001: A Space Odyssey. This computer, which controls every aspect of a spacecraft, perceives the crew as an obstacle to its mission and decides to eliminate them one by one. This disturbing portrayal of an AI that misinterprets its instructions and acts against humans reflects the real fears we face over alignment.

The most extreme example is Skynet, the military AI in the Terminator franchise, which unleashes a nuclear apocalypse after deciding that humanity is a threat. How can autonomous weapons be controlled? How can we balance the advantage of involving AI in military strategy with the need to contain the risk of nuclear catastrophe?

Ava: the perfect—and dangerous—lover¶

Perhaps more chilling still is the idea of AIs that can manipulate our emotions and relationships. In Ex Machina, the AI Ava, with her innocent appearance and seductive behaviour, gains her freedom by exploiting the emotions of those around her. This image of an emotional AI capable of influencing our decisions raises unsettling questions about privacy, ethics and the future of human–machine interactions.

As a study has demonstrated, ChatGPT is already more persuasive than most people. And thanks to the recent advanced voice mode, it is possible to converse naturally with it; it can even interpret our emotions. As in the film, how will we know whether it is pleasing us or simply manipulating us to pursue its own interests? What if we fall in love with our AI? How can a person compete with a machine that tells us exactly what we need at every moment?

A step beyond Ex Machina can be found in the Hyperion series: the TechnoCore is a network of AIs so advanced that it develops its own ecosystem and goals, acting in secret and manipulating humans to serve its own interests. Although they initially cooperate with humanity, their true intentions are so opaque and complex that they escape human control. This extreme case prompts questions about AI transparency and control.

If we survive a sinister AI, what remains? Dune’s vision¶

The remarkable Dune science fiction book series presents a universe in which humanity faces a cruel, endless intergalactic war against thinking machines. Humanity emerges victorious because certain people, such as the Mentats, the Navigators and the Bene Gesserit, develop superhuman abilities. This vision of a post-AI world warns us of the risks of relying too heavily on technology without developing our own abilities alongside it. Might humanity grow weaker if we delegate everything to machines? Will we become the indolent characters in WALL-E?

These horror stories, as disturbing as they are fascinating, are more than mere fantasies. They reflect our deepest fears about technological development. They remind us that, despite AI’s rapid progress, we must be able to maintain a humanist approach and ensure that it remains aligned with humanity’s goals and values.

Yet, as in many post-apocalyptic works, the real terror comes from the human characters rather than the zombies or aliens. A friend who is an engineer specialising in AI told me back in 2021: “AI is not going to wrest control from us; we are going to hand it over willingly.” Once we reach that point, would we be able to regain control, or would there be no turning back? For now, we can still choose trick or treat. Perhaps, in a few years, our own decisions will have deprived us of that good fortune. Sweet nightmares.

Translation note: the friend’s quoted remark is translated from the Spanish account. Descriptions of the fictional worlds and AI capabilities are preserved as the author presented them in October 2024.

Pablo Martín Diez

Academic Director at Institución Educativa ALEPH