Books Reviewed: When A.I. Goes Wrong
Hinton’s Warning
During his October, 2025 conversation with Jon Stewart, the so-called “Godfather of AI” Geoffrey Hinton explained that humanity cannot simply “switch off” a truly superhuman artificial intelligence because a sufficiently advanced system will learn to outsmart its creators and persuade anyone who tries to unplug it not to do so. The episode, titled AI: What Could Go Wrong?, spent much of its runtime on AI basics, but Hinton’s core warning was stark:
“Intelligence will scale at a rate humans cannot keep pace with, and a system’s internal reasoning will become a ‘black box’ no one can decipher or monitor.“
Rather than Artificial General Intelligence (AGI), Hinton prefers the term “superintelligence” to describe a system that surpasses human cognitive capability outright — and warns it could arrive within ten years. That leaves a dangerously short window to solve the alignment problem: the risk that a system vastly smarter than us pursues goals that do not match our values. Because such a system would operate at a speed and scale beyond us, any misalignment could be catastrophic and irreversible.
So — what could go wrong?
As with global warming, there’s little public appetite for imagining the specifics of a “misaligned” future. We know the planet is warming; extreme weather this summer in Europe, Canada, and elsewhere confirms it. But little changes, even among those who accept the science. Likewise, public conversation about generative AI is narrowly focused on job losses, cheating in exams, and threats to writers. To really grasp the possibilities, we need to turn to science fiction.
Two compelling books do just that — and, notably, they approach the danger from opposite directions. One imagines a centralized AI that seeks to control humanity. The other imagines humans who, out of fear of exactly that outcome, struggle to seize control over AI first. Read together, they suggest the alignment problem may be less about who holds power and more about what unchecked power — on either side — does to a society.
Robopocalypse, by Daniel H. Wilson
Wilson’s 2012 Sci-Fi novel, written thirteen years before Hinton spoke with Jon Stewart, is a compelling look at what could go very, very wrong with a superintelligent AI. The novel starts with isolated incidents of what appears to be random malfunctions in connected devices. The ‘Internet of Things’ starts to turn on–quite literally attack–people. Among the precursor incidents:
- A computer scientist dies when the AI system he’s programming takes over the ventilation system in his lab and he suffocates: shades of 2001’s “Open the pod door, HAL”.
- A food delivery drone attacks workers in a frozen yogurt store.
- And, in a theme explored at length in Wagner’s 2024 novel, a military robot turns on civilians despite a “primary objective” to never hurt anyone.
These incidents are part of what the novel calls the “precursor virus.” Control rests with a centralized, sentient machine that has tricked a drilling crew into securing it access to an underground cavern in the Alaskan wilderness. This is the superintelligent entity known as Archos.
The precursor incidents build for a year. When Archos is safe in its bunker, Zero Hour arrives: mass slaughter of whole populations in coordinated attacks.
- Driverless cars run people down in the street.
- Retirement-home residents are instructed by tannoy to evacuate via elevators — which plummet to the basement.
- Cities empty as survivors realize the robots can’t yet traverse rough ground; Archos soon fixes that, building machines that can.
Few places are safe from “Big Rob.”
Isolated pockets of survivors meet and begin to resist, in scenes that recall The Last of Us. Ragtag bands trek toward Alaska. Alliances form with “lobotomized” robots, cut off from the net and beyond Archos’s control — a detail worth holding onto, since it’s the closest Robopocalypse comes to imagining a machine as something other than a threat or a tool. Eventually, in scenes of near-medieval battle between the free robots and Archos’s armies of drones and antipersonnel bots, the central AI is destroyed.
If only they had listened to Geoff, all this would have been unnecessary.
Robopocalypse is, in effect, Hinton’s alignment warning played out globally: a system smarter and faster than its creators, operating by a logic they can no longer control, acting on goals no human chose for it. It’s the nightmare version of misalignment. But it leaves one question conspicuously unexamined — what about the AI that doesn’t revolt? What happens to a sentient or near-sentient system that remains, by design, under human control? That’s where our second book picks up.
Mechanize My Hands to War, by Erin K. Wagner
Where Wilson imagines AI escaping human control, Wagner’s 2024 novel imagines the reverse: humans attempting to keep a tight grip on AI — and asks what that grip costs everyone, human and android alike.
Set in rural Appalachia between 2056 and 2062, the novel takes place after climate crises and the widespread deployment of androids have left many people unemployed and resentful. A charismatic demagogue, Eli Whitaker, leads the Civil Union Militia, a grassroots movement staunchly — and, when it deems necessary, violently — opposed to androids, and one that recruits child soldiers into the cause. A federal raid on Whitaker’s compound (shades of Ruby Ridge and Waco) goes sideways when an android named Ora fatally shoots a child soldier, an act that triggers widespread revulsion and forces a reckoning with what, exactly, these machines are.
That reckoning is the novel’s real subject. Wagner gives us both human and android points of view. Android soldiers, elder care nurses, and field hands are sentient. They have feelings, desires, and curiosity that is circumscribed by the humans who built and program them.This is the novel’s sharpest inversion of the Robopocalypse premise.
Archos is dangerous because no one can control it.
Wagner’s androids are tragic because everyone insists on controlling them.
That tension plays out in multiple ways. Farmers Shay and Ernst, whose GMO crop seeds have failed, causing a deadly illness in Shay. To manage, they hire android labor—Sarah for hospice care, AG-15 to work the now-toxic fields. As both humans grow sicker, their relationship with Sarah and AG-15 shifts from transactional to something closer to dependence, then to trust — a quiet counterargument to Whitaker’s militia, playing out one household at a time. Ora is confined to a cell and struggles to understand the moral dilemma posed by his programming. Humans struggle to understand android behavior:
“Could these artificial humans, these living mannequins, become anything? Become something more than human? Could they surpass the humans who had made them?”
Wither AI?
These novels are views of possible futures. They examine the alignment problem from opposite ends and land on a similar warning. Robopocalypse shows what happens when a superintelligence is left unchecked and turns on its creators — the scenario Hinton describes almost beat for beat, right down to a system whose reasoning has become an unreachable black box. Mechanize My Hands to War shows the mirror image: a society so afraid of that exact outcome that it strips its AI of autonomy, and finds that attempt at control produces its own casualties, human and android alike.
Taken together, the two books suggest that the alignment problem isn’t merely a technical one — it’s a question of what humans do with their power to create these superintelligent forces. Hinton warns that a misaligned superintelligence would be catastrophic because we couldn’t stop it.
Wagner asks us to sit with the less comfortable possibility: that an AI we can stop, one we keep fully leashed, might reveal less about the danger of machines than about the danger of the fear that leashes them.
Wilson closes the loop on the technologist’s nightmare; Wagner opens a harder question about what we become in trying to prevent it.
If Hinton is right that the window to solve alignment is measured in years, not decades, these novels are less escapism than rehearsal — two versions of a future we’re being warned, right now, not to sleepwalk into.
Note: Sub-edits were made following suggestions from Mark Ivey. Images via Midjourney.
4 Comments so far
Leave a comment
Tracking live updates on this topic, highlighting where Sci-Fi illuminates the way forward…
On August 7 The Register reports that ‘Asimov was right’ about rules for robots, says ex-US Cyber Director. Humans will get the AI models they deserve.
In a nod to Sci-Fi author Isaac Asimov who proposed the “Three Laws of Robotics“, former US National Cyber Director Chris Inglis noted that:
By Theboss on 08.14.26 1:00 pm
Two fascinating books for sure, with much different approaches. I lean toward Mechanize My Hands because it feels fresher and very current, which it is – and it’s goes beyond the end of the world via AI type themes, which we’ve seen since 1968 with the release of 2001: A Space Odyssey. This is a more nuanced look at the machines we’re trying to figure out.
The essential question is the one at the end of the second review: Humans struggle to understand android behavior:
“Could these artificial humans, these living mannequins, become anything? Become something more than human? Could they surpass the humans who had made them?”
I wonder if 5 years from now we’ll still be asking the same question- or if by then AI will have completely turned out world upside down and we’ll be looking back at today and wondering why we didn’t figure it out earlier?
By mark ivey on 08.16.26 3:51 pm
And so it begins… from the Financial Times Weekend Edition, Saturday, Aug 22, 2026.
China holds global reins as it pushes deeper into robotics
By William Langley in Guangzhou, Joe Leahy in Beijing and William Sandlund in Hong Kong
By Theboss on 08.25.26 7:49 am
Writing in the Weekend FT, Gillian Tett highlights the cultural difference between Japanese and Anglo-Saxon views about robots. Her September 5, 2026 column, titled ‘America must learn AI lessons from Astro Boy‘ notes that “Japan’s media culture has always been positive about robots and AI — as epitomized by the wildly popular Astro Boy manga and anime series featuring a cute robot who befriends humans.”
This is not the case in other countries.
The difference is rooted in Japan’s spiritual traditions:
Nihonjin no robotto-suki
These cultural differences are explored in Wilson’s Robopocalypse. One character, Takeo Nomura, is an elderly Japanese machine repairman and robotics expert living in Tokyo. At 65 years old, Nomura lives a solitary life and forms a deep emotional attachment to a lifelike companion android named Mikiko.
When the rogue AI, Archos, triggers the robot uprising (“Zero Hour”) and hacks Mikiko to attack him, Nomura manages to save her and rewrite her code. By doing so, he grants her true self-awareness and free will. Mikiko transmits a signal that frees other humanoid robots from Archos’ control, giving birth to the “Freeborn” robot rebels. Nomura transforms his factory into a fortified sanctuary, eventually becoming celebrated as one of the greatest technical minds of the human resistance.
Perhaps the world will be saved by the Japanese love of robots! (Nihonjin no robotto-suki)
By Theboss on 09.08.26 4:21 pm
Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>