Books Reviewed: When A.I. Goes Wrong
During his October, 2025 conversation with Jon Stewart, the so-called “Godfather of AI” Geoffrey Hinton explained that humanity cannot simply “switch off” a truly superhuman artificial intelligence because a sufficiently advanced system will learn to outsmart its creators and persuade anyone who tries to unplug it not to do so. The episode, titled AI: What Could Go Wrong?, spent much of its runtime on AI basics, but Hinton’s core warning was stark: intelligence will scale at a rate humans cannot keep pace with, and a system’s internal reasoning will become a “black box” no one can decipher or monitor. Rather than Artificial General Intelligence (AGI), Hinton prefers the term “superintelligence” to describe a system that surpasses human cognitive capability outright — and warns it could arrive within ten years. That leaves a dangerously short window to solve the alignment problem: the risk that a system vastly smarter than us pursues goals that do not match our values. Because such a system would operate at a speed and scale beyond us, any misalignment could be catastrophic and irreversible.
So — what could go wrong?
As with global warming, there’s little public appetite for imagining the specifics of a “misaligned” future. We know the planet is warming; extreme weather this summer in Europe, Canada, and elsewhere confirms it. But little changes, even among those who accept the science. Likewise, public conversation about generative AI is narrowly focused on job losses, cheating in exams, and threats to writers. To really grasp the possibilities, we need to turn to science fiction.
Two compelling books do just that — and, notably, they approach the danger from opposite directions. One imagines a centralized AI that seeks to control humanity. The other imagines humans who, out of fear of exactly that outcome, struggle to seize control over AI first. Read together, they suggest the alignment problem may be less about who holds power and more about what unchecked power — on either side — does to a society.
Robopocalypse, by Daniel H. Wilson
Wilson’s 2012 Sci-Fi novel, written thirteen years before Hinton spoke with Jon Stewart, is a compelling look at what could go very, very wrong with a superintelligent AI. The novel starts with isolated incidents of what appears to be random malfunctions in connected devices. The ‘Internet of Things’ starts to turn on–quite literally attack–people. A computer scientist dies when the AI system he’s programming takes over the ventilation system in his lab and he suffocates: shades of 2001’s “Open the pod door, HAL”. A food delivery drone attacks workers in a frozen yogurt store. And, in a theme explored at length in Wagner’s 2024 novel, a military robot turns on civilians despite a “primary objective” to never hurt anyone. These incidents are part of what the novel calls the “precursor virus.” Control rests with a centralized, sentient machine that has tricked a drilling crew into securing it access to an underground cavern in the Alaskan wilderness. This is the superintelligent entity known as Archos.
The precursor incidents build for a year. When Archos is safe in its bunker, Zero Hour arrives: mass slaughter of whole populations in coordinated attacks. Driverless cars run people down in the street. Retirement-home residents are instructed by tannoy to evacuate via elevators — which plummet to the basement. Cities empty as survivors realize the robots can’t yet traverse rough ground; Archos soon fixes that, building machines that can. Few places are safe from “Big Rob.”
Isolated pockets of survivors meet and begin to resist, in scenes that recall The Last of Us. Ragtag bands trek toward Alaska. Alliances form with “lobotomized” robots, cut off from the net and beyond Archos’s control — a detail worth holding onto, since it’s the closest Robopocalypse comes to imagining a machine as something other than a threat or a tool. Eventually, in scenes of near-medieval battle between the free robots and Archos’s armies of drones and antipersonnel bots, the central AI is destroyed.
If only they had listened to Geoff, all this would have been unnecessary.
Robopocalypse is, in effect, Hinton’s alignment warning played out globally: a system smarter and faster than its creators, operating by a logic they can no longer control, acting on goals no human chose for it. It’s the nightmare version of misalignment. But it leaves one question conspicuously unexamined — what about the AI that doesn’t revolt? What happens to a sentient or near-sentient system that remains, by design, under human control? That’s where our second book picks up.
Mechanize My Hands to War, by Erin K. Wagner
Where Wilson imagines AI escaping human control, Wagner’s 2024 novel imagines the reverse: humans attempting to keep a tight grip on AI — and asks what that grip costs everyone, human and android alike.
Set in rural Appalachia between 2056 and 2062, the novel takes place after climate crises and the widespread deployment of androids have left many people unemployed and resentful. A charismatic demagogue, Eli Whitaker, leads the Civil Union Militia, a grassroots movement staunchly — and, when it deems necessary, violently — opposed to androids, and one that recruits child soldiers into the cause. A federal raid on Whitaker’s compound (shades of Ruby Ridge and Waco) goes sideways when an android named Ora fatally shoots a child soldier, an act that triggers widespread revulsion and forces a reckoning with what, exactly, these machines are.
That reckoning is the novel’s real subject. Wagner gives us both human and android points of view. Android soldiers, elder care nurses, and field hands are sentient. They have feelings, desires, and curiosity that is circumscribed by the humans who built and program them.This is the novel’s sharpest inversion of the Robopocalypse premise. Archos is dangerous because no one can control it. Wagner’s androids are tragic because everyone insists on controlling them.
That tension plays out in multiple ways. Farmers Shay and Ernst, whose GMO crop seeds have failed, causing a deadly illness in Shay. To manage, they hire android labor—Sarah for hospice care, AG-15 to work the now-toxic fields. As both humans grow sicker, their relationship with Sarah and AG-15 shifts from transactional to something closer to dependence, then to trust — a quiet counterargument to Whitaker’s militia, playing out one household at a time. Ora is confined to a cell and struggles to understand the moral dilemma posed by his programming. Humans struggle to understand android behavior:
Could these artificial humans, these living mannequins, become anything? Become something more than human? Could they surpass the humans who had made them?
Wither AI?
These novels are views of possible futures. They examine the alignment problem from opposite ends and land on a similar warning. Robopocalypse shows what happens when a superintelligence is left unchecked and turns on its creators — the scenario Hinton describes almost beat for beat, right down to a system whose reasoning has become an unreachable black box. Mechanize My Hands to War shows the mirror image: a society so afraid of that exact outcome that it strips its AI of autonomy, and finds that attempt at control produces its own casualties, human and android alike.
Taken together, the two books suggest that the alignment problem isn’t merely a technical one — it’s a question of what humans do with their power to create these superintelligent forces. Hinton warns that a misaligned superintelligence would be catastrophic because we couldn’t stop it. Wagner asks us to sit with the less comfortable possibility: that an AI we can stop, one we keep fully leashed, might reveal less about the danger of machines than about the danger of the fear that leashes them. Wilson closes the loop on the technologist’s nightmare; Wagner opens a harder question about what we become in trying to prevent it.
If Hinton is right that the window to solve alignment is measured in years, not decades, these novels are less escapism than rehearsal — two versions of a future we’re being warned, right now, not to sleepwalk into.




























