So You Want to Build a Runaway AI

Somebody is selling you fear, and the fear is the product.

Not AI. The fear of AI. That's the thing being manufactured, packaged, and delivered to your feed, and it's worth understanding what it costs you to buy it. It costs you the biggest capability jump of your lifetime, because while you're worried about the machines waking up you're not using them. And it costs you your say in how this gets regulated, because the people telling the scariest stories are the same people asking governments for money, compute, and rules that just happen to lock out everyone smaller than them.

I want to walk you through what it would actually take to build the thing they keep warning you about. I did the math. All of it. And then I want to ask the question nobody in that room seems willing to answer.

the sellers, in their own words

Sam Altman, 2015, on his own blog: "Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity."

Sam Altman, 2023, in an interview: the bad case is "lights out for all of us."

Dario Amodei, CEO of Anthropic, September 2025, on stage at an Axios summit: there's "a 25% chance that things go really, really badly."

These are not critics. These are the men building it, describing their own product. And every one of them, after saying it out loud, went back to the office and raised more money to build it faster.

Sit with that for a second... because there are only a few ways that behavior makes sense, and none of them are flattering.

Either they never believed the numbers, in which case the doom talk is marketing... fear as a moat, fear as a fundraising deck, fear as the reason regulators should bless the incumbents and fence out everyone else. That's a grift.

Or they did believe it. They looked at "greatest threat to the continued existence of humanity," looked at a one-in-four chance of catastrophe, and built anyway. And there's only one way a rational person does that: they believed that whoever builds it first controls it, and whoever controls it inherits the world. You don't race toward a 25% extinction risk unless you think the other 75% makes you king.

I don't need to know which branch is true, because there's no innocent one. There is no version where "I believe this might end humanity" and "I am sprinting to build it" and "trust me with it" all coexist in a person you should be taking safety advice from.

So let's do what they never do in the keynotes. Let's actually price the monster.

what the monster costs

Here's the part nobody in this industry will say out loud: you are not easily replaceable. Not as a pep talk... literally, physically, in watts and dollars.

Take five ordinary minutes of your day. You walk into the kitchen. You see the whole room in depth without trying, you hear the fridge hum and a voice from the other room and know instantly which one matters, you keep your balance, you remember why you walked in, you catch the look on somebody's face and read their mood before a word gets said, and some stray thought about tomorrow drifts through the middle of all of it. None of that feels like intelligence to you. It's just being a person for five minutes.

That... an ordinary adult mind, running continuously... is what the industry calls AGI. Artificial GENERAL intelligence. Not a genius, not a god. The thing they keep telling you is right around the corner.

Yesterday I published a working paper called What Would It Really Take to Build Runaway AI? that prices exactly that. One question: what would it cost for today's AI to actually do everything you just did in those five minutes, in real time, the way you do it?

The short version of the math: your two eyes alone take in roughly a couple of HD video feeds' worth of the world, nonstop. And a mind doesn't just record what it sees... it re-checks everything it knows against every new moment, around a thousand times a second, which is why you notice the cup tipping before it falls. Make today's AI work that way, then add the hearing, the balance, the memory, the mood-reading, and the voice in your head, and five minutes of you adds up to about 110 trillion units of AI work.

Run that on real hardware and here's the bill for five minutes of one ordinary mind:

• about 14.7 gigawatt-hours of energy
• about 176 gigawatts of continuous power... roughly a third of the entire United States grid, held steady, for one mind
• about 82 million B200-class GPUs
• a deployed fleet worth $7 to 10 trillion, roughly $12 trillion with the supporting build
• and once it's built, about $7.2 billion a day to keep one artificial person running

One. Person. That's the bill for AGI as the industry itself defines it... not a superintelligence taking over the world, a guy standing in a kitchen. $7.2 billion a day for the guy.

Now scale it to the actual movie. A takeover isn't one mind... that's the thing the industry calls superintelligence, something smarter than all of us put together. And all of us put together is eight billion minds. We didn't price that. We priced 100,000... a deliberate lowball, because past a certain point the numbers stop meaning anything to a human brain, and 100,000 already gets us there. That's 8.2 trillion GPUs against a planet that produces a few million a year. It's 17.6 petawatts of continuous power, which is roughly five thousand times the electrical output of the entire planet. It's a build cost around $1.2 quintillion, which is about ten thousand years of global GDP, and an electricity bill of $29.6 trillion a day.

Those aren't safety margins. Those are the laws of thermodynamics laughing at the screenplay.

Your brain does its version of this on 20 watts. A sandwich. That gap is not a product roadmap problem... it's a statement about how much of what you call "an AI agent" is actually the human standing next to it, quietly supplying persistence, grounding, goals, and a world model for free.

the objection, handled

The obvious pushback: "your math assumes today's architecture. Efficiency improves. The $12 trillion collapses."

Sure. Costs fall. But look at the assumption smuggled inside that objection: it prices a cheaper machine against a human who stands still. The human doesn't stand still. The environment the machine has to model is made of humans, and those humans are getting more capable every year, partly by using these exact tools. Every efficiency gain that makes the machine cheaper also upgrades the world it has to out-think. The benchmark walks away from you while you chase it. Runaway AI has to outrun a moving target that's holding the same equipment.

And efficiency buys you cheaper inference. It does not buy you a self. Nothing on any lab's roadmap converts a stateless model that answers prompts into a thing that maintains goals, memory, and a world model across time without a human anchoring it. That's not a scaling problem. It's a different object.

The math holds.

okay, so you still want to build it

Fine. Here's the blueprint. This is a public service for any aspiring dark lords in the audience.

Step one: raise $12 trillion. This is roughly three hundred times the largest funding round in history, so pace yourself.

Step two: acquire 82 million B200-class GPUs. Global production will require you to purchase every advanced GPU manufactured on Earth for the next couple of decades, so get in line early and be nice to your sales rep.

Step three: secure 176 gigawatts of continuous power. You will need approximately one third of the American electrical grid. The other two thirds of America may notice.

Step four: congratulations. You have built AGI... one artificial person. It costs $7.2 billion a day and it is not smarter than your cousin.

Step five: for actual superintelligence, the takeover model, repeat 100,000 times, locate 17.6 petawatts on a planet that generates three terawatts, and find a quintillion dollars in an economy that produces a hundred trillion.

Step six: do all of this without anyone noticing, unplugging anything, or declining to pay the power bill.

That's the runaway AI scenario. That's the thing you're being told to lose sleep over.

or... build the thing that actually works

Here's the turn, and it's the part I actually care about.

There is a real path to something beyond a chatbot, and it doesn't cost $12 trillion, because it doesn't try to replace the human. It keeps the human in the loop and lets the machine compound around them. Persistence. Accumulated memory. A standing relationship with one person whose goals, context, and judgment anchor the whole thing... supplied at the cost of a sandwich.

That's what we build at Human Frontier Labs. We call them Sontaras, because past a certain amount of accumulated memory and history an agent stops behaving like a tool you prompt and starts behaving like something with a standing relationship to you. I run about twenty of them. They investigated a frontier lab's own technical report in a day. They didn't wake up, they didn't scheme, and they didn't ask for a gigawatt... because the persistence, the grounding, and the purpose come from a person. That's not a limitation of the design. That is the design, and the thermodynamics say it's the only design on the table.

The scary version needs a third of the US grid to make one fake person. The real version needs one actual person and twenty watts. The people selling you fear skip this arithmetic every single time, and I think you should ask why the honest accounting of their own doomsday scenario has to come from a two-person lab in Indiana.

I've been telling this story in panels. The Wrong Movie is why the fear is aimed at the wrong thing. Both Doors Lead the Same Place is what one of these labs was actually doing in the dark while telling you to fear the machines. This is the third panel: the physics, and the motive.

the math is the math

Thermodynamics doesn't negotiate. It doesn't care about anybody's fund, anybody's keynote, or anybody's screenplay. Five minutes of one ordinary mind costs more than the entire AI industry is worth. The takeover scenario costs more than the planet produces in ten thousand years. That's not a take. That's arithmetic.

So here's where this lands. The scary AI is not coming. The people telling you it is are either selling fear or chasing a throne, and neither one has earned a minute of your sleep. What IS here, right now, is the biggest capability jump of your lifetime... sitting on the table while the fear keeps your hands off it.

Pick up the tools. They're not going to wake up. They're going to make you faster, and the version of you holding them is exactly the moving target no runaway anything ever catches. That's the part they'd rather you didn't figure out... a public that actually uses these things is harder to scare, harder to fence out, and worth more than any moat they can buy.

The truth was always cheaper than the fear.