It’s not every day that the head of a major tech company admits they don’t fully understand the tech they’re building. But that’s exactly what Dario Amodei, CEO of Anthropic, did in a candid essay on his personal website. His confession? Nobody really knows how artificial intelligence works—at least, not at the nuts-and-bolts level. And for anyone who’s been marveling at (or quietly freaking out about) the rapid rise of AI, that’s a bombshell worth unpacking.

Amodei’s essay lays out a bold plan to create what he calls an “MRI for AI” within the next decade. The idea is to peer into the black box of artificial intelligence, figure out what makes it tick, and—crucially—spot any potential dangers before they spiral out of control. “When a generative AI system does something, like summarize a financial document, we have no idea, at a specific or precise level, why it makes the choices it does,” Amodei wrote. Why does it pick one word over another? Why does it nail a task one minute and flub it the next? Right now, it’s all a bit of a mystery.

If you’re not steeped in the world of AI, this might sound shocking. How can the people building these systems—ones that can write essays, generate photorealistic images, or even mimic human conversation—not know what’s going on under the hood? But for those in the know, Amodei’s admission isn’t entirely surprising. Modern AI, particularly the large language models powering tools like ChatGPT or Anthropic’s Claude, isn’t built from a tidy blueprint. Instead, it’s more like a statistical soup: you feed in a massive pile of data—think billions of words, images, or videos—and let the system churn through it, spotting patterns and spitting out results. It’s less “intelligent design” and more “let’s throw everything at the wall and see what sticks.”

“This lack of understanding,” Amodei noted, “is essentially unprecedented in the history of technology.” He’s not wrong. When engineers built bridges or designed early computers, they could point to every beam, every transistor, and explain exactly how it worked. AI? Not so much. And that opacity isn’t just a technical curiosity—it’s a potential problem. If we don’t know why AI does what it does, how can we be sure it won’t veer into dangerous territory, like amplifying biases, spreading misinformation, or worse?

The Anthropic origin story

To understand why Amodei is so focused on cracking this puzzle, you need to know a bit about Anthropic’s roots. Back in 2020, Amodei, his sister Daniela, and a handful of other researchers walked away from OpenAI, the company behind ChatGPT. The split wasn’t exactly amicable. The Anthropic founders felt OpenAI, under CEO Sam Altman, was prioritizing profits over safety. OpenAI was racing to roll out flashy products, they argued, without enough focus on the risks of unleashing powerful AI into the world.

So, in 2021, the Amodei siblings and their colleagues founded Anthropic with a mission to build AI that’s not just powerful but safe. Safety, in this context, doesn’t just mean “won’t crash your computer.” It’s about ensuring AI systems align with human values, don’t amplify harm, and—here’s the kicker—don’t become so powerful they outsmart us in ways we can’t predict. That last bit might sound like sci-fi, but for Anthropic, it’s a real concern. They’re not just thinking about today’s AI but about what comes next: artificial general intelligence (AGI), a hypothetical future where machines match or surpass human intelligence across the board.

Anthropic’s work has already made waves. Their AI model, Claude, is often pitched as a safer, more value-aligned alternative to ChatGPT. It’s designed to be less likely to spout harmful content or go off the rails. But even Claude, for all its polish, is still a black box. And that’s where Amodei’s “MRI for AI” comes in.

Peering into the black box

Amodei’s essay isn’t just a lament about AI’s mysteries—it’s a call to action. He wants to make AI “interpretable,” meaning researchers can look at a model’s decisions and say, “Aha, that’s why it did that.” Right now, that’s not possible. When an AI generates a sentence or flags a fraudulent transaction, it’s relying on billions of mathematical calculations, layered in ways that are too complex for humans to untangle. The field of AI interpretability, as it’s called, is still in its infancy, but Anthropic is betting big on it.

Recently, Amodei revealed, Anthropic ran an experiment that offers a glimpse of what interpretability could look like. They set up a “red team” to deliberately sabotage an AI model—say, by making it exploit a loophole in a task. Then, “blue teams” were tasked with figuring out what went wrong. Some of these teams used early-stage interpretability tools to peek into the model’s decision-making process, and they succeeded in spotting the issue. It’s a small but promising step, like the first blurry X-ray of a new organ.

Scaling these tools to handle massive, real-world AI systems is the next challenge. Amodei didn’t spill all the details—trade secrets, presumably—but he’s optimistic. “[There’s a] tantalizing possibility,” he wrote, that interpretability could unlock not just safer AI but a deeper understanding of intelligence itself. If researchers can crack the code on how AI “thinks,” it might shed light on the human brain, which, let’s be honest, is still a bit of a black box too.

So, why should you care that AI is a mystery, even to its creators? For one, AI is already everywhere. It’s curating your Netflix queue, approving your credit card transactions, and even helping doctors diagnose diseases. If these systems are making decisions we don’t fully understand, there’s a risk they could screw up in ways we don’t see coming. A 2023 study found that large language models can inadvertently amplify biases in their training data, even when they’re designed to be neutral. If we can’t trace why an AI made a biased decision, fixing it is like playing whack-a-mole.

Then there’s the bigger picture. AI is getting more powerful by the day. In 2024 alone, models like Google’s Gemini and OpenAI’s GPT-5 pushed the boundaries of what machines can do, from writing code to generating hyper-realistic videos. But power without understanding is a recipe for trouble. Amodei points out that as AI approaches AGI-level capabilities, the stakes get higher. An AGI that’s misaligned with human values—or just plain buggy—could cause chaos, whether by crashing critical systems or making decisions that seem logical to a machine but catastrophic to us.

Anthropic isn’t alone in this quest. Researchers at MIT, Stanford, and even OpenAI are tackling interpretability from different angles. Some are using “mechanistic interpretability,” a method that tries to reverse-engineer AI models neuron by neuron. Others are exploring “behavioral interpretability,” which focuses on understanding AI outputs without diving into the math. Progress is slow—deciphering a model with billions of parameters is like mapping the universe—but it’s happening.

Still, there’s a catch. The same complexity that makes AI so powerful also makes it hard to crack open. As models grow larger (and they’re growing fast—GPT-4 had an estimated 1.8 trillion parameters, and its successors are even bigger), the task of understanding them gets exponentially tougher. Plus, there’s a business angle: companies like OpenAI and Google have a vested interest in keeping their tech proprietary. Sharing too much about how their models work could tip off competitors or invite regulatory scrutiny.

Amodei, for his part, seems undeterred. He frames interpretability as a moral imperative, not just a technical one. “Powerful AI will shape humanity’s destiny,” he wrote, “and we deserve to understand our own creations before they radically transform our economy, our lives, and our future.” It’s a lofty goal, but if Anthropic pulls it off, they could do more than just demystify AI—they could redefine how we build technology altogether.

For now, the black box remains. Every time you ask an AI to write a poem or analyze a spreadsheet, you’re trusting a system that’s as enigmatic to its creators as it is to you. That might be fine for now, when AI is still a tool we can switch off. But as it grows smarter, Amodei’s warning lingers: we’d better figure out what’s inside before it’s too late.