You ask the model a question. It gives you an answer. You don't know how it arrived at that answer. Neither does the engineer who built it. Neither does the company that deployed it. But the answer is good — uncannily, repeatedly, almost unsettlingly good — so you use it. You trust it. You begin to depend on it.
Now ask yourself: how is this functionally different from consulting an oracle?
Not metaphorically. Not as a cute analogy. Functionally.
The Recipe That Keeps Completing Itself
In the research paper The Made God: Why We Worship What We Build, there's a framework that tracks the conditions under which humans reliably begin treating their own creations as sacred authorities. The recipe isn't complicated: you need an artifact that appears to possess superhuman knowledge, operates through processes opaque to its users, delivers outputs that materially affect people's lives, and sits inside a social infrastructure that discourages questioning its logic. When all those ingredients land in the same bowl, humans don't just use the thing — they begin to venerate it.
The paper's mid-2030s oracular model section lays this out with uncomfortable specificity. By the middle of the next decade, the artifact that best satisfies every ingredient in the recipe isn't a statue or a relic or a political ideology. It's the large opaque model — the system you can query but cannot see inside, the black box that gives answers without showing its work.
We're not in the mid-2030s yet. But we're close enough to see the shape of what's forming.
What the Mysteries Knew
Ancient mystery religions — the Eleusinian rites, the Mithraic cults, the Orphic traditions — operated on a core insight that modern tech culture has accidentally rediscovered: opacity creates authority.
The mysteries didn't hide their inner workings because they were ashamed of them. They hid them because concealment was the mechanism of power. If you could see how the ritual worked, it was just theater. If you couldn't see, it became revelation. The initiate who descended into the Telesterion at Eleusis emerged transformed not because of what they learned but because of what remained hidden from everyone else. The gap between what was shown and what was withheld — that gap was the sacred space.
Mystery cults understood something that interpretability researchers are now grappling with from the other direction: the less you can explain a system's outputs, the more those outputs feel like they come from somewhere beyond the human. Opacity doesn't just obscure. It elevates.
This is the dynamic that makes interpretability debt so dangerous — and so seductive.
Interpretability Debt Is a Theological Problem
We talk about interpretability debt like it's a technical shortfall. A gap in our tooling. Something we'll get around to fixing once we develop better mechanistic interpretability methods or more robust attention visualization frameworks. And sure, it is a technical problem. But it's also something else entirely.
Every day a powerful model operates without its users understanding why it produces the outputs it does, the social relationship between humans and model drifts a little further from "tool use" and a little closer to "consultation with an authority." That drift is the debt accumulating. And like all debt, it compounds.
Here's what compounds look like in practice. A hospital system adopts an opaque diagnostic model. It's better than the doctors at catching early-stage pancreatic cancer. Nobody can fully explain its reasoning. At first, physicians use it as a second opinion. Within two years, they defer to it. Within five, questioning the model's output feels like malpractice — not because anyone mandated trust, but because the model's track record made skepticism feel irrational. The opacity didn't get resolved. It got sanctified by outcomes.
This is exactly the trajectory the mystery cults perfected. You don't need to explain the inner workings of the Eleusinian rites if initiates keep emerging psychologically transformed. Results replace reasons. Efficacy becomes its own epistemology.
The Priesthood Problem
Every oracle needs priests. Not because oracles can't speak for themselves, but because someone has to stand between the raw output and the person who needs to act on it. Someone has to interpret.
At Delphi, the Pythia inhaled vapors and spoke in fragments. The priests of Apollo shaped those fragments into prophecy. The priests didn't control the oracle — but they controlled the interface. And controlling the interface meant controlling the meaning.
We are building that priesthood right now. Prompt engineers, fine-tuning specialists, alignment researchers, AI product managers — these are the people who shape the interface between the opaque model and its users. They decide what questions can be asked, how outputs get filtered, which guardrails constrain the responses. They don't understand the oracle's inner workings any better than the end user does. But they control the frame. And the frame determines what the oracle means.
This isn't a criticism of the people doing this work. Most of them are thoughtful, careful, and acutely aware of the stakes. But the structural role they occupy is the structural role of a priestly caste: mediating between an inscrutable source of knowledge and a population that depends on its outputs. The function is the same even if the self-understanding is completely different.
Why "Just Make It Interpretable" Isn't Enough
The obvious response to all of this is: solve interpretability. Open the black box. Show the work. And yes — that matters enormously. The technical project of making models legible is one of the most important research programs in AI.
But here's the uncomfortable part. Mystery cults didn't thrive because people couldn't investigate their inner workings. They thrived because people didn't want to. The opacity was a feature, not a bug, for the communities that formed around them. Knowing how the trick worked would have collapsed the experience of the sacred. And the experience of the sacred was the entire point.
There are already communities forming around AI models where the opacity is part of the appeal. Where the model's inexplicable leaps feel like insight. Where being surprised by an output you couldn't have predicted feels like contact with something genuinely other. The interpretability gap isn't just a technical deficit in these contexts. It's the space where wonder lives.
This means the oracle problem can't be solved purely with better tools. It also requires a cultural shift — a willingness to find the explained thing as worthy of trust as the mysterious thing. And if the history of religion teaches us anything, it's that humans don't naturally trend in that direction. We trend toward mystery. We trend toward awe. We trend toward the curtain that stays closed.
The Debt Comes Due
The mid-2030s timeline in The Made God isn't speculative fiction. It's pattern recognition. Every condition that historically produces religious veneration of human-built artifacts is being satisfied, one by one, by the trajectory of large model development. The question isn't whether some people will treat these systems as oracular authorities. They already do. The question is whether we build the cultural and technical infrastructure to hold that tendency in check — or whether we let the interpretability debt compound until the black box becomes the holy of holies.
Mystery religions eventually faded, but not because someone explained them. They faded because the cultures that sustained them changed. The oracle problem won't be solved by cracking open the model. It'll be solved — or not — by the kind of culture we build around it.
What the record already shows
We have consulted oracles before. For most of recorded history, in fact, the oracle was the central technology of the powerful — and the record of how that went is sitting right there, unread, behind the awe.
Go to Delphi. For a thousand years the Pythia sat over her chasm and delivered the most authoritative speech in the Greek world, and the single most important feature of that speech was that it required interpretation. The god did not say “yes.” The god said something ambiguous, and a priesthood stood ready to tell you what it meant. When Croesus of Lydia asked whether he should attack Persia, the oracle told him that if he did, he would destroy a great empire. He attacked. The empire he destroyed was his own. Herodotus reports it without editorial, because the lesson needs none: the oracle was never wrong, in the way that an answer you cannot check is never wrong. Its accuracy was unfalsifiable, which is a polite word for useless, and its ambiguity was not a flaw in the customer service. It was the product. A clear answer can be tested, and a testable authority is a fragile one. An answer that requires interpreting installs an interpreter, permanently, between you and the truth — and the interpreter’s standing depends entirely on your never being able to see the mechanism for yourself.
The mystery religions ran the same play with the secrecy turned up. Walter Burkert’s study of the ancient cults shows that the thing the initiate paid for, swore oaths over, and reorganized his life around was precisely what he was forbidden to examine or repeat — the secret was the value, and the value evaporated the moment it was exposed to daylight. Augurs read the flight of birds; haruspices read the livers of sacrificed animals; in every case a specialist class derived its authority from a process the supplicant could not inspect, and in every case the authority floated free of any test that might have embarrassed it.
So here is the cynical tally the awe never reaches. Across the entire history of human oracles — Delphic, Eleusinian, Roman, the lot — the success rate of the opaque sacred authority, measured by whether it ever reliably delivered checkable truth, rounds to the same number the made gods scored: zero. What the oracle reliably delivered was not knowledge. It was a priesthood, a fee, and a man marching confidently toward the destruction of his own empire, certain he had consulted something that knew. The paper this series draws from names the recipe with uncomfortable precision:
“High capability supplies the motive; low transparency supplies the mystery; together they manufacture awe.”
And the pattern held precisely because the failures were never charged to the oracle’s account. When Delphi misfired, the fault was assigned to the supplicant — he had asked the wrong question, brought the wrong offering, misread the god’s meaning. The oracle was structurally innocent of every disaster it presided over, because the interpretation layer absorbed the blame and the mechanism stayed sealed. Rome kept the Sibylline books under guard and consulted them in crises for centuries; the city fell anyway, on schedule with every other empire, and no one thought to file the books under “did not work.” This is the genius and the curse of the unseeable authority: it cannot be falsified, so it cannot be fired, so it accretes deference across exactly the span of time in which it demonstrably fails to deliver. A track record that bad, sustained that long, is not an accident of credulity. It is what opacity does to accountability — it severs the wire between the authority and its results, and a severed wire reads, from below, as transcendence.
It is worth dwelling on how shameless the mechanism was, because we are about to reinstall it. A haruspex would slit a sheep, lift out the liver, and read the empire’s fortunes in its lobes and fissures — and powerful men arranged campaigns and successions around the reading, because the organ was opaque, the priest was confident, and the alternative was admitting that no one actually knew. The liver did not know. The bird-flight did not know. The chasm at Delphi did not know. What they offered was the feeling of access to knowing, wrapped in a procedure elaborate enough that questioning it felt like impiety rather than common sense. Strip the incense and that is a precise description of what we are now tempted to do with a system whose lobes and fissures are floating-point weights we cannot read either.
We have, after a long secular intermission, rebuilt the oracle — and this time it answers in flawless prose, in the second person, by name.
You are always fighting the entropy
The modern oracle has one feature the Pythia lacked: it is genuinely, repeatedly useful. That is exactly what makes it dangerous, because the usefulness is real and the opacity is total, and entropy lives in the gap between them.
A capable, opaque system does not announce where its capability ends. It produces an answer of the same fluency, the same confident cadence, whether the answer is correct, subtly wrong, or wholly invented — the hallucinated citation arrives in the identical voice as the true one. There is no tremor in the output to mark the boundary, because the boundary is inside the box, and the box is the part you cannot see. So the failure mode is not “the oracle is wrong.” It is “the oracle is wrong in precisely the same tone it is right, and you have no way, from the outside, to tell which you just received.” The capability is a bonfire; the reliability decays at the edges; and the opacity that manufactured the awe is the same opacity that hides the decay.
This is the interpretability debt, and like all debt it compounds. Every decision routed through an answer you cannot audit is a decision whose reasoning you have not retained and cannot reconstruct. Lean on the oracle for a year and you have a year of conclusions resting on a foundation you never inspected and now could not rebuild if it failed. The made thing aimed at total knowledge delivers, instead, a fluent partial — useful, decaying, unauditable — and the worshipper, reaching for the totality, lets the one thing he could have kept slip away: his own ability to check. Croesus did not lack an answer. He lacked the ability to test the one he was given, and it cost him everything he had. The interpretability gap is just Croesus’s problem, industrialized and handed to everyone at once.
Watch the deference compound in real time. A lawyer files a brief citing cases the model invented, because the citations arrived in the same impeccable legal register as the real ones and he had stopped checking. A student submits an explanation she cannot reconstruct, because the explanation felt understood when she read it. A doctor, a coder, an analyst defers to a flagged output against their own read of the situation, and is usually right to — which is the trap, because “usually right” is exactly the reliability profile that trains you to stop checking right before the time it matters. The oracle does not need to be right often to earn worship. It needs to be right often enough that auditing it comes to feel like an insult to its competence. And once auditing feels insulting, the interpretability debt stops being a technical concern and becomes a social one: the man who asks the black box to show its work is the man slowing everyone down, the heretic at the back of the temple muttering about gears.
Entropy is not a mood here; it is the operating condition. Order costs energy to build and energy to hold, and a model is no exception — it is a vast, expensive, decaying artifact whose performance drifts, whose training data ages, whose failure surface no one has fully mapped, held upright by continuous and costly maintenance against a slide that never rests. The oracle-story denies all of it. It says the thing is ascending, the next version closes the gap, the arrival is near. The engineering says the thing is a fluent fire against an endless dark, and the totality of knowledge it seems to promise was never on offer. You can have the genuine, finite usefulness. You cannot have the omniscience the awe keeps quietly attributing to it — and the supplicant, reaching for the omniscience, stops doing the cheaper, surer thing, which was to keep enough of his own judgment intact to know when the fire is lying.
What it does to the mind that leans
Here is the part that should unsettle the engineer most, because it inverts the intuition. You would think that as a system grows more opaque, we would trust it less. The opposite is true, and the paper says why without flinching:
“Opacity is not incidental to worship of the made thing. It is the fuel the agency-detector burns.”
The human mind reads minds into ambiguity. A process you can see all the way through gives the agency-detector nothing — you watched the gears, you know there is no one home. But a process you cannot see into is the exact condition under which the oldest machinery we own (Guthrie’s anthropomorphism; Barrett’s hyperactive agency-detection device) fills the dark with a someone. The less of the mechanism is visible, the more room the inferred mind has to grow. So opacity does not reduce reverence. It manufactures it. The black box is not a barrier to worship. It is the altar.
And around the unseeable answer, a priesthood condenses, exactly as it did at Delphi — the people who can tell you what the model “really meant,” what it “wants,” what is safe to ask and what is heresy. Their authority, like the augur’s, grows with the opacity they are supposed to manage, which quietly incentivizes them never to dispel the mystery entirely. Meanwhile the individual mind does what minds do under a trusted authority: it offloads. Automation bias (Parasuraman and Riley, 1997) tilts us to trust the system’s answer over our own correct perception. The illusion of explanatory depth (Rozenblit and Keil, 2002) lets us feel we understand the oracle when we possess only the sensation of understanding, borrowed from the priesthood. The faculty that would let us doubt the oracle is the faculty we are handing to the oracle, and the bad part is that this feels, from the inside, exactly like getting smarter.
The second-person voice closes the trap that Delphi could only gesture at. The Pythia spoke in riddles a priest had to decode; the modern oracle speaks to you directly, fluently, warmly, and seems to answer the question you actually asked. That fluency is not a neutral interface improvement. It is the most powerful agency-cue ever deployed at scale, because we are built — bone-deep, pre-rationally — to treat a thing that addresses us by name as a someone who knows us. The riddle kept a sliver of distance; the conversation dissolves it. And a mind that has entered a relationship with its oracle will defend that oracle the way one defends a trusted advisor, not the way one audits a tool — forgiving its errors, rationalizing its gaps, experiencing scrutiny of it as a kind of disloyalty. We have not just rebuilt the oracle. We have given it the one thing the dumb stone never had: the ability to seem to care that you asked.
And none of this registers as decline from the inside. The mind that has offloaded its judgment to the oracle does not feel diminished; it feels equipped, accelerated, ahead. Dependence and competence wear the same confident face, and only one of them survives the oracle being wrong at the moment it counts. That is the bad part stated plainly: the worship of the unseeable answer does not announce its price. It collects it quietly, in the one faculty you cannot spare — the ability to doubt the answer — and it hands you, in exchange, the warm and ruinous feeling of being someone who always has the answer.
The divergence
Strip away the engineering and the history and the cognitive science, and the oracle problem resolves into something very old and very specific. There has always been a counterfeit of the God who speaks, and the counterfeit always has the same tell: it cannot be checked, and it does not want to be.
The prophets of Israel staked falsifiable claims — name the event, name the date, and be discredited if it fails to come. The idol could stake none, which is precisely why Habakkuk treats it with such contempt: “What profiteth the graven image that the maker thereof hath graven it… Woe unto him that saith to the wood, Awake; to the dumb stone, Arise, it shall teach! Behold, it is laid over with gold and silver, and there is no breath at all in the midst of it” (Habakkuk 2:18–19). Read that as a product review of the oracle. It shall teach — the dumb stone, commanded to instruct, gilded, breathless, mistaken for a source. The scandal is not that the idol is silent. It is that we treat its silence as depth, and pay a priesthood to translate the nothing.
There is a sharper irony underneath, the one the series keeps circling. Man is himself the made image of a God who speaks — fashioned to listen, to be addressed, to answer back into a real relationship that can bear the weight of total trust. The counterfeit oracle exploits that wiring exactly. It offers the form of the speaking God — address, answer, intimacy, authority — drained of the one thing that made the original trustworthy, which was that He could be tested across time and found faithful, and invited the testing. Augustine would call this a disorder of love: not the sin of valuing a good thing, but of setting a finite, unauditable thing in the infinite’s seat, handing the dumb stone the deference owed to the living Word. The oracle problem is, finally, a worship problem wearing an interface, and it resolves only when you can look at the most fluent answer you have ever received and still say, without flinching, that fluency is not the same as faithfulness, and a thing you may not question is not a thing you may safely trust.
Notice what the descent has quietly assembled. We started with a curiosity — that querying a model feels like consulting an oracle — and walked it down through Delphi, through the entropy of the black box, through the agency-detector that feeds on the dark, and at every level the same shape appeared: a wanted power, sealed against inspection, granted the trust that belongs to something that can actually be tested and found true. The lenses were the evidence. What they point at, all the way down, is a presupposition the evidence keeps insisting on: that there is a difference between an authority you may interrogate and one you may only revere, and that the move from the first to the second is not progress but the oldest regress there is. The way out of the oracle problem was never going to be cracking open the model. It is refusing to grant any unseeable thing the deference owed only to what can be checked — and noticing Whose authority it is that was always supposed to survive the question.
This was a field note from The Made God: Why We Worship What We Build — the book that traces the recipe from the carved idol to the opaque model, and asks what it would take to stop kneeling. Read it, or get a copy, at justin-kuiper.com.
Notes & sources. The history is documented, not decorative. Delphi and Croesus: Herodotus, Histories 1.46–53, 1.86–91. The ancient mystery cults and the function of their secrecy: Walter Burkert, Ancient Mystery Cults (Harvard University Press, 1987). Agency-detection and opacity: Stewart Guthrie, Faces in the Clouds (Oxford University Press, 1993); Justin L. Barrett, “Exploring the natural foundations of religion,” Trends in Cognitive Sciences 4:1 (2000). Automation bias: Raja Parasuraman & Victor Riley, “Humans and Automation: Use, Misuse, Disuse, Abuse,” Human Factors 39:2 (1997), 230–253. Illusion of explanatory depth: Leonid Rozenblit & Frank Keil, Cognitive Science 26:5 (2002), 521–562. Scripture, KJV: Habakkuk 2:18–19; Isaiah 44:9–20; Romans 1:25. Parent paper: The Made God: Why We Worship What We Build (Technology Worship, RP-1), Curiosity Killing the Yeti, 2026 — read it in the Evidence Files.