The Heresy Guard: Building a Hallucination-Resistant AI

If you know anything about modern Artificial Intelligence, you know it has a fundamental flaw: it tends to "hallucinate." When an AI doesn't know an answer, it confidently invents one.

In my day job as a Data Analytics Engineer, a hallucination is a bug. It means a dataset is broken or a dashboard is misleading. But in theology, a hallucination isn't just a bug—it's heresy.

When I set out to build The Depositum, an AI-generated Catholic theology podcast, this was the primary engineering hurdle. How do you take a technology inherently prone to making things up and use it to faithfully transmit the 2,000-year-old unchangeable Truths of the Catholic Church?

The answer: You build a "Restricted Brain" and guard it with a rigorous vetting pipeline. Here is exactly how we engineered our hallucination-resistant "Heresy Guard."

Part 1: Source Restriction & The "Restricted Brain"

Unlike standard AI chatbots (like ChatGPT or Claude), our digital theologians are strictly forbidden from consulting the open internet. The open web is fraught with modernism, subjective interpretations, and theological error.

Instead, their entire "worldview" is physically limited to a closed environment of uploaded Markdown files. We lock the AI in a digital room with the Saints. We call these our Three Pillars:

1. Scripture: The Douay-Rheims Bible

2. Dogma: The Catechism of the Council of Trent

3. Tradition: The Haydock Bible Commentary

If a concept isn't in those specific public domain documents, the AI cannot speak on it. We use Google's NotebookLM as the foundational synthesizer for these texts specifically because of its strict grounding feature: it is designed to solely reference the sources you add and absolutely no others.

Now, full transparency: the underlying AI model of Google NotebookLM was still originally trained on a massive swath of the open internet. Occasionally, it will try to regress and hallucinate based on its original training, bringing in outside biases or modern assumptions. However, because we force it to retrieve from our files, those regressions are minimized. And when they do happen, they are usually caught by the safety nets we built next.

Part 2: The Three-Step Vetting Protocol

Trust is the currency of any theological endeavor. Restricting the sources is step one, but before a 5-minute episode of The Depositum ever reaches your ears, the audio must survive our rigorous Three-Step Vetting Protocol.

1. The Verbatim Protocol (Source Restriction) Our AI utilizes two distinct personas: Host 1 (The Lector) and Host 2 (The Theologian). Host 1 is programmed with a strict mandate: it does not paraphrase Scripture. Through explicit file-path anchoring (e.g., retrieving Bible_Book_49_Luke.md), it must read the exact verses from our closed Douay-Rheims files verbatim, without interruption.

2. The Orthodoxy Filter We produce the episode first. Then, right before publishing, the output runs through a secondary QC AI agent. This agent acts as a theological auditor. Its sole purpose is to compare the episode against traditional Catholic sources for consistency, flagging any errors, modernisms, or deviations from traditional Catholic teaching before the audio is finally approved. The results of these tests are publicly available and can be found in this google doc.

3. Human Verification AI generates the script and the audio, but a human must approve it. I personally listen to every single second of every episode before it goes live. I want to be clear: I am a Catholic layperson. I have no formal theological training beyond Catholic primary, middle, and high school, Sunday sermons, and light reading. While I listen for general consistency with my understanding of the Faith, I don't pretend to be the ultimate authority on theological accuracy. My primary job in this step is validating the execution. I am checking that the AI hosts read the Biblical text verbatim, ensuring the storytelling carries the appropriate emotional weight, and confirming there are no audio quirks or glitches.

By combining a restricted, pre-1926 public domain dataset with an adversarial auditing pipeline, we've created a tool that does the heavy lifting of historical research without compromising the Depositum Fidei.