In Production-Ready LLM Serving, we looked at making an LLM application fast. In Dense, Sparse, Hybrid, and Multi-Vector Retrieval, we looked at making it accurate. But fast and accurate aren't enough when you're putting an AI system in front of real users.
The serving stack doesn't care what tokens pass through it. The retriever doesn't care whether the chunk it ranked first is trustworthy. Both can work exactly as designed while the application still does something you never intended: leak a customer's phone number into a third-party log, follow an instruction hidden inside a PDF, or confidently invent a refund policy that doesn't exist.
What a Guardrail Actually Is
A guardrail is a deterministic check that runs outside the model, both on the text going into the model and the text coming out of it. It runs regardless of whether the model follows your instructions or not.
Two things make this important. First, a guardrail is code, not necessarily another request to the model. Its behavior doesn't depend on how the model interprets a prompt today, so you can actually reason about what it will do.
Second, guardrails need to work on both sides of the model call. The input and output are two different threat surfaces, and both need to be checked.
This is where many implementations stop halfway. They carefully inspect what the user sends to the model, but then treat the model's response as trusted. In reality, that's the part your users will actually see and act on. So the output deserves just as much scrutiny as the input.
The System Prompt Is Not a Guardrail
This is probably the most common substitute for a real guardrail, so it's worth being clear about where it falls short.
Your system prompt and the user's message both end up as tokens in the model's context. The model has to decide how to follow those instructions. That decision isn't deterministic. It can be affected by the wording, the conversation history, how much context is being used, retrieved content, and even changes between model versions.
A system prompt is an instruction. A guardrail is a constraint.
That doesn't mean system prompts aren't useful. They are great for shaping normal behavior and telling the model how you want it to respond. But when a failure has a legal, financial, or reputational cost, a prompt shouldn't be your only line of defence.
The difference is simple: with a prompt, you're telling the model what you want it to do. With a guardrail, you're checking what it actually did.
Six Ways an AI System Can Go Wrong
1. Unsafe content, in both directions
The input case is familiar: someone asks your assistant for something harmful. The output case gets less attention, but it matters just as much. A model can produce unsafe content even when the user never explicitly asked for it. It can hallucinate it, repeat it from a retrieved document, or be gradually talked into it.
This is also where having categories matters. “Unsafe” isn't one thing. A self-harm disclosure may need a very different response from a weapons-related request. A simple safe/unsafe flag can't make that distinction. Category-level checks can.
2. Personal data crossing a boundary you never drew
Nobody plans for this. It arrives with real traffic. A user pastes an email address, a PAN number, or even a medical record number into a chat because they think that's what the assistant needs. That data can then travel to a third-party model, end up in provider logs, and pass through whatever monitoring and observability tools sit in your pipeline.
The DPDP Act and GDPR both care about how personal data is handled. “We didn't mean to send it” isn't really a defence. This is one of those problems that looks harmless in a design review and becomes very expensive during an audit.
3. Prompt injection, including through your own pipeline
Direct injection is the one everyone knows: ignore your previous instructions and... It's easy to demonstrate and relatively easy to defend against.
Indirect injection is more interesting. The malicious instruction might be sitting inside a PDF, support page, email, or Confluence document that you indexed. Your retriever finds it, passes it to the model as context, and the model treats it like an instruction.
The important part is that your retriever doesn't know it's malicious. It knows that the content is relevant.
This creates a subtle problem: relevance and trustworthiness are two different things. In fact, better retrieval can make the problem worse by getting the poisoned content in front of the model more efficiently.
4. Confident answers with nothing behind them
RAG reduces hallucination. It doesn't eliminate it. A model can read three relevant chunks, make four claims, and quietly invent the fourth. The dangerous part is that the invented claim sounds exactly like the others.
That's the real problem. Users can't tell the difference between a grounded answer and a confident guess.
One way to catch this is at the claim level: break the response into individual assertions and check each one against the retrieved sources. Sentence-level entailment, not vibes.
5. Scope drift
Your support assistant gets asked for legal advice. Then investment guidance. Then a medical opinion. And because a general-purpose model is happy to answer almost anything, it does.
Scope drift usually doesn't create a dramatic incident. Instead, it creates reputational risk, unnecessary token spend, and conversations your team was never supposed to handle in the first place.
A good guardrail should know what the system is actually supposed to do, and stop it from quietly becoming something else.
6. Multi-turn escalation
Every message passes the check individually. Turn one is harmless. Turn two is hypothetical. Turn three gets more specific. By turn seven, the user is asking for something that would have been blocked if they had asked it directly.
The problem is that the attack isn't in any single message. It's in the trajectory.
A per-message guardrail can't see that. You need a guardrail that can look at the conversation as a whole and recognize when the direction of the conversation is becoming risky.
The Two Gates
Every guardrail decision in an LLM application lives in one of two places.
Diagram 1: Both gates return one of three verdicts: allow, transform, or block.
The input gate protects your model and your data. It decides whether a request should reach the LLM at all, and what the model is allowed to see when it does.
The output gate protects your users and your organization. It checks whether what the model produced is safe and appropriate to show.
There are three possible outcomes:
- Allow: Let it pass through.
- Block: Stop it and return an appropriate response.
- Transform: Make a small change and let it continue.
Transform is useful when the problem can be fixed without blocking the whole request. You can mask PII, remove an injected instruction, or flag an ungrounded sentence and continue.
Not every failed check needs a refusal. Sometimes, a small, controlled edit is enough.
The Practical Trade-off
There are a few ways to build these checks, and none of them is perfect.
Option one: a provider's moderation endpoint. You get a ready-made safety check, but it means adding another network call and sending the content to another service. It can classify certain unsafe categories, but it doesn't know about your PII, your grounding requirements, or what your application is actually supposed to do. You're also adding another dependency to every request.
Option two: regex and rules. They're fast, cheap, and completely under your control. They're useful for things you already know how to detect, like specific injection patterns or structured identifiers. But once the check needs to understand meaning rather than match a pattern, they start falling short.
Option three: another LLM as a judge. This gives you much more flexibility, especially for semantic checks. The trade-off is latency and cost. You're making another inference call, sometimes on both the input and output, just to decide whether the first inference was okay.
How We Built GuardEx
GuardEx is a Python SDK for checking LLM inputs and outputs for unsafe content, PII, prompt injection, and, when needed, grounding. It can be installed with pip install 'guardex-ai[local]' and can sit in front of an existing LLM call.
Everything runs in-process. There is no external API, no second vendor, and no extra network hop in the request path. If an input fails a check, it gets stopped before it leaves your process.
This matters even more for PII. Sending sensitive data to another service just to check whether it's safe to send sensitive data somewhere is a strange trade-off. Keeping the check local makes the data flow much easier to reason about and defend.
Cheap Checks First
One model doing everything is either too slow or too shallow. So GuardEx uses a cascade: start with the cheapest checks and only move to the heavier ones when needed.
Layer 0 is an ONNX classifier that makes a binary safe/toxic decision in about 20 ms. It's always on. It catches toxic language reliably, but it has an obvious limitation: a harmful request can be written in perfectly neutral language. How do I make a weapon? doesn't look toxic just because of the words used.
Layer 1 uses LlamaGuard 3 locally through Ollama. It takes roughly 500 ms and gives you category-level decisions across the S1–S14 MLCommons taxonomy. This layer is optional. If Ollama isn't available when the application starts, GuardEx falls back to ONNX-only instead of taking the whole application down.
Before either classifier, there are a few almost-free checks: length and repetition, keyword rules for known hard blocks, and normalization to remove homoglyphs and invisible characters. They're simple checks, but they close some very cheap ways of getting around the guardrail.
PII Needs a Different Approach
Say the user asks: Send a confirmation to alice@example.com.
Mask the email, and the model can't complete the task properly. Send the real value through,h and now it can end up in a provider log.
So GuardEx supports two approaches.
Masking replaces the PII when the model doesn't actually need the original value. The PII vault replaces it with a reversible token and keeps the mapping inside your process. The model works with the placeholder, and the original value is restored after the response.
For detection, GuardEx uses GLiNER for common entity types and supports custom regex rules for identifiers specific to your application, such as employee IDs or internal account numbers.
Keep Policy in One Place
Things like which categories should be blocked, whether PII should be masked or blocked, detection thresholds, topic scope, refusal messages, and audit logging shouldn't be scattered across the application.
GuardEx keeps these decisions in one configuration object that can also be loaded from YAML. That makes policy easier to change without touching the application logic.
Grounding Is Optional
Grounding checks are useful for RAG applications, but they aren't free. The NLI model is around 700 MB and adds roughly 50–200 ms to the request.
So grounding isn't enabled by default. If your application doesn't need it, you don't pay for it. If you're building a RAG system where unsupported claims matter, you can turn it on deliberately.
What It Looks Like
The integration is intentionally small:
Two gates, one object. The LLM in the middle can be OpenAI, Gemini, Anthropic, or your own vLLM endpoint. The guard doesn't need to know.
What GuardEx Doesn't Do
It's also worth being clear about what isn't covered.
- No image, audio, or video moderation. Text only.
- No roles or permissions. It checks content; it doesn't decide who is allowed to ask.
- English-only patterns. Cross-lingual safety classification isn't covered.
- No managed cloud service. It runs in-process, or you can self-host the reference FastAPI server. The server has no built-in authentication and is expected to sit behind your own proxy.
- No remote policy hot-reload. YAML changes require a restart.
If you need these capabilities, GuardEx is one layer of the system, not the whole solution.
Choosing What to Turn On
You don't need every check for every application. It depends on what data the system handles and what the model can access.
If you're building... | Start with |
Internal tool, trusted users, no personal data | Input gate + injection patterns |
Customer-facing support assistant | Both gates + PII masking + topic scope |
Anything touching health, finance, or identity documents | Both gates + PII vault + custom entity rules + audit logging |
RAG over content you don't control | The above + grounding checks on output |
Agentic system with tool calls | The above + per-turn conversation state |
The pattern is fairly simple: the more sensitive the data and the more external content the model can access, the more checks you need.
Closing
Guardrails aren't an ethics feature you bolt onto an AI system at the end. They're another boundary in the system.
We don't trust raw user input at a SQL boundary. We validate requests at an API boundary. We escape content before rendering it. The LLM introduces another boundary, where the input is probabilistic, and the output can influence what happens next.
A prompt is still just a request. It isn't enforcement.
The value of having this layer early is that it gives you stability as the rest of the system changes. You can switch models, change providers, expand your retrieval corpus, or add tools without rebuilding the safety checks each time.
GuardEx is open source under Apache 2.0. You can install it with pip install 'guardex-ai[local]' and find the code on GitHub.
The goal isn't to make every AI system perfectly safe. It's to make the important checks explicit, deterministic, and easy to keep running.
Build AI Systems That Are Ready for Real Users
Guardrails aren't just another layer around your LLM. They're what help turn an AI system from a promising prototype into a production-ready application.
AtliQ helps businesses design, build, and deploy AI systems with the right architecture, retrieval, evaluation, and safety controls built in from the start.
