METAHEURISTIC
How to build AI LLM guardrails? I’d take a comprehensive approach: run checks on the input before the model, on the output after it, on the input and output together, and on the chat history. Decide per security policy whether a flagged request passes, gets a warning or gets blocked. Detect with layers of regex rules, trained classifiers and an LLM judge, with a separate policy engine making the final call. All of that is application dependent. It depends on your domain, the type of traffic you have and how exposed your application is in production.
The scenario I’m working through: you’re in a company that wants to build an AI gateway layer behind some sort of chat interface, something similar to OpenRouter. They only have regex-based guardrails, and you want something more advanced. Here is how I’d sketch the solution, with the trade-offs for each step.
How do you build AI LLM guardrails? The short answer
Where should guardrails sit in the request flow?
There’s a trade-off here on how much you trust the input, and on your security restrictions and policies. Generally, you can run guardrail checks on the inputs and you can run checks on the output. For the input, it’s to make sure there are no jailbreaks or prompt injection, and all the other vectors of attack. On the output, it’s to make sure you don’t expose any PII or any information that might be sensitive to your domain, application or database.
You can also scan the input and output together as a pair, to get broader context over what’s going on. And there’s another layer of guardrails: scanning in the context of the chat history. In a small context, a request might look like a pretty normal message in a chat. In the broad picture, you can spot vectors of attack, especially if your context also pulls in messages from the history.
What about images, audio and files?
Take into account that you’re not only dealing with traditional text generation models. You might also run a multimodal application, where inputs come as text and also as audio, images or any file format. You’d very likely normalize those to some textual or feature representation that goes to the LLM. Nevertheless, an attacker can embed attacks in those files too. On an image, they can just print some characters that form a prompt attack, or they can embed a metadata header in a PNG or JPEG to run a prompt injection. The same applies to PDF files and similar formats.
What should happen when a guardrail flags something?
That also depends on the security policy and the application. I think we can do all of the above, and also keep the logs for further calibration and improvement over time. So basically we can have three different modes.
I’ve never done editing of the responses, and I don’t think it works well in general. I’ve seen some applications where it makes sense. For instance, you might be concerned about exposing sensitive information from your development team’s environment, credentials or API keys, and the guardrail application is supposed to prevent exactly those attacks. For those cases, I agree you can redact. Still, I’d rather take the radical approach and just block those requests, since they’re trying to expose information. The user sending that to the chat is probably not using the application or the LLM as intended.
Redaction is a policy Editing responses is possible, but generic guardrail interfaces usually don’t do it unless you implement it as a policy. When something tries to pull out credentials, I block.
How do you evaluate whether guardrails are working?
Since I’m describing a multi-layer architecture for guardrails, with different levels of checks, we should have a separate dataset per layer: a golden evaluation set. What goes into it always depends on the implementation details. You might use rule-based guardrails, a classifier trained on an encoder transformer, or a more classical ML classifier to reach your latency goals or SLA. Either way, I’d keep a golden dataset of the things I want each of those three layers to catch.
I’d also measure the full path end to end, to make sure we’re not exposed to attacks. And I’d keep tracking and collecting the long tail of what we see in production, obviously with respect for the privacy of the customer. I’d also test the classifier on data it was never exposed to during training. I covered how a golden set catches regressions in a retrieval pipeline in RAG beyond the tutorial. Here it’s one set per layer plus the end-to-end run.
Three more things belong in the evaluation. First, be explicit about false positives versus false negatives. Blocking too much can make the product unusable, so you report both, per layer. Second, adversarial red teaming that evolves over time, because static test sets get stale fast and attackers adapt. Third, track the latency impact of each layer. That number decides whether you can afford a check in the hot path or whether it has to run async.
Regex, classifier or LLM judge: which detection should you use?
Let’s evaluate all of these approaches. They trade simplicity and latency against accuracy and resilience.
Regular expression rules
Regex rules are simple, and they scale roughly linearly with the number of rules you add. That still requires some performance optimization, because some regexes are expensive to run and carry overhead from backtracking and other factors. They give you a surface of protection. I wouldn’t call it ideal. It’s a trade-off between simplicity and overall resilience, and it’s probably the layer you can add fairly quickly.
The backtracking cost depends on the engine. Engines like RE2 guarantee time linear in the
input length for a given pattern, while backtracking engines can hit catastrophic,
exponential blowups. So I’d use RE2 or the Rust regex crate, and I wouldn’t assume regex
is always the weak layer. For structured secrets and known signatures, regex is probably
better than a probabilistic classifier.
A trained classifier
The next layer is a classifier. You can label data and train a binary classifier, whether it’s an encoder or encoder-decoder transformer, something else, or even a naive Bayes classifier, to decide whether a message is, say, a prompt injection. This takes time to train, and there’s another penalty in latency to run the classifier. The encoder option is the one I wrote about in BERT’s revenge.
Where it runs matters. I’d design the guardrails so they don’t run as part of the router. A router is an IO-heavy application with many connections, threads or coroutines, whatever your technology uses. You might build it in Go, Rust, JavaScript or Python, that’s up to you, but embedding a CPU-heavy classifier into that service isn’t a wise decision. It introduces CPU spikes that would totally mess up your IO, even with threads. So that’s a good reason to move the classifier to a separate service. You pay extra latency for the network hop and extra latency for running the classifier itself.
It isn’t mandatory in every case, though. For small models, in-process inference can be fine if you isolate execution from the event loop. For transformer inference, a dedicated service makes sense for batching, GPU use and multi-replica setups.
Keep heavy inference out of the router The router is IO-bound. A CPU-heavy model inside it causes spikes that stall every other connection. Small models can stay in-process off the event loop; transformers get their own service.
I think a classifier gives you much better accuracy and precision than regex rules. It still requires constant maintenance, updates and retraining, which might be expensive or might not, depending on the technology and approach. Or it could be an ensemble classifier that you gradually add to for certain use cases. Instead of retraining the same classifier over and over, you add one slim model and aggregate the signals from all of them.
The hard part of an ensemble is arbitrating disagreement. If one model flags prompt injection at 92% and another says 15%, what does the policy engine do? Calibration across models is hard. You need either a meta classifier on top or well calibrated per-model thresholds, otherwise your policies become really brittle.
An LLM as a judge
The last layer is an LLM judge. Its advantage is that it can spot things like attacks in other languages. Regex and classifier solutions are exposed to multi-language attacks. If I write a prompt injection in Ukrainian, that’s a blind spot for any classifier that was never trained on Ukrainian or didn’t have enough exposure to it. The same applies to regex rules, since they’re usually written as patterns in the most common languages. And you don’t know which language an attacker will choose.
There’s another option: you can prohibit using the application in languages that aren’t spoken by the primary majority of your customers. That’s fine as well, it’s also a security policy, and it’s easy to detect from the text in question, from certain words, or from the Unicode range of the characters.
The LLM judge also exposes a surface of attack. The judge is another place where prompt injection can happen, and it can fall for the injection itself and let the request bypass. Structured outputs and a strict separation between instructions and untrusted content help, but they’re not a silver bullet. Generally the judge gives you better confidence in the results, but its latency is all over the place, up to a second per response if you use a very high parameter model. The judge, and to a lesser degree the transformer classifier, can also be much more expensive for your budget. I compared a judge with a cheaper setup on a real client in quality gates without LLM-as-a-judge.
Language is a policy too Prohibiting languages outside your primary customer base is a valid security policy, and it’s easy to detect from words or the Unicode range of the text.
Which classifier models should you use for guardrails?
Here are the options I’d consider for the classifier layer, from the cheapest baseline to the most expensive encoder, with the trade-offs that decide between them.
I wouldn’t trust TF-IDF with logistic regression alone. And rather than standing up five transformers in an ensemble, I’d benchmark three things: a cheap baseline like TF-IDF plus logistic regression, a compact encoder like MiniLM, and an LLM judge. Then measure, and not just accuracy. What you want is the incremental lift each layer adds over the one before it.
How do you handle a new attack type?
Six months in, a new attack type shows up. Do you add another specialized classifier or retrain the general one? I’d go hybrid. A general multi-head classifier is more efficient at inference, because you run one encoder instead of three separate transformers. Specialized classifiers give you modular updates without retraining everything.
So I’d start with a shared encoder and a multi-head setup, and keep the option to bolt on a specialist when a new threat appears. Once the threat is stable and there’s enough data, fold it back into the general model on the next training cycle. That hedges between agility and efficiency.
What about attacks that look benign in every category?
A multi-head design has a gap: an attack can be benign under each single category and malicious in combination. I’d handle it in two ways. First, a shared encoder with multi-label outputs. Second, a policy layer that looks at combinations of signals. If none of the heads fires high enough on its own, but several are moderately suspicious, the policy can trigger a higher risk treatment.
Don’t let the models make access decisions. They emit signals, and a policy engine decides.
Models emit, policy decides Per-head thresholds miss an attack spread thin across categories. A policy engine that reads all the scores together can escalate it, and can apply different rules per customer.
How do you keep guardrails under 15 ms at 10,000 requests per second?
Say the gateway handles 10,000 requests per second and you want under 15 milliseconds added at P99. That target also answers which requests hit which classifiers without turning the pipeline into a latency and cost bomb. I’d split it into three tiers.
At 10,000 requests per second, GPU batching in tier 2 can be cheaper than CPU, but watch the queueing. Keep batch windows tiny and use dynamic batching. For tier 3, don’t count on an external LLM staying under 50 milliseconds on the hot path. Beside the tiers, add a separate policy engine that takes the signals and applies per-customer rules.
For metrics, I’d track the added P99 latency per layer, cost per million requests, attack success rate on held-out sets, recall at a fixed false positive rate, and queue depth. And before building any of it, I’d estimate what percent of your traffic really needs a transformer at all. That one number could swing your cost by 10x.
How do you monitor LLM guardrails in production?
For production, I think in five threads, plus a cadence that keeps them running.
Observability first. Trace every request across the whole path: gateway, regex, transformer, policy engine and the LLM output, and attach the policy decisions to the trace. Don’t store raw prompts by default.
Security monitoring means tracking attack success, not just blocks. If your block rate drops from 8% to 2%, maybe attacks stopped, or maybe your detector broke. You can’t tell from the block rate alone, so you also monitor distribution shifts in language mix, prompt length and attack types.
On failure modes, decide fail open versus fail closed per policy. If the transformer service is down, do you allow with logging, or do you block? A public chatbot might lean toward fail open. An AI agent with production credentials should fail closed.
The improvement loop samples and labels production traffic with privacy controls, feeds it back into the eval sets, retrains, then goes through shadow, canary and ramp-up. Don’t train on every flagged request. Attackers will poison you.
Don't train on every flag Flagged traffic is partly written by attackers. Feed it into retraining unfiltered and they get to shape your detector.
How do you detect an attack that actually succeeded?
Separate detection from confirmation. For live detection, run your usual classifiers and add conversation-level signals. If a request gets a high prompt injection score and you block it, that’s not a successful attack. It’s a prevented attempt.
To spot actual success, you need ground truth signals: tool call logs showing that a prohibited action was attempted, or secrets detected in the output.
What if the customer says no data, no training?
There’s a tension between customer privacy and the need to collect production examples for retraining. If privacy prevents storing prompts, keep derived signals only: scores, decision types, language, model version and latency, and not the content itself. That lets you alert in real time without retaining sensitive text. For retraining without data retention, lean on synthetic adversarial data.
So, how do you build AI LLM guardrails?
Check the input for jailbreaks and prompt injection, the output for PII and anything sensitive to your domain, the pair together, and the chat history, including whatever comes in through images, audio and files. Let each policy choose pass, warn or block, and keep redaction as an explicit policy. Detect in tiers: linear-time regex and a cheap baseline in the gateway, a compact encoder like MiniLM in its own service, and an LLM judge only for the small slice of traffic that needs it, with a policy engine combining the signals per customer. Keep a golden set per layer, report false positives and false negatives, red team continuously, and in production track attack success, drift and fail-open rules per policy. If you’re building this into a gateway and want a second pair of eyes on the design, that’s the kind of work a fractional AI CTO does.
