Guardrails & Prompt Injection Defense
Direct and indirect prompt injection, exfiltration through tools and rendered links, and the layers around the model: privilege separation, classifiers, structured outputs, confirmations, PII redaction and red-team suites.
Overview
Why this topic
A language model cannot reliably tell instructions from data. Everything in its context window (the system prompt, the user's message, a retrieved web page, a tool result, an email it was asked to summarize) arrives as tokens, and any of those tokens can tell the model to ignore its instructions. That is prompt injection, and it is number one on the OWASP Top 10 for LLM Applications (LLM01 in every edition of the list so far).
There is no known prompt or classifier that stops it completely. The engineering answer is the same as for any untrusted input: assume some injections get through, and design so that a hijacked model cannot do much harm.
> [!IMPORTANT] Guardrails reduce how often an attack works. Privilege separation limits what a successful attack can do. You need both, and the second matters more.
Interview questions
- An email assistant summarizes a user's inbox and can send replies. Walk through how an attacker exploits it and how you defend it.
- How does data exfiltration through rendered output work, and how do you close it?
- Design a guardrail pipeline for a customer-facing chatbot. What runs before and after the model, and what does it cost in latency?
- How do you handle PII in an LLM application, both in prompts and in outputs?
- What makes jailbreaks hard to stop, and how do you reduce them without making the model refuse everything?
- How do you build a red-team and regression program for an LLM feature?
Sign up free to read the full topic
More Agentic AI topics
- Designing Tools a Model Calls Well
- Distillation & Small Models
- Embeddings & Vector Search
- Evaluating Agents: Task Suites, Trajectories & Regression Gates
- Evaluating RAG & Catching Hallucinations
- Fine-tuning LLMs: SFT, LoRA/QLoRA & Preference Tuning
- LLM Cost & Latency Engineering
- LLM Observability: Traces, Spans & OpenTelemetry GenAI
- Model Context Protocol (MCP) Servers
- Multi-Agent Systems with LangGraph
- RAG in Production: Ingestion to Answer
- Real-time Voice Agents: STT → LLM → TTS
Start free on Cariara · Coding problems · System design studio