RAG in Production: Ingestion to Answer

Overview

Why this topic

A RAG demo is twenty lines: embed some text, store the vectors, take the top 5 and paste them into a prompt. A RAG system in production is a pipeline of about ten stages, and almost every bad answer traces back to one stage upstream of the model: a PDF table parsed into noise, a chunk cut in the middle of a step list, a keyword query that vector search could not match, or a document the user was never allowed to read.

The interview question is rarely what RAG is. It is a walk through your pipeline: where it breaks, and how you know.

Interview questions

• Walk me through a production RAG pipeline end to end. What does each stage do and what fails there?
• How do you choose a chunking strategy and chunk size? Show how you would test it.
• Why do you need hybrid search and RRF? When does pure vector search fail?
• How do you enforce per-document access control in RAG?
• Documents change every day. How do you keep the index fresh without re-embedding everything?
• Your RAG answers take 4 seconds to start. Break down the latency and tell me what you would cut.

More Agentic AI topics

• Fine-tuning LLMs: SFT, LoRA/QLoRA & Preference Tuning
• Guardrails & Prompt Injection Defense
• LLM Cost & Latency Engineering
• LLM Observability: Traces, Spans & OpenTelemetry GenAI
• Model Context Protocol (MCP) Servers
• Multi-Agent Systems with LangGraph
• Real-time Voice Agents: STT → LLM → TTS