Fine-tuning LLMs: SFT, LoRA/QLoRA & Preference Tuning

Overview

Why this topic

Most teams reach for fine-tuning too early. A better prompt or a retrieval step fixes most quality problems for a fraction of the cost. Fine-tuning earns its place when the model must learn a behavior (a format, a tone, a tool-calling style, a domain's way of reasoning) that it cannot pick up from a few examples in the prompt, or when a small tuned model can replace a large general one and cut cost and latency.

The interview questions follow the work: should we fine-tune at all, what data, which method, how much GPU memory, how do we know it got better, and how do we serve it.

Interview questions

• A product team wants to fine-tune a model so it answers in the company support style and uses internal product names. Do you agree, and what would you do first?
• How do you build and format a fine-tuning dataset from production logs?
• Explain the LoRA math. How do rank, alpha and target modules change parameter count and quality?
• You have one 24 GB GPU and want to fine-tune an 8B model. Walk through the memory budget and the method you pick.
• Walk through SFT followed by DPO. What does each stage fix, and how do you build the preference data?
• How do you evaluate a fine-tuned model before shipping, and how do you serve several fine-tunes cheaply?

More Agentic AI topics

• Context Window Management for Persistent Agents
• Designing Tools a Model Calls Well
• Distillation & Small Models
• Embeddings & Vector Search
• Evaluating Agents: Task Suites, Trajectories & Regression Gates
• Evaluating RAG & Catching Hallucinations
• Guardrails & Prompt Injection Defense
• LLM Cost & Latency Engineering
• LLM Observability: Traces, Spans & OpenTelemetry GenAI
• Model Context Protocol (MCP) Servers
• Multi-Agent Systems with LangGraph
• RAG in Production: Ingestion to Answer