NPC Dialogue System

LLM-Driven Game Characters

Design a service that lets game characters hold open conversations with players: in character, aware of the game world, remembering past meetings, fast enough to feel live, and never able to change game state on their own.

Overview

In most games, a non-player character (NPC) speaks from a dialogue tree: designers write every line and every branch. Players exhaust it quickly, and the character cannot answer anything the writers did not foresee. A large language model (LLM) can answer anything, in the character's voice, and can mention what the player did last week. This problem is the service that makes that work inside a game.

A game is harder than a chat app in four ways. Speed: a character who stays silent for three seconds feels broken, so replies must start almost at once. Consistency: a medieval blacksmith must not talk like a chatbot or know about events far away. Memory: the character should remember the player, but a model only knows what is in its prompt. Control: the model must never be able to hand out gold or finish a quest just because a player asked cleverly; the game server owns game state.

Questions to ask first

Text, voice, or both? — Text first; voice adds speech-to-text before and text-to-speech after
How many talk at once? — 100,000 conversations at peak, a player line every 20 seconds
How fast must a reply start? — First words on screen within 1 second (our target)
What should a character remember? — Facts and events with each player, across sessions
Can a character act in the game? — It may propose actions from a fixed list; the game server decides
Who writes characters? — Game designers, through a tool, without writing prompts by hand

What you will be able to do

Set a latency budget for one line of dialogue and meet it with streaming and prompt caching.
Build the prompt for each line from persona, knowledge limits, game state, memory and recent turns.
Store and retrieve a character's memories of a player, and explain the scoring.
Keep the model from changing game state: structured actions, validated by the game server.
Degrade gracefully when the model is slow, unsafe or down.

Key concepts

Latency budget
Streaming tokens
Prompt assembly
Prompt caching
Memory retrieval
Knowledge boundaries
Structured game actions
Output guardrails
Graceful fallback

Functional requirements

A player sends a line to a character and sees the reply appear as it is generated
Each character speaks from a persona (voice, backstory, what it knows and does not know) written by designers
Characters remember facts and events with each player across sessions
Replies reflect current game state: location, time of day, active quests, the player's standing with the character
Characters may propose game actions (offer a quest, open the shop, refuse to talk) from an allowed list; the game server applies or rejects them
Unsafe or out-of-character replies are blocked and replaced
When the model is slow or unavailable, the character answers with a written fallback line

Non-functional requirements

First words on screen within 1 second at p95, full reply within about 3 seconds (targets we chose)
The model can never change game state directly
No player can read another player's memories or conversation
The game keeps running if the dialogue service fails
Cost per line tracked and bounded; prompts kept short

Interview questions

How do you make a reply feel instant when the model takes seconds?
How does a character remember a player across sessions?
How do you stop the character from knowing things it should not?
Can the character give the player items or quests?
How many model calls and tokens do you need to plan for?
What happens when the model provider fails?
How do you keep characters in character over time?