AI Tutoring System

Adaptive Practice with an AI Tutor

Design a practice platform that tracks what each student knows skill by skill, picks the next problem to fit, schedules reviews before skills are forgotten, and offers an AI tutor that gives hints without handing over the answer.

Overview

An AI tutoring system is a practice app with two brains. The first is a student model: for every student and every skill in the curriculum it keeps an estimate of how likely the student is to know that skill. It updates after every answer and decides what to practice next and when to review. The second is a tutor: a chat, backed by a large language model (LLM), that helps a stuck student with a hint, a question back, or an explanation of a mistake.

The student model is classic, cheap and exact: a few numbers per skill and a Bayes update per answer. The tutor is slow and costly, and it has one rule that is easy to break: it must help the student think, not give the answer. Khan Academy described its own experience with Khanmigo in 2026: for students who engaged, the Socratic tutor worked as designed, but many students did not know when to ask for help, and the first version did not change learning as much as hoped. Its fix was to put the tutor inside practice problems and have it prompt students to explain their answers. That lesson shapes this design: the tutor is part of the practice flow, not a separate chat page.

Questions to ask first

Who are the students? — School students, many under 13, so children's privacy rules apply
Which subjects? — Math first: answers can be checked exactly
How many students? — 5 million with accounts, 1 million practicing at the school-day peak
Who writes the problems? — Curriculum staff; each problem has a worked solution and a skill tag
Does the tutor grade answers? — No: a checker grades; the tutor only helps
Do teachers need a view? — Yes: class mastery by skill, and flagged tutor conversations

What you will be able to do

Explain Bayesian Knowledge Tracing and update a student's mastery after an answer.
Pick the next problem from a prerequisite graph and a target success rate.
Design a tutor that gives graded hints and never leaks the answer, with checks in code, not only in the prompt.
Schedule spaced reviews from a forgetting curve, and size the whole system with numbers.

Key concepts

Knowledge tracing (BKT)
Deep knowledge tracing
Prerequisite graph
Target success rate
Hint ladder
Answer leak guard
Spaced repetition
Half-life regression
Idempotent answers
COPPA

Functional requirements

Serve a practice problem, check the answer, and return the result with the next problem
Keep a mastery estimate per student per skill, updated after every answer
Offer only skills whose prerequisites the student has mastered
Let a stuck student ask the tutor; the tutor replies with a hint, a question or an explanation of the mistake, never the final answer
Schedule reviews of learned skills before they are likely to be forgotten
Show teachers class mastery by skill and let them read tutor conversations

Non-functional requirements

Answer path (check, update, next problem) under 200 ms at p99 (a target we chose)
First tutor words within 2 seconds, streamed (a target we chose)
An answer is counted exactly once, even when the app retries
The answer path keeps working when the LLM provider is slow or down
Children's data handled under COPPA: verifiable parental consent for under-13 users, no tutor transcripts used for ads

Interview questions

How does knowledge tracing work, and why not just use accuracy?
How do you stop the tutor from giving away the answer?
How do you pick the next problem?
How do you schedule reviews?
What happens at the school-day peak?
How do you handle children's data?