Email Auto-Response System

Triage, Suggested Replies and AI Drafts

Design an assistant that connects to a user's mailbox, decides which new emails need a reply, offers three one-tap replies, writes full drafts in the user's own style, and sends automatic replies only where it is safe to do so.

Overview

An email assistant sits beside a user's mailbox. When mail arrives it answers three questions: does this email need a reply at all, what are two or three short replies the user could send with one tap, and, if the user asks, what would a full reply in their own words look like. A small set of users also want the assistant to reply on its own, for example an out-of-office notice or an acknowledgement from a shared support inbox.

Two things make this harder than calling a language model. The first is volume against value: most mail is newsletters, receipts and notifications that need no reply, so running an expensive model on every message wastes money and shows useless suggestions. Google's Smart Reply paper describes a cheap triggering model that runs first and lets only about 11% of messages through to the response model. The second is trust: a reply goes out under the user's name. The design keeps a person in the loop for everything the model writes, and allows fully automatic replies only for fixed templates that follow the email standards for auto-responders.

Questions to ask first

Which mail providers? — Gmail and Outlook through their APIs, with the user's OAuth consent
Suggest, draft, or send? — Suggest and draft for everyone; send automatically only for opt-in templates
How many users and emails? — 1 million mailboxes, 50 emails received and 20 sent per user a day
How fast? — Short replies ready when the email is opened; a draft starts streaming within 2 seconds
May we store email bodies? — No: bodies are processed in memory; only derived data is kept
Enterprise needs? — Opt out of shared models, a sensitive-mail list that never gets drafts, audit logs

What you will be able to do

Explain why a cheap triggering step runs before any expensive model, and size both with numbers.
Pick three short replies that are not three ways of saying yes.
Build a writing-style profile without storing the user's mail.
Send automatic replies that never loop, never answer a mailing list, and reach each sender at most once a week.
Defend the design against instructions hidden in an incoming email.

Key concepts

Triggering model
Fixed response set
Intent diversity
Style profile
Thread summarization
RFC 3834 auto-replies
Once per sender per 7 days
Indirect prompt injection
Transactional outbox

Functional requirements

Receive a notification for each new email in a connected mailbox and fetch it through the provider API
Classify each email: needs a reply, meeting request, FYI, newsletter or automated, sensitive
Offer up to three short replies with different intents (for example yes, a question, no)
Stream a full draft reply on request, using the thread, the user's style and calendar availability
Never offer drafts for mail the user or admin marked sensitive (legal, HR, finance)
Send opt-in automatic replies from fixed templates, following RFC 3834
Record which suggestions were used and how much the user edited a draft

Non-functional requirements

Triage within 5 seconds of the provider notification (a target we chose)
Short replies ready before the user opens the email; if not ready, show none rather than wait
First draft tokens within 2 seconds, full draft within 10 seconds (targets we chose)
No email body stored after processing; derived data only, deletable per user
An automatic reply is sent at most once per sender per rule per 7 days, and never twice for the same email
A provider outage or model outage delays suggestions; it never sends anything unexpected

Interview questions

How do you avoid running an expensive model on every email?
How do you choose the three short replies?
How do you match the user's writing style without storing their mail?
How do you make automatic replies safe?
What if an incoming email contains instructions aimed at your model?
How do you receive new mail for a million mailboxes?
How do you support meeting requests?
How do you know the assistant is any good?