AI Healthcare Chatbot

Design a Patient Assistant for a Health System

Design a patient chat assistant for a health system that books visits, routes symptoms to the right level of care and escalates emergencies, while keeping patient data inside HIPAA's rules.

Overview

A health system (hospitals and clinics under one organization) wants a chat assistant in its patient app. Patients ask it to book a visit, ask whether a symptom needs urgent care, ask how to take a medicine, request a refill or say they are not feeling like themselves. Some of those messages are routine; a few are emergencies.

Two things make this harder than a shopping or banking bot. Safety: a wrong answer can hurt someone, and a language model (LLM) can state wrong medical facts fluently. Law: in the US, patient data is protected health information (PHI) under HIPAA, so every system and vendor that touches it must be inside a contract and a set of safeguards. There is also a regulatory line: software that analyzes a patient's symptoms to tell them what condition they have or what treatment to take can be a medical device under FDA rules. This design stays on the safe side of that line: it routes people to the right level of care and to clinicians, and it does not diagnose.

Questions to ask first

Who uses it? — Patients of one health system, signed in to its app
Does it diagnose? — No: it routes to a level of care and to people
Can it change the medical record? — Only appointments; refills become tasks for the care team
Which EHR interface? — The EHR's FHIR R4 API with SMART on FHIR patient scopes
Can the model provider see PHI? — Only a provider that signs a BAA, and only what the turn needs
How big? — 5 million patients in the app, 200,000 chat sessions a day

What you will be able to do

Explain why emergencies are caught by fixed rules before the model, and what the assistant says then.
Draw the HIPAA boundary: which vendors need a BAA, what 'minimum necessary' means for a prompt, what goes in the access log.
Book an appointment slot so two patients cannot get the same one, using FHIR and SQL.
Separate routing to a level of care from diagnosis, and say why that matters for FDA rules.
Size the service and plan for the model being wrong or down.

Key concepts

Red-flag screening before the model
Care level, not diagnosis
Business associate agreement (BAA)
Minimum necessary
FHIR R4 and SMART scopes
Conditional slot booking
PHI access log
PHQ-2
Human in the loop

Functional requirements

Book, move and cancel appointments from the EHR's open slots
Screen every message for emergency and crisis signs before anything else runs
Route symptom questions to a level of care (emergency, same day, routine, self-care information) using clinician-approved protocols
Answer medication questions from approved drug information, citing the source
Turn refill and clinical questions into tasks for the care team
Offer the PHQ-2 depression screen and route positive screens to behavioral health
Log every read or write of patient data

Non-functional requirements

Emergency and crisis replies are fixed text, shown within 1 second (a target we chose), with no model call
Patient data reaches only systems and vendors under a BAA, and only the minimum a turn needs
No appointment slot is ever booked twice
The model never states a diagnosis, a dose change or a treatment decision
Scheduling and escalation keep working when the model is down

Interview questions

How do you make sure an emergency is never handled by the model?
Where is the line between routing and diagnosing, and why does it matter?
Can you send patient data to an LLM provider?
How do you stop two patients from booking the same slot?
How does the assistant answer medication questions safely?
What does the PHQ-2 do, and what happens after a positive screen?
How do you size the system?