Enterprise AI Assistant

A company-wide assistant over internal knowledge and tools

Design an assistant every employee can ask about internal wikis, tickets, documents and chat, and ask to act in those tools, without ever showing anyone a document they cannot open.

Overview

An enterprise AI assistant is a chat box every employee can use to ask questions about the company's own knowledge ('What is our parental leave policy?', 'Why did the VPN fail last Tuesday?') and to get work done ('File a ticket for this', 'Summarize this thread'). It reads from the systems the company already uses: a wiki, a ticket tracker, shared drives and team chat.

The retrieval and generation part is the same as any retrieval-augmented generation (RAG) system, which the RAG System topic covers: chunk, embed, search with keywords and vectors, rerank, answer with citations. This topic is about what makes the enterprise version hard. Each source has its own identities and sharing rules, and the assistant must respect all of them at once. It acts inside other systems on the user's behalf. Every passage it reads may contain instructions planted by someone else. Legal and security teams need to know what it saw, what it said and what it did.

In the interview, say early that permissions are the central problem. A wrong answer is a quality bug; an answer that quotes a document the user cannot open is a security incident.

Questions to ask first

One company or many? — One company of 50,000 employees; multi-tenant SaaS is out of scope
Which sources? — Wiki, ticket tracker, document drive, team chat; each has an API with change tracking
How do people sign in? — Company single sign-on (OIDC); groups come from the identity provider
May it act, or only answer? — It may read and write in connected tools, as the user, and asks before any write
How fresh must permissions be? — A removed permission takes effect before the next answer that would use it
How fast? — First word within about 1.5 seconds at p95 (a target we chose)
What is kept? — Conversations for 30 days, audit records for 1 year (policy choices to confirm with legal)

What you will be able to do

Map one employee to their identities and groups in every source, and keep that map current.
Capture permissions at ingest, enforce them inside the search, and recheck them before an answer is written.
Explain the window between a permission change and the index learning about it, and how to close it.
Let the assistant act in other tools with the user's own scopes, with a confirmation that cannot be forged or reused.
Defend against instructions hidden in retrieved documents, and govern cost, PII, retention and audit.

Key concepts

Single sign-on
Identity mapping
Connectors and delta sync
ACLs at ingest
Query-time principal expansion
Answer-time recheck
Delegated OAuth scopes
Write confirmation
Indirect prompt injection
Model gateway and quotas

Functional requirements

Sign in with company SSO; every request runs as one employee
Ingest wiki pages, tickets, documents and chat messages, with their permissions, and keep both in sync
Answer questions from documents the asker may open, with citations to each source
Call tools in connected systems as the asker; ask for confirmation before any write
Keep an audit record of every question, the passages used, the answer and every tool call
Collect feedback on answers and let admins see usage and cost per team

Non-functional requirements

Never show content from a document the asker cannot open in its source system
A removed permission is enforced by the next answer, even before the index syncs, for every source with a permission check API
First token within about 1.5 seconds at p95 (a target we chose)
Content edits searchable within about 15 minutes (a target we chose)
Fail closed: when a permission cannot be resolved, the document is not used
99.9% availability for questions; a model provider outage degrades to a fallback model (a target we chose)

Interview questions

How do you make sure the assistant never shows a user a document they cannot open?
Why not just filter the model's answer for forbidden content afterwards?
A user is removed from a group. When does that take effect?
How do connectors stay in sync without re-reading everything?
How does the assistant create a ticket for the user safely?
How do you defend against prompt injection in retrieved documents?
How do you control cost per team?
What do you log, and for how long?
How would you evaluate this before each release?