Autonomous Agent Platform

Running LLM agents that plan and use tools

Design a platform that runs LLM agents: each task loops through model calls and tool calls for minutes or hours, survives crashes, stays inside a budget, asks a person before anything irreversible, and runs untrusted code in a sandbox.

Overview

An agent receives a goal ('find last quarter's numbers and email a summary to the CFO') and works toward it in a loop: the model reads the history and decides the next action, the platform runs that action as a tool call (search, run code, call an API), the result is appended to the history, and the loop repeats until the model gives a final answer or a limit stops it. The ReAct paper (Yao et al., 2022) named this pattern of interleaving reasoning and actions.

Anthropic's 2024 guide 'Building effective agents' draws a line worth using in the interview. Workflows run models and tools through code paths you wrote in advance; agents let the model direct its own process and tool use. It advises finding the simplest solution possible and adding agent autonomy only when needed, and says agents need extensive testing in sandboxed environments, guardrails, and stopping conditions such as a maximum number of iterations. The platform is where those rules are enforced, for every agent at once.

Three properties make this different from a normal request-response service. Tasks are long: minutes to hours, across many machine restarts. Each step costs money and the cost grows as history grows. And tool calls have real side effects, triggered by a model that can be misled by the content it reads.

Questions to ask first

Who writes the agents? — Developers on the platform define agents: a model, a prompt, a set of tools
How long do tasks run? — Typically 30 steps of about 10 seconds; some run for hours
How many at once? — About 10,000 running tasks at peak
What can tools do? — Read data, call external APIs, run generated code, send messages
Which actions need a person? — Anything irreversible or external: sending, paying, deleting, deploying
Multi-tenant? — Yes: many organizations, strict isolation

What you will be able to do

Explain the agent loop, its stop conditions, and when a fixed workflow is the better design.
Make a task survive crashes with a durable step log and idempotent tool calls.
Enforce budgets and approvals in code, atomically, rather than trusting the model.
Design the tool sandbox and the defenses against prompt injection.
Keep the cost of long tasks under control with compaction and caching.

Key concepts

Agent loop (ReAct)
Workflow vs agent
Durable execution
Idempotent tool calls
Tool schemas (MCP)
Human approval
Sandboxing (microVMs)
Prompt injection
Context compaction
Budgets and stop conditions

Functional requirements

Start a task from a goal and an agent definition (model, instructions, allowed tools, limits)
Run the loop: call the model, validate and run the tools it asks for, record every step
Stream progress (steps, tool calls, results) to the user
Pause for human approval before actions marked as requiring it, and resume on the decision
Stop on a final answer, a step limit, a budget limit, a time limit, or a user cancel
Resume a task after any crash without repeating completed side effects
Run generated code in an isolated sandbox with no access to other tenants or secrets
Let a task start sub-agents for independent subtasks, within the parent's budget

Non-functional requirements

No step is lost and no side effect is repeated when a worker crashes
A task never spends more than its budget (checked before each model call)
Sandbox escape or cross-tenant access treated as a critical failure; defense in depth
Orchestrator overhead small compared with model time: under 100 ms per step (a target we chose)
Every tool call auditable: who, which task, which inputs, which result

Interview questions

Walk me through the agent loop. What stops it?
How does a task survive a crash in the middle of a step?
How do you run code the agent writes?
How do you defend against prompt injection?
How do you keep a long task from costing too much?
When would you not build an agent?
How do sub-agents fit, and what can go wrong?