AI Art Generation System

Text-to-Image Service

Design a service that turns a text prompt into a set of images in seconds: an asynchronous job pipeline in front of a scarce GPU fleet, with safety checks before and after generation and provenance on every image.

Overview

A text-to-image service takes a prompt such as 'a lighthouse in a storm, oil painting' and returns a few candidate images. The user picks one, then upscales it, varies it or edits part of it. Behind the simple form sits a diffusion model: a neural network that starts from random noise and removes a little of it at each of many steps, guided by the prompt, until an image appears.

That model sets the shape of the whole system. One job takes seconds of GPU time, not milliseconds of CPU, so the API cannot hold a request open while it works: it accepts a job and reports back later. GPUs are the scarce and costly resource, so the scheduler's job is to keep them busy with the right model already in memory. And people will ask for images the service must not make, so prompts are checked before any GPU time is spent and images are checked before anyone sees them.

Questions to ask first

How many users and jobs? — 1 million users a day, 5 million jobs a day, 4 images a job
What sizes? — Square and wide images around 1024 pixels; upscaling is a separate job
How fast must a result arrive? — Paid users see results within about 20 seconds at peak (our target)
One model or many? — A few base models plus many small style adapters (LoRA)
What is forbidden? — Sexual content involving minors always; other categories by policy and account settings
Must images be labeled as AI-made? — Yes, with signed provenance metadata on every file

What you will be able to do

Explain why generation is an asynchronous job and design the submit, status and result API.
Size the GPU fleet from jobs a day and GPU-seconds a job, and say what drives the cost.
Schedule jobs so that GPUs stay busy, the right model is already loaded, and one heavy user cannot starve others.
Place the safety checks and provenance signing, and say what each one can and cannot catch.
Keep credits correct when jobs fail, time out or have images filtered.

Key concepts

Asynchronous jobs
Latent diffusion
Batching variants
Model affinity
Priority queue
Admission control
Prompt and image safety
Content credentials (C2PA)
Credits reservation

Functional requirements

Submit a prompt with size, model, optional style adapter, optional seed, and get back a job id
Get a job's status and, when done, the images (4 by default)
Upscale, vary, or edit a masked region of an image as a new job that points at the source image
Reject prompts that break policy before any GPU work, and filter unsafe images before delivery
Attach provenance metadata marking every image as AI-generated
Show each user their history, and re-run any past job with the same settings
Charge credits per job, and refund them for failed or fully filtered jobs

Non-functional requirements

Paid-tier results within about 20 seconds at peak, free tier within a few minutes (targets we chose)
Prompt safety check adds under 200 ms to submit (a target we chose)
No image is ever readable by the user before it passes the image safety check
A lost worker never loses a job: it is retried on another worker
GPU utilization kept high, because GPUs are most of the cost

Interview questions

Why can't the generate endpoint just return the image?
How many GPUs do you need?
How do you keep GPUs busy without making some users wait forever?
Where do the safety checks go, and why two of them?
How do you keep credits correct?
How do you label images as AI-generated, and can people remove the label?
How would you support custom styles (LoRA adapters)?
What happens when a worker dies mid-job?