Music Generation System

Text-to-Music on a GPU Job Queue

Design a service that turns a text prompt into a finished music track: an asynchronous GPU job queue, a credit system that charges exactly once, and checks that catch outputs copying existing recordings before anyone hears them.

Overview

A user types 'warm lo-fi hip hop with a mellow piano, 80 BPM' and gets a two-minute track. Behind that is a large model running on a GPU for several seconds per track. Published research shows the two main model families: autoregressive models such as Meta's MusicGen, which predict compressed audio tokens one step at a time like a text model predicts words, and latent diffusion models such as Stable Audio, which start from noise and refine a whole clip at once. Commercial services such as Suno and Udio have not published their architectures, so this design treats the model as a black box with a measured speed.

There are three things that shape the system. The work is slow and expensive, so requests go through a queue to a pool of GPU workers. Users pay with credits, so charging must happen exactly once and be refunded if the job fails. And the legal risk is real: in June 2024 the major record companies sued Suno and Udio for copying recordings to train their models (Warner Music settled with both in November 2025 and signed licensing deals), so a service must at least make sure its outputs are not copies of existing recordings.

Questions to ask first

How long are tracks? — 30 seconds to 4 minutes; about 2 minutes on average
How many a day? — 500,000, with evening peaks 3 times the average
How fast must results arrive? — Paid users within about a minute; free users may wait longer
Do we train the model? — No; we serve a model we have the rights to use
What audio format? — 44.1 kHz stereo, delivered as MP3 with a lossless download for paid users
What must be checked before delivery? — Prompt policy, and a fingerprint match against a licensed reference catalog

What you will be able to do

Explain why generation is an asynchronous job and size the GPU pool from a model's measured speed.
Charge credits exactly once, and refund exactly once, in SQL that is safe under retries.
Run priority and standard lanes on one queue without starving either.
Explain how audio fingerprinting finds a copied recording, and what it cannot find.
Describe how the two model families differ in speed, streaming and memory.

Key concepts

Asynchronous job queue
GPU worker pool
Priority lanes
Credit ledger
Autoregressive audio tokens
Latent diffusion
Audio fingerprinting
Stem separation
CDN delivery

Functional requirements

Accept a prompt with optional genre, tempo, key, duration and instrumental flag; return a job id at once
Report job progress and deliver the finished track with a waveform preview
Charge credits by duration when a job is accepted; refund automatically if it fails or is blocked
Check every output against a reference catalog and the content policy before it is published
Keep a personal library with playback, download and sharing
Split a finished track into stems (vocals, drums, bass, other) as a separate paid job

Non-functional requirements

Priority-lane jobs start within 30 seconds at p95 during normal load (a target we chose)
No track reaches a user without passing the output check
Credits are charged once per accepted job and refunded once per failed job, whatever retries happen
A worker crash loses no job: the job returns to the queue
Playback starts in under 1 second from the CDN (a target we chose)

Interview questions

Why not generate the track inside the HTTP request?
How many GPUs do you need?
How do you make sure credits are charged exactly once?
How do you stop outputs that copy existing songs?
Can the user hear the track before it finishes?
How do free and paid users share the GPUs fairly?
What legal context should the design reflect?