Live Video Transcoding Pipeline

Ingest, encode and package live streams

Design the pipeline behind live streaming: take one stream from a broadcaster, turn it into several qualities in real time, and publish it as low-latency HLS within a few seconds of the camera.

Overview

A broadcaster's encoder sends one live stream, say 1080p at 60 frames a second. Viewers are on fiber, on hotel Wi-Fi and on a train, so the platform must turn that one stream into several versions at different sizes and bitrates, cut them into small files that ordinary HTTP caches can serve, and keep doing it for hours without falling behind.

This problem is the pipeline in the middle: ingest (receive the stream), transcode (decode it and encode each quality), package (cut segments and write playlists) and publish (hand them to the CDN). The Twitch topic covers the whole product and its delivery network; this topic goes deeper into the pipeline's timing, the Low-Latency HLS mechanics, and what happens when a machine dies mid-stream.

The pipeline is real time in the strict sense. A batch job that runs slowly finishes late; a live transcoder that runs at 0.95 times real time falls further behind every second until viewers stall. So capacity is planned per channel, not per request, and failures must be handled in seconds.

Questions to ask first

How many channels live at once? — 10,000 at peak
What do broadcasters send? — 1080p60 H.264 at about 6 Mbps, over RTMP or SRT
What latency is needed? — About 5 seconds camera to screen; 20 seconds or more is acceptable as a fallback
Which devices play it? — Browsers, phones and TVs: HLS with fMP4 (CMAF) segments
Can viewers rewind? — Yes, a 2-hour DVR window; streams are recorded
Do all channels get the full ladder? — Yes in this design; a real service may limit it (see the Twitch topic)

What you will be able to do

Choose an ingest protocol and explain what RTMP, SRT and WebRTC ingest each trade.
Build a quality ladder whose renditions switch cleanly, with aligned keyframes.
Explain how Low-Latency HLS parts, preload hints and blocking playlist reloads cut delay from about 25 seconds to about 5.
Place channels on transcoders, detect a dead machine, and fail over in seconds.
Write a latency budget and say which knob moves each line.

Key concepts

RTMP and SRT ingest
Quality ladder
Keyframe (GOP) alignment
CMAF segments
Partial segments
Blocking playlist reload
Hold-back
Transcoder leases
Redundant pipelines
Latency budget

Functional requirements

Create a channel and give the broadcaster ingest URLs and a secret stream key
Accept a live stream over RTMP or SRT and reject a wrong or revoked key
Transcode each stream into a ladder of video renditions plus audio, in real time
Publish Low-Latency HLS playlists and fMP4 segments and parts to the CDN
Keep a 2-hour DVR window and record the stream for playback afterwards
Report stream health (ingest bitrate, keyframe interval, dropped frames) to the broadcaster

Non-functional requirements

About 5 seconds camera to screen in low-latency mode (a target we chose); a standard HLS fallback for players without it
Every transcoder keeps up with real time; a machine falling behind is taken out of rotation
A transcoder failure interrupts a channel for no more than a few seconds
Renditions stay aligned so a player can switch quality without a stall
Ingest keeps working when recording or the control plane is slow

Interview questions

Why transcode at all? Why not send the broadcaster's stream to everyone?
How does Low-Latency HLS cut the delay?
Why must keyframes be aligned across renditions?
A transcoder machine dies. What happens?
How do you size the transcoder fleet?
How do DVR and recording work?
When would you not use HLS?
What would you monitor?