Facebook Live Comments

Real-time Comment Streaming System

Design the comment stream beside a live video: anyone watching can post, and every viewer sees a steady, relevant trickle of comments within seconds, even when millions watch and thousands post each second.

Overview

Live comments appear beside a video while it is being broadcast. Viewers post comments, and every viewer sees a stream of other people's comments, a few seconds after they were written. A comment that is a minute old is no longer interesting, so the system trades completeness for freshness.

The scale problem runs in two directions. A popular broadcast has millions of viewers, so one comment has millions of possible readers. And the same broadcast can receive thousands of comments a second, far more than any person can read. Meta's Bladerunner paper (SOSP 2021), which describes the system that serves Facebook's LiveVideoComments, says a live video with a hundred million viewers can produce over a million comments a second at peak, and that the rate for any given video cannot be predicted.

So the answer is not to deliver every comment to everyone. It is to store each comment, fan it out in tiers to the servers that hold viewer connections, and let those servers pick, per viewer, the few comments worth showing.

The building blocks this design reuses are covered in the Real-time Updates and Polling vs WebSockets vs Webhooks topics.

Questions to ask first

How many viewers per broadcast? — Up to about 10 million on the biggest one
How many comments? — Up to 10,000 a second on one broadcast
Must every viewer see every comment? — No: a ranked selection, favoring friends and quality
How fresh? — A few seconds; older than about 10 seconds is dropped
Can a comment be lost? — Its author must see it saved; other viewers missing one is acceptable
Replay? — Yes: comments reappear at the same point in the video

What you will be able to do

Explain why polling fails for live comments and how push with topic subscriptions replaces it.
Design two-tier fan-out: a pub/sub layer to gateway servers, and gateways to devices.
Rank and rate-limit comments per viewer, and handle a video that goes viral.
Filter spam fast enough for live use, and handle deletes and bans.
Say what the system guarantees (best effort) and why that is the right choice here.

Key concepts

Push, not polling
Topic pub/sub
Two-tier fan-out
Per-viewer ranking
Rate limiting per stream
Best-effort delivery
Hot-topic pre-ranking
Sticky reconnects
Moderation fast path
Replay by video offset

Functional requirements

Post a comment on a live video; the author sees it immediately
Viewers see a steady stream of other viewers' comments within seconds
Show friends' comments and high-quality comments first
Filter spam and abuse; let the broadcaster and moderators delete comments and ban users
Limit how fast one user can comment (slow mode)
Show comments at the matching moment when the video is replayed

Non-functional requirements

A comment reaches other viewers in under 5 seconds at p95 (a target we chose)
A saved comment is never lost for its author; live delivery to others is best effort
One broadcast with 10 million viewers must not slow other broadcasts
No polling from viewers while they watch
Degrade by showing fewer comments, never by stopping the video

Interview questions

Why not just have clients poll for new comments?
How do you deliver one comment to ten million viewers?
How do you decide which comments a viewer sees?
What happens when a video suddenly goes viral?
Do you guarantee every viewer sees every comment? Should you?
How do you handle spam and abuse in real time?
How do reconnects work without losing the viewer's place?
How do comments show up when someone watches the replay?