Pastebin

Text Sharing

Design a service where anyone can paste text or code, get a link back, and have the paste disappear on schedule, while keeping it from becoming free file hosting or a malware drop.

Overview

A paste service takes a block of text, stores it, and returns a link. Whoever opens the link sees the text, until the paste expires or its owner deletes it. It looks like a URL shortener, and it shares that problem's short code and read-heavy traffic (see the URL Shortener topic for code generation). This page covers what is different: the content itself.

Pastes are bodies, not pointers. A short link stores a URL of about 100 bytes; a paste can be anything up to the size limit. Where the bytes live decides cost and read latency.
Pastes die. Most expire after minutes, days or weeks. The system must stop serving them at exactly the right time, from every cache, and then reclaim the space.
The link is the password. An unlisted paste has no access check except that nobody knows its key. Keys that can be guessed leak other people's pastes.
Anonymous uploads attract abuse. Leaked credentials, malware, phishing pages and people using the service as free file hosting.

Questions to ask first

How much traffic? — 1 million new pastes a day, each read about 10 times
How big? — 10 KB on average; a hard limit of 1 MB
How long do pastes live? — Owner's choice: 10 minutes, 1 hour, 1 day, 1 week, 1 month or never; half never expire
Who can read a paste? — Public (listed), unlisted (anyone with the link) or password-protected
Accounts? — Optional; anonymous pastes allowed with stricter limits
Editing? — No; a paste never changes after it is created

What you will be able to do

Decide where each paste's bytes live and justify the size cut-off.
Make an expired or deleted paste unreadable at once, everywhere, while reclaiming its space lazily.
Choose key lengths for public and unlisted pastes from a guessing budget.
Enforce size limits and rate limits before storing anything.
Serve untrusted text so it can never run as a web page on your domain.

Key concepts

Inline vs object storage
Unguessable keys
Expiry at read time
Batched sweeping
Burn after reading
Size limits at the edge
Cache lifetime capped by expiry
Serving untrusted text safely
Rate limiting

Functional requirements

Create a paste with text, an optional title and language, an expiry and a visibility
Return a link; open the link to read the paste, or its raw text
Burn after reading: the paste is deleted by its first read
Password-protected pastes need the password to read
The owner can delete a paste at any time
Report a paste; moderators can take it down

Non-functional requirements

An expired, deleted or taken-down paste is never served again, from the origin or a cache
Read p99 under about 100 ms for a cached paste (a target we chose)
A stored paste is not lost before its expiry (durable storage, replicated)
Unlisted pastes cannot be found by guessing keys within any realistic number of requests
Rate limits per IP and per account keep abuse from crowding out normal users

Interview questions

How is this different from a URL shortener?
Where would you store the paste content?
How do you make sure an expired paste is never shown?
How long should the keys be?
How do you implement burn after reading safely?
How do you stop the service from being abused?
Would you deduplicate identical pastes?
What does the system need at 100 times this traffic?