Dealing with Contention
What to do when many requests want to change the same thing at the same moment: seats, stock, balances, counters. Atomic updates, locks, version checks, holds, fencing tokens and ways to split a hot row.
Overview
Contention happens when many requests try to change the same piece of data at once: the last seat on a flight, the stock count of a product in a flash sale, one account's balance, the like counter on a viral post. Each request on its own is correct. Together, they can sell a seat twice, lose a deposit, or slow the whole system to the speed of one row.
The classic bug is the lost update. Two requests read the same value, each computes a new value from it, and each writes. The second write overwrites the first, and one change disappears without an error.
There is no single fix. The tools range from one atomic SQL statement, through locks and version checks inside one database, to reservations with an expiry and locks shared across machines. Each suits a different amount of contention and a different cost of being wrong. Interviewers listen for two things: that you name the invariant you are protecting, and that you choose the simplest tool that protects it.
What you will be able to do
- Recognize a lost update and a check-then-act race in a design.
- Fix one with an atomic conditional update, a row lock or a version check, and say which fits.
- Design a reservation that holds an item for a few minutes and releases it safely.
- Explain why a distributed lock with a timeout needs a fencing token.
- Raise the write rate of a hot row by batching, sharding or putting its writes in one queue.
Key concepts
- Lost updates
- Atomic conditional updates
- Pessimistic locking
- Optimistic concurrency
- Isolation levels
- Reservations with expiry
- Distributed locks and leases
- Fencing tokens
- Hot rows and sharded counters
Interview questions
- Two users try to book the last seat at the same moment. How do you make sure only one gets it?
- Explain optimistic and pessimistic locking. When do you use each?
- How would you implement a distributed lock, and what can go wrong?
- A product's stock row gets thousands of updates a second during a flash sale. What do you do?
- What is write skew, and how do you prevent it?
- How do you keep a like counter correct on a post that gets 50,000 likes a minute?
- When would you route all writes for a key through a single queue consumer?
- Your service uses a Redis lock to make sure only one worker sends each day's invoices. Is that safe?
Sign up free to read the full topic
More System Design topics
Start free on Cariara · Coding problems · System design studio