Real-Time Fraud Detection

Payment Risk Scoring

Design a service that scores every card payment for fraud before it is authorized, using rules, fresh velocity counts and a model, and that keeps learning from disputes that arrive weeks later.

Overview

A payment fraud system looks at every card payment before the card is charged and returns one of three answers: allow, review (let it through but have a person look), or block. It has a budget of about a tenth of a second, because the buyer is waiting on the checkout page.

The two kinds of mistakes cost different things. A false negative (fraud that got through) usually ends in a dispute, and the merchant loses the money and pays a dispute fee. A false positive (a good customer blocked) loses the sale and often the customer. The system does not remove mistakes; it chooses where to put the threshold between them.

Three things make the problem hard. The strongest signals are velocity: how many times this card, device or IP address was used in the last minute, hour or day, and those counts must be fresh at scoring time. The truth arrives late: card networks typically let a cardholder dispute a payment for 120 days. And the model changes its own training data: a blocked payment never gets a dispute, so you never learn whether blocking it was right.

Questions to ask first

Who calls us, and when? — The payment service, synchronously, before sending the charge to the card network
How many payments? — 100 million a day, with peaks at 10 times the average
How fast must we answer? — 100 ms at p99, a target we chose; the checkout is waiting
Which actions? — Allow, review, block; review lets the payment through and queues it for a person
Where do labels come from? — Disputes, merchant fraud reports, early fraud warnings and review outcomes
What if we are down? — The payment goes ahead on rules only; checkout must not stop

What you will be able to do

Lay out the scoring path and split a 100 ms budget across its steps.
Keep velocity counts exact and fresh with sorted sets, and make them safe against replayed events.
Combine rules and a model, and choose thresholds with precision, recall and review capacity in mind.
Explain why labels arrive late and why blocked payments hide their own outcome, and fix it with exploration and weighting.
Find fraud rings through shared cards, devices and addresses, and ship a new model without risk.

Key concepts

Velocity features
Sliding-window counters
Rules plus model
Decision thresholds
Delayed labels
Counterfactual exploration
Inverse propensity weighting
Fraud rings
Shadow rollout
Fail-safe defaults

Functional requirements

Score each payment before authorization and return allow, review or block with a score and reasons
Evaluate merchant and platform rules (block lists, amount limits, velocity limits) with the model score as an input
Keep velocity counts per card, device, IP address and account over 1 minute, 1 hour and 24 hours
Collect labels from disputes, fraud reports, early fraud warnings and manual reviews
Retrain models on matured labels and roll them out in shadow first
Link accounts that share cards, devices or addresses into rings and use ring size as a feature

Non-functional requirements

Scoring answers in 100 ms at p99 (a target we chose), feature reads in about 20 ms
Scoring is idempotent per payment: a retry gets the same decision and does not count twice
Available at all times: if the model or features fail, fall back to rules and allow by default
Every decision is logged with the model version, features used and the exploration probability
A new model or rule changes decisions only after it has run in shadow

Interview questions

How do you compute velocity features fast enough?
Why have rules at all if you have a model?
Labels take months. How do you train?
If you block a payment, how do you know you were right?
How do you choose the block and review thresholds?
How do you catch fraud rings that use each card once?
What happens when the fraud service is down?
How do you roll out a new model safely?