AI E-commerce Search Ranking System

Product Search for a Large Online Store

Design the search behind an online store's search box: understand the query, find matching products among hundreds of millions, rank them by what shoppers actually buy, and apply stock and merchandising rules, all in a fraction of a second.

Overview

Product search starts from a short query ('running shoes', 'burgundy dress', 'latex exam gloves') and ends with a page of products the shopper is likely to buy. Amazon's engineers wrote in 2016 that Amazon Search powers the majority of Amazon's sales, which is why small ranking gains matter so much in this domain.

Keyword matching alone fails in both directions. Amazon's Semantic Product Search paper (KDD 2019) gives examples: the query 'running shoes' may miss products described as 'sneakers', 'burgundy dress' misses a 'red dress', and 'latex free examination gloves' are retrieved for 'latex examination gloves' because every word matches. The same paper notes that product search differs from web search: queries are short, and purchases, the strongest signal, are much sparser than clicks.

The standard answer has two phases. Matching (retrieval) builds a candidate set from an inverted index plus a semantic model. Ranking orders the candidates with machine-learned models in stages. A rule layer then applies what the model must not decide: stock, blocked items, merchandising and labeled sponsored slots.

Questions to ask first

How big is the catalog? — 200 million products, with prices and stock changing all day
How much traffic? — 300 million searches a day, peak 3 times the average
Personalized? — Lightly: ranking features from the shopper's recent activity; the candidate set is not personalized
What does success mean? — Purchases and revenue per search, in A/B tests; relevance judged offline with NDCG
Sponsored products? — Yes, in fixed labeled slots; the auction is out of scope
How fresh? — Price and stock visible within a minute; new products searchable within minutes

What you will be able to do

Turn a raw query into filters plus free text, and explain why that improves precision.
Combine BM25 and semantic retrieval, and merge their lists with reciprocal rank fusion.
Rank in stages with learning to rank, and measure it with NDCG offline and revenue per search online.
Keep stock, blocks, boosts and sponsored slots in a rule layer, and keep price and stock fresh in the index.

Key concepts

Query understanding
BM25
Semantic retrieval
Reciprocal rank fusion
Multi-stage ranking
Learning to rank
NDCG
Position bias
Merchandising rules
Near-real-time indexing

Functional requirements

Return a ranked page of products for a text query, with facets (brand, price range, rating, size)
Correct spelling and extract brand, category, color and size from the query
Retrieve products that match the meaning of the query, not only its words
Never show out-of-stock or blocked products; apply merchandising boosts and buries per query
Place sponsored products in fixed slots, labeled as sponsored
Reflect price and stock changes within a minute; make new products searchable within minutes
Log every impression, click, add-to-cart and purchase with the query and position

Non-functional requirements

p99 latency of 250 ms for a results page (our target)
High availability; a failed ranker degrades to the L1 order, never to an error page
Stock and blocks are enforced on every response, even in degraded mode
Ranking changes ship behind A/B tests with revenue per search as the main metric

Interview questions

Why is keyword search not enough for product search?
How do you combine BM25 and vector search results?
Walk me through the ranking stack.
What labels do you train the ranker on, and how do you evaluate it?
How does the Amazon semantic search model treat products that were shown but not bought?
How do you keep prices and stock fresh in a 200-million-product index?
How do merchandising rules and sponsored products fit without ruining relevance?