OCR System (Text Recognition)

Document Text Extraction Service

Design a service that turns photos, scans and PDFs into text with positions, tables and form fields: a fast path for single images, a batch path for millions of pages, and confidence scores that decide what a person must check.

Overview

Optical character recognition (OCR) reads text from images: a phone photo of a receipt, a scanned contract, a 400-page PDF of old records. A useful OCR service returns more than a string. It returns every word with its position on the page and a confidence score, and for business documents it returns structure: which words form a table, which value belongs to the 'Total' label.

Two very different users call it. A mobile app wants one receipt read while the user waits. A back office wants a million archived pages processed by tomorrow. The first needs low latency; the second needs throughput and cost. Hosted OCR services draw the same line: Amazon Textract, for example, offers synchronous operations for single-page documents and asynchronous jobs for large multipage ones, with completion announced on a notification topic.

The other hard part is knowing when OCR is wrong. No model reads every smudged scan correctly, so the service must say how sure it is, in numbers that mean something, and send the unsure parts to a person.

Questions to ask first

How many pages, and how much is interactive? — 10 million pages a day; 1 million real-time, 9 million batch
How fast for real time? — One page in under 3 seconds at p95 (our target)
When must batch finish? — Within 24 hours of submission
What output? — Words and lines with boxes and confidence; tables and form fields on request; searchable PDF
Printed only, or handwriting too? — Both; handwriting is slower and less accurate
What happens to low-confidence results? — Fields below a threshold go to human review

What you will be able to do

Walk through the OCR pipeline stage by stage and say what each stage costs and fixes.
Split real-time and batch work and size both from pages a day.
Track a multi-page batch job so it completes exactly once, even with retries.
Explain confidence scores, why they need calibration, and how they drive human review.
Measure accuracy with character and word error rates, and catch errors with business checks.

Key concepts

Pipeline stages
Text detection
Text recognition (CTC)
Layout analysis
Page-level parallelism
Separate real-time and batch paths
Confidence calibration
Human review loop
Character error rate

Functional requirements

Real-time: send one image or a short PDF and get text, word boxes and confidence in the response
Batch: submit many documents stored in object storage and get a job id; results land in an output location; a webhook fires on completion
Return words, lines and blocks with bounding boxes and confidence scores
Optionally extract tables (rows, columns, cells) and key-value form fields
Accept language hints; detect the script when none is given
Produce a searchable PDF: the original image with an invisible text layer
Route low-confidence fields to human review and store the corrections

Non-functional requirements

Real-time: under 3 seconds a page at p95 (our target)
Batch: every job finishes within 24 hours of submission
A batch never slows real-time traffic
No page is lost or counted twice; a job completes exactly once
Customer documents are encrypted and deleted after the retention period

Interview questions

Walk me through what happens to one page.
How do you support both a user waiting for one receipt and a million-page backlog?
How do you make sure a batch job completes exactly once?
What does a confidence score of 0.9 mean, and how do you use it?
How would you extract tables from scanned invoices?
How do you measure and keep accuracy?
How many GPUs and reviewers do you need?