AI Video Surveillance System

Edge video analytics and alerts

Design a platform that analyzes video from thousands of cameras at the sites where they are installed, raises alerts for rule-based events such as loitering or entering a restricted zone, and keeps as little footage as the purpose allows.

Overview

An AI video surveillance system watches camera feeds and tells a person when something needs attention: someone lingering by a back door after hours, a bag left on a platform, a crowd growing too dense near an exit. The operator then looks at a short clip and decides what to do.

Two facts shape the whole design. First, video is huge: 10,000 cameras at 4 Mbit/s each is 40 Gbit/s, around the clock. Sending all of it to a data center is expensive and slow, so the analysis runs at the edge, on a small computer at each site, and only events leave the building. Second, video is personal data about people who did nothing wrong. European regulators' guidance on video devices (EDPB Guidelines 3/2019) says footage should in most cases be erased, ideally automatically, after a few days, and that keeping it beyond 72 hours needs more justification. Designing for less data is both cheaper and safer.

The third problem is people. An alert that is usually wrong teaches operators to ignore alerts, so precision per rule and per site is a first-class metric.

Questions to ask first

How many cameras and sites? — 10,000 cameras across 500 sites (malls, depots, stations)
What should it detect? — Rule-based events: loitering, zone entry, left objects, crowd counts
How fast? — Operator sees the alert and clip within 10 s at p95 (our target)
Identify people by face? — No. Out of scope; it raises the legal bar sharply
How long is video kept? — Continuous video 72 hours on site; event clips 30 days (our choices, per purpose)
Who can view footage? — Named operators per site; every view is logged

What you will be able to do

Explain why analysis runs at the edge and compute how much bandwidth that saves.
Turn detections and tracks into events with zone rules, and deduplicate alerts.
Measure and raise alert precision per rule and per site with operator feedback.
Build privacy into the pipeline: masking, blurring, retention and audited access.
Link a person across cameras on one site without face recognition, and know its limits.

Key concepts

Edge inference
Motion gating
Object tracking
Zone rules
Alert deduplication
Precision per rule
Privacy masking
Retention limits
Person re-identification
Fleet management

Functional requirements

Ingest video from IP cameras at each site and run detection and tracking there
Evaluate per-camera rules (zones, time windows, thresholds) and create events with a short clip
Deliver alerts to the operators of that site in real time, with the clip and a thumbnail
Let operators acknowledge and resolve alerts as true or false positives
Mask privacy zones and blur faces outside security zones before any clip leaves the site
Delete footage automatically at the end of its retention period and log every footage view

Non-functional requirements

Event to alert on the operator console within 10 s at p95 (a target we chose)
A site keeps detecting and recording when its internet link is down, and uploads events later
A camera or site outage never affects other sites
Alert precision is measured per rule and per site; a rule below its target is reviewed
Continuous video never leaves the site; retention is enforced by the system, not by a manual process

Interview questions

Why not send all video to the cloud and run bigger models there?
How do you avoid flooding operators with false alerts?
How long do you keep video, and how do you enforce it?
How do you follow a person across cameras without face recognition?
What happens when a site goes offline?
How do you roll out a new detection model to 500 sites?