AI Social Media Bot Detection System

Finding Fake and Coordinated Accounts

Design the system a social network uses to find automated, fake and coordinated accounts, from checks at sign-up to graph models and group detection, and to act on them without silencing real people.

Overview

A social network has to find accounts that are not what they claim to be: scripts that post spam, farms of fake profiles that sell follows and likes, and networks that pretend to be many independent people. Meta's policy calls the most sophisticated form coordinated inauthentic behavior: fake identities are central to the operation and the operators work to evade detection. Meta also states that it enforces this on behavior, regardless of content or ideology. That is the right frame for the design: the system judges how accounts act, not what they say.

The design is shaped by two facts. First, the opponent adapts: any public rule or threshold becomes a target. Second, a wrong decision against a real person (a journalist, a small business) is costly, so severe actions need very high precision and a way back.

Questions to ask first

How big is the network? — 1 billion accounts, 300 million active a day, 2 million sign-ups a day
What counts as a bot? — Automated or fake accounts and coordinated groups; not what they post
How fast must we act? — Spam bursts in seconds; account scores daily; group detection daily
Which actions exist? — Limit reach, challenge (captcha, phone check), suspend, all appealable
Are good bots allowed? — Yes, registered and labeled automated accounts are fine
What is the error budget? — Suspensions must be very rarely wrong; mild limits may be wrong more often

What you will be able to do

Split bot detection into per-action rules, an account model and group detection, and say what each one catches.
Explain why graph features and group behavior are hard for attackers to fake, with published results.
Build a loose-synchrony detector: matching actions, similarity, clustering, and why hot objects need partitioning.
Choose thresholds per action from a precision target, and design review and appeals as part of the system.
Keep the loop learning while the opponent adapts.

Key concepts

Inauthentic behavior
Rate limits and challenges
Account score
Graph features
Loose synchrony
Jaccard similarity
Connected components
Precision-first thresholds
Graded enforcement
Appeals as labels

Functional requirements

Check each sign-up and each write action (post, follow, like, message) against rate limits and known-bad signals
Keep a bot score per account, refreshed daily and on suspicious events
Find groups of accounts acting in step on the same targets
Apply graded actions: limit reach, challenge, suspend; record why
Send uncertain cases to human review, and accept appeals on every action
Feed review and appeal outcomes back as training labels
Allow registered automated accounts and label them

Non-functional requirements

Edge checks add under 10 ms to a write (a target we chose)
Suspensions made without a human reviewed to 99% precision or better, measured on samples (our target)
Every action explainable and reversible; appeals answered within days
Detection details (features, thresholds) kept confidential
The action pipeline survives a burst of several times normal load (spam waves)

Interview questions

How do you detect coordinated accounts when each one looks legitimate?
What features would you give an account-level model?
How do you keep false positives low for real users with unusual behavior?
How does the system scale the pairwise comparison?
Where do labels come from, and how do you retrain against an adaptive opponent?
What happens at sign-up?
Would you use a graph neural network?
How do you measure whether the system works?