Public S3 Bucket Exposed Customer PII for 8 Months
A single Block Public Access setting left unchecked exposed 3.2 million customer records in a logging bucket.
Overview
Every application bucket was locked down, but the logging bucket was left public during setup, and for 8 months it held customer PII from request payloads.
What happened
The team's S3 practices were strong: every application bucket required server-side encryption and versioning and had Block Public Access enabled at the bucket level. The gap was a bucket created for a new observability pipeline.
Why it broke
The engineer was working quickly and meant to restrict the bucket after testing. Meanwhile the application logs included request payloads, and error logs carried the most sensitive data.
> [!WARNING] Bucket-level controls protect only the buckets someone remembers to configure. One bucket set up outside the usual path was enough.
Impact
- Exposure period — 8 months
- Log data in the bucket — 847 GB
- Distinct email addresses found by Macie — 3.2 million
- Credit card-like patterns found by Macie — 180,000
The fix
Move the guarantee from each bucket to the account, and check it continuously.
Key concepts
- S3 Block Public Access
- Four settings that can be applied at the account or bucket level to prevent any bucket policy or ACL from granting public access. Should be enabled at the account level as a baseline.
- AWS Macie
- Managed service that uses ML to automatically discover, classify, and protect sensitive data in S3, alerting on PII, credentials, and financial data.
- Data minimization
- Logging only the fields necessary for debugging, not full request/response payloads. PII should be masked or excluded from logs by default.
- AWS Config rules
- Automated compliance checks that evaluate resource configurations against rules, alerting or auto-remediating when resources drift from the expected configuration.
When to use it
- Discussing AWS security baseline requirements and account-level controls
- Explaining why logging infrastructure must meet the same security standards as application infrastructure
- Justifying data classification and PII filtering in logs
Interview questions
- How do you ensure no S3 bucket in your AWS account can be made public accidentally?
Sign up free to read the full topic
More War Stories topics
- Failover to a Lagging Replica Lost 4 Minutes of Orders
- Internal DNS Cache Poisoning That Redirected API Traffic
- Malicious npm Package Exfiltrated Environment Variables at Build Time
- N+1 Query Pattern That Collapsed Under Load
- NAT Gateway Port Exhaustion Under Bursty Lambda Traffic
- Node.js Memory Leak That Required Daily Restarts
- Silent Data Corruption from an Off-By-One Migration
- Single-Region Dependency That Caused a Global Outage
- The Cascade That Took Down Three Services
- Third-Party API Outage That Was Mistaken for Own Code
- Thundering Herd After a Cache Cluster Restart
- TLS Certificate Expiry That Took Down an Internal API
Start free on Cariara · Coding problems · System design studio