AWS Credentials Committed to a Public GitHub Repository
An access key ID and secret committed in a config file were found by an automated scanner within 4 minutes, leading to $34,000 in EC2 charges overnight.
Overview
An AWS key pushed to a public GitHub repository was found by scanners in 4 minutes and ran up $34,200 of GPU mining before it was revoked.
What happened
A developer writing a tutorial blog post committed a sample config file to a public repository. It held a real AWS access key left over from local testing. The plan was to remove it before publishing, but the push to the public branch came first.
How it spread
Automated scanners watch public GitHub commits for credential patterns. Before the key was revoked, the attacker had already set up mining and persistence.
> [!WARNING] Revoking the key was not the whole cleanup: the attacker had also created 12 IAM users and an SQS dead drop for ongoing communications.
Impact
- Time to discovery by scanners — 4 minutes
- Time until the developer revoked the key — 3 hours
- GPU instances launched — 400, in 6 regions
- IAM users created — 12
- EC2 charges — $34,200 in 14 hours
- Recovery time — 3 days
AWS waived the charges after a Security Incident Report.
The fix
- Long-lived access key in a config file — IAM roles and AWS Secrets Manager
- Nothing checked commits for secrets — Pre-commit hooks such as detect-secrets
Key concepts
- Credential scanning
- Automated tools (GitHub secret scanning, truffleHog, gitleaks) that search code repositories for patterns matching API keys, passwords, and credentials.
- IAM least privilege
- Granting only the permissions required for the specific task — a key used for S3 read access during a tutorial should have no EC2 or IAM permissions.
- Secret rotation
- Regularly replacing credentials and automatically distributing the new values using a secrets manager, reducing the window of exposure for any compromised key.
- AWS Secrets Manager
- Managed service for storing, rotating, and auditing access to secrets. Applications retrieve credentials at runtime rather than storing them in code or config files.
When to use it
- Discussing secrets management and credential hygiene
- Explaining the response procedure for a leaked AWS key
- Justifying IAM least-privilege and credential rotation policies
Interview questions
- You discover an AWS access key has been committed to a public GitHub repository. Walk me through your incident response.
Sign up free to read the full topic
More War Stories topics
- A Deployment That Could Not Be Rolled Back
- A Failed Migration That Left the Database in a Partial State
- A Feature Flag Change That Triggered a Production Outage
- Accidental DELETE Without a WHERE Clause in Production
- An IAM Misconfiguration That Let a Developer Become Admin
- Backups That Were Never Actually Completing
- BGP Route Leak That Rerouted Traffic Through an Unexpected AS
- Blue-Green Deployment That Resulted in Split-Brain Sessions
- Cache Stampede on a Flash-Sale Product Page
- Connection Pool Exhaustion That Brought Down 12 Microservices
- DynamoDB Hot Partition That Throttled a Flash Sale
- Failover to a Lagging Replica Lost 4 Minutes of Orders
Start free on Cariara · Coding problems · System design studio