P1HardRunbook

Key Compromise Response Runbook

Private key exposed or suspected breach? This runbook guides you through incident response, remediation, and prevention. Time is critical - act fast.

1-2 hours

Last Updated: December 2025

Progress: 0/33 complete0%

When to Use

• Private key file was exposed (committed to Git, leaked in logs, etc.)
• Server was compromised and key may be stolen
• Employee with key access terminated without key rotation
• Suspicious activity suggesting key misuse

Do NOT Use For

• Certificate expired (use Emergency Replacement Runbook)
• Chain issues (use Certificate Not Trusted Runbook)
• Just want to rotate keys proactively (use Certificate Renewal Runbook)

Quick Reference (TL;DR)

Confirm compromise and gather evidence
Generate NEW key pair immediately
Request new certificate from CA
Deploy new certificate
REVOKE the old certificate
Conduct post-incident analysis

1Confirm Compromise

Objective: Verify the key is actually compromised and assess scope

How was the key exposed? (Git, logs, breach, stolen device, insider)

Identify which certificate(s) use this key

When did the exposure occur (or when was it discovered)?

Is the compromised key still in active use?

Notify security team and incident commander

Begin incident documentation (timeline, actions, decisions)

Identify certificate using this key (match modulus):

# Get key fingerprint
openssl rsa -in compromised.key -noout -modulus | md5sum

# Check certificate
openssl x509 -in certificate.crt -noout -modulus | md5sum

# If MD5 hashes match, this cert uses that key

Check Certificate Transparency for rogue certificates:

# Search crt.sh for any certificates issued with your domains
curl "https://crt.sh/?q=%.example.com&output=json" | jq '.[] | {id, issuer_name, not_before}'

# Look for unexpected issuances

💡 Document EVERYTHING - you may need this for compliance/legal

💡 Take screenshots of the exposure if possible

💡 Do not delete evidence of the compromise

2Immediate Actions

Objective: Replace the compromised certificate as fast as possible

Generate NEW private key (NEVER reuse compromised key)

Generate new CSR

Request new certificate from CA (emergency/expedited if available)

Deploy new certificate immediately

Verify new certificate is serving correctly

Generate new key (use ECC for better security):

# ECC P-256 (recommended)
openssl ecparam -genkey -name prime256v1 -out new-key.pem
chmod 600 new-key.pem

# Or RSA 2048
openssl genrsa -out new-key.pem 2048
chmod 600 new-key.pem

Generate CSR with new key:

openssl req -new -key new-key.pem -out new.csr \
  -subj "/CN=example.com/O=Your Org/L=City/ST=State/C=US"

# Verify CSR
openssl req -in new.csr -noout -text

Deploy and reload:

# Copy files to server
sudo cp new-key.pem /etc/ssl/private/
sudo cp new-cert.pem /etc/ssl/certs/

# nginx
sudo nginx -t && sudo systemctl reload nginx

# Apache
sudo apachectl configtest && sudo systemctl reload apache2

3Revoke Compromised Certificate

Objective: Prevent the compromised certificate from being used

Log into CA portal

Locate the compromised certificate

Submit revocation request with reason "Key Compromise"

Confirm revocation (CA must revoke within 24 hours per BR)

Verify certificate shows as revoked

Verify revocation via OCSP (after CA processes):

# Get OCSP URL from certificate
openssl x509 -in old-cert.pem -noout -ocsp_uri

# Query OCSP (you'll need issuer cert)
openssl ocsp -issuer issuer.crt -cert old-cert.pem \
  -url http://ocsp.digicert.com -resp_text | grep "Cert Status"

# Should show: Cert Status: revoked

💡 Per CA/Browser Forum Baseline Requirements, CAs must revoke within 24 hours for key compromise

💡 Revocation is PERMANENT - cannot undo

💡 Keep the old certificate files for audit purposes (but delete the private key)

4Assess Impact

Objective: Understand what damage may have occurred

What could an attacker do with this key?

Was any traffic potentially intercepted?

Check CT logs for unauthorized certificate issuance

Review server/application logs for suspicious activity

Check if key was used for other purposes (code signing, etc.)

💡 With a compromised private key, attacker could: impersonate your server, intercept traffic (MITM), sign malicious content

💡 If key was used for mTLS client auth, attacker could access protected services

💡 Consider resetting any sessions/tokens that may have been established with compromised cert

5Secure the New Key

Objective: Prevent the same exposure from happening again

Restrict file permissions on new key (chmod 600)

Ensure key is owned by appropriate service user

Update .gitignore to exclude key files

Consider HSM or secrets management (AWS KMS, HashiCorp Vault)

Implement key rotation policy

Add secrets scanning to CI/CD pipeline

Secure key file permissions:

chmod 600 /etc/ssl/private/new-key.pem
chown root:ssl-cert /etc/ssl/private/new-key.pem

# Or for nginx user
chown www-data:www-data /etc/ssl/private/new-key.pem

Add to .gitignore:

# Add to .gitignore
*.key
*.pem
*.p12
*.pfx
*private*

Scan git history for leaked secrets:

# Using trufflehog
trufflehog git file://./

# Using gitleaks
gitleaks detect --source .

6Post-Incident

Objective: Document, learn, and prevent recurrence

Complete incident documentation with full timeline

Conduct root cause analysis (5 Whys)

Update security procedures based on lessons learned

Brief relevant stakeholders and management

File compliance reports if required (PCI, HIPAA, etc.)

Schedule follow-up review

Rollback Procedure

If new certificate has issues, you may temporarily use a different certificate while troubleshooting
DO NOT go back to the compromised key/certificate
If CA issues wrong certificate, contact CA support for reissuance

Troubleshooting

Problem: CA won't revoke within 24 hours

Solution: For key compromise, CAs are required to revoke within 24 hours per BR. Escalate with CA support.

Problem: Found unauthorized certificates in CT logs

Solution: Contact the issuing CA immediately - this indicates CA compromise or fraudulent issuance.

Problem: Key was committed to public GitHub

Solution: Use git filter-branch or BFG Repo Cleaner to remove from history, then rotate key. Assume it's compromised.

Problem: Multiple services used the same key

Solution: All services need new certificates. Never share keys between services - each should have unique key pair.

Escalation

Internal: Security team lead, CISO, Legal (if data breach)

CA Support: DigiCert: 1-801-877-2100 | Sectigo: 1-888-266-6361 | GlobalSign: 1-877-775-4562

After Hours: Follow security incident escalation procedures

Related Resources

📖 Certificate Revocation Requirements 📖 Revocation Guide (CRL vs OCSP)🎬 Certificate Revocation Demo 🎬 Certificate Transparency Demo ✅ Emergency Replacement Runbook 🎬 Certificate Lifecycle Demo

← Back to Checklists Library