Operational procedures for common scenarios.
Structure
Each runbook includes:
- Trigger โ When to use this runbook
- Prerequisites โ Access, tools needed
- Steps โ Detailed procedure
- Verification โ How to confirm success
- Rollback โ How to undo if needed
- Escalation โ When and who to contact
Runbooks
Incidents
- โ Service Outage Response โ generic outage triage, mitigate-before-investigate decision tree
- โ Database Failover โ promote replica to primary, engine-specific gotchas
- โ Rollback Deployment โ revert prod to last known good, schema-migration tax
- โ Certificate Rotation โ TLS cert renewal including emergency expiry response
Operations
- โ Scaling a service
- โ Database maintenance
- โ Log rotation
- โ Backup verification
Security
- โ Credential Rotation โ secret rotation, scheduled vs compromise differentiation
- โ Security incident response
- โ Access revocation
Template
# [Runbook Title]
## Trigger
When to use this runbook.
## Prerequisites
- [ ] Access to X
- [ ] Tool Y installed
## Steps
1. First step
command here
Expected output: ...
2. Second step
...
## Verification
How to confirm the issue is resolved.
## Rollback
How to undo changes if needed.
## Escalation
- First: @on-call
- Then: @team-lead
- Finally: @incident-commander