Change Failure Rate

Definition

Change failure rate (CFR) is the percentage of changes released to production that result in degraded service and require immediate remediation — a rollback, a hotfix, a feature-flag kill, or a forward fix under incident conditions. It is one of the four DORA metrics, sitting alongside deployment frequency, lead time for changes, and time to restore service. Its role is specific: it is the quality counterweight to the two speed metrics.

Why It Matters

Deployment frequency alone is easy to game — ship more, faster, and the dashboard improves while customers suffer. CFR prevents that by asking what proportion of those changes hurt. Read together, the four metrics describe a delivery system honestly: high frequency with low CFR and short restore time indicates genuine engineering capability; high frequency with rising CFR indicates a team outrunning its safety net. The research consistently shows these move together rather than trading off, because the practices that make deployment safe are what make it fast.

How to Measure It Credibly

1. Define failure before measuring. Most teams count any change requiring unplanned remediation within a defined window — commonly 24 hours.
2. Count changes, not incidents. The denominator is deployments to production; one bad deployment causing three alerts is one failure.
3. Derive it from systems, not surveys. Link deployment records to incident and rollback records automatically, or the number becomes an opinion.
4. Segment by service. An aggregate 12% may hide one legacy service at 40% and everything else at 3%.
5. Track the trend, not the number. Direction over quarters is the signal; a single month is noise.
6. Review failures qualitatively. The metric points at where to look; the post-incident review explains why.

Real-World Example

A payments team deploying twice a week reported a 22% change failure rate and used it to argue for a slower, more heavily gated release process. Segmenting the data told a different story. Nineteen of the twenty-three failures that quarter came from one service whose deployments carried database migrations, and in every case the failure appeared at the migration step, not in application code. The team adopted expand-and-contract schema changes, separated migration deployment from code deployment, and added a migration dry-run against a production-shaped copy in CI. Within two quarters CFR fell to 6% while deployment frequency rose to daily. Adding gates would have slowed everything and fixed nothing, because the problem was one specific practice, not the pace.

Practical Lessons Learned

• Aggregate CFR hides the actionable detail. Segment by service, change type and time of day before drawing conclusions.
• Batch size is the strongest lever. Large infrequent releases fail more often and are harder to diagnose when they do.
• Never make CFR an individual performance metric. The instant it is, failures stop being reported and the data dies.
• Feature flags shift the shape of failure. A flag kill is still a failure, but a much cheaper one — count it and celebrate the mechanism.
• A CFR of zero is a warning sign. It usually means under-reporting or a team shipping too conservatively to learn anything.

Expert Tips

• Instrument the link automatically: deployment IDs in incident records, so CFR is a query rather than a spreadsheet exercise.
• Review the failure list, not just the rate, at each retrospective — three failures with the same root cause is a project, not a statistic.
• Publish CFR alongside deployment frequency and restore time. Any one metric in isolation invites the wrong behaviour.
• Distinguish rollbacks from forward fixes in the data; a team that always fixes forward may simply lack a working rollback path.
• Set an improvement target on the trend and the top contributing service, not a blanket organisational threshold.

Common Mistakes

• Counting incidents rather than failed changes, which double-counts noisy failures.
• Reducing deployment frequency to protect the ratio — fewer, larger releases usually make things worse.
• Excluding configuration and infrastructure changes, which are a large share of real failures.
• Using self-reported survey estimates as though they were measurements.
• Attaching the metric to individuals or to bonus schemes, which guarantees silent failures.

Key Takeaways

• CFR is the quality counterweight to deployment frequency and lead time.
• Define failure precisely and derive the number from deployment and incident systems.
• Segment before acting; the aggregate usually conceals one dominant cause.
• Smaller batches, progressive delivery and reliable rollback are the practical levers.
• Use it for system improvement, never for individual evaluation.

Related Concepts

Pairs with Progressive Delivery, Rollback Plan, Schema Change Discipline, and Post-Incident Review.