Postmortems sem culpa

Nível: B2

Objetivo

Aplicar linguagem blameless para escrever uma análise madura de incidente em inglês, identificando contributing factors, o detection gap, action items concretos e medidas de prevenção, sem apontar indivíduos e descrevendo o que o sistema e o processo permitiram.

Explicação

Um blameless postmortem é uma análise de incidente que procura como o sistema e o processo permitiram a falha, não quem errou. O objetivo é aprender e prevenir, não distribuir culpa. Em B2, isso exige vocabulário próprio e padrões de frase consistentes.

1. A mudança de foco: do indivíduo para o sistema

A pergunta base deixa de ser `Who broke it?` e passa a ser `How did the system allow this?`. Frases típicas em B2:

• `The deployment introduced a regression on the checkout path.`
• `The system did not surface the error rate early enough.`
• `The runbook did not cover this scenario.`
• `The alert fired, but the escalation policy did not page the secondary on-call.`

Perceba o sujeito: the deployment, the system, the runbook, the policy. A pessoa é descrita pelo que fez naquele momento, não rotulada.

2. Vocabulário central de um postmortem blameless

• Contributing factor: uma condição que contribuiu para o incidente, sem ser necessariamente a causa única. Incidentes maduros listam vários.
• Detection gap: tempo entre o início do problema e o momento em que um humano ouve o alerta. Reduzir esse tempo é uma métrica de prevenção.
• Action item: uma ação concreta com owner (responsável) e due date (prazo). Sem isso, o postmortem vira documento de gaveta.
• Mitigation: a ação que conteve o incidente em produção (por exemplo, um rollback). Diferente de prevention.
• Prevention: medida que reduz a chance de o incidente se repetir (mudança de processo, alerta novo, teste novo).
• Lessons learned: o que a organização passa a saber depois do incidente. Em B2, vêm listadas e referenciadas por action item.
• Severity: o nível de impacto (geralmente SEV1 a SEV4 em SRE). Define o que entra no documento.
• Timeline: ordem cronológica de eventos com timestamps. Sustenta a análise, mas não a substitui.

3. Linguagem blameless: padrões e substituições

O princípio é: descreva a ação observável e a lacuna de sistema/processo que tornou a falha possível. Pessoas aparecem como autores de action items e de decisões no timeline, nunca como causa isolada.

4. Estrutura padrão de um postmortem blameless em B2

Um documento profissional segue, em ordem:

1. Summary: duas a três frases com o que aconteceu, o impacto e a mitigação. Sem nomes próprios.
2. Impact: métrica objetiva (downtime, error rate, customer-facing incidents, data loss).
3. Timeline: eventos em ordem com timestamps UTC e fatos, não interpretações.
4. Contributing factors: lista numerada, cada item com o sistema ou processo envolvido.
5. Detection gap: quanto tempo entre o início e a detecção, e por que o alerta não chegou antes.
6. Action items: tabela com `action item`, `owner`, `due date`, `type` (mitigation/prevention/process).
7. Lessons learned: bullets curtos, cada um ligado a um action item.

Linguagem útil para abrir cada seção:

• Summary: `On <date>, the <service> experienced a <SEV> incident that lasted <duration> and affected <metric>.`
• Contributing factors: `The contributing factors were: (1) ..., (2) ..., (3) ...`
• Detection gap: `The detection gap was <N> minutes, because <system reason>.`
• Action item: `Action item #N: <verb in base form> <object>, owned by <team>, due by <date>.`
• Prevention: `To prevent recurrence, we will <action>; this addresses contributing factor #N.`

5. Conectores e hedge próprios do postmortem

O tom é factual, mas precisa de hedge para distinguir fato de interpretação:

• `Based on the logs, the regression appeared around 14:02 UTC.`
• `We believe the canary stage would have caught this, but it was skipped because of <reason>.`
• `Going forward, we will ...` em vez de `We must ...`.
• `This addresses contributing factor #2.` referencia de volta, em vez de repetir.
• `On balance, the trade-off here is speed of rollout versus coverage of the canary.` amarra a decisão ao trade-off.

Exemplos

Erros comuns

• Apontar um indivíduo como causa: `Bob shipped a bad change.` Em postmortem blameless, descreva a lacuna do sistema: `The change was not covered by a canary stage.`
• Usar `human error` sem contexto: o leitor profissional sempre pergunta 'qual lacuna do sistema tornou esse erro possível?'.
• Misturar mitigation e prevention: mitigation é o que conteve o incidente em produção; prevention é o que reduz a chance de recorrência.
• Listar contributing factors vagos: `monitoring was bad` deve virar `the alert threshold was set too high, which delayed the page by 40 minutes`.
• Escrever action items sem owner ou due date: `we should monitor better` não é um action item; `add a synthetic check, owned by SRE, due by 2026-08-15` é.
• Confundir root cause com contributing factors: a maioria dos incidentes tem vários fatores; listar um único 'root cause' empobrece a análise.
• Tratar o detection gap como falha pessoal: `Alice took 30 minutes to look at the page` deve ser `the alert reached the on-call 30 minutes after the incident started, because the threshold was set above the affected traffic level`.
• Fechar com lições genéricas: `we learned a lot` deve ser substituído por bullets concretos, cada um referenciando um action item.
• Escrever `we must never let this happen again`: tom categórico demais; o padrão B2 é `going forward, we will ...` com prazo e owner.

Prática guiada

Reescreva cada frase blaming como blameless, preservando o fato.

1. `Carlos deployed a broken change.`

→ `The change was deployed without a canary stage, which allowed the regression to reach production.`

1. `Marina didn't see the alert for 30 minutes.`

→ `The detection gap was 30 minutes, because the alert was configured on a metric that did not move during the incident.`

1. `Human error caused the outage.`

→ `The runbook did not cover the rollback path for this service, which made the diagnosis slower.`

1. `We need to be more careful with deploys.`

→ `Action item #1: add a mandatory canary stage to the release pipeline, owned by the platform team, due by the end of next sprint.`

Repare que, em cada reescrita, o foco sai do indivíduo (`Carlos`, `Marina`, `human error`) e vai para o sistema (canary stage, alert metric, runbook, release pipeline). Esse é o movimento central de um postmortem blameless.

Resumo

Um postmortem blameless descreve como o sistema e o processo permitiram a falha, sem apontar indivíduos como causa. O vocabulário central em B2 é contributing factor, detection gap, action item, mitigation, prevention, lessons learned e timeline. Linguagem blaming (`Bob broke it`, `human error`, `we must be more careful`) é substituída por padrões factuais: `the deployment introduced a regression`, `the alert fired on the wrong metric`, `the runbook did not cover this path`, `action item #N: <verb> <object>, owned by <team>, due by <date>`. O documento segue summary → impact → timeline → contributing factors → detection gap → action items → lessons learned, sempre referenciando o número do contributing factor ou action item. O tom é factual, com hedge (`based on the logs`, `we believe`, `going forward`), e fecha com um trade-off explícito quando a decisão causou parte da falha.

História: Postmortems sem culpa

Nível: B2

História em inglês

The checkout incident

On the morning of July 2nd, the checkout service started returning 500 errors for about 12% of the requests. The on-call at the time, an engineer named Priya, was paged by a dashboard alert, not by a threshold-based alert. She opened the runbook, but the rollback path described there was three versions behind, and the steps no longer matched the current deployment. After thirty-eight minutes of diagnosis, the team decided to roll back the change that had shipped two hours earlier. The rollback brought the error rate back to zero within two minutes, and the incident was declared mitigated at 14:58 UTC.

The first blameless draft

Two days later, the team met to write the postmortem, and the goal was to keep the language blameless from the first sentence. The first draft said, "Priya took thirty minutes to find the right rollback command," and the reviewer changed it to, "The detection gap was thirty minutes, because the alert did not include a direct link to the runbook." Another sentence blamed a developer named Tomás for skipping the canary stage, and the reviewer rewrote it as, "The release pipeline marked the canary stage as optional, so the regression was not caught before full rollout." The team agreed that human error was not a useful label, because the same person, with the same runbook, would have made the same decision under the same alerts. They replaced every sentence that named a person with a sentence that named a system, a process, or a document.

Contributing factors and the detection gap

By the end of the meeting, the team had listed three contributing factors. Contributing factor #1 was that the canary stage had been marked as optional in the release pipeline about six months earlier, when the team was trying to speed up deploys. Contributing factor #2 was that the alert was configured on the average error rate over five minutes, which masked a sharp spike in a smaller subgroup of requests. Contributing factor #3 was that the runbook had not been updated since the previous major migration, so the rollback path was out of date. The detection gap was thirty-eight minutes, which matched the time from the first failed request to the moment Priya opened the runbook. The team wrote that the alert reached the on-call, but the alert did not include a direct link to the rollback runbook, and the runbook itself was out of date.

Action items and prevention

The team then wrote four action items, each with an owner, a due date, and a link to the contributing factor it addressed. Action item #1: make the canary stage mandatory in the release pipeline, owned by the platform team, due by the end of next sprint; this addresses contributing factor #1. Action item #2: reconfigure the alert to fire on a per-endpoint error rate, owned by the observability team, due by the end of this month; this addresses contributing factor #2. Action item #3: update the rollback runbook and link it from every alert, owned by the SRE team, due by the end of next sprint; this addresses contributing factor #3. Action item #4: add a synthetic check on the checkout path, owned by the SRE team, due by 2026-08-15; this is a prevention measure that should reduce the detection gap to under five minutes. The team closed the document with a short lessons learned section, listing what the organization now knows and what it will do differently going forward. On balance, the trade-off that contributed to the incident was speed of rollout versus coverage of the canary stage, and the postmortem made that trade-off explicit instead of hiding it.

Tradução linha por linha

Notas de vocabulário

• The on-call at the time, an engineer named Priya, was paged by a dashboard alert = Pessoas aparecem como autores de decisões no timeline, não como causa do incidente; o que falhou foi o alerta baseado em limiar.
• The detection gap was thirty minutes = Detection gap sempre vem com um número; é a métrica que o time quer reduzir com action items.
• The release pipeline marked the canary stage as optional = Sujeito: the release pipeline. A frase descreve uma lacuna de processo, não uma decisão pessoal.
• human error was not a useful label = Rótulo genérico que um postmortem blameless evita; em vez dele, descreve-se a lacuna do sistema.
• Contributing factor #1 was that the canary stage had been marked as optional = Contributing factor numerado e datado; preserva o fato (marcado como opcional cerca de seis meses antes).
• the alert did not include a direct link to the rollback runbook = Lacuna concreta do sistema de alertas; serve de base para o action item #3.
• Action item #1: make the canary stage mandatory in the release pipeline, owned by the platform team, due by the end of next sprint = Estrutura canônica de action item: verbo no infinitivo, owner, due date e link ao contributing factor.
• this addresses contributing factor #1 = Fechar o action item com referência ao número do factor evita repetir a explicação.
• a synthetic check on the checkout path = Synthetic check é uma prevention measure: detecta a falha antes que vire incidente.
• On balance, the trade-off that contributed to the incident was speed of rollout versus coverage of the canary stage = Fechar com trade-off explícito mostra maturidade; o postmortem não finge que havia escolha óbvia.

Perguntas de compreensão

1. What metric did the alert reach the on-call on, and why did that delay the page?
2. How long was the detection gap, and what did the team decide to do about it?
3. Why was the runbook described as out of date?
4. What did the reviewer change in the sentence that originally named Priya?
5. What did the reviewer change in the sentence that originally blamed Tomás for skipping the canary stage?
6. List the three contributing factors in the order the team wrote them.
7. Which action item addresses contributing factor #2, and what does it ask for?
8. Why did the team decide that "human error" was not a useful label?
9. What trade-off did the team name as contributing to the incident?
10. What did the team say they would do differently going forward?

Prática com a história

1. Reescreva em blameless: `The on-call took thirty minutes to find the rollback command.`
2. Classifique: `We will add a synthetic check on the checkout path; this addresses contributing factor #2.`
3. Preencha: The ___ gap was thirty-eight minutes, because the alert fired on the average error rate over five minutes.
4. Reescreva o action item em forma compacta: `Action item #1: tornar o estágio canário obrigatório no pipeline de release, responsabilidade do time de plataforma, prazo para o fim do próximo sprint.`
5. Escreva uma frase blameless de Summary para o incidente do story (SEV2, checkout, 38 minutos, 12% de error rate, mitigado por rollback).