Incidents and metrics

Observer’s status model has two layers:

1. Metrics drive status by default. A metric flips to unhealthy when its measured value crosses the threshold; the page status rolls up from the worst metric. No human action required.
2. Incidents are the customer comm layer on top. An incident is what the operator publishes to explain context: what is broken, what we know, what we are doing about it.

Both layers can fire independently, and they often do.

​

Why two layers

A metric flip is an automated signal. The threshold breach happened at 14:32:18 because the agent reported 4.2% errors and the unhealthy rule said over 2%. That is precise, but it is not customer communication. Customers want to know:

• Are you aware?
• What is the impact?
• When will it be fixed?
• How will I know it is fixed?

Those are operator-authored sentences. The metric flip cannot answer them on its own.

​

How they relate at runtime

Metric status drives each metric’s state on the page. The page headline can only get worse from incidents: an open incident raises it to match its severity (minor to degraded, major or critical to unhealthy) and never lowers a measured outage. Resolving the incident removes that escalation. The exception is manual metrics (see Manual metrics): when an open incident lists a service, the manual metrics that service’s SLOs observe auto-set their status to mirror the incident severity. This is the case where incidents drive status (by design) because manual metrics have no probe to measure them.

​

The “draft from metric” flow

When a metric is unhealthy or degraded, the metric edit page surfaces an Open incident button. One click creates a draft that pre-fills:

• Title: Investigating: <metric-title>
• Severity: major if metric is unhealthy, minor if degraded
• Affected services: every service that has an SLO bound to this metric
• Pages: every status page that shows this metric
• Initial message: Investigating <metric>. Current status: <status> (value: <value>).

The operator reviews, edits if needed, and publishes. The “metric flipped → I need to update status” loop drops from minutes to one click. The button is idempotent within 30 minutes: a second click on the same metric in the same window opens the existing draft instead of creating a duplicate.

​

Auto-drafts (opt-in)

The same “draft from metric” path can run automatically. Opt a metric in via the Automatic incident creation section on its edit form (Pro+). When the metric flips unhealthy, Observer creates the draft for you and emails your org owners with publish / dismiss buttons. The auto flow shares the same dedup rule as the manual CTA. If an open incident already affects the metric’s service, a message is appended to the existing incident instead of opening a new one. Per-metric cooldown is one hour. Drafts that go unactioned for 24 hours auto-expire. See the full setup walkthrough at Auto-incident creation.