Skip to main content
When a metric flips unhealthy in the middle of the night, the on-call already knows. The question is whether the customer-facing status page should be updated to reflect that. Auto-incident creation does the typing-out part for you, without ever publishing without a human pressing a button.

​
How it works

  1. You opt a metric in to the feature in its settings (Pro+).
  2. The metric flips unhealthy (after its confirmation time, like any other status change).
  3. The auto-incident worker creates a draft incident on the metric’s bound service.
  4. Observer emails your org owners with two links: Review and publish (flip to published; customers see it) and Dismiss the draft (soft-delete the draft).
  5. If neither button is clicked within 24 hours, the draft auto-expires. Nothing ever reaches the public page without a human action.
Drafts are invisible to customersA draft incident is not visible to the public; your status page renders only published incidents. The draft exists for you to verify and act on. It can be safely dismissed if it turned out to be noise.

​
Enable for a metric

  1. Open Metrics, select your metric, and open its Settings tab.
  2. Open the Incidents tab (Automatic incident creation).
  3. Pick a Policy:
    • Off: auto-creation is disabled for this metric.
    • On: create immediately: a draft is created the moment the metric flips unhealthy.
    • On: wait, then check again: Observer waits the configured number of seconds, then re-checks the metric’s current status. If it’s still unhealthy, the draft is created. If the metric recovered during the wait, nothing happens. This is the recommended setting for metrics that occasionally flap.
  4. Pick a Severity (minor / major / critical). This value is stamped on every auto-drafted incident.
  5. With On: wait, then check again, set Confirmation time (seconds) between 60 and 3600. Defaults to 300 (5 minutes).
  6. Save.

​
What gets created

When the worker fires, you get:
  • A new incident row with:
    • title: Investigating elevated errors on <metric title>
    • severity: as configured on the metric
    • affected_services: every service that has an SLO pointing at the metric
    • An initial Information message describing the value vs the threshold and the timestamp.
  • An audit row (incident.auto_drafted on the metric, plus the parent row on the incident itself).
  • An email to every org owner who hasn’t opted out (see Notification preferences).

​
Email CTAs

Each email has two links:
  • Review and publish: flips the draft to published so customers see it.
  • Dismiss the draft: removes the draft.
Each link opens a confirmation page, and the action runs only when you confirm there (Publish incident or Dismiss draft), so a mail scanner that opens the link changes nothing. Each link is cryptographically signed, bound to a single action, and expires after 24 hours. Editing the URL cannot turn a dismiss into a publish. Both endpoints are idempotent. Re-clicking publish after the incident is already published returns a success page. Re-clicking dismiss after it’s already dismissed returns a success page.

​
Dedup, cooldown, and expiry

Three guardrails keep the auto-incident flow from spamming you:
  1. Dedup against open incidents on the service. If you (or a prior auto-draft) filed an unresolved incident affecting the metric’s service in the last hour, the worker appends a new Information message to the existing incident instead of creating a duplicate. This applies to drafts and published incidents alike. On a published incident the message is public and notifies subscribers like any other update. Message text: Metric <name> is now unhealthy (auto-detected). Value: <value>.
  2. One auto-draft per metric per hour. If a metric was already auto-drafted or auto-dismissed in the last hour, the worker skips. Flapping metrics never produce more than one draft per hour.
  3. 24-hour auto-expiry. Drafts older than 24 hours that haven’t been published or dismissed are soft-deleted by a 15-minute cron and audited as incident.auto_expired.

​
Notification preferences

Per-user opt-out lives at Console > Settings > Notifications > Auto-incident draft emails. Default is ON for org owners. Owners who toggle this off do not receive auto-incident emails (any other type of email is unaffected).

​
Plan gate

This feature is Pro+ only. On Free and Starter plans the Automatic incident creation section of the metric form shows an upgrade note and its fields are read-only. Leave the Policy at Off (the default) on lower plans, or upgrade.

​
Alert destinations

Auto-incident lifecycle events are not currently available as alerts on the Alerts page. Once a draft is published, it sends incident.published to your alert destinations like any incident published from the console. For most teams:
  • On: wait, then check again with 300 seconds for any latency or error-rate metric. The wait catches noisy alarms before they generate an email.
  • On: create immediately for binary signals (TLS expiry hit zero, a service is unreachable). These should not flap, so a wait adds nothing.
  • Leave auto-creation off for noisy dashboards that are not customer-visible. The console already shows unhealthy metrics; not every internal alarm deserves a draft.