Observer
Observer

Grafana IRM

Fire and resolve a Grafana IRM alert from Observer via an incoming webhook.

Observer sends incidents to Grafana IRM (Incident Response and Management) through a custom incoming-webhook integration. One Observer incident maps to one Grafana IRM alert.

Set up#

  1. Create a Webhook integration#

    In Grafana IRM, add an Integration of type Webhook. Grafana issues a unique URL for it, for example https://<stack>.grafana.net/integrations/v1/webhook/<token>/.

  2. Add the integration in Observer#

    In the console, open Alerts, choose Add alert, pick Grafana IRM, and paste the integration URL. The URL embeds a token, so it is stored encrypted. Choose org-wide or a single page, then save.

  3. Test it#

    Use Test on the row. A test alert should appear in Grafana IRM. Resolve it there when you are done.

Lifecycle mapping#

Observer eventGrafana IRM action
Incident publishedFire an alert (grouped by alert id)
Incident update postedFire on the same alert id
Incident resolvedResolve the grouped alert

Correlation is by alert_uid, derived from the Observer incident id (observer-incident-<id>). New events group onto that alert; resolution posts to the integration URL's resolve action for the same id, so the lifecycle produces one alert that fires and then resolves.

Troubleshooting#

  • No alert appears: confirm the URL is the full integration webhook URL, including the trailing token segment, and that the integration is enabled in Grafana IRM. A 4xx is surfaced in the Observer delivery result.
  • Resolve does not close the alert: the resolve posts to the same URL with a resolve action. Make sure the integration is a Webhook integration that accepts incoming alerts.
Was this page helpful?