> ## Documentation Index
> Fetch the complete documentation index at: https://docs.use.observer/llms.txt
> Use this file to discover all available pages before exploring further.

# Thresholds and confirmation time

> How a metric's status is decided, and how confirmation time prevents flapping.

A metric's status flips through three layers in order:

1. **Threshold evaluation**: the strict-operator rule applied to
   each pushed value (see
   [threshold operators](https://docs.use.observer/docs/reference/threshold-operators.md)).
2. **Confirmation time**: a status only flips after holding the new
   state for the configured time.
3. **Snooze (optional)**: a metric can be snoozed for a window so
   it does not affect status pages or fire webhook events while
   operators tune it.

This page covers steps 2 and 3.

## Why confirmation time exists

A naive implementation would publish every status change the
agent reports. In practice, metrics flap. A network blip pushes
one bad sample, the next sample is fine, the on-call gets paged
twice per minute.

Confirmation time requires the new status to hold for a minimum
duration before propagating. Configure two values per metric:

* **Confirmation time, getting worse**: how long the metric must
  report the new status before flipping into a worse band
  (healthy to degraded, or healthy to unhealthy). API and config
  field: `dwell_seconds_to_breach`.
* **Confirmation time, recovering**: how long the metric must report
  the new status before flipping into a better band (unhealthy to
  healthy). API and config field: `dwell_seconds_to_recover`.

The defaults are conservative: 180 seconds getting worse, 300
seconds recovering. Heartbeat metrics and inbound alert metrics
default to 0 for both: a heartbeat already has its grace period, and
an alert sender already debounces before it fires. Asymmetric values
("quick to flag, slow to recover") are appropriate for systems where
premature recovery announcements have a higher cost than a delayed
unhealthy alert.

Both values are editable per metric under **Thresholds** >
**Missing data and confirmation time** in the metric's settings, from
0 to 3600 seconds. A confirmation time of 0 flips on the next
confirming sample. Configuration-as-code documents accept the same
values through the optional `dwell_seconds_to_breach` and
`dwell_seconds_to_recover` keys on each metric entry, and the API
returns them on the metric detail endpoint.

Transitions involving `no_data` or `unknown` always use the
recovering confirmation time, in either direction. A single missed poll never declares a
metric down or up.

**Confirmation runs on the clock.** An agent stores a reading when the
status changes and on its regular report, not on every poll. So
Observer checks pending changes every 15 seconds. Once the
confirmation time has passed with no contrary reading, the status
flips about a minute later (a short grace for late readings), without
waiting for the next report.

**Late readings don't page.** An agent that was offline sends its
buffered readings when it reconnects. They are recorded in history, but
an outage that already ended inside that backlog pages no one. If the
problem is still happening when the readings arrive, you get one
alert.

## Status sources

Three statuses come from the strict-operator rule:

* `healthy`
* `degraded`
* `unhealthy`

Two outcomes come from the collection layer, not from values:

* `no_data`: the agent attempted a probe but produced no value.
  The reason code is recorded alongside (`ECONNREFUSED`, `ETIMEDOUT`,
  `no_data_for_query`, etc.; see [Reason codes](https://docs.use.observer/agent/reference/reason-codes.md)).
* `no_data` with reason `stale_agent_silent`: no push arrived
  within the staleness cutoff.

Both are operational signals and surface in the agent dashboard.
Probe-side `no_data` fires `metric.no_data`; staleness does not (see
below). Probe-side `no_data` is not free with respect to SLOs,
though: unless the metric's no-data treatment folds it to `healthy`
or `degraded`, `no_data` samples burn under the default `down` unknown-data policy.
A stale metric does not burn (see below). See
[SLOs and error budgets](https://docs.use.observer/docs/concepts/slos-and-error-budgets.md).

## Stale data tolerance

Confirmation time handles the small flaps. A separate read-time rule
handles a much larger gap: what happens when no sample arrives at
all, because the agent is crashed, the network between the agent
and Observer Cloud is partitioned, or Observer Cloud itself is
degraded.

A metric is **stale** when its last push timestamp is older than a
cutoff derived from its push interval: three times the interval,
capped at 15 minutes, except that the cap never falls below one
minute past the next expected push. The push interval is the
metric's **Report at least every** setting (`interval_agent_push`):
5 to 60 minutes, 15 by default, with a plan floor of 10 on Free and
Starter.

```
cutoff = min(3 × push_interval, max(15 min, push_interval + 1 min))
stale  = (now - last_push_timestamp) > cutoff
```

Worked through:

| Push interval | Cutoff |
| - | - |
| 5 to 14 minutes | 15 minutes (the cap) |
| 15 minutes (the default) | 16 minutes (push + 1) |
| 20 minutes | 21 minutes (push + 1) |
| 60 minutes | 61 minutes (push + 1) |

A metric with no push interval set is treated as 15 minutes.

The 3× multiplier gives the agent one full retry-and-backoff window
before a missing push is considered a problem. The 15-minute cap
stops a moderately slow cadence from masking an outage. The
`push_interval + 1` term keeps a genuinely slow cadence from being
declared stale before its next push is even due: a metric that
pushes hourly is not blind at 15 minutes.

Staleness is enforced on both sides. A sweeper stamps the stored
status to `no_data` with reason `stale_agent_silent` once the cutoff
passes, and the status-page and embed renderers apply the same rule
at read time, so a page loaded between sweeps still reads correctly.

A stale metric is **excluded** from the service rollup. It is **not**
counted toward SLO burn. The `metric.status_changed` and
`metric.no_data` webhooks are **not** fired on staleness transitions.
What does fire: `agent.lag_high` and `agent.offline`, which speak
to the actual cause. They are operator-facing.

When every metric on a service is stale, the service renders as
`monitoring_delayed` with a "Last known: Operational" caption
alongside. See
[Observer availability](https://docs.use.observer/docs/concepts/observer-availability.md) for
the full trust contract.

## Snooze

A metric can be snoozed until a future timestamp (sometimes called
shadow mode). While snoozed:

* The metric still pushes status to the cloud.
* Status pages do not consume the snoozed metric in the
  rolled-up page status.
* Webhook events for the snoozed metric are suppressed.
* The metric's history is still recorded for later inspection.

Use snooze when introducing a new metric, tuning its threshold, or
rolling out a new probe runtime. Once the metric behaves as
expected, end the snooze and it joins the public status surface.

> [!WARNING]
>
> **Strict everywhere**
>
> Threshold evaluation is strict in both the agent and in the read
> path that renders status pages. The same value cannot flip
> status depending on which surface read it. See the
> [threshold operators reference](https://docs.use.observer/docs/reference/threshold-operators.md)
> for examples.

For the full picture of how these verdicts become the page badge,
the history bars, and the SLO budgets, see
[How status is calculated](https://docs.use.observer/docs/concepts/how-status-is-calculated.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.