Observer availability

A status page exists to be honest with your customers. If the status page itself becomes a source of misinformation when the monitoring infrastructure has a bad day, it is worse than no status page at all. This page describes what Observer does when Observer itself is degraded, when the agent in your network goes silent, or when the link between them is broken.

​

The two failure modes that look identical

A metric ends up without a recent value for two reasons. They look identical in the database; they mean very different things.

• The probe ran and got nothing. The agent reached out to your Prometheus, your HTTP endpoint, your TLS certificate, and could not produce a value. The query returned empty, the connection was refused, the TLS handshake failed. This is a real signal about your service. It is no_data with a reason code.
• The agent has not pushed anything recently. The agent crashed. The host running it lost the network. Observer Cloud could not accept the push. This is a signal about our or your monitoring infrastructure, not your service. It is stale.

We treat these two cases differently.

​

What “stale” means

A metric is stale when its last push timestamp is older than three times its push interval, capped at 15 minutes. The 3× allows for the agent’s normal retry-and-backoff window before declaring a problem. The 15-minute cap prevents a slow push cadence from masking a multi-hour outage. The cap never falls below one minute past the next expected push, so a metric with a long push interval is not marked stale before its next push is due (see Thresholds and confirmation time). Staleness is applied at read time, and a periodic sweep also marks the stored status as no_data with reason stale_agent_silent, so every reader agrees.

​

What happens when your agent stops pushing

Every metric driven by that agent becomes stale within minutes. Your customer-facing status page does not flip to red. Each stale metric is excluded from the live rollup. If only some metrics on a service are stale, the service rollup uses the fresh metrics and shows a small “X of N delayed” caption. If every metric on a service is stale, the service renders as Monitoring delayed, with a muted “Last known: Operational” (or whatever it was) pill alongside. Your SLOs do not burn during the stale window. Observer’s SLO calculation counts samples, not wall-clock time, so a quiet agent contributes zero to both the numerator (good samples) and the denominator (total samples). The error budget reads No data until the agent resumes pushing; it neither burns nor recovers. We do not fire metric.status_changed or metric.no_data webhooks on staleness transitions. We do fire agent.lag_high and agent.offline. Those are operator-facing and tell you the actual cause.

​

What happens when Observer Cloud is degraded

The agent’s local buffer holds up to 10,000 readings by default (BUFFER_MAX_ROWS), evicting the oldest first. When the cloud receiver recovers, the agent drains the buffer in order. Status pages catch up to the real customer state as the backlog clears. While the cloud is degraded:

• The same staleness rule applies: as time without a fresh sample exceeds the threshold, services on the page roll up to Monitoring delayed rather than flipping to red.
• No false-positive webhooks fire. The operator-facing agent.lag_high event tracks the cloud-side outage from each agent’s vantage point.

​

Will my SLO burn during these gaps?

No. Observer’s SLO computation is sample-counting, not time-counting. The error budget is (good_samples - target × total_samples) / ((1 - target) × total_samples) × 100. When the agent is silent, no samples are added to either side of that fraction, and the budget reads No data until the agent resumes pushing. When the agent resumes, the new samples land in the same window and contribute on their own merits. A short gap during which your service was actually unhealthy does not get retroactively counted as a healthy window. There is just no data for it. This trade-off has a name. Observer is explicit about absence: when we cannot say, we say so. We do not infer healthy minutes between samples and we do not infer unhealthy minutes either.

​

Why pages show Monitoring delayed

Pages render Monitoring delayed when we cannot confirm health. The last-known status sits alongside in a muted pill so your customer can see the trajectory rather than a blank red square. We would rather say “we don’t currently know” than guess.