> ## Documentation Index
> Fetch the complete documentation index at: https://docs.use.observer/llms.txt
> Use this file to discover all available pages before exploring further.

# Heartbeat payload

> JSON shape the agent posts to /api/agent/heartbeat.

The agent emits a heartbeat to the cloud on a fixed interval
(typically every 30 seconds). The payload is the agent's view of
its own runtime state. The cloud uses the payload to compute
24-hour uptime, restart counts, and lag alerts.

## Endpoint

```
POST <CLOUD_SERVER_URL>/api/agent/heartbeat
Authorization: provided via Agent-Key header (key transport detail)
Content-Type: application/json
```

## Body

```json theme={null}
{
  "version": "1.2.3",
  "uptime_seconds": 12345,
  "buffer_size": 0,
  "buffer_oldest_age_seconds": 0,
  "queue_depth": 0,
  "queue_oldest_age_seconds": 0,
  "queue_capacity": 10000,
  "agent_started_at": "2026-05-09T12:00:00Z",
  "source_types_active": ["prometheus", "http", "tcp"],
  "custom_probes": [
    { "name": "queue_lag", "has_config_schema": false }
  ],
  "build": {
    "channel": "official",
    "commit": "0a1b2c3d4e5f",
    "source_hash": "9f2c4a..."
  },
  "hostname": "ip-10-0-3-17"
}
```

## Field reference

| Field | Type | Meaning |
| - | - | - |
| `version` | string | Build-time version of the agent. |
| `uptime_seconds` | integer | Wall-clock seconds since process start. |
| `buffer_size` | integer | Deprecated alias for `queue_depth`; kept for older cloud versions. |
| `buffer_oldest_age_seconds` | integer | Legacy alias for `queue_oldest_age_seconds`. |
| `queue_depth` | integer | Pushes currently waiting in the local queue. |
| `queue_oldest_age_seconds` | integer | Age of the oldest pending push, in seconds. |
| `queue_capacity` | integer | Hard cap on the queue (`BUFFER_MAX_ROWS`). |
| `agent_started_at` | ISO timestamp | When the process started. The cloud uses changes to this value to detect restarts. |
| `source_types_active` | string\[] | Distinct source types of the metric definitions currently assigned to the agent. |
| `otlp_stats` | object | OTLP receiver counters, sent when the agent runs the receiver: `running`, `listen_addr`, `bearer_required`, `data_points_received`, `data_points_dropped`, `requests_authenticated`, `requests_rejected_auth`, `requests_rejected_payload`, `unique_streams`, `active_subscriptions`. Omitted otherwise. |
| `custom_probes` | object\[] | Custom probes registered on the agent, each with `name` and `has_config_schema`. The probe function and its `description` are never sent (agents before 1.7.0 also sent `description`; Observer discards it). Sent as an empty list when none are registered, so removing a probe clears it in the console; older agents omit the field. The console fills its custom probe list from this field. |
| `build` | object | Build provenance (agent 1.5.0 and later). `channel` is `official` when the process runs the published image, `source` when it runs from a checkout. `source_hash` is a deterministic hash of the agent's source tree; `commit` is the git commit the official image was built from. See [Build provenance](https://docs.use.observer/agent/concepts/build-provenance.md). |
| `hostname` | string | The agent host's name, as the operating system reports it. Shown on the agent detail page. Older agents omit it. |

## Response

The cloud acknowledges a heartbeat with a small JSON body:

```json theme={null}
{
  "ok": true,
  "uptime_pct_24h": 99.9,
  "restart_count_24h": 0,
  "definitions_version": "5f0c2a9e41b7d3c8a6e1f2b4c9d0e7a3"
}
```

`definitions_version` is an opaque string that changes whenever the
metric definitions assigned to this agent change: a metric is
created, edited, deleted, or moved to or from this agent. The
`GET /api/agent/metrics-definitions` response carries the same value
in its `X-Observer-Definitions-Version` header. When a heartbeat
returns a version that differs from the one of the agent's last
fetch, the agent fetches its definitions again right away, so a
change in the console reaches the agent within about one heartbeat
interval (30 seconds). Agent 1.6.0 and later act on it.

When two processes share one agent key, the response to the process
that lost ownership is `{ "ok": true, "fenced": true }` with no other
fields. See [Duplicate key detection](#duplicate-key-detection).

The `definitions_version` field is optional. An agent that does not recognise it, or a
cloud that does not send it, falls back to the periodic fetch:
every 30 seconds for the first 10 minutes after start while the
agent has no definitions, then every 5 minutes. Agents older than
1.6.0 fetch every 5 minutes.

## Lag and uptime alerts

The queue and uptime numbers in the heartbeat drive two alerts:

* **`agent.lag_high`**: opens when `queue_depth` is above 1000 or
  `queue_oldest_age_seconds` is above 300.
* **`agent.uptime_degraded`**: opens when uptime over the last
  24 hours falls below 95%.

These alerts surface on the agent detail page and as alert events
to any destination on the Alerts page that subscribes to them. A
separate `agent.offline` event fires when the cloud stops receiving
heartbeats from the agent.

## Duplicate key detection

The agent sends its start time in an `Agent-Started-At` header on
every request. When two processes share one agent key, the cloud
treats the most recently started process as the owner:

* Heartbeats from the older process are acknowledged but ignored.
  They do not count toward uptime and do not update the dashboard.
* Pushes from the older process are rejected with HTTP 409. The
  agent drops the rejected samples instead of retrying and logs a
  warning at most once every five minutes.
* The dashboard shows a duplicate key warning on the agent, and an
  `agent.duplicate_key` alert event fires (state `open`) to any
  destination on the Alerts page that subscribes to it. Once the
  older process goes quiet for ten minutes, the warning clears and a
  `cleared` event follows.

Restart detection is monotonic: only a strictly newer
`agent_started_at` records a restart. There is one escape hatch:
when the process holding the newer start time stops heartbeating
for ten minutes, a fenced process may take over as the owner. This
recovers a single agent whose clock moved backwards across a
restart without operator action. To resolve a duplicate key
situation, stop one of the two processes, or rotate the key and give
each deployment its own agent.

## Versioning

Field additions to the payload are additive. Older agents that omit
newer fields stay compatible; older cloud versions ignore fields
they do not recognise.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.