Skip to main content
Heartbeat checks expect your job to call a ping URL on Observer. A job in a private network with no internet access can ping the agent instead, and the agent forwards the ping over its own outbound connection. Use the relay when a cron job, batch task or Kubernetes CronJob can reach an Observer agent but cannot reach the internet. Jobs that can reach the internet should keep calling the ping URL directly.
The heartbeat relay requires agent 1.6.0 or later.

​
Enable the relay

The relay is off by default. Enable it with environment variables on the agent:
Agent environment
See Environment variables for the full list.

​
Point the job at the agent

The paths are the same as on the ping URL, with the agent’s address in front. Replace <agent-host> with the address of the agent and <token> with the check’s ping token:
Job script
The check’s Send pings section in the console has a Via agent tab with the relay URL and these snippets filled in for that check.

​
URL forms

The relay also accepts the /api/heartbeat/<token> prefix, so a job that already uses the ping URL path only needs its host changed. GET, POST and HEAD all work. For a POST, the first 10 KB of the request body is kept as the run’s output.

​
Responses

The agent answers 202 as soon as the ping is queued. The agent cannot tell whether a token is valid, so a well-formed ping always gets 202. An unknown token, or a token for a check in another organization, is dropped by Observer and logged by the agent as not_found. Pings show up in the check’s history marked as sent via agent; if a ping does not appear, check the agent log for not_found.

​
Delivery

  • Pings wait in a durable local queue, a file next to the agent’s buffer, holding up to 5000 pings. They are delivered in order, so a short outage between the agent and Observer loses nothing.
  • Each ping keeps the time the agent received it, so a delayed delivery does not make the check look late.
  • Pings delayed by more than an hour are recorded as one hour old.
  • If the agent stops, relayed checks go late like any missed run, and the agent offline alert fires.

​
Limits

  • 60 pings a minute per token.
  • 600 pings a minute in total.
Tokens never appear in full in the agent’s logs.

​
Security

  • The relay listens on 127.0.0.1 unless you set HEARTBEAT_RELAY_HOST.
  • It has no bearer token. The ping token is the only credential, the same as on the ping URL, and the relay returns nothing but a status code.
  • The relay forwards only what a job sends it: the ping, its exit code and up to 10 KB of request body.
  • Keep a non-loopback relay inside your private network, for example behind a cluster-internal Kubernetes Service. Do not publish the port to the internet.