Metrics

Test a probe

A key with write:metrics or write:config may call it. Validates source_config with the same rules as a metric write, then runs the probe once: on the given agent (on its network, with its env-var secrets), or, with no agent_id, from Observer’s cloud for a public HTTP check. Waits up to wait_seconds (default 20) for the agent; if it has not answered by then the response is 202 with a job to poll via getProbeTest. Nothing is stored: no metric, no sample, no status change. The result carries only the value, its unit, the status the given thresholds would produce, a reason code and the duration; never response bodies, rows or headers.

Value units by source_type:

prometheus: Value: scalar result of the PromQL query, as returned by the query; Observer does not rescale it.
http: Value: ms: response time in milliseconds. With json_path set: the number extracted from the JSON response at json_path (booleans read as 1/0); unit as in the response.
tcp: Value: ms: TCP connect time in milliseconds.
dns: Value: ms: DNS resolve time in milliseconds.
icmp: Value by interpretation: interpretation=latency (default) → ms: average round-trip time in milliseconds; interpretation=packet_loss → %: percentage of pings lost, 0 to 100; interpretation=reachability → 1 when any ping returned, 0 when none did.
tls_cert: Value: days: days until the certificate expires.
grpc: Value by interpretation: interpretation=health_state (default) → 1 when the health check reports SERVING, 0 when NOT_SERVING; interpretation=latency → ms: health check round-trip time in milliseconds.
websocket: Value by interpretation: interpretation=handshake_latency (default) → ms: handshake time in milliseconds; interpretation=round_trip_latency → ms: message round-trip time in milliseconds; interpretation=connection_success → 1 when the socket opened, 0 when it did not.
mtls_http: Value: ms: response time in milliseconds (deprecated alias of http). With json_path set: the number extracted from the JSON response at json_path (booleans read as 1/0); unit as in the response.
database: Value: scalar returned by the read-only query (one row, one column), as returned by the query; Observer does not rescale it.
otlp: Value: aggregated value of the pushed OTLP data points, in the instrument’s own unit.
cloudwatch: Value: latest CloudWatch datapoint for the chosen statistic, in the metric’s CloudWatch unit.
custom: Value: number returned by the registered probe function, in whatever unit it uses.
loki: Value: scalar result of the LogQL aggregation, as returned by the query; Observer does not rescale it.
elasticsearch: Value: value of the named aggregation, as returned by the query; Observer does not rescale it.
host: Value by metric: metric=cpu (default) → %: CPU utilisation percentage, 0 to 100; metric=memory → %: used memory percentage, 0 to 100; metric=filesystem → %: used disk space percentage of the mountpoint, 0 to 100; metric=network → B/s: network throughput in bytes per second (received + sent); metric=load → 1-minute load average divided by the CPU count.
manual: Value: no measured value; the status is set by hand or by an open incident.
heartbeat: Value: s: seconds since the last successful ping.
alert: Value: worst firing alert as a score: 0 none, 1 degraded, 2 unhealthy.

POST

metrics

test

Run a draft probe once and return the value it measures, without creating a metric

curl --request POST \
  --url https://use.observer/api/v1/metrics/test \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "source_config": {}
}
'

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({source_config: {}})
};

fetch('https://use.observer/api/v1/metrics/test', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://use.observer/api/v1/metrics/test"

payload = { "source_config": {} }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "value": 123,
  "unit": "<string>",
  "status": "healthy",
  "duration_ms": 123,
  "mode": "agent",
  "value_description": "<string>",
  "reason": "<string>",
  "reason_label": "<string>",
  "reason_hint": "<string>",
  "job_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "thresholds_complete": true,
  "skipped_secret_headers": [
    "<string>"
  ]
}

{
  "job_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "status": "pending",
  "poll": "<string>",
  "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "expires_at": "2023-11-07T05:31:56Z",
  "detail": "<string>"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

Authorizations

Authorization

string

header

required

An organization API key (obs_pub_…) sent as Authorization: Bearer <key>. Each key carries scopes; an endpoint lists the scope it needs. read:change_events is reserved: keys can carry it, but no endpoint needs it yet.

Body

application/json

source_type

enum<string>

required

The probe type. custom, otlp (push), manual, heartbeat and alert cannot be tested.

Available options:

prometheus,

http,

tcp,

dns,

icmp,

tls_cert,

grpc,

websocket,

mtls_http,

database,

cloudwatch,

loki,

elasticsearch,

host

source_config

object

required

The same object a metric's source_config takes for this source_type (see the SourceConfig schemas used by applyConfig); validated the same way before anything runs.

agent_id

string<uuid> | null

The Observer Agent to run the test on (listAgents). It must be online and run agent 1.7.0 or later. Omit it to test a public HTTP check from Observer's cloud instead (header_refs secrets are not sent then); every other source type needs an agent.

thresholds

object

Optional. With all four set, status says what the metric's status would be. Strict comparisons: over is >, under is <, equal is =.

Show child attributes

wait_seconds

integer

default:20

How long to wait for the agent's answer before returning 202 with a job to poll. Cloud tests always answer at once.

Required range: 0 <= x <= 20

Response

the probe ran

value

number | null

required

What the probe measured, in unit. null when it got no value (see reason).

unit

string | null

required

Unit of value (ms, %, days, s, B/s), or null for a bare number. Same rules as a metric's unit.

status

enum<string> | null

required

The status these thresholds give the value; no_data when the probe got no value. null when a value was measured but thresholds were not all given.

Available options:

healthy,

degraded,

unhealthy,

no_data,

null

duration_ms

integer

required

How long the probe took, in milliseconds.

mode

enum<string>

required

agent: ran on the agent's network with its secrets. cloud: a public HTTP check run from Observer's cloud.

Available options:

agent,

cloud

value_description

string | null

What the value is, in words.

reason

string | null

Why there is no value: the agent's reason code (e.g. ETIMEDOUT, unexpected_status:503). null on success.

reason_label

string | null

Short human description of reason.

reason_hint

string | null

What usually causes reason and how to fix it.

job_id

string<uuid> | null

The agent job (agent mode only).

agent_id

string<uuid> | null

thresholds_complete

boolean

skipped_secret_headers

string[]

Cloud mode only: header_refs names that were not sent, because their values live on an agent host.

Run a draft probe once and return the value it measures, without creating a metric

curl --request POST \
  --url https://use.observer/api/v1/metrics/test \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "source_config": {}
}
'

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({source_config: {}})
};

fetch('https://use.observer/api/v1/metrics/test', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

import requests

url = "https://use.observer/api/v1/metrics/test"

payload = { "source_config": {} }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

{
  "value": 123,
  "unit": "<string>",
  "status": "healthy",
  "duration_ms": 123,
  "mode": "agent",
  "value_description": "<string>",
  "reason": "<string>",
  "reason_label": "<string>",
  "reason_hint": "<string>",
  "job_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "thresholds_complete": true,
  "skipped_secret_headers": [
    "<string>"
  ]
}

{
  "job_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "status": "pending",
  "poll": "<string>",
  "agent_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "expires_at": "2023-11-07T05:31:56Z",
  "detail": "<string>"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

{
  "type": "https://docs.use.observer/api/getting-started/errors#forbidden",
  "title": "forbidden",
  "status": 403,
  "detail": "missing scope: write:incidents"
}

Getting started

Account

Status pages

Services

Metrics

SLOs

SLA statements

Incidents

Maintenances

Config as code

Agents

Change events

Audit log

Test a probe

Authorizations

Body

Response