Observer Docs

When a probe runs but gets no value, the metric reports no_data with a reason code. The console shows the label and hint below in place of the code. The API returns the same three fields on GET /metrics/{id}: reason is the stable code to branch on, reason_label the short description and reason_hint what usually causes it. POST /metrics/test returns them for a one-off test. Codes are case-sensitive. A failed sample has no value, so current_value is null while a metric reports no_data. For a step-by-step diagnosis, see Metric shows no data.

Network and TLS

HTTP, TCP, DNS, TLS certificate and Prometheus checks report these when the connection itself fails. The agent normalises its runtime’s error codes into this set. Loki and Elasticsearch report their own loki_unreachable and es_unreachable instead. A refused port reports ECONNREFUSED and a hostname that does not resolve reports ENOTFOUND; older agents report both as ConnectionRefused (see Codes from older agents).

Code	Meaning	What to check
`ECONNREFUSED`	Connection refused	The target refused the TCP connection. Service down, wrong port, or firewall.
`ETIMEDOUT`	Request timed out	The probe didn’t finish within the configured timeout.
`ENOTFOUND`	DNS lookup failed	The hostname could not be resolved.
`EAI_AGAIN`	DNS lookup temporarily failed	The resolver returned a transient error. Usually self-heals on the next tick.
`ECONNRESET`	Connection reset	The target closed the connection before answering. Often a crashing service, a proxy dropping the request, or a plain-text port spoken to over TLS.
`EHOSTUNREACH`	Host unreachable	The agent has no network route to the target. Check routing, VPN and firewall rules between the agent and the host.
`connect_failed`	Could not connect	The agent could not open a connection to the target. Check that the host and port are right and reachable from the agent.
`ENODATA`	No DNS records of that type	The name exists but has no records of the requested type.
`ESERVFAIL`	DNS server failure	The resolver answered SERVFAIL. Often a broken DNSSEC chain or an unreachable authoritative server.
`EREFUSED`	DNS query refused	The resolver refused the query. It may not allow recursion for the agent’s address.
`CERT_HAS_EXPIRED`	Server certificate expired	The target’s TLS certificate is past its expiry date. Renew it, or turn off TLS verification for this probe if you accept the risk.
`CERT_NOT_YET_VALID`	Server certificate not yet valid	The certificate’s start date is in the future. Check the certificate, or the agent host’s clock.
`DEPTH_ZERO_SELF_SIGNED_CERT`	Self-signed server certificate	The target presents a self-signed certificate. Provide its CA with ca_cert_ref, or turn off TLS verification for this probe.
`SELF_SIGNED_CERT_IN_CHAIN`	Untrusted certificate chain	The chain ends in a root the agent doesn’t trust (often a private CA). Provide the CA with ca_cert_ref.
`UNABLE_TO_VERIFY_LEAF_SIGNATURE`	Certificate chain incomplete	The agent couldn’t verify the server certificate, usually because the server doesn’t send its intermediate certificates.
`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`	Certificate issuer unknown	The certificate’s issuer isn’t in the agent’s trust store. Provide the CA with ca_cert_ref, or fix the server’s chain.
`ERR_TLS_CERT_ALTNAME_INVALID`	Certificate name mismatch	The server certificate isn’t valid for the hostname the probe connects to. Check the URL or the certificate’s names.
`tls_handshake_failed`	TLS handshake failed	The agent and the target could not agree on a TLS session. Check that the port speaks TLS and supports a modern protocol version.
`tcp_error`	TCP connection failed	The connection failed with an error the agent doesn’t classify. Check the agent log.
`tls_error`	TLS connection failed	The TLS connection failed with an error the agent doesn’t classify. Check the agent log.
`dns_error`	DNS query failed	The resolver returned an error the agent doesn’t classify. Check the resolver and the agent log.

HTTP

Configuration: HTTP guide.

Code	Meaning	What to check
`body_mismatch`	Body match failed	The response body did not contain the configured match string. Only the first 4 KB is read.
`json_body_too_large`	Response body too large for JSON extraction	The body exceeded 10 MB. Use a dedicated probe endpoint that returns just the metric.
`http_error`	HTTP request failed	The agent could not complete the request. Check the agent log for details.

HTTP JSON extraction

Configuration: HTTP JSON extraction guide.

Code	Meaning	What to check
`json_parse_failed`	Response body isn’t valid JSON	The agent couldn’t parse the response. Check the Content-Type and the body shape.
`json_path_invalid`	JSONPath expression is invalid	Test the path against the response body locally before deploying.
`json_path_no_match`	JSONPath matched nothing	Verify the field exists in the response and the path is spelled correctly.
`json_path_multi_match`	JSONPath matched multiple values	Use an array index or filter to pick exactly one value.
`json_path_non_numeric`	Extracted value isn’t numeric	Booleans cast to 0/1; everything else fails. Cast inside the source if you control it.
`json_path_unsupported_type`	Extracted value is an array, object, or null	The path must point at a leaf scalar.

HTTP mTLS

Configuration: HTTP mTLS guide.

Code	Meaning	What to check
`mtls_ref_missing`	mTLS env var not set on the agent	The certificate / key environment variable named in the config isn’t set on the agent host. Export it and restart the agent; or, if you entered certificate content into the field, replace it with the NAME of the env var that holds it.
`mtls_handshake_failed`	mTLS handshake failed	The TLS handshake failed for a reason the agent couldn’t categorise. Check the agent log and the server’s client-auth requirements.
`mtls_file_unreadable`	mTLS cert file unreadable	The env var points at a file path the agent can’t read. Check the path and file permissions.
`mtls_cert_parse_failed`	Client certificate didn’t parse	The PEM material isn’t a valid certificate. Verify it’s the cert, not the key, and that it’s complete.
`mtls_key_parse_failed`	Client private key didn’t parse	The PEM material isn’t a valid private key. PKCS#1, PKCS#8, and EC keys are supported.
`mtls_weak_key`	Client key too weak	RSA keys under 2048 bits are rejected. Reissue the client certificate with a stronger key.
`mtls_cert_expired`	Client certificate expired	The client certificate’s validity period has passed. Reissue and update the env var on the agent.
`mtls_incomplete`	mTLS config incomplete	Both the client certificate and private key env vars must be set together.
`mtls_server_cert_expired`	Server certificate expired	The endpoint’s TLS certificate has expired. This is a server-side problem, not the client cert.
`mtls_server_verify_failed`	Couldn’t verify the server certificate	The server’s cert didn’t validate against the configured CA (or the system trust store). Set the CA cert env var if the server uses a private CA.
`mtls_client_cert_rejected`	Server rejected the client certificate	The handshake failed: the server didn’t accept the client cert. This is the server’s trust decision, not your cert’s validity; confirm the cert chains to a CA the server trusts for client auth and isn’t revoked.

Secret headers (HTTP, WebSocket)

Configuration: Secret headers guide.

Code	Meaning	What to check
`header_ref_missing`	Secret header env var not set on the agent	A header in header_refs names an environment variable that isn’t set (or is blank) on the agent host, so the probe was not sent. Export it on the agent process and restart the agent. metadata.header and metadata.header_ref name the header and the variable.
`header_ref_invalid`	Secret header value is invalid	The env var named in header_refs contains a line break or control character, which isn’t allowed in an HTTP header. Fix the value on the agent host and restart the agent.

DNS and TLS certificate

Configuration: DNS and TLS certificate guide.

Code	Meaning	What to check
`expected_value_mismatch`	DNS record didn’t match	The resolved value didn’t contain the configured expected value.
`no_cert`	TLS handshake failed	The agent couldn’t obtain a certificate from the target.
`bad_cert_date`	TLS certificate has no expiry	The certificate’s notAfter field couldn’t be parsed.

ICMP

Configuration: ICMP guide.

Code	Meaning	What to check
`icmp_privilege_denied`	ICMP needs elevated privilege	The agent can’t send ICMP. Grant CAP_NET_RAW (Linux) or run with the documented capability. See the ICMP probe guide.
`icmp_unavailable`	ping not available on the agent	The agent host has no usable `ping` binary. Install iputils-ping (Linux) or add it to the agent image.
`icmp_dns_failed`	Host didn’t resolve	The hostname couldn’t be resolved to an address. Check the name and the agent’s DNS.
`icmp_all_timeout`	All pings timed out	No ICMP echo replies came back. The host may be down, or the network blocks ICMP. Try a TCP probe if ICMP is filtered.
`icmp_error`	Ping failed	The ping command failed for an uncategorised reason. Check the agent log.

gRPC

Configuration: gRPC guide.

Code	Meaning	What to check
`grpc_health_unknown`	Health status UNKNOWN	The service answered the health check with UNKNOWN. It hasn’t decided its own status yet, or is mid-startup. Re-check shortly.
`grpc_service_unknown`	gRPC service not registered	The named service isn’t registered with the server’s health service. Check the service name (leave it blank to check overall server health).
`grpc_unimplemented`	Server doesn’t implement health checks	The target returned UNIMPLEMENTED. Add the standard grpc.health.v1.Health service to the server, or probe a different endpoint.
`grpc_unavailable`	gRPC server unavailable	The connection failed (refused, DNS, or network). Distinct from an auth or TLS error. Confirm the host:port and that the server is up.
`grpc_timeout`	gRPC health check timed out	The Check call didn’t complete within the timeout. The server may be slow or unreachable.
`grpc_unauthenticated`	gRPC authentication failed	The server rejected the call as unauthenticated. Check the secret metadata (for example the authorization entry) and the env var it names on the agent host.
`grpc_permission_denied`	gRPC permission denied	Authenticated but not authorized to call the health service. Check the credential’s permissions.
`grpc_tls_failed`	gRPC TLS handshake failed	The TLS handshake to the server failed. Verify the TLS mode, and set the CA cert env var if the server uses a private CA.
`grpc_ca_unreadable`	gRPC CA certificate unreadable	The env var named for the CA certificate is unset or points at an unreadable file on the agent. Check the name and file permissions.
`grpc_error`	gRPC health check failed	An uncategorised gRPC error. Check the agent log for the status code and detail.
`metadata_ref_missing`	Secret metadata env var not set on the agent	A key in metadata_refs names an environment variable that isn’t set (or is blank) on the agent host, so the health check was not sent. Export it on the agent process and restart the agent. metadata.metadata_key and metadata.metadata_ref name the key and the variable.
`metadata_ref_invalid`	Secret metadata value is invalid	The env var named in metadata_refs contains a line break, control character, or non-ASCII character, which gRPC metadata doesn’t allow. Fix the value on the agent host and restart the agent.

WebSocket

Configuration: WebSocket guide.

Code	Meaning	What to check
`ws_open_failed`	WebSocket connection failed	The socket never opened. The endpoint refused the connection, the handshake was rejected, or the URL is wrong. Check the agent log’s metadata.error field.
`ws_handshake_timeout`	WebSocket handshake timed out	The connection didn’t open within the timeout. The endpoint may be slow, unreachable, or not speaking WebSocket on that path.
`ws_reply_timeout`	No WebSocket reply in time	The socket opened and the message was sent, but no matching reply arrived before the timeout. Check the endpoint echoes or responds to the configured message.
`ws_closed_early`	WebSocket closed before the reply	The endpoint closed the connection before sending the awaited reply. The close code is in metadata.
`ws_reply_mismatch`	WebSocket reply didn’t match	A reply arrived but didn’t contain the configured expected message. Verify the endpoint’s response format.

Prometheus

Configuration: Prometheus guide.

Code	Meaning	What to check
`prometheus_url_missing`	No Prometheus server configured	The agent has no Prometheus server configured (PROMETHEUS_SERVER_URL). Configure it or switch this metric to an active probe type.
`no_data_for_query`	Query returned no data	The PromQL query evaluated to an empty result. The series may not exist yet, or the time range excluded all samples.
`Unauthorized`	Prometheus rejected the request	Authentication or authorization failed. Check the agent’s Prometheus credentials (basic auth or PROMETHEUS_BEARER_TOKEN) and, for Grafana Mimir, PROMETHEUS_TENANT_ID.
`BadQuery`	PromQL query rejected	Prometheus parsed the query as invalid. Test it in the Prometheus UI first.
`PromUpstream`	Prometheus upstream error	Prometheus returned a 5xx. Usually transient.
`DNS`	DNS lookup failed	The agent could not resolve the Prometheus hostname.
`AgentInternal`	Agent error	An internal error in the agent. Check the agent log.

Database

Configuration: Database guide.

Code	Meaning	What to check
`db_dsn_missing`	Database connection string missing	The env var named in connection_string_ref is not set on the agent. Export it and restart.
`db_query_not_allowed`	Query rejected by the SELECT-only parser	The query failed the safety check. The parser only accepts a single SELECT or WITH … SELECT.
`db_empty_result`	Query returned zero rows	Change the query so it always returns one row, or add a coalesce to handle the empty case.
`db_multi_row`	Query returned more than one row	Reduce with count / max / sum or add LIMIT 1 so the probe gets a single scalar.
`db_multi_column`	Query returned more than one column	Project to a single column. The probe contract is one scalar per query.
`db_null_value`	Query returned NULL	Wrap in coalesce(value, 0) if zero is the right fallback, or filter the NULL upstream.
`db_non_numeric`	Query result isn’t numeric	Cast inside the query (::int, ::numeric, CAST(… AS UNSIGNED)). Booleans cast to 0/1.
`db_timeout`	Query exceeded statement timeout	Optimize the query or raise statement_timeout_ms (max 30 s).
`db_access_denied`	Database role lacks SELECT privilege	Grant SELECT on the table to the read-only role the agent uses.
`db_auth_failed`	Database authentication failed	The DSN’s username or password is wrong, or the role doesn’t exist.
`db_syntax_error`	Query has a syntax error	Test the query directly against the database before saving the metric.
`db_connection_failed`	Database unreachable	Network error reaching the database. Check firewall rules and DNS from the agent host.
`db_invalid_timeout`	Statement timeout is invalid	An internal error caused a non-finite timeout. File an issue with the agent log.
`db_error`	Database returned an error	Uncategorised driver error. Check the agent log for the underlying SQLSTATE / errno.

OpenTelemetry (OTLP)

Configuration: OpenTelemetry guide.

Code	Meaning	What to check
`otlp_receiver_disabled`	OTLP receiver disabled on the agent	The agent was started with OBSERVER_OTLP_DISABLE=true. Restart with that variable unset.
`otlp_no_samples`	No OTLP samples received yet	Verify the sender is pushing to the right port and the metric name matches exactly.
`otlp_stale`	OTLP samples are stale	The most recent matching sample is older than the configured staleness window. The sender may have stopped pushing.
`otlp_aggregation_unavailable`	Aggregation not supported for this data point	Quantiles on exponential histograms aren’t supported in v1. Switch to mean, count, or sum.
`otlp_invalid_json`	OTLP payload isn’t valid JSON	Check the exporter is configured with encoding: json. Protobuf encoding returns HTTP 415.
`otlp_malformed_payload`	OTLP payload shape is malformed	The body parsed as JSON but didn’t contain a `resourceMetrics` array.

AWS CloudWatch

Configuration: AWS CloudWatch guide.

Code	Meaning	What to check
`cloudwatch_no_data`	CloudWatch returned no data	Verify the dimensions match the metric exactly. CloudWatch matches on the full dimension set.
`cloudwatch_access_denied`	AWS IAM access denied	The agent’s role lacks cloudwatch:GetMetricData against this metric.
`cloudwatch_throttled`	AWS is rate-limiting the agent	Raise the period for non-critical metrics or split the load across agents.
`cloudwatch_invalid_parameter`	Invalid CloudWatch parameter	Usually a dimension CloudWatch doesn’t recognise or a statistic the metric doesn’t support.
`cloudwatch_resource_not_found`	Metric not found in CloudWatch	The namespace, metric name, and dimension combination doesn’t exist in this region and account.
`cloudwatch_expired_credentials`	AWS credentials expired	The STS session expired. The SDK refreshes automatically on the next tick.
`cloudwatch_server_error`	AWS returned a server error	Transient. Check the AWS service health dashboard if it persists.
`cloudwatch_error`	CloudWatch returned an error	An uncategorised AWS error. Check the AWS service health dashboard.

Grafana Loki

Configuration: Grafana Loki guide.

Code	Meaning	What to check
`loki_not_aggregation`	LogQL query isn’t a metric	The query returned log lines, not a number. Wrap it in an aggregation like count_over_time, rate, or sum(…). See the Loki source guide.
`loki_no_data`	Loki query returned no data	No log events matched in the query window. May be a label / filter typo, or genuinely no events.
`loki_multiple_series`	Loki query returned multiple series	The query produced more than one label set. Aggregate further (sum without (…)) so it returns a single value.
`loki_unreachable`	Loki unreachable	The agent couldn’t connect to the Loki base URL (refused, DNS, or network). Distinct from an auth or query error.
`loki_timeout`	Loki query timed out	The query didn’t finish within the timeout. Narrow the time range or simplify the query.
`loki_unauthorized`	Loki rejected the request	401/403 from Loki. Check the auth mode + the token/password env var on the agent, and the tenant ID for multi-tenant Loki.
`loki_auth_ref_missing`	Loki auth env var not set	The env var named for the bearer token / basic password isn’t set on the agent. Export it and restart the agent.
`loki_query_error`	Loki rejected the query	Loki returned a 400/422. The LogQL is invalid; test it in the Loki / Grafana explore view. The error text is in the metadata.
`loki_server_error`	Loki server error	Loki returned a 5xx. Usually transient; check Loki’s own health if it persists.
`loki_bad_response`	Unexpected Loki response	The query succeeded but didn’t return a vector/scalar the probe can read. Use an instant metric query.
`loki_error`	Loki query failed	An uncategorised error talking to Loki. Check the agent log.

Elasticsearch and OpenSearch

Configuration: Elasticsearch and OpenSearch guide.

Code	Meaning	What to check
`es_agg_not_found`	Aggregation not found in the response	The aggregation name doesn’t appear in Elasticsearch’s response. Check the name matches a key under `aggs`. The available names are in the metadata.
`es_agg_unsupported`	Aggregation has no numeric value	The named aggregation didn’t return a single number. Use a metric aggregation (value_count, sum, avg, cardinality, percentiles + a percentile).
`es_no_data`	Aggregation returned no value	The aggregation value was null (often zero matching documents on avg/min/max). value_count returns 0; metric aggs over an empty set return null.
`es_index_not_found`	Index not found	Elasticsearch returned 404 for the index pattern. Check the index name / wildcard exists.
`es_unreachable`	Elasticsearch unreachable	The agent couldn’t connect to the base URL (refused, DNS, network). Distinct from an auth or query error.
`es_timeout`	Elasticsearch query timed out	The search didn’t finish within the timeout. Narrow the query window or simplify the aggregation.
`es_unauthorized`	Elasticsearch rejected the request	401/403 from Elasticsearch. Check the auth mode + the token / password / API key env var on the agent.
`es_auth_ref_missing`	Elasticsearch auth env var not set	The env var named for the token / password / API key isn’t set on the agent. Export it and restart the agent.
`es_query_error`	Elasticsearch rejected the query	A 400/422 from Elasticsearch. The Query DSL is invalid; the parser error is in the metadata. Test it in Kibana / Dev Tools.
`es_server_error`	Elasticsearch server error	Elasticsearch returned a 5xx. Usually transient; check the cluster health if it persists.
`es_bad_response`	Unexpected Elasticsearch response	The search succeeded but didn’t contain the expected aggregations block.
`es_error`	Elasticsearch query failed	An uncategorised error talking to Elasticsearch. Check the agent log.

Custom probes

Configuration: Custom probes guide.

Code	Meaning	What to check
`custom_probe_not_found`	Custom probe not registered	No probe with this name is registered on the agent. Check the name, and confirm the agent that runs it has been redeployed with the probe.
`custom_probe_config_invalid`	Custom probe config rejected	The probe’s own config schema rejected the supplied config. Check the JSON against what the probe expects.
`custom_probe_timeout`	Custom probe timed out	The probe didn’t finish within its timeout. Optimise the probe or raise the timeout (max 30s).
`custom_probe_error`	Custom probe threw an error	The probe function raised an exception. The metric’s metadata carries the message (metadata.error) and a short stack (metadata.stack); fix it in the agent code and redeploy.
`custom_probe_bad_return`	Custom probe returned a non-number	A probe must return a finite number, or `{ value, metadata }`. Check what the function returns.

Host metrics

Code	Meaning	What to check
`host_cpu_unavailable`	CPU usage unavailable	The agent couldn’t read CPU counters on its host.
`host_memory_unavailable`	Memory usage unavailable	The agent couldn’t read memory counters on its host.
`host_filesystem_unavailable`	Filesystem usage unavailable	The agent couldn’t read the configured mountpoint. Check the path exists on the agent host (inside the container, if it runs in one).
`host_network_unavailable`	Network throughput unavailable	The agent couldn’t read interface counters, or the configured interface doesn’t exist on its host.
`host_metric_unsupported_platform`	Not supported on this platform	This host metric isn’t available on the agent’s operating system. Network throughput is Linux only.
`host_unknown_metric`	Unknown host metric	The agent doesn’t know this host metric. Update the agent.
`host_read_error`	Host metric read failed	Reading the host counters failed unexpectedly. Check the agent log.

Inbound alerts

Configuration: Inbound alerts guide.

Code	Meaning	What to check
`alert_heartbeat_missing`	Heartbeat alert missing	The inbound alert source’s heartbeat alert (for example Alertmanager’s Watchdog) stopped arriving, so its alerts can’t be trusted. Status returns when the heartbeat resumes.

Hosted checks

Configuration: Hosted checks guide.

Code	Meaning	What to check
`dns_failed`	DNS lookup failed	The prober couldn’t resolve the hostname with public DNS. Internal names aren’t reachable from hosted checks.
`dns_no_answer`	Hostname has no address	Public DNS returned no A or AAAA records for the name.
`source_not_allowed_in_prober`	Check type not available outside-in	Hosted checks run HTTP, TCP, DNS, TLS certificate and ping only. Use an in-network agent for this source.
`icmp_unavailable_on_prober`	Ping not available on this prober	This prober can’t send ICMP. Use a TCP or HTTP check instead.
`redirect_invalid`	Invalid redirect	The target redirected to a Location that isn’t a valid URL.
`redirect_scheme_not_allowed`	Redirect to a non-HTTP URL	The target redirected to a scheme other than http or https, which hosted checks don’t follow.
`url_credentials_not_allowed`	URL contains credentials	Hosted checks don’t send user:password in URLs, including on redirects.
`invalid_host`	Invalid host	The host isn’t a valid domain name or IP address.
`host_not_fqdn`	Host is not a full domain name	Hosted checks need a fully qualified name, like api.example.com.
`host_internal`	Internal host name	Hosted checks run from the public internet and can’t reach internal names. Use an in-network agent.
`host_not_public`	Target is not public	Hosted checks only reach public addresses. Use an in-network agent for private targets.
`invalid_resolver`	Invalid DNS resolver	Hosted DNS checks need a public resolver IP on port 53.
`resolver_not_public`	Resolver is not public	Hosted DNS checks can only query public resolvers.

Configuration and agent runtime

Code	Meaning	What to check
`unknown_source_type`	Unknown probe type	The agent doesn’t know how to dispatch this source_type. Likely an older agent against a newer cloud.
`invalid_config`	Probe configuration is invalid	Edit the metric and fix the highlighted fields. The agent rejects the config before it runs.
`push_source_init_failed`	Receiver could not start	Usually a port collision. Free the port or change the receiver’s listen address.
`push_source_read_failed`	Receiver read failed	An error during a push-source read. Check the agent log.
`not_implemented`	Probe runtime not implemented	This source type is reserved in the schema but doesn’t have an agent runtime yet.
`invalid_url`	Invalid URL	The configured URL could not be parsed. Fix the metric’s source configuration.
`invalid_url_scheme`	Unsupported URL scheme	Only http:// and https:// URLs are supported for HTTP probes.
`dns_resolver_invalid`	Invalid DNS resolver	The custom resolver in the probe configuration is not a valid address.
`grpc_metadata_invalid`	Invalid gRPC metadata	A metadata key or value in the probe configuration is not valid. Fix the metric’s gRPC metadata entries.
`invalid_prometheus_url`	Invalid Prometheus URL	The agent’s PROMETHEUS_SERVER_URL is not a valid URL. Fix the agent’s environment configuration.
`prometheus_auth_conflict`	Conflicting Prometheus auth	The agent has both a bearer token and basic auth set (or PROMETHEUS_HEADERS repeats one of them). Keep one in the agent’s environment; the agent log names the variables.
`prometheus_headers_invalid`	Invalid PROMETHEUS_HEADERS	The agent’s PROMETHEUS_HEADERS is neither a JSON object nor a “Name: value; Name2: value” list. The agent log names the problem.
`prom_bad_response`	Unreadable Prometheus response	The query endpoint answered with something that isn’t JSON. Check that PROMETHEUS_SERVER_URL points at the Prometheus API, not a login page or proxy.
`prom_multiple_series`	Query returned multiple series	The PromQL result has more than one series; Observer needs a single scalar. Aggregate (sum/avg/max) or add label matchers so exactly one series returns.
`source_threw`	Probe crashed	The source runtime threw an unexpected error. The agent recovered and will retry; check the agent logs for the stack trace.
`non_finite_value`	Value is not a number	The source returned NaN or infinity, which can’t be compared to thresholds.
`unknown`	Unknown failure	The agent recorded a failure without a cause. Check the agent log.
`test_timeout`	Test timed out	The test run didn’t finish in time. The target may be slow, or the timeout too short.
`agent_error`	Agent error	The agent failed while running this request. Check the agent log.

Reported by Observer

Written by Observer itself, not by an agent.

Code	Meaning	What to check
`stale_agent_silent`	Agent went silent	No samples have arrived for over 3× this metric’s push interval, so the last verdict expired. Usually the agent process is stopped or can’t reach Observer; check the agent’s status and logs.
`no_data`	No data	The probe didn’t produce a measurable value on the last tick.

Codes with a variable part

Code	Meaning	What to check
`unexpected_status:<status>`	Unexpected HTTP status	The response status was not one the probe accepts (for HTTP probes, the metric’s expected_status set). For example `unexpected_status:503`.
`target_not_public:<reason>`	Target is not public	Hosted checks only reach public addresses, and refuse a name if any of its DNS answers is private. Use an in-network agent for this target. The suffix names the block, for example `loopback`, `private` or `cloud_metadata`.
`refused:<problem>`	Check refused by the prober	The prober re-validates every hosted check and refused this one. Edit the check to fix the named problem.

Codes from older agents

Older agents report some failures with the raw error code of their runtime, or with a code that has since been renamed. Observer still recognises them and shows them like the current code below.

Older code	Shown as
`ConnectionRefused`	Could not connect. The connection was refused, or the hostname did not resolve (older agents report both the same way). Check the host, port and DNS from the agent.
`ConnectionFailed`	`connect_failed` (Could not connect)
`FailedToOpenSocket`	`connect_failed` (Could not connect)
`ConnectionClosed`	`ECONNRESET` (Connection reset)
`ConnectionResetByPeer`	`ECONNRESET` (Connection reset)
`BrokenPipe`	`ECONNRESET` (Connection reset)
`ConnectionTimedOut`	`ETIMEDOUT` (Request timed out)
`ConnectionTimeout`	`ETIMEDOUT` (Request timed out)
`Timeout`	`ETIMEDOUT` (Request timed out)
`TimeoutError`	`ETIMEDOUT` (Request timed out)
`AbortError`	`ETIMEDOUT` (Request timed out)
`EPIPE`	`ECONNRESET` (Connection reset)
`ECONNABORTED`	`ETIMEDOUT` (Request timed out)
`ESOCKETTIMEDOUT`	`ETIMEDOUT` (Request timed out)
`ETIMEOUT`	`ETIMEDOUT` (Request timed out)
`ENETUNREACH`	`EHOSTUNREACH` (Host unreachable)
`ENETDOWN`	`EHOSTUNREACH` (Host unreachable)
`EHOSTDOWN`	`EHOSTUNREACH` (Host unreachable)
`EADDRNOTAVAIL`	`connect_failed` (Could not connect)
`EAI_NONAME`	`ENOTFOUND` (DNS lookup failed)
`EAI_NODATA`	`ENOTFOUND` (DNS lookup failed)
`EAI_FAIL`	`ENOTFOUND` (DNS lookup failed)
`CERT_UNTRUSTED`	`SELF_SIGNED_CERT_IN_CHAIN` (Untrusted certificate chain)
`UNABLE_TO_GET_ISSUER_CERT`	`UNABLE_TO_GET_ISSUER_CERT_LOCALLY` (Certificate issuer unknown)
`HOSTNAME_MISMATCH`	`ERR_TLS_CERT_ALTNAME_INVALID` (Certificate name mismatch)
`no_prometheus_url`	`prometheus_url_missing` (No Prometheus server configured)
`body_too_large`	`json_body_too_large` (Response body too large for JSON extraction)
`ERR_TLS_`, `ERR_SSL_`	`tls_handshake_failed` (TLS handshake failed)

A code that matches nothing on this page is shown as is, with a note that it is not recognised. It usually means the agent is newer than this page, or a custom build reports its own codes.

Quickstart

Concepts

Guides

Reference

Reason codes

Network and TLS

HTTP

HTTP JSON extraction

HTTP mTLS

Secret headers (HTTP, WebSocket)

DNS and TLS certificate

ICMP

gRPC

WebSocket

Prometheus

Database

OpenTelemetry (OTLP)

AWS CloudWatch

Grafana Loki

Elasticsearch and OpenSearch

Custom probes

Host metrics

Inbound alerts

Hosted checks

Configuration and agent runtime

Reported by Observer

Codes with a variable part

Codes from older agents

​Network and TLS

​HTTP

​HTTP JSON extraction

​HTTP mTLS

​Secret headers (HTTP, WebSocket)

​DNS and TLS certificate

​ICMP

​gRPC

​WebSocket

​Prometheus

​Database

​OpenTelemetry (OTLP)

​AWS CloudWatch

​Grafana Loki

​Elasticsearch and OpenSearch

​Custom probes

​Host metrics

​Inbound alerts

​Hosted checks

​Configuration and agent runtime

​Reported by Observer

​Codes with a variable part

​Codes from older agents

Network and TLS

HTTP

HTTP JSON extraction

HTTP mTLS

Secret headers (HTTP, WebSocket)

DNS and TLS certificate

ICMP

gRPC

WebSocket

Prometheus

Database

OpenTelemetry (OTLP)

AWS CloudWatch

Grafana Loki

Elasticsearch and OpenSearch

Custom probes

Host metrics

Inbound alerts

Hosted checks

Configuration and agent runtime

Reported by Observer

Codes with a variable part

Codes from older agents