Skip to main content
Prometheus is the most common metric source. The agent runs the PromQL query against the Prometheus URL configured at deploy time and reports the scalar result.

​
Configuration shape

A Prometheus metric carries a source_config of:
The optional prometheus_url overrides the agent’s PROMETHEUS_SERVER_URL for this single metric. Use it when one agent serves multiple Prometheus servers.

​
Query requirements

  • The query must return a single scalar value. Use aggregation (sum, avg, rate, etc.) to collapse vector results.
  • Empty results are reported as no_data; the metric does not flip status until the query produces a value.
  • A result with more than one series is reported as no_data with reason prom_multiple_series.
  • The agent does not interpret the query content. The cloud’s push payload contains the precomputed status, not the query string.

​
Reason codes

The full catalog, with the hint the console shows for each code, is in Reason codes. A metric that cannot produce a value reports no_data with one of:
  • prometheus_url_missing: neither prometheus_url nor PROMETHEUS_SERVER_URL is set.
  • invalid_prometheus_url: the configured URL is not a valid URL.
  • no_data_for_query: the query returned no series, or a value that is not a finite number.
  • prom_multiple_series: the query returned more than one series. Aggregate it down to one.
  • Unauthorized, BadQuery, PromUpstream: Prometheus answered 401 or 403, 400, or 5xx.
  • prometheus_auth_conflict: a bearer token and basic auth are both set, or PROMETHEUS_HEADERS repeats a header another variable already sets (agent 1.6.2+).
  • prometheus_headers_invalid: PROMETHEUS_HEADERS is neither a JSON object nor a Name: value; Name2: value list (agent 1.6.2+).
  • ECONNREFUSED, ETIMEDOUT, ENOTFOUND, ECONNRESET, EHOSTUNREACH, connect_failed: the request did not reach Prometheus. Older agents report a failed lookup as DNS, and a refused port or a failed lookup as ConnectionRefused.

​
Authentication

When Prometheus requires basic auth, set on the agent:
These apply to every Prometheus probe the agent runs. From agent 1.6.2 the agent can also send a bearer token (PROMETHEUS_BEARER_TOKEN), a tenant header (PROMETHEUS_TENANT_ID) and any other header (PROMETHEUS_HEADERS). A bearer token and basic auth cannot be used together. Header values are never written to the agent log.
Grafana CloudGrafana Cloud’s hosted Prometheus uses basic auth. The credentials issued by Grafana for read access drop in here unchanged. See Connect to Grafana Cloud.

​
Grafana Mimir and other Prometheus-compatible backends

Requires agent 1.6.2 or later. Any backend that serves the Prometheus HTTP API (/api/v1/query) works as a Prometheus source: Grafana Mimir, Cortex, Thanos Query, VictoriaMetrics and Grafana Cloud. Set PROMETHEUS_SERVER_URL to the base URL of the query API, including any path prefix. The agent keeps the prefix and appends /api/v1/query.

​
Grafana Mimir with a tenant

Mimir is multi-tenant: each request names its tenant in the X-Scope-OrgID header. Set the tenant with PROMETHEUS_TENANT_ID:
If Mimir runs with multi-tenancy turned off, leave PROMETHEUS_TENANT_ID unset. If a gateway in front of Mimir also asks for credentials, add basic auth or a bearer token as below.

​
Grafana Cloud with basic auth

Grafana Cloud’s hosted Prometheus uses basic auth: the username is the numeric user ID shown on the stack’s Prometheus details page and the password is an access policy token with the metrics:read scope. See Connect to Grafana Cloud.

​
Bearer token

For a backend or gateway that expects Authorization: Bearer, set PROMETHEUS_BEARER_TOKEN and turn basic auth off:
A bearer token and basic auth cannot be combined. When both are in effect (basic auth enabled with PROMETHEUS_PASSWORD set), the agent logs an error at startup and Prometheus metrics report no_data with reason prometheus_auth_conflict.

​
Other headers

PROMETHEUS_HEADERS adds any other header, as a JSON object or a Name: value; Name2: value list:
Keep the password, the bearer token and PROMETHEUS_HEADERS on the agent host like any other secret. The Add agent screen in the console writes them into the install command as variables you fill in on the host, never as values.

​
Threshold examples

The strict-comparison rule applies (see threshold operators in the Documentation tab).