Observer Docs

The Elasticsearch source runs one search per interval against /{index}/_search with size:0 and reads a named aggregation’s numeric value as the metric. OpenSearch uses the same search API, so this source covers both; the flavor setting only changes labels. Observer is not a log store. The source extracts one number from an aggregation; it never reads or keeps documents. Same shape as the SQL and Loki sources: the query returns a number, that number is the metric.

When to use it

Anything you have indexed in Elasticsearch / OpenSearch and want as a status signal: error counts from log events, average or p95 request latency, unique active users, business event rates.

Query structure

Provide the full search body with a query (optional) and an aggs block. Name the aggregation; the source reads its value.

{
  "query": {
    "bool": {
      "must": [
        { "match": { "level": "error" } },
        { "range": { "@timestamp": { "gte": "now-5m" } } }
      ]
    }
  },
  "aggs": {
    "error_count": { "value_count": { "field": "@timestamp" } }
  }
}

Set the aggregation name to error_count; the source reads aggregations.error_count.value. The agent always sends size:0, so documents are never returned.

Supported aggregations

Single-value metric aggregations are read from .value:

Aggregation	Use
`value_count`	how many documents matched (error count, event count)
`sum`	total of a numeric field
`avg`	average (request latency, response size)
`min` / `max`	extremes
`cardinality`	distinct values (unique users)

percentiles is read from .values. Set the Percentile field to the key you want (for example 95.0):

{ "aggs": { "latency_p95": { "percentiles": { "field": "duration_ms", "percents": [95] } } } }

The query must resolve to one number. value_count over no matches returns 0; avg/min/max over no matches return null, which the probe reports as es_no_data.

Configuration shape

{
  "base_url": "https://es.internal.example.com:9200",
  "index": "logs-*",
  "query": { "query": {}, "aggs": { "error_count": { "value_count": { "field": "@timestamp" } } } },
  "agg_name": "error_count",
  "flavor": "elasticsearch",
  "auth_mode": "api_key",
  "api_key_ref": "OBSERVER_ES_API_KEY",
  "timeout_ms": 10000
}

Field reference

Field	Default	Notes
`base_url`	required	Elasticsearch / OpenSearch HTTP endpoint.
`index`	required	Index or pattern. Wildcards + date math allowed (`logs-`, `events-2026.`).
`query`	required	Full search body containing an `aggs` block.
`agg_name`	required	The key under `aggs` to read the value from.
`percentile`	none	For a percentiles aggregation: which percentile key (`95.0`).
`flavor`	`elasticsearch`	`elasticsearch` or `opensearch`. Same request either way.
`auth_mode`	`none`	`none`, `bearer`, `basic`, or `api_key`.
`token_ref`	none	Env var NAME holding the bearer token. Required for `bearer`.
`username` / `password_ref`	none	Basic auth username + env var NAME of the password. Both required for `basic`.
`api_key_ref`	none	Env var NAME holding the base64 `id:api_key`. Required for `api_key`.
`timeout_ms`	`10000`	100 to 60000 ms.

Authentication

Auth secrets stay on the agent. You store the NAME of an environment variable on the agent host; the agent reads the value at query time, so the credential never reaches the cloud and never appears in logs. For Elastic Cloud, the usual choice is an API key. Create one in Kibana (Stack Management, API keys); Elastic gives you a base64 id:api_key value. Export it on the agent and reference it:

export OBSERVER_ES_API_KEY="VnVhQ2ZHY0JDZGJrU..."   # api_key_ref: OBSERVER_ES_API_KEY

The agent sends it as Authorization: ApiKey <value>. Bearer and basic auth work the same way through token_ref and username + password_ref.

OpenSearch

OpenSearch is a fork of Elasticsearch with a compatible search API. Set the flavor to OpenSearch for clarity; the request is identical. Known differences (security plugin endpoints, version fields) don’t affect the _search + aggregations path this source uses. Basic auth and bearer tokens both work; OpenSearch doesn’t use the Elastic API-key format, so prefer basic auth there.

Testing the query

Observer cannot reach a private Elasticsearch from the cloud, so there is no live “run query” button. Validate the query in Kibana Dev Tools (or OpenSearch Dashboards) against the same cluster, confirm the aggregation returns a single number, then paste the body in. The first value arrives on the next agent tick.

Reason codes

es_agg_not_found: the aggregation name isn’t in the response. The available names are in the metadata; check the name matches a key under aggs.
es_agg_unsupported: the named aggregation didn’t yield a single number. Use a metric aggregation (and set the percentile for percentiles).
es_no_data: the aggregation value was null (often an empty match set on avg/min/max).
es_index_not_found: 404 for the index pattern. Check it exists.
es_unreachable: couldn’t connect (refused, DNS, network).
es_timeout: the search didn’t finish in time.
es_unauthorized: 401/403. Check auth + the credential env var.
es_auth_ref_missing: the token / password / API-key env var isn’t set on the agent.
es_query_error: 400/422. The Query DSL is invalid; the parser error is in the metadata. Test it in Dev Tools.
es_server_error: 5xx from the cluster. Usually transient.
es_error: any other failure, such as an unexpected HTTP status or a network error not listed above.

Troubleshooting

es_agg_not_found with the right query. The agg_name must match the key under aggs exactly. For nested aggregations, name the top-level one whose .value you want.
es_no_data on avg. No documents matched. avg over an empty set is null; if you want 0, use value_count or widen the time range.
es_unauthorized on Elastic Cloud. Confirm the API key is the base64 id:api_key, not the raw key, and that the env var is set on the agent.

Quickstart

Concepts

Guides

Reference

Configure an Elasticsearch or OpenSearch source

When to use it

Query structure

Supported aggregations

Configuration shape

Field reference

Authentication

OpenSearch

Testing the query

Reason codes

Troubleshooting

​When to use it

​Query structure

​Supported aggregations

​Configuration shape

​Field reference

​Authentication

​OpenSearch

​Testing the query

​Reason codes

​Troubleshooting

When to use it

Query structure

Supported aggregations

Configuration shape

Field reference

Authentication

OpenSearch

Testing the query

Reason codes

Troubleshooting