Configure an Elasticsearch or OpenSearch source

The Elasticsearch source runs one search per interval against /{index}/_search with size:0 and reads a named aggregation's numeric value as the metric. OpenSearch uses the same search API, so this source covers both; the flavor setting only changes labels.

Observer is not a log store. The source extracts one number from an aggregation; it never reads or keeps documents. Same shape as the SQL and Loki sources: the query returns a number, that number is the metric.

When to use it

Anything you have indexed in Elasticsearch / OpenSearch and want as a status signal: error counts from log events, average or p95 request latency, unique active users, business event rates.

Query structure

Provide the full search body with a query (optional) and an aggs block. Name the aggregation; the source reads its value.

{
  "query": {
    "bool": {
      "must": [
        { "match": { "level": "error" } },
        { "range": { "@timestamp": { "gte": "now-5m" } } }
      ]
    }
  },
  "aggs": {
    "error_count": { "value_count": { "field": "@timestamp" } }
  }
}

Set the aggregation name to error_count; the source reads aggregations.error_count.value. The agent always sends size:0, so documents are never returned.

Supported aggregations

Single-value metric aggregations are read from .value:

Aggregation	Use
value_count	how many documents matched (error count, event count)
sum	total of a numeric field
avg	average (request latency, response size)
min / max	extremes
cardinality	distinct values (unique users)

percentiles is read from .values. Set the Percentile field to the key you want (for example 95.0):

{ "aggs": { "latency_p95": { "percentiles": { "field": "duration_ms", "percents": [95] } } } }

The query must resolve to one number. value_count over no matches returns 0; avg/min/max over no matches return null, which the probe reports as es_no_data.

Configuration shape

{
  "base_url": "https://es.internal.example.com:9200",
  "index": "logs-*",
  "query": { "query": {}, "aggs": { "error_count": { "value_count": { "field": "@timestamp" } } } },
  "agg_name": "error_count",
  "flavor": "elasticsearch",
  "auth_mode": "api_key",
  "api_key_ref": "OBSERVER_ES_API_KEY",
  "timeout_ms": 10000
}

Field reference

Field	Default	Notes
base_url	required	Elasticsearch / OpenSearch HTTP endpoint.
index	required	Index or pattern. Wildcards + date math allowed (logs-*, events-2026.*).
query	required	Full search body containing an aggs block.
agg_name	required	The key under aggs to read the value from.
percentile	none	For a percentiles aggregation: which percentile key (95.0).
flavor	elasticsearch	elasticsearch or opensearch. Same request either way.
auth_mode	none	none, bearer, basic, or api_key.
token_ref	none	Env var NAME holding the bearer token. Required for bearer.
username / password_ref	none	Basic auth username + env var NAME of the password. Both required for basic.
api_key_ref	none	Env var NAME holding the base64 id:api_key. Required for api_key.
timeout_ms	10000	100 to 60000 ms.

Authentication

Auth secrets stay on the agent. You store the NAME of an environment variable on the agent host; the agent reads the value at query time, so the credential never reaches the cloud and never appears in logs.

For Elastic Cloud, the usual choice is an API key. Create one in Kibana (Stack Management, API keys); Elastic gives you a base64 id:api_key value. Export it on the agent and reference it:

The agent sends it as Authorization: ApiKey <value>. Bearer and basic auth work the same way through token_ref and username + password_ref.

OpenSearch

OpenSearch is a fork of Elasticsearch with a compatible search API. Set the flavor to OpenSearch for clarity; the request is identical. Known differences (security plugin endpoints, version fields) don't affect the _search + aggregations path this source uses. Basic auth and bearer tokens both work; OpenSearch doesn't use the Elastic API-key format, so prefer basic auth there.

Testing the query

Observer cannot reach a private Elasticsearch from the cloud, so there is no live "run query" button. Validate the query in Kibana Dev Tools (or OpenSearch Dashboards) against the same cluster, confirm the aggregation returns a single number, then paste the body in. The first value arrives on the next agent tick.

Reason codes

• es_agg_not_found: the aggregation name isn't in the response. The available names are in the metadata; check the name matches a key under aggs.
• es_agg_unsupported: the named aggregation didn't yield a single number. Use a metric aggregation (and set the percentile for percentiles).
• es_no_data: the aggregation value was null (often an empty match set on avg/min/max).
• es_index_not_found: 404 for the index pattern. Check it exists.
• es_unreachable: couldn't connect (refused, DNS, network).
• es_timeout: the search didn't finish in time.
• es_unauthorized: 401/403. Check auth + the credential env var.
• es_auth_ref_missing: the token / password / API-key env var isn't set on the agent.
• es_query_error: 400/422. The Query DSL is invalid; the parser error is in the metadata. Test it in Dev Tools.
• es_server_error: 5xx from the cluster. Usually transient.

Troubleshooting

• es_agg_not_found with the right query. The agg_name must match the key under aggs exactly. For nested aggregations, name the top-level one whose .value you want.
• es_no_data on avg. No documents matched. avg over an empty set is null; if you want 0, use value_count or widen the time range.
• es_unauthorized on Elastic Cloud. Confirm the API key is the base64 id:api_key, not the raw key, and that the env var is set on the agent.