> ## Documentation Index > Fetch the complete documentation index at: https://docs.use.observer/llms.txt > Use this file to discover all available pages before exploring further. # Run behind a proxy or in a restricted network > Outbound requirements, HTTP proxies, TLS-intercepting proxies, private CAs, offline installs, and how to verify the agent reaches Observer Cloud. This page covers installing the agent in networks that restrict outbound traffic: an HTTP proxy is mandatory, a TLS-intercepting proxy re-signs traffic with a corporate CA, internal services use a private CA, or the host cannot pull from public registries. ## Outbound requirements The agent only makes outbound connections. Nothing needs to reach the agent from the internet, and no inbound firewall rule is required. | Destination | Port | When | Purpose | | - | - | - | - | | Observer Cloud (`use.observer`, or the host in your `CLOUD_SERVER_URL`) | 443 (HTTPS) | Always | Heartbeats, metric definitions, result pushes, optional log broadcast. Every request goes to a path under `/api/agent/` and carries the `Agent-Key` header. | | Your probe targets | Whatever each probe uses | Always | Prometheus, HTTP endpoints, databases, DNS servers and the rest. These are usually internal. | | `ghcr.io` | 443 | Install and upgrade only | Pulling the container image. Layer downloads are redirected to `pkg-containers.githubusercontent.com`. | | `github.com` | 443 | Install and upgrade only | Downloading a release binary. Downloads are redirected to a GitHub asset host (currently `release-assets.githubusercontent.com`). | The agent does not follow redirects from Observer Cloud, so a proxy or gateway that answers with a 3xx (for example a captive portal or a login page) shows up as a failed request in the agent logs, not as a silent success. Everything the agent sends to the cloud is described in [The agent / cloud boundary](/agent/concepts/agent-cloud-boundary). ## Use an HTTP proxy The agent's runtime honours the standard proxy variables. Set them in the agent's environment: | Variable | Example | Notes | | - | - | - | | `HTTPS_PROXY` | `http://proxy.corp.example:3128` | Proxy for HTTPS requests, including the connection to Observer Cloud. The URL scheme is the scheme used to talk to the proxy, which is usually `http://` even for HTTPS traffic. | | `HTTP_PROXY` | `http://proxy.corp.example:3128` | Proxy for plain HTTP requests. | | `NO_PROXY` | `localhost,127.0.0.1,prometheus.internal,.corp.example` | Comma-separated hosts and domain suffixes that bypass the proxy. A leading dot matches every subdomain. CIDR ranges are not supported; list hostnames or individual IP addresses. | If the proxy needs credentials, put them in the URL: `http://user:password@proxy.corp.example:3128`. Treat the value as a secret. ### Keep internal probe targets out of the proxy The proxy variables apply to every HTTP request the agent makes, not just the connection to Observer Cloud. That includes Prometheus queries and HTTP probes. Most probe targets are internal services that the proxy either cannot reach or should not see, so add them to `NO_PROXY`: * The Prometheus host from `PROMETHEUS_SERVER_URL`. * The hosts of internal HTTP, Loki and Elasticsearch targets. * `localhost` and `127.0.0.1`. * In Kubernetes, the cluster domain suffix (for example `.svc,.svc.cluster.local`), so in-cluster Services are reached directly. Probes that open their own connections (TCP, DNS, ICMP, TLS certificate and database probes) connect directly to the target and are not routed through the proxy. gRPC and WebSocket probes use their own client libraries; confirm their behaviour with a test probe before relying on the proxy for them. ## TLS-intercepting proxies and private CAs A TLS-intercepting proxy (also called TLS inspection or SSL inspection) terminates HTTPS and re-signs it with a corporate CA. The agent rejects that certificate unless it trusts the CA, and logs a certificate verification error on every request to the cloud. Add the CA to the agent's trust store with `NODE_EXTRA_CA_CERTS`: point it at a PEM file on the agent host (or inside the container) that holds the CA certificate, or several certificates concatenated. The runtime trusts these certificates in addition to its built-in trust store, so public endpoints keep working. The same variable covers internal HTTPS targets signed by a private CA: Prometheus, Loki, Elasticsearch and HTTP probes all trust the extra certificates. `NODE_EXTRA_CA_CERTS` is read when the process starts. After you change the file or the variable, restart the agent. Ask your network or security team for the CA certificate in PEM format (a text file that starts with `-----BEGIN CERTIFICATE-----`). If you only have a DER file (`.cer` or `.crt` with binary content), convert it: ```bash theme={null} openssl x509 -inform der -in corp-ca.cer -out corp-ca.pem ``` The container image runs as a non-root user (UID 65532), so make sure a mounted PEM file is world-readable (`chmod 644`). ### Per-probe CA certificates Some probes take a CA certificate in their own configuration instead. This scopes the trust to that one probe: * gRPC probes: set `ca_cert_ref` in TLS or mTLS mode. See [gRPC probes](/agent/guides/grpc-probes#transport-security). * HTTP probes: `ca_cert_ref` applies together with mTLS client certificates. See [mTLS authentication](/agent/guides/http-probes#mtls-authentication). For a plain HTTPS target behind a private CA, use `NODE_EXTRA_CA_CERTS`. `ca_cert_ref` holds the name of an environment variable on the agent host, never the certificate itself. The variable's value is either the PEM text or a path to a PEM file. ### Do not disable TLS verification Two variables turn certificate checks off. Neither belongs in production: * `SKIP_SSL_VERIFICATION=true` disables verification on the connection to Observer Cloud. Anyone who can intercept that connection can read your agent key and feed the agent false metric definitions. * `NODE_TLS_REJECT_UNAUTHORIZED=0` disables verification for every HTTPS request the agent makes: the cloud, Prometheus, and every probe. An HTTP probe then reports a target as healthy even when its certificate is invalid or forged. If verification fails, fix the trust store with `NODE_EXTRA_CA_CERTS` instead. To accept a self-signed certificate on a single internal HTTP probe, set `verify_tls: false` on that probe only. ## Examples Each example sets a proxy, bypasses it for internal targets, and trusts a corporate CA stored in `corp-ca.pem`. Remove the parts you do not need. ```bash theme={null} docker run -d \ --name observer-agent \ --restart unless-stopped \ -v observer-agent-buffer:/data \ -v /etc/observer/corp-ca.pem:/etc/observer/corp-ca.pem:ro \ -e AGENT_KEY=obs_live_... \ -e CLOUD_SERVER_URL=https://use.observer \ -e BUFFER_PATH=/data/observer-agent-buffer.db \ -e HTTPS_PROXY=http://proxy.corp.example:3128 \ -e HTTP_PROXY=http://proxy.corp.example:3128 \ -e NO_PROXY=localhost,127.0.0.1,prometheus.internal,.corp.example \ -e NODE_EXTRA_CA_CERTS=/etc/observer/corp-ca.pem \ ghcr.io/useobserver/agent:1.6.0 ``` ```yaml title="docker-compose.yml" theme={null} services: observer-agent: image: ghcr.io/useobserver/agent:1.6.0 container_name: observer-agent restart: unless-stopped env_file: [.env] environment: BUFFER_PATH: /data/observer-agent-buffer.db HTTPS_PROXY: http://proxy.corp.example:3128 HTTP_PROXY: http://proxy.corp.example:3128 NO_PROXY: localhost,127.0.0.1,prometheus.internal,.corp.example NODE_EXTRA_CA_CERTS: /etc/observer/corp-ca.pem volumes: - observer-agent-buffer:/data - ./corp-ca.pem:/etc/observer/corp-ca.pem:ro volumes: observer-agent-buffer: ``` Keep `AGENT_KEY` (and proxy credentials, if any) in `.env`. Store the CA in a ConfigMap (a CA certificate is public, so a Secret is optional): ```bash theme={null} kubectl -n observer create configmap observer-agent-ca \ --from-file=corp-ca.pem=./corp-ca.pem ``` Then add the variables and the volume to the Deployment from [Install on Kubernetes](/agent/quickstart/install-kubernetes): ```yaml title="agent.yaml (excerpt)" theme={null} spec: containers: - name: agent image: ghcr.io/useobserver/agent:1.6.0 env: - name: AGENT_KEY valueFrom: secretKeyRef: { name: observer-agent, key: agent-key } - name: CLOUD_SERVER_URL value: https://use.observer - name: HTTPS_PROXY value: http://proxy.corp.example:3128 - name: HTTP_PROXY value: http://proxy.corp.example:3128 - name: NO_PROXY value: localhost,127.0.0.1,.svc,.svc.cluster.local,.corp.example - name: NODE_EXTRA_CA_CERTS value: /etc/observer/ca/corp-ca.pem volumeMounts: - { name: data, mountPath: /data } - { name: corp-ca, mountPath: /etc/observer/ca, readOnly: true } volumes: - name: data emptyDir: {} - name: corp-ca configMap: name: observer-agent-ca ``` If the proxy URL carries credentials, put `HTTPS_PROXY` in the `observer-agent` Secret and reference it with `secretKeyRef`, like `AGENT_KEY`. To use a Secret for the CA instead of a ConfigMap, replace the `configMap` volume with `secret: { secretName: observer-agent-ca }`. Add the variables to the environment file used by the unit from [Install from a release binary](/agent/quickstart/install-binary): ```bash title="/etc/observer-agent.env" theme={null} AGENT_KEY=obs_live_... CLOUD_SERVER_URL=https://use.observer HTTPS_PROXY=http://proxy.corp.example:3128 HTTP_PROXY=http://proxy.corp.example:3128 NO_PROXY=localhost,127.0.0.1,prometheus.internal,.corp.example NODE_EXTRA_CA_CERTS=/etc/observer/corp-ca.pem ``` Or set them directly in the unit: ```ini title="/etc/systemd/system/observer-agent.service (excerpt)" theme={null} [Service] EnvironmentFile=/etc/observer-agent.env Environment=HTTPS_PROXY=http://proxy.corp.example:3128 Environment=NO_PROXY=localhost,127.0.0.1,prometheus.internal,.corp.example Environment=NODE_EXTRA_CA_CERTS=/etc/observer/corp-ca.pem ExecStart=/usr/local/bin/observer-agent ``` ```bash theme={null} sudo install -m 644 corp-ca.pem /etc/observer/corp-ca.pem sudo systemctl daemon-reload sudo systemctl restart observer-agent ``` The binary reads `NODE_EXTRA_CA_CERTS` from the file system of the host, so the path must be readable by the service user. ## Install without access to public registries The host that runs the agent does not need to reach `ghcr.io` or `github.com` after installation. Fetch the artifact on a machine that can, then move it across. ### Container image: save and load On a machine with internet access, pull the image for the architecture of the target host and save it to a file: ```bash theme={null} docker pull --platform linux/amd64 ghcr.io/useobserver/agent:1.6.0 docker save ghcr.io/useobserver/agent:1.6.0 -o observer-agent-1.6.0.tar sha256sum observer-agent-1.6.0.tar ``` Use `--platform linux/arm64` for ARM hosts. Copy the tar file to the target host, check the checksum, and load it: ```bash theme={null} sha256sum observer-agent-1.6.0.tar docker load -i observer-agent-1.6.0.tar ``` Then run the agent with the same image reference. ### Container image: mirror into an internal registry If your organisation runs its own registry, copy the image into it and point the install at the mirror. `ghcr.io/useobserver/agent` is a multi-architecture image (`linux/amd64` and `linux/arm64`); use a tool that copies every architecture, such as `skopeo` or `crane`: ```bash theme={null} skopeo copy --all \ docker://ghcr.io/useobserver/agent:1.6.0 \ docker://registry.corp.example/observer/agent:1.6.0 # or crane copy ghcr.io/useobserver/agent:1.6.0 registry.corp.example/observer/agent:1.6.0 ``` Replace `ghcr.io/useobserver/agent` with `registry.corp.example/observer/agent` in the Docker, Compose or Kubernetes examples. Pin exact versions so every host runs the image you mirrored. ### Release binary: download and transfer On a machine with internet access, download the binary and the checksum file from [github.com/useobserver/agent/releases](https://github.com/useobserver/agent/releases): ```bash theme={null} VERSION=1.6.0 curl -fLO https://github.com/useobserver/agent/releases/download/agent-v${VERSION}/observer-agent-linux-x64 curl -fLO https://github.com/useobserver/agent/releases/download/agent-v${VERSION}/SHA256SUMS.txt ``` Copy both files to the target host and verify the binary there, after the transfer: ```bash theme={null} sha256sum -c SHA256SUMS.txt --ignore-missing chmod +x observer-agent-linux-x64 sudo mv observer-agent-linux-x64 /usr/local/bin/observer-agent ``` Then continue with the systemd steps in [Install from a release binary](/agent/quickstart/install-binary). ## Air-gapped and isolated networks Observer Cloud is a hosted service, so the agent needs an outbound HTTPS path to it, directly or through a proxy. No inbound connection is ever required. A network with no route to Observer Cloud at all is not supported: the agent can run and collect readings there, but nothing reaches your status pages. What the agent does tolerate is losing that path for a while. Every result is written to the [local queue](/agent/concepts/local-queue) first, a SQLite file at `BUFFER_PATH`, and a background process delivers it to the cloud in batches. When the cloud is unreachable: * The agent keeps probing and keeps queueing results. * Delivery retries with exponential backoff, from 1 second up to a maximum of 5 minutes between attempts. * The queue holds up to `BUFFER_MAX_ROWS` rows (default `10000`). Once it is full, the oldest rows are evicted to make room and the agent logs an error with the number dropped. * So that one bad row cannot block the queue forever, a row that fails 20 delivery attempts in a row is dropped with an error log. During a long outage this costs at most one row roughly every hour or two. * When the path returns, the backlog drains in batches of up to 100 rows per request, oldest first. How long an outage the default cap covers depends on how many probes the agent runs and how often. Raise `BUFFER_MAX_ROWS` if you expect long interruptions, and keep `BUFFER_PATH` on a persistent volume so the queue survives a restart. While the agent cannot reach the cloud it also stops sending heartbeats, so the console marks it offline and the `agent.offline` alert fires even though probing continues locally. For jobs on hosts that cannot reach the internet at all, the [heartbeat relay](/agent/guides/heartbeat-relay) lets them report heartbeat checks through the agent instead. ## Verify connectivity After starting the agent, check three places. **Agent logs.** A working connection logs `Successfully fetched metric definitions` within about 30 seconds. A failed one logs `Error fetching metric definitions:` followed by the cause. Heartbeat failures are only logged with `VERBOSE=true`, which is worth setting while you debug the network path. Typical causes: | Log content | Likely cause | | - | - | | `unable to get local issuer certificate`, `self signed certificate in certificate chain` | A TLS-intercepting proxy or private CA that the agent does not trust. Set `NODE_EXTRA_CA_CERTS`. | | `ECONNREFUSED`, `ETIMEDOUT`, `DNS` | No route to the cloud. Set `HTTPS_PROXY`, or ask for an egress rule to port 443. | | `407` or a proxy authentication error | The proxy requires credentials. Add them to the proxy URL. | | `redirect (not followed)` | Something between the agent and the cloud answers with a redirect, such as a captive portal. | | `HTTP 401` | The agent key is wrong or was rotated. | Where to read the logs: `docker logs -f observer-agent`, `docker compose logs -f observer-agent`, `kubectl -n observer logs -f deploy/observer-agent`, or `journalctl -u observer-agent -f`. **Local dashboard.** The agent serves a read-only dashboard on port `10101`, bound to loopback by default. Its *Cloud link* card shows the last heartbeat and the last push, with the error text when one failed, and the queue card shows whether results are piling up. See [Read the agent dashboard](/agent/guides/read-the-dashboard). **Console.** The agent's page in the console shows when it last checked in. A connected agent is marked **running** within 90 seconds of starting. If it stays offline while the logs show no errors, see [Diagnose a stalled agent](/agent/guides/diagnose-stalled-agent). To test the network path from the agent host before installing, make a request through the same proxy. Pass `--cacert` only when a TLS-intercepting proxy re-signs the traffic, because it replaces curl's default trust store: ```bash theme={null} curl -sS -o /dev/null -w '%{http_code}\n' \ --proxy http://proxy.corp.example:3128 \ --cacert /etc/observer/corp-ca.pem \ https://use.observer/ ``` Any HTTP status code means the path and the certificate chain work. A TLS error means the CA file is wrong or incomplete. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.