Threshold suggestions from observed data

Picking good thresholds is hard. Round numbers (warn at 500ms because it's round) are either too sensitive (noise) or too lax (real degradations slip through). Once a metric has about a week of history, Observer suggests thresholds grounded in its actual observed range, so you can calibrate to reality instead of guessing.

The suggestion appears on the metric's edit page, beside the threshold inputs. It is a starting point you apply, adjust, or dismiss. Observer never changes your thresholds on its own.

How it works

When you open a metric's edit page, Observer looks at the last 7 days of that metric's values and computes percentiles (p50, p95, p99, and the low end p5 / p1). From those it suggests bounds based on the metric's direction:

• Higher is worse (latency, error rate, queue depth): healthy under the 95th percentile, unhealthy over the 99th. Most values stay healthy; only the rare high outliers cross into unhealthy.
• Lower is worse (uptime, success rate, remaining quota): healthy over the 5th percentile, unhealthy under the 1st.

The panel shows the percentiles and a small distribution bar so you can see the data behind the suggestion before trusting it.

Why percentiles

Percentiles are transparent and defensible: "95% of observed values were under this number" is something you can reason about, unlike an opaque model output. This is deliberately a simple, explainable v1. Anomaly detection and seasonal models are not part of it.

Direction

Observer infers whether higher or lower is worse:

1. From your current threshold operators. If you already set "healthy under / unhealthy over", the metric is treated as higher-is-worse.
2. Failing that, from the metric name (latency, error, lag lean higher-is-worse; uptime, success, remaining lean lower-is-worse).
3. If neither is clear, no suggestion is shown. The panel asks you to set the operators yourself, and the next suggestion calibrates to them.

When a suggestion isn't shown

• Not enough history. Below about a week of data the panel says so and asks you to set thresholds from your operational targets.
• Unclear direction. If Observer can't tell whether higher or lower is worse and you haven't set operators, it shows no suggestion.

Trusting the suggestion

The suggestion is based on observed data, not a guaranteed optimum. If the last week was unusual (a sustained incident, a noisy deploy), the distribution reflects that. The distribution bar is there so you can spot a skewed window before you apply. Apply, then adjust the numbers to match what "healthy" should mean for your service.

Applying, adjusting, dismissing

• Apply writes the suggested operators and values into the form. You can still edit them before saving.
• Adjust anything by typing over the values.
• Dismiss hides the panel for this edit session.

Suggestions are computed from recent data and refreshed periodically; they are not recomputed on every page load.