Availability: uptime, outages, and honest percentages
RootTrace records the state of every check over time and turns that history into availability. Not just "is it broken now", but how often it breaks, how long it stays broken, whether it always breaks on a Tuesday at 03:00, and what you can honestly claim in an SLA review.
Nothing to configure. Once a collector reports, the ledger fills.
The three numbers, and why there are three
A single availability percentage always hides an assumption. RootTrace shows which one:
| Number | What it assumes | Use it for |
|---|---|---|
| Raw | Time nobody measured counts as downtime | The honest floor. Never flattering, never wrong. |
| Over known time | Only measured time counts | An entity monitored for part of the window. |
| Adjusted | Known time, minus declared maintenance and minus time the host was unreachable | SLA reviews and error budgets. |
Every report also carries a confidence figure: how much of the window was actually measured. Below 95% the UI greys the percentage and labels it provisional, because a 100% uptime figure computed from four hours of a 30-day window is not a fact.
Nagios comparison. Nagios shows two of these (the plain number and the bracketed known-time one). It has no equivalent of the adjusted number: scheduled downtime is counted as downtime by design, with no setting to change it, and the documented workaround is to disable checks before maintenance so the data never exists. RootTrace derives maintenance at report time instead, so a window you declare after an incident still corrects the report.
What gets measured
Every check result becomes a state:
| Result | State | Counts as available |
|---|---|---|
| pass | Up | yes |
| warn | Degraded | yes |
| fail | Down | no |
| missing / unreadable | Not measured | neither |
Warn counts as available on purpose. A disk at 82% is serving traffic. Counting every warning as downtime would put most fleets under 90% and make the number useless. Degraded time is reported on its own line so it stays visible.
An entity is one monitored thing: the same (environment, service, host, check, resource) grouping that RootTrace already uses for issues. "Availability of X" and "the thing that pages me about X" are the same X.
What is covered
| Signal | In the availability ledger |
|---|---|
| Every collector check (reference), including custom Python checks | yes |
| Collector heartbeat, per host | yes |
| APM services (latency, error rate, throughput) | no (use a latency or error-rate SLO) |
| Logs, OTLP | no |
APM is deliberately out of scope. A request-based service already has the right instrument in the SLO page: good and bad requests, counted exactly from histograms. Recasting that as "seconds up" would be a worse measurement of the same thing. Availability answers the question for signals that are sampled rather than counted: every collector check, and the heartbeat.
Undetermined time
When a collector stops reporting, the record goes dark. RootTrace credits the previous state for one more check interval and then records the rest as not measured.
This is the difference between an availability report and a lie. Silence is not uptime: a collector that dies during an outage would otherwise show a perfect month precisely because nobody was watching.
The check interval is learned per entity from its own cadence, so a check that runs every ten minutes is not mistaken for a collector outage every ten minutes.
Confirmed and unconfirmed failures
RootTrace records Nagios's soft/hard distinction: a failure becomes confirmed after three consecutive failing samples.
- Downtime counts all failing time, confirmed or not.
- Outage statistics (count, MTTR, MTBF, longest outage) count only confirmed failures, so a one-sample blip does not drag MTTR toward zero.
This is recorded and displayed only. It does not change when you get paged. RootTrace still opens an issue on the first failing sample exactly as before.
Flap detection
An entity that alternates between states scores a flap percentage over its last 21 states, weighted so recent changes count more than old ones. That is the same algorithm Nagios uses. Above 20% it is marked as flapping; the mark clears below 5%, so the flag itself does not flap.
Flapping is surfaced on the entity, in the sidebar badge, and as its own ranking under Top alert producers.
Dependency-aware downtime
When a host's collector heartbeat is confirmed down, every other check on that host is failing because the host is unreachable, not because each thing is independently broken.
RootTrace attributes that time to host unreachable rather than to the check, reports it on its own line, and removes it from the adjusted number. This is Nagios's parent/child relationship, with no configuration: the heartbeat is automatically the parent of everything else on its host.
The heartbeat is recorded when it changes, not on a cadence, so its state holds until something contradicts it. A heartbeat that fails at 02:00 and recovers at 05:00 is three hours of downtime, not three minutes followed by undetermined time.
The page
Availability in the sidebar, under Operate.
The window selector offers 24h/7d/30d/90d, or Custom range for an exact start and end date and time. A custom range applies to every tab and to the per-entity detail sheet, and each report states the window it measured. Pick a preset while you are scanning; pick a range once you know roughly when something broke and want to see only that.
- Report: every entity with its availability, trend against the previous window, MTTR, outage count, and a confidence chip.
- Timeline: a state strip per entity across the window, with deploys and other change events drawn over it. An outage band and the deploy that preceded it line up visually.
- Patterns: a day-by-hour heatmap of outage starts, and top alert producers ranked three ways: most downtime, most outages, most flapping. Nagios ranks by alert count alone, which flatters the check that has been down for a solid week and produced one alert.
Availability SLOs
An SLO can target an availability entity instead of an APM service. Pick Availability as the indicator and choose the entity.
The error budget is denominated in known seconds, and maintenance and unreachable time leave the budget entirely: an SLO cannot spend minutes nobody was watching, and should not spend minutes you announced in advance. Burn-rate alerting works exactly as it does for latency and error-rate SLOs.
This is what lets a disk check, a TLS expiry, or a host heartbeat carry an objective. See the SLO page for the burn-rate model.
Maintenance windows
Declare them under Settings → Maintenance windows. A window suppresses new issues while it is open, and (separately) removes its time from the adjusted availability number and from any availability error budget.
Because the exclusion is computed when a report is read rather than stamped when the data arrives, a window recorded after the work still corrects historical reports. Windows that have already ended are accepted for exactly this reason: nobody declares maintenance at 3 a.m. with the database down, and the report should still reflect that the outage was planned. Backdating is bounded by your data retention: a window older than the state history it would adjust has nothing left to correct.
Raw and known-time availability are deliberately left alone. Only the adjusted figure moves, so the honest floor is always still on the page.
Retention
State history follows your plan's data retention, like every other tenant record. It is far smaller than the raw check results it summarises: one document per state change rather than one per sample. That is what makes a 90-day availability report a cheap query when scanning the underlying samples would not be.
On first upgrade RootTrace reconstructs history from the per-minute check rollups it still holds, so the page is not blank for a month. Set ROOTTRACE_AVAILABILITY_BACKFILL_DAYS (default 30, 0 to skip) to bound that replay. Reconstruction is only as good as the surviving rollups: gaps in them become undetermined time, which is the honest outcome.
API
All endpoints are workspace- and environment-scoped like the rest of the API, and take either a preset time_range of 1h, 24h, 7d, 30d, 90d, or 365d, or an explicit start and end.
Explicit bounds win over time_range, and start alone means "from then until now". Use them once you know roughly when something broke. A preset cannot answer "what happened between 21:00 and 23:30 last night":
GET /api/availability/entities/{key}?start=2026-07-29T21:00:00Z&end=2026-07-29T23:30:00ZThe window is capped at 366 days and must be at least a minute. An end in the future is clamped to now rather than rejected, so a client whose clock is ahead still gets a sensible answer instead of a block of undetermined time dragging every percentage down. Every response reports the window_start and window_end actually measured, which is what the UI displays. Quote that, not what you asked for.
| Endpoint | Returns |
|---|---|
GET /api/availability/overview | Every entity, with the current and previous window |
GET /api/availability/entities/{entity_key} | One entity's full report plus its state runs |
GET /api/availability/timeline | State bands for the timeline strip |
GET /api/availability/histogram | Outage starts by hour and by day-and-hour |
GET /api/availability/top-producers | Worst offenders, ranked three ways |
GET /api/availability/sla | An SLA table against a target_pct |