Skip to main content

Availability and SLA

Operations > Availability and SLA measures the uptime of your Proxmox nodes and clusters, follows SLA targets over groups of guests, and takes declared maintenance out of the figures. An incident register groups the alerts of an outage, with a journal, corrective actions and a post-mortem that exports as a PDF.

Both are Enterprise features. The provider sees every measured connection; a tenant that owns whole connections sees those, and a vDC tenant does not see the page.

Quick Path​

Before you start

  • An Enterprise license that includes availability monitoring.
  • At least one Proxmox VE connection.

Start measuring

  1. Open Operations > Availability and SLA and click the settings icon at the top right.
  2. Tick the connections to measure and save, see Measured connections.
  3. Create an SLA policy in the Policies tab, see SLA policies.
  4. Declare planned work in the Maintenance tab before it starts, see Maintenance windows.

Check the result

  • The Overview tab draws a timeline per connection, filling from the moment the connection was ticked.
  • The Policies tab shows each policy against its target.

Measured connections​

Nothing is measured until an administrator ticks connections. The orchestrator then records, for every ticked connection, the state of its nodes, the quorum of its cluster and the state of its guests.

Measured connections dialog: the availability registry only records the connections ticked here, their nodes, cluster quorum and guests; PVE-DR and PVE-PROD are ticked
Tick the connections the availability registry records.

Unticking a connection stops its recording; its history is kept until the retention removes it. The dialog names the SLA policies that still cover guests of a connection before you untick it.

The history is kept 400 days by default, and new policies start with a 99.9 % target. Both are set in Settings > Availability and SLA: the retention from 30 to 1825 days, the default target from 90 to 99.999 %.

Settings > Availability and SLA: history retention 400 days and default SLA target 99.9 percent, with the Save button
How long the history is kept and which target new policies start with.

Overview​

Availability and SLA overview over the last hour: tiles for nodes up 5 of 6, clusters with quorum 2 of 2, covered guests down 0, nodes down 0, in maintenance 1 and connections under 100 percent coverage 0, then one timeline per connection, PVE-DR up and PVE-PROD in maintenance
The tiles summarise the measured estate, and each connection gets one timeline, folded by default.

The tiles count the nodes up, the clusters with quorum, the covered guests down, the nodes down or in maintenance, and the connections whose coverage is below 100 %. Pick the period at the top right: the last hour, 24 hours, 7 or 30 days, a year, or a custom range.

Each connection shows one summary timeline, drawn as status-page bars. Expand all, or the arrow of a connection, unfolds its rates and one timeline per host.

PVE-PROD unfolded: node availability, coverage and cluster availability at 100 percent, then one timeline for the cluster and for each of pve1, pve2 and pve3, pve1 in maintenance, with the legend Up, Down, Maintenance, Not observed and Absent
An unfolded connection: its three rates, then the cluster and each host.
StateMeaning
UpThe object answered and was healthy.
DownThe object was observed down.
MaintenanceThe node was in maintenance mode, or inside a declared maintenance window.
Not observedThe registry could not see the object, for instance while the orchestrator was stopped.
AbsentThe object did not exist yet, or was not measured at that time.

How the rates are calculated:

  • Availability = up time / (up time + down time).
  • Coverage = (up time + down time) / (up time + down time + not observed time). It tells how much of the period the registry actually saw.
  • Maintenance and absent periods are left out of both rates.
  • A rate is shown as not measured when nothing was observed, never as 0 %.

Click an outage on a timeline to open the incident that holds it, or to create one from the matching alert.

SLA policies​

A policy sets an availability target for a group of guests, measured over the last 30 days.

New SLA policy dialog: name Production web tier, target 99.9 percent, policy enabled, covered guests with the Guests, Tags and Pools tabs, the guest list filtered on web-0 with web-01 and web-02 ticked under PVE-PROD
A policy covers ticked guests, guests carrying a tag, or the members of a Proxmox pool.
FieldDescription
NameUnique within the tenant.
Target (%)From 90 to 99.999.
Policy enabledA disabled policy keeps its definition but is not evaluated.
Covered guestsA guest is covered when it is ticked, carries one of the tags, or belongs to one of the pools. Tag and pool membership is evaluated when the figures are calculated, not when the policy is saved.

Only the guests of measured connections are offered. A policy refuses a newly ticked guest on an unmeasured connection, keeps the guests it already holds, and shows how many of them are not measured.

Policies tab: the policy Production web tier covering 5 guests, status Met, target 99.9 percent, availability over 30 days 100 percent and the downtime budget used
Each policy against its target, with the downtime budget it has used. Unfold a row to see the guests below target.

The downtime budget is the downtime the target allows over the period, and the bar shows how much of it is used. Open a row to list the guests below target.

Budget notifications​

SLA is a notification category: it sends a warning once a policy has spent the chosen share of its downtime budget, then a critical notification once the target is lost for the period. Set the Budget period (calendar month or last 30 days, rolling) and the Warning threshold, and subscribe channels, in Notifications. Covered guests going down and a lost cluster quorum come through the Alerts category.

SLA report​

The SLA report type of Reports exports the policies, their availability and the outages, in PDF or CSV, on demand or on a schedule.

Maintenance windows​

During a declared window, the downtime of the targeted objects counts as maintenance, not as an outage. Declare a window before planned work so that it does not count against the targets.

New maintenance window dialog: scope Node, connection PVE-PROD, node pve2, start and end dates, End after 1 h, 4 h, 8 h or 24 h, reason Proxmox VE 9.1 upgrade of pve2
A window targets a connection, a node, a guest or every guest of a policy.
FieldDescription
ScopeA whole connection, a node, a guest, or the guests of a policy.
Start / EndA window lasts 31 days at most. End after sets the end 1, 4, 8 or 24 hours after the start.
ReasonRequired, shown in the list and in the SLA report.
Maintenance tab: an upcoming window on PVE-PROD / pve2, with its start, end, duration of 4 hours, reason and author
Windows are listed as upcoming, ongoing or past, with who declared them.

A node put in maintenance mode from ProxCenter, for a rolling update for instance, is counted as maintenance even without a declared window.

Incident register​

An incident groups the alerts of one outage. Open one from an alert, with the incident button of its row in Alerts, or from an outage of a timeline. The Incidents tab of the Alerts page lists them, filtered as open, resolved or closed, and the alert list gains an Incident column.

An open critical incident for Node pve1-dr: Storage critical: start date, ongoing, duration, one active alert, SLA report not included, post-mortem to do; the Alerts card with the alerts of the same outage to attach; the Post-mortem card with root cause, impact and a corrective action; the Journal with one comment; and the Incident report card with Download the PDF
An incident: its alerts, the journal, the post-mortem and the PDF report.
  • Alerts. The incident follows its alerts by start time. Alerts of the same outage proposes the other alerts of the same node, its guests or its cluster raised within 15 minutes: tick those with the same cause and attach them.
  • Journal. Comments record the exchanges and decisions as the incident is followed. An internal note stays out of the SLA report and of the PDF.
  • Lifecycle. The incident resolves on its own once every alert is resolved. It closes only when the post-mortem is validated, with a root cause and an impact.
  • Post-mortem. Write the root cause and the impact, list the corrective actions and tick them as they are done, then Validate the post-mortem.
  • Incident report. Download the PDF exports the key facts, the alerts, the post-mortem and the public journal. Until the post-mortem is validated, the report is marked as a draft.

An alert held by an incident cannot be deleted. A batch delete of alerts says how many were kept for that reason.

The register belongs to the provider tenant, and every view is narrowed to the alerts the user may see.