Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

28.9 KB
Raw Blame

Recovery

Pulse collects backup, snapshot and replication evidence from connected providers. Use it to find recorded artifacts, identify gaps in protection and check the source of a backup before using the provider's own recovery tools. Pulse does not restore workloads from these views.

Choose the right backup

The Recovery settings panel and the platform backup views serve different jobs. Their backups are not interchangeable:

Backup What it is for What it does not establish
Provider backup, snapshot or replication artifact Recovering a monitored workload through the provider's own tools; see where to look A successful task does not prove guest thaw or application recovery
Create Backup in Settings → System → Recovery Encrypted transfer of the included Pulse configuration Not a workload backup or a full Pulse installation backup; history and agent enrolment state are excluded
Consistent filesystem/volume backup of Pulse Full-state recovery, including retained history and the matching encryption key Not a backup of the workloads Pulse monitors
Updater installation snapshot Version-specific server update recovery A recorded snapshot path does not prove a complete or consistent data backup

Restore Backup in the Recovery settings panel imports Pulse configuration, not a VM, container, dataset or PVC. Import replaces the included settings and API-token records; it does not merge them. Export the destination's existing configuration before importing, retain the original passphrase privately, and follow the transfer and validation steps. First confirm the installation, organisation and access for each side of the transfer. The archive passphrase does not grant permission to replace settings, and configuration import does not migrate organisation membership or rebind API tokens to a different organisation. Do not import configuration, restore Pulse data or restart monitoring to clear a guest freeze or an incorrect backup badge. Follow the independent guest-safety checks below and the provider's recovery procedure instead.

Where to look

There is no top-level Recovery page. Open the existing platform view:

  • Proxmox → Backups (/proxmox/backups): By date lists recorded artifacts; Coverage groups them by workload. Expand a coverage row to inspect its posture reason and individual artifacts. Backup servers have their own rows on the same page.
  • TrueNAS → Protection (/truenas/protection): snapshots and replication activity reported by the TrueNAS connection.
  • Kubernetes: recovery records depend on the agent reporting VolumeSnapshot or Velero inventory. An installed agent or a connected cluster does not establish that either is available.

Provider collection is separate from presentation. Not every platform displays the same columns or filters, and a missing reading is not proof that no backup exists. Check missing evidence before changing the provider or Pulse's stored data.

Why Proxmox backup counts differ

In Proxmox → Backups → By date, the inventory and completed-point totals answer different questions:

  • Backup entries includes running and failed entries, kept for inspection. The completed restore points total in Backup health excludes both, even when an incomplete entry has a size or a verification flag. The total covers the loaded inventory, not just the rows matching your list filters. For a row marked Unknown size, use the individual size checks.
  • Backup jobs counts the Proxmox backup tasks Pulse has retained for current workloads, including failed runs, and ignores the list filters. A run's outcome is separate evidence from the restore points: a failed run usually leaves none, and retention prunes older points from the list, so the two need not agree for any day.
  • Guest snapshots can contribute to completed-point totals. They are not independent backups and do not fill Coverage's Last backup. If a completed backup's date is unavailable or in the future, the latest backup age stays unknown rather than choosing an older dated backup.

The list follows the By date search, Node, Backup location, Type and Source filters, including Verified or Unverified. Backup health still covers the loaded inventory: a non-zero total there can sit beside an empty filtered list. An unresolved or ambiguous saved backup location shows no matching points; choose the intended location again or clear that filter, rather than treating the empty view as missing backups. An empty list is not proof that no backups exist; check any inventory loading or access warning first. None of these display controls changes collection, permissions, retention or protection policy. Older versions may use different labels or show a per-day restore point chart.

Completed is not Verified, a tested restore or guest thaw. Compare the same provider, guest, location and time using existing records, not just a count. Do not run a new backup, verification or restore, delete entries, or clear History to make totals agree. For contradictory PBS verification or protection, use the PBS evidence checks. An unresponsive guest or uncertain freeze/thaw needs the backup safety precaution, not a count check.

Restore-point sizes and unknown values

In Proxmox → Backups → By date, Unknown size means no usable byte count was reported, not that the restore point is empty, corrupt or failed. Missing sizes and zero placeholders both remain unknown. The point stays in the inventory; its completion, verification and protection are separate evidence. A positive size is the reported artifact size, not the guest's disk capacity, PBS datastore usage or deduplicated physical space.

On a wide screen, use the Size column. Sorting it keeps measured sizes ahead of unknowns in both directions; reversing the sort reverses only the measured order. On a phone, expand the restore-point row and read Size under Restore point. A hidden column is not a missing reading. In Coverage, expand the workload to see each point's size; narrow layouts show it alongside the point's identity. One known size does not fill the gaps for other points. For totals, see why backup counts differ.

Compare the same workload, repository and time in existing provider records before drawing a conclusion. Do not run another backup, verification or restore, delete a point or clear History just to obtain a size. Older versions may show a dash or zero instead; neither establishes an empty backup. If a guest is unresponsive or thaw is uncertain, stop this display comparison and follow Backup safety.

What the evidence means

Keep these observations separate:

Observation What it establishes What it does not establish
Successful task or success outcome A provider-reported result or collected artifact Guest thaw, application health or a tested restore
Running task or partial artifact Work or an incomplete artifact is observed A completed recovery point, even if a size is shown
Verified backup The provider's verification result for that artifact A successful application restore or guest thaw
Recent snapshot A point on the source system An independent copy that survives loss of that system
Protected posture Current, linked evidence meets Pulse's protection policy A recovery guarantee or permission to discard other backups

A backup task's OK status is not confirmation that the guest has thawed. For an affected installation, use the backup safety precaution before a planned freeze-enabled Proxmox backup. It covers the actual server deployment and automatic updaters, not just a guest agent. A planned pause is not an incident recovery procedure, and stopping Pulse does not cancel a guest-agent request already issued. If an existing operation's state is unknown, do not start a backup on the strength of a stopped service or an elapsed waiting period.

Keep Pulse stopped until the backup has ended and independent post-backup checks confirm all three:

  1. Guest thaw.
  2. Fresh successful workload writes to every filesystem covered by the backup.
  3. Workload liveness.

Use the workload's established safe checks, independent of Pulse and the QEMU Guest Agent. An OK task, an absent lock, a console connection, a successful read or a write to only the OS disk is not enough. If any check fails or is unavailable, leave Pulse and its automatic updater paused and use the guest/platform's recovery procedure. Do not force writes, repeat a backup or send guest-agent probes to fill the gap.

After all checks pass, restore only services and timers that were active before the pause. Unknown pre-pause states are not permission to start them. An updater must not restart a previously inactive server; follow the deployment-specific restoration steps in the precaution. Pulse monitoring and alert delivery are unavailable while it is stopped; arrange independent outage coverage. Do not disable filesystem freezing to make a task look successful. This precaution is not proof of a repaired or reproduced native thaw failure.

Before relying on an artifact, confirm its provider, repository, workload identity and time in the provider's own tools. For recovery assurance, test a restore to an isolated destination using that provider's procedure and check the restored application's data and operation. Do not overwrite the live workload as a diagnostic test, delete older backups, or change retention to hide a Pulse display problem.

Artifacts and subjects

A recovery point is a collected backup, snapshot or replication result. Its subject is the protected workload, dataset or PVC; its repository is the storage location. Pulse links subjects to monitored resources where it can, and retains provider-local references when it cannot.

Independent Proxmox installations can reuse a VMID. Compare the connection, node, guest type/ID, datastore, namespace and artifact time; matching names or VMIDs alone are not sufficient to identify a duplicate or the right restore. Direct PBS observations and PBS-through-PVE observations can differ in detail. See PBS data sources.

Backup location and guest placement

In Coverage and By date, expand the workload or backup row before deciding which installation owns it. Keep these identities separate:

Evidence Meaning and limit
Workload type, Target ID, PVE connection and Node The guest and its reported PVE scope and placement. A VMID is unique only within its owning installation; a missing connection or node stays Not reported.
Artifact Source and Location The inventory reader and backup repository, including a PBS datastore or namespace. A repository name is not a guest node, and its reader is not necessarily the guest's owning PVE connection.
Provider evidence → Scope The collection context for that evidence, not a substitute for missing workload ownership or placement.

Node: Not reported does not mean that a backup failed, a guest was deleted or its node is offline. An unmatched PBS backup may retain its repository and guest type/ID without a known PVE owner. Do not infer that owner or node from the datastore, namespace, connection label or a repeated VMID. Older versions may have shown one of those hints as a Node; that label alone is not native placement evidence.

The Node filter offers reported placement, not every backup location. If an expected row is absent under a selected node, choose All nodes and compare its Backup location or search for its repository instead. A namespace missing from the Node options is not evidence that its backups are missing. Clearing a display filter neither refreshes collection nor repairs identity or protection posture. On a phone, expand the row to read its full source and location rather than relying on the shortened summary.

Compare existing provider records with consistent private aliases for the installation, guest type/ID, repository and time. Keep unknown ownership unknown; do not rename nodes, move backups, widen permissions or recreate a connection to fill the label. Keep full identifiers and private paths out of public reports. If node errors or agent-backed Docker readings are also mixed, follow the cross-installation identity checks: one corrected backup label does not establish recovery in those other views. An unresponsive guest or uncertain thaw needs the separate backup safety procedure, not a filter change.

Outcomes and verification

Recorded outcomes are success, warning, failed, running or unknown. An unknown outcome is not success; an error fetching records is not an empty history. Where available, the API returns startedAt, completedAt, sizeBytes and provider-specific details. Missing times and sizes are not zero-valued measurements. A source backup timestamp is not necessarily the completion time of a later copy or sync task.

The verified field is optional: true is a positive provider verification observation; false supplies no positive verification, so inspect the provider's detailed state; an omitted value supplies no verification result. Do not interpret an omitted field as a failed verification. Rollups may also report verifyIntent (verified, stale or unknown) and lastVerifiedAt; these summarise observed verification recency, not an end-to-end restore test.

Protection posture

Pulse combines subject-linked artifacts with current provider collection evidence:

  • Protected: a qualifying current recovery point is linked to the resource and complete provider evidence does not invalidate the claim.
  • Attention: evidence is stale, failing, incomplete or unverified when verification is expected.
  • Unprotected: complete evidence confirms that no qualifying protection exists.
  • Unknown: identity, permissions or collection completeness cannot support a stronger claim.

A retained artifact can remain visible while posture is unknown. A running backup is not a new completed recovery point; inspect any earlier completed point separately rather than treating current activity as protection.

For PBS artifacts that disagree with the health strip, use the backup health checks to compare one workload's explanation and provider evidence without running a new backup or changing stored history. For local vzdump archives, use the PVE comparison below; PBS is not required.

Protection assessment times

In Proxmox → Backups → Coverage, expand the affected workload and read Protection assessment, where available. Its recorded times are shown in UTC, not your browser's local timezone:

  • Evaluated: when the server assessed protection, not when the backup completed or the provider last collected data. A recent evaluation can use older or incomplete evidence; read Provider evidence and the posture explanation as well.
  • Latest successful point: the successful-point time recorded by that assessment. It is not the latest task's result or necessarily Last backup in the current filtered list.
  • Last verified: the verification time recorded by the assessment, not proof of a tested restore, application health or guest thaw.

These clocks are workload-wide. Selecting a Backup location changes the listed artifacts and their backup age, not the assessment or its times. Compare the matching archive, owning installation and guest; repeated names or VMIDs do not establish identity. Keep an archive's creation time separate from its task's completion time, and compare the same instant with its timezone, not just the calendar date.

Not reported means that clock was not supplied; Unknown means its supplied value could not be interpreted. Neither means zero, no backups or a failed verification. Do not borrow another clock to fill it. If the section is absent or the assessment cannot be loaded, a retained artifact or green connection does not establish a current assessment. Older versions may not show these details; use existing provider records without rerunning collection.

Use existing observations only; do not run Diagnostics, another backup or a guest-agent probe, restart monitoring or clear History to reconcile the clocks. If a guest is unresponsive or freeze/thaw is uncertain, follow Backup safety instead. A current timestamp or corrected badge does not establish safe workload recovery.

Backup task results and backup age

A recent backup and a failed task can both be genuine. In Proxmox → Backups → Coverage, read these columns separately:

Column What to compare
Last backup (Backup or Age at narrower widths) The latest completed backup visible for the selected backup location, not the result of the latest task. A guest snapshot does not fill this column.
Task A collected task result for this workload, where the evidence establishes which task started last. It does not verify an artifact or replace its timestamp.
Posture Pulse's assessment of subject-linked protection evidence, not a synonym for either task success or backup age. A location filter does not recalculate this assessment.

Expand the workload, including on a phone, to inspect Latest task: Status, Started, Duration, Error and Task ID, where reported. Keep the workload's type, ID, node and PVE connection separate from the provider reader's Scope. Match the existing native task and backup record to that same owning installation and guest, not just a repeated name or VMID.

  • Failed does not erase an earlier completed backup. Inspect the task's error and the existing artifact independently; a recent backup age does not cancel the failure either.
  • Running is not a new completed recovery point. OK is a task result, not verification, a tested restore or confirmation that the guest thawed.
  • If Task order says Latest task: Unknown, missing, future or tied start times prevent Pulse from ordering several possible latest tasks. Inspect each Possible latest task section. Its number is not a time order: do not choose an OK task over a failed one to declare recovery.
  • None or a dash means no matching recent task is shown, not that no task ran in the provider. An Unknown status is not OK, and Not reported under Error is not proof that there was no error. If your version has no task details, use the existing native task record rather than assuming success.

Duration is a reported completed interval, not a live running timer or backup age. Read it separately from Status and Error:

  • Not reported: no end time was supplied. This alone does not establish whether the task is still running, completed or failed.
  • Unknown: an end time was supplied, but the timestamps or running status do not establish a completed duration. Examples include a missing or invalid start, a future time, an end at or before the start, or Running with a reported end. Unknown timing does not turn an OK status into a failure or clear a reported error.
  • 0s: a valid positive interval shorter than half a second can round to zero. Equal start and end times are Unknown, not a measured zero; absent or invalid timing is never replaced with 0s.

Compare the matching native task's existing start, end and status if available; do not fill a missing duration with the time since Started or another task's time. A displayed duration does not establish success, verification or guest safety. Older versions may not distinguish missing and invalid timing; use the existing native record if the labels differ.

Use the records already available. Do not run another backup, verification, restore or guest-agent probe, restart monitoring, clear History or change retention merely to reconcile these readings. Keep full task IDs, private paths and unredacted errors private; a public report needs only the affected view, running Pulse version, task status/time and a locally reviewed error summary with consistent aliases.

If a guest is unresponsive or freeze/thaw is uncertain, stop this display comparison and follow Backup safety. Neither an OK task nor a corrected badge confirms independent thaw, fresh successful writes to every covered filesystem and workload liveness.

Local PVE backups are newer than Coverage

A recent archive in PVE and an old or Unprotected Coverage row are different observations, not proof that the archive failed or the workload is protected. Compare one existing archive with the same workload in Pulse:

  1. Open Proxmox → Backups → By date. Clear search, start with All nodes and All backup locations, then select PVE backup files (PVE files on narrow screens). A list filter is not the protection policy; widening it does not refresh collection or change a backup's age.
  2. Expand the archive to read Source, Location, Created, State and Identifier under Details, including on a phone. Compare these with the existing record in the owning PVE installation: connection, native node, guest type/ID, storage, volume identifier and archive time. Keep the native task's completion result separate from the archive's creation time. The PVE source names the inventory reader, not necessarily local storage: a PBS-backed storage target can also appear as a PVE file.
  3. Open Coverage, clear its posture filter and expand that same workload. Read the posture explanation and Provider evidence: provider, Scope, Source, Reported outcome, History and Access. A successful inventory read is not a backup-job completion result; keep task and restore-point states separate. Keep All backup locations for a whole-workload comparison. Selecting a location limits the listed artifacts and backup age, but does not recalculate the workload's posture. A visible archive and a green connection do not establish that the posture assessment received the same complete, correctly linked evidence. Read the Protection assessment times separately from the archive and task clocks; a recent evaluation is not a fresh collection.
  4. Compare Last backup (Backup or Age at narrower widths) with the archive's timestamp. Restore evidence can show only a selection of the workload's points; use By date to locate an omitted archive. Unknown is not None: unavailable or future dates, or incomplete readable history, can prevent a reliable latest age. Do not substitute an older dated point for an unknown latest backup.

Use consistent private aliases when comparing installations: names, VMIDs and storage names can repeat. If the readings still disagree, report the running Pulse version, affected view, one archive's time/state and that workload's posture explanation and Provider evidence. Keep the full volume identifier, private paths, hostnames, credentials and full inventories out of public reports; retain the exact originals privately.

Do not run another backup, verification or restore, restart or re-enrol a node, clear History, change retention or widen permissions to change the badge. If a guest is unresponsive or freeze/thaw is uncertain, stop this display comparison and follow Backup safety. An OK task or a corrected posture does not establish guest thaw, successful writes to every covered filesystem or workload liveness.

Missing or inconsistent evidence

  1. Open the relevant platform view, clear its search and filters, and check the selected connection and organisation. Record the route that is affected.
  2. Inspect connection health, the latest observation time and any collection error. Wait for that provider's configured polling interval, not a fixed 30-second assumption. A green connection badge does not prove that every history or backup read succeeded.
  3. Check the expected artifact in the provider's own inventory using an existing authorised session. Verify its identity, location and permissions; do not grant write or restore permissions merely to fill a monitoring row.
  4. For an Unknown posture, expand the coverage row and use the specific limiting evidence. Resolve that collection or identity gap; do not turn retained artifacts into a protection claim by clearing Pulse history.

If the guest stopped responding during backup, preserve the task's warning or thaw error and the observations already available. Do not repeat the backup, reset the guest, or run active guest-agent probes just to reproduce it. Use the provider's recovery procedure; a separate trusted guest console can help assess the guest, but a console connection alone does not clear the precaution. Keep monitoring paused until the independent post-backup checks above confirm thaw, writes to every covered filesystem and workload liveness, not just until Pulse's backup badge looks healthy.

When reporting missing records, share the failing view, running Pulse version, affected provider, time range, expected reading and relevant redacted error. Use consistent placeholders for private hostnames, datasets and IDs. Do not share full backup inventories, request headers, tokens, cookies or recovery keys. See safe issue reporting.

API reference

These are read-only monitoring endpoints. Use an existing authenticated browser session or a monitoring:read token supplied through a private header file, never a token in a URL or diagnostic command argument. See API access.

Method Endpoint Result
GET /api/recovery/points Paged recovery points
GET /api/recovery/rollups Paged subject summaries
GET /api/recovery/series Completed-point counts grouped by day
GET /api/recovery/facets Filter values from matching points
GET /api/recovery/postures Canonical resource postures and limiting evidence

Point and rollup filters

Points, rollups, series and facets accept these filters:

Parameter Values or meaning
platform proxmox-pve, proxmox-pbs, truenas or kubernetes; provider is an alias
kind backup, snapshot or other; TrueNAS replication results use backup
mode local, remote or snapshot
outcome success, warning, failed, running or unknown
from Lower time bound in RFC3339, for example 2026-10-01T00:00:00Z
to Upper time bound in RFC3339
subjectResourceId Exact linked resource identity; itemResourceId is an alias
rollupId Exact subject summary identity
q Text search; query is an alias

For example, this relative browser request selects PBS backup records within the stated time bounds without putting a credential in the URL:

/api/recovery/points?platform=proxmox-pbs&kind=backup&from=2026-10-01T00:00:00Z&to=2026-10-02T00:00:00Z&page=1&limit=100

Use from/to, not since/until, and subjectResourceId, not subject. Unrecognised query names do not apply those filters. Invalid from or to values return an error rather than an empty result.

Points and rollups accept page (starting at 1) and limit (default 100, maximum 500). Their response has a data array and meta pagination; inspect every required page rather than assuming the first is the inventory. Series and facets return aggregate data, not that paged-list contract.

Posture lookup

/api/recovery/postures uses a separate filter contract. Repeat resourceId for a batch of at most 200 resource IDs, or omit it for a paged list. state=attention selects attention postures; page and limit control list pagination. Its response includes policy and provider evidence states. Do not apply point filters to this endpoint or re-derive posture from artifact count.

See also