Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Make copied webhook verification freshness-aware
pulse-triage[bot]41457c8a82bc
diff --git a/docs/WEBHOOKS.md b/docs/WEBHOOKS.mdindex 9899d5706d..3acf6649b4 100644--- a/docs/WEBHOOKS.md+++ b/docs/WEBHOOKS.md@@ -86,16 +86,58 @@ These fields and behaviors are stable; ticket-routing integrations can rely on t- `X-Pulse-Timestamp`: Unix seconds at send time.- `X-Pulse-Signature`: `v1=` + hex HMAC-SHA256 over `timestamp + "." + body`, keyed with the shared secret.-To verify: recompute the HMAC over the received timestamp and raw body, compare with constant-time equality, and reject requests whose timestamp is outside your tolerance window (e.g. 5 minutes) to block replays.+Verify before parsing the JSON or performing any action. Read the original body+as bytes and the complete `X-Pulse-Timestamp` and `X-Pulse-Signature` header+values; reject missing or duplicate signing headers. Do not re-serialize JSON,+trim the body, or trust a timestamp from the payload instead of the header.++This Python example accepts timestamps within five minutes of the receiver's+clock, rejects malformed headers, and compares the HMAC in constant time. Keep+both machines' clocks synchronised. Load the shared secret from private receiver+configuration, not from the request, a command argument or a log. Pulse trims+surrounding whitespace from its configured secret; the verifier does the same.+An empty secret must never authenticate a request.```python-import hashlib, hmac+import hashlib+import hmac+import re+import time++MAX_SKEW_SECONDS = 300+def verify(secret: str, timestamp: str, body: bytes, signature: str) -> bool:- expected = "v1=" + hmac.new(secret.encode(), f"{timestamp}.".encode() + body, hashlib.sha256).hexdigest()+ secret = secret.strip()+ if not secret or not isinstance(body, bytes):+ return False+ if not isinstance(timestamp, str) or re.fullmatch(r"[0-9]{1,12}", timestamp) is None:+ return False+ if not isinstance(signature, str) or re.fullmatch(r"v1=[0-9a-f]{64}", signature) is None:+ return False+ if abs(time.time() - int(timestamp)) > MAX_SKEW_SECONDS:+ return False+ expected = "v1=" + hmac.new(+ secret.encode(), timestamp.encode() + b"." + body, hashlib.sha256+ ).hexdigest()return hmac.compare_digest(expected, signature)```+A time window rejects old captures, **not every replay**: the same signed+request can arrive more than once within that window, and Pulse signs each+retry with its current send time. After verification, deduplicate atomically+before creating a ticket or taking another action. Use a durable key scoped to+the configured Pulse sender and destination, built from the authenticated+payload's alert ID and event (`{{.ID}}` and `{{.Event}}` in your template); keep+it across retries and receiver restarts. An `alert` and its later `resolved`+are different events, not duplicates. The `X-Pulse-Event-ID` header is a useful+correlation hint, but is **not covered by this HMAC**: do not let a changed+header bypass deduplication of an otherwise identical signed event. The sample+PSA payloads below include `alertId` and `event` for this purpose. A valid HMAC+establishes integrity, not that processing succeeded; retain your receiver's+normal durable processing and retry handling. Never log the secret or full+credential-bearing request while diagnosing a rejection.+The secret is write-only through the API: list responses mask it, and an update that echoes the masked placeholder keeps the stored secret.```httpdiff --git a/frontend-modern/browser-tests/webhook-verification-help.cjs b/frontend-modern/browser-tests/webhook-verification-help.cjsnew file mode 100644index 0000000000..f093455341--- /dev/null+++ b/frontend-modern/browser-tests/webhook-verification-help.cjs@@ -0,0 +1,81 @@+// Verify the copyable receiver example through production Docs/router/Markdown.+// This does not send a webhook or exercise a native receiver or deduplication.+const assert = require('node:assert/strict');+const fs = require('node:fs');+const path = require('node:path');+const { createHash } = require('node:crypto');+const { chromium, webkit } = require('playwright');++(async () => {+ const root = path.resolve('frontend-modern');+ const artifacts = path.join(root, 'node_modules', 'webhook-verification-browser');+ fs.mkdirSync(artifacts, { recursive: true });+ const guide = fs.readFileSync(path.join(root, 'public/docs/WEBHOOKS.md'), 'utf8');+ const example = guide.match(/```python\n([\s\S]*?)```/)[1].trim();+ process.chdir(root);+ const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js'));+ const server = await createServer({+ root, configFile: path.join(root, 'vite.config.ts'),+ cacheDir: path.join(root, 'node_modules', '.vite-webhook-verification'),+ server: { host: '127.0.0.1', port: 5255, strictPort: true },+ optimizeDeps: { noDiscovery: true, include: [] },+ });+ const results = [];+ let browser;+ try {+ await server.listen();+ for (const [engine, width, tone] of [['chromium', 1280, 'light'], ['webkit', 390, 'dark']]) {+ browser = await { chromium, webkit }[engine].launch(engine === 'chromium'+ ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] }+ : { headless: true });+ const page = await browser.newPage({ viewport: { width, height: 1000 },+ ...(engine === 'webkit' ? { isMobile: true, hasTouch: true } : {}) });+ const errors = [];+ page.on('pageerror', error => errors.push(error.message));+ page.on('console', message => { if (message.type() === 'error') errors.push(message.text()); });+ await page.goto('http://127.0.0.1:5255/browser-tests/docs-fragment-navigation.html',+ { waitUntil: 'domcontentloaded', timeout: 60_000 });+ if (tone === 'dark') await page.evaluate(() => document.documentElement.classList.add('dark'));+ await page.getByRole('link', { name: '← All documentation' }).click();+ const entry = page.locator('a[data-doc-link][href="/docs/WEBHOOKS"]').first();+ await entry.focus();+ await page.keyboard.press('Enter');+ const heading = page.getByRole('heading', { name: '📦 Delivery Contract', exact: true });+ await heading.waitFor();+ const code = page.locator('pre').filter({ hasText: 'MAX_SKEW_SECONDS = 300' });+ await code.scrollIntoViewIfNeeded();+ assert.equal((await code.locator('code').textContent()).trim(), example);+ assert.ok(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth + 1));+ await page.screenshot({ path: path.join(artifacts, `${engine}-verifier.png`) });+ const warning = page.locator('p').filter({ hasText: 'A time window rejects old captures' });+ await warning.scrollIntoViewIfNeeded();+ const text = await page.locator('article').innerText();+ for (const phrase of ['within five minutes', 'empty secret', 'not every replay',+ 'deduplicate atomically', 'receiver restarts', 'not covered by this HMAC',+ 'missing or duplicate signing headers', 'before parsing the JSON']) {+ assert.ok(text.includes(phrase), phrase);+ }+ assert.ok(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth + 1));+ await page.screenshot({ path: path.join(artifacts, `${engine}-deduplication.png`) });+ const payloadHeading = page.getByRole('heading', { name: 'Sample PSA payloads', exact: true });+ await payloadHeading.scrollIntoViewIfNeeded();+ assert.ok((await page.locator('pre').filter({ hasText: '"alertId": "{{.ID | jsonString}}"' })+ .textContent()).includes('"event": "{{.Event}}"'));+ assert.deepEqual(errors, []);+ results.push({ engine, browserVersion: browser.version(), width, tone,+ keyboardEntry: true, exactCopiedPython: true, freshnessAndReplayLimits: true,+ payloadIdentity: true, noBodyOverflow: true, errors });+ await browser.close();+ browser = undefined;+ }+ const result = { result: 'passed', playwrightVersion: require('playwright/package.json').version,+ scope: 'rendered guide and exact copied code; not webhook delivery or receiver processing',+ docs: { 'frontend-modern/public/docs/WEBHOOKS.md':+ createHash('sha256').update(guide).digest('hex') }, results };+ fs.writeFileSync(path.join(artifacts, 'result.json'), JSON.stringify(result, null, 2) + '\n');+ console.log(JSON.stringify(result));+ } finally {+ if (browser) await browser.close();+ await server.close();+ }+})().catch(error => { console.error(error); process.exitCode = 1; });diff --git a/frontend-modern/public/docs/WEBHOOKS.md b/frontend-modern/public/docs/WEBHOOKS.mdindex 9899d5706d..3acf6649b4 100644--- a/frontend-modern/public/docs/WEBHOOKS.md+++ b/frontend-modern/public/docs/WEBHOOKS.md@@ -86,16 +86,58 @@ These fields and behaviors are stable; ticket-routing integrations can rely on t- `X-Pulse-Timestamp`: Unix seconds at send time.- `X-Pulse-Signature`: `v1=` + hex HMAC-SHA256 over `timestamp + "." + body`, keyed with the shared secret.-To verify: recompute the HMAC over the received timestamp and raw body, compare with constant-time equality, and reject requests whose timestamp is outside your tolerance window (e.g. 5 minutes) to block replays.+Verify before parsing the JSON or performing any action. Read the original body+as bytes and the complete `X-Pulse-Timestamp` and `X-Pulse-Signature` header+values; reject missing or duplicate signing headers. Do not re-serialize JSON,+trim the body, or trust a timestamp from the payload instead of the header.++This Python example accepts timestamps within five minutes of the receiver's+clock, rejects malformed headers, and compares the HMAC in constant time. Keep+both machines' clocks synchronised. Load the shared secret from private receiver+configuration, not from the request, a command argument or a log. Pulse trims+surrounding whitespace from its configured secret; the verifier does the same.+An empty secret must never authenticate a request.```python-import hashlib, hmac+import hashlib+import hmac+import re+import time++MAX_SKEW_SECONDS = 300+def verify(secret: str, timestamp: str, body: bytes, signature: str) -> bool:- expected = "v1=" + hmac.new(secret.encode(), f"{timestamp}.".encode() + body, hashlib.sha256).hexdigest()+ secret = secret.strip()+ if not secret or not isinstance(body, bytes):+ return False+ if not isinstance(timestamp, str) or re.fullmatch(r"[0-9]{1,12}", timestamp) is None:+ return False+ if not isinstance(signature, str) or re.fullmatch(r"v1=[0-9a-f]{64}", signature) is None:+ return False+ if abs(time.time() - int(timestamp)) > MAX_SKEW_SECONDS:+ return False+ expected = "v1=" + hmac.new(+ secret.encode(), timestamp.encode() + b"." + body, hashlib.sha256+ ).hexdigest()return hmac.compare_digest(expected, signature)```+A time window rejects old captures, **not every replay**: the same signed+request can arrive more than once within that window, and Pulse signs each+retry with its current send time. After verification, deduplicate atomically+before creating a ticket or taking another action. Use a durable key scoped to+the configured Pulse sender and destination, built from the authenticated+payload's alert ID and event (`{{.ID}}` and `{{.Event}}` in your template); keep+it across retries and receiver restarts. An `alert` and its later `resolved`+are different events, not duplicates. The `X-Pulse-Event-ID` header is a useful+correlation hint, but is **not covered by this HMAC**: do not let a changed+header bypass deduplication of an otherwise identical signed event. The sample+PSA payloads below include `alertId` and `event` for this purpose. A valid HMAC+establishes integrity, not that processing succeeded; retain your receiver's+normal durable processing and retry handling. Never log the secret or full+credential-bearing request while diagnosing a rejection.+The secret is write-only through the API: list responses mask it, and an update that echoes the masked placeholder keeps the stored secret.```httpdiff --git a/scripts/tests/test_webhook_verification_docs.py b/scripts/tests/test_webhook_verification_docs.pynew file mode 100644index 0000000000..6092c377b5--- /dev/null+++ b/scripts/tests/test_webhook_verification_docs.py@@ -0,0 +1,151 @@+#!/usr/bin/env python3+"""Execute the receiver verifier copied from the shipped webhook guide.++Synthetic signatures exercise freshness, malformed inputs and byte integrity.+This is not a running receiver, durable deduplication or notification delivery.+"""++from __future__ import annotations++import hashlib+import hmac+from pathlib import Path+import re+import unittest+from unittest.mock import patch+++ROOT = Path(__file__).resolve().parents[2]+DOC = ROOT / "docs/WEBHOOKS.md"+NOW = 1_791_030_000+SECRET = "synthetic-receiver-test-secret"+BODY = b'{"alertId":"alert-42","event":"alert","message":"test"}\r\n'+++def copied_verifier():+ blocks = re.findall(r"```python\n(.*?)```", DOC.read_text(), re.DOTALL)+ if len(blocks) != 1:+ raise AssertionError("expected one copyable webhook verifier")+ namespace = {}+ exec(compile(blocks[0], str(DOC), "exec"), namespace)+ return namespace["verify"]+++def sign(timestamp: str, body: bytes = BODY, secret: str = SECRET) -> str:+ # Independent wire-format oracle; do not use the copied implementation.+ message = timestamp.encode("utf-8") + b"." + body+ return "v1=" + hmac.new(secret.encode("utf-8"), message, hashlib.sha256).hexdigest()+++class WebhookVerificationDocsTest(unittest.TestCase):+ def check(self, timestamp=str(NOW), *, body=BODY, signature=None, secret=SECRET, now=NOW):+ if signature is None and isinstance(timestamp, str):+ signature = sign(timestamp, secret=secret.strip())+ with patch("time.time", return_value=now):+ return copied_verifier()(secret, timestamp, body, signature)++ def test_shipped_copy_matches_the_executed_guide(self):+ self.assertEqual(DOC.read_bytes(),+ (ROOT / "frontend-modern/public/docs/WEBHOOKS.md").read_bytes())++ def test_current_signed_payload_is_accepted(self):+ self.assertIs(self.check(), True)+ self.assertIs(self.check(secret=" " + SECRET + "\n"), True)++ def test_five_minute_boundary_and_nearby_delivery_are_accepted(self):+ for delta in (-300, -1, 0, 1, 300):+ with self.subTest(delta=delta):+ self.assertIs(self.check(str(NOW + delta)), True)++ def test_old_valid_signature_cannot_be_replayed_after_the_window(self):+ timestamp = str(NOW)+ self.assertIs(self.check(timestamp), True)+ for delta in (301, 3_600, 86_400):+ with self.subTest(delta=delta):+ self.assertIs(self.check(timestamp, now=NOW + delta), False)++ def test_future_valid_signature_is_not_accepted_beyond_the_window(self):+ for delta in (301, 3_600, 86_400):+ with self.subTest(delta=delta):+ self.assertIs(self.check(str(NOW + delta)), False)++ def test_signed_malformed_timestamps_are_rejected_without_an_exception(self):+ # These are valid MACs for invalid headers, not ordinary bad signatures.+ # Bound integer parsing and accept only the sender's ASCII seconds.+ values = ("", " " + str(NOW), str(NOW) + "\n", "+" + str(NOW),+ str(NOW) + ".0", "1e9", "١٧٩١٠٣٠٠٠٠", str(NOW) + "," + str(NOW),+ "9" * 5_000)+ for timestamp in values:+ with self.subTest(timestamp=timestamp[:30]):+ self.assertIs(self.check(timestamp), False)+ self.assertIs(self.check(None, signature=sign(str(NOW))), False)++ def test_malformed_or_missing_signatures_fail_closed(self):+ good = sign(str(NOW))+ for signature in ("", "v2=" + good[3:], good.upper(), good + "\n",+ good + "," + good, "v1=" + "é" * 64, b"v1=" + b"a" * 64):+ with self.subTest(signature=str(signature)[:30]):+ self.assertIs(self.check(signature=signature), False)+ with patch("time.time", return_value=NOW):+ self.assertIs(copied_verifier()(SECRET, str(NOW), BODY, None), False)++ def test_empty_or_whitespace_key_cannot_authenticate(self):+ for secret in ("", " \n\t"):+ with self.subTest(secret=repr(secret)):+ self.assertIs(self.check(secret=secret), False)++ def test_body_timestamp_key_and_signature_are_all_authenticated(self):+ good = sign(str(NOW))+ cases = (+ {"body": BODY.replace(b"alert-42", b"alert-43")},+ {"timestamp": str(NOW + 1)},+ {"secret": SECRET + "-other"},+ {"signature": "v1=" + ("0" if good[3] != "0" else "1") + good[4:]},+ )+ for changed in cases:+ with self.subTest(changed=list(changed)):+ self.assertIs(self.check(**{"signature": good, **changed}), False)++ def test_raw_bytes_not_reserialised_json_are_signed(self):+ body = '{"message":"héllo", "alertId":"alert-42", "event":"alert"}\r\n'.encode()+ signature = sign(str(NOW), body=body)+ self.assertIs(self.check(body=body, signature=signature), True)+ self.assertIs(self.check(body=body.rstrip(), signature=signature), False)+ self.assertIs(self.check(body=body.replace(b", ", b","), signature=signature), False)+ self.assertIs(self.check(body=body.decode(), signature=signature), False)++ def test_go_sender_deterministic_vector_uses_the_same_wire_format(self):+ # The production sender's TestSignWebhookPayloadDeterministic uses+ # these exact key/timestamp/body bytes and an independent Go MAC.+ sender_test = (ROOT / "internal/notifications/webhook_signing_test.go").read_text()+ self.assertIn('signWebhookPayload("secret", "1700000000", []byte(`{"a":1}`))', sender_test)+ signature = "v1=" + hmac.new(b"secret", b'1700000000.{"a":1}', hashlib.sha256).hexdigest()+ self.assertIs(self.check("1700000000", body=b'{"a":1}', signature=signature,+ secret="secret", now=1_700_000_000), True)++ def test_digest_comparison_remains_constant_time(self):+ verifier = copied_verifier()+ compare = hmac.compare_digest+ with patch("time.time", return_value=NOW), patch("hmac.compare_digest", wraps=compare) as spy:+ self.assertIs(verifier(SECRET, str(NOW), BODY, sign(str(NOW))), True)+ spy.assert_called_once_with(sign(str(NOW)), sign(str(NOW)))++ def test_signed_retry_is_valid_but_not_a_new_logical_event(self):+ # Freshness alone is not at-most-once processing. A receiver needs+ # durable/atomic payload-derived deduplication, not a new MAC per retry.+ first, retry = sign(str(NOW)), sign(str(NOW + 60))+ self.assertNotEqual(first, retry)+ self.assertIs(self.check(signature=first), True)+ self.assertIs(self.check(str(NOW + 60), signature=retry, now=NOW + 60), True)+ # Changing the correlation header cannot change a timestamp/body MAC.+ # The docs must not claim that X-Pulse-Event-ID is authenticated.+ text = " ".join(DOC.read_text().split())+ for phrase in ("not every replay", "deduplicate atomically", "authenticated payload's",+ "receiver restarts", "different events", "not covered by this HMAC",+ "missing or duplicate signing headers", "before parsing the JSON",+ "not from the request", "Never log the secret"):+ self.assertIn(phrase, text)+++if __name__ == "__main__":+ unittest.main()