Skip to content

Autonomous development for real products. Governed agents build from specs you approve, follow the rules you set, and nothing merges until another agent proves it. Your decisions, judgement and taste are the bar that work must clear. Declare your vision and let agents build it.

fix(relay): probe blocked uploads without resealing [H15,H17]

Corey Olson692351b349bc
diff --git a/docs/api.md b/docs/api.md
index c4d8139a..359d79cf 100644
--- a/docs/api.md
+++ b/docs/api.md
@@ -367,7 +367,7 @@ The boards response keeps `boards` and may include `warnings` for registered ent
The state from `pullboard serve` lists every item as a summary of what the lists and the roadmap read: `id`, `title`, `lane`, `status`, `route`, `milestone`, `specs`, `claim`, `builtBy`, `blockedBy`, `updatedAt`, `stage`, `since`, `counts` and `verdict`. `milestone` names the first roadmap milestone that lists the item, or is `null`. `claim` is the live lease on the item, `{by, until}`: its builder's while it is claimed, or its reviewer's while a submitted item is reserved for review; otherwise it is `null`. `stage` is the item's status, except `accepted` for a verified item and `merged` once a merge is recorded. `since` is when the item entered that stage: edits, review reservations and renewals do not restart it, and a lapsed claim's stage starts at its recorded lease deadline. `counts` has the item's number of `verdicts` and `facts`. `verdict` keeps only the latest verdict's `decision` and `reason`, or is `null`; for an item sent back and not verified, it also keeps `note`, the first nonblank line of that reject's note. The state also has `moveCounts`, the board's lifecycle moves keyed `from>to:verb`, as the view's lifecycle drawing counts them.
-`GET /api/v1/boards/:board/items/:id` returns `{item}`: the item's `id` and every field its summary leaves out, which are `owner`, `reviewer`, `reviewUntil`, `verifiedBy`, `criterion`, `brief`, `commit`, `merged`, `history`, `check`, `checkBaseline` when recorded, `verdicts`, `thread`, and the latest `verdict` in full. Laid over its summary, the detail is the item exactly as the full state carries it. An unknown id returns `NO_ITEM` with HTTP 404; a malformed id is refused with HTTP 400. `state?detail=full` returns every item in full, summary fields included, and `detail=summary` returns summaries; any other value is refused with `BAD_REQUEST`. Without `detail`, a server answers with its own default, `stateDetail: "summary"` or `"full"`: `serveApi`, which `pullboard serve` runs, defaults to summaries, and `createLocalApiHandler`, which a page's own server builds, defaults to full items until that page reads summaries. `GET /api/v1/boards/:board/items?q=text` returns `{ids}`, in state order, for the items whose `#id`, title, lane, spec ids or criterion contain the trimmed text, ignoring case, as the view's search matches; empty text matches every item. An adapter without these reads, such as the relay, whose state carries every item in full, returns `ITEM_NOT_AVAILABLE`.
+`GET /api/v1/boards/:board/items/:id` returns `{item}`: the item's `id` and every field its summary leaves out, which are `owner`, `reviewer`, `reviewUntil`, `verifiedBy`, `criterion`, `brief`, `commit`, `merged`, `history`, `check`, `checkBaseline` when recorded, `verdicts`, `thread`, and the latest `verdict` in full. Laid over its summary, the detail is the item exactly as the full state carries it. An unknown id returns `NO_ITEM` with HTTP 404; a malformed id is refused with HTTP 400. `state?detail=full` returns every item in full, summary fields included, and `detail=summary` returns summaries; any other value is refused with `BAD_REQUEST`. Without `detail`, a server answers with its own default, `stateDetail: "summary"` or `"full"`: `serveApi`, which `pullboard serve` runs, defaults to summaries, and `createLocalApiHandler`, which a page's own server builds, defaults to full items until that page reads summaries. `GET /api/v1/boards/:board/items?q=text` returns `{ids}`, in state order, for the items whose `#id`, title, lane, spec ids or criterion contain the trimmed text, ignoring case, as the view's search matches; empty text matches every item. An adapter without these reads returns `ITEM_NOT_AVAILABLE`. The paired API2 phone transport implements these reads by decrypting addressed pages; its initial state is bounded and full detail follows separately.
Every state names the event it reflects as `sequence`, the latest event id when it was read. `GET /api/v1/boards/:board/state?since=N` answers `{since, sequence, eventLogVersion, delta}`, where `sequence` is the new latest and `delta` brings the summary state this server sent at event N up to date: `delta.replace` holds whole fields that changed, `delta.merge` holds keyed lists by field, as `{key, set, ids?}`, with only their changed entries and, when entries came, went or moved, the full key order, and `delta.remove` names fields that went away. Items merge by `id`, `shouts` and `asked` by `shout_id`, `events` by `event_id`, `agents` by `agent_id`, `spec` and `practice` rows by `id`, and `milestones` by `name`. `unseen`, `proofStats` and the Needs-you lists, `decisions`, `holds`, `requests` and `personRequests`, always come whole. Apply a delta without recomputing anything: copy `replace` over the state, delete each `remove` field, and for each `merge` field, overlay `set` on the entries by `key` and order them by `ids`, or keep their order. `applyStateDelta(state, delta)` in `src/api-http.js` does exactly this, and the result equals a fresh snapshot at `sequence`. A delta also carries changes that wrote no event, such as a spec or doctrine edit, so its `sequence` may equal N. N alone cannot say which of the snapshots sent at N a client holds, so after such a change a later delta from the same N may repeat it; it still applies exactly.
@@ -450,6 +450,33 @@ The initial command output names `.pullboard/signers`, `.pullboard/first-commit`
Every signed row and signer-list change binds the initial signer-list hash in its canonical text. Rewriting `.pullboard/signers.initial`, or relabelling that hash in earlier receipts, invalidates those receipts.
+## Paged relay snapshots
+
+Relay API2 preserves the `version: 1` response envelope, executable moves and
+seal envelope. `GET /api/v2` advertises sealed-page support. The authenticated
+`/api/v2/boards/:board/state`, `/uploads`, `/uploads/:id/objects/:hash`,
+`/uploads/:id/publish`, `/objects/:hash`, and `/storage` routes exchange bounded
+opaque objects and server-derived storage totals; their full transport contract
+is in [relay/README.md](../relay/README.md#paged-snapshots-and-account-storage).
+
+The phone installs a current-state presentation under 900,000 bytes before
+streaming complete history. It uses the same lifecycle `stage`, `since`, claim,
+counts and summary projection as the local API. Full item details, search and old
+shout addresses can read selected encrypted pages before the complete stream.
+The transport's `historyProgress(boardId)` returns `{loaded, total, complete,
+sequence}` and optional `error`. Completion means all declared native rows and
+item details arrived for that checkpoint, not merely that current lists rendered.
+A newer current checkpoint can appear while an older pinned stream completes;
+historical completion never rewinds the live cursor.
+
+Account quota responses use `ACCOUNT_STORAGE_LIMIT` with HTTP 507 and
+`error.storage: {account: {id, login}, usedBytes, limitBytes}`. The same validated
+fields apply to API1 and API2 refusals. `GET /api/v2/boards/:board/storage` returns
+`{version: 1, apiVersion: 2, storage, warning?}` without downloading the snapshot.
+All routes retain current authorization and credential-derived sender checks.
+An API1 request for a paged head fails explicitly with `RELAY_API_VERSION` and
+upgrade guidance; it never returns an older or partial image.
+
## Sealed person requests
The authenticated relay page calls `cockpitPage` with `readOnly: true` and the separate `requests: true` capability. This enables only the person's literal intents through the sealing transport; generic API writes remain refused. Local read-only pages and exported snapshots omit this capability and refuse every action.
diff --git a/docs/formats.md b/docs/formats.md
index 370b7e14..2c42ff81 100644
--- a/docs/formats.md
+++ b/docs/formats.md
@@ -116,6 +116,22 @@ The board file is in the repository's Git common directory at `.git/pullboard/bo
| `move engine` | `56` | `ENGINE_VERSION` in `src/machine.js`, carried as `engine` in every sealed executable move | Bump when a move's meaning changes, independently of the event log and sealed envelope. A newer engine is refused with `ENGINE_VERSION`, naming both versions and asking you to upgrade Pullboard. |
<!-- format-versions:end -->
+Relay transport capability `apiVersion: 2` is independent of these versions and
+of the unchanged `version: 1` HTTP envelope and seal envelope. API2 uses
+`pullboard-paged-snapshot/1` inside encrypted directory pages, with a current-state
+presentation below 900,000 bytes, full current fields, native tables and keyed
+item/shout history references. Each sealed object is content-addressed by the
+SHA-256 of its ciphertext; plaintext blocks are at most 262,144 bytes, ciphertext
+at most 400,000 bytes and descriptor fanout at most 64. Devices validate bounds,
+board/sequence binding, range counts, ancestry and the complete restored native
+shape. The relay validates only the opaque graph. Its journal format 4 migrates
+formats 2 and 3 in place and stores pages in the same SQLite backup as the
+current root. Old API1 clients explicitly refuse an API2 head with upgrade
+guidance. A private device-only page cache stores no board key or plaintext
+record; its fingerprints and hashed session-notice receipts never cross HTTP.
+See [the operator protocol](../relay/README.md#paged-snapshots-and-account-storage).
+
+
Pullboard 0.6.1 released engine 1. Engine 2 records a fresh explicit-build claim's skipped-review snapshot. Pullboard 0.8.1 released engine 3, including lane-sensitive graft refusals and authenticated relay senders. Engine 4 adds captured background-baseline completions through `completeCheckBaseline`; released engine-3 clients stop with `ENGINE_VERSION` before interpreting this new operation or advancing their replay cursor. Engine 5 requires a one-line reason when releasing a review and gives that reviewer a one-hour cooldown for the current submission. Each sealed move replays under its recorded engine version: engines 1 through 4 retain the original release event and review-reservation behavior. Engine 5 retains support for older moves, including engine-4 baseline completions. Pullboard 0.8.3 released engine 6, adding board-scoped machine credentials and permitting machine execution only for one exact, earlier authenticated phone request, including its payload digest. Engine 7 adds ordered item holds through `holdItem` and `releaseItemHold`; engine-6 clients stop with `ENGINE_VERSION` before interpreting either operation or advancing their replay cursor. Engine 8, shared by the 0.8.5 move changes, retains merged-item exception notes, selects the next review in roadmap order, and adds coordinator-only `reopen` for a submission returned without a verdict; engine-7 clients stop with `ENGINE_VERSION` before interpreting this new operation or advancing their replay cursor. Recorded engines 1 through 7 retain their original tier-first review selection. Engine 8 also adds durable refusal recovery for blocked relay positions; each explicit skip publishes the readable prefix even when another unsupported record follows. Engine 8 also permits coordinator criterion/check edits on a live claim as pending corrections; the holder cannot renew or submit until a refreeze adopts the correction while preserving the claim. Older request receipts retain their recorded engine semantics. Upgrade every linked device before using the new moves. Each release shares one engine bump across its changed moves. Engine 8 also retains declared model labels on registration and records each registered agent move's model snapshot; engines 1 through 7 retain their original registration and event details without these model fields. Pullboard 0.8.8 released engine 9, which allows resubmitting the same rejected commit after a corrected criterion is refrozen, but only when the rejection names a different frozen digest. Engine 8 retains its HEAD_NOT_NEW refusal and prior submit-event detail. Engine-9 submit events add `criterionDigest`, and engine-8 boards replay with the original refusal and event history.
Engine 10 adds coordinator-only reopening for an unmerged verified item after a red landing candidate. Its append-only `reopen` event names the superseded accept, failing test and landing merge commit; the original accept and verdict remain in history. Engines 8 and 9 retain their submitted-only `reopen` behavior: a verified-item reopen is refused without changing the replayed board. Only engine 10 may return a verified item to open, with a failing test and landing merge commit in the note. Dependency edits also require engine 10: an `editItem` move with `after`, including `after: []` to clear dependencies, cannot be replayed under an older engine. Ordinary edits without `after` retain their historical behavior. Older clients refuse an engine-10 move before changing their board or replay cursor; upgrade Pullboard before sending or applying dependency edits.
diff --git a/relay/README.md b/relay/README.md
index 19f84a5b..281defe3 100644
--- a/relay/README.md
+++ b/relay/README.md
@@ -120,7 +120,7 @@ credentials, repositories, or board databases are needed.
## Sealed API relay
createRelayHandler({ directory, auth, publicOrigin, pollMs }) from service.js
-mounts API v1 on the existing sign-in component. Mount the auth handler first,
+mounts the version-1 envelope and the API1/API2 transports on the existing sign-in component. Mount the auth handler first,
then this handler, on one HTTP server; call the handler's close() at shutdown.
serveRelay({ directory, auth, port, host, publicOrigin }) is a convenience
server for these API routes. Its default address is loopback and its default
@@ -155,12 +155,74 @@ The client owns the encryption format and engine-version checks.
It requires a person's signed-in session and current write access.
It removes the link and its board-scoped credentials too; other boards and person sessions remain valid.
-All responses and refusals retain API version 1. Moves use the shared 100,000
-byte JSON-body limit. Snapshots allow a bounded 14,000,000 byte JSON body and
-10,000,000 decoded bytes. One private SQLite journal per board stores only its
+All responses and refusals retain envelope `version: 1`. Moves use the shared
+100,000 byte JSON-body limit. Legacy API1 snapshots allow a bounded 14,000,000
+byte JSON body and 10,000,000 decoded bytes; API2 uses the bounded objects below. One private SQLite journal per board stores only its
identity, format, head cursor, receive times, public sender identities and sealed payloads. Compaction
keeps the head cursor, so the next move never reuses an earlier sequence.
+### Paged snapshots and account storage
+
+Upgrade the relay before linking a board larger than the legacy cap. `GET /api/v2`
+advertises `apiVersion: 2`, `snapshot: "sealed-pages"` and the per-object bounds.
+A new CLI preserves legacy reads and small legacy uploads, then negotiates API2
+for a large upload or an existing paged head. Once paged, a board cannot silently
+downgrade. An older API1 reader receives `RELAY_API_VERSION` and
+`npm i -g pullboard@latest`; it never receives an incomplete legacy snapshot.
+This transport does not change the executable move engine or seal envelope.
+
+API2 routes are under `/api/v2/boards/:id`:
+
+- `POST /uploads` takes `{sequence, resume?}` and returns a principal-bound
+ upload id. An unfinished upload lasts one hour and can resume after a lost reply.
+- `PUT /uploads/:upload/objects/:hash` takes `{sequence, sealed, children}`.
+ Ciphertext has a SHA-256 content id, at most 400,000 bytes, at most 64 opaque
+ child ids, and a JSON request smaller than 600,000 bytes. Devices authenticate
+ and decrypt every object; a private plaintext fingerprint is never uploaded.
+- `POST /uploads/:upload/publish` takes `{root, initialOnly?}`. The relay checks
+ the complete opaque graph and commits its root with journal coverage in one
+ SQLite transaction. A partially staged graph never replaces the readable head.
+- `GET /state` returns a legacy head or `{apiVersion: 2, sequence, sender,
+ root, read, storage, warning?}`. The short-lived read pin retains this exact
+ root while a newer head is published. It expires after fifteen idle minutes.
+- `GET /objects/:hash?pin=...&via=...` reads only an object reachable along the
+ bounded path from that principal's pinned root. Knowing a staged hash grants
+ no read access. Every request rechecks the board's authorization.
+- `GET /storage` returns only `{storage, warning?}` plus version fields. It
+ reclaims expired staging across the server-derived account before reporting
+ whether uploads can resume. It sends no snapshot or plaintext board record.
+
+The phone decrypts a current-state presentation under 900,000 bytes first.
+Full current fields and all native tables, item threads and old shouts follow
+in sealed background pages, newest history first; addressed reads can fetch
+one older item or shout immediately. `transport.historyProgress(boardId)` names
+`loaded`, `total`, `complete`, and `sequence`; partial or failed streams do not
+claim completion. Live ordered moves continue while a prior read pin finishes.
+Native restoration retains every table, counter, event and receipt.
+
+The default account quota is **256,000,000 decimal bytes**. Set
+`PULLBOARD_ACCOUNT_STORAGE_BYTES` to a positive whole-byte value and restart the
+service to change it. The board's link owner is assigned by authenticated server
+state; no request can choose another account. Usage counts logical retained
+sealed journal payloads and cipher pages, including unfinished uploads and
+live pinned roots, across that account's boards. SQLite allocation, auth
+metadata and backups are separate operator disk costs. A quota refusal is
+`ACCOUNT_STORAGE_LIMIT` (HTTP 507) with validated account, used-byte and limit
+fields; the phone notice and CLI relay status name that account and limit.
+Unlinking deletes its stored relay copy only through the normal approved flow.
+Maintenance and storage probes collect unreachable pages after upload/read
+expiry; live read pins and the current complete head remain protected.
+
+While a size or quota limit blocks an upload, ordinary commands probe only
+capabilities or account totals. They retain the encrypted pending candidate
+without re-sealing it and report the pause once per local session. A relay
+upgrade resumes a baseline containing all locally accumulated work. Quota
+recovery retries a retained checkpoint before capturing newer state; accepted
+baselines keep subsequent moves in relay order. General malformed-upload
+refusals retain their existing retry policy. Device cache files hold ciphertext,
+private fingerprints and hashed notice receipts under `PULLBOARD_HOME/relay-pages/`;
+no board key or decrypted record is written there.
+
Every CLI relay request carries `X-Pullboard-Engine` with its executable engine
version. Board-content requests require engine 3 once the board has issued any
agent token. Revocation, expiry and a service restart do not lower that minimum;
diff --git a/relay/pages-http.js b/relay/pages-http.js
index 9b25872f..cd783654 100644
--- a/relay/pages-http.js
+++ b/relay/pages-http.js
@@ -29,7 +29,7 @@ function cipherBytes(value) {
}
/** Add the new snapshot transport without changing API1 native board documents or executable move formats. */
-export function createRelayPagesHandler({ directory, now, authorize, withJournal, sender, quota, annotated, enabled = true }) {
+export function createRelayPagesHandler({ directory, now, authorize, withJournal, sender, quota, annotated, refreshStorage, enabled = true }) {
/** Open objects only while the caller's shared board lifecycle lock and fresh authorization hold. */
function withPages(id, who, work, create = false) {
return withJournal(id, journal => {
@@ -61,12 +61,13 @@ export function createRelayPagesHandler({ directory, now, authorize, withJournal
? { apiVersion: 2, snapshot: 'sealed-pages', pageBytes: PAGE_BYTES, cipherBytes: PAGE_CIPHER_BYTES, fanout: PAGE_FANOUT }
: { apiVersion: 1, snapshot: 'legacy' });
if (!enabled) throw new Refused('RELAY_API_VERSION', 'Upgrade the relay authentication adapter to serve relay API version2.');
- const route = /^\/api\/v2\/boards\/([0-9a-f]{32})\/(state|uploads(?:\/([A-Za-z0-9-]{1,128})(?:\/(publish|objects\/([0-9a-f]{64})))?)?|objects\/([0-9a-f]{64}))$/u.exec(url.pathname);
+ const route = /^\/api\/v2\/boards\/([0-9a-f]{32})\/(state|storage|uploads(?:\/([A-Za-z0-9-]{1,128})(?:\/(publish|objects\/([0-9a-f]{64})))?)?|objects\/([0-9a-f]{64}))$/u.exec(url.pathname);
if (!route) throw new Refused('BAD_REQUEST', 'Use a supported relay API version2 sealed-page route.');
const id = route[1];
const write = req.method !== 'GET';
const who = await authorize(req, id, write);
const principal = sender(who, write ? 'snapshot' : false);
+ if (req.method === 'GET' && route[2] === 'storage') return reply(res, 200, { apiVersion: 2, ...refreshStorage(id, principal) });
if (req.method === 'GET' && route[2] === 'state') {
const state = withPages(id, who, (journal, pages) => head(journal, pages, principal, id));
return reply(res, 200, { apiVersion: 2, state });
diff --git a/relay/service.js b/relay/service.js
index 2da129b4..ed61775a 100644
--- a/relay/service.js
+++ b/relay/service.js
@@ -13,7 +13,7 @@ import { createPairingStore } from './pairing-store.js';
import { createPairingHandler } from './pairing-http.js';
import { createRelayBrowserHandler } from './browser-page.js';
import { VERIFIER_STANDING_ENGINE } from '../src/machine.js';
-import { createRelayPagesHandler } from './pages-http.js';
+import { createRelayPagesHandler, snapshotPageRoot } from './pages-http.js';
import { createStorageQuota } from './storage-quota.js';
const SNAPSHOT_BODY = 14_000_000;
@@ -128,7 +128,11 @@ export function createRelayHandler({ directory, auth, pollMs = 200, publicOrigin
/** Expose only a stable failure code to the trusted operator; never log paths, keys or provider errors. */
function maintain() {
- try { const result = retention.maintain(); maintenanceError = null; return result; }
+ try {
+ const result = retention.maintain();
+ if (quota) for (const link of auth.linkedBoards()) collectPages(link.id);
+ maintenanceError = null; return result;
+ }
catch (error) { maintenanceError = error instanceof Refused ? error.code : 'RELAY_MAINTENANCE'; throw error; }
}
const timer = maintenanceMs ? setInterval(() => { try { maintain(); } catch { /* operator reads maintenanceStatus */ } }, maintenanceMs) : null;
@@ -212,6 +216,28 @@ export function createRelayHandler({ directory, auth, pollMs = 200, publicOrigin
});
}
+ /** Collect expired staging and read pins without opening a missing board or decoding its opaque bytes. */
+ function collectPages(id) {
+ try { lstatSync(join(root, identity(id) + '.journal.sqlite')); }
+ catch (error) { if (error.code === 'ENOENT') return; throw error; }
+ auth.withBoard(id, () => {
+ const journal = createRelayJournal({ directory: root, boardId: id, maxBytes: STORED_BYTES, now: () => new Date(now()) });
+ const pages = journal.pages({ now });
+ try { const current = snapshotPageRoot(journal.snapshot()); pages.prune(current ? [current.id] : []); }
+ finally { pages.close(); journal.close(); }
+ });
+ }
+
+ /** Probe only account totals after expired staging is reclaimed, never a full snapshot or a device-supplied account. */
+ function refreshStorage(id, principal) {
+ return auth.withBoard(id, () => {
+ const account = auth.storageAccount(id, principal.userId);
+ if (account) for (const board of auth.storageBoards(account.id)) collectPages(board);
+ const storage = quota.usage(id, principal), warning = quota.notice(id);
+ return { storage, ...(warning ? { warning } : {}) };
+ });
+ }
+
/**
* Let token issuance see a published seat receipt under the same board lock that appends it, so
* a receipt claims its target before, and whether or not, its enrollment finishes (#579). An
@@ -264,7 +290,7 @@ export function createRelayHandler({ directory, auth, pollMs = 200, publicOrigin
}
const browser = origin ? createRelayBrowserHandler({ authenticate: (req) => auth.authenticate(credential(req, origin)) }) : null;
- const pagesHandler = createRelayPagesHandler({ directory: root, now, authorize: authorizedRequest, withJournal, sender, quota, annotated, enabled: Boolean(quota) });
+ const pagesHandler = createRelayPagesHandler({ directory: root, now, authorize: authorizedRequest, withJournal, sender, quota, annotated, refreshStorage, enabled: Boolean(quota) });
const common = createApiHandler({
authenticate: async (req, { board, write }) => {
const token = credential(req, origin, write);
diff --git a/src/api-http.js b/src/api-http.js
index d6213b0a..09b416d8 100644
--- a/src/api-http.js
+++ b/src/api-http.js
@@ -1,5 +1,6 @@
/** API v1's shared HTTP boundary; local and relay adapters supply the same board operations (A2). */
import { Refused } from './refused.js';
+import { relayStorage } from './relay-storage.js';
import { refusalDocument } from './json.js';
import { withGitFacts } from './git.js';
@@ -7,7 +8,10 @@ const BODY_BYTES = 100_000;
/** Keep unexpected internal failures out of the public refusal document. */
function refusal(error) {
- return refusalDocument(error instanceof Refused ? error : new Refused('SERVICE_ERROR', 'the board service could not complete the call; retry or check the service'));
+ const document = refusalDocument(error instanceof Refused ? error : new Refused('SERVICE_ERROR', 'the board service could not complete the call; retry or check the service'));
+ const storage = error instanceof Refused && error.code === 'ACCOUNT_STORAGE_LIMIT' ? relayStorage(error.storage) : null;
+ if (storage) document.error.storage = storage;
+ return document;
}
/** Emit exactly one versioned JSON response with no caching or content sniffing. */
@@ -352,6 +356,7 @@ export function applyStateDelta(state, delta) {
/** Choose the same HTTP refusal class for local and sealed relay adapters. */
export function apiStatus(error) {
+ if (error instanceof Refused && error.code === 'ACCOUNT_STORAGE_LIMIT') return 507;
return { RELAY_BOARD_DELETED: 410, AUTH_REQUIRED: 401, DEVICE_REVOKED: 403, TOKEN_BOARD: 403, API_ORIGIN: 403, BAD_ORIGIN: 403, WRITE_REQUIRED: 403, HUMAN_REQUIRED: 403, NO_REPO_ACCESS: 403, NO_BOARD: 404, NO_SHOUT: 404, BOARD_NOT_LINKED: 404, NO_SNAPSHOT: 404, BASELINE_EXISTS: 412, SNAPSHOT_REQUIRED: 409, SEQUENCE_REPEAT: 409, SEQUENCE_GAP: 409 }[error.code] ?? (error instanceof Refused ? 400 : 500);
}
diff --git a/src/relay-page-cache.js b/src/relay-page-cache.js
index 92d166b9..7de1572f 100644
--- a/src/relay-page-cache.js
+++ b/src/relay-page-cache.js
@@ -31,6 +31,7 @@ export function createRelayPageCache(state) {
CREATE TABLE IF NOT EXISTS meta (key TEXT PRIMARY KEY,value TEXT NOT NULL);
CREATE TABLE IF NOT EXISTS object (id TEXT PRIMARY KEY,sequence INTEGER NOT NULL,payload BLOB NOT NULL,children TEXT NOT NULL,ack INTEGER NOT NULL DEFAULT 0);
CREATE TABLE IF NOT EXISTS fingerprint (hash TEXT PRIMARY KEY,id TEXT NOT NULL REFERENCES object(id) ON DELETE CASCADE,reference TEXT NOT NULL);
+ CREATE TABLE IF NOT EXISTS notice (session TEXT PRIMARY KEY,created INTEGER NOT NULL);
CREATE TABLE IF NOT EXISTS job (source TEXT PRIMARY KEY,reference TEXT NOT NULL,upload TEXT);
CREATE TABLE IF NOT EXISTS job_object (source TEXT NOT NULL REFERENCES job(source) ON DELETE CASCADE,id TEXT NOT NULL REFERENCES object(id),PRIMARY KEY(source,id));`);
const saved = db.prepare('SELECT key,value FROM meta ORDER BY key').all();
@@ -153,7 +154,18 @@ export function createRelayPageCache(state) {
} catch (error) { db.exec('ROLLBACK'); throw error; }
}
+ /** Retain one credential-free pause notice receipt per session across CLI processes, with bounded private metadata. */
+ function firstNotice(session) {
+ if (!/^[0-9a-f]{64}$/u.test(session ?? '')) throw invalid();
+ const inserted = db.prepare('INSERT OR IGNORE INTO notice VALUES (?,?)').run(session, Date.now()).changes;
+ db.exec('DELETE FROM notice WHERE session NOT IN (SELECT session FROM notice ORDER BY created DESC,rowid DESC LIMIT 128)');
+ return inserted === 1;
+ }
+
+ /** A successful upload ends the pause interval for all local sessions. */
+ function clearNotices() { db.exec('DELETE FROM notice'); }
+
/** Close the private cache after every request, including refused or interrupted uploads. */
function close() { db.close(); }
- return { file, object, put, lookup, remember, job, prepare, upload, pending, acknowledge, missing, complete, close };
+ return { file, object, put, lookup, remember, job, prepare, upload, pending, acknowledge, missing, complete, firstNotice, clearNotices, close };
}
diff --git a/src/relay-page-client.js b/src/relay-page-client.js
index 31999406..78b0d3ba 100644
--- a/src/relay-page-client.js
+++ b/src/relay-page-client.js
@@ -1,5 +1,7 @@
/** Native bounded snapshot upload/resume and complete restore; no private cache fingerprint crosses HTTP [H15,H17]. */
import { Refused } from './refused.js';
+import { relayStorage } from './relay-storage.js';
+export { relayStorage, relayStorageLine } from './relay-storage.js';
import { createRelayPageCache, snapshotSourceId } from './relay-page-cache.js';
import { openPagedSnapshot } from './relay-page-layout.js';
import { writePagedSnapshot } from './relay-page-writer.js';
@@ -18,20 +20,6 @@ function bytes(value) {
return Uint8Array.from(decoded);
}
-/** Keep account quota status only after validating its credential-free response fields. */
-export function relayStorage(value) {
- if (!value || typeof value.account?.id !== 'string' || typeof value.account?.login !== 'string' ||
- !/^[A-Za-z0-9_-]{1,100}$/u.test(value.account.id) || !/^[A-Za-z0-9_-]{1,100}$/u.test(value.account.login) ||
- !Number.isSafeInteger(value.usedBytes) || value.usedBytes < 0 || !Number.isSafeInteger(value.limitBytes) || value.limitBytes < 1) return null;
- return { account: { id: value.account.id, login: value.account.login }, usedBytes: value.usedBytes, limitBytes: value.limitBytes };
-}
-
-/** Print account, used bytes and the operator-set limit without importing server-only quota storage into the CLI. */
-export function relayStorageLine(value) {
- const storage = relayStorage(value);
- return storage ? `relay storage: account ${storage.account.login}, ${(storage.usedBytes / 1_000_000).toFixed(1)} MB used of ${(storage.limitBytes / 1_000_000).toFixed(1)} MB` : '';
-}
-
/** Adapt the old internal pending-record interface only after the new protocol has proved an exact root ACK. */
export async function pagedSnapshotRequest(state, options, { request, key, decode, encode, maxEngine }) {
const path = '/api/v2/boards/' + state.board;
@@ -168,6 +156,8 @@ export async function relaySnapshotRequest(state, path, options = {}, io, hooks)
if (snapshot && (state.snapshotApiVersion === 2 || oversized) && await supportsPages()) return paged();
try { return await hooks.raw(path, options); }
catch (error) {
+ const storage = relayStorage(error.storage);
+ if (storage) { state.storage = storage; hooks.persist(); }
if (!snapshot || error.code !== 'RELAY_API_VERSION' || !await supportsPages()) throw error;
return paged();
}
diff --git a/src/relay-storage.js b/src/relay-storage.js
new file mode 100644
index 00000000..13fd5af0
--- /dev/null
+++ b/src/relay-storage.js
@@ -0,0 +1,15 @@
+/** Shared credential-free account storage status for all transport versions [H15,H17]. */
+/** Keep account quota status only after validating its credential-free response fields. */
+export function relayStorage(value) {
+ if (!value || typeof value.account?.id !== 'string' || typeof value.account?.login !== 'string' ||
+ !/^[A-Za-z0-9_-]{1,100}$/u.test(value.account.id) || !/^[A-Za-z0-9_-]{1,100}$/u.test(value.account.login) ||
+ !Number.isSafeInteger(value.usedBytes) || value.usedBytes < 0 || !Number.isSafeInteger(value.limitBytes) || value.limitBytes < 1) return null;
+ return { account: { id: value.account.id, login: value.account.login }, usedBytes: value.usedBytes, limitBytes: value.limitBytes };
+}
+
+/** Print account, used bytes and the operator-set limit without importing server-only quota storage into the CLI. */
+export function relayStorageLine(value) {
+ const storage = relayStorage(value);
+ return storage ? `relay storage: account ${storage.account.login}, ${(storage.usedBytes / 1_000_000).toFixed(1)} MB used of ${(storage.limitBytes / 1_000_000).toFixed(1)} MB` : '';
+}
+
diff --git a/src/relay-upload-pause.js b/src/relay-upload-pause.js
new file mode 100644
index 00000000..564cc902
--- /dev/null
+++ b/src/relay-upload-pause.js
@@ -0,0 +1,50 @@
+/** A blocked upload probes small metadata and reports once per real device session [H15,H17]. */
+import { createRelayPageCache, snapshotSourceId } from './relay-page-cache.js';
+import { relayStorage } from './relay-storage.js';
+import { agentSessionDigest } from './person.js';
+import { Refused } from './refused.js';
+
+/** Classify a durable transport limit without trusting arbitrary relay text as executable advice. */
+export function uploadPauseType(pause) {
+ if (pause?.code === 'ACCOUNT_STORAGE_LIMIT') return 'quota';
+ if (pause?.size || /(?:exceeds?|over|larger than|too large).{0,40}(?:bytes|snapshot|limit)|(?:snapshot|limit).{0,40}(?:exceeds?|over|too large)/iu.test(pause?.message ?? '')) return 'size';
+ return null;
+}
+
+/** Definitive quota and HTTP client refusals pause; throttling and uncertain transport failures retain exact retries. */
+export function permanentUploadRefusal(error) {
+ return error?.code === 'ACCOUNT_STORAGE_LIMIT' && error.httpStatus === 507 ||
+ Number.isInteger(error?.httpStatus) && error.httpStatus >= 400 && error.httpStatus < 500 && ![408, 425, 429].includes(error.httpStatus);
+}
+
+/** Ordinary commands never materialize or re-seal a known blocked snapshot just to rediscover its limit. */
+export async function uploadRetryReady(state, pause, request) {
+ const type = uploadPauseType(pause);
+ if (!type) return true;
+ if (type === 'size') {
+ try {
+ const ready = (await request('/api/v2')).apiVersion === 2;
+ if (ready) state.snapshotApiVersion = 2;
+ return ready;
+ }
+ catch (error) { if (error.httpStatus === 404 && error.relayVersion === 1) return false; throw error; }
+ }
+ const result = await request('/api/v2/boards/' + state.board + '/storage');
+ const storage = relayStorage(result.storage);
+ if (result.apiVersion !== 2 || !storage) throw new Refused('RELAY_RESPONSE', 'Retry the configured relay to read the account storage limit.');
+ state.storage = storage;
+ return result.warning?.code !== 'ACCOUNT_STORAGE_LIMIT';
+}
+
+/** Hash an existing session identity locally; never change it or persist a raw shell marker. */
+export function firstUploadPauseNotice(state) {
+ const session = agentSessionDigest() ?? snapshotSourceId(process.env.TERM_SESSION_ID || 'terminal-parent:' + process.ppid);
+ const cache = createRelayPageCache(state);
+ try { return cache.firstNotice(session); } finally { cache.close(); }
+}
+
+/** End notice suppression only after a real accepted baseline or checkpoint, not a successful probe. */
+export function clearUploadPauseNotices(state) {
+ const cache = createRelayPageCache(state);
+ try { cache.clearNotices(); } finally { cache.close(); }
+}
diff --git a/src/relay.js b/src/relay.js
index 6da555a6..533f12dd 100644
--- a/src/relay.js
+++ b/src/relay.js
@@ -1,3 +1,4 @@
+import { clearUploadPauseNotices, firstUploadPauseNotice, permanentUploadRefusal, uploadPauseType, uploadRetryReady } from './relay-upload-pause.js';
/** Opt-in, sealed relay ordering, with crash-safe migration from the older mirror [H1,H3,H5,H16,H17,P5]. */
import {
createHash,
@@ -114,7 +115,6 @@ const BASELINE_ATTEMPTS = new WeakMap();
const BASELINE_NOTICES = new WeakSet();
const CHECKPOINT_NOTICES = new WeakMap();
const CHECKPOINT_MOVES = new WeakMap();
-const BASELINE_LOCAL_MOVES = new WeakSet();
const BASELINE_FAILURES = new WeakMap();
const AUTH_NOTICES = new WeakSet();
const DELETION_NOTICES = new WeakSet();
@@ -500,7 +500,7 @@ export function pendingSnapshotSize(root) {
export function formatSnapshotLimit(problem, { paused = false } = {}) {
const size = `${(problem.sealedBytes / 1_000_000).toFixed(1)} MB`;
const limit = `${(problem.limit / 1_000_000).toFixed(1)} MB`;
- if (paused) return `relay paused for ${problem.board}: snapshot ${size} over the relay limit ${limit}; it resumes on its own once a snapshot fits`;
+ if (paused) return `relay paused for ${problem.board}: snapshot ${size} over the relay limit ${limit}; it resumes when the relay supports bounded snapshots`;
return `relay snapshot for ${problem.board}: sealed snapshot ${size} over the relay limit ${limit}`;
}
@@ -540,18 +540,15 @@ function expiredSignInMoveRefusal(state, error) {
return new Refused('AUTH_REQUIRED', `relay sign-in expired or was revoked for board ${state.board}: ${reason}; run pullboard relay on to give this machine a credential or pullboard relay off to stop syncing`);
}
-/** Pause only definitive baseline 4xx refusals; throttling and transient errors retain retries. */
-function permanentBaselineRefusal(error) {
- return Number.isInteger(error?.httpStatus) && error.httpStatus >= 400 && error.httpStatus < 500 &&
- ![408, 425, 429].includes(error.httpStatus);
-}
-
/** Explain a refused later checkpoint without changing the ordered-move recovery policy. */
function checkpointProblem(state) {
const refusal = state.checkpointRefusal;
const code = safePauseCode(refusal);
const reason = safePauseMessage(refusal);
const size = snapshotSize(state);
+ if (refusal.code === 'ACCOUNT_STORAGE_LIMIT') return { code: refusal.code,
+ message: `relay checkpoint paused for ${state.board}: ${relayStorageLine(state.storage)}; free relay storage or ask the operator to raise the quota; moves still sync in order`,
+ next: 'Free relay storage or ask the operator to raise the account quota; uploads resume automatically.' };
const next = 'upgrade pullboard (npm i -g pullboard) or run pullboard relay off then relay on --all';
const message = `relay checkpoint refused for ${state.board}: ${code} ${reason}`
+ (size ? '; ' + formatSnapshotLimit(size) : '')
@@ -571,6 +568,7 @@ function reportCheckpointRefusal(state, io) {
if (!boards) { boards = new Set(); CHECKPOINT_NOTICES.set(io, boards); }
if (boards.has(state.board)) return;
boards.add(state.board);
+ if (uploadPauseType(state.checkpointRefusal) && !firstUploadPauseNotice(state)) return;
const line = 'pullboard: ' + checkpointProblem(state).message;
if (['status', 'doctor'].includes(io.relayCommand?.cliOperation) && io.relayJson === false) io.say(line);
else io.err(line);
@@ -580,7 +578,7 @@ function reportCheckpointRefusal(state, io) {
function pauseFix(state) {
const pause = state.baselinePause;
if (pause.size || /(?:exceeds?|over|larger than|too large).{0,40}(?:bytes|snapshot|limit)|(?:snapshot|limit).{0,40}(?:exceeds?|over|too large)/iu.test(pause.message)) {
- return 'it resumes automatically once a snapshot fits';
+ return 'upgrade the configured relay to API version 2 for bounded snapshots, then it resumes automatically';
}
if ([401, 403].includes(pause.status) || pause.code === 'AUTH_REQUIRED') return 'run pullboard relay on to sign in again';
return 'upgrade Pullboard or run pullboard relay on again, then report the refusal if it persists';
@@ -599,7 +597,7 @@ function baselinePauseLine(state) {
function reportBaselinePause(state, io) {
if (!BASELINE_NOTICES.has(io)) {
BASELINE_NOTICES.add(io);
- io.err(baselinePauseLine(state));
+ if (firstUploadPauseNotice(state)) io.err(baselinePauseLine(state));
}
}
@@ -667,16 +665,18 @@ async function uploadInitialBaseline(root, file, state, io) {
state.baselineAccepted = true;
state.cursor = Number.isSafeInteger(state.baselineCursor) ? state.baselineCursor : localRecords(root, (board) => board.db.prepare('SELECT COALESCE(MAX(event_id), 0) AS id FROM event').get().id);
delete state.baselineCursor;
+ const resumed = Boolean(state.baselinePause);
delete state.baselinePause;
delete state.snapshot;
state.presentationDigest = state.snapshotPresentationDigest ?? state.presentationDigest;
delete state.snapshotPresentationDigest;
state.needsPresentation = false;
saveLink(file, state);
+ if (resumed) clearUploadPauseNotices(state);
return true;
} catch (error) {
if (error.relayVersion === 1 && error.code === 'BASELINE_EXISTS') fenceBaseline(file, state);
- if (!permanentBaselineRefusal(error) || state.baselineAccepted) {
+ if (!permanentUploadRefusal(error) || state.baselineAccepted) {
rememberBaselineFailure(io, file, error);
throw error;
}
@@ -693,19 +693,20 @@ async function uploadInitialBaseline(root, file, state, io) {
message: safePauseMessage(error),
...(size ? { size: { sealedBytes: size.sealedBytes, limit: size.limit } } : {}),
};
- delete state.snapshot;
- delete state.snapshotPresentationDigest;
- delete state.baselineCursor;
+ // Retain the refused ciphertext and its source cursor; blocked commands do not re-seal it.
saveLink(file, state);
reportBaselinePause(state, io);
return false;
}
}
-/** Rebuild from current local state once per command, so paused moves become the accepted baseline. */
+/** Probe blocked limits first; only a recovered relay receives a fresh baseline containing all paused local moves. */
async function retryPausedBaseline(root, file, state, io) {
if (!state.baselinePause || attemptedBaselineThisCommand(io, file)) return !state.baselinePause;
try {
+ if (!await uploadRetryReady(state, state.baselinePause, path => request(state, path, {}, io))) {
+ saveLink(file, state); reportBaselinePause(state, io); return false;
+ }
const document = relaySnapshot(root);
state.snapshot = { sequence: 0, sealed: await sealedRecord(readBoardKey(state.board), document, state, 'snapshot', 0) };
state.snapshotPresentationDigest = presentationDigest(document.presentation);
@@ -862,21 +863,30 @@ async function replayRelayRecord(board, key, state, record) {
/** Persist the exact checkpoint before uploading, so a lost reply retries identical ciphertext. */
async function publishCheckpoint(root, file, state, io) {
if (!state.token?.startsWith('pm_') || process.env.PULLBOARD_RELAY_TOKEN) return;
- const document = relaySnapshot(root);
- const sequence = Number(document.tables.board_meta.find((row) => row.meta_key === 'relay_applied_sequence')?.meta_value ?? 0);
- const digest = presentationDigest(document.presentation);
- if (!state.checkpoint || state.checkpoint.sequence !== sequence || state.checkpointPresentationDigest !== digest) {
+ const resumed = Boolean(state.checkpointRefusal);
+ if (state.checkpointRefusal && uploadPauseType(state.checkpointRefusal)) {
+ if (!await uploadRetryReady(state, state.checkpointRefusal, path => request(state, path, {}, io))) {
+ saveLink(file, state); reportCheckpointRefusal(state, io); return;
+ }
delete state.checkpointRefusal;
- state.checkpointPresentationDigest = digest;
- state.checkpoint = { sequence, sealed: await sealedRecord(readBoardKey(state.board), document, state, 'snapshot', sequence) };
saveLink(file, state);
+ } else {
+ const document = relaySnapshot(root);
+ const sequence = Number(document.tables.board_meta.find((row) => row.meta_key === 'relay_applied_sequence')?.meta_value ?? 0);
+ const digest = presentationDigest(document.presentation);
+ if (!state.checkpoint || state.checkpoint.sequence !== sequence || state.checkpointPresentationDigest !== digest) {
+ delete state.checkpointRefusal;
+ state.checkpointPresentationDigest = digest;
+ state.checkpoint = { sequence, sealed: await sealedRecord(readBoardKey(state.board), document, state, 'snapshot', sequence) };
+ saveLink(file, state);
+ }
+ // A definitive rejection retries only a fresh image after the board changes.
+ if (state.checkpointRefusal) { reportCheckpointRefusal(state, io); return; }
}
- // A definitive rejection retries only a fresh image after the board changes.
- if (state.checkpointRefusal) { reportCheckpointRefusal(state, io); return; }
let reply;
try { reply = await request(state, '/api/v1/boards/' + state.board + '/state', { method: 'PUT', body: state.checkpoint }, io); }
catch (error) {
- if (!(error instanceof Refused) || !permanentBaselineRefusal(error)) throw error;
+ if (!(error instanceof Refused) || !permanentUploadRefusal(error)) throw error;
state.checkpointRefusal = { code: safePauseCode(error), message: safePauseMessage(error), status: error.httpStatus };
saveLink(file, state);
reportCheckpointRefusal(state, io);
@@ -887,6 +897,7 @@ async function publishCheckpoint(root, file, state, io) {
delete state.checkpoint;
delete state.checkpointPresentationDigest;
saveLink(file, state);
+ if (resumed) clearUploadPauseNotices(state);
}
/** Drain a legacy source once, refusing foreign history before it can be silently checkpointed. */
@@ -1154,10 +1165,7 @@ export async function relayOperation(root, operation, args, io, command, applyLo
moves.add(state.board);
try {
if (!await ensureOrdered(root, file, state, io)) {
- // Preflight's notice covers the first move; long-running commands still name every later move.
- if (BASELINE_LOCAL_MOVES.has(io)) BASELINE_NOTICES.delete(io);
reportBaselinePause(state, io);
- BASELINE_LOCAL_MOVES.add(io);
return applyPausedMoveLocally();
}
await catchUp(root, file, state, io);
diff --git a/test/relay-client-fixture.js b/test/relay-client-fixture.js
index dee8a76b..f3f7da3c 100644
--- a/test/relay-client-fixture.js
+++ b/test/relay-client-fixture.js
@@ -112,7 +112,7 @@ function childResult(root, env, argv, observedSnapshotUploads, initialSnapshotUp
}
/** Start an isolated board and authorized loopback relay; a null coordinatorModel keeps its initial identity unknown. */
-export async function relayClientFixture(t, { cliChildren, coordinatorModel } = {}) {
+export async function relayClientFixture(t, { cliChildren, coordinatorModel, accountStorageBytes } = {}) {
const scratch = realpathSync(mkdtempSync(join(tmpdir(), 'pullboard-relay-cleanup-')));
const root = join(scratch, 'repo');
const home = join(scratch, 'home');
@@ -129,6 +129,7 @@ export async function relayClientFixture(t, { cliChildren, coordinatorModel } =
const env = {
...process.env, HOME: home, PULLBOARD_HOME: join(home, '.pullboard'),
PULLBOARD_MACHINE_HOME: join(scratch, 'machine'), PATH: privateBin,
+ TERM_SESSION_ID: createHash('sha256').update(scratch).digest('hex'),
GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_NOSYSTEM: '1',
GIT_AUTHOR_NAME: 'Relay Fixture', GIT_AUTHOR_EMAIL: '[email protected]',
GIT_COMMITTER_NAME: 'Relay Fixture', GIT_COMMITTER_EMAIL: '[email protected]',
@@ -435,7 +436,7 @@ globalThis.fetch = function tracedFetch(input, init) {
const origin = 'http://127.0.0.1:' + relayPort;
auth = createRelayAuth({ database: authDatabase, github: createGitHubClient({ ...provider.config, callbackURL: origin + '/auth/github/callback' }) });
signIn = createAuthHandler({ auth, publicOrigin: origin });
- api = createRelayHandler({ directory: join(scratch, 'relay'), auth, publicOrigin: origin, pollMs: 10, maintenanceMs: 0, now: () => Date.now() + timeOffsetMs });
+ api = createRelayHandler({ directory: join(scratch, 'relay'), auth, publicOrigin: origin, pollMs: 10, maintenanceMs: 0, accountStorageBytes, now: () => Date.now() + timeOffsetMs });
/** Sign a separate phone in over actual OAuth HTTP; its person session stays only in fixture RAM. */
async function phoneSession() {
if (phone) return phone;
@@ -607,8 +608,10 @@ globalThis.fetch = function tracedFetch(input, init) {
/** Start a saved real relay link without exposing the pairing fragment. */
async function link() {
- assert.equal((await cli('relay', 'on', '--url', origin)).code, 0, 'real device sign-in links the board');
+ const result = await cli('relay', 'on', '--url', origin);
+ assert.equal(result.code, 0, 'real device sign-in links the board');
privateKeys.add(readFileSync(keyFile, 'utf8').trim());
+ return result;
}
/** Raise this fixture board's durable minimum to the current engine before compatibility checks. */
async function requireEngineThree() {
@@ -702,6 +705,13 @@ globalThis.fetch = function tracedFetch(input, init) {
overrideDelete(value) { override = value; },
failTokenMints(count) { mintFailures = count; },
refuseSnapshotWrites(value) { refuseSnapshotWrites = value; },
+ /** Restart only the private service with a changed operator quota, retaining real journals and credentials. */
+ setStorageQuota(bytes) {
+ assert.ok(Number.isSafeInteger(bytes) && bytes > 0);
+ api.close();
+ api = createRelayHandler({ directory: join(scratch, 'relay'), auth, publicOrigin: origin, pollMs: 10,
+ maintenanceMs: 0, accountStorageBytes: bytes, now: () => Date.now() + timeOffsetMs });
+ },
/** Return a chosen HTTP refusal for snapshot writes, without inspecting the sealed body. */
rejectSnapshots(value) { rejectSnapshotWrites = value ? { ...value } : null; },
/** Model an older relay's capability response without changing the real API1 request and journal bounds. */
diff --git a/test/relay-pages-http.test.js b/test/relay-pages-http.test.js
index 2f1c373c..a192e169 100644
--- a/test/relay-pages-http.test.js
+++ b/test/relay-pages-http.test.js
@@ -31,10 +31,11 @@ async function fixture(t, accountStorageBytes) {
await auth.linkBoard(signed.token, BOARD, 'fixture/repository');
await auth.linkBoard(signed.token, OTHER, 'fixture/repository');
let handler;
+ let clock = Date.now();
const server = createServer((req, res) => handler(req, res));
server.listen(0, '127.0.0.1'); await once(server, 'listening');
const origin = 'http://127.0.0.1:' + server.address().port;
- handler = createRelayHandler({ directory: join(root, 'boards'), auth, publicOrigin: origin, maintenanceMs: 0, accountStorageBytes });
+ handler = createRelayHandler({ directory: join(root, 'boards'), auth, publicOrigin: origin, maintenanceMs: 0, accountStorageBytes, now: () => clock });
t.after(() => { server.closeAllConnections(); server.close(); handler.close(); auth.close(); rmSync(root, { recursive: true, force: true }); });
const key = await generateBoardKey();
/** Return only protocol responses, never bearer credentials, in assertion diagnostics. */
@@ -80,7 +81,9 @@ async function fixture(t, accountStorageBytes) {
const directory = await readSealedDescriptor(state.root, context);
return readSealedValue(directory.value.value, { ...context, parents: [state.root.id] });
}
- return { root, auth, signed, origin, key, call, stage, publish, read, handler };
+ return { root, auth, signed, origin, key, call, stage, publish, read, handler,
+ /** Advance the private service clock to observe real staging expiry, without mutating its database. */
+ advance(ms) { clock += ms; } };
}
test('API2 reads legacy heads and publishes opaque pages while old clients explicitly upgrade [H15,H17]', async t => {
@@ -172,3 +175,29 @@ test('a consistent journal backup carries every opaque page and unlink removes t
assert.equal((await box.call('DELETE', '/api/v1/boards/' + BOARD)).status, 200);
assert.equal((await box.call('GET', '/api/v2/boards/' + BOARD + '/state')).status, 404);
});
+
+
+test('small authenticated storage probes collect expired staging across the account without downloading a snapshot [H15,H17,H18]', async t => {
+ const box = await fixture(t, 12_000);
+ const empty = await box.call('GET', '/api/v2/boards/' + BOARD + '/storage');
+ assert.equal(empty.status, 200); assert.equal(empty.document.storage.usedBytes, 0);
+ const first = await box.stage({ marker: 'retained current head' });
+ assert.equal((await box.publish(first)).status, 200);
+ await box.stage({ text: randomBytes(8_000).toString('base64') });
+ await assert.rejects(box.stage({ text: randomBytes(8_000).toString('base64') }), { code: 'ACCOUNT_STORAGE_LIMIT' });
+ const blocked = await box.call('GET', '/api/v2/boards/' + OTHER + '/storage');
+ assert.ok(blocked.document.storage.usedBytes > 8_000);
+ const own = await box.call('GET', '/api/v2/boards/' + BOARD + '/storage');
+ assert.equal(own.document.warning.code, 'ACCOUNT_STORAGE_LIMIT');
+ box.advance(3_600_001);
+ const collected = await box.call('GET', '/api/v2/boards/' + OTHER + '/storage');
+ assert.ok(collected.document.storage.usedBytes < 2_000, 'another linked board can release expired staging charged to the same account');
+ const ready = await box.call('GET', '/api/v2/boards/' + BOARD + '/storage');
+ assert.equal(ready.document.warning, undefined);
+ assert.ok(JSON.stringify(ready.document).length < 2048);
+ assert.deepEqual(Object.keys(ready.document).sort(), ['apiVersion', 'storage', 'version']);
+ const head = await box.call('GET', '/api/v2/boards/' + BOARD + '/state');
+ assert.equal((await box.read(head.document.state)).marker, 'retained current head');
+ const invalid = await box.call('GET', '/api/v2/boards/' + BOARD + '/storage', undefined, 'pa_' + 'x'.repeat(43));
+ assert.equal(invalid.status, 401);
+});
diff --git a/test/relay-snapshot-size.test.js b/test/relay-snapshot-size.test.js
index 8c11ba1b..c73193cc 100644
--- a/test/relay-snapshot-size.test.js
+++ b/test/relay-snapshot-size.test.js
@@ -312,22 +312,22 @@ test('thirty thousand events open on the phone within three seconds and stream a
});
await chrome.navigate(box.origin + '/#board=' + link.board + '&key=' + readFileSync(box.keyFile, 'utf8').trim());
// Budget: the owner's explicit first-readable-board bar is three seconds, independent of historical streaming.
- await chrome.waitFor('Number.isFinite(window.__phoneTiming?.firstBoardPaint) && typeof transport !== "undefined" && !!transport?.historyProgress', 3000, 'bounded current board paints before all history');
+ await chrome.waitFor('Number.isFinite(window.__phoneTiming?.firstBoardPaint) && typeof transport !== "undefined" && !!transport?.historyProgress', 3000, 'bounded current board paints before all history'); // budget: declared frozen #903 first-readable-board ceiling is 3000ms; it cannot be relaxed.
const first = await chrome.evaluate(`({ ...__phoneTiming, phone: document.documentElement.dataset.relayDevice, progress: transport.historyProgress('${link.board}') })`);
assert.equal(first.phone, 'phone'); assert.ok(first.firstBoardPaint < 3000);
assert.equal(first.loopbackReads, 0); assert.equal(first.progress.complete, false);
// Budget: this fixture streams thirty thousand historical records before reaching its deliberately held oldest event leaf.
- await chrome.waitFor('window.__phoneTiming.historyHeld', 30_000, 'old history is streaming after the first readable board');
+ await chrome.waitFor('window.__phoneTiming.historyHeld', 30_000, 'old history is streaming after the first readable board'); // budget: hang ceiling for the deliberately held oldest-event-page handshake, independent of the 3s boot bar.
try {
assert.equal((await box.cli('add', box.lane, liveMarker)).code, 0, 'ordinary native moves continue while old history is pinned');
- await chrome.waitFor(`document.querySelector('#chain')?.textContent.includes('${liveMarker}')`, 15_000, 'new current state arrives while an older history page is held');
+ await chrome.waitFor(`document.querySelector('#chain')?.textContent.includes('${liveMarker}')`, 15_000, 'new current state arrives while an older history page is held'); // budget: hang ceiling for the live-move handshake while old history is held, not a throughput claim.
} finally {
await chrome.evaluate('window.__olderHistoryAllowed = true; window.__releaseOlderHistory?.(); true');
}
const expected = relaySnapshot(box.root);
const finalSequence = Number(expected.tables.board_meta.find(row => row.meta_key === 'relay_applied_sequence').meta_value);
// Budget: a complete loopback replay includes both the pinned predecessor and its live successor, never the three-second boot bar.
- await chrome.waitFor(`transport.historyProgress('${link.board}').complete && transport.historyProgress('${link.board}').sequence === ${finalSequence}`, 45_000, 'all sealed history is held at the live checkpoint');
+ await chrome.waitFor(`transport.historyProgress('${link.board}').complete && transport.historyProgress('${link.board}').sequence === ${finalSequence}`, 45_000, 'all sealed history is held at the live checkpoint'); // budget: hang ceiling for two complete pinned background streams; the 3s boot bar is measured separately.
const oldestHash = createHash('sha256').update(oldestMarker).digest('hex');
const read = await chrome.evaluate(`(async () => {
const before = __phoneTiming.objectRequests;
@@ -339,7 +339,7 @@ test('thirty thousand events open on the phone within three seconds and stream a
assert.equal(read.hash, oldestHash); assert.equal(read.extraObjectReads, 0, 'the oldest shout is already held after the background stream');
assert.equal(read.progress.loaded, read.progress.total); assert.equal(read.loopbackReads, 0);
await chrome.evaluate(`location.hash = '#shout-${oldest}'; true`);
- await chrome.waitFor(`document.body.innerText.includes('${oldestMarker}')`, 3000, 'the actual phone view opens the streamed oldest shout');
+ await chrome.waitFor(`document.body.innerText.includes('${oldestMarker}')`, 3000, 'the actual phone view opens the streamed oldest shout'); // budget: hang ceiling for the addressed-shout render handshake after history completion.
assert.equal((await second.cli('status')).code, 0, 'an independent paired CLI restores the same paged checkpoint');
const restored = await second.cli('export');
assert.equal(restored.code, 0);
diff --git a/test/relay-stuck-upload.test.js b/test/relay-stuck-upload.test.js
index c935ffd4..d24d7550 100644
--- a/test/relay-stuck-upload.test.js
+++ b/test/relay-stuck-upload.test.js
@@ -26,8 +26,9 @@ test('a refused upload never blocks board moves [H16,B26]', async (t) => {
'--check', gate);
assert.equal(added.code, 0, `the fixture creates its task; result=${JSON.stringify(added.document)}`);
const item = added.document.item.item_id;
- box.rejectSnapshots({ status: 400, code: 'BAD_REQUEST', message: 'the body exceeds 14000000 bytes; send a smaller snapshot' });
- await box.link();
+ box.rejectSnapshots({ status: 400, code: 'BAD_REQUEST', message: 'fixture refuses the initial baseline until repaired' });
+ const linked = await box.link();
+ assert.equal((linked.document.diagnostics ?? []).filter(line => /relay paused/u.test(line)).length, 1);
const status = await box.cli('status');
assert.equal(status.code, 0);
assertPaused(status.document);
@@ -37,7 +38,7 @@ test('a refused upload never blocks board moves [H16,B26]', async (t) => {
assert.equal(pausedLink.baselinePause.status, 400);
assert.ok(pausedLink.baselinePause.message.length <= 240);
assert.doesNotMatch(JSON.stringify(pausedLink.baselinePause), /(?:ps|pa)_[A-Za-z0-9_-]{20,}/u);
- assert.equal(pausedLink.snapshot, undefined, 'refused ciphertext is discarded so recovery seals current local state');
+ assert.ok(pausedLink.snapshot, 'refused ciphertext remains durable until a recovered baseline includes all paused local work');
const diagnosed = await box.cli('doctor');
assert.ok([0, 1].includes(diagnosed.code));
assertPaused(diagnosed.document);
@@ -97,9 +98,9 @@ test('a refused upload never blocks board moves [H16,B26]', async (t) => {
/** Each internal move in a long-running command retains its own loud pause notice. */
- await t.test('multiple local moves in one command each name the refused baseline [H16,B26]', async subtest => {
+ await t.test('multiple local moves in one session do not repeat the refused baseline notice [H16,B26]', async subtest => {
const repeated = await relayClientFixture(subtest);
- repeated.rejectSnapshots({ status: 400, code: 'BAD_REQUEST', message: 'the body exceeds 14000000 bytes; send a smaller snapshot' });
+ repeated.rejectSnapshots({ status: 400, code: 'BAD_REQUEST', message: 'fixture refuses the initial baseline until repaired' });
await repeated.link();
const before = repeated.calls.filter(call => call.method === 'PUT' && call.path.endsWith('/state')).length;
const result = await repeated.script(`
@@ -115,8 +116,8 @@ test('a refused upload never blocks board moves [H16,B26]', async (t) => {
} catch (error) { process.stdout.write(JSON.stringify({ error: { code: error.code, message: error.message } })); process.exitCode = 1; }
`);
assert.equal(result.code, 0);
- assert.equal(result.document.diagnostics.filter(line => /relay paused for /u.test(line)).length, 2,
- 'each move gets one notice, with no duplicate preflight or postflight notice');
+ assert.equal(result.document.diagnostics.filter(line => /relay paused for /u.test(line)).length, 0,
+ 'the session was already notified when linking; internal moves do not repeat it');
assert.equal(repeated.calls.filter(call => call.method === 'PUT' && call.path.endsWith('/state')).length - before, 1,
'one command attempts only one fresh baseline despite multiple local moves');
assert.equal(repeated.moveAcks.length, 0, 'unaccepted local moves never become sequence-one uploads');
@@ -197,10 +198,11 @@ test('a refused upload never blocks board moves [H16,B26]', async (t) => {
await t.test('malformed baseline refusals carry their reason and recover [H16]', async (subtest) => {
const malformed = await relayClientFixture(subtest);
malformed.rejectSnapshots({ status: 400, code: 'BAD_UPLOAD', message: 'send only a sequence and sealed payload' });
- await malformed.link();
+ const linked = await malformed.link();
const paused = await malformed.cli('status');
assert.equal(paused.code, 0);
- const notice = (paused.document.diagnostics ?? []).join('\n');
+ assert.equal((paused.document.diagnostics ?? []).filter(line => /relay paused/u.test(line)).length, 0, 'the link already delivered this session notice');
+ const notice = (linked.document.diagnostics ?? []).join('\n');
assert.match(notice, /BAD_UPLOAD/u);
assert.match(notice, /send only a sequence and sealed payload/u);
assert.match(notice, /upgrade pullboard|pullboard relay on/u, 'the malformed-upload notice gives its recovery step');
@@ -377,13 +379,10 @@ test('a refused upload never blocks board moves [H16,B26]', async (t) => {
});
});
-/** Require the one-line durable baseline pause notice on a local move. */
+/** The session's link already named its pause; later local commands keep working without repeating it. */
function assertPaused(document) {
- const notices = (document.diagnostics ?? []).filter(line => /relay paused/u.test(line));
- assert.equal(notices.length, 1, `each move names the paused relay upload and recovery path in one line; diagnostics=${JSON.stringify(document.diagnostics ?? [])}`);
- assert.match(notices[0], /BAD_REQUEST/u, 'the notice keeps the relay refusal code');
- assert.match(notices[0], /body exceeds 14000000 bytes/u, 'the notice keeps the relay refusal reason');
- assert.match(notices[0], /snapshot.*fits/u, 'the notice says how automatic recovery proceeds');
+ assert.equal((document.diagnostics ?? []).filter(line => /relay paused/u.test(line)).length, 0,
+ 'one session does not repeat a durable baseline pause on each command');
}
/** Authenticate the actual relay checkpoint, accepting both supported plaintext snapshot formats. */
diff --git a/test/relay-upload-pause.test.js b/test/relay-upload-pause.test.js
new file mode 100644
index 00000000..c4353420
--- /dev/null
+++ b/test/relay-upload-pause.test.js
@@ -0,0 +1,106 @@
+/** Paused size and quota limits do not re-seal history or repeat notices across commands [H15,H17]. */
+import assert from 'node:assert/strict';
+import { randomBytes } from 'node:crypto';
+import { mkdirSync, readFileSync, writeFileSync } from 'node:fs';
+import { join } from 'node:path';
+import { test } from 'node:test';
+import * as store from '../src/board.js';
+import { relayClientFixture } from './relay-client-fixture.js';
+import { findChromeExecutable, startChrome } from './relay-browser-fixture.js';
+
+/** Read private pending metadata without including ciphertext or credentials in an assertion diagnostic. */
+function state(box) { return JSON.parse(readFileSync(box.linkFile, 'utf8')); }
+
+/** A real cryptographic counter distinguishes no re-seal from merely discarding a freshly generated snapshot. */
+async function pausedCrypto(box) {
+ return box.script(`
+ const original = crypto.subtle.encrypt.bind(crypto.subtle);
+ let encryptions = 0;
+ crypto.subtle.encrypt = (...args) => { encryptions++; return original(...args); };
+ const { syncRelay } = await import(${JSON.stringify(new URL('../src/relay.js', import.meta.url).href)});
+ const lines = [], io = { err(line) { lines.push(line); } };
+ await syncRelay(process.cwd(), io);
+ await syncRelay(process.cwd(), io);
+ console.log(JSON.stringify({ encryptions, notices: lines.filter(line => /relay (?:checkpoint )?paused/u.test(line)).length }));
+ `);
+}
+
+test('a size-paused baseline keeps its cipher, warns once per session and resumes with all local moves after API2 [H15,H17]', async t => {
+ const box = await relayClientFixture(t);
+ box.legacySnapshotApi();
+ box.rejectSnapshots({ status: 400, code: 'BAD_REQUEST', message: 'the body exceeds 14000000 bytes; send a smaller snapshot' });
+ const linked = await box.link();
+ assert.equal((linked.document.diagnostics ?? []).filter(line => /relay paused/u.test(line)).length, 1);
+ const saved = state(box).snapshot.sealed;
+ const puts = () => box.calls.filter(call => call.method === 'PUT' && call.path.endsWith('/state')).length;
+ const before = puts();
+ const probe = await pausedCrypto(box);
+ assert.equal(probe.code, 0); assert.equal(probe.document.encryptions, 0); assert.equal(probe.document.notices, 0);
+ assert.equal((await box.cli('shout', 'all', 'local work while size is paused')).code, 0);
+ assert.equal((await box.cli('status')).code, 0);
+ assert.equal(state(box).snapshot.sealed === saved, true, 'blocked commands retain exact durable pending ciphertext');
+ assert.equal(puts(), before, 'small capabilities probes replace failed whole-board uploads');
+ const originalSession = box.env.TERM_SESSION_ID;
+ box.env.TERM_SESSION_ID = 'another-private-terminal-session';
+ const newSession = await box.cli('status');
+ assert.equal((newSession.document.diagnostics ?? []).filter(line => /relay paused/u.test(line)).length, 1);
+ box.env.TERM_SESSION_ID = originalSession;
+ box.legacySnapshotApi(false); box.rejectSnapshots(null);
+ assert.equal((await box.cli('status')).code, 0);
+ const resumed = state(box);
+ assert.equal(resumed.baselinePause, undefined); assert.equal(resumed.snapshot, undefined); assert.equal(resumed.snapshotApiVersion, 2);
+ const exported = await box.cli('export');
+ assert.equal(exported.code, 0);
+ assert.ok(exported.document.tables.shout.some(row => row.shout_text === 'local work while size is paused'));
+ assert.equal(box.moveAcks.length, 0, 'all paused local work belongs to the fresh accepted baseline');
+ assert.equal((await box.cli('shout', 'all', 'first ordered move after upgrade')).code, 0);
+ assert.deepEqual(box.moveAcks.map(row => row.event_id), [1]);
+});
+
+test('quota-paused checkpoints keep exact pending bytes, permit ordered moves and resume after an operator quota change [H15,H17]', async t => {
+ const box = await relayClientFixture(t, { accountStorageBytes: 256_000 });
+ await box.link();
+ const board = store.openBoard(join(box.root, '.git/pullboard/board.sqlite'));
+ try { store.shout(board, { from: 'coordinator', to: 'all', lanes: [box.lane], text: randomBytes(300_000).toString('base64') }); }
+ finally { store.closeBoard(board); }
+ const refused = await box.cli('status');
+ assert.equal(refused.code, 0);
+ const saved = state(box);
+ assert.equal(saved.checkpointRefusal.code, 'ACCOUNT_STORAGE_LIMIT');
+ assert.equal(refused.document.relay.storage.limitBytes, 256_000);
+ await t.test('the actual phone names the exhausted account quota before any newer checkpoint [H15,H17]', {
+ skip: !findChromeExecutable() && 'Chrome is not installed',
+ }, async phoneTest => {
+ const chrome = await startChrome(); phoneTest.after(() => chrome.close());
+ await chrome.send('Emulation.setDeviceMetricsOverride', { width: 390, height: 844, deviceScaleFactor: 1, mobile: true });
+ await chrome.send('Emulation.setTouchEmulationEnabled', { enabled: true, maxTouchPoints: 5 });
+ await chrome.send('Network.setCookie', { name: 'pb_session', value: (await box.phoneSession()).token, url: box.origin, httpOnly: true, sameSite: 'Lax' });
+ await chrome.navigate(box.origin + '/#board=' + saved.board + '&key=' + readFileSync(box.keyFile, 'utf8').trim());
+ await chrome.waitFor("document.querySelector('#relay-notice')?.textContent.includes('relay storage limit')", 3000, 'the account quota is visible on the phone'); // budget: hang ceiling for the account-notice render handshake, not a throughput claim.
+ const notice = await chrome.evaluate("document.querySelector('#relay-notice').textContent");
+ assert.ok(notice.includes(saved.storage.account.login));
+ assert.match(notice, /0\.3 MB relay storage limit/u);
+ if (process.env.PULLBOARD_RELAY_PAGE_PROOF) {
+ mkdirSync(process.env.PULLBOARD_RELAY_PAGE_PROOF, { recursive: true, mode: 0o700 });
+ const screenshot = await chrome.send('Page.captureScreenshot', { format: 'png' });
+ writeFileSync(join(process.env.PULLBOARD_RELAY_PAGE_PROOF, 'quota-phone.png'), Buffer.from(screenshot.data, 'base64'), { mode: 0o600 });
+ }
+ });
+ const probe = await pausedCrypto(box);
+ assert.equal(probe.code, 0); assert.equal(probe.document.encryptions, 0); assert.equal(probe.document.notices, 0);
+ assert.equal((await box.cli('shout', 'all', 'ordered move while checkpoint quota is paused')).code, 0);
+ assert.deepEqual(box.moveAcks.map(row => row.event_id), [1]);
+ assert.equal(state(box).checkpoint.sealed === saved.checkpoint.sealed, true);
+ const priorWrites = box.transit.length;
+ box.setStorageQuota(3_000_000);
+ assert.equal((await box.cli('status')).code, 0);
+ const attempts = box.transit.slice(priorWrites).filter(row => row.method === 'PUT' && row.path.endsWith('/state'));
+ assert.ok(attempts.length >= 1);
+ assert.equal(JSON.parse(attempts[0].request).sealed === saved.checkpoint.sealed, true, 'recovery retries the retained checkpoint before sealing newer state');
+ assert.equal((await box.cli('status')).code, 0);
+ assert.equal(state(box).checkpointRefusal, undefined); assert.equal(state(box).checkpoint, undefined);
+ assert.equal(state(box).storage.limitBytes, 3_000_000);
+ const exported = await box.cli('export');
+ assert.equal(exported.code, 0);
+ assert.equal(exported.document.tables.shout.filter(row => row.shout_text === 'ordered move while checkpoint quota is paused').length, 1);
+});