Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

test(convergence): a synced recovery run that reports commits landed behind it is followed by the second run (busy-agent (l) at 50 ms)

garrytan1e2bbe3e7083
diff --git a/scripts/persistence/README.md b/scripts/persistence/README.md
index c6643abb39..2cc4294a5c 100644
--- a/scripts/persistence/README.md
+++ b/scripts/persistence/README.md
@@ -579,7 +579,9 @@ anyway, which is how the PR that fixes it turns it on.
After the chaos every scenario gets a fresh serve (what a supervisor restarts)
and at most two plain `gbrain sync --no-pull` runs, with no operator step; only
-exit 0 with outcome `synced` counts. Then the oracle, each check its own test
+exit 0 with outcome `synced` counts, and a synced run whose
+receipt reports commits landed behind it (`watermark.pending.commits_behind`, a
+queued write's Git commit) is followed by the second. Then the oracle, each check its own test
named by this gate's violation class: disk == database both ways (except the
pages the mode's end state names), no request queued, running, recovering or
holding a recovery record, every non-committed terminal request superseded or
diff --git a/test/helpers/convergence-brain.ts b/test/helpers/convergence-brain.ts
index e0f22c0dd2..b169becffa 100644
--- a/test/helpers/convergence-brain.ts
+++ b/test/helpers/convergence-brain.ts
@@ -334,7 +334,7 @@ export class Brain {
this.note(`phase 0: fault-free import in ${(this.elapsed.baseline_import_ms / 1000).toFixed(1)}s; oracle calibration problems: ${this.calibration.length}`);
}
- /** A fresh resident serve and at most MAX_RECOVERY_RUNS plain syncs, no operator step. */
+ /** A fresh resident serve and at most MAX_RECOVERY_RUNS plain syncs, no operator step; a synced run that reports commits landed behind it (a queued write's Git commit) is followed by the next. */
async recover(): Promise<void> {
const started = performance.now();
const serve = new StdioServe(this);
@@ -344,8 +344,9 @@ export class Brain {
const envelope = lastJson(r.out);
this.recoveryRuns.push({ ...r, envelope });
this.keepLog(`recovery-${run}-client.log`, `${r.out}\n${r.err}`);
- this.note(`recovery run ${run}: exit ${r.code} in ${(r.ms / 1000).toFixed(1)}s outcome=${String(envelope?.outcome ?? envelope?.sync_status ?? envelope?.error ?? 'none')}`);
- if (r.code === 0 && envelope?.outcome === 'synced') break;
+ const behind = ((envelope?.watermark as { pending?: { commits_behind?: { count?: number } } } | undefined)?.pending?.commits_behind?.count) ?? 0;
+ this.note(`recovery run ${run}: exit ${r.code} in ${(r.ms / 1000).toFixed(1)}s outcome=${String(envelope?.outcome ?? envelope?.sync_status ?? envelope?.error ?? 'none')}${behind ? `, ${behind} commit(s) landed behind it` : ''}`);
+ if (r.code === 0 && envelope?.outcome === 'synced' && behind === 0) break;
}
this.elapsed.recovery_ms = performance.now() - started;
this.keepLog('recovery-serve.log', serve.stderr);