Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Stop the stale link sweep re-extracting a page forever on a bare-name link that matches a top-level page
garrytan0b7b73a5d061
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 01ff2049e3..fc382d821c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -12,7 +12,7 @@ identifiers and attribution are available in the pre-removal Git revision## [Unreleased]-**Doctor, dream, enrich and `maintain --safe` fixes, landed together.** `gbrain maintain --safe` no longer deletes pages or calls a paid provider, several doctor checks now report what they find accurately, dream phases stop and report correctly, and enrich, model-call errors, atom provenance and transcript dates get community-reported fixes. Each fix has a test that fails without it.+**Doctor, dream, enrich and `maintain --safe` fixes, landed together.** `gbrain maintain --safe` no longer deletes pages or calls a paid provider, several doctor checks now report what they find accurately, dream phases stop and report correctly, and enrich, model-call errors, atom provenance, transcript dates and the stale link sweep get community-reported fixes. Each fix has a test that fails without it.**The `unindexed_files` maintenance phase no longer reports a complete scan when a source could not be scanned.** A source whose Git checkout failed to read was counted as an error and the phase warned, but its `details.complete` still said `true`, so anything reading the phase result took the run as a full scan of every managed source. `complete` is now `true` only when no source stopped at its time budget and none errored. Reported by @Masashi-Ono0611 (#8887).@@ -40,6 +40,8 @@ identifiers and attribution are available in the pre-removal Git revision**The admin dashboard's live feed keeps reconnecting after a dropped connection,** and four smaller fixes land with it: the load throttle still applies its concurrency and memory limits when the machine reports no load data (Windows), a `remember` TTL past the representable date range is refused as `invalid_params` instead of throwing, a worker pool whose error callback throws now stops every worker before rejecting, and claude-cli tool loops read the previous turn's prompt cache instead of rewriting the whole transcript to it.+**The stale link sweep no longer re-extracts one page forever when a bare-name frontmatter link matches a top-level page.** A person page with `founded: ["acme-toolkit"]` resolves that name under `companies/`, so a top-level page named `acme-toolkit` does not satisfy it and the link stays wanted. The wanted-link row was still stamped as if its target existed, which marks its page stale on every pass, so the sweep queued the same page again and again (thousands of extract jobs an hour) and starved lower-priority jobs. Only a slug reference to an existing page is stamped that way now, and a row an earlier release stamped settles the next time its page is extracted, which the sweep does on its own. Reported and first fixed by @akinduroifedayo (#9140).+### Itemized changes- `src/core/cycle/unindexed-files.ts`: `details.complete` also requires `errors.length === 0`. Test: `test/unindexed-files.test.ts` (a source whose `.git/HEAD` is unreadable: phase `warn`, one error, `complete: false`).@@ -69,6 +71,7 @@ identifiers and attribution are available in the pre-removal Git revision- `src/core/cycle/atom-source-date.ts`: new `transcriptAtomSourceDate` (frontmatter date, else the path date that names the `atoms/<date>/` folder); an undated transcript still gets none. Test: `test/extract-atoms-source-date.test.ts`. Reported by @spiky02plateau (#7268).- Admin dashboard: the EventSource `onerror` only sets the status from `readyState` and no longer closes the stream; `admin/dist` and `src/admin-embedded.ts` are rebuilt. Test: `test/admin-dashboard-stream-reconnect.test.ts`. Reported by @ramybarsoum (#8913).- Load throttle (`shouldProceed`), `remember` TTL parsing and the worker pool: see the paragraph above. Tests: `test/backoff.test.ts`, `test/writeback-config.test.ts`, `test/worker-pool.test.ts`. Reported by @ramybarsoum (#8913).+- `src/core/wanted-links-store.ts` (`replaceWantedLinks`): `checked_at` is `-infinity` only for a `slug` row whose exact page exists; a `name` row gets `now()`, and the existing `ON CONFLICT` update re-stamps old rows on their origin's next extraction. Test: `test/wanted-links.test.ts` and `test/e2e/wanted-links-postgres.test.ts` (`topLevelSlugNameReferenceSettles`: the origin is not stale after one sweep, and a row forced to `-infinity` settles after one more). Reported by @akinduroifedayo (#9140).## [0.60.659.0] - 2026-11-11diff --git a/src/core/wanted-links-store.ts b/src/core/wanted-links-store.tsindex 78855c1f9d..cb95e14660 100644--- a/src/core/wanted-links-store.ts+++ b/src/core/wanted-links-store.ts@@ -54,9 +54,10 @@ function pageSlugShaped(target: string): boolean {* one that replaces its derived links), keeping only possibleWantedRows. Slug targets are locked first, so a* concurrent writer creating the target either committed before this check* (and the row is stamped `-infinity`, leaving the origin stale) or commits- * after it with an `updated_at` past `checked_at`. Only an exact slug counts- * here: a bare-name reference matching some page's basename may legitimately- * stay unresolved (basename resolution is opt-in), and stamping it+ * after it with an `updated_at` past `checked_at`. Only a slug reference to an+ * exact page counts here: a bare-name reference matching some page's slug or+ * basename may legitimately stay unresolved (basename resolution is opt-in,+ * and a directory hint can skip a top-level slug), and stamping it* `-infinity` would re-list its origin forever.*/export async function replaceWantedLinks(@@ -83,8 +84,8 @@ export async function replaceWantedLinks(const written = await executeRawJsonb<{ id: number }>(tx, `INSERT INTO wanted_links(origin_page_id, source_id, producer, ref_kind, target_source_id, target_ref, link_type, context, checked_at)SELECT $1, $2, v.producer, v.ref_kind, v.target_source_id, v.target_ref, v.link_type, v.context,- CASE WHEN EXISTS (SELECT 1 FROM pages t WHERE t.source_id = v.target_source_id AND t.deleted_at IS NULL- AND t.slug = v.target_ref)+ CASE WHEN v.ref_kind = 'slug' AND EXISTS (SELECT 1 FROM pages t WHERE t.source_id = v.target_source_id+ AND t.deleted_at IS NULL AND t.slug = v.target_ref)THEN '-infinity'::timestamptz ELSE now() ENDFROM jsonb_to_recordset(($3::jsonb)->'rows')AS v(producer text, ref_kind text, target_source_id text, target_ref text, link_type text, context text)diff --git a/test/e2e/wanted-links-postgres.test.ts b/test/e2e/wanted-links-postgres.test.tsindex 246e52f045..a0641bd191 100644--- a/test/e2e/wanted-links-postgres.test.ts+++ b/test/e2e/wanted-links-postgres.test.ts@@ -7,13 +7,14 @@ import { PostgresEngine } from '../../src/core/postgres-engine.ts';import { replaceWantedLinks } from '../../src/core/wanted-links.ts';import { bareNameReferenceSettles, disabledClearsRows, forwardReferenceHeals, impossibleTargetsNeverAbortTheSweep,impossibleTargetsStayOutOfWrites, onlyUnresolvedAuthoredReferences, privateOriginsStayPrivate,- restoredTargetHeals } from '../helpers/wanted-links-scenarios.ts';+ restoredTargetHeals, topLevelSlugNameReferenceSettles } from '../helpers/wanted-links-scenarios.ts';const url = process.env.DATABASE_URL;describe.skipIf(!url)('Postgres wanted pages', () => {test('a forward reference heals once its target exists', () => forwardReferenceHeals(url), 180_000);test('only unresolved authored references are wanted', () => onlyUnresolvedAuthoredReferences(url), 180_000);test('a bare-name reference settles', () => bareNameReferenceSettles(url), 180_000);+ test('#9140: a bare-name reference equal to a top-level slug settles', () => topLevelSlugNameReferenceSettles(url), 180_000);test('private origins stay private', () => privateOriginsStayPrivate(url), 180_000);test('restoring a deleted target heals', () => restoredTargetHeals(url), 180_000);test('disabling clears rows', () => disabledClearsRows(url), 180_000);diff --git a/test/helpers/wanted-links-scenarios.ts b/test/helpers/wanted-links-scenarios.tsindex 70521e58b1..5c0c65230e 100644--- a/test/helpers/wanted-links-scenarios.ts+++ b/test/helpers/wanted-links-scenarios.ts@@ -85,6 +85,28 @@ export async function bareNameReferenceSettles(databaseUrl?: string) {}, { databaseUrl });}+/**+ * #9140: a bare-name frontmatter reference resolved under a directory hint stays unresolved even when a+ * top-level page has the exact slug. It settles after one extraction, and a row an older build stamped+ * `-infinity` settles on its origin's next extraction.+ */+export async function topLevelSlugNameReferenceSettles(databaseUrl?: string) {+ await managedBrain(async ({ engine, ctx }) => {+ await put(ctx, 'acme-toolkit', 'Acme toolkit.');+ await put(ctx, 'people/alice-example', 'Alice.', 'person', 'founded: ["acme-toolkit"]\n');+ await extractStale(engine);+ expect(await wantedRows(engine)).toEqual([{ target_ref: 'acme-toolkit', ref_kind: 'name', producer: 'frontmatter' }]);+ expect(await stale(engine)).toBe(0);+ await engine.executeRaw("UPDATE wanted_links SET checked_at = '-infinity'::timestamptz WHERE target_ref = 'acme-toolkit'");+ expect(await stale(engine)).toBe(1);+ await extractStale(engine);+ expect(await stale(engine)).toBe(0);+ const [row] = await engine.executeRaw<{ settled: boolean }>(+ "SELECT checked_at > '-infinity'::timestamptz AS settled FROM wanted_links WHERE target_ref = 'acme-toolkit'");+ expect(row?.settled).toBe(true);+ }, { databaseUrl });+}+/** A remote caller never sees targets referenced only from private pages, nor counts that include them. */export async function privateOriginsStayPrivate(databaseUrl?: string) {await managedBrain(async ({ engine, ctx }) => {diff --git a/test/wanted-links.test.ts b/test/wanted-links.test.tsindex 5080e97038..f45e428513 100644--- a/test/wanted-links.test.ts+++ b/test/wanted-links.test.ts@@ -10,11 +10,12 @@ import { collectWantedLinks, possibleWantedRows } from '../src/core/wanted-linksimport { LINK_EXTRACTOR_VERSION_TS, type LinkCandidate } from '../src/core/link-extraction.ts';import { bareNameReferenceSettles, BRACKETED_CODE, disabledClearsRows, FOREIGN_PREFIXES, forwardReferenceHeals,impossibleTargetsNeverAbortTheSweep, impossibleTargetsStayOutOfWrites, onlyUnresolvedAuthoredReferences, POSSIBLE_WANTED,- privateOriginsStayPrivate, restoredTargetHeals, wantedKeys } from './helpers/wanted-links-scenarios.ts';+ privateOriginsStayPrivate, restoredTargetHeals, topLevelSlugNameReferenceSettles, wantedKeys } from './helpers/wanted-links-scenarios.ts';test('a link written before its target exists becomes an edge after the target is created', () => forwardReferenceHeals(), 120_000);test('resolved references, prose paths and code spans are never wanted', () => onlyUnresolvedAuthoredReferences(), 120_000);test('a bare-name reference matched only by basename settles after one re-extraction', () => bareNameReferenceSettles(), 120_000);+test('#9140: a bare-name reference equal to a top-level slug settles after one re-extraction', () => topLevelSlugNameReferenceSettles(), 120_000);test('remote callers never see targets or counts from private origins', () => privateOriginsStayPrivate(), 120_000);test('restoring a deleted target heals links written while it was deleted', () => restoredTargetHeals(), 120_000);test('wanted_pages.enabled=false clears an origin\'s rows on its next extraction', () => disabledClearsRows(), 120_000);