Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

docs(retrieval): recall's query arm and fetch write the stale-page signal; pin the writers (#7200)

garrytand9e98b5bc562
diff --git a/docs/architecture/key-files/core-services-1.md b/docs/architecture/key-files/core-services-1.md
index f27373a76d..bb4435d3f6 100644
--- a/docs/architecture/key-files/core-services-1.md
+++ b/docs/architecture/key-files/core-services-1.md
@@ -21,7 +21,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/brainstorm/checkpoint.ts` — crash-resilient checkpoint for `gbrain brainstorm` and `gbrain lsd`. Persists FULL idea bodies (~50KB/run) so resume MERGES pre-crash ideas with post-resume ideas before the judge runs (a resume that produces only second-run output is silent partial output). `run_id = sha256(question + profile + sort(close_slugs) + sort(far_slugs)).slice(0,16)` — NO embedding bits, stable across embedding-model swaps. Atomic write via `.tmp + rename`. ONE resume flag (`--resume <run_id>` covers failed crosses, never-attempted crosses, AND a failed judge phase — the orchestrator keeps the checkpoint with `judge_done=false` whenever the judge throws, so `--resume` short-circuits every completed cross from disk and re-runs only the judge; there is no separate retry-judge flag); only a fully clean run (all crosses + judge) clears the file. The checkpoint also banks the CLI's saved-page slug (`idea_slug`, from `RunBrainstormOpts.ideaSlug`) and the orchestrator echoes it back as `BrainstormResult.idea_slug` on resume, so the re-scored page overwrites the one the failed run saved instead of forking a second page under a fresh nonce. `--list-runs` prints run_ids mtime-newest-first; `--force-resume` bypasses the 7-day staleness gate. Cycle purge phase (`gbrain dream --phase purge`) GCs checkpoints older than 7 days via `gcStaleCheckpoints(7)`. Pinned by `test/e2e/brainstorm-resume.test.ts` (20 unit + 5 E2E cases incl. the merge contract and judge-failed retention/re-score).
-- `src/core/brainstorm/{domain-bank,orchestrator,judges}.ts` + `src/commands/{brainstorm,lsd,eval-brainstorm}.ts` + `src/core/last-retrieved.ts` — bisociation-grounded idea generation pair: `gbrain brainstorm <question>` (defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) and `gbrain lsd <question>` (Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias via `pages.last_retrieved_at`, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (`SELECT DISTINCT substring(slug from '^[^/]+/[^/]+')` cached 1h-TTL in `config` per source) tiebroken by `JOIN page_links` connection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via `1 - clamp(cosine_distance, 0, 2) / 2`. `judges.ts` exports `runJudge(config, ideas)` + two configs (`BRAINSTORM_JUDGE_CONFIG` weighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vs `LSD_JUDGE_CONFIG` cognitive_load 0.50 + inversion rule). Calibration cold-start fallback: when `calibration_profiles.active_bias_tags` is empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back in `src/core/operations.ts` `search`/`query`/`get_page` handlers fires `bumpLastRetrievedAt(engine, pageIds)` (fire-and-forget, 5-min throttled via SQL clause, default-on with `search.track_retrieval` config escape hatch, read per engine); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scoped `Set<Promise<unknown>>`; `awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}>` resolves once all tracked promises settle, bounded by a 5s `Promise.race` timeout that stderr-warns the pending count. `src/cli.ts` awaits the drain unconditionally for every op in the op-dispatch finally block BEFORE `engine.disconnect()`, then a fallback `process.exit(0)` fires ONLY when `outcome === 'timeout'` AND `shouldForceExitAfterMain(argv)` (excludes `serve` so daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive. The timestamp lives in `page_retrievals(page_id, last_retrieved_at)` (v224): the bump is an `INSERT ... ON CONFLICT DO UPDATE` throttled to 5 minutes, a HOT update with no triggers that never advances `page_generation_clock_seq` (the old `UPDATE pages` advanced it on every read, expiring the query-cache bookmark and the get_health and projection-readiness memos). No foreign key; the `pages_forget_retrievals` statement trigger removes rows of hard-deleted pages. Readers (domain-bank samplers, volunteer stats, entity card) take `GREATEST(pages.last_retrieved_at, page_retrievals.last_retrieved_at)`; the legacy column stays (unindexed since v224) for older binaries; full forward-reference bootstrap probe on both engines. Frontmatter `mode: lsd` makes the dream-cycle synthesize phase skip LSD output via `isLsdOutput()` in `src/core/cycle/transcript-discovery.ts` short-circuiting `isDreamOutput()`. `gbrain eval brainstorm <fixture.jsonl>` is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable). `gbrain doctor` has a `brainstorm_health` check (migration applied, `search.track_retrieval` setting, an unpriced cross or judge chat model, calibration cold-start status). Cost policy lives in `src/core/brainstorm/cost-gate.ts`: no cap flag is the $5 `default` tracker cap (an unpriced model warns and runs, priced calls stay metered; the estimate, mid-run guard and pre-judge check count it at Sonnet rates), `--max-usd N` (legacy `--max-cost`) is a `user` cap that refuses a run touching an unpriced chat, judge or embedding model before any work with the `no-pricing.ts` refusal, `--max-usd off` drops the USD ceiling; every price reads `pricing.overrides`; results carry `cost.cap_usd`/`cost.cap_source`. `judges.ts` computes the judge token budget via `computeJudgeMaxTokens(ideaCount, modelId)` (named constants `TOKEN_BUDGET_PER_IDEA`, `TOKEN_BUDGET_ENVELOPE`, `LEGACY_MIN_MAX_TOKENS`, `MAX_OUTPUT_TOKENS_CEIL`; `ANTHROPIC_OUTPUT_CAPS` map: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with no `modelOverride` the cap routes through the gateway's actual configured chat model via `getChatModel()`. `--save` for both commands persists through the canonical ingestion path: `persistSavedIdea(engine, {slug, content, provenanceVia})` calls `importFromContent({noEmbed:true, sourcePath, holdNoncanonicalPrivate:true})` (a held private request refuses put_page's `visibility_not_applied` and any other refusal is a failed save, #7897; chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the shared `writePageThrough` helper (file rendered FROM the row so the two sinks can't diverge and `gbrain sync` doesn't churn it). `formatSaveOutcome(outcome, ctx)` returns an honest per-branch message (both-sinks, DB-only when no `sync.repo_path`/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loud `save FAILED … NOT persisted` on stderr + nonzero exit) — `--save` never prints "Saved" when the DB write failed. `buildIdeaSlug(question, label, nonce?)` adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber. `--json` callers stay DB-only. `buildBrainstormFrontmatterObject(result)` in orchestrator.ts returns the object form for `serializeMarkdown`. Pinned by `test/last-retrieved.test.ts`, `test/e2e/pglite-cli-exit.serial.test.ts` (IRON-RULE: real `bun src/cli.ts` subprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival), `test/fix-wave-structural.test.ts` (asserts the drain `await` is textually BEFORE `engine.disconnect`), `test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts`. Open Collider source: `github.com/CL-ML/open-collider`.
+- `src/core/brainstorm/{domain-bank,orchestrator,judges}.ts` + `src/commands/{brainstorm,lsd,eval-brainstorm}.ts` + `src/core/last-retrieved.ts` — bisociation-grounded idea generation pair: `gbrain brainstorm <question>` (defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) and `gbrain lsd <question>` (Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias via `pages.last_retrieved_at`, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (`SELECT DISTINCT substring(slug from '^[^/]+/[^/]+')` cached 1h-TTL in `config` per source) tiebroken by `JOIN page_links` connection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via `1 - clamp(cosine_distance, 0, 2) / 2`. `judges.ts` exports `runJudge(config, ideas)` + two configs (`BRAINSTORM_JUDGE_CONFIG` weighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vs `LSD_JUDGE_CONFIG` cognitive_load 0.50 + inversion rule). Calibration cold-start fallback: when `calibration_profiles.active_bias_tags` is empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back in the `search`/`query`/`get_page`/`fetch` handlers and `recall`'s `query` arm (never its facts-only reads) fires `bumpLastRetrievedAt(engine, pageIds)` (fire-and-forget, 5-min throttled via SQL clause, default-on with `search.track_retrieval` config escape hatch, read per engine); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scoped `Set<Promise<unknown>>`; `awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}>` resolves once all tracked promises settle, bounded by a 5s `Promise.race` timeout that stderr-warns the pending count. `src/cli.ts` awaits the drain unconditionally for every op in the op-dispatch finally block BEFORE `engine.disconnect()`, then a fallback `process.exit(0)` fires ONLY when `outcome === 'timeout'` AND `shouldForceExitAfterMain(argv)` (excludes `serve` so daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive. The timestamp lives in `page_retrievals(page_id, last_retrieved_at)` (v224): the bump is an `INSERT ... ON CONFLICT DO UPDATE` throttled to 5 minutes, a HOT update with no triggers that never advances `page_generation_clock_seq` (the old `UPDATE pages` advanced it on every read, expiring the query-cache bookmark and the get_health and projection-readiness memos). No foreign key; the `pages_forget_retrievals` statement trigger removes rows of hard-deleted pages. Readers (domain-bank samplers, volunteer stats, entity card) take `GREATEST(pages.last_retrieved_at, page_retrievals.last_retrieved_at)`; the legacy column stays (unindexed since v224) for older binaries; full forward-reference bootstrap probe on both engines. Frontmatter `mode: lsd` makes the dream-cycle synthesize phase skip LSD output via `isLsdOutput()` in `src/core/cycle/transcript-discovery.ts` short-circuiting `isDreamOutput()`. `gbrain eval brainstorm <fixture.jsonl>` is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable). `gbrain doctor` has a `brainstorm_health` check (migration applied, `search.track_retrieval` setting, an unpriced cross or judge chat model, calibration cold-start status). Cost policy lives in `src/core/brainstorm/cost-gate.ts`: no cap flag is the $5 `default` tracker cap (an unpriced model warns and runs, priced calls stay metered; the estimate, mid-run guard and pre-judge check count it at Sonnet rates), `--max-usd N` (legacy `--max-cost`) is a `user` cap that refuses a run touching an unpriced chat, judge or embedding model before any work with the `no-pricing.ts` refusal, `--max-usd off` drops the USD ceiling; every price reads `pricing.overrides`; results carry `cost.cap_usd`/`cost.cap_source`. `judges.ts` computes the judge token budget via `computeJudgeMaxTokens(ideaCount, modelId)` (named constants `TOKEN_BUDGET_PER_IDEA`, `TOKEN_BUDGET_ENVELOPE`, `LEGACY_MIN_MAX_TOKENS`, `MAX_OUTPUT_TOKENS_CEIL`; `ANTHROPIC_OUTPUT_CAPS` map: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with no `modelOverride` the cap routes through the gateway's actual configured chat model via `getChatModel()`. `--save` for both commands persists through the canonical ingestion path: `persistSavedIdea(engine, {slug, content, provenanceVia})` calls `importFromContent({noEmbed:true, sourcePath, holdNoncanonicalPrivate:true})` (a held private request refuses put_page's `visibility_not_applied` and any other refusal is a failed save, #7897; chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the shared `writePageThrough` helper (file rendered FROM the row so the two sinks can't diverge and `gbrain sync` doesn't churn it). `formatSaveOutcome(outcome, ctx)` returns an honest per-branch message (both-sinks, DB-only when no `sync.repo_path`/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loud `save FAILED … NOT persisted` on stderr + nonzero exit) — `--save` never prints "Saved" when the DB write failed. `buildIdeaSlug(question, label, nonce?)` adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber. `--json` callers stay DB-only. `buildBrainstormFrontmatterObject(result)` in orchestrator.ts returns the object form for `serializeMarkdown`. Pinned by `test/last-retrieved.test.ts`, `test/last-retrieved-writers.test.ts` (the writer verbs and every call site), `test/e2e/pglite-cli-exit.serial.test.ts` (IRON-RULE: real `bun src/cli.ts` subprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival), `test/fix-wave-structural.test.ts` (asserts the drain `await` is textually BEFORE `engine.disconnect`), `test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts`. Open Collider source: `github.com/CL-ML/open-collider`.
- `src/core/budget/budget-tracker.ts` — the keystone budget primitive. One typed error (`BudgetExhausted` with `reason: 'cost' | 'runtime' | 'no_pricing'`), one schema-stable audit JSONL at `~/.gbrain/audit/budget-YYYY-Www.jsonl`. Contracts: `record()` throws when cumulative spend exceeds cap (the cap is a real ceiling, not a suggestion); `reserve()` admission checks cumulative + OUTSTANDING spend and returns an opaque `BudgetReservation` id for each held projection; `record({ reservation })` settles that id and `release(id)` frees an attempt that never billed (no-op when already settled), so N concurrent callers can't all pass against the same cumulative and a small same-model call finishing first can't free a large in-flight call's hold (settlement is by id, never by model; an id-less record settles nothing; a priced reservation recorded under an unpriced served id charges its held ceiling). `gateway.chat` reserves, then runs the attempt in one try/finally that releases the id (provider-resolution failure, invocation-policy refusal); embed reserves after its preflights; rerank, decide, the TypeSafe rerank adapter and embed-facts pass their ids. Record audit rows carry `requested_model` and `reservation`; `reserve()` hard-fails with `reason: 'no_pricing'` when `maxCostUsd` is set AND the model is missing from pricing maps, its message and `BudgetExhausted.pricing` built by `no-pricing.ts` (warn-once preserved when cap is unset, naming the registration command); `lookupPricing` normalizes recipe aliases through `resolveRecipe` before consulting the pricing tables, so `claude-cli:haiku` prices exactly like `claude-cli:claude-haiku-4-5-20251001` at `reserve()`, `record()` and `isModelPriceable()` alike (the gateway reserves with the pre-resolution string the user configured); `extractUsageFromError(err, fallback)` returns `err.usage` when the SDK provides it, else the pessimistic fallback (caller passes `maxOutputTokens`, not the optimistic pre-call estimate). `onExhausted(cb)` fires once synchronously BEFORE the throw propagates so callers can persist checkpoints. Pricing math lives in the pure `src/core/budget/reservation-cost.ts`: `priceFor(model, kind, overrides) → { pricing, source: 'override' | 'provider_wildcard' | 'table' } | null` is the one resolver (exact override, then a `<provider>:*` wildcard admitted only for recipes with `billing: 'subscription'` by `isAllowedPricingOverrideKey`, then aliases, the claude-cli → Anthropic sibling, canonical rows and free local providers); `isModelPriceable`, `usageCostUsd`/`reservationCostUsd`, skillopt preflight and synthesize_concepts all use it. `parsePricingOverrides` drops disallowed keys and adds each alias key's canonical twin. Every `record()` also feeds an in-memory model ledger keyed by (requested model, served model, touchpoint, purpose) — updated BEFORE the cap throw and before the unpriced early return — exposed as `snapshot().models`; `BudgetActualUsage` carries `requestedModelId`, `purpose`, `failed`, `countsAsCall` (default true) and `estimated`, and rows track `calls` (gateway operations), `attempts` (every record), `failed_calls`, tokens, `cost_usd` (`null` when unpriced) and `cost_basis` (`estimated` if any contributing record was). `src/core/budget/models-used.ts` maps labels to touchpoints (`gateway.chat`→chat, `gateway.expand*`→expansion, `gateway.embed*`→embedding, `gateway.rerank*`→rerank, `gateway.ocr*`→ocr, else other), builds receipt rows (`buildModelsUsed`), merges resumed segments and renders the "Models called" table. No DB change. Shared by brainstorm, cycle/budget-meter, and eval-contradictions; `BudgetMeter` keeps its public shape over it (`schema_version: 1` stamped on every dream-budget audit line). Pinned by 18 unit cases. A4 cap sources: `capSource: 'derived' | 'default' | 'user'` (getter `capSource`; an explicit `maxCostUsd` with no declared source is `user`, the pre-A4 behaviour). Only a `user` cap hard-fails an unpriced model, and that `BudgetExhausted` carries `fix` (`noPricingFix`, actor `agent` on the CLI, `host_admin` when `transport` is remote); `derived`/`default` caps warn once and run the model unmetered. Cost exhaustion carries `capSource`, and a `derived` one appends `derived_cap_exhausted` to the agent-contract log (callers report it with `consent.ts`'s `derivedCapExhaustedError`). Pinned by `test/budget-tracker-cap-source.test.ts`.
- `src/core/budget/daily-ledger.ts` — #6188 the durable per-UTC-day USD ledger over `budget_ledger`/`budget_reservations` (migration v012; no new schema), one ledger per (scope, resolver), e.g. `FENCE_REPAIR_LEDGER` (`llm_repair`/`fences`). `reserve(estimate, { capUsd })` is one conditional upsert (`reserved + committed + estimate <= cap`) plus a reservation row with `expires_at`; `dispatch` marks it sent; `settle(id, actual)` is idempotent, bound to the reservation's own day, and records an overrun; `release` returns an undispatched hold; expired holds are reclaimed before each reserve (a dispatched one is charged its estimate). Any ledger error means no call (`ledger_unavailable`); `meteredCall` runs the whole protocol. Amounts are NUMERIC(12,4), spend rounded up and caps down. `estimateChatCallUsd` prices a chat call through `pricing.overrides` and the canonical table; an unpriced model is metered at the highest canonical rate (`estimated: true`) under a default cap and refused with `noPricingGuidance` under a user cap. Never opens a BudgetTracker, so a caller's ambient tracker keeps metering. Read by doctor `fence_integrity` (today's spend). Tests: `test/daily-ledger.test.ts` (+ `test/e2e/daily-ledger-postgres.test.ts`, concurrent sessions).
diff --git a/src/core/last-retrieved.ts b/src/core/last-retrieved.ts
index b4bb29f14d..2012e165e7 100644
--- a/src/core/last-retrieved.ts
+++ b/src/core/last-retrieved.ts
@@ -3,11 +3,14 @@
*
* Architecture (codex round 2 #3 + D11 + D2 + D13):
*
- * - Op-layer, NOT engine-layer. This module is called from the `search` /
- * `query` / `get_page` op handlers in `operations.ts` AFTER the engine
- * returns. Internal callers (sync, migrations, helper flows) bypass the
- * op layer entirely, so this never fires from `import-file.ts`, the
- * dream cycle, doctor probes, etc. Pure signal: "a user-facing surface
+ * - Op-layer, NOT engine-layer. The writers are the `search`, `query`,
+ * `get_page` and `fetch` op handlers and `recall`'s `query` arm (the page
+ * search it runs beside the facts), each AFTER the engine returns; recall's
+ * facts-only reads write nothing. test/last-retrieved-writers.test.ts pins
+ * this list, so a new reader that bumps is a deliberate choice. Internal
+ * callers (sync, migrations, helper flows) bypass the op layer entirely,
+ * so this never fires from `import-file.ts`, the dream cycle, doctor
+ * probes, etc. Pure signal: "a user-facing surface
* just surfaced this page."
*
* - 5-min throttle (D2). The upsert only moves a timestamp older than
@@ -196,8 +199,8 @@ export const BUMP_LAST_RETRIEVED_SQL = `INSERT INTO page_retrievals (page_id, la
* MUST NOT await this for the op response. Empty ids list is a no-op.
*
* @param engine The BrainEngine handling the op.
- * @param pageIds The page ids surfaced by the op (search hits, query results,
- * or the single id returned by get_page).
+ * @param pageIds The page ids surfaced by the op (search, query or recall
+ * query hits, or the single page get_page or fetch returned).
*/
export function bumpLastRetrievedAt(engine: BrainEngine, pageIds: number[]): void {
if (pageIds.length === 0) return;
diff --git a/test/last-retrieved-writers.test.ts b/test/last-retrieved-writers.test.ts
new file mode 100644
index 0000000000..1c8e35f5fa
--- /dev/null
+++ b/test/last-retrieved-writers.test.ts
@@ -0,0 +1,104 @@
+/**
+ * #7200: which read verbs write the stale-page signal (page_retrievals).
+ *
+ * 1. Protects: the documented writer set in src/core/last-retrieved.ts:
+ * search, query, get_page, fetch and recall with `query` record a page
+ * retrieval; recall without `query` (a read of facts) does not.
+ * 2. Fails when: a verb drops its bump, recall's facts-only path starts
+ * bumping, or a new call site appears without a decision recorded here and
+ * in the last-retrieved.ts header.
+ * 3. Existing coverage: test/last-retrieved.test.ts covers the drain and the
+ * throttle with a stub engine; nothing pinned which verbs write.
+ * 4. No production seam: the real op handlers on PGLite.
+ */
+import { afterAll, beforeAll, beforeEach, describe, expect, test } from 'bun:test';
+import { readdirSync, readFileSync, statSync } from 'node:fs';
+import { join, relative } from 'node:path';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { operations, type OperationContext } from '../src/core/operations.ts';
+import { awaitPendingLastRetrievedWrites, _resetTrackRetrievalCacheForTests } from '../src/core/last-retrieved.ts';
+import { importFromContent } from '../src/core/import-file.ts';
+import { resetPgliteState } from './helpers/reset-pglite.ts';
+
+let engine: PGLiteEngine;
+const SLUG = 'notes/alice-example-ledger';
+
+beforeAll(async () => {
+ engine = new PGLiteEngine();
+ await engine.connect({});
+ await engine.initSchema();
+});
+
+afterAll(async () => {
+ await engine.disconnect();
+});
+
+beforeEach(async () => {
+ await resetPgliteState(engine);
+ _resetTrackRetrievalCacheForTests();
+ await importFromContent(engine, SLUG, '# Alice Example ledger\nThe quillmarsh ledger kept by alice-example.', { noEmbed: true });
+ await engine.executeRaw(`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, source) VALUES ('default', $1, 'alice-example keeps the quillmarsh ledger.', 'fact', 'world', 'fixture')`, [SLUG]);
+});
+
+function localCtx(): OperationContext {
+ return {
+ engine, config: {} as never, dryRun: false, remote: false, transport: 'stdio', sourceId: 'default',
+ logger: { info: () => {}, warn: () => {}, error: () => {}, debug: () => {} } as never,
+ } as OperationContext;
+}
+
+async function retrievedAfter(op: string, params: Record<string, unknown>): Promise<string[]> {
+ await engine.executeRaw('DELETE FROM page_retrievals');
+ await operations.find(o => o.name === op)!.handler(localCtx(), params);
+ await awaitPendingLastRetrievedWrites();
+ const rows = await engine.executeRaw<{ slug: string }>('SELECT p.slug FROM page_retrievals r JOIN pages p ON p.id = r.page_id');
+ return rows.map(r => r.slug);
+}
+
+describe('page retrieval writers', () => {
+ const WRITERS: Array<[string, Record<string, unknown>]> = [
+ ['search', { query: 'quillmarsh ledger' }],
+ ['query', { query: 'quillmarsh ledger', expand: false }],
+ ['get_page', { slug: SLUG }],
+ ['fetch', { id: SLUG }],
+ ['recall', { query: 'quillmarsh ledger' }],
+ ];
+ for (const [op, params] of WRITERS) {
+ test(`${op} ${JSON.stringify(params)} records the page it surfaced`, async () => {
+ expect(await retrievedAfter(op, params)).toEqual([SLUG]);
+ });
+ }
+
+ const NON_WRITERS: Array<[string, Record<string, unknown>]> = [
+ ['recall', { entity: SLUG }],
+ ['recall', { question: 'who keeps the quillmarsh ledger' }],
+ ];
+ for (const [op, params] of NON_WRITERS) {
+ test(`${op} ${JSON.stringify(params)} reads facts and records no page retrieval`, async () => {
+ expect(await retrievedAfter(op, params)).toEqual([]);
+ });
+ }
+
+ test('every bumpLastRetrievedAt call site is a documented writer', () => {
+ const root = join(import.meta.dir, '..');
+ const sites: Record<string, number> = {};
+ const walk = (dir: string): void => {
+ for (const name of readdirSync(dir)) {
+ const path = join(dir, name);
+ if (statSync(path).isDirectory()) walk(path);
+ else if (path.endsWith('.ts')) {
+ const count = readFileSync(path, 'utf8').match(/\bbumpLastRetrievedAt\(ctx\.engine/g)?.length ?? 0;
+ if (count) sites[relative(root, path)] = count;
+ }
+ }
+ };
+ walk(join(root, 'src'));
+ // A new reader that bumps is a deliberate choice: add it to the writer
+ // list in src/core/last-retrieved.ts and to WRITERS above.
+ expect(sites).toEqual({
+ 'src/core/ops/facts.ts': 1,
+ 'src/core/ops/pages.ts': 2,
+ 'src/core/ops/search.ts': 3,
+ });
+ });
+});