Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Skill resolver runtime: warm catalog handle, off-thread build, session GC

garrytan3582341bfb76
diff --git a/docs/architecture/key-files/skills.md b/docs/architecture/key-files/skills.md
index a6e199801e..ee94ff1238 100644
--- a/docs/architecture/key-files/skills.md
+++ b/docs/architecture/key-files/skills.md
@@ -45,8 +45,8 @@ Contract schemas: `docs/schemas/skill-resolver.v1.*.json`. Every module under
`src/core/skills/resolver/` is a leaf: no engine, operations table, provider or
`skill-catalog.ts` import, static or dynamic (`test/skill-resolver-module-graph.test.ts`).
-- `src/core/skills/resolver/index.ts` — the library entry (`gbrain/skill-resolver` export, and the entry of the Node bundle `gbrain-skill-resolver.mjs`). Re-exports build, status, remove, resolve, `ResolverError`, `resolverHome`; `resetSkillSession` deletes one session's state.
-- `src/core/skills/resolver/resolve.ts` — one turn: `normalizeTurnInput` (the turn-input schema, unknown fields refused, message capped at 8 KB), catalog loading (receipt and ranker identity, index size, gbrain version, the sliced stat freshness check: at most `STAT_CHECK_SLICE` entries per turn, a full sweep at most once per `STAT_CHECK_INTERVAL_MS`, cursor in `stat-check`), ack and no_match, explicit mentions first, merged-catalog interleave, the `min_match_strength` floor before the cut to k, `previous_user_message` as the no-match fallback, the session rules (k 5/3/1 at 50%/80% of the budget, strong-match pacing, no-repeat, one re-suggestion after 10 turns, a named skill past a spent budget), the block, `explain`, `failOpen`, `deadlineMs`. Returns `commit()`; session state changes only through it. Pinned by `test/skill-resolver.test.ts` and `test/skill-resolver-contract.test.ts`.
+- `src/core/skills/resolver/index.ts` — the library entry (`gbrain/skill-resolver` export, and the entry of the Node bundle `gbrain-skill-resolver.mjs`). Re-exports build, status, remove, resolve, `openCatalog`, `buildSkillIndexOffThread`, `ResolverError`, `resolverHome`; `resetSkillSession` deletes one session's state; `gcSkillSessions` runs one bounded async GC pass; `RESOLVER_FEATURES` names what an adapter may rely on.
+- `src/core/skills/resolver/resolve.ts` — one turn: `normalizeTurnInput` (the turn-input schema, unknown fields refused, message capped at 8 KB), catalog loading through a `TurnSource` (`openCatalogIndex`: receipt and ranker identity, index size; then per turn the gbrain version and, for `dir` and `claude_plugins` catalogs, the sliced stat freshness check: at most `STAT_CHECK_SLICE` entries per turn, a full sweep at most once per `STAT_CHECK_INTERVAL_MS`, cursor in `stat-check`; `stale.stamp` is written without fsync, only when absent and only for a catalog with `rebuild_argv`, never an adapter-owned stdin one), ack and no_match, explicit mentions first, merged-catalog interleave, the `min_match_strength` floor before the cut to k, `previous_user_message` as the no-match fallback, the session rules (k 5/3/1 at 50%/80% of the budget, strong-match pacing, no-repeat, one re-suggestion after 10 turns, a named skill past a spent budget), the block, `explain`, `failOpen`, `deadlineMs`. Returns `commit()`; session state changes only through it. Pinned by `test/skill-resolver.test.ts` and `test/skill-resolver-contract.test.ts`.
- `src/core/skills/resolver/rank.ts` — the A5 BM25F ranker over an `IndexReader` (scores summed in the prototype's order, ranked by score over summed idf, ties by name then id), the multilingual acknowledgement list, `mentionsSkill` (a `/name` counts only as a leading slash command) and `reasonText`. `test/skill-resolver-reproduction.test.ts` pins top-5 equality with the Step 1 prototype.
- `src/core/skills/resolver/tokenize.ts` — `splitWords` (shared with `scoreSkillMatch` in `skill-catalog.ts`) and `lexTerms`, the frozen Step 1 term rule (stopwords, light stemming); CJK runs become overlapping bigrams. Vectors: `test/fixtures/skill-resolver/tokenizer-vectors.json`.
- `src/core/skills/resolver/index-format.ts` — `index.bin` (magic `GBSKIDX1`, JSON header, field-length averages, per-skill lengths, sorted term dictionary, flat postings with per-field tf, per-skill meta), byte-identical for identical records; `VARIANT_PARAMS` (D and T constants), `fieldText`, `summaryOf` (the description cut at 120 characters), `IndexReader` (lazy postings and meta reads).
@@ -54,7 +54,9 @@ Contract schemas: `docs/schemas/skill-resolver.v1.*.json`. Every module under
- `src/core/skills/resolver/sources.ts` — the three sources: `--dir` (real paths only, symlinks out of the root skipped, breadth-first, first id wins, plugin namespace from `.claude-plugin/plugin.json`, stat fingerprint), `--bundled` (dynamic import of `src/core/skills/bundled.ts`, never the operations table) and `--from-json` (8 MB, 10,000 entries); `applyRouting`; bounds (`MAX_SKILLS`, `MAX_DIRS_VISITED`, `MAX_DEPTH`).
- `src/core/skills/resolver/frontmatter.ts` — bounded SKILL.md reader (64 KB file, 4 KB body): name, description, `when_to_use`, triggers, `disable-model-invocation`.
- `src/core/skills/resolver/routing.ts` — one RESOLVER-style routing file's table and compact-list rows, skipping the generated skills block.
-- `src/core/skills/resolver/session.ts` — session state (`sessions/<sha256(scope, harness, id)>.json`: ids and counters only), `commitSession` under a short lease, file and id caps, 7-day GC with a per-run work bound.
+- `src/core/skills/resolver/session.ts` — session state (`sessions/<sha256(scope, harness, id)>.json`: ids and counters only), `commitSession` under a short lease, file and id caps, 7-day GC with a per-run work bound; every commit (sync or async) then starts an async GC pass after it settles, at most once per `GC_INTERVAL_MS` (10 min) per home by the mtime of `logs/session-gc.stamp`, so new sessions keep deduping past `MAX_SESSION_FILES`.
+- `src/core/skills/resolver/handle.ts` — `openCatalog(catalogs, opts)`, the warm handle for long-lived hosts: keeps each catalog's receipt and open `IndexReader` and the settings, re-stats `receipt.json`, `index.bin` and `config.json` once per turn and reloads only what changed; turns run `resolve.ts` `runTurn`, so output, `pending`, `commit()` and errors equal `resolveSkills` apart from `meta.timing_ms` (`test/skill-resolver-runtime.test.ts`).
+- `src/core/skills/resolver/build-thread.ts` — `buildSkillIndexOffThread`: `buildSkillIndex` in a `node:worker_threads` Worker whose entry is the module's own URL (in the Node bundle, the bundle), run by a top-level guard keyed on `BUILD_THREAD_MARKER` in `workerData`; no worker file ships; a `ResolverError` comes back with its code, message, fix and context. The hook lane never reaches it (`test/skill-resolver-module-graph.test.ts`).
- `src/core/skills/resolver/block.ts` — the `<relevant_skills>` block: header, one JSON-quoted line per suggestion (id, name, path for codex and openclaw, summary unless names-only), one load line per harness or `--load-hint`, cut at whole lines to 400 tokens by `estimateTokens`.
- `src/core/skills/resolver/lock.ts` — ownership leases (O_EXCL create, random token, expiry; reclaimed only after expiry; release and renewal check the token).
- `src/core/skills/resolver/config.ts` — `skills.resolver.session_token_budget` (0 or 100-50,000, default 2,000) and `auto_rebuild` / `GBRAIN_SKILLS_NO_AUTOREBUILD=1`, read straight from `config.json`.
diff --git a/docs/guides/skill-resolver.md b/docs/guides/skill-resolver.md
index 3fe06fd8e9..88c2a2c51b 100644
--- a/docs/guides/skill-resolver.md
+++ b/docs/guides/skill-resolver.md
@@ -192,7 +192,9 @@ With `--session <id>` the resolver remembers which skills it suggested:
The key is `sha256(on|shadow, harness, session id)`: the same id under two
harnesses is two sessions, and merged catalogs share one. Session files idle
-for 7 days are deleted.
+for 7 days are deleted: after a commit (at most once every 10 minutes per
+home, at most 1,000 files a pass, with async I/O after the commit settles),
+and on each build and status.
Measured block size at k=5 over the Step 1 test turns on gbrain's own skills:
median 267 tokens (Claude Code), 268 (Hermes), 270 (qm), 334 (Codex) and 339
@@ -201,7 +203,9 @@ median 267 tokens (Claude Code), 268 (Hermes), 270 (qm), 334 (Codex) and 339
## Freshness and rebuilds
`resolve` never rebuilds. When the source changed it still answers from the
-existing index with `meta.index_stale: true` and touches `stale.stamp`; the
+existing index with `meta.index_stale: true` and writes `stale.stamp` once (a
+later stale turn leaves it alone; a `--from-json -` catalog, which only its
+adapter rebuilds, never gets one); the
rebuild runs from `gbrain skills index build` (which prints the exact command
in `skills index status`). For a `--dir` catalog the turn checks file sizes and
mtimes, at most 1,024 per turn and a full sweep at most once a minute;
@@ -227,6 +231,7 @@ Nothing leaves the machine. Everything lives under `$GBRAIN_HOME/.gbrain/skill-i
| `<catalog>/fingerprint.json`, `stat-check`, `stale.stamp`, `lock`, `build.log` | freshness, lease and the last build error |
| `sessions/<key>.json` | skill ids and turn counters; never turn text |
| `logs/lane-<date>.jsonl`, `logs/resolver-<date>.log` | one line per lane turn, and scrubbed stack traces of unexpected failures; never turn text; one file per day, kept 14 days |
+| `logs/session-gc.stamp` | empty; its mtime is the last session GC |
The block never echoes the user's text. Summaries (each skill's description
cut at 120 characters, the `summary` field) are catalog text, so they are
@@ -283,6 +288,36 @@ On an event loop that must not block, commit with async I/O instead:
`if (r.pending) await commitSessionAsync(home, r.pending.key, r.pending.delta)`
(same file, lease and rules as `commit()`).
+A host that stays up across turns (`open_catalog` in `RESOLVER_FEATURES`)
+opens a warm handle once instead of reopening the index every turn:
+
+```ts
+const h = openCatalog(['codex-user'], { failOpen: true });
+const r = await h.resolve({ message, session_id, harness: 'codex' });
+inject(r.output.block);
+await r.commit();
+h.close(); // on shutdown
+```
+
+The handle keeps each catalog's receipt, term dictionary and settings in
+memory and checks once per turn (a stat of `receipt.json`, `index.bin` and
+`config.json`) whether a rebuild or a config edit replaced them, reloading
+only that catalog. Its results are the same as `resolveSkills` for the same
+input, apart from `meta.timing_ms`; a removed catalog refuses the same way.
+`h.resolve(input, { explainTarget, now, ... })` takes per-turn options.
+At 1,000 skills it took a turn's p50 from 0.39 ms to 0.14 ms when the
+freshness sweep had nothing to do.
+
+`await buildSkillIndexOffThread(input, { home })` (`build_off_thread`) runs
+`buildSkillIndex` on a worker thread so a build of 100 ms or more doesn't
+block the host's event loop. It returns the same receipt and rethrows a
+`ResolverError` with its code and fix. It works from Bun and from the Node
+bundle, whose own file is the worker's entry, so nothing else ships.
+
+Commits already collect idle session files every 10 minutes; a host that
+wants its own schedule calls `await gcSkillSessions({ home })` (`gc_sessions`),
+one bounded pass that returns how many files it deleted.
+
## Error codes
### skill_index_missing
diff --git a/scripts/build-skill-resolver-bundle.ts b/scripts/build-skill-resolver-bundle.ts
index 5ad01c1891..dde86cf13e 100644
--- a/scripts/build-skill-resolver-bundle.ts
+++ b/scripts/build-skill-resolver-bundle.ts
@@ -8,7 +8,9 @@
* Usage: bun scripts/build-skill-resolver-bundle.ts --out <file.mjs> [--smoke]
* Writes <file.mjs>.sha256 beside it (sha256sum format). --smoke imports the
* bundle with node, builds a catalog from JSON and the bundled catalog in a
- * scratch home, resolves one turn, and exits 1 on any failure.
+ * scratch home, resolves one turn, builds one catalog on a worker thread
+ * (the bundle is its own worker entry) and resolves it through a warm handle,
+ * and exits 1 on any failure.
*/
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
@@ -36,7 +38,14 @@ if (args.includes('--smoke')) {
"if (t.output.suggestions[0]?.id !== 'x:deploy' || t.output.suggestions[0]?.match_strength !== 'name' || b.catalog.skill_count < 1) { console.error(JSON.stringify(t.output)); process.exit(1); }",
"if (!(await m.commitSessionAsync(home, t.pending.key, t.pending.delta))) { console.error('commitSessionAsync did not commit'); process.exit(1); }",
`if (m.GBRAIN_VERSION !== ${JSON.stringify(VERSION)}) { console.error('GBRAIN_VERSION ' + m.GBRAIN_VERSION); process.exit(1); }`,
- "console.log(JSON.stringify({ node: process.version, gbrain_version: m.GBRAIN_VERSION, bundled_skills: b.catalog.skill_count, top: t.output.suggestions[0].id }));",
+ "const off = await m.buildSkillIndexOffThread({ catalog: 'w', fromJson: JSON.stringify([{ id: 'x:invoice', name: 'invoice', description: 'Draft an invoice for a customer' }]), fromJsonStdin: true }, { home });",
+ "const bad = await m.buildSkillIndexOffThread({ catalog: '../w', fromJson: '[]', fromJsonStdin: true }, { home }).then(() => null, e => e);",
+ "if (!(bad instanceof m.ResolverError) || bad.code !== 'skill_catalog_unknown' || !bad.fix.suggestion) { console.error('off-thread error ' + String(bad)); process.exit(1); }",
+ "const h = m.openCatalog(['w'], { home });",
+ "const w = await h.resolve({ message: 'draft an invoice' });",
+ "h.close();",
+ "if (off.catalog.skill_count !== 1 || w.output.suggestions[0]?.id !== 'x:invoice') { console.error(JSON.stringify(w.output)); process.exit(1); }",
+ "console.log(JSON.stringify({ node: process.version, gbrain_version: m.GBRAIN_VERSION, bundled_skills: b.catalog.skill_count, top: t.output.suggestions[0].id, off_thread: off.catalog.name, warm_top: w.output.suggestions[0].id }));",
].join('\n'));
const run = Bun.spawnSync(['node', probe], { stdout: 'pipe', stderr: 'pipe', cwd: home });
process.stdout.write(run.stdout.toString());
diff --git a/scripts/core-world-reads-baseline.tsv b/scripts/core-world-reads-baseline.tsv
index 1155ac0ad1..839a05f13b 100644
--- a/scripts/core-world-reads-baseline.tsv
+++ b/scripts/core-world-reads-baseline.tsv
@@ -1071,6 +1071,8 @@ src/core/skills/resolver/config.ts process_env 1 GBRA-377 skill resolver: GBRAIN
src/core/skills/resolver/catalog-files.ts fs_import 1 GBRA-377 skill resolver: a filesystem tool by contract (reads SKILL.md dirs, writes its index and session files under GBRAIN_HOME/skill-index); it never opens a brain or an engine, so there are no engine settings to take it from; stays
src/core/skills/resolver/daily-log.ts fs_import 2 GBRA-399 skill resolver daily logs: appends and prunes lane-<day>.jsonl and resolver-<day>.log under GBRAIN_HOME/skill-index (sync for the CLI, async in the hook); no brain or engine; stays
src/core/skills/resolver/fsutil.ts fs_import 1 GBRA-377 skill resolver: a filesystem tool by contract (reads SKILL.md dirs, writes its index and session files under GBRAIN_HOME/skill-index); it never opens a brain or an engine, so there are no engine settings to take it from; stays
+src/core/skills/resolver/handle.ts fs_import 1 GBRA-399 skill resolver warm handle: stats each catalog's receipt.json and index.bin and GBRAIN_HOME/config.json once per turn to see a rebuild or a config edit; no brain or engine; stays
+src/core/skills/resolver/handle.ts process_env 1 GBRA-399 skill resolver warm handle: GBRAIN_SKILLS_NO_AUTOREBUILD from the injectable env (process.env by default, as config.ts reads it) keys its cached settings; no brain or engine; stays
src/core/skills/resolver/index-format.ts fs_import 1 GBRA-377 skill resolver: a filesystem tool by contract (reads SKILL.md dirs, writes its index and session files under GBRAIN_HOME/skill-index); it never opens a brain or an engine, so there are no engine settings to take it from; stays
src/core/skills/resolver/lock.ts fs_import 1 GBRA-377 skill resolver: a filesystem tool by contract (reads SKILL.md dirs, writes its index and session files under GBRAIN_HOME/skill-index); it never opens a brain or an engine, so there are no engine settings to take it from; stays
src/core/skills/resolver/lane-worker.ts process_env 1 GBRA-399 skill resolver lane worker: process.env as the default of an injectable env parameter, handed to the worker the hook starts first; no brain or engine; stays
diff --git a/scripts/structural-suites.tsv b/scripts/structural-suites.tsv
index 43b33482fc..0eedc23ab2 100644
--- a/scripts/structural-suites.tsv
+++ b/scripts/structural-suites.tsv
@@ -388,6 +388,7 @@ test/shared-skills-writer-boundary.test.ts (file-level) 7 readFileSync
test/skill-resolver-hook-lane.test.ts skill-resolver lane: compiled binaries 1 readFileSync
test/skill-resolver-hook-lane.test.ts skill-resolver lane: delivery per harness 4 readFileSync
test/skill-resolver-hook-lane.test.ts skill-resolver lane: fail open 6 readFileSync
+test/skill-resolver-module-graph.test.ts the skill resolver module graph 4 readFileSync
test/skill-resolver-off-pin.test.ts (file-level) 1 readFileSync
test/skill-resolver-packaging.test.ts skill resolver packaging 3 bun-file,readFileSync
test/skillify-scaffold.test.ts 11-item scaffold contract (T9 + Phase 3 cross-modal eval) 1 readFileSync
diff --git a/src/core/skills/resolver/build-thread.ts b/src/core/skills/resolver/build-thread.ts
new file mode 100644
index 0000000000..0d007c5b7b
--- /dev/null
+++ b/src/core/skills/resolver/build-thread.ts
@@ -0,0 +1,54 @@
+/**
+ * `buildSkillIndexOffThread(input, opts)`: buildSkillIndex in a
+ * node:worker_threads Worker, so a Node or Bun host's event loop keeps
+ * running through a 100+ ms build. It resolves with the same BuildReceipt;
+ * a ResolverError thrown in the worker is rethrown here with the same code,
+ * message, fix and context.
+ *
+ * No separate worker file ships: the worker's entry is this module's own
+ * URL (in the Node bundle gbrain-skill-resolver.mjs, the bundle itself), and
+ * the guard at the bottom runs the build only in a worker started with
+ * BUILD_THREAD_MARKER in its workerData. The hook lane's worker never loads
+ * this module.
+ */
+import { isMainThread, parentPort, Worker, workerData } from 'node:worker_threads';
+import { buildSkillIndex, type BuildInput, type BuildReceipt } from './build.ts';
+import { ResolverError, type ResolverErrorCode, type ResolverFix } from './errors.ts';
+import { resolverHome } from './paths.ts';
+
+export const BUILD_THREAD_MARKER = 'gbrain-skill-resolver-build-thread-v1';
+
+interface Job { [BUILD_THREAD_MARKER]: true; input: BuildInput; home: string }
+type Failure = { resolver: true; code: ResolverErrorCode; message: string; fix: ResolverFix; context: Record<string, string | number | boolean> } | { resolver: false; name: string; message: string; code?: string };
+type Reply = { receipt: BuildReceipt } | { failure: Failure };
+
+function revive(f: Failure): Error {
+ if (f.resolver) return new ResolverError(f.code, f.message, f.fix, f.context);
+ return Object.assign(new Error(f.message), { name: f.name, ...(f.code ? { code: f.code } : {}) });
+}
+
+/** Build a catalog's index on a worker thread; same input, receipt and errors as buildSkillIndex. */
+export function buildSkillIndexOffThread(input: BuildInput, opts: { home?: string } = {}): Promise<BuildReceipt> {
+ const home = opts.home ?? resolverHome();
+ return new Promise((resolve, reject) => {
+ let settled = false;
+ const settle = (fn: () => void) => { if (!settled) { settled = true; fn(); } };
+ const job: Job = { [BUILD_THREAD_MARKER]: true, input, home };
+ const worker = new Worker(new URL(import.meta.url), { workerData: job });
+ worker.once('message', (m: Reply) => settle(() => ('receipt' in m ? resolve(m.receipt) : reject(revive(m.failure)))));
+ worker.once('error', e => settle(() => reject(e)));
+ worker.once('exit', () => settle(() => reject(new ResolverError('skill_resolver_internal', 'The skill index build thread exited without a result.', { suggestion: 'Run the build again; if it repeats, run gbrain skills index build in a shell to see the error.' }))));
+ });
+}
+
+if (!isMainThread && (workerData as Record<string, unknown> | null)?.[BUILD_THREAD_MARKER] === true) {
+ const { input, home } = workerData as Job;
+ buildSkillIndex(input, { home }).then(
+ receipt => parentPort!.postMessage({ receipt } satisfies Reply),
+ (e: unknown) => {
+ const failure: Failure = e instanceof ResolverError ? { resolver: true, code: e.code, message: e.message, fix: e.fix, context: e.context }
+ : { resolver: false, name: (e as Error)?.name ?? 'Error', message: String((e as Error)?.message ?? e), ...(typeof (e as NodeJS.ErrnoException)?.code === 'string' ? { code: (e as NodeJS.ErrnoException).code } : {}) };
+ parentPort!.postMessage({ failure } satisfies Reply);
+ },
+ );
+}
diff --git a/src/core/skills/resolver/handle.ts b/src/core/skills/resolver/handle.ts
new file mode 100644
index 0000000000..51597c9b51
--- /dev/null
+++ b/src/core/skills/resolver/handle.ts
@@ -0,0 +1,60 @@
+/**
+ * A warm catalog handle for a long-lived host (an adapter's process, a
+ * daemon): `openCatalog(catalogs, opts)` keeps each catalog's receipt and
+ * open index (term dictionary, lengths, offsets) and the resolver settings
+ * in memory, so a turn reads only the postings and metadata it ranks. Each
+ * turn stats every catalog's receipt.json and index.bin, and config.json,
+ * once, and reloads only what a rebuild or an edit replaced; a removed
+ * catalog refuses exactly as it does cold. Turns run through resolve.ts
+ * runTurn, the code resolveSkills runs, so the output, the pending commit
+ * and every error match resolveSkills byte for byte apart from
+ * meta.timing_ms. The per-turn freshness sweep and stamps are unchanged.
+ */
+import { statSync } from 'node:fs';
+import { join } from 'node:path';
+import { readResolverSettings, type ResolverSettings } from './config.ts';
+import { catalogFile } from './paths.ts';
+import { openCatalogIndex, runTurn, type OpenedCatalog, type ResolveOptions, type ResolveResult, type TurnInput, type TurnSource } from './resolve.ts';
+
+export type CatalogHandleOptions = Omit<ResolveOptions, 'catalogs'>;
+export type CatalogTurnOptions = Omit<ResolveOptions, 'catalogs' | 'home' | 'env'>;
+
+export interface CatalogHandle {
+ /** One turn, as resolveSkills(input, { ...opts, ...turn, catalogs }). */
+ resolve(input: TurnInput | unknown, turn?: CatalogTurnOptions): Promise<ResolveResult>;
+ /** Close the open index files; a later resolve opens them again. */
+ close(): void;
+}
+
+const fileId = (path: string): string => { try { const s = statSync(path); return `${s.dev}:${s.ino}:${s.size}:${s.mtimeMs}`; } catch { return '-'; } };
+
+/** Open a warm handle over the named catalogs (none: the only built catalog, as resolveSkills picks it each turn). */
+export function openCatalog(catalogs: string[], opts: CatalogHandleOptions = {}): CatalogHandle {
+ const kept = new Map<string, { id: string; c: OpenedCatalog }>();
+ let settings: { key: string; value?: ResolverSettings; error?: unknown } | null = null;
+ const drop = (name: string) => { kept.get(name)?.c.ix.close(); kept.delete(name); };
+ const source: TurnSource = {
+ settings(home, env) {
+ const key = `${home}|${fileId(join(home, 'config.json'))}|${(env ?? process.env).GBRAIN_SKILLS_NO_AUTOREBUILD}`;
+ if (settings?.key !== key) {
+ try { settings = { key, value: readResolverSettings(home, env) }; } catch (error) { settings = { key, error }; }
+ }
+ if (settings.error !== undefined) throw settings.error;
+ return settings.value!;
+ },
+ open(home, name) {
+ const id = `${home}|${fileId(catalogFile(home, name, 'receipt.json'))}|${fileId(catalogFile(home, name, 'index.bin'))}`;
+ const k = kept.get(name);
+ if (k?.id === id) return k.c;
+ drop(name);
+ const c = openCatalogIndex(home, name);
+ kept.set(name, { id, c });
+ return c;
+ },
+ done(opened) { for (const name of [...kept.keys()]) if (!opened.some(c => c.name === name)) drop(name); },
+ };
+ return {
+ resolve: (input, turn = {}) => runTurn(input, { ...opts, ...turn, catalogs }, source),
+ close: () => { for (const name of [...kept.keys()]) drop(name); },
+ };
+}
diff --git a/src/core/skills/resolver/index.ts b/src/core/skills/resolver/index.ts
index 6c1decd3d0..84af8815b8 100644
--- a/src/core/skills/resolver/index.ts
+++ b/src/core/skills/resolver/index.ts
@@ -13,22 +13,30 @@
export { buildSkillIndex, catalogStatus, listCatalogs, removeSkillIndex, sessionStatus, type BuildInput, type BuildReceipt, type CatalogStatus } from './build.ts';
export { resolveSkills, normalizeTurnInput, errorTurnOutput, SCHEMA_VERSION, EXCLUDE_MAX, MATCH_STRENGTHS, type Explain, type MatchStrength, type ResolveOptions, type ResolveResult, type Suggestion, type TurnInput, type TurnOutput } from './resolve.ts';
export { commitSessionAsync, type TurnDelta } from './session.ts';
+export { openCatalog, type CatalogHandle, type CatalogHandleOptions, type CatalogTurnOptions } from './handle.ts';
+export { buildSkillIndexOffThread } from './build-thread.ts';
export { VERSION as GBRAIN_VERSION } from '../../../version.ts';
export { ResolverError, type ResolverErrorCode } from './errors.ts';
export { HARNESSES, BLOCK_TOKEN_CAP } from './block.ts';
export { resolverHome } from './paths.ts';
import { resolverHome } from './paths.ts';
-import { resetSession, sessionKey } from './session.ts';
+import { gcSessionsAsync, resetSession, sessionKey } from './session.ts';
/**
* What this build of the resolver supports beyond the first v1 release, so
* an adapter can check before it sends a field: min_match_strength in the
* turn input, exclude up to EXCLUDE_MAX ids, the index's catalog_hash on
* skill_catalog_mismatch, a named skill's line past a spent budget,
- * previous_user_message as the fallback ranking text, and the pi and cursor
- * harnesses.
+ * previous_user_message as the fallback ranking text, the pi and cursor
+ * harnesses, the warm handle (openCatalog), the worker-thread build
+ * (buildSkillIndexOffThread) and gcSkillSessions.
*/
-export const RESOLVER_FEATURES: readonly string[] = ['min_match_strength', 'exclude_1024', 'mismatch_catalog_hash', 'named_past_budget', 'previous_user_message', 'harness_pi', 'harness_cursor'];
+export const RESOLVER_FEATURES: readonly string[] = ['min_match_strength', 'exclude_1024', 'mismatch_catalog_hash', 'named_past_budget', 'previous_user_message', 'harness_pi', 'harness_cursor', 'open_catalog', 'build_off_thread', 'gc_sessions'];
+
+/** One bounded pass (at most 1,000 files, async I/O) deleting session files idle for 7 days; returns how many it deleted. Commits already run one at most every 10 minutes. */
+export async function gcSkillSessions(opts: { home?: string; now?: number } = {}): Promise<number> {
+ return gcSessionsAsync(opts.home ?? resolverHome(), opts.now);
+}
/** Drop a session's dedupe and budget state (what a compaction does). */
export async function resetSkillSession(sessionId: string, opts: { home?: string; harness?: string; shadow?: boolean } = {}): Promise<void> {
diff --git a/src/core/skills/resolver/resolve.ts b/src/core/skills/resolver/resolve.ts
index 125881c450..7cfe0de3a8 100644
--- a/src/core/skills/resolver/resolve.ts
+++ b/src/core/skills/resolver/resolve.ts
@@ -30,10 +30,10 @@ import { VERSION } from '../../../version.ts';
import { BLOCK_TOKEN_CAP, BLOCK_TOKENIZER, HARNESSES, renderBlock, type BlockLine } from './block.ts';
import type { BuildReceipt } from './build.ts';
import { dirFingerprintSlice, listCatalogs, readReceipt } from './catalog-files.ts';
-import { readResolverSettings, validateSessionTokenBudget } from './config.ts';
+import { readResolverSettings, type ResolverSettings, validateSessionTokenBudget } from './config.ts';
import { ResolverError } from './errors.ts';
import { appendDailyLogSync, RESOLVER_LOG } from './daily-log.ts';
-import { ensureDir, writeFileAtomic, writeFileDurable } from './fsutil.ts';
+import { ensureDir, writeFileAtomic } from './fsutil.ts';
import { EXCLUDE_MAX, MATCH_STRENGTHS, type MatchStrength, STRENGTH_FIELDS, TOP_K_DEFAULT, TOP_K_MAX } from './limits.ts';
import { IndexReader, RANKER_ID, RANKER_VERSION } from './index-format.ts';
import { assertCatalogName, catalogFile, logsDir, resolverHome } from './paths.ts';
@@ -165,7 +165,20 @@ export function normalizeTurnInput(raw: unknown): Required<Pick<TurnInput, 'mess
};
}
-interface Loaded { name: string; receipt: BuildReceipt; ix: IndexReader; stale: string | null }
+/** A catalog's receipt and open index: what a turn reads once per catalog, and what a warm handle keeps between turns. */
+export interface OpenedCatalog { name: string; receipt: BuildReceipt; ix: IndexReader; publishing: boolean }
+interface Loaded extends OpenedCatalog { stale: string | null }
+
+/** Where a turn gets its settings and catalogs: the cold path reads them every turn, a warm handle (handle.ts) keeps them. */
+export interface TurnSource {
+ settings(home: string, env?: NodeJS.ProcessEnv): ResolverSettings;
+ open(home: string, name: string): OpenedCatalog;
+ /** Called once per turn with every catalog it opened. */
+ done(opened: OpenedCatalog[]): void;
+}
+
+const COLD: TurnSource = { settings: readResolverSettings, open: openCatalogIndex, done: opened => { for (const c of opened) c.ix.close(); } };
+const STAT_TIER_KINDS: readonly string[] = ['dir', 'claude_plugins'];
function defaultCatalogs(home: string): string[] {
const all = listCatalogs(home);
@@ -174,7 +187,8 @@ function defaultCatalogs(home: string): string[] {
throw new ResolverError('skill_catalog_unknown', `Several catalogs exist (${all.join(', ')}); pick one with --catalog.`, { suggestion: `Pass --catalog with one of: ${all.join(', ')}.` }, { catalogs: all.join(',') });
}
-function loadCatalog(home: string, name: string, now: number, autoRebuild: boolean): Loaded {
+/** Read a catalog's receipt and open its index; throws the turn's refusal when either is missing or incompatible. */
+export function openCatalogIndex(home: string, name: string): OpenedCatalog {
let receipt = readReceipt(home, name);
if (!receipt) {
if (existsSync(catalogFile(home, name, 'index.bin'))) throw new ResolverError('skill_index_incompatible', `The catalog "${name}" has an index but no readable receipt.`, { suggestion: 'Rebuild it with gbrain skills index build.' }, { catalog: name });
@@ -186,14 +200,28 @@ function loadCatalog(home: string, name: string, now: number, autoRebuild: boole
fix ? { suggestion: 'Rebuild it with the command it was built with.', argv: fix } : { suggestion: 'The adapter that owns this catalog rebuilds it by passing a new snapshot to gbrain skills index build --from-json -.' }, { catalog: name });
}
const ix = IndexReader.open(catalogFile(home, name, 'index.bin'), fix);
- let stale: string | null = null;
- const size = statSync(catalogFile(home, name, 'index.bin')).size;
- if (size !== receipt.index.size) {
- const again = readReceipt(home, name);
- if (again && again.index.size === size) receipt = again; else stale = 'publishing';
- }
+ let publishing = false;
+ try {
+ const size = statSync(catalogFile(home, name, 'index.bin')).size;
+ if (size !== receipt.index.size) {
+ const again = readReceipt(home, name);
+ if (again && again.index.size === size) receipt = again; else publishing = true;
+ }
+ } catch (e) { ix.close(); throw e; }
+ return { name, receipt, ix, publishing };
+}
+
+/**
+ * The turn's staleness verdict for an opened catalog: publishing, a gbrain
+ * version skew, or (directory catalogs) one slice of the stat sweep. A stale
+ * catalog gbrain can rebuild gets stale.stamp once (no fsync; an existing
+ * stamp is left alone); an adapter-owned stdin catalog never does.
+ */
+function turnStaleness(home: string, c: OpenedCatalog, now: number, autoRebuild: boolean): string | null {
+ const { name, receipt } = c;
+ let stale: string | null = c.publishing ? 'publishing' : null;
if (!stale && receipt.gbrain_version !== VERSION) stale = 'gbrain_version_changed';
- if (!stale && (receipt.catalog.source.kind === 'dir' || receipt.catalog.source.kind === 'claude_plugins')) {
+ if (!stale && STAT_TIER_KINDS.includes(receipt.catalog.source.kind)) {
const stamp = catalogFile(home, name, 'stat-check');
let last: { verdict: 'fresh' | 'stale'; cursor: number } | null = null;
let age = Infinity;
@@ -205,8 +233,11 @@ function loadCatalog(home: string, name: string, now: number, autoRebuild: boole
try { writeFileAtomic(stamp, JSON.stringify({ verdict: slice.changed ? 'stale' : 'fresh', cursor: slice.changed ? 0 : slice.next })); } catch { /* read-only home: check again next turn */ }
}
}
- if (stale && autoRebuild) { try { writeFileDurable(catalogFile(home, name, 'stale.stamp'), `${JSON.stringify({ reason: stale })}\n`); } catch { /* best effort */ } }
- return { name, receipt, ix, stale };
+ if (stale && autoRebuild && receipt.build?.rebuild_argv) {
+ const stamp = catalogFile(home, name, 'stale.stamp');
+ if (!existsSync(stamp)) { try { writeFileAtomic(stamp, `${JSON.stringify({ reason: stale })}\n`); } catch { /* best effort */ } }
+ }
+ return stale;
}
function emptyOutput(input: { shadow?: boolean }, extra: Partial<TurnOutput>): TurnOutput {
@@ -235,12 +266,17 @@ function logUnexpected(home: string, e: unknown): void {
/** Resolve one turn. Throws ResolverError on a usage error always, and on a runtime error unless failOpen. */
export async function resolveSkills(rawInput: TurnInput | unknown, opts: ResolveOptions = {}): Promise<ResolveResult> {
+ return runTurn(rawInput, opts, COLD);
+}
+
+/** One turn over a TurnSource: resolveSkills with the cold source, a warm handle with its own. */
+export async function runTurn(rawInput: TurnInput | unknown, opts: ResolveOptions, source: TurnSource): Promise<ResolveResult> {
const t0 = performance.now();
const input = normalizeTurnInput(rawInput);
let home: string;
try { home = opts.home ?? resolverHome(opts.env); } catch (e) { throw new ResolverError('skill_resolver_config_invalid', (e as Error).message, { suggestion: 'Set GBRAIN_HOME to an absolute path without .. segments, or unset it.' }); }
try {
- return await resolveInner(input, opts, home, t0);
+ return await resolveInner(input, opts, home, t0, source);
} catch (e) {
const err = e instanceof ResolverError ? e : new ResolverError('skill_resolver_internal', 'The skill resolver failed unexpectedly; a scrubbed trace was written under skill-index/logs/.', { suggestion: 'Run gbrain skills index status and report the trace if it repeats.', argv: ['gbrain', 'skills', 'index', 'status', '--json'] });
if (!(e instanceof ResolverError)) logUnexpected(home, e);
@@ -251,17 +287,17 @@ export async function resolveSkills(rawInput: TurnInput | unknown, opts: Resolve
}
}
-async function resolveInner(input: ReturnType<typeof normalizeTurnInput>, opts: ResolveOptions, home: string, t0: number): Promise<ResolveResult> {
+async function resolveInner(input: ReturnType<typeof normalizeTurnInput>, opts: ResolveOptions, home: string, t0: number, source: TurnSource): Promise<ResolveResult> {
const now = (opts.now ?? Date.now)();
const check = opts.deadlineMs === undefined ? () => {} : () => {
if (performance.now() - t0 > opts.deadlineMs!) throw new ResolverError('skill_resolve_timeout', `The resolver ran past its ${opts.deadlineMs} ms deadline; nothing was committed.`, { suggestion: 'The turn proceeds without a block; check the index size with gbrain skills index status.' });
};
- const settings = readResolverSettings(home, opts.env);
+ const settings = source.settings(home, opts.env);
const budget = opts.sessionTokenBudget === undefined ? settings.sessionTokenBudget : validateSessionTokenBudget(opts.sessionTokenBudget, 'sessionTokenBudget');
const names = opts.catalogs?.length ? [...new Set(opts.catalogs.map(assertCatalogName))] : defaultCatalogs(home);
const loaded: Loaded[] = [];
try {
- for (const n of names) { loaded.push(loadCatalog(home, n, now, settings.autoRebuild)); check(); }
+ for (const n of names) { const l: Loaded = { ...source.open(home, n), stale: null }; loaded.push(l); l.stale = turnStaleness(home, l, now, settings.autoRebuild); check(); }
const single = loaded.length === 1 ? loaded[0]! : null;
const catalog = loaded.map(l => l.name).join('+');
const catalogHash = single ? single.receipt.catalog.catalog_hash : sha(JSON.stringify(loaded.map(l => [l.name, l.receipt.catalog.catalog_hash])));
@@ -382,6 +418,6 @@ async function resolveInner(input: ReturnType<typeof normalizeTurnInput>, opts:
}
return finish(out, { tokens: r.tokens, suggested: kept.map(x => x.id), resuggested: resuggested.filter(id => kept.some(x => x.id === id)) }, explain);
} finally {
- for (const l of loaded) l.ix.close();
+ source.done(loaded);
}
}
diff --git a/src/core/skills/resolver/session.ts b/src/core/skills/resolver/session.ts
index 725041b6c9..d0dcfe12a9 100644
--- a/src/core/skills/resolver/session.ts
+++ b/src/core/skills/resolver/session.ts
@@ -11,20 +11,27 @@
* recorded blocks follow the same rules without touching real state.
* Commits are serialized per file under a short lease with a bounded wait;
* a lost commit only means a skill may be suggested again.
+ *
+ * Every commit (either kind) then starts a session GC pass, at most once per
+ * GC_INTERVAL_MS per home (the mtime of logs/session-gc.stamp), after the commit
+ * has settled and with async I/O only, so it never costs a turn's deadline;
+ * without it, past MAX_SESSION_FILES new sessions would never dedupe.
*/
import { createHash } from 'node:crypto';
import { promises, readdirSync, readFileSync, statSync, unlinkSync } from 'node:fs';
-const { mkdir, open, readdir, readFile, rename, stat, unlink } = promises;
+const { mkdir, open, readdir, readFile, rename, stat, unlink, writeFile } = promises;
import { randomBytes } from 'node:crypto';
import { join } from 'node:path';
import { ensureDir, writeFileDurable } from './fsutil.ts';
import { acquireLease, type Lease, releaseLease, releaseLeaseAsync, tryAcquireLeaseAsync } from './lock.ts';
-import { sessionsDir } from './paths.ts';
+import { logsDir, sessionsDir } from './paths.ts';
export const MAX_SESSION_FILES = 10_000;
export const MAX_SESSION_IDS = 10_000;
export const SESSION_TTL_MS = 7 * 24 * 3600 * 1000;
export const GC_WORK_BOUND = 1000;
+export const GC_INTERVAL_MS = 10 * 60 * 1000;
+const gcStamp = (home: string) => join(logsDir(home), 'session-gc.stamp');
const STATE_CAP_BYTES = 2 * 1024 * 1024;
export interface SessionState {
@@ -63,6 +70,10 @@ export interface TurnDelta { tokens: number; suggested: string[]; resuggested: s
/** Apply one delivered turn. False when the commit was skipped (busy lock, file cap). */
export async function commitSession(home: string, key: string, delta: TurnDelta): Promise<boolean> {
+ try { return await commitSync(home, key, delta); } finally { scheduleSessionGc(home); }
+}
+
+async function commitSync(home: string, key: string, delta: TurnDelta): Promise<boolean> {
const dir = sessionsDir(home);
ensureDir(dir);
const p = statePath(home, key);
@@ -96,6 +107,10 @@ function applyDelta(s: SessionState, delta: TurnDelta): SessionState {
export interface CommitHold { lease?: Lease | null; tmp?: string | null; at?: (stage: 'held' | 'written') => Promise<void> }
export async function commitSessionAsync(home: string, key: string, delta: TurnDelta, hold: CommitHold = {}): Promise<boolean> {
+ try { return await commitAsync(home, key, delta, hold); } finally { scheduleSessionGc(home); }
+}
+
+async function commitAsync(home: string, key: string, delta: TurnDelta, hold: CommitHold): Promise<boolean> {
const dir = sessionsDir(home);
const p = statePath(home, key);
try {
@@ -141,15 +156,52 @@ export async function resetSession(home: string, key: string): Promise<void> {
try { await unlink(statePath(home, key)); } catch { /* none */ }
}
+/** At most GC_WORK_BOUND of a directory's names, from a random start once there are more. */
+function gcSlice(names: string[]): string[] {
+ const start = names.length > GC_WORK_BOUND ? Math.floor(Math.random() * names.length) : 0;
+ return Array.from({ length: Math.min(names.length, GC_WORK_BOUND) }, (_, i) => names[(start + i) % names.length]!);
+}
+
/** Delete session files idle for 7 days, examining at most GC_WORK_BOUND files per run. */
export function gcSessions(home: string, now: number = Date.now()): number {
let removed = 0;
let names: string[];
try { names = readdirSync(sessionsDir(home)); } catch { return 0; }
- const start = names.length > GC_WORK_BOUND ? Math.floor(Math.random() * names.length) : 0;
- for (let i = 0; i < Math.min(names.length, GC_WORK_BOUND); i++) {
- const p = join(sessionsDir(home), names[(start + i) % names.length]!);
+ for (const name of gcSlice(names)) {
+ const p = join(sessionsDir(home), name);
try { if (now - statSync(p).mtimeMs > SESSION_TTL_MS) { unlinkSync(p); removed++; } } catch { /* raced */ }
}
return removed;
}
+
+/** gcSessions with async I/O only; it touches the GC stamp first, so the commit-path pass skips the next interval. */
+export async function gcSessionsAsync(home: string, now: number = Date.now()): Promise<number> {
+ const dir = sessionsDir(home);
+ let names: string[];
+ try {
+ names = await readdir(dir);
+ await mkdir(logsDir(home), { recursive: true, mode: 0o700 });
+ await writeFile(gcStamp(home), '', { mode: 0o600 });
+ } catch { return 0; }
+ let removed = 0;
+ for (const name of gcSlice(names)) {
+ const p = join(dir, name);
+ try { if (now - (await stat(p)).mtimeMs > SESSION_TTL_MS) { await unlink(p); removed++; } } catch { /* raced */ }
+ }
+ return removed;
+}
+
+let gcPass: Promise<number> | null = null;
+
+/** Start a GC pass once the current task ends, unless one ran in this home within GC_INTERVAL_MS. Never throws, never waited on. */
+export function scheduleSessionGc(home: string): void {
+ const prev = gcPass;
+ gcPass = new Promise<void>(r => setImmediate(r)).then(async () => {
+ await prev;
+ try { if (Date.now() - (await stat(gcStamp(home))).mtimeMs < GC_INTERVAL_MS) return 0; } catch { /* none yet */ }
+ return gcSessionsAsync(home);
+ }).catch(() => 0);
+}
+
+/** Resolves once the GC passes started so far have finished (tests). */
+export const sessionGcSettled = async (): Promise<void> => { await gcPass; };
diff --git a/test/skill-resolver-module-graph.test.ts b/test/skill-resolver-module-graph.test.ts
index 6fa35eb36f..c010ac2237 100644
--- a/test/skill-resolver-module-graph.test.ts
+++ b/test/skill-resolver-module-graph.test.ts
@@ -13,9 +13,9 @@
* (pattern: test/cold-read-module-graph.test.ts).
*/
import { describe, expect, test } from 'bun:test';
-import { mkdtempSync, rmSync } from 'node:fs';
+import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
-import { join, resolve } from 'node:path';
+import { dirname, join, resolve } from 'node:path';
const REPO_ROOT = resolve(import.meta.dir, '..');
@@ -47,6 +47,20 @@ describe('the skill resolver module graph', () => {
}
}, 60_000);
+ test('the hook lane worker reaches neither the build thread nor the warm handle', () => {
+ const transpiler = new Bun.Transpiler({ loader: 'ts' });
+ const reached = new Set<string>();
+ const walk = (file: string) => {
+ if (reached.has(file)) return;
+ reached.add(file);
+ for (const imp of transpiler.scanImports(readFileSync(file, 'utf8'))) if (imp.path.startsWith('.') && imp.path.endsWith('.ts')) walk(resolve(dirname(file), imp.path));
+ };
+ for (const entry of ['hook-worker.ts', 'lane-worker.ts', 'hook-lane.ts']) walk(join(REPO_ROOT, 'src/core/skills/resolver', entry));
+ const local = [...reached].map(f => f.slice(REPO_ROOT.length + 1));
+ expect(local).toContain('src/core/skills/resolver/resolve.ts');
+ expect(local.filter(f => /resolver\/(build-thread|handle|build|index)\.ts$/.test(f))).toEqual([]);
+ });
+
test('the CLI module loads no engine, driver, operations table or provider', () => {
const loaded = loadedBy(`await import(${JSON.stringify(join(REPO_ROOT, 'src/commands/skill-resolver.ts'))})`);
expect(loaded).toContain('src/commands/skill-resolver.ts');
diff --git a/test/skill-resolver-packaging.test.ts b/test/skill-resolver-packaging.test.ts
index 0a0b462ead..ff8e59ad25 100644
--- a/test/skill-resolver-packaging.test.ts
+++ b/test/skill-resolver-packaging.test.ts
@@ -42,8 +42,9 @@ describe('skill resolver packaging', () => {
const out = join(dir, 'gbrain-skill-resolver.mjs');
const run = Bun.spawnSync([process.execPath, 'scripts/build-skill-resolver-bundle.ts', '--out', out, '--smoke'], { cwd: ROOT, stdout: 'pipe', stderr: 'pipe' });
expect(run.exitCode, run.stderr.toString()).toBe(0);
- const smoke = JSON.parse(run.stdout.toString().trim().split('\n').at(-1)!) as { node: string; gbrain_version: string; bundled_skills: number; top: string };
+ const smoke = JSON.parse(run.stdout.toString().trim().split('\n').at(-1)!) as { node: string; gbrain_version: string; bundled_skills: number; top: string; off_thread: string; warm_top: string };
expect(smoke.top).toBe('x:deploy');
+ expect([smoke.off_thread, smoke.warm_top]).toEqual(['w', 'x:invoice']);
expect(smoke.gbrain_version).toBe(VERSION);
expect(smoke.bundled_skills).toBeGreaterThan(20);
const sum = createHash('sha256').update(readFileSync(out)).digest('hex');
diff --git a/test/skill-resolver-runtime.test.ts b/test/skill-resolver-runtime.test.ts
new file mode 100644
index 0000000000..a1cc617cdc
--- /dev/null
+++ b/test/skill-resolver-runtime.test.ts
@@ -0,0 +1,219 @@
+/**
+ * GBRA-399 PR 2: the skill resolver library's runtime for long-lived hosts.
+ *
+ * 1. Protects: a warm handle (openCatalog) answers byte-identically to
+ * resolveSkills across turns, a rebuild and a removal, and keeps its
+ * receipt and index in memory between rebuilds; buildSkillIndexOffThread
+ * returns the in-thread receipt and its ResolverError; a committed turn
+ * collects expired session files at most once per interval; a stale
+ * turn never stamps an adapter-owned stdin catalog and never rewrites a
+ * stamp that is already there.
+ * 2. Fails when: the warm path forks from the cold one, a rebuild or a
+ * removal goes unseen, the worker loses an error's code or fix, session
+ * files pile up past the cap, or a skewed stdin catalog writes (and
+ * fsyncs) a stamp every turn.
+ * 3. Every test runs on a scratch GBRAIN_HOME with no brain and no database.
+ */
+import { afterAll, describe, expect, test } from 'bun:test';
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, utimesSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+ buildSkillIndex, buildSkillIndexOffThread, catalogStatus, gcSkillSessions, openCatalog, removeSkillIndex, resolveSkills, RESOLVER_FEATURES, ResolverError,
+ type ResolveResult, type TurnInput,
+} from '../src/core/skills/resolver/index.ts';
+import { catalogFile } from '../src/core/skills/resolver/paths.ts';
+import { GC_INTERVAL_MS, SESSION_TTL_MS, sessionGcSettled } from '../src/core/skills/resolver/session.ts';
+
+const scratch: string[] = [];
+afterAll(() => { for (const d of scratch) rmSync(d, { recursive: true, force: true }); });
+function tmp(prefix = 'skres-rt-'): string { const d = mkdtempSync(join(tmpdir(), prefix)); scratch.push(d); return d; }
+
+interface J { id: string; name: string; description?: string; triggers?: string[] }
+const BASIC: J[] = [
+ { id: 'gbrain:briefing', name: 'briefing', description: 'Prepare a daily briefing from calendar meetings and open notes' },
+ { id: 'gbrain:reports', name: 'reports', description: 'Write weekly status reports for a team' },
+ { id: 'gbrain:email-triage', name: 'email-triage', description: 'Sort an inbox and draft replies to urgent email' },
+ { id: 'gbrain:meeting-notes', name: 'meeting-notes', description: 'Summarize a meeting transcript into decisions and action items' },
+ { id: 'gbrain:deploy', name: 'deploy', description: 'Ship a service to production after the tests pass' },
+];
+const MORE: J[] = [
+ ...BASIC,
+ { id: 'gbrain:invoices', name: 'invoices', description: 'Draft invoices for acme-example and track payments', triggers: ['send an invoice'] },
+ { id: 'gbrain:standup', name: 'standup', description: 'Collect standup updates from the team channel', triggers: ['daily standup'] },
+];
+function skillDir(root: string, rel: string, name: string, description: string): string {
+ const d = join(root, rel);
+ mkdirSync(d, { recursive: true });
+ writeFileSync(join(d, 'SKILL.md'), `---\nname: ${name}\ndescription: ${JSON.stringify(description)}\n---\nSteps.\n`);
+ return join(d, 'SKILL.md');
+}
+function skewVersion(home: string, catalog: string): void {
+ const p = catalogFile(home, catalog, 'receipt.json');
+ writeFileSync(p, JSON.stringify({ ...JSON.parse(readFileSync(p, 'utf8')), gbrain_version: '0.0.1' }));
+}
+const comparable = (r: ResolveResult) => JSON.stringify({
+ output: { ...r.output, meta: { ...r.output.meta, timing_ms: 0 } },
+ shadow: r.shadowOutput ? { ...r.shadowOutput, meta: { ...r.shadowOutput.meta, timing_ms: 0 } } : null,
+ explain: r.explain ?? null, failOpen: r.failOpen ?? false, pending: r.pending ?? null,
+});
+
+describe('warm handle', () => {
+ test('openCatalog answers byte-identically to resolveSkills over many turns, a rebuild, a source edit and a removal', async () => {
+ const src = tmp('skres-rt-src-');
+ const json = join(src, 'a.json');
+ const root = join(src, 'skills');
+ skillDir(root, 'deploy', 'deploy', 'Ship a service to production');
+ skillDir(root, 'notes', 'notes', 'Keep meeting notes tidy');
+ const T0 = Date.parse('2026-11-01T00:00:00Z');
+ const homes = [tmp(), tmp()];
+ const buildAll = async (skills: J[]) => {
+ writeFileSync(json, JSON.stringify(skills));
+ for (const home of homes) await buildSkillIndex({ catalog: 'a', fromJson: json }, { home, now: () => T0 });
+ };
+ await buildAll(BASIC);
+ for (const home of homes) await buildSkillIndex({ catalog: 'b', dirs: [root] }, { home, now: () => T0 });
+ const [coldHome, warmHome] = homes as [string, string];
+ const handle = openCatalog(['a', 'b'], { home: warmHome, failOpen: true });
+ const messages = ['deploy the service', 'summarize the meeting notes', 'thanks', 'write weekly reports', 'qqq zzz', 'use $deploy please', 'send an invoice to acme-example', 'daily standup', 'triage my urgent email', 'prepare the daily briefing'];
+ let clock = Date.now();
+ let answered = 0;
+ let failed = 0;
+ const seen = new Set<string>();
+ for (let i = 0; i < 48; i++) {
+ if (i === 14) await buildAll(MORE);
+ if (i === 22) skillDir(root, 'standup', 'standup', 'Run the daily standup');
+ if (i === 30) for (const home of homes) await removeSkillIndex('a', { home });
+ if (i === 36) await buildAll(BASIC);
+ if (i % 4 === 0) clock += 120_000;
+ const input: TurnInput = {
+ message: messages[i % messages.length]!,
+ session_id: i % 3 === 0 ? 'alice-example' : 'acme-example',
+ ...(i % 5 === 0 ? { harness: 'codex' } : {}),
+ ...(i % 7 === 3 ? { shadow: true } : {}),
+ ...(i === 20 ? { reset_session: true } : {}),
+ ...(i % 6 === 1 ? { top_k: 2 } : {}),
+ ...(i % 9 === 4 ? { previous_user_message: 'deploy the service' } : {}),
+ };
+ const turn = { now: () => clock, ...(i % 8 === 2 ? { explainTarget: 'gbrain:deploy' } : {}) };
+ const cold = await resolveSkills(input, { home: coldHome, catalogs: ['a', 'b'], failOpen: true, ...turn });
+ const warm = await handle.resolve(input, turn);
+ expect({ turn: i, warm: comparable(warm) }).toEqual({ turn: i, warm: comparable(cold) });
+ expect(await warm.commit()).toBe(await cold.commit());
+ if (warm.output.error) failed++; else answered++;
+ if (warm.output.meta.index_stale) seen.add('stale');
+ for (const s of [...warm.output.suggestions, ...(warm.shadowOutput?.suggestions ?? [])]) seen.add(s.id);
+ }
+ expect(answered).toBeGreaterThan(30);
+ expect(failed).toBe(6);
+ expect([...seen].filter(x => x === 'stale' || x === 'gbrain:invoices').sort()).toEqual(['gbrain:invoices', 'stale']);
+ handle.close();
+ });
+
+ test('the handle keeps its receipt in memory until the catalog is rebuilt, then reloads it', async () => {
+ const home = tmp();
+ writeFileSync(join(home, 'cat.json'), JSON.stringify(BASIC));
+ await buildSkillIndex({ catalog: 'c', fromJson: join(home, 'cat.json') }, { home });
+ const rp = catalogFile(home, 'c', 'receipt.json');
+ const second = Math.floor(Date.now() / 1000) - 60;
+ utimesSync(rp, second, second);
+ const handle = openCatalog(['c'], { home });
+ expect((await handle.resolve({ message: 'deploy' })).output.suggestions[0]?.id).toBe('gbrain:deploy');
+ const before = statSync(rp);
+ writeFileSync(rp, readFileSync(rp, 'utf8').replace('"schema_version": 1', '"schema_version": 2'));
+ utimesSync(rp, second, second);
+ expect([statSync(rp).ino, statSync(rp).size, statSync(rp).mtimeMs]).toEqual([before.ino, before.size, before.mtimeMs]);
+ try { await resolveSkills({ message: 'deploy' }, { home }); throw new Error('expected a refusal'); } catch (e) { expect((e as ResolverError).code).toBe('skill_index_incompatible'); }
+ expect((await handle.resolve({ message: 'deploy' })).output.suggestions[0]?.id).toBe('gbrain:deploy');
+ writeFileSync(join(home, 'cat.json'), JSON.stringify(MORE));
+ await buildSkillIndex({ catalog: 'c', fromJson: join(home, 'cat.json') }, { home });
+ expect((await handle.resolve({ message: 'send an invoice' })).output.suggestions[0]?.id).toBe('gbrain:invoices');
+ await removeSkillIndex('c', { home });
+ try { await handle.resolve({ message: 'deploy' }); throw new Error('expected a refusal'); } catch (e) { expect((e as ResolverError).code).toBe('skill_index_missing'); }
+ handle.close();
+ expect(RESOLVER_FEATURES).toContain('open_catalog');
+ });
+});
+
+describe('off-thread build', () => {
+ test('buildSkillIndexOffThread publishes the receipt buildSkillIndex would, and a ResolverError keeps its code, message and fix', async () => {
+ const [h1, h2] = [tmp(), tmp()];
+ const input = { catalog: 'c', fromJson: JSON.stringify(MORE), fromJsonStdin: true };
+ const off = await buildSkillIndexOffThread(input, { home: h1 });
+ const on = await buildSkillIndex(input, { home: h2 });
+ const strip = (r: typeof on) => ({ ...r, build: { ...r.build, ms: 0, built_at: '' } });
+ expect(strip(off)).toEqual(strip(on));
+ expect(JSON.parse(readFileSync(catalogFile(h1, 'c', 'receipt.json'), 'utf8'))).toEqual(off);
+ expect((await resolveSkills({ message: 'send an invoice' }, { home: h1 })).output.suggestions[0]?.id).toBe('gbrain:invoices');
+ for (const bad of [{ catalog: '../escape', fromJson: '[]', fromJsonStdin: true }, { catalog: 'c', fromJson: '{not json', fromJsonStdin: true }]) {
+ const inThread = await buildSkillIndex(bad, { home: h2 }).then(() => null, (e: unknown) => e);
+ const offThread = await buildSkillIndexOffThread(bad, { home: h1 }).then(() => null, (e: unknown) => e);
+ expect(inThread).toBeInstanceOf(ResolverError);
+ expect(offThread).toBeInstanceOf(ResolverError);
+ const view = (e: ResolverError) => ({ code: e.code, message: e.message, fix: e.fix, context: e.context });
+ expect(view(offThread as ResolverError)).toEqual(view(inThread as ResolverError));
+ }
+ expect(RESOLVER_FEATURES).toContain('build_off_thread');
+ }, 60_000);
+});
+
+describe('session GC on the resolve path', () => {
+ test('a committed turn deletes expired session files at most once per interval and keeps fresh ones', async () => {
+ const home = tmp();
+ await buildSkillIndex({ catalog: 'c', fromJson: JSON.stringify(BASIC), fromJsonStdin: true }, { home });
+ const dir = join(home, 'skill-index', 'sessions');
+ mkdirSync(dir, { recursive: true });
+ const old = (Date.now() - SESSION_TTL_MS - 3_600_000) / 1000;
+ const plant = (name: string, expired: boolean) => { const p = join(dir, name); writeFileSync(p, '{}'); if (expired) utimesSync(p, old, old); return p; };
+ const expired = plant(`${'a'.repeat(64)}.json`, true);
+ const fresh = plant(`${'b'.repeat(64)}.json`, false);
+ const turn = async (id: string) => { const r = await resolveSkills({ message: 'deploy', session_id: id }, { home }); expect(await r.commit()).toBe(true); await sessionGcSettled(); };
+ await turn('alice-example');
+ expect(existsSync(expired)).toBe(false);
+ expect(existsSync(fresh)).toBe(true);
+ const later = plant(`${'c'.repeat(64)}.json`, true);
+ await turn('alice-example');
+ expect(existsSync(later)).toBe(true);
+ const past = (Date.now() - GC_INTERVAL_MS - 1000) / 1000;
+ utimesSync(join(home, 'skill-index', 'logs', 'session-gc.stamp'), past, past);
+ await turn('acme-example');
+ expect(existsSync(later)).toBe(false);
+ expect(existsSync(fresh)).toBe(true);
+ });
+
+ test('gcSkillSessions runs one bounded pass for an adapter', async () => {
+ const home = tmp();
+ const dir = join(home, 'skill-index', 'sessions');
+ mkdirSync(dir, { recursive: true });
+ const old = (Date.now() - SESSION_TTL_MS - 3_600_000) / 1000;
+ for (let i = 0; i < 3; i++) { const p = join(dir, `${String(i).repeat(64)}.json`); writeFileSync(p, '{}'); utimesSync(p, old, old); }
+ writeFileSync(join(dir, `${'f'.repeat(64)}.json`), '{}');
+ expect(await gcSkillSessions({ home })).toBe(3);
+ expect(await gcSkillSessions({ home: join(home, 'none') })).toBe(0);
+ expect(RESOLVER_FEATURES).toContain('gc_sessions');
+ });
+});
+
+describe('stale.stamp', () => {
+ test('a version skew never stamps an adapter-owned stdin catalog; a dir catalog is stamped once and not rewritten', async () => {
+ const home = tmp();
+ await buildSkillIndex({ catalog: 'j', fromJson: JSON.stringify(BASIC), fromJsonStdin: true }, { home });
+ skewVersion(home, 'j');
+ for (let i = 0; i < 2; i++) expect((await resolveSkills({ message: 'deploy' }, { home, catalogs: ['j'] })).output.meta.index_stale).toBe(true);
+ expect(existsSync(catalogFile(home, 'j', 'stale.stamp'))).toBe(false);
+ expect((await catalogStatus('j', { home })).stale_reason).toBe('gbrain_version_changed');
+ const root = tmp('skres-rt-dir-');
+ skillDir(root, 'deploy', 'deploy', 'Ship it');
+ await buildSkillIndex({ catalog: 'd', dirs: [root] }, { home });
+ skewVersion(home, 'd');
+ expect((await resolveSkills({ message: 'deploy' }, { home, catalogs: ['d'] })).output.meta.index_stale).toBe(true);
+ const stamp = catalogFile(home, 'd', 'stale.stamp');
+ expect(JSON.parse(readFileSync(stamp, 'utf8'))).toEqual({ reason: 'gbrain_version_changed' });
+ const hourAgo = (Date.now() - 3_600_000) / 1000;
+ utimesSync(stamp, hourAgo, hourAgo);
+ const pinned = statSync(stamp).mtimeMs;
+ expect((await resolveSkills({ message: 'deploy' }, { home, catalogs: ['d'] })).output.meta.index_stale).toBe(true);
+ expect(statSync(stamp).mtimeMs).toBe(pinned);
+ expect((await catalogStatus('d', { home })).rebuild.stale_stamp).toBe(true);
+ });
+});