Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.
TypeScript browser-shellcontrol-planefile-managerhomelabmobile-firstnextjsptyreactself-hostedserver-managementshadcn-uisysadmintailwindcssterminaltypescriptvpsweb-desktopweb-terminal
fix(mcp): lossless scan continuation and exact-id project resolution
rahmanef63dcae9b8a3bf7
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.mdindex adc94451..d4e5debe 100644--- a/docs/CHANGELOG.md+++ b/docs/CHANGELOG.md@@ -14,6 +14,7 @@ this is the *what*, and it is what Settings → About shows as “What's new”.**Fixed**+- `mcp` lossless scan continuation and exact-id project resolution- `mcp` one project validator, dirent budgets, resumable caps- `mcp` contain, bound and uniquely identify project discovery- `mcp` make ChatGPT file imports region agnosticdiff --git a/docs/CONNECTORS-GATEWAY-INTEGRATION.md b/docs/CONNECTORS-GATEWAY-INTEGRATION.mdindex 9a4817c9..04e3b496 100644--- a/docs/CONNECTORS-GATEWAY-INTEGRATION.md+++ b/docs/CONNECTORS-GATEWAY-INTEGRATION.md@@ -4,7 +4,7 @@## Cross-repo contract-MSO currently exposes **26 tool names** (server `1.5.2`, toolset `2026.08.20.5`). The gateway manifest still maps **15** of them through literal `x-upstream` strings:+MSO currently exposes **26 tool names** (server `1.5.3`, toolset `2026.08.20.6`). The gateway manifest still maps **15** of them through literal `x-upstream` strings:```json{ "id": "mso.fs.delete", "x-upstream": "fs_delete" }diff --git a/docs/MCP.md b/docs/MCP.mdindex cc69cb85..cf973dfd 100644--- a/docs/MCP.md+++ b/docs/MCP.md@@ -104,8 +104,8 @@ The catalog has a stable server version plus a schema-derived toolset signature.Settings → MCP shows the current version/hash/count and stores a browser-local acknowledgement when the operator marks ChatGPT refreshed. A later signature change becomes an explicit stale-snapshot warning. This does not mutate ChatGPT remotely; it makes the required refresh visible instead of relying on memory.-Current catalog: **26 tools** (14 read, 10 write, 2 exec), server `1.5.2` / toolset-`2026.08.20.5`. `projects_list`, `skills_list` and `skills_read` are the new public+Current catalog: **26 tools** (14 read, 10 write, 2 exec), server `1.5.3` / toolset+`2026.08.20.6`. `projects_list`, `skills_list` and `skills_read` are the new publicnames in this release; `image_generation_status` and `image_generate` were removed(see the migration note below).@@ -138,11 +138,19 @@ never reaches the `package.json` or `.git` readers at all. A path hint is validacomponent by component from the container down, because canonicalizing first and checkingafterwards is exactly what let a symlinked intermediate through.-A caller-supplied `rootHint` gets the same treatment: a symlinked or hidden root is-refused rather than canonicalized into something the caller never named. It is authorized-against *every* configured read root rather than the scan-capped subset — naming a root-never widens the jail, but the jail must not shrink because twelve other roots were-configured ahead of it.+A caller-supplied `rootHint` gets the same treatment: a symlinked root is refused rather+than canonicalized into something the caller never named, and **any** dot-prefixed+component below the authorized root is refused — measured relative to that root, so a+checkout legitimately living under `~/.claude/worktrees` still works while nothing may+hide beneath it. It is authorized against *every* configured read root rather than the+scan-capped subset: naming a root never widens the jail, but the jail must not shrink+because twelve other roots were configured ahead of it.++**An exact `<rootId>/<project-name>` is resolved first**, before alias, package or fuzzy+matching. The `rootId` maps to exactly one container across every configured root, the+project name is then checked by the shared validator, and an unknown `rootId` is refused+outright. Falling through to fuzzy matching is what made two same-named projects+unaddressable: asking for the second one's id returned the first.### Bounds, and telling the truth about them@@ -168,13 +176,28 @@ so an oversized `package.json`, `SKILL.md` or `packed-refs` costs one stat, notand a count of entries rejected by the containment/ownership checks. **Do not conclude aproject or skill is absent from a truncated scan** — the tool descriptions say so too.-**Every cap is resumable.** A truncated report carries `scan.continuation` with the-containers still pending, a per-container cursor, and an opaque `cursor` string to pass-back as the `cursor` argument to `projects_list` / `skills_list`. Cursors are positional-in readdir order (`cursorSemantics: "readdir-position"`) and are valid while the-directories are unchanged — making them name-ordered would require visiting every dirent-to find the next N names, which is the unbounded walk the entry cap exists to prevent.-Ordinary paging *within* one scan uses `offset`/`limit` with `hasMore`/`nextOffset`.+**Every cap is losslessly resumable.** Both walks are a single streaming pass: each dirent+is validated as it arrives and the recorded position advances **only after** that entry is+fully processed, so a cap or deadline that trips mid-entry re-reads it rather than skipping+it. A truncated report carries `scan.continuation` with the pending roots and an opaque+`cursor` to pass back.++The project position is `(rootIndex, containerIndex, entriesConsumed)`, where `rootIndex`+indexes the **uncapped** configured-root list — that offset is what lets `maxRoots` advance+at all; without it every call rebuilt the same capped prefix and a 13th configured root was+unreachable forever. The skill cursor records the roots that finished *cleanly*, the+projects whose every root finished cleanly, and the exact dirent position inside the one+root that was interrupted; a partially consumed project is re-listed and resumed, never+marked done.++Cursors are raw readdir stream positions (`cursorSemantics: "readdir-stream-position"`)+and are valid while the directories are unchanged — name-ordered cursors would require+visiting every dirent to find the next N names, which is the unbounded walk the entry cap+exists to prevent. Rows inside a returned page are sorted for presentation; *which* rows+land in a truncated page is stream order.++There are deliberately **two** paginations and a client needs both: `hasMore`/`nextOffset`+walks the rows of one scan, `scan.continuation.cursor` resumes the scan itself past a cap.### Ids, so nothing shadows anything@@ -184,7 +207,10 @@ of sha256 over the canonical container path. It was 32 bits, and a review foundcollision — `/tmp/mso-root-50323` and `/tmp/mso-root-125549` both hashed to `51e156ef` —which would have merged two roots' same-named projects back into one row. Belt andbraces, nothing dedupes on the hash: the internal key is the full canonical path, so even-a collision cannot merge two containers. That makes ids **globally unique**: two configured roots may each hold a+a collision cannot merge two containers. That exact pair is an end-to-end regression+(`lib/mcp/collision-e2e.test.ts`) asserting that `projects_list`, `skills_list`,+`skills_read`, `skills_search` and `workflow_start` each return the **second** project+when handed the second id. That makes ids **globally unique**: two configured roots may each hold a`widget` shipping `deploy`, and both stay visible, readable and searchable. The derived`projects/` container gets its own `rootId` for the same reason, so `~/widget` and`~/projects/widget` cannot collide either. Within one project, `.mso/skills` outranks thediff --git a/docs/PROGRESS.md b/docs/PROGRESS.mdindex 4fb9016f..c327667f 100644--- a/docs/PROGRESS.md+++ b/docs/PROGRESS.md@@ -8,6 +8,60 @@ Running log of what shipped each phase. Newest at top.> Read those phases as history. **This file is the source of truth for what exists** —> `ARCHITECTURE.md` is no longer maintained and carries a stale-warning banner.+## 2026-08-20 — lossless continuation and exact-id project resolution (DONE)++The final fail-closed review found six remaining items. Five were continuation bugs of the+same species — a cursor that *described* where a scan stopped instead of *being* a position+you could resume from.++**Hidden `rootHint` still resolved.** `validateRootHint` checked the final entry for+symlink, shape, uid and credentials but never for a dot-prefixed component, so supplying+`<authorized-root>/.hidden` as the root resolved a project underneath it by exact name+while enumeration refused the same tree. The hidden check is now measured *relative to the+authorized root*: the root's own path may contain dot components (a checkout under+`~/.claude/worktrees` does), but nothing below it may.++**`maxRoots` could not advance, ever.** `listProjectDirs()` rebuilt the same capped+`projectContainers()` on every call, so a 13th configured root stayed pending no matter how+many times a client followed the cursor. Root identity is now an index into the *uncapped*+configured list, `authorizedRoots(startIndex)` slides that window forward, and the+continuation carries the index of the first root the scan could not honour.++**`maxProjects` and `maxProjectSkills` lost entries.** Both derived a readdir position from+sorted accepted rows and a global result count — arithmetic that cannot be truthful, and+which emitted `entriesConsumed=400` for a root holding 250 dirents. Both walks are now a+single streaming pass: each dirent is validated as it arrives, every cap is checked+*before* the entry is touched, and the recorded position advances *only after* the entry is+fully handled. The skill cursor separately records roots that finished cleanly, projects+whose every root finished cleanly, and the exact position inside the one interrupted root —+a partially consumed project is re-listed and resumed rather than marked done.++**Deadlines skipped unprocessed entries.** Both scanners read every name, sorted, then+validated; a deadline expiring during validation left the position past names nothing had+looked at. Same streaming fix — the cursor cannot outrun the work.++**Exact project ids were unusable through `workflow_start`.** `resolveProjectHint` had no+root-qualified-id branch, so `<rootId>/<name>` fell through to fuzzy matching and, with two+same-named projects, returned the *wrong* one. An exact `<32-hex>/<name>` is now parsed and+resolved before alias, package or fuzzy; the rootId maps to exactly one container across+every configured root; an unknown rootId is refused rather than guessed.++The colliding pair from the previous review (`/tmp/mso-root-50323` and+`/tmp/mso-root-125549`, both `51e156ef` at 8 hex) is now an end-to-end regression:+`projects_list`, `skills_list`, `skills_read`, `skills_search` and `workflow_start` each+return the **second** project when handed the second id. Continuation reproducers drain+13 roots, 2×250 projects, a 437-entry root and a 150+160-skill pair to completion and+assert no row is dropped or repeated. Note that there are deliberately two paginations —+`hasMore`/`nextOffset` within one scan, `scan.continuation.cursor` across scans — and the+docs now say so, because conflating them looks exactly like data loss.++The positive suites are unchanged and green: all-agents-all-tools, the exec/read/write+ladder, image generation removed everywhere, `fs_upload_file` with regional ChatGPT import,+and auth/OAuth untouched. `project-identity.ts`, `project-authorized-roots.ts` and+`catalog-cursor.ts` split the work so every file stays under 200 lines and the value-cycle+count stays at zero. The MCP server/toolset advance to `1.5.3` / `2026.08.20.6`; the catalog+stays at **26 tools** (14 read, 10 write, 2 exec).+## 2026-08-20 — one validator, dirent-counted budgets, resumable caps (DONE)A second fail-closed re-review of the discovery hardening found six things still open.diff --git a/lib/host/project-authorized-roots.ts b/lib/host/project-authorized-roots.tsnew file mode 100644index 00000000..aa225c87--- /dev/null+++ b/lib/host/project-authorized-roots.ts@@ -0,0 +1,60 @@+// WHICH configured roots a scan may touch, and where a continuation picks them back up.+//+// Two lists on purpose. Authorization is uncapped — a root the owner configured is inside+// the jail whether or not this call had budget to walk it. A SCAN is capped, and the cap+// is a sliding window over that list, which is the only thing that makes `maxRoots`+// continuable at all.+import { promises as fs } from "fs";+import { isCredentialPath, readRootList } from "./paths";+import { PROJECT_LIMITS, shortId, type AuthorizedRoot } from "./project-identity";++export async function allConfiguredRoots(): Promise<AuthorizedRoot[]> {+ const out: AuthorizedRoot[] = [];+ const seen = new Set<string>();+ for (const configured of readRootList()) {+ const real = await fs.realpath(configured).catch(() => null);+ if (!real || real === "/" || seen.has(real)) continue;+ if (!(await fs.stat(real).catch(() => null))?.isDirectory()) continue;+ if (isCredentialPath(real)) continue;+ seen.add(real);+ out.push({ id: shortId(real), configured, path: real, index: out.length });+ }+ return out;+}++/**+ * The roots ONE scan will honour: `maxRoots` of them, starting at `startIndex`.+ *+ * The start offset is what makes `maxRoots` continuable. Without it the second call+ * rebuilt the identical capped prefix and the 13th configured root was unreachable+ * forever, however many times the caller followed the cursor.+ */+export async function authorizedRoots(startIndex = 0): Promise<AuthorizedRoot[]> {+ return (await allConfiguredRoots()).slice(startIndex, startIndex + PROJECT_LIMITS.maxRoots);+}++/**+ * EVERY configured root, canonicalized, with NO cap.+ *+ * `authorizedRoots()` caps at `maxRoots` because a SCAN has to be bounded. Authorization+ * is a different question: a root the owner configured is inside the jail whether or not+ * this call had budget to walk it. Using the capped list as the jail predicate is what+ * made an explicitly named `rootHint` unresolvable once 12 other roots were configured.+ */+export async function configuredRootPaths(): Promise<string[]> {+ return (await allConfiguredRoots()).map((r) => r.path);+}++/** Roots configured but NOT honoured, so the caller can say so rather than imply the+ * configuration was fully covered. */+export async function overflowRoots(startIndex = 0): Promise<{ roots: Array<{ path: string; reason: string }>; nextIndex: number }> {+ const all = await allConfiguredRoots();+ const end = startIndex + PROJECT_LIMITS.maxRoots;+ const remaining = all.slice(end);+ return {+ roots: remaining.map((r) => ({ path: r.path, reason: "maxRoots" })),+ // Where a continuation must pick the root list back up.+ nextIndex: remaining.length ? end : all.length,+ };+}+diff --git a/lib/host/project-candidate.ts b/lib/host/project-candidate.tsindex 4bfcd05b..3d88fe84 100644--- a/lib/host/project-candidate.ts+++ b/lib/host/project-candidate.ts@@ -73,10 +73,21 @@ export async function validateProjectDescendant(container: ProjectContainer, tarreturn { ok: true, path: parent };}-/** The same rules applied to a caller-supplied ROOT directory, before it is trusted as- * a container. A symlinked or hidden `rootHint` is refused rather than canonicalized- * into something the caller never named. */-export async function validateRootHint(absolute: string): Promise<CandidateResult> {+/**+ * The same rules applied to a caller-supplied ROOT directory, before it is trusted as a+ * container. A symlinked or hidden `rootHint` is refused rather than canonicalized into+ * something the caller never named.+ *+ * `authorizedRoot` is required for the hidden check, and the relative calculation is the+ * whole point: the authorized root's OWN path may legitimately contain dot components+ * (a checkout under `~/.claude/worktrees`, say), but nothing BELOW it may. Checking only+ * the final component let `<authorized-root>/.hidden/widget` resolve by exact name while+ * enumeration refused the same directory.+ */+export async function validateRootHint(absolute: string, authorizedRoot: string): Promise<CandidateResult> {+ if (!isUnderRoot(absolute, authorizedRoot)) return { ok: false, reason: "escape" };+ const relative = path.relative(authorizedRoot, absolute);+ if (relative && relative.split(path.sep).some((segment) => segment.startsWith("."))) return { ok: false, reason: "hidden" };const stat = await fs.lstat(absolute).catch(() => null);if (!stat) return { ok: false, reason: "missing" };if (stat.isSymbolicLink()) return { ok: false, reason: "symlink" };diff --git a/lib/host/project-containers.ts b/lib/host/project-containers.tsindex 5e5e64cf..e602fdea 100644--- a/lib/host/project-containers.ts+++ b/lib/host/project-containers.ts@@ -20,28 +20,22 @@// Everything the scan omits is REPORTED. `truncated:false` means "this is all of it";// hitting a root, entry, project or time cap sets `truncated:true` with a named// reason. A silent slice that claims completeness is worse than a refusal.-import { createHash } from "crypto";import { promises as fs } from "fs";import path from "path";-import { isCredentialPath, isUnderRoot, readRootList } from "./paths";-import { boundedGitMeta, packageMeta } from "./project-meta";--export const PROJECT_LIMITS = {- /** Authorized OS_FS_READ_ROOTS entries honoured per call. */- maxRoots: 12,- /** Directory entries READ from one container before the walk stops. */- maxEntriesPerRoot: 400,- /** Projects enumerated in total, across all containers. */- maxProjects: 400,- /** Wall-clock ceiling for one whole enumeration. */- maxScanMs: 4000,- defaultPageSize: 50,- maxPageSize: 200,-} as const;--export type AuthorizedRoot = { id: string; configured: string; path: string };+import { isCredentialPath, isUnderRoot } from "./paths";+import { PROJECT_LIMITS, shortId, type AuthorizedRoot } from "./project-identity";+import { allConfiguredRoots, authorizedRoots } from "./project-authorized-roots";++export { allConfiguredRoots, authorizedRoots };+export { PROJECT_LIMITS, shortId, containerKey } from "./project-identity";+export type { AuthorizedRoot } from "./project-identity";++++export type { ScanCursor, ScanReport } from "./project-scan-types";+export { configuredRootPaths, overflowRoots } from "./project-authorized-roots";export type ProjectContainer = {/** Short sha256 of the canonical container path. The DERIVED `projects/` child gets@@ -52,6 +46,9 @@ export type ProjectContainer = {authorizedRootId: string;authorizedRoot: string;derived: boolean;+ /** (rootIndex, containerIndex) is this container's exact address for a cursor. */+ rootIndex: number;+ containerIndex: number;};export type ProjectRow = {@@ -68,22 +65,7 @@ export type ProjectRow = {git?: { branch?: string; head?: string };};-/**- * 128 bits of sha256 over the canonical container path.- *- * This was 8 hex characters — 32 bits — and a review found a REAL collision in the- * fixture space it was tested in: `/tmp/mso-root-50323` and `/tmp/mso-root-125549`- * both hashed to `51e156ef`. Two colliding roots holding same-named projects would- * have merged back into one row, re-creating the exact bug root-qualified ids exist to- * fix. 32 hex characters make that computationally unreachable — and, belt and braces,- * nothing DEDUPES on this value: the internal key is the full canonical path (see- * `containerKey`), so even a collision cannot merge two containers.- */-export const shortId = (real: string) => createHash("sha256").update(real).digest("hex").slice(0, 32);-/** The internal identity. Always the full canonical path — never the hash — so- * dedupe/precedence can never be decided by a truncated digest. */-export const containerKey = (real: string) => real;const currentUid = (): number | undefined => (typeof process.getuid === "function" ? process.getuid() : undefined);@@ -101,48 +83,6 @@ export async function ownedByUs(target: string): Promise<boolean> {* Following a symlinked CONFIGURED root is intended — its realpath simply becomes the* authorized real root, and everything else is measured against that.*/-export async function authorizedRoots(): Promise<AuthorizedRoot[]> {- const out: AuthorizedRoot[] = [];- const seen = new Set<string>();- for (const configured of readRootList()) {- if (out.length >= PROJECT_LIMITS.maxRoots) break;- const real = await fs.realpath(configured).catch(() => null);- if (!real || real === "/" || seen.has(real)) continue;- if (!(await fs.stat(real).catch(() => null))?.isDirectory()) continue;- if (isCredentialPath(real)) continue;- seen.add(real);- out.push({ id: shortId(real), configured, path: real });- }- return out;-}--/**- * EVERY configured root, canonicalized, with NO cap.- *- * `authorizedRoots()` caps at `maxRoots` because a SCAN has to be bounded. Authorization- * is a different question: a root the owner configured is inside the jail whether or not- * this call had budget to walk it. Using the capped list as the jail predicate is what- * made an explicitly named `rootHint` unresolvable once 12 other roots were configured.- */-export async function configuredRootPaths(): Promise<string[]> {- const out: string[] = [];- for (const configured of readRootList()) {- const real = await fs.realpath(configured).catch(() => null);- if (!real || real === "/" || out.includes(real)) continue;- if (!(await fs.stat(real).catch(() => null))?.isDirectory()) continue;- if (isCredentialPath(real)) continue;- out.push(real);- }- return out;-}--/** Roots configured but NOT honoured, so the caller can say so rather than imply the- * configuration was fully covered. */-export async function overflowRoots(): Promise<Array<{ path: string; reason: string }>> {- const honoured = new Set((await authorizedRoots()).map((r) => r.configured));- return readRootList().filter((c) => !honoured.has(c)).map((path) => ({ path, reason: "maxRoots-or-unreadable" }));-}-/** `<root>/projects`, only when it is a real, non-symlink directory contained in that* same authorized root. */async function derivedContainerPath(root: AuthorizedRoot): Promise<string | null> {@@ -156,19 +96,43 @@ async function derivedContainerPath(root: AuthorizedRoot): Promise<string | null/** Every container, deterministically: configured root order, each root followed by* its derived `projects/` child, deduped by canonical path. */-export async function projectContainers(): Promise<ProjectContainer[]> {+export async function projectContainers(startIndex = 0): Promise<ProjectContainer[]> {const out: ProjectContainer[] = [];const seen = new Set<string>();- for (const root of await authorizedRoots()) {+ for (const root of await authorizedRoots(startIndex)) {+ let containerIndex = 0;for (const [candidate, derived] of [[root.path, false], [await derivedContainerPath(root), true]] as const) {- if (!candidate || seen.has(candidate)) continue;+ if (!candidate || seen.has(candidate)) { containerIndex += 1; continue; }seen.add(candidate);- out.push({ id: shortId(candidate), path: candidate, authorizedRootId: root.id, authorizedRoot: root.path, derived });+ out.push({+ id: shortId(candidate), path: candidate, authorizedRootId: root.id, authorizedRoot: root.path,+ derived, rootIndex: root.index, containerIndex,+ });+ containerIndex += 1;}}return out;}+/** Resolve a 32-hex rootId to its container, across EVERY configured root — not just the+ * scan-capped window. An exact id must address the same project on every call. */+export async function containerById(rootId: string): Promise<ProjectContainer | null> {+ if (!/^[a-f0-9]{32}$/.test(rootId)) return null;+ for (const root of await allConfiguredRoots()) {+ let containerIndex = 0;+ for (const [candidate, derived] of [[root.path, false], [await derivedContainerPath(root), true]] as const) {+ if (candidate && shortId(candidate) === rootId) {+ return {+ id: rootId, path: candidate, authorizedRootId: root.id, authorizedRoot: root.path,+ derived, rootIndex: root.index, containerIndex,+ };+ }+ containerIndex += 1;+ }+ }+ return null;+}+/** Back-compat view: just the container paths. */export async function projectRoots(): Promise<string[]> {return (await projectContainers()).map((c) => c.path);@@ -179,6 +143,9 @@ export async function projectRoots(): Promise<string[]> {* when the global container list is already at its cap. Same id derivation, same* containment rules — only the discovery path differs. */export function containerFor(realDir: string): ProjectContainer {- return { id: shortId(realDir), path: realDir, authorizedRootId: shortId(realDir), authorizedRoot: realDir, derived: false };+ return {+ id: shortId(realDir), path: realDir, authorizedRootId: shortId(realDir), authorizedRoot: realDir,+ derived: false, rootIndex: 0, containerIndex: 0,+ };}diff --git a/lib/host/project-continuation.test.ts b/lib/host/project-continuation.test.tsnew file mode 100644index 00000000..c4974883--- /dev/null+++ b/lib/host/project-continuation.test.ts@@ -0,0 +1,126 @@+import { afterAll, describe, expect, it } from "vitest";+import { promises as fs } from "fs";+import os from "os";+import path from "path";++// CONTINUATION MUST BE LOSSLESS. A truncated scan that cannot be resumed to completion is+// data loss wearing a label, and every one of these cases previously returned zero rows+// on the second call while insisting there was more.+const base = await fs.mkdtemp(path.join(os.tmpdir(), "mso-cont-"));+const previous = process.env.OS_FS_READ_ROOTS;+const { listProjects, PROJECT_LIMITS } = await import("./project-roots");+const setRoots = (...roots: string[]) => { process.env.OS_FS_READ_ROOTS = roots.join(":"); };++afterAll(async () => {+ if (previous === undefined) delete process.env.OS_FS_READ_ROOTS;+ else process.env.OS_FS_READ_ROOTS = previous;+ await fs.rm(base, { recursive: true, force: true });+});++/**+ * Drain everything. There are deliberately TWO paginations and a caller has to use both:+ * `hasMore`/`nextOffset` walks the rows of ONE scan, `scan.continuation.cursor` resumes+ * the scan itself past a cap. Conflating them is how you conclude data is missing when+ * it is merely on the next page.+ */+async function drain(cursor?: string) {+ const seen = new Set<string>();+ for (let scan = 0; scan < 40; scan += 1) {+ let page = await listProjects({ limit: PROJECT_LIMITS.maxPageSize, cursor });+ for (const p of page.projects) seen.add(p.path);+ while (page.hasMore) {+ page = await listProjects({ limit: PROJECT_LIMITS.maxPageSize, offset: page.nextOffset, cursor });+ for (const p of page.projects) seen.add(p.path);+ }+ if (!page.scan.truncated || !page.scan.continuation?.cursor) break;+ if (page.scan.continuation.cursor === cursor) break; // no forward progress+ cursor = page.scan.continuation.cursor;+ }+ return seen;+}++describe("maxRoots continuation advances past the capped prefix", () => {+ it("reaches projects in the 13th configured root", async () => {+ const roots: string[] = [];+ for (let i = 0; i <= PROJECT_LIMITS.maxRoots; i += 1) {+ const dir = path.join(base, `mr-${String(i).padStart(2, "0")}`);+ await fs.mkdir(path.join(dir, `p${i}`), { recursive: true });+ roots.push(dir);+ }+ setRoots(...roots);++ const first = await listProjects({ limit: PROJECT_LIMITS.maxPageSize });+ expect(first.scan.truncated).toBe(true);+ expect(first.scan.truncationReasons).toContain("maxRoots");++ const all = await drain();+ // The last root's project must be reachable, not merely reported as pending.+ expect([...all].some((p) => p.endsWith(`mr-${PROJECT_LIMITS.maxRoots}/p${PROJECT_LIMITS.maxRoots}`))).toBe(true);+ expect(all.size).toBe(PROJECT_LIMITS.maxRoots + 1);+ });+});++describe("maxProjects continuation keeps the exact raw dirent position", () => {+ it("returns the remaining projects instead of zero", async () => {+ // Two roots of 250 each: the cap trips mid-way through the SECOND root, which is+ // exactly where a cursor derived from a global accepted-count went wrong.+ const a = path.join(base, "mp-a");+ const b = path.join(base, "mp-b");+ const per = 250;+ await Promise.all([a, b].flatMap((root) =>+ Array.from({ length: per }, (_, i) => fs.mkdir(path.join(root, `p${String(i).padStart(4, "0")}`), { recursive: true }))));+ setRoots(a, b);++ const first = await listProjects({ limit: PROJECT_LIMITS.maxPageSize });+ expect(first.scan.truncated).toBe(true);+ expect(first.scan.truncationReasons).toContain("maxProjects");++ const all = await drain();+ expect(all.size).toBe(per * 2);+ });+});++describe("maxEntriesPerRoot continuation resumes at the exact position", () => {+ it("returns every project across repeated pages", async () => {+ const root = path.join(base, "me-root");+ const count = PROJECT_LIMITS.maxEntriesPerRoot + 37;+ await Promise.all(Array.from({ length: count }, (_, i) =>+ fs.mkdir(path.join(root, `p${String(i).padStart(4, "0")}`), { recursive: true })));+ setRoots(root);++ const first = await listProjects({ limit: PROJECT_LIMITS.maxPageSize });+ expect(first.scan.truncated).toBe(true);+ const all = await drain();+ expect(all.size).toBe(count);+ });++ it("never re-reports a project already returned by an earlier page", async () => {+ const root = path.join(base, "me-dup");+ const count = PROJECT_LIMITS.maxEntriesPerRoot + 5;+ await Promise.all(Array.from({ length: count }, (_, i) =>+ fs.mkdir(path.join(root, `p${String(i).padStart(4, "0")}`), { recursive: true })));+ setRoots(root);++ // One full scan (both its pages), then the resumed scan: no row may appear twice.+ const firstA = await listProjects({ limit: PROJECT_LIMITS.maxPageSize });+ const firstB = await listProjects({ limit: PROJECT_LIMITS.maxPageSize, offset: firstA.nextOffset });+ const firstPaths = new Set([...firstA.projects, ...firstB.projects].map((p) => p.path));+ expect(firstPaths.size).toBe(PROJECT_LIMITS.maxEntriesPerRoot);++ const second = await listProjects({ limit: PROJECT_LIMITS.maxPageSize, cursor: firstA.scan.continuation!.cursor });+ expect(second.projects.every((p) => !firstPaths.has(p.path))).toBe(true);+ expect(firstPaths.size + second.projects.length).toBe(count);+ });+});++describe("a complete scan still reports no continuation", () => {+ it("stays truncated=false with the cursor absent", async () => {+ const root = path.join(base, "small");+ await fs.mkdir(path.join(root, "one"), { recursive: true });+ setRoots(root);+ const { scan, projects } = await listProjects();+ expect(scan.truncated).toBe(false);+ expect(scan.continuation).toBeUndefined();+ expect(projects.map((p) => p.name)).toEqual(["one"]);+ });+});diff --git a/lib/host/project-cursor.ts b/lib/host/project-cursor.tsindex 7c93a907..f6c8ddbe 100644--- a/lib/host/project-cursor.ts+++ b/lib/host/project-cursor.ts@@ -4,10 +4,11 @@// way to fetch the rest is data loss with a label on it.import type { ScanCursor } from "./project-containers";-/** Cursors are positional in readdir order. That is a deliberate trade: making them- * name-ordered would require visiting every dirent to find the next N names, which is- * the unbounded walk the entry cap exists to prevent. Valid while the directory is- * unchanged; `note` says so in the payload. */+/** Cursors are raw readdir STREAM positions. That is a deliberate trade: a name-ordered+ * cursor would require visiting every dirent to find the next N names, which is the+ * unbounded walk the entry cap exists to prevent. Rows within a returned page are+ * sorted for presentation; WHICH rows land in a truncated page is stream order. Valid+ * while the directory is unchanged; `note` says so in the payload. */export function encodeCursor(cursor: ScanCursor): string {return Buffer.from(JSON.stringify(cursor), "utf8").toString("base64url");}@@ -17,9 +18,12 @@ export function decodeCursor(raw: string | undefined): ScanCursor | undefined {try {const parsed = JSON.parse(Buffer.from(raw, "base64url").toString("utf8")) as ScanCursor;if (!parsed || typeof parsed !== "object") return undefined;+ const int = (v: unknown) => (Number.isFinite(v) ? Math.max(0, Math.floor(v as number)) : 0);return {- roots: Array.isArray(parsed.roots) ? parsed.roots.filter((r) => typeof r?.root === "string") : [],- skipRoots: Array.isArray(parsed.skipRoots) ? parsed.skipRoots.filter((r) => typeof r === "string") : [],+ rootIndex: int(parsed.rootIndex),+ containerIndex: int(parsed.containerIndex),+ entriesConsumed: int(parsed.entriesConsumed),+ ...(typeof parsed.containerPath === "string" ? { containerPath: parsed.containerPath } : {}),};} catch {return undefined;diff --git a/lib/host/project-hint.test.ts b/lib/host/project-hint.test.tsindex 23e99147..f69867ab 100644--- a/lib/host/project-hint.test.ts+++ b/lib/host/project-hint.test.ts@@ -19,6 +19,9 @@ await fs.mkdir(path.join(named, "orchard"), { recursive: true });await fs.writeFile(path.join(named, "orchard", "package.json"), JSON.stringify({ name: "@farm/orchard" }));await fs.mkdir(path.join(named, "greenhouse-annex"), { recursive: true });await fs.mkdir(path.join(named, ".hidden-proj"), { recursive: true });+// A real project sitting inside a HIDDEN directory. Supplying that hidden directory as+// the rootHint used to resolve it by exact name, while enumeration refused the same tree.+await fs.mkdir(path.join(named, ".hidden-root", "widget"), { recursive: true });await fs.mkdir(outside, { recursive: true });await fs.mkdir(path.join(outside, "elsewhere"), { recursive: true });await fs.symlink(path.join(outside, "elsewhere"), path.join(named, "linked-proj"));@@ -89,6 +92,12 @@ describe("the exact-name probe rejects what enumeration excludes", () => {await expect(resolveProjectHint(".hidden-proj", named)).resolves.toBeNull();});+ it("refuses a rootHint that is itself inside a HIDDEN directory", async () => {+ // The hidden check is measured relative to the authorized root, so a legitimately+ // dot-bearing authorized root still works while nothing below it may hide.+ await expect(resolveProjectHint("widget", path.join(named, ".hidden-root"))).resolves.toBeNull();+ });+it("refuses a name that resolves outside the root it was probed in", async () => {// `../outside/elsewhere` must not traverse out of the named root.await expect(resolveProjectHint("../outside/elsewhere", named)).resolves.toBeNull();diff --git a/lib/host/project-identity.ts b/lib/host/project-identity.tsnew file mode 100644index 00000000..8e8a092d--- /dev/null+++ b/lib/host/project-identity.ts@@ -0,0 +1,42 @@+// The leaf: scan budgets and container identity. Nothing here imports anything of ours,+// so both the root list and the container builder can depend on it without a cycle.+import { createHash } from "crypto";++export const PROJECT_LIMITS = {+ /** Authorized OS_FS_READ_ROOTS entries honoured per call. */+ maxRoots: 12,+ /** Directory entries READ from one container before the walk stops. */+ maxEntriesPerRoot: 400,+ /** Projects enumerated in total, across all containers. */+ maxProjects: 400,+ /** Wall-clock ceiling for one whole enumeration. */+ maxScanMs: 4000,+ defaultPageSize: 50,+ maxPageSize: 200,+} as const;++export type AuthorizedRoot = {+ id: string;+ configured: string;+ path: string;+ /** Position in the UNCAPPED configured list. A cursor stores this so a later call can+ * slide the `maxRoots` window forward instead of re-scanning the same prefix. */+ index: number;+};++/**+ * 128 bits of sha256 over the canonical container path.+ *+ * This was 8 hex characters — 32 bits — and a review found a REAL collision in the+ * fixture space it was tested in: `/tmp/mso-root-50323` and `/tmp/mso-root-125549`+ * both hashed to `51e156ef`. Two colliding roots holding same-named projects would+ * have merged back into one row, re-creating the exact bug root-qualified ids exist to+ * fix. 32 hex characters make that computationally unreachable — and, belt and braces,+ * nothing DEDUPES on this value: the internal key is the full canonical path (see+ * `containerKey`), so even a collision cannot merge two containers.+ */+export const shortId = (real: string) => createHash("sha256").update(real).digest("hex").slice(0, 32);++/** The internal identity. Always the full canonical path — never the hash — so+ * dedupe/precedence can never be decided by a truncated digest. */+export const containerKey = (real: string) => real;diff --git a/lib/host/project-roots.ts b/lib/host/project-roots.tsindex dac65495..60578453 100644--- a/lib/host/project-roots.ts+++ b/lib/host/project-roots.ts@@ -1,77 +1,91 @@// The project WALK: bounded enumeration of the containers project-containers.ts-// authorized, plus the truthful, RESUMABLE report of everything it could not cover.+// authorized, plus a truthful, LOSSLESSLY RESUMABLE report of everything it could not+// cover.//-// Three rules earned the hard way:-// 1. `truncated:false` means "this is all of it". Any cap sets `truncated:true` with-// a named reason — a silent slice that claims completeness is how a model ends up+// Four rules earned the hard way:+// 1. `truncated:false` means "this is all of it". Any cap sets `truncated:true` with a+// named reason — a silent slice that claims completeness is how a model ends up// telling the owner a project does not exist.// 2. EVERY dirent counts against the entry cap, accepted or not. Counting only the-// entries we kept meant a container of a million regular files still cost a-// million iterations before the "400 entry" cap was reached.-// 3. A cap is only honest if it can be continued. Every cap emits a cursor, and-// passing it back resumes where the walk stopped.+// entries we kept meant a container of a million regular files still cost a million+// iterations before the "400 entry" cap was reached.+// 3. ONE STREAMING PASS. Entries are validated as they arrive and the position advances+// only AFTER an entry is fully processed. The previous version read every name,+// sorted them, then validated — so a cap or deadline that tripped during validation+// left the cursor pointing past names nothing had looked at, and they were lost.+// 4. Caps are addressed by (rootIndex, containerIndex, entriesConsumed), where+// rootIndex indexes the UNCAPPED configured-root list. Without that, `maxRoots`+// could never advance: every call rebuilt the same capped prefix.import { promises as fs } from "fs";import path from "path";import { validateProjectChild } from "./project-candidate";import { decodeCursor, encodeCursor } from "./project-cursor";import {- overflowRoots, projectContainers,- PROJECT_LIMITS, type ProjectContainer, type ProjectRow, type ScanReport, type ScanCursor,+ allConfiguredRoots, overflowRoots, projectContainers,+ PROJECT_LIMITS, type ProjectContainer, type ScanCursor, type ScanReport,} from "./project-containers";export {- authorizedRoots, configuredRootPaths, containerFor, projectContainers, projectRoots, shortId, PROJECT_LIMITS,+ allConfiguredRoots, authorizedRoots, configuredRootPaths, containerById, containerFor,+ projectContainers, projectRoots, shortId, PROJECT_LIMITS,} from "./project-containers";export type { AuthorizedRoot, ProjectContainer, ProjectRow, ScanReport, ScanCursor } from "./project-containers";export { decodeCursor, encodeCursor } from "./project-cursor";-type Walk = { dirs: string[]; entriesVisited: number; hitEntryCap: boolean; skipped: number };+type StopReason = "maxEntriesPerRoot" | "maxProjects" | "deadline";+type Walk = {+ dirs: string[];+ /** Raw dirents FULLY PROCESSED. Never includes one a cap stopped us before validating. */+ consumed: number;+ stop?: StopReason;+ skipped: number;+};/**- * Bounded `opendir` iteration. Every dirent yielded increments the budget, so a- * container full of rejected entries costs the cap and no more. `skipEntries` resumes a- * previously truncated walk at the same readdir position.+ * One streaming `opendir` pass. `skipEntries` fast-forwards to a recorded position, and+ * `budget` is how many more projects the whole scan may still accept.*/-async function walkContainer(container: ProjectContainer, containerPaths: Set<string>, skipEntries: number, deadlineAt: number): Promise<Walk> {- const names: string[] = [];- let entriesVisited = skipEntries;+async function walkContainer(+ container: ProjectContainer,+ containerPaths: Set<string>,+ skipEntries: number,+ deadlineAt: number,+ budget: number,+): Promise<Walk> {+ const dirs: string[] = [];+ let consumed = skipEntries;let seen = 0;- let hitEntryCap = false;+ let processed = 0;let skipped = 0;+ let stop: StopReason | undefined;const handle = await fs.opendir(container.path).catch(() => null);- if (!handle) return { dirs: [], entriesVisited, hitEntryCap: false, skipped: 0 };+ if (!handle) return { dirs, consumed, skipped };try {for await (const entry of handle) {seen += 1;if (seen <= skipEntries) continue;- // EVERY dirent costs budget — that is the whole point of counting here rather- // than after the accept/reject decision.- if (entriesVisited - skipEntries >= PROJECT_LIMITS.maxEntriesPerRoot || Date.now() > deadlineAt) {- hitEntryCap = true;- break;+ // Every check happens BEFORE the entry is processed, so stopping here leaves+ // `consumed` on the last fully-handled dirent and this one is re-read next time.+ if (processed >= PROJECT_LIMITS.maxEntriesPerRoot) { stop = "maxEntriesPerRoot"; break; }+ if (Date.now() > deadlineAt) { stop = "deadline"; break; }+ if (dirs.length >= budget) { stop = "maxProjects"; break; }++ processed += 1;+ const full = path.join(container.path, entry.name);+ // `~/projects` under `~` is a CONTAINER, not a project inside its parent.+ if (containerPaths.has(full)) skipped += 1;+ else if (!entry.isDirectory() || entry.name.startsWith(".")) skipped += 1;+ else {+ const candidate = await validateProjectChild(container, entry.name);+ if (candidate.ok) dirs.push(candidate.path);+ else skipped += 1;}- entriesVisited += 1;- if (!entry.isDirectory() || entry.name.startsWith(".")) { skipped += 1; continue; }- names.push(entry.name);+ consumed = seen; // ONLY now — the entry is fully handled.}} catch {- // A directory that vanishes mid-walk yields what we already have.- }- names.sort((a, b) => a.localeCompare(b));-- const dirs: string[] = [];- for (const name of names) {- // The deadline is enforced THROUGH the metadata work too, not only in the dirent- // loop: 400 lstat+realpath+stat triples on a cold or networked filesystem is where- // a slow scan actually spends its time.- if (Date.now() > deadlineAt) { hitEntryCap = true; break; }- // `~/projects` under `~` is a CONTAINER, not a project inside its parent.- if (containerPaths.has(path.join(container.path, name))) { skipped += 1; continue; }- const candidate = await validateProjectChild(container, name);- if (!candidate.ok) { skipped += 1; continue; }- dirs.push(candidate.path);+ // A directory that vanishes mid-walk yields what we already fully processed.}- return { dirs, entriesVisited, hitEntryCap, skipped };+ return { dirs, consumed, stop, skipped };}export type ProjectDirs = {@@ -80,59 +94,70 @@ export type ProjectDirs = {scan: ScanReport;};-/** The one walk. `listProjectDirs()` is this over every discovered container; an- * explicit `rootHint` is this over exactly one. */+const complete = (rootCount: number): ScanCursor => ({ rootIndex: rootCount, containerIndex: 0, entriesConsumed: 0 });++/** The one walk. `listProjectDirs()` is this over every discovered container; an explicit+ * `rootHint` is this over exactly one. */export async function listProjectDirsIn(containers: ProjectContainer[],- options: { explicit?: boolean; cursor?: ScanCursor } = {},+ options: { explicit?: boolean; cursor?: ScanCursor; startIndex?: number } = {},): Promise<ProjectDirs> {const deadlineAt = Date.now() + PROJECT_LIMITS.maxScanMs;+ const startIndex = options.startIndex ?? 0;const dirs: Array<{ container: ProjectContainer; dir: string }> = [];const reasons: string[] = [];const scannedRoots: string[] = [];- const skippedRoots = options.explicit ? [] : await overflowRoots();+ const overflow = options.explicit ? { roots: [], nextIndex: startIndex } : await overflowRoots(startIndex);+ const skippedRoots = [...overflow.roots];const containerPaths = new Set(containers.map((c) => c.path));- const resumeRoots = new Map((options.cursor?.roots ?? []).map((r) => [r.root, r.entriesConsumed]));- const skipRoots = new Set(options.cursor?.skipRoots ?? []);- const cursors: ScanCursor["roots"] = [];+ const cursor = options.cursor;const pendingRoots: string[] = [];+ let position: ScanCursor | undefined;let skippedProjects = 0;- if (skippedRoots.length) { reasons.push("maxRoots"); pendingRoots.push(...skippedRoots.map((r) => r.path)); }for (const container of containers) {- if (skipRoots.has(container.path)) continue;+ // A recorded position addresses one exact container; earlier ones are already done.+ if (cursor && (container.rootIndex < cursor.rootIndex+ || (container.rootIndex === cursor.rootIndex && container.containerIndex < cursor.containerIndex))) continue;+ const resumeHere = cursor+ && container.rootIndex === cursor.rootIndex+ && container.containerIndex === cursor.containerIndex+ && (!cursor.containerPath || cursor.containerPath === container.path);+ const skipEntries = resumeHere ? cursor!.entriesConsumed : 0;+if (Date.now() > deadlineAt) {reasons.push("deadline");- skippedRoots.push({ path: container.path, reason: "deadline" });- pendingRoots.push(container.path);- continue;+ position = { rootIndex: container.rootIndex, containerIndex: container.containerIndex, entriesConsumed: skipEntries, containerPath: container.path };+ pendingRoots.push(...containers.slice(containers.indexOf(container)).map((c) => c.path));+ break;}- const walk = await walkContainer(container, containerPaths, resumeRoots.get(container.path) ?? 0, deadlineAt);++ const walk = await walkContainer(container, containerPaths, skipEntries, deadlineAt, PROJECT_LIMITS.maxProjects - dirs.length);scannedRoots.push(container.path);skippedProjects += walk.skipped;- if (walk.hitEntryCap) {- reasons.push(`maxEntriesPerRoot:${container.path}`);- cursors.push({ root: container.path, entriesConsumed: walk.entriesVisited });- }- let capped = false;- for (const dir of walk.dirs) {- if (dirs.length >= PROJECT_LIMITS.maxProjects) { reasons.push("maxProjects"); capped = true; break; }- dirs.push({ container, dir });- }- if (capped) {- // Resume this container from where its accepted entries stopped, and leave the- // remaining containers pending rather than pretending they were covered.- cursors.push({ root: container.path, entriesConsumed: walk.entriesVisited - walk.dirs.length + dirs.length });- pendingRoots.push(...containers.slice(containers.indexOf(container) + 1).map((c) => c.path));+ for (const dir of walk.dirs) dirs.push({ container, dir });++ if (walk.stop) {+ reasons.push(walk.stop === "maxEntriesPerRoot" ? `maxEntriesPerRoot:${container.path}` : walk.stop);+ position = { rootIndex: container.rootIndex, containerIndex: container.containerIndex, entriesConsumed: walk.consumed, containerPath: container.path };+ pendingRoots.push(...containers.slice(containers.indexOf(container)).map((c) => c.path));break;}}+ // Nothing stopped mid-container, so the next position is the first root this scan+ // could not honour — which is how `maxRoots` finally becomes continuable.+ if (!position && overflow.roots.length) {+ reasons.push("maxRoots");+ pendingRoots.push(...overflow.roots.map((r) => r.path));+ position = { rootIndex: overflow.nextIndex, containerIndex: 0, entriesConsumed: 0 };+ }+const truncationReasons = [...new Set(reasons)];- const done = [...new Set(scannedRoots)].filter((r) => !cursors.some((c) => c.root === r) && !pendingRoots.includes(r));+ const rootCount = (await allConfiguredRoots()).length;return {containers,- dirs,+ dirs: dirs.sort((a, b) => a.container.path.localeCompare(b.container.path) || a.dir.localeCompare(b.dir)),scan: {truncated: truncationReasons.length > 0,truncationReasons,@@ -142,10 +167,10 @@ export async function listProjectDirsIn(...(truncationReasons.length ? {continuation: {pendingRoots: [...new Set(pendingRoots)],- cursors,- cursorSemantics: "readdir-position" as const,- note: "Pass `cursor` back to resume. Positions are readdir order and are valid while the directory is unchanged.",- cursor: encodeCursor({ roots: cursors, skipRoots: done }),+ position: position ?? complete(rootCount),+ cursorSemantics: "readdir-stream-position" as const,+ note: "Pass `cursor` back to resume at the exact dirent the scan stopped on. Positions are readdir stream order and are valid while the directories are unchanged.",+ cursor: encodeCursor(position ?? complete(rootCount)),},} : {}),},@@ -154,7 +179,8 @@ export async function listProjectDirsIn(/** Every project directory across every container, in container-then-name order. */export async function listProjectDirs(cursor?: ScanCursor): Promise<ProjectDirs> {- return listProjectDirsIn(await projectContainers(), { cursor });+ const startIndex = cursor?.rootIndex ?? 0;+ return listProjectDirsIn(await projectContainers(startIndex), { cursor, startIndex });}export { listProjects } from "./project-list";diff --git a/lib/host/project-scan-types.ts b/lib/host/project-scan-types.tsindex f4414620..cfb416d3 100644--- a/lib/host/project-scan-types.ts+++ b/lib/host/project-scan-types.ts@@ -2,11 +2,28 @@// separate so both the project walk and the skill catalog can describe incompleteness// in the same shape without importing each other's scanners.-/** A resumable position. Positional in readdir order — see encodeCursor's note. */+/**+ * An EXACT resumable position, not a description of one.+ *+ * The previous shape recorded "entries consumed" derived from sorted accepted rows and a+ * global result count, which could point past dirents that were never processed — and it+ * had no way at all to advance past the `maxRoots` prefix, so a 13th configured root was+ * permanently unreachable. This is a raw stream position instead:+ *+ * rootIndex index into the UNCAPPED configured-root list to start from, so the+ * cap window slides forward across calls;+ * containerIndex 0 = the root itself, 1 = its derived `projects/` child;+ * entriesConsumed raw dirents FULLY PROCESSED in that container. A dirent that was+ * read but not validated before a cap or deadline tripped is NOT+ * counted, so it is re-processed rather than skipped.+ */export type ScanCursor = {- roots: Array<{ root: string; entriesConsumed: number }>;- /** Containers already fully covered by the previous page; skipped on resume. */- skipRoots: string[];+ rootIndex: number;+ containerIndex: number;+ entriesConsumed: number;+ /** Canonical path the position was recorded against; a mismatch discards the position+ * rather than resuming into a different directory. */+ containerPath?: string;};export type ScanReport = {@@ -20,8 +37,9 @@ export type ScanReport = {* cannot resume is just data loss with a label on it. */continuation?: {pendingRoots: string[];- cursors: Array<{ root: string; entriesConsumed: number }>;- cursorSemantics: "readdir-position";+ /** The exact stream position the next call resumes from. */+ position: ScanCursor;+ cursorSemantics: "readdir-stream-position";note: string;cursor: string;};diff --git a/lib/host/projects.ts b/lib/host/projects.tsindex 22b4f296..3ae9e1e6 100644--- a/lib/host/projects.ts+++ b/lib/host/projects.ts@@ -4,7 +4,7 @@ import { normalizeProjectKey, projectAliasesFor, projectAliasTarget } from "./primport { boundedGitMeta, fullGitMeta, packageMeta } from "./project-meta";import { homeDir, isUnderRoot } from "./paths";import { validateProjectChild, validateProjectDescendant, validateRootHint } from "./project-candidate";-import { configuredRootPaths, containerFor, listProjectDirsIn, projectContainers, type ProjectContainer } from "./project-roots";+import { configuredRootPaths, containerById, containerFor, listProjectDirsIn, projectContainers, type ProjectContainer } from "./project-roots";export type ProjectResolution = {hint: string;@@ -16,7 +16,7 @@ export type ProjectResolution = {root: string;packageName?: string;aliases: string[];- matchedBy: "path" | "name" | "alias" | "package" | "fuzzy";+ matchedBy: "id" | "path" | "name" | "alias" | "package" | "fuzzy";};async function resolutionFor(container: ProjectContainer, dir: string, hint: string, matchedBy: ProjectResolution["matchedBy"]): Promise<ProjectResolution> {@@ -47,13 +47,14 @@ function expandHome(p: string): string {async function containersFor(rootHint?: string): Promise<ProjectContainer[]> {if (!rootHint) return projectContainers();const absolute = expandHome(rootHint);- const validated = await validateRootHint(absolute);- if (!validated.ok) return [];// Checked against EVERY configured root, not the scan-capped subset: naming a root// must never widen the jail, but the jail must not shrink just because 12 other roots- // are configured ahead of it.- const authorized = await configuredRootPaths();- if (!authorized.some((root) => isUnderRoot(validated.path, root))) return [];+ // are configured ahead of it. The owning root is also what the hidden-component check+ // is measured against.+ const owner = (await configuredRootPaths()).find((root) => isUnderRoot(absolute, root));+ if (!owner) return [];+ const validated = await validateRootHint(absolute, owner);+ if (!validated.ok) return [];return [containerFor(validated.path)];}@@ -72,6 +73,19 @@ export async function resolveProjectHint(hint: string, rootHint?: string): Promiconst containers = await containersFor(rootHint);if (!containers.length) return null;+ // EXACT `<32-hex-rootId>/<project-name>` first — the id projects_list advertises and+ // workflow_start is told to pass. It must never fall through to fuzzy matching: with+ // two same-named projects that returned the WRONG one, which is the entire failure the+ // root-qualified id exists to prevent.+ const exactId = /^([a-f0-9]{32})\/(.+)$/.exec(raw);+ if (exactId) {+ const container = await containerById(exactId[1]);+ if (!container) return null;+ if (rootHint && !containers.some((c) => c.path === container.path)) return null;+ const candidate = await validateProjectChild(container, exactId[2]);+ return candidate.ok ? resolutionFor(container, candidate.path, raw, "id") : null;+ }+const pathHint = raw.startsWith("projects/") ? `~/${raw}` : raw;if (/^(?:~\/|\/|\.\.?\/)/.test(pathHint)) {// A path hint gets the SAME component-by-component validation an enumerated entrydiff --git a/lib/mcp/collision-e2e.test.ts b/lib/mcp/collision-e2e.test.tsnew file mode 100644index 00000000..dd753066--- /dev/null+++ b/lib/mcp/collision-e2e.test.ts@@ -0,0 +1,144 @@+import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";+vi.mock("server-only", () => ({}));+import { promises as fs } from "fs";+import path from "path";++// THE COLLISION, END TO END.+//+// `/tmp/mso-root-50323` and `/tmp/mso-root-125549` both hash to `51e156ef` at 8 hex —+// a real pair found by a probe, not a hypothetical. Each holds a project called `widget`+// shipping a skill called `deploy`. Under the old 32-bit id these two collapsed into one+// row and half the box became invisible.+//+// It is not enough that the ids differ: every surface a client actually uses must return+// the SECOND project when handed the second id. That is what this file pins —+// projects_list, skills_list, skills_read, skills_search and workflow_start.+const ROOT_A = "/tmp/mso-root-50323";+const ROOT_B = "/tmp/mso-root-125549";+const previous = {+ read: process.env.OS_FS_READ_ROOTS,+ write: process.env.OS_FS_WRITE_ROOTS,+ memory: process.env.OS_SKILL_MEMORY_STORE,+};++async function seed(root: string, marker: string) {+ const skills = path.join(root, "widget", ".claude/skills/deploy");+ await fs.mkdir(skills, { recursive: true });+ await fs.writeFile(+ path.join(skills, "SKILL.md"),+ `---\nname: deploy\ndescription: Deploy the widget service from ${marker}.\n---\n\n# deploy ${marker}\n\nRun the ${marker} pipeline.\n`,+ );+ await fs.writeFile(path.join(root, "widget", "package.json"), JSON.stringify({ name: `widget-${marker}` }));+}++beforeAll(async () => {+ await seed(ROOT_A, "alpha");+ await seed(ROOT_B, "bravo");+ process.env.OS_FS_READ_ROOTS = `${ROOT_A}:${ROOT_B}`;+ process.env.OS_FS_WRITE_ROOTS = `${ROOT_A}:${ROOT_B}`;+ process.env.OS_SKILL_MEMORY_STORE = path.join(ROOT_A, "memory.json");+});++afterAll(async () => {+ for (const [key, value] of Object.entries(previous)) {+ const name = key === "read" ? "OS_FS_READ_ROOTS" : key === "write" ? "OS_FS_WRITE_ROOTS" : "OS_SKILL_MEMORY_STORE";+ if (value === undefined) delete process.env[name];+ else process.env[name] = value;+ }+ await Promise.all([ROOT_A, ROOT_B].map((r) => fs.rm(r, { recursive: true, force: true })));+});++const { DISCOVERY_TOOLS } = await import("./tools-discovery");+const { LEARNING_TOOLS } = await import("./tools-learning");+const { shortId } = await import("@/lib/host/project-roots");+const { searchSkillMemory } = await import("@/lib/skills/search");++const tool = (name: string) => {+ const found = [...DISCOVERY_TOOLS, ...LEARNING_TOOLS].find((t) => t.name === name);+ if (!found) throw new Error(`missing ${name}`);+ return found;+};+const run = (name: string, args: Record<string, unknown>, scope: "read" | "write" = "read") =>+ tool(name).run(args, { scope, actor: "mcp:collision" });++const idA = () => `${shortId(ROOT_A)}/widget`;+const idB = () => `${shortId(ROOT_B)}/widget`;++describe("the 8-hex collision pair stays distinct on every surface", () => {+ it("collides at 8 hex and separates at 32 — the premise of this file", () => {+ expect(shortId(ROOT_A).slice(0, 8)).toBe(shortId(ROOT_B).slice(0, 8));+ expect(shortId(ROOT_A)).not.toBe(shortId(ROOT_B));+ });++ it("projects_list returns BOTH widgets with distinct ids", async () => {+ const result = await run("projects_list", { limit: 200 }) as {+ projects: Array<{ id: string; name: string; path: string; packageName?: string }>;+ };+ const widgets = result.projects.filter((p) => p.name === "widget");+ expect(widgets).toHaveLength(2);+ expect(widgets.map((p) => p.id).sort()).toEqual([idA(), idB()].sort());+ expect(widgets.find((p) => p.id === idB())?.packageName).toBe("widget-bravo");+ });++ it("skills_list returns both projects' deploy skills under distinct ids", async () => {+ const { skills } = await run("skills_list", { limit: 200 }) as {+ skills: Array<{ id: string; description: string; project?: { id: string } }>;+ };+ const deploys = skills.filter((s) => s.id.endsWith("/deploy"));+ expect(deploys.map((s) => s.id).sort()).toEqual([`${idA()}/deploy`, `${idB()}/deploy`].sort());+ });++ it("skills_list filtered by the SECOND project id returns only its skill", async () => {+ const only = await run("skills_list", { project: idB(), limit: 200 }) as {+ skills: Array<{ id: string; description: string }>;+ };+ expect(only.skills.map((s) => s.id)).toEqual([`${idB()}/deploy`]);+ expect(only.skills[0].description).toContain("bravo");+ });++ it("skills_read on the SECOND id returns the SECOND project's instructions", async () => {+ const result = await run("skills_read", { name: `${idB()}/deploy` }) as { content: string; project: { id: string } };+ expect(result.project.id).toBe(idB());+ expect(result.content).toContain("bravo");+ expect(result.content).not.toContain("alpha");+ });++ it("skills_read refuses the ambiguous bare name and offers both exact ids", async () => {+ await expect(run("skills_read", { name: "deploy" })).rejects.toThrow(/ambiguous across projects/);+ await expect(run("skills_read", { name: "widget/deploy" })).rejects.toThrow(new RegExp(idB()));+ });++ it("skills_search surfaces both, each carrying its own project id", async () => {+ const { hits } = await searchSkillMemory("deploy the widget service");+ const deploys = hits.filter((h) => h.kind === "skill" && h.id.endsWith("/deploy"));+ expect(deploys.map((h) => h.id).sort()).toEqual([`${idA()}/deploy`, `${idB()}/deploy`].sort());+ expect(deploys.find((h) => h.id === `${idB()}/deploy`)?.project?.id).toBe(idB());+ });++ it("workflow_start resolves the SECOND project from its exact id, not the first", async () => {+ const result = await run("workflow_start", {+ intent: "deploy the widget service", project: idB(),+ }, "write") as {+ bootstrap: { project: { id: string; path: string; matchedBy: string } };+ };+ expect(result.bootstrap.project.matchedBy).toBe("id");+ expect(result.bootstrap.project.id).toBe(idB());+ expect(result.bootstrap.project.path).toBe(path.join(ROOT_B, "widget"));+ });++ it("workflow_start resolves the FIRST project from its own id", async () => {+ const result = await run("workflow_start", {+ intent: "deploy the widget service", project: idA(),+ }, "write") as { bootstrap: { project: { id: string; path: string } } };+ expect(result.bootstrap.project.id).toBe(idA());+ expect(result.bootstrap.project.path).toBe(path.join(ROOT_A, "widget"));+ });++ it("refuses an id whose rootId is unknown rather than falling through to fuzzy", async () => {+ const bogus = "f".repeat(32);+ const result = await run("workflow_start", { intent: "x", project: `${bogus}/widget` }, "write") as {+ bootstrap: { project?: { matchedBy?: string } };+ };+ expect(result.bootstrap.project?.matchedBy).toBe("unresolved");+ });+});diff --git a/lib/mcp/dispatch.test.ts b/lib/mcp/dispatch.test.tsindex 3f1c3d2a..06ff4c16 100644--- a/lib/mcp/dispatch.test.ts+++ b/lib/mcp/dispatch.test.ts@@ -44,7 +44,7 @@ describe("protocol", () => {it("publishes server and toolset metadata so action drift is visible", async () => {const r = await dispatch({ id: 1, method: "initialize" }, "exec");const result = r.result as { serverInfo: { version: string }; _meta: { toolset: { toolCount: number; hash: string } } };- expect(result.serverInfo.version).toBe("1.5.2");+ expect(result.serverInfo.version).toBe("1.5.3");expect(result._meta.toolset.toolCount).toBe(TOOLS.length);expect(result._meta.toolset.hash).toMatch(/^[a-f0-9]{16}$/);});diff --git a/lib/mcp/toolset.ts b/lib/mcp/toolset.tsindex bd2ca363..3d20d576 100644--- a/lib/mcp/toolset.ts+++ b/lib/mcp/toolset.ts@@ -2,9 +2,9 @@ import { createHash } from "crypto";import type { Scope } from "./scope";import type { McpTool } from "./tool-kit";-export const MCP_SERVER_VERSION = "1.5.2";-export const MCP_TOOLSET_VERSION = "2026.08.20.5";-export const MCP_TOOLSET_CHANGED_AT = "2026-08-20T16:00:00Z";+export const MCP_SERVER_VERSION = "1.5.3";+export const MCP_TOOLSET_VERSION = "2026.08.20.6";+export const MCP_TOOLSET_CHANGED_AT = "2026-08-20T16:40:00Z";export type McpToolsetInfo = {serverVersion: string;diff --git a/lib/skills/catalog-continuation.test.ts b/lib/skills/catalog-continuation.test.tsnew file mode 100644index 00000000..7800114c--- /dev/null+++ b/lib/skills/catalog-continuation.test.ts@@ -0,0 +1,114 @@+import { afterEach, describe, expect, it } from "vitest";+import { mkdir, mkdtemp, rm, writeFile } from "fs/promises";+import os from "os";+import path from "path";+import { catalogSkillsDetailed } from "./catalog";+import { SKILL_SCAN_LIMITS } from "./catalog-types";+import { projectRefFor } from "./project-skills";++const temps: string[] = [];+async function temp() {+ const dir = await mkdtemp(path.join(os.tmpdir(), "mso-skill-cont-"));+ temps.push(dir);+ return dir;+}+afterEach(async () => { await Promise.all(temps.splice(0).map((d) => rm(d, { recursive: true, force: true }))); });++const body = (name: string) => `---\nname: ${name}\ndescription: ${name} desc\n---\n\n# ${name}\n`;++async function skillsIn(root: string, count: number, prefix: string) {+ await mkdir(root, { recursive: true });+ await Promise.all(Array.from({ length: count }, async (_, i) => {+ const dir = path.join(root, `${prefix}${String(i).padStart(4, "0")}`);+ await mkdir(dir, { recursive: true });+ await writeFile(path.join(dir, "SKILL.md"), body(`${prefix}${i}`));+ }));+}++/** Follow the cursor until the catalog reports itself complete. */+async function drain(opts: { appDir: string; homeDir: string; projects: ReturnType<typeof projectRefFor>[] }) {+ const ids = new Set<string>();+ let cursor: string | undefined;+ for (let i = 0; i < 40; i += 1) {+ const page = await catalogSkillsDetailed({ ...opts, cursor });+ for (const s of page.skills) ids.add(s.id);+ if (!page.scan.truncated || !page.scan.continuation?.cursor) break;+ if (page.scan.continuation.cursor === cursor) break; // no forward progress+ cursor = page.scan.continuation.cursor;+ }+ return ids;+}++describe("maxProjectSkills continuation returns the remainder of a partially consumed root", () => {+ it("drains 150 + 160 project skills across pages without losing the last 10", async () => {+ const workspace = await temp();+ const one = path.join(workspace, "one");+ const two = path.join(workspace, "two");+ await skillsIn(path.join(one, ".claude/skills"), 150, "a");+ await skillsIn(path.join(two, ".claude/skills"), 160, "b");+ const projects = [projectRefFor(one, workspace), projectRefFor(two, workspace)];+ const opts = { appDir: await temp(), homeDir: await temp(), projects };++ const first = await catalogSkillsDetailed(opts);+ expect(first.skills.filter((s) => s.project).length).toBe(SKILL_SCAN_LIMITS.maxProjectSkills);+ expect(first.scan.truncationReasons).toContain("maxProjectSkills");+ expect(first.scan.continuation!.pendingProjects).toBeGreaterThan(0);++ const all = await drain(opts);+ // 310 project skills in total, none dropped.+ expect([...all].filter((id) => id.includes("/")).length).toBe(310);+ });++ it("records the exact interrupted root rather than marking it complete", async () => {+ const workspace = await temp();+ const one = path.join(workspace, "one");+ await skillsIn(path.join(one, ".claude/skills"), SKILL_SCAN_LIMITS.maxProjectSkills + 20, "c");+ const projects = [projectRefFor(one, workspace)];+ const opts = { appDir: await temp(), homeDir: await temp(), projects };++ const first = await catalogSkillsDetailed(opts);+ const resume = first.scan.continuation!.cursors[0];+ expect(resume.root).toBe(path.join(one, ".claude/skills"));+ expect(resume.entriesConsumed).toBeGreaterThan(0);+ expect(first.scan.continuation!.pendingRoots).toContain(resume.root);++ const all = await drain(opts);+ expect([...all].filter((id) => id.includes("/")).length).toBe(SKILL_SCAN_LIMITS.maxProjectSkills + 20);+ });+});++describe("a deadline never advances the cursor past an unprocessed entry", () => {+ it("keeps every skill reachable when the deadline expires immediately", async () => {+ const app = await temp();+ await skillsIn(path.join(app, "claude-skills"), 3, "d");+ const opts = { appDir: app, homeDir: await temp(), projects: [] };++ // maxScanMs is a constant, so the honest way to force the deadline branch is a clock+ // that is already past it on the first check.+ const realNow = Date.now;+ let calls = 0;+ Date.now = () => (calls++ === 0 ? realNow() : realNow() + SKILL_SCAN_LIMITS.maxScanMs + 1_000);+ let first;+ try {+ first = await catalogSkillsDetailed(opts);+ } finally {+ Date.now = realNow;+ }+ expect(first.scan.truncated).toBe(true);+ expect(first.scan.truncationReasons).toContain("deadline");+ // Whatever it could not process must still be reachable on resume.+ const resumed = await catalogSkillsDetailed({ ...opts, cursor: first.scan.continuation!.cursor });+ const ids = new Set([...first.skills, ...resumed.skills].map((s) => s.id));+ expect([...ids].filter((id) => id.startsWith("d"))).toHaveLength(3);+ });+});++describe("a complete catalog build reports no continuation", () => {+ it("stays truncated=false", async () => {+ const app = await temp();+ await skillsIn(path.join(app, "claude-skills"), 2, "e");+ const { scan } = await catalogSkillsDetailed({ appDir: app, homeDir: await temp(), projects: [] });+ expect(scan).toMatchObject({ truncated: false, truncationReasons: [] });+ expect(scan.continuation).toBeUndefined();+ });+});diff --git a/lib/skills/catalog-cursor.ts b/lib/skills/catalog-cursor.tsnew file mode 100644index 00000000..d08b8c87--- /dev/null+++ b/lib/skills/catalog-cursor.ts@@ -0,0 +1,25 @@+// Encoding for a resumable skill-catalog position. Same contract as the project walk's+// cursor: a cap the caller cannot continue past is data loss with a label on it.+import type { SkillScanCursor } from "./catalog-types";++export function encodeSkillCursor(cursor: SkillScanCursor): string {+ return Buffer.from(JSON.stringify(cursor), "utf8").toString("base64url");+}++export function decodeSkillCursor(raw: string | undefined): SkillScanCursor | undefined {+ if (!raw) return undefined;+ try {+ const parsed = JSON.parse(Buffer.from(raw, "base64url").toString("utf8")) as SkillScanCursor;+ if (!parsed || typeof parsed !== "object") return undefined;+ return {+ doneRoots: Array.isArray(parsed.doneRoots) ? parsed.doneRoots.filter((r) => typeof r === "string") : [],+ projectOffset: Number.isFinite(parsed.projectOffset) ? Math.max(0, Math.floor(parsed.projectOffset)) : 0,+ ...(parsed.resume && typeof parsed.resume.root === "string"+ ? { resume: { root: parsed.resume.root, entriesConsumed: Math.max(0, Math.floor(parsed.resume.entriesConsumed ?? 0)) } }+ : {}),+ };+ } catch {+ return undefined;+ }+}+diff --git a/lib/skills/catalog-scan.ts b/lib/skills/catalog-scan.tsindex 527504f8..d3145a88 100644--- a/lib/skills/catalog-scan.ts+++ b/lib/skills/catalog-scan.ts@@ -70,72 +70,77 @@ async function verifiedBundledSkill(dir: string, md: string): Promise<Pick<Skill};}-/** Bounded `opendir` walk of ONE skill root — global roots included, which previously- * had no entry budget at all. Names are capped before anything is stat'd or read. */-async function skillDirNames(root: string, deadlineAt: number, skipEntries = 0): Promise<{ names: string[]; hitCap: boolean; entriesVisited: number }> {- const names: string[] = [];- let hitCap = false;- let entriesVisited = skipEntries;+/**+ * ONE streaming pass over a skill root.+ *+ * The previous version read every dirent name, sorted them, then validated — so a+ * deadline that expired during validation left `entriesVisited` pointing past names+ * nothing had looked at, and continuation skipped them silently. Now each dirent is+ * fully processed before the position advances, and every cap is checked BEFORE the+ * entry is touched, so stopping never consumes it.+ *+ * `budget` is how many more skills the whole catalog may still accept, which is how the+ * overall `maxProjectSkills` ceiling is enforced mid-root rather than only between roots.+ */+export async function scanRoot(+ spec: RootSpec,+ deadlineAt: number,+ skipEntries = 0,+ budget = Number.POSITIVE_INFINITY,+): Promise<{ found: Array<{ skill: SkillInfo; priority: number }>; stop?: "maxEntriesPerRoot" | "deadline" | "budget"; consumed: number }> {+ const found: Array<{ skill: SkillInfo; priority: number }> = [];+ let consumed = skipEntries;let seen = 0;- const handle = await fs.opendir(root).catch(() => null);- if (!handle) return { names, hitCap, entriesVisited };+ let processed = 0;+ let stop: "maxEntriesPerRoot" | "deadline" | "budget" | undefined;+ const handle = await fs.opendir(spec.path).catch(() => null);+ if (!handle) return { found, consumed };try {for await (const entry of handle) {seen += 1;if (seen <= skipEntries) continue;- // EVERY dirent costs budget, accepted or not. Counting only directories meant a- // root holding a million regular files still required a million iterations before- // the advertised entry cap was reached.- if (entriesVisited - skipEntries >= SKILL_SCAN_LIMITS.maxEntriesPerRoot || Date.now() > deadlineAt) { hitCap = true; break; }- entriesVisited += 1;- if (!entry.isDirectory() && !entry.isSymbolicLink()) continue;- names.push(entry.name);- }- } catch {- // A root that vanishes mid-walk yields what we already have.- }- return { names: names.sort((a, b) => a.localeCompare(b)), hitCap, entriesVisited };-}--export async function scanRoot(spec: RootSpec, deadlineAt: number, skipEntries = 0): Promise<{ found: Array<{ skill: SkillInfo; priority: number }>; hitCap: boolean; entriesVisited: number }> {- const { names, hitCap: dirCap, entriesVisited } = await skillDirNames(spec.path, deadlineAt, skipEntries);- let hitCap = dirCap;- const found: Array<{ skill: SkillInfo; priority: number }> = [];- for (const name of names) {- // The deadline is enforced THROUGH the per-skill stat/read work too, not only in- // the dirent loop, which is where a slow filesystem actually spends its time.- if (Date.now() > deadlineAt) { hitCap = true; break; }- const dir = path.join(spec.path, name);- if (!(await fs.stat(dir).catch(() => null))?.isDirectory()) continue;- const file = path.join(dir, SKILL_FILE);- const md = await readSkillFile(file);- if (md === null) continue;+ // EVERY dirent costs budget, accepted or not — and every stop check happens before+ // the entry is processed, so `consumed` never runs ahead of the work.+ if (processed >= SKILL_SCAN_LIMITS.maxEntriesPerRoot) { stop = "maxEntriesPerRoot"; break; }+ if (Date.now() > deadlineAt) { stop = "deadline"; break; }+ if (found.length >= budget) { stop = "budget"; break; }+ processed += 1;- let trust = spec.trust;- let provenance: SkillInfo["provenance"];- if (spec.verifyClawHub) {- const verified = await verifiedBundledSkill(dir, md);- trust = verified.trust;- provenance = verified.provenance;- } else if (spec.project) {- // Path alone grants nothing: containment, ownership and SKILL.md shape decide.- trust = await projectSkillTrust(dir, spec.project.path);+ if (entry.isDirectory() || entry.isSymbolicLink()) {+ const dir = path.join(spec.path, entry.name);+ if ((await fs.stat(dir).catch(() => null))?.isDirectory()) {+ const md = await readSkillFile(path.join(dir, SKILL_FILE));+ if (md !== null) {+ let trust = spec.trust;+ let provenance: SkillInfo["provenance"];+ if (spec.verifyClawHub) {+ const verified = await verifiedBundledSkill(dir, md);+ trust = verified.trust;+ provenance = verified.provenance;+ } else if (spec.project) {+ // Path alone grants nothing: containment, ownership and SKILL.md shape decide.+ trust = await projectSkillTrust(dir, spec.project.path);+ }+ found.push({+ priority: spec.priority,+ skill: {+ id: spec.project ? `${spec.project.id}/${entry.name}` : entry.name,+ name: entry.name,+ path: path.join(dir, SKILL_FILE),+ description: skillDescription(md),+ source: spec.source,+ trust,+ ...(spec.project ? { project: spec.project } : {}),+ ...(provenance ? { provenance } : {}),+ },+ });+ }+ }+ }+ consumed = seen; // ONLY now — the entry is fully handled.}-- found.push({- priority: spec.priority,- skill: {- id: spec.project ? `${spec.project.id}/${name}` : name,- name,- path: file,- description: skillDescription(md),- source: spec.source,- trust,- ...(spec.project ? { project: spec.project } : {}),- ...(provenance ? { provenance } : {}),- },- });+ } catch {+ // A root that vanishes mid-walk yields what we already fully processed.}- return { found, hitCap, entriesVisited };+ return { found, stop, consumed };}-diff --git a/lib/skills/catalog-types.ts b/lib/skills/catalog-types.tsindex 97ce5610..044f01d7 100644--- a/lib/skills/catalog-types.ts+++ b/lib/skills/catalog-types.ts@@ -50,11 +50,13 @@ export type SkillInfo = {* walk's cursor: name-ordered resume would require visiting every dirent, which is the* unbounded walk the entry cap exists to prevent. */export type SkillScanCursor = {- roots: Array<{ root: string; entriesConsumed: number }>;- /** Roots fully covered by the previous page; skipped on resume. */- skipRoots: string[];- /** Projects already consumed, so a maxProjects/maxProjectSkills cap can be continued. */+ /** Roots that finished CLEANLY; skipped on resume. */+ doneRoots: string[];+ /** Projects every one of whose roots finished cleanly. A partially consumed project is+ * deliberately NOT counted here — it is re-listed and resumed at `resume`. */projectOffset: number;+ /** The exact dirent position inside the one root that was interrupted. */+ resume?: { root: string; entriesConsumed: number };};/** What a catalog build could NOT cover. Mirrors lib/host's ScanReport so the twodiff --git a/lib/skills/catalog.ts b/lib/skills/catalog.tsindex 76a81fbf..bc37690a 100644--- a/lib/skills/catalog.ts+++ b/lib/skills/catalog.ts@@ -1,6 +1,7 @@import os from "os";import path from "path";import { scanRoot, type RootSpec } from "./catalog-scan";+import { decodeSkillCursor, encodeSkillCursor } from "./catalog-cursor";import {SKILL_FILE, SKILL_SCAN_LIMITS, skillIsExecutableByDefault,type ProjectRef, type SkillInfo, type SkillScanCursor, type SkillScanReport, type SkillSource, type SkillTrust,@@ -9,6 +10,7 @@ import { discoveredProjects, projectSkillRoots } from "./project-skills";export { SKILL_FILE, SKILL_SCAN_LIMITS, skillIsExecutableByDefault };export { readSkillFile, skillDescription } from "./catalog-scan";+export { decodeSkillCursor, encodeSkillCursor } from "./catalog-cursor";export type { ProjectRef, SkillInfo, SkillScanCursor, SkillScanReport, SkillSource, SkillTrust };type CatalogOptions = {@@ -21,25 +23,6 @@ type CatalogOptions = {cursor?: string;};-export function encodeSkillCursor(cursor: SkillScanCursor): string {- return Buffer.from(JSON.stringify(cursor), "utf8").toString("base64url");-}--export function decodeSkillCursor(raw: string | undefined): SkillScanCursor | undefined {- if (!raw) return undefined;- try {- const parsed = JSON.parse(Buffer.from(raw, "base64url").toString("utf8")) as SkillScanCursor;- if (!parsed || typeof parsed !== "object") return undefined;- return {- roots: Array.isArray(parsed.roots) ? parsed.roots.filter((r) => typeof r?.root === "string") : [],- skipRoots: Array.isArray(parsed.skipRoots) ? parsed.skipRoots.filter((r) => typeof r === "string") : [],- projectOffset: Number.isFinite(parsed.projectOffset) ? Math.max(0, Math.floor(parsed.projectOffset)) : 0,- };- } catch {- return undefined;- }-}-/*** Root priority is a security boundary, not just display order.*@@ -75,9 +58,6 @@ export async function catalogSkillsDetailed(options: CatalogOptions = {}): Promiconst homeDir = options.homeDir ?? os.homedir();const deadlineAt = Date.now() + SKILL_SCAN_LIMITS.maxScanMs;const cursor = decodeSkillCursor(options.cursor);- const resumeRoots = new Map((cursor?.roots ?? []).map((r) => [r.root, r.entriesConsumed]));- const skipRoots = new Set(cursor?.skipRoots ?? []);- const projectOffset = cursor?.projectOffset ?? 0;const reasons: string[] = [];let projects = options.projects;@@ -86,78 +66,90 @@ export async function catalogSkillsDetailed(options: CatalogOptions = {}): Promiprojects = discovered.projects;reasons.push(...discovered.truncationReasons);}+ // `projectOffset` counts projects FULLY consumed by earlier pages; the partially+ // consumed one is re-listed and resumed at its exact dirent position.+ const projectOffset = Math.min(cursor?.projectOffset ?? 0, projects.length);const remaining = projects.slice(projectOffset);const { roots: projectRoots, truncated: projectsCapped } = await projectSkillRoots(remaining);if (projectsCapped) reasons.push("maxProjects");+ const globalSpecs = skillRoots(appDir, homeDir);const specs: RootSpec[] = [- ...skillRoots(appDir, homeDir),+ ...globalSpecs,...projectRoots.map((root): RootSpec => ({path: root.path, source: "project", trust: "untrusted", priority: root.priority, project: root.project,})),];+ const doneRoots = new Set(cursor?.doneRoots ?? []);const chosen = new Map<string, { skill: SkillInfo; priority: number }>();- const cursors: SkillScanCursor["roots"] = [];+ const completedRoots: string[] = [];const pendingRoots: string[] = [];- const doneRoots: string[] = [];- const consumedProjects = new Set<string>();+ const consumedProjects: string[] = [];let projectSkills = 0;+ let resume: SkillScanCursor["resume"];let stoppedAt = -1;for (const [index, spec] of specs.entries()) {- if (skipRoots.has(spec.path)) { doneRoots.push(spec.path); continue; }- if (Date.now() > deadlineAt) { reasons.push("deadline"); stoppedAt = index; break; }- if (spec.project && projectSkills >= SKILL_SCAN_LIMITS.maxProjectSkills) {- reasons.push("maxProjectSkills");+ if (doneRoots.has(spec.path)) { completedRoots.push(spec.path); continue; }+ const skipEntries = cursor?.resume?.root === spec.path ? cursor.resume.entriesConsumed : 0;+ const budget = spec.project ? SKILL_SCAN_LIMITS.maxProjectSkills - projectSkills : Number.POSITIVE_INFINITY;++ if (Date.now() > deadlineAt) {+ reasons.push("deadline");+ resume = { root: spec.path, entriesConsumed: skipEntries };stoppedAt = index;break;}- const { found, hitCap, entriesVisited } = await scanRoot(spec, deadlineAt, resumeRoots.get(spec.path) ?? 0);- if (hitCap) {- reasons.push(`maxEntriesPerRoot:${spec.path}`);- cursors.push({ root: spec.path, entriesConsumed: entriesVisited });- } else {- doneRoots.push(spec.path);+ if (spec.project && budget <= 0) {+ reasons.push("maxProjectSkills");+ resume = { root: spec.path, entriesConsumed: skipEntries };+ stoppedAt = index;+ break;}- let overflowed = false;++ const { found, stop, consumed } = await scanRoot(spec, deadlineAt, skipEntries, budget);for (const candidate of found) {- // The overall cap is checked INSIDE the loop. Checking it only before scanRoot- // let one root carry the total from just under 300 to nearly 500.- if (spec.project && projectSkills >= SKILL_SCAN_LIMITS.maxProjectSkills) {- reasons.push("maxProjectSkills");- overflowed = true;- break;- }if (spec.project) projectSkills += 1;const current = chosen.get(candidate.skill.id);if (!current || candidate.priority > current.priority) chosen.set(candidate.skill.id, candidate);}- if (spec.project) consumedProjects.add(spec.project.id);- if (overflowed) { stoppedAt = index; break; }++ if (stop) {+ reasons.push(stop === "budget" ? "maxProjectSkills" : stop === "deadline" ? "deadline" : `maxEntriesPerRoot:${spec.path}`);+ // The exact dirent this root stopped on — never "the whole root is done".+ resume = { root: spec.path, entriesConsumed: consumed };+ stoppedAt = index;+ break;+ }+ completedRoots.push(spec.path);+ // A project counts as consumed only once EVERY one of its roots finished cleanly.+ if (spec.project && specs.slice(index + 1).every((later) => later.project?.id !== spec.project!.id)) {+ consumedProjects.push(spec.project.id);+ }}if (stoppedAt >= 0) pendingRoots.push(...specs.slice(stoppedAt).map((r) => r.path));const truncationReasons = [...new Set(reasons)];- const pendingProjects = Math.max(0, projects.length - projectOffset - consumedProjects.size);+ const pendingProjects = Math.max(0, projects.length - projectOffset - consumedProjects.length);return {skills: [...chosen.values()].map(({ skill }) => skill).sort((a, b) => a.id.localeCompare(b.id)),scan: {truncated: truncationReasons.length > 0,truncationReasons,scannedRoots: specs.length,- scannedProjects: consumedProjects.size,+ scannedProjects: consumedProjects.length,...(truncationReasons.length ? {continuation: {pendingRoots: [...new Set(pendingRoots)],- cursors,+ cursors: resume ? [resume] : [],pendingProjects,cursorSemantics: "readdir-position" as const,- note: "Pass `cursor` back to resume. Positions are readdir order and are valid while the directories are unchanged.",+ note: "Pass `cursor` back to resume at the exact dirent the scan stopped on. Positions are readdir stream order and are valid while the directories are unchanged.",cursor: encodeSkillCursor({- roots: cursors,- skipRoots: [...new Set(doneRoots)],- projectOffset: projectOffset + consumedProjects.size,+ doneRoots: completedRoots,+ projectOffset: projectOffset + consumedProjects.length,+ ...(resume ? { resume } : {}),}),},} : {}),diff --git a/skills/README.md b/skills/README.mdindex 23444409..05514f3f 100644--- a/skills/README.md+++ b/skills/README.md@@ -57,8 +57,11 @@ checks the cap against `fstat` before any bytes move.Every discovery response carries a **scan report**. `truncated:false` means "this is allof it"; hitting a cap sets `truncated:true`, names the reason, and includes-`continuation` — pending roots, per-root cursors and an opaque `cursor` to pass back and-resume. Do not conclude a skill is absent from a truncated scan.+`continuation` — pending roots, the exact interrupted position and an opaque `cursor` to+pass back. Resumption is lossless: each dirent is validated as it streams and the position+advances only after that entry is fully handled, so a cap or deadline re-reads the entry+it stopped on instead of skipping it. A partially consumed project is resumed, never+marked done. Do not conclude a skill is absent from a truncated scan.## Contract