Beautiful, AI-native markdown IDE and LLM wiki
TypeScript 2nd-brainagent-skillsclaudecodexcompany-braindocsknowledge-knowledge-baseknowledge-managementllmllm-wikillm-wiki-karpathymarkdownmarkdown-editormdmdxnotespkmsecond-brainskills
fix(ok): keep OpenKnowledge's own writes out of saved versions (#4530)
Nick Gomez1ffee5e84179
diff --git a/.changeset/saved-versions-exclude-machine-local-state.md b/.changeset/saved-versions-exclude-machine-local-state.mdnew file mode 100644index 000000000..8f45ed3db--- /dev/null+++ b/.changeset/saved-versions-exclude-machine-local-state.md@@ -0,0 +1,5 @@+---+'@inkeep/open-knowledge': patch+---++Saved versions no longer capture per-machine OpenKnowledge state or half-written files, and no longer fail when a save runs while the editor is still writing a document to disk.diff --git a/docs/content/reference/what-open-knowledge-writes.mdx b/docs/content/reference/what-open-knowledge-writes.mdxindex 8d80a2a93..d6bb67ca8 100644--- a/docs/content/reference/what-open-knowledge-writes.mdx+++ b/docs/content/reference/what-open-knowledge-writes.mdx@@ -126,7 +126,7 @@ The app runs a collaboration server for the open project. It writes runtime stat| `.ok/local/terminal/claude-settings-{mcp,tools,mcp-tools}.json` | Windows-only Claude launch settings written when **Open in Claude** requests MCP or tool preapproval; one of three possible files, overwritten rather than appended on each launch that uses that combination. macOS and Linux pass the same settings inline | In-project, gitignored || `.ok/local/comments/` | [Comments](/docs/features/comments) left on documents, one JSON file each — the note, the passage it quotes, and whether it has been sent | In-project, gitignored || `.ok/worktrees/<branch>/` | Git worktree checkout created when you open a branch in its own window (project switcher); the path is also added to `.git/info/exclude` | In-project, gitignored |-| `.git/ok/` — or `.git/worktrees/<name>/ok/` in a linked worktree, or the enclosing repo's `.git/ok-<slug>/` when your project is a subfolder of a larger repo | Shadow git repo holding per-writer work-in-progress refs; always inside a git admin dir, never in your content tree | In-project, gitignored (inside `.git/`) |+| `.git/ok/` — or `.git/worktrees/<name>/ok/` in a linked worktree, or the enclosing repo's `.git/ok-<slug>/` when your project is a subfolder of a larger repo | Shadow git repo holding per-writer work-in-progress refs; always inside a git admin dir, never in your content tree. Its own `info/exclude` file, written when OpenKnowledge opens the repo, keeps machine-local runtime state — `.ok/local/` and `.ok/worktrees/`, plus the pre-`.ok/local/` equivalents that older projects still keep at the `.ok/` root (`.ok/principal.json`, `.ok/server.lock`, `.ok/cache/`, …) — and the `.tmp.<uuid>` files an in-flight atomic write leaves behind out of every saved version, the way `.ok/.gitignore` keeps them out of git | In-project, gitignored (inside `.git/`) |Diagnostic logs and telemetry are **on by default but local-only** — credential-bearing attributes are redacted (`[REDACTED]`) before anything is written, the files rotate at the size caps above, and nothing leaves the machine until you explicitly run `ok diagnose bundle`. Turn the local sink off with `telemetry.localSink.enabled: false`; see [Configuration](./configuration.mdx).@@ -247,7 +247,7 @@ On Windows, Claude Desktop and OpenCode configs live under `%APPDATA%\Claude\` a| `.ok/local/installed-skills.json` | Per-project skill-install markers (which editors each skill is projected into) | In-project, gitignored || `.ok/local/skill-move-retained.json` | Bookkeeping for a skill move between Project and Global scope that could not finish: which destination copy was kept, why, and what the source looked like at the time. Keyed by scope and skill name, one record each. A record clears when a later move of that skill completes, when the copy at the destination stops matching the one recorded, or when you delete that skill at that scope. No document content | In-project, gitignored || `.ok/local/comments/` | [Comments](/docs/features/comments) left on documents, one JSON file each — the note, the passage it quotes, and whether it has been sent | In-project, gitignored |-| `.git/ok/` — or `.git/worktrees/<name>/ok/` in a linked worktree, or the enclosing repo's `.git/ok-<slug>/` when your project is a subfolder of a larger repo | Shadow git repo holding per-writer work-in-progress refs (powers the timeline); always inside a git admin dir, never in your content tree | In-project, gitignored (inside `.git/`) |+| `.git/ok/` — or `.git/worktrees/<name>/ok/` in a linked worktree, or the enclosing repo's `.git/ok-<slug>/` when your project is a subfolder of a larger repo | Shadow git repo holding per-writer work-in-progress refs (powers the timeline); always inside a git admin dir, never in your content tree. Its own `info/exclude` file, written when OpenKnowledge opens the repo, keeps machine-local runtime state — `.ok/local/` and `.ok/worktrees/`, plus the pre-`.ok/local/` equivalents that older projects still keep at the `.ok/` root (`.ok/principal.json`, `.ok/server.lock`, `.ok/cache/`, …) — and the `.tmp.<uuid>` files an in-flight atomic write leaves behind out of every saved version, the way `.ok/.gitignore` keeps them out of git | In-project, gitignored (inside `.git/`) |Diagnostic logs and telemetry are **on by default but local-only** — credential-bearing attributes are redacted (`[REDACTED]`) before anything is written, the files rotate at the size caps above, and nothing leaves the machine until you explicitly run `ok diagnose bundle`. Turn the local sink off with `telemetry.localSink.enabled: false`; see [Configuration](./configuration.mdx).@@ -285,6 +285,12 @@ Disable both with `OK_RECLAIM_DISABLE=1`. `ok start` does not touch skill files;</Tabs>+## Versions saved before the shadow exclusions++The shadow repo's `info/exclude` is forward-only. Versions saved by an earlier OpenKnowledge build, and the work-in-progress commits folded into them, still carry the machine-local paths in their trees: the contents of `.ok/local/`, including `principal.json` and `server.lock`, and the caches and logs beside them. A project that predates `.ok/local/` carries the same runtime state at the `.ok/` root instead — `.ok/principal.json`, `.ok/state.json`, `.ok/server.lock`, `.ok/ui.lock`, `.ok/sync-state.json`, `.ok/conflicts.json` and `.ok/last-spawn-error.log`, alongside `.ok/cache/` and `.ok/tmp/`. Opening the project on a current build drops those paths from the shadow repo's staging index and stops adding them to new versions; it does not rewrite the versions already written, and those older commits stay reachable from the versions that folded them.++All of it stays in the shadow git repo inside your own `.git/` directory, which OpenKnowledge never pushes anywhere. A version you saved yourself is kept rather than expired, so this does not age out on its own. `ok deinit` removes the shadow repo along with the rest of a project's OpenKnowledge state.+## Pi folder trustWhen you use Pi inside OpenKnowledge and accept its bridge setup prompt, OpenKnowledge also grants Pi trust in the project folder. This applies to both the desktop app and the web app served by `ok start`; `ok init` writes the project bridge without adding this trust grant.diff --git a/knip.config.ts b/knip.config.tsindex e902b1013..dd620604a 100644--- a/knip.config.ts+++ b/knip.config.ts@@ -142,15 +142,11 @@ export default {'packages/server': {entry: ['src/**/*.test.ts', 'src/parse-worker.ts'],project: 'src/**',- ignoreDependencies: [- '@types/shell-quote',- ],+ ignoreDependencies: ['@types/shell-quote'],},'packages/cli': {entry: ['src/**/*.test.ts', 'scripts/*.ts', 'tests/**/*.ts', 'src/parse-worker.ts'],- ignoreDependencies: [- 'yjs',- ],+ ignoreDependencies: ['yjs'],},'packages/desktop': {entry: [diff --git a/lint-plugins/ok-rules/README.md b/lint-plugins/ok-rules/README.mdindex 07a94b438..f42fc3c4d 100644--- a/lint-plugins/ok-rules/README.md+++ b/lint-plugins/ok-rules/README.md@@ -611,13 +611,13 @@ Rule: [`lint-plugins/ok-rules/rules/no-demoted-dialog-confirm.mjs`](rules/no-dem### `no-unconverted-git-pathspec`-Git pathspec discipline. Fires on a hand-written `'--'` string literal in an array literal, or as a direct argument of a `.raw(...)` varargs call, anywhere under `packages/{core,server,cli,desktop}/src/**` — unless the argv's leading non-flag token is one of the four verbs whose `--` operands are not pathspecs. The array branch consults no git token of any kind; see the over-catch note below. The tail after the separator is built with `pathspecArgs(paths)` from [`packages/core/src/git-pathspec.ts`](../../packages/core/src/git-pathspec.ts), which emits the separator and the `:(literal)` conversion together.+Git pathspec discipline. Fires on a hand-written `'--'` string literal in an array literal, or as a direct argument of a `.raw(...)` varargs call, anywhere under `packages/{core,server,cli,desktop}/src/**` — unless the argv's leading non-flag token is one of the five verbs whose `--` operands are not pathspecs. The array branch consults no git token of any kind; see the over-catch note below. The tail after the separator is built with `pathspecArgs(paths)` from [`packages/core/src/git-pathspec.ts`](../../packages/core/src/git-pathspec.ts), which emits the separator and the `:(literal)` conversion together.**Why.** git parses a pathspec argument position as a pattern language, and `--` does not disable that parsing: it stops option parsing only. A filesystem path handed over bare is therefore reinterpreted whenever its first path segment carries a metacharacter. PRD-8638 is what that costs: a doc named `:colon.md` fatals the push cycle, a doc named `:!bang.md` reads as exclusion magic and is silently subtracted from the set it was meant to join, and a wildcard-named skill directory makes `ls-files --error-unmatch` report an untracked file as tracked. Thirty-six `pathspecArgs` call sites across fourteen files now carry the conversion. That count is post-fix call sites, not pre-fix `--` literals: the two differ because one `sync-engine.ts` staging loop collapsed two pre-fix literals into a single call, and four `shadow-repo.ts` / `persistence.ts` sites had no `--` to count in the first place. Every converted site that did carry a `--` already had it, which is what made the omission look handled. The pattern is not a regression: three of those sites are present verbatim in the initial Copybara mirror-import commit `6bd7d8ba7d`.**Prior art in the dependency.** `[email protected]` ships a `pathspec(...)` helper (`@simple-git/[email protected]`), and its `suffixPathsPlugin` — registered unconditionally in `gitInstanceFactory` — already routes marked arguments, and everything after a `'--'`, into the command suffix and re-emits exactly one separator. The separator half of `pathspecArgs`'s contract is therefore something simple-git already guarantees for every `sg.raw(...)` site here. The helper adds no magic word, so `pathspec(':colon.md')` still reaches git bare: the `:(literal)` half, which is the half PRD-8638 is about, is not available from it. `pathspecArgs` is also package-agnostic, covering the two converted sites the plugin never sees — `rename-log.ts`'s `spawn` and `git-exclude.ts`'s `execFileSync`. The rule deliberately does **not** treat `pathspec(...)` as a non-violation, because accepting it would wave through an unconverted operand. (Same check as the `require-windowshide-on-spawn` note above, which records what `simple-git` already covers.)-**A fail-closed allowlist, not suppressions.** Four verbs take `--` operands that genuinely are not pathspecs, and the rule carries them in `NON_PATHSPEC_VERBS` rather than asking four production files to suppress it: `hash-object` takes file arguments, `clone` takes a URL and a directory, `worktree add` takes a commit-ish, and `mv` takes literal source and destination paths (verified against real git 2.49.0: `git mv -- ':(literal):colon.md' moved.md` reports `fatal: bad source, source=:(literal):colon.md, destination=moved.md`, so converting that operand would break the case-only rename it implements). This README treats an `oxlint-disable` header as a backlog marker, so a permanent legitimate use is a semantic mismatch there. The verb is resolved positionally — the first token that is not a flag, skipping the value of a `-c` / `-C` pair, so `clone` is still recognised behind a `-c core.symlinks=false` prefix — rather than by membership anywhere in the argv, so a path operand or a commit message that happens to read `mv` cannot exempt the argv it sits in. Each argv array is judged on its own: an array of argvs is not flattened into one bag, so one allowlisted row cannot disarm its siblings. The polarity is deliberate: a verb the rule has never seen fires rather than silently passing, so a new git surface is opted in by a reviewer rather than by omission.+**A fail-closed allowlist, not suppressions.** Five verbs take `--` operands that genuinely are not pathspecs, and the rule carries them in `NON_PATHSPEC_VERBS` rather than asking the production files that use them to suppress it: `hash-object` takes file arguments, `clone` takes a URL and a directory, `worktree add` takes a commit-ish, `mv` takes literal source and destination paths (verified against real git 2.49.0: `git mv -- ':(literal):colon.md' moved.md` reports `fatal: bad source, source=:(literal):colon.md, destination=moved.md`, so converting that operand would break the case-only rename it implements), and `update-index` takes literal file names (verified against real git 2.50.1: with `lit.md` staged, `git update-index --force-remove -- ':(literal)lit.md'` exits 0 and leaves `lit.md` in the index, and `git update-index --force-remove -- 'sub/*'` leaves `sub/tracked.md` in the index, so neither the magic word nor a wildcard is parsed). That verification covers `--force-remove`, the only `update-index` mode that passes a `--` separator here (the other four call sites are `--add --cacheinfo`, which carry no separator, so the rule never inspects them); the allowlist entry is nevertheless verb-wide, and `update-index --again` does parse its `--` operands as pathspecs (verified against real git 2.50.1: with `sub/tracked.md` and `other.md` both staged ahead of `HEAD` and then modified again, `git update-index --again -- 'sub/*'` refreshes `sub/tracked.md` and leaves `other.md` at its earlier blob, so the wildcard matched). Nothing calls `--again` today, so nothing is unprotected; an author who adds one needs `pathspecArgs` for its operands and will not be reminded by this rule. `update-index` still needs the separator, which is why it is on this list rather than off the rule's radar: without it, a staged path named `-foo.md.tmp.<uuid>` reports `error: unknown switch 'f'`-class option parsing instead of being removed. This README treats an `oxlint-disable` header as a backlog marker, so a permanent legitimate use is a semantic mismatch there. The verb is resolved positionally — the first token that is not a flag, skipping the value of a `-c` / `-C` pair, so `clone` is still recognised behind a `-c core.symlinks=false` prefix — rather than by membership anywhere in the argv, so a path operand or a commit message that happens to read `mv` cannot exempt the argv it sits in. Each argv array is judged on its own: an array of argvs is not flattened into one bag, so one allowlisted row cannot disarm its siblings. The polarity is deliberate: a verb the rule has never seen fires rather than silently passing, so a new git surface is opted in by a reviewer rather than by omission.**Scope** (`RULE_SCOPES['no-unconverted-git-pathspec']` in [`scope.mjs`](scope.mjs)). The four packages that build git argvs today. `git-pathspec.ts` is excluded because it is the one sanctioned emitter of the separator. Tests and test helpers are excluded per the `no-raw-html-interactive-element` precedent: fixture code that seeds a hostile filename deliberately passes a correct `:(literal)` pathspec by hand, and that is the control proving git can do the right thing, not a production argv.diff --git a/lint-plugins/ok-rules/__fixtures__/no-unconverted-git-pathspec.fixture.tsx b/lint-plugins/ok-rules/__fixtures__/no-unconverted-git-pathspec.fixture.tsxindex 0dc40b0dc..5d5113481 100644--- a/lint-plugins/ok-rules/__fixtures__/no-unconverted-git-pathspec.fixture.tsx+++ b/lint-plugins/ok-rules/__fixtures__/no-unconverted-git-pathspec.fixture.tsx@@ -2,15 +2,17 @@// part of the main lint: `__fixtures__/` is in `oxlint.config.ts#ignorePatterns`, and// `__fixtures__/oxlint.fixtures.json` re-enables the rules for the test.//-// Pairs 5 positive cases (rule MUST fire) with negative cases (rule must NOT fire). Every positive-// is a real pre-fix argv from PRD-8638 rather than a synthetic one: the bug supplied them.+// Pairs 7 positive cases (rule MUST fire) with negative cases (rule must NOT fire). P1-P5 are real+// pre-fix argv from PRD-8638 rather than synthetic ones: the bug supplied them. P6 and P7 are+// constructed; the closing paragraph says what they pin.//-// The negatives are the precision boundary, not clean code. Four are the verbs whose `--` operands-// genuinely are not pathspecs (`hash-object` takes file arguments, `clone` takes a URL and a-// directory, `worktree add` takes a commit-ish, `mv` takes literal source and destination paths) —-// these prove the in-rule allowlist works from any position in the argv and through a nested-// conditional tail. Two are `parse-command.ts`'s string comparisons against the `--` token, which-// are not argv construction at all and are the shape most likely to draw a widened pattern.+// The negatives are the precision boundary, not clean code. Six cover the five verbs whose `--`+// operands genuinely are not pathspecs (`hash-object` takes file arguments, `clone` takes a URL and+// a directory, `worktree add` takes a commit-ish, `mv` takes literal source and destination paths,+// `update-index` takes literal file names) — these prove the in-rule allowlist works from any+// position in the argv and through a nested conditional tail. Two are `parse-command.ts`'s string+// comparisons against the `--` token, which are not argv construction at all and are the shape most+// likely to draw a widened pattern.//// The fixture test asserts the diagnostic count with exact equality (`toBe(7)`) so both a weakened// pattern (drops below 7) and a widened pattern that catches a negative (rises above 7) fail the@@ -60,13 +62,14 @@ async function positives() {}async function negatives() {- // N1: `hash-object` takes file arguments, in the varargs form (shadow-repo.ts:1289).+ // N1: `hash-object` takes file arguments, in the varargs form (shadow-repo.ts#parkBranchInner).await git.raw('hash-object', '-w', '--', ...chunk);- // N2: `clone` takes a URL and a directory, and is not the first element (fetch.ts:114).+ // N2: `clone` takes a URL and a directory, and is not the first element (fetch.ts#fetchSource).await execFileAsync('git', ['-c', 'core.symlinks=false', 'clone', '-q', '--', url, tmp]);- // N3: `worktree add` takes a commit-ish (worktree-service.ts:240).+ // N3: `worktree add` takes a commit-ish (worktree-service.ts#buildAddArgs).const added = ['worktree', 'add', worktreePath, '--', branch];- // N4: the same, with the separator in a nested conditional tail (worktree-service.ts:237).+ // N4: the same, with the separator in a nested conditional tail+ // (worktree-service.ts#buildAddArgs).const created = ['worktree','add',@@ -75,18 +78,23 @@ async function negatives() {worktreePath,...(baseBranch ? ['--', baseBranch] : []),];- // N5: `git mv` takes literal path arguments, not pathspecs (api-extension.ts:1412). Verified+ // N5: `git mv` takes literal path arguments, not pathspecs+ // (api-extension.ts#renameTrackedPathInGit, which carries four calls of this shape). Verified// against real git 2.49.0: `git mv -- ':(literal):colon.md' moved.md` reports// `fatal: bad source, source=:(literal):colon.md, destination=moved.md`, so converting this// operand would break the case-only rename it implements.await git.raw('mv', '--', p, rel);- // N6: a string comparison against the token, not argv construction (parse-command.ts:55).+ // N6: `update-index` takes literal file names, in the varargs form+ // (shadow-repo.ts#dropExcludedIndexEntries).+ await git.raw('update-index', '--force-remove', '--', ...chunk);+ // N7: a string comparison against the token, not argv construction+ // (parse-command.ts#isRecursiveGrepFlag).const isFlag = arg.startsWith('--');- // N7: an equality test against the token (parse-command.ts:220).+ // N8: an equality test against the token (parse-command.ts#classifyArgs).const isSeparator = value === '--';- // N8: the converted shape the fix installs.+ // N9: the converted shape the fix installs.await git.raw(['add', ...pathspecArgs([...batch])]);- // N9: flags that merely begin with the separator's characters.+ // N10: flags that merely begin with the separator's characters.const flags = ['diff', '--name-only', '--cached'];return { added, created, isFlag, isSeparator, flags };}diff --git a/lint-plugins/ok-rules/rules/no-unconverted-git-pathspec.mjs b/lint-plugins/ok-rules/rules/no-unconverted-git-pathspec.mjsindex 6805d7c2b..1f51e199d 100644--- a/lint-plugins/ok-rules/rules/no-unconverted-git-pathspec.mjs+++ b/lint-plugins/ok-rules/rules/no-unconverted-git-pathspec.mjs@@ -2,9 +2,9 @@ import { docsUrl } from '../docs.mjs';const URL = docsUrl('no-unconverted-git-pathspec');-const MESSAGE = `Hand-written \`'--'\` in a git argv leaves the operands after it as bare paths, and git parses a pathspec position as a pattern language — \`--\` stops option parsing, not pathspec-magic parsing. Build the tail with \`pathspecArgs(paths)\` from @inkeep/open-knowledge-core, which emits the separator and the \`:(literal)\` conversion together. If this verb's \`--\` operands are not pathspecs (as for clone, hash-object, mv and worktree), add the verb to \`NON_PATHSPEC_VERBS\` in this rule rather than suppressing. See ${URL}`;+const MESSAGE = `Hand-written \`'--'\` in a git argv leaves the operands after it as bare paths, and git parses a pathspec position as a pattern language — \`--\` stops option parsing, not pathspec-magic parsing. Build the tail with \`pathspecArgs(paths)\` from @inkeep/open-knowledge-core, which emits the separator and the \`:(literal)\` conversion together. If this verb's \`--\` operands are not pathspecs (as for clone, hash-object, mv, update-index and worktree), add the verb to \`NON_PATHSPEC_VERBS\` in this rule rather than suppressing. See ${URL}`;-export const NON_PATHSPEC_VERBS = ['clone', 'hash-object', 'mv', 'worktree'];+export const NON_PATHSPEC_VERBS = ['clone', 'hash-object', 'mv', 'update-index', 'worktree'];const ALLOWED = new Set(NON_PATHSPEC_VERBS);diff --git a/packages/app/tests/integration/bug3-source-mode-writeback.test.ts b/packages/app/tests/integration/bug3-source-mode-writeback.test.tsindex a28cc2b18..8a60e5826 100644--- a/packages/app/tests/integration/bug3-source-mode-writeback.test.ts+++ b/packages/app/tests/integration/bug3-source-mode-writeback.test.ts@@ -1,4 +1,3 @@-import { readFileSync } from 'node:fs';import { join } from 'node:path';import { setTimeout as wait } from 'node:timers/promises';diff --git a/packages/app/tests/integration/indented-jsx-byte-sacred.test.ts b/packages/app/tests/integration/indented-jsx-byte-sacred.test.tsindex 4fc2c11f9..6fcdc23f2 100644--- a/packages/app/tests/integration/indented-jsx-byte-sacred.test.ts+++ b/packages/app/tests/integration/indented-jsx-byte-sacred.test.ts@@ -1,4 +1,3 @@-import { setTimeout as wait } from 'node:timers/promises';import { normalizeBridge } from '@inkeep/open-knowledge-core';import { updateYFragment } from '@tiptap/y-tiptap';diff --git a/packages/app/tests/integration/interior-edit-freshness.test.ts b/packages/app/tests/integration/interior-edit-freshness.test.tsindex 9f2f6feb6..40b5cdfd6 100644--- a/packages/app/tests/integration/interior-edit-freshness.test.ts+++ b/packages/app/tests/integration/interior-edit-freshness.test.ts@@ -1,4 +1,3 @@-import { setTimeout as wait } from 'node:timers/promises';import { updateYFragment, yXmlFragmentToProseMirrorRootNode } from '@tiptap/y-tiptap';import { afterAll, beforeAll, describe, expect, test } from 'vitest';diff --git a/packages/app/tests/integration/keystroke-cadence-danger-space.test.ts b/packages/app/tests/integration/keystroke-cadence-danger-space.test.tsindex 2a085742b..93666af70 100644--- a/packages/app/tests/integration/keystroke-cadence-danger-space.test.ts+++ b/packages/app/tests/integration/keystroke-cadence-danger-space.test.ts@@ -1,4 +1,3 @@-import { setTimeout as wait } from 'node:timers/promises';import { updateYFragment, yXmlFragmentToProseMirrorRootNode } from '@tiptap/y-tiptap';import { afterAll, beforeAll, describe, expect, test } from 'vitest';diff --git a/packages/app/tests/integration/producer-guard-wiring.test.ts b/packages/app/tests/integration/producer-guard-wiring.test.tsindex 878d5196b..867598d34 100644--- a/packages/app/tests/integration/producer-guard-wiring.test.ts+++ b/packages/app/tests/integration/producer-guard-wiring.test.ts@@ -1,4 +1,3 @@-import { setTimeout as wait } from 'node:timers/promises';import { updateYFragment, yXmlFragmentToProseMirrorRootNode } from '@tiptap/y-tiptap';import { afterAll, beforeAll, describe, expect, test } from 'vitest';diff --git a/packages/app/tests/integration/qa-039-reconciliation-counters-http.test.ts b/packages/app/tests/integration/qa-039-reconciliation-counters-http.test.tsindex 4ada3cf26..0970ac12f 100644--- a/packages/app/tests/integration/qa-039-reconciliation-counters-http.test.ts+++ b/packages/app/tests/integration/qa-039-reconciliation-counters-http.test.ts@@ -1,4 +1,3 @@-import { randomUUID } from 'node:crypto';import { afterEach, describe, expect, test } from 'vitest';import { agentWriteMd, createTestServer, type TestServer } from './test-harness';diff --git a/packages/app/tests/integration/qa-045-restore-after-gc.test.ts b/packages/app/tests/integration/qa-045-restore-after-gc.test.tsindex cbe2b705f..b632185e1 100644--- a/packages/app/tests/integration/qa-045-restore-after-gc.test.ts+++ b/packages/app/tests/integration/qa-045-restore-after-gc.test.ts@@ -1,4 +1,3 @@-import { execFileSync } from 'node:child_process';import { randomUUID } from 'node:crypto';import { join } from 'node:path';diff --git a/packages/app/tests/integration/qa-050-rollback-undo-shipped-path.test.ts b/packages/app/tests/integration/qa-050-rollback-undo-shipped-path.test.tsindex 89bc9b4d9..c530ce4ba 100644--- a/packages/app/tests/integration/qa-050-rollback-undo-shipped-path.test.ts+++ b/packages/app/tests/integration/qa-050-rollback-undo-shipped-path.test.ts@@ -1,4 +1,3 @@-import { randomUUID } from 'node:crypto';import { afterAll, afterEach, beforeAll, describe, expect, test } from 'vitest';import type * as Y from 'yjs';diff --git a/packages/app/tests/integration/undo-after-rollback.test.ts b/packages/app/tests/integration/undo-after-rollback.test.tsindex a78aeb0e4..d3767adc9 100644--- a/packages/app/tests/integration/undo-after-rollback.test.ts+++ b/packages/app/tests/integration/undo-after-rollback.test.ts@@ -1,4 +1,3 @@-import { ROLLBACK_ORIGIN } from '@inkeep/open-knowledge-server';import { afterAll, beforeAll, describe, expect, test } from 'vitest';import * as Y from 'yjs';diff --git a/packages/app/tests/integration/undo-recycle-reset.test.ts b/packages/app/tests/integration/undo-recycle-reset.test.tsindex e7cc69c3f..b076f955c 100644--- a/packages/app/tests/integration/undo-recycle-reset.test.ts+++ b/packages/app/tests/integration/undo-recycle-reset.test.ts@@ -1,4 +1,3 @@-import { writeFileSync } from 'node:fs';import { join } from 'node:path';import { setTimeout as wait } from 'node:timers/promises';diff --git a/packages/app/tests/integration/undo-rederive-survival.test.ts b/packages/app/tests/integration/undo-rederive-survival.test.tsindex 9ddcd0730..7935e35f8 100644--- a/packages/app/tests/integration/undo-rederive-survival.test.ts+++ b/packages/app/tests/integration/undo-rederive-survival.test.ts@@ -1,4 +1,3 @@-import { prependFrontmatter } from '@inkeep/open-knowledge-core';import { Plugin, PluginKey } from '@tiptap/pm/state';import { updateYFragment, yXmlFragmentToProseMirrorRootNode } from '@tiptap/y-tiptap';diff --git a/packages/app/tests/lint-plugins/no-unconverted-git-pathspec.test.ts b/packages/app/tests/lint-plugins/no-unconverted-git-pathspec.test.tsindex a652fab31..0a4903dbb 100644--- a/packages/app/tests/lint-plugins/no-unconverted-git-pathspec.test.ts+++ b/packages/app/tests/lint-plugins/no-unconverted-git-pathspec.test.ts@@ -67,6 +67,12 @@ describe('no-unconverted-git-pathspec oxlint rule', () => {});test('exempts every verb whose "--" operands are not pathspecs, and nothing else', () => {- expect([...NON_PATHSPEC_VERBS].sort()).toEqual(['clone', 'hash-object', 'mv', 'worktree']);+ expect([...NON_PATHSPEC_VERBS].sort()).toEqual([+ 'clone',+ 'hash-object',+ 'mv',+ 'update-index',+ 'worktree',+ ]);});});diff --git a/packages/cli/src/content/lint-runner.ts b/packages/cli/src/content/lint-runner.tsindex 7fdcca0ed..42958fd4a 100644--- a/packages/cli/src/content/lint-runner.ts+++ b/packages/cli/src/content/lint-runner.ts@@ -18,6 +18,7 @@ import {SUPPORTED_DOC_EXTENSIONS,summarizeLintPluginFailures,} from '@inkeep/open-knowledge-core';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import {composeEffectiveLinterConfig,composeFrontmatterSchemasConfig,@@ -136,7 +137,7 @@ export async function runLint(opts: RunLintOptions): Promise<LintRunResult> {if (fix && cfg.enabled) {const fixedText = fixDocument(text, cfg, rel, pushPluginFailure);if (fixedText !== text) {- const tmp = `${abs}.tmp.${process.pid}.${Date.now()}`;+ const tmp = atomicTempPath(abs);try {writeFileSync(tmp, fixedText, 'utf-8');renameSync(tmp, abs);diff --git a/packages/core/src/bridge/compose.ts b/packages/core/src/bridge/compose.tsindex 2a597ec3d..f4cc8a286 100644--- a/packages/core/src/bridge/compose.ts+++ b/packages/core/src/bridge/compose.ts@@ -1,4 +1,3 @@-import { prependFrontmatter, stripFrontmatter } from '../extensions/frontmatter.ts';export type ComposeAdjustment = 'none' | 'body' | 'frontmatter' | 'unresolved';diff --git a/packages/core/src/bridge/doc-boundary-space.ts b/packages/core/src/bridge/doc-boundary-space.tsindex 01ef76a9f..8fb428b90 100644--- a/packages/core/src/bridge/doc-boundary-space.ts+++ b/packages/core/src/bridge/doc-boundary-space.ts@@ -1,4 +1,3 @@-import { FRONTMATTER_RE, stripFrontmatter } from '../extensions/frontmatter.ts';import { carriedEdgeEmpties, type DocEdgeEmpties } from '../markdown/doc-edge-blank-runs.ts';diff --git a/packages/core/src/constants/ok-dir.ts b/packages/core/src/constants/ok-dir.tsindex 054fda883..d048e3564 100644--- a/packages/core/src/constants/ok-dir.ts+++ b/packages/core/src/constants/ok-dir.ts@@ -4,10 +4,39 @@ export const OK_PROJECT_MARKER = '.ok/config.yml';export const LOCAL_DIR = 'local';+export const WORKTREES_DIRNAME = 'worktrees';+export const SAVED_THEMES_DIRNAME = 'themes';export const OK_BIN_DIRNAME = 'bin';+export const OK_MACHINE_LOCAL_ROOT_FILES = [+ 'principal.json',+ 'state.json',+ 'server.lock',+ 'ui.lock',+ 'sync-state.json',+ 'conflicts.json',+ 'last-spawn-error.log',+] as const;++export const OK_ACTIVE_MACHINE_LOCAL_ROOT_DIRS = [LOCAL_DIR, WORKTREES_DIRNAME] as const;++export const OK_MACHINE_LOCAL_ROOT_DIRS = [+ ...OK_ACTIVE_MACHINE_LOCAL_ROOT_DIRS,+ 'cache',+ 'tmp',+] as const;++export type OkMachineLocalRootDir = (typeof OK_MACHINE_LOCAL_ROOT_DIRS)[number];++export type OkMachineLocalRootFile = (typeof OK_MACHINE_LOCAL_ROOT_FILES)[number];++const activeMachineLocalRootDirs: ReadonlySet<string> = new Set(OK_ACTIVE_MACHINE_LOCAL_ROOT_DIRS);++export const OK_LEGACY_MACHINE_LOCAL_ROOT_DIRS: readonly OkMachineLocalRootDir[] =+ OK_MACHINE_LOCAL_ROOT_DIRS.filter((name) => !activeMachineLocalRootDirs.has(name));+export function posixOkManagedBinDir(homeDir: string): string {const base = homeDir.replace(/\/+/g, '/').replace(/\/+$/, '');return `${base}/${OK_DIR}/${OK_BIN_DIRNAME}`;diff --git a/packages/core/src/git/worktree-path.ts b/packages/core/src/git/worktree-path.tsindex 497c31011..85a6149b0 100644--- a/packages/core/src/git/worktree-path.ts+++ b/packages/core/src/git/worktree-path.ts@@ -1,4 +1,6 @@-export const WORKTREES_PARENT_DIR = '.ok/worktrees';+import { OK_DIR, WORKTREES_DIRNAME } from '../constants/ok-dir.ts';++export const WORKTREES_PARENT_DIR = `${OK_DIR}/${WORKTREES_DIRNAME}`;export function worktreeRelativeDir(branch: string): string | null {const trimmed = branch.trim();diff --git a/packages/core/src/index.ts b/packages/core/src/index.tsindex 8273f324c..31345780b 100644--- a/packages/core/src/index.ts+++ b/packages/core/src/index.ts@@ -497,10 +497,17 @@ export {} from './constants/native-menu-labels.ts';export {LOCAL_DIR,+ OK_ACTIVE_MACHINE_LOCAL_ROOT_DIRS,OK_DIR,+ OK_LEGACY_MACHINE_LOCAL_ROOT_DIRS,+ OK_MACHINE_LOCAL_ROOT_DIRS,+ OK_MACHINE_LOCAL_ROOT_FILES,OK_PROJECT_MARKER,+ type OkMachineLocalRootDir,+ type OkMachineLocalRootFile,posixOkManagedBinDir,SAVED_THEMES_DIRNAME,+ WORKTREES_DIRNAME,} from './constants/ok-dir.ts';export {PREVIEW_EMBED_STARTERS,diff --git a/packages/core/src/markdown/comment-promoter.ts b/packages/core/src/markdown/comment-promoter.tsindex 20a68d2a1..bfbd5a2e8 100644--- a/packages/core/src/markdown/comment-promoter.ts+++ b/packages/core/src/markdown/comment-promoter.ts@@ -1,4 +1,3 @@-import type { Nodes, Paragraph, PhrasingContent, Root, RootContent, Text } from 'mdast';import { SKIP, visit } from 'unist-util-visit';import type { VFile } from 'vfile';diff --git a/packages/core/src/markdown/merged-walker.ts b/packages/core/src/markdown/merged-walker.tsindex fa3cfcd86..bfd7e3109 100644--- a/packages/core/src/markdown/merged-walker.ts+++ b/packages/core/src/markdown/merged-walker.ts@@ -1,4 +1,3 @@-import type { Nodes, Parent, Root } from 'mdast';import { SKIP, visit } from 'unist-util-visit';import type { VFile } from 'vfile';diff --git a/packages/core/src/markdown/safe-url.ts b/packages/core/src/markdown/safe-url.tsindex 8be632ba4..4d3250d3a 100644--- a/packages/core/src/markdown/safe-url.ts+++ b/packages/core/src/markdown/safe-url.ts@@ -1,4 +1,3 @@-export const SAFE_URL_SCHEMES = ['https', 'http', 'mailto', 'tel', 'ftp', 'sms'] as const;const SCHEME_ALT = SAFE_URL_SCHEMES.map((s) => `${s}:`).join('|');diff --git a/packages/core/src/server.ts b/packages/core/src/server.tsindex 5e281b572..437e1345f 100644--- a/packages/core/src/server.ts+++ b/packages/core/src/server.ts@@ -31,6 +31,12 @@ export {type NodeLocaleSignal,readNodeLocaleSignal,} from './i18n/node-locale-provider.ts';+export {+ ATOMIC_TEMP_GLOB,+ ATOMIC_TEMP_INFIX,+ ATOMIC_TEMP_PATH_RE,+ atomicTempPath,+} from './util/atomic-temp-path.ts';export {type AtomicWriteFsAdapter,type AtomicWriteOptions,diff --git a/packages/core/src/util/atomic-temp-path.test.ts b/packages/core/src/util/atomic-temp-path.test.tsnew file mode 100644index 000000000..fda981aad--- /dev/null+++ b/packages/core/src/util/atomic-temp-path.test.ts@@ -0,0 +1,172 @@+import { execFileSync, spawnSync } from 'node:child_process';+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { afterAll, beforeAll, describe, expect, test } from 'vitest';+import {+ ATOMIC_TEMP_GLOB,+ ATOMIC_TEMP_INFIX,+ ATOMIC_TEMP_PATH_RE,+ atomicTempPath,+} from './atomic-temp-path.ts';++const UUID = '0123abcd-0123-0123-0123-0123456789ab';+const PRODUCER_BASE = 'notes/a.md';+const PRODUCED_SUFFIX = atomicTempPath(PRODUCER_BASE).slice(PRODUCER_BASE.length);++const MATCHING: ReadonlyArray<readonly [string, string]> = [+ [`notes/a.md${PRODUCED_SUFFIX}`, 'suffix taken from a real atomicTempPath() call'],+ [`notes/a.md.tmp.${UUID}`, 'hand-written instance of the same shape'],+ [`notes/a b.md.tmp.${UUID}`, 'space in the basename'],+ [`dir/.tmp.${UUID}`, 'whole basename is the infix plus uuid'],+];++const REJECTED: ReadonlyArray<readonly [string, string]> = [+ ['notes/a.md', 'no temp infix at all'],+ [`notes/a.md.tmpX${UUID}`, 'non-dot character in the second dot position'],+ [`notes/a.md.tmp_${UUID}`, 'underscore in the second dot position'],+ [`notes/a.mdXtmp.${UUID}`, 'non-dot character in the first dot position'],+ ['notes/a.md.tmp.12345.1699999999999', 'the .tmp.<pid>.<ms> shape other writers produced'],+ [`notes/a.md.tmp.${UUID}.bak`, 'trailing suffix after the uuid'],+ ['notes/a.md.tmp.', 'infix with nothing after it'],+ ['notes/a.md.tmp.0123abc-0123-0123-0123-0123456789ab', 'first hex group one short'],+ ['notes/a.md.tmp.0123abcde-0123-0123-0123-0123456789ab', 'first hex group one long'],+ ['notes/a.md.tmp.0123abcd-0123-0123-0123-0123456789abc', 'last hex group one long'],+ ['notes/a.md.tmp.0123abcd-0123-0123-0123456789ab', 'one hex group missing'],+ ['notes/a.md.tmp.0123abcz-0123-0123-0123-0123456789ab', 'non-hex character in a group'],+ ['notes/a.md.tmp.0123ABCD-0123-0123-0123-0123456789ab', 'uppercase hex'],+ [+ 'notes/a.md.tmp.0123abcd_0123-0123-0123-0123456789ab',+ 'underscore where a group separator belongs',+ ],+];++const LEAF_CANDIDATES: readonly string[] = [+ ...MATCHING.map(([path]) => path),+ ...REJECTED.map(([path]) => path),+];++const DESCENDANT_OF_TEMP_DIR = `notes/a.md.tmp.${UUID}/inner.md`;+const UPPERCASE_HEX = 'notes/a.md.tmp.0123ABCD-0123-0123-0123-0123456789ab';++let repoDir: string;++function gitEnv(): NodeJS.ProcessEnv {+ const env: NodeJS.ProcessEnv = {+ ...process.env,+ GIT_CONFIG_GLOBAL: '/dev/null',+ GIT_CONFIG_SYSTEM: '/dev/null',+ };+ delete env.GIT_DIR;+ delete env.GIT_WORK_TREE;+ delete env.GIT_INDEX_FILE;+ return env;+}++function globVerdicts(paths: readonly string[], ignoreCase: boolean): Map<string, boolean> {+ const run = spawnSync(+ 'git',+ [+ '-c',+ `core.ignoreCase=${ignoreCase}`,+ 'check-ignore',+ '--verbose',+ '--non-matching',+ '-z',+ '--stdin',+ ],+ { cwd: repoDir, env: gitEnv(), encoding: 'utf-8', input: `${paths.join('\0')}\0` },+ );+ if (run.status !== 0 && run.status !== 1) {+ throw new Error(`git check-ignore failed (status ${run.status}): ${run.stderr}`);+ }+ const fields = run.stdout.split('\0');+ fields.pop();+ expect(fields.length).toBe(paths.length * 4);+ const verdicts = new Map<string, boolean>();+ for (let i = 0; i < fields.length; i += 4) {+ verdicts.set(fields[i + 3] ?? '', (fields[i] ?? '') !== '');+ }+ expect(verdicts.size).toBe(paths.length);+ return verdicts;+}++beforeAll(() => {+ repoDir = mkdtempSync(join(tmpdir(), 'ok-atomic-temp-glob-'));+ execFileSync('git', ['init', '-q'], { cwd: repoDir, stdio: 'pipe', env: gitEnv() });+ writeFileSync(join(repoDir, '.git', 'info', 'exclude'), `${ATOMIC_TEMP_GLOB}\n`);+});++afterAll(() => {+ if (repoDir) rmSync(repoDir, { recursive: true, force: true });+});++describe('ATOMIC_TEMP_PATH_RE', () => {+ for (const [path, why] of MATCHING)+ test(`matches (${why}): ${JSON.stringify(path)}`, () =>+ expect(ATOMIC_TEMP_PATH_RE.test(path)).toBe(true));++ for (const [path, why] of REJECTED)+ test(`rejects (${why}): ${JSON.stringify(path)}`, () =>+ expect(ATOMIC_TEMP_PATH_RE.test(path)).toBe(false));++ test('repeated calls on the same path return the same verdict', () => {+ const path = `notes/a.md.tmp.${UUID}`;+ expect([+ ATOMIC_TEMP_PATH_RE.test(path),+ ATOMIC_TEMP_PATH_RE.test(path),+ ATOMIC_TEMP_PATH_RE.test(path),+ ]).toEqual([true, true, true]);+ });+});++describe('atomicTempPath', () => {+ test('appends to the given path instead of replacing any part of it', () => {+ const given = '/Users/x/Project/notes/My Note.md';+ const produced = atomicTempPath(given);+ expect(produced.slice(0, given.length)).toBe(given);+ expect(produced.slice(given.length).startsWith(ATOMIC_TEMP_INFIX)).toBe(true);+ expect(ATOMIC_TEMP_PATH_RE.test(produced)).toBe(true);+ });++ test('leaves a basename that already contains the infix intact', () => {+ const given = '/Users/x/Project/notes/report.tmp.md';+ const produced = atomicTempPath(given);+ expect(produced.slice(0, given.length)).toBe(given);+ expect(ATOMIC_TEMP_PATH_RE.test(produced)).toBe(true);+ });++ test('two calls for the same target do not collide', () => {+ const given = '/Users/x/Project/notes/a.md';+ expect(atomicTempPath(given)).not.toBe(atomicTempPath(given));+ });+});++describe('ATOMIC_TEMP_GLOB and ATOMIC_TEMP_PATH_RE describe the same shape', () => {+ test('git and the regex return the same verdict for every leaf candidate', () => {+ const verdicts = globVerdicts(LEAF_CANDIDATES, false);+ const byGit = LEAF_CANDIDATES.map((path) => `${path} -> ${verdicts.get(path)}`);+ const byRegex = LEAF_CANDIDATES.map((path) => `${path} -> ${ATOMIC_TEMP_PATH_RE.test(path)}`);+ expect(byGit).toEqual(byRegex);+ });++ test('no candidate is force-removed by the regex while git would still track it', () => {+ const all = [...LEAF_CANDIDATES, DESCENDANT_OF_TEMP_DIR];+ for (const ignoreCase of [false, true]) {+ const verdicts = globVerdicts(all, ignoreCase);+ const regexOnly = all.filter((path) => ATOMIC_TEMP_PATH_RE.test(path) && !verdicts.get(path));+ expect(regexOnly).toEqual([]);+ }+ });++ test('known one-way breadth: git also excludes descendants of a temp-named directory', () => {+ expect(globVerdicts([DESCENDANT_OF_TEMP_DIR], false).get(DESCENDANT_OF_TEMP_DIR)).toBe(true);+ expect(ATOMIC_TEMP_PATH_RE.test(DESCENDANT_OF_TEMP_DIR)).toBe(false);+ });++ test('known one-way breadth: a case-insensitive repo folds the hex classes, the regex does not', () => {+ expect(globVerdicts([UPPERCASE_HEX], true).get(UPPERCASE_HEX)).toBe(true);+ expect(globVerdicts([UPPERCASE_HEX], false).get(UPPERCASE_HEX)).toBe(false);+ expect(ATOMIC_TEMP_PATH_RE.test(UPPERCASE_HEX)).toBe(false);+ });+});diff --git a/packages/core/src/util/atomic-temp-path.ts b/packages/core/src/util/atomic-temp-path.tsnew file mode 100644index 000000000..7b611db1a--- /dev/null+++ b/packages/core/src/util/atomic-temp-path.ts@@ -0,0 +1,23 @@+import { randomUUID } from 'node:crypto';++export const ATOMIC_TEMP_INFIX = '.tmp.';++const UUID_HEX_GROUP_LENGTHS = [8, 4, 4, 4, 12] as const;++function escapeRegExpLiteral(literal: string): string {+ return literal.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');+}++const UUID_HEX_GLOB = UUID_HEX_GROUP_LENGTHS.map((length) => '[0-9a-f]'.repeat(length)).join('-');++const UUID_HEX_PATTERN = UUID_HEX_GROUP_LENGTHS.map((length) => `[0-9a-f]{${length}}`).join('-');++export const ATOMIC_TEMP_GLOB = `*${ATOMIC_TEMP_INFIX}${UUID_HEX_GLOB}`;++export const ATOMIC_TEMP_PATH_RE = new RegExp(+ `${escapeRegExpLiteral(ATOMIC_TEMP_INFIX)}${UUID_HEX_PATTERN}$`,+);++export function atomicTempPath(absPath: string): string {+ return `${absPath}${ATOMIC_TEMP_INFIX}${randomUUID()}`;+}diff --git a/packages/core/src/util/atomic-yaml-write.ts b/packages/core/src/util/atomic-yaml-write.tsindex 85fe7a8af..3ccd87276 100644--- a/packages/core/src/util/atomic-yaml-write.ts+++ b/packages/core/src/util/atomic-yaml-write.ts@@ -7,6 +7,7 @@ import {writeFile as nodeWriteFile,} from 'node:fs/promises';import { basename, dirname, join } from 'node:path';+import { ATOMIC_TEMP_INFIX, atomicTempPath } from './atomic-temp-path.ts';const STALE_TMP_AGE_MS = 30_000;@@ -37,7 +38,7 @@ export interface AtomicWriteSyncOptions {async function sweepStaleTmps(absPath: string): Promise<void> {try {const parent = dirname(absPath);- const prefix = `${basename(absPath)}.tmp.`;+ const prefix = `${basename(absPath)}${ATOMIC_TEMP_INFIX}`;const cutoff = Date.now() - STALE_TMP_AGE_MS;const entries = await nodeReaddir(parent);await Promise.all(@@ -56,7 +57,7 @@ async function sweepStaleTmps(absPath: string): Promise<void> {function sweepStaleTmpsSync(absPath: string): void {try {const parent = dirname(absPath);- const prefix = `${basename(absPath)}.tmp.`;+ const prefix = `${basename(absPath)}${ATOMIC_TEMP_INFIX}`;const cutoff = Date.now() - STALE_TMP_AGE_MS;for (const name of readdirSync(parent)) {if (!name.startsWith(prefix)) continue;@@ -76,7 +77,7 @@ export async function atomicWriteFile(): Promise<void> {if (opts.sweepStaleTmps !== false) await sweepStaleTmps(absPath);const fs = opts.fs ?? DEFAULT_FS;- const tmpPath = `${absPath}.tmp.${crypto.randomUUID()}`;+ const tmpPath = atomicTempPath(absPath);try {await fs.writeFile(tmpPath, content, { encoding: 'utf-8', mode: opts.mode ?? 0o644 });await fs.rename(tmpPath, absPath);@@ -94,7 +95,7 @@ export function atomicWriteFileSync(opts: AtomicWriteSyncOptions = {},): void {sweepStaleTmpsSync(absPath);- const tmpPath = `${absPath}.tmp.${crypto.randomUUID()}`;+ const tmpPath = atomicTempPath(absPath);try {writeFileSync(tmpPath, content, { encoding: 'utf-8', mode: opts.mode ?? 0o644 });renameSync(tmpPath, absPath);diff --git a/packages/server/src/boot.ts b/packages/server/src/boot.tsindex 2225cced6..fa66df730 100644--- a/packages/server/src/boot.ts+++ b/packages/server/src/boot.ts@@ -10,6 +10,8 @@ import {isLoopbackOnlyBind,LOCAL_DIR,OK_DIR,+ OK_LEGACY_MACHINE_LOCAL_ROOT_DIRS,+ OK_MACHINE_LOCAL_ROOT_FILES,requiresExternalConsent,resolveServerRuntimeConfig,type ServerRuntimeConfig,@@ -70,18 +72,6 @@ import {teardownToleranceTelemetryWriter,} from './tolerance-telemetry-writer.ts';-const LEGACY_RUNTIME_FILENAMES = [- 'server.lock',- 'ui.lock',- 'state.json',- 'principal.json',- 'sync-state.json',- 'conflicts.json',- 'last-spawn-error.log',-] as const;--const LEGACY_RUNTIME_DIRNAMES = ['cache', 'tmp'] as const;-export function findLegacyRuntimeFiles(okDir: string): string[] {const localDir = resolve(okDir, LOCAL_DIR);const localDirEmpty = (() => {@@ -95,10 +85,10 @@ export function findLegacyRuntimeFiles(okDir: string): string[] {if (!localDirEmpty) return [];const found: string[] = [];- for (const name of LEGACY_RUNTIME_FILENAMES) {+ for (const name of OK_MACHINE_LOCAL_ROOT_FILES) {if (existsSync(resolve(okDir, name))) found.push(name);}- for (const name of LEGACY_RUNTIME_DIRNAMES) {+ for (const name of OK_LEGACY_MACHINE_LOCAL_ROOT_DIRS) {const candidate = resolve(okDir, name);try {if (existsSync(candidate) && statSync(candidate).isDirectory()) {diff --git a/packages/server/src/config-persistence.ts b/packages/server/src/config-persistence.tsindex edf7c453a..4cba06649 100644--- a/packages/server/src/config-persistence.ts+++ b/packages/server/src/config-persistence.ts@@ -15,6 +15,7 @@ import {withConfigSpanSync,} from '@inkeep/open-knowledge-core';import {+ atomicTempPath,FileLockTimeoutError,resolveConfigPath,withFileLock,@@ -211,7 +212,7 @@ export function loadConfigDoc(async function atomicWriteConfig(absPath: string, content: string): Promise<void> {await tracedMkdir(dirname(absPath), { recursive: true });- const tmpPath = `${absPath}.tmp.${crypto.randomUUID()}`;+ const tmpPath = atomicTempPath(absPath);try {await tracedWriteFile(tmpPath, content, 'utf-8');await tracedRename(tmpPath, absPath);diff --git a/packages/server/src/content/folder-frontmatter-write.ts b/packages/server/src/content/folder-frontmatter-write.tsindex affba9543..ee3f21a08 100644--- a/packages/server/src/content/folder-frontmatter-write.ts+++ b/packages/server/src/content/folder-frontmatter-write.ts@@ -9,6 +9,7 @@ import {writeFileSync,} from 'node:fs';import { isAbsolute, join, resolve, sep } from 'node:path';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import { parse as parseYaml, stringify as stringifyYaml } from 'yaml';import { checkSymlinkLeaf } from '../fs-safety.ts';import { type FrontmatterRecord, mergePatch } from './frontmatter-merge.ts';@@ -92,7 +93,7 @@ export function applyFolderFrontmatterPatch(mkdirSync(okDir, { recursive: true });const yaml = stringifyYaml(merged);- const tmpPath = `${fmPath}.tmp.${process.pid}.${Date.now()}`;+ const tmpPath = atomicTempPath(fmPath);writeFileSync(tmpPath, yaml, 'utf-8');renameSync(tmpPath, fmPath);return { ok: true, path: relPathOf(contentAbs, fmPath), action: 'written' };diff --git a/packages/server/src/content/skills-write.ts b/packages/server/src/content/skills-write.tsindex 17c9c49d7..d6ea6a52f 100644--- a/packages/server/src/content/skills-write.ts+++ b/packages/server/src/content/skills-write.ts@@ -9,6 +9,7 @@ import {type SkillAuthoringWarningCode,type SkillFrontmatter,} from '@inkeep/open-knowledge-core';+import { ATOMIC_TEMP_INFIX, atomicTempPath } from '@inkeep/open-knowledge-core/server';import { stringify as stringifyYaml } from 'yaml';/** STOP: every disk write in this module goes through `fs-traced.ts` so it@@ -110,7 +111,7 @@ export function composeSkillContent(input: {function sweepStaleTmpSiblings(targetPath: string): void {const dir = dirname(targetPath);- const prefix = `${basename(targetPath)}.tmp.`;+ const prefix = `${basename(targetPath)}${ATOMIC_TEMP_INFIX}`;try {for (const entry of readdirSync(dir)) {if (entry.startsWith(prefix)) tracedUnlinkSync(join(dir, entry));@@ -147,7 +148,7 @@ export function applySkillWrite(input: WriteSkillInput): SkillWriteResult {const created = !existsSync(filePath);sweepStaleTmpSiblings(filePath);- const tmpPath = `${filePath}.tmp.${process.pid}.${Date.now()}`;+ const tmpPath = atomicTempPath(filePath);try {tracedWriteFileSync(tmpPath, content, 'utf-8');tracedRenameSync(tmpPath, filePath);@@ -200,7 +201,7 @@ export function applySkillDirNameSync(input: {const renamed = applyPatchToFm(fenced, { name: input.toName });if (!renamed.ok) return { ok: false, stage: 'patch', error: renamed.error };sweepStaleTmpSiblings(filePath);- const tmpPath = `${filePath}.tmp.${process.pid}.${Date.now()}`;+ const tmpPath = atomicTempPath(filePath);try {tracedWriteFileSync(tmpPath, `${renamed.nextFenced}${body}`, 'utf-8');tracedRenameSync(tmpPath, filePath);@@ -360,7 +361,7 @@ export function applySkillBundleFileWrite(};}sweepStaleTmpSiblings(abs);- const tmpPath = `${abs}.tmp.${process.pid}.${Date.now()}`;+ const tmpPath = atomicTempPath(abs);try {if (typeof payload === 'string') tracedWriteFileSync(tmpPath, payload, 'utf-8');else tracedWriteFileSync(tmpPath, payload);diff --git a/packages/server/src/content/templates-write.ts b/packages/server/src/content/templates-write.tsindex 74a8f6a9b..57fc17492 100644--- a/packages/server/src/content/templates-write.ts+++ b/packages/server/src/content/templates-write.ts@@ -16,6 +16,7 @@ import {type TemplateIdentity,unwrapFrontmatterFences,} from '@inkeep/open-knowledge-core';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import { validateSubstitution } from './substitution.ts';type TemplateWriteResult =@@ -146,7 +147,7 @@ export function applyTemplateWrite(input: WriteTemplateInput): TemplateWriteResuconst created = !existsSync(filePath);- const tmpPath = `${filePath}.tmp.${process.pid}.${Date.now()}`;+ const tmpPath = atomicTempPath(filePath);try {writeFileSync(tmpPath, content, 'utf-8');renameSync(tmpPath, filePath);diff --git a/packages/server/src/document-durability-state.ts b/packages/server/src/document-durability-state.tsindex 047c2f950..49671ed5f 100644--- a/packages/server/src/document-durability-state.ts+++ b/packages/server/src/document-durability-state.ts@@ -1,6 +1,6 @@-import { randomUUID } from 'node:crypto';import { existsSync, readFileSync } from 'node:fs';import { dirname } from 'node:path';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import {tracedMkdirSync,tracedRenameSync,@@ -622,7 +622,7 @@ export class DocumentDurabilityState {}const serialized = parts.join('');tracedMkdirSync(dirname(this.persistencePath), { recursive: true });- const temporaryPath = `${this.persistencePath}.tmp.${randomUUID()}`;+ const temporaryPath = atomicTempPath(this.persistencePath);try {tracedWriteFileSync(temporaryPath, serialized, {encoding: 'utf-8',diff --git a/packages/server/src/init-project.test.ts b/packages/server/src/init-project.test.tsindex 8b671dfdc..07a28b59e 100644--- a/packages/server/src/init-project.test.ts+++ b/packages/server/src/init-project.test.ts@@ -11,7 +11,11 @@ import {} from 'node:fs';import { tmpdir } from 'node:os';import { dirname, join, resolve } from 'node:path';-import { OK_DIR } from '@inkeep/open-knowledge-core';+import {+ OK_DIR,+ OK_MACHINE_LOCAL_ROOT_DIRS,+ OK_MACHINE_LOCAL_ROOT_FILES,+} from '@inkeep/open-knowledge-core';import { afterEach, beforeEach, describe, expect, it } from 'vitest';import {buildConfigYmlContent,@@ -130,7 +134,7 @@ describe('initContent', () => {const after = readFileSync(join(okDir, '.gitignore'), 'utf-8');expect(after).toBe(- `cache/\nserver.lock\nui.lock\nsync-state.json\nlocal/\nworktrees/\nprincipal.json\nstate.json\nlast-spawn-error.log\n`,+ `cache/\nserver.lock\nui.lock\nsync-state.json\nlocal/\nworktrees/\ntmp/\nprincipal.json\nstate.json\nconflicts.json\nlast-spawn-error.log\n`,);expect(result.updated).toContain('.gitignore');expect(result.created).not.toContain('.gitignore');@@ -202,6 +206,21 @@ describe('initContent', () => {});});+ it('ignores every machine-local name the product can leave at the .ok/ root', () => {+ initContent(testDir);+ const lines = readFileSync(join(testDir, OK_DIR, '.gitignore'), 'utf-8')+ .split('\n')+ .map((l) => l.trim());++ for (const name of OK_MACHINE_LOCAL_ROOT_FILES) {+ expect(lines, `${name} must be ignored`).toContain(name);+ }+ for (const name of OK_MACHINE_LOCAL_ROOT_DIRS) {+ expect(lines, `${name}/ must be ignored`).toContain(`${name}/`);+ }+ expect(lines).not.toContain('config.yml');+ });+it('does not duplicate .gitignore entries on repeated initContent calls', () => {initContent(testDir);initContent(testDir);diff --git a/packages/server/src/init-project.ts b/packages/server/src/init-project.tsindex 930ee6599..092bdf652 100644--- a/packages/server/src/init-project.ts+++ b/packages/server/src/init-project.ts@@ -1,7 +1,14 @@-import { randomUUID } from 'node:crypto';import { existsSync, lstatSync, readFileSync } from 'node:fs';import { join, resolve } from 'node:path';-import { CONFIG_SCHEMA_MAJOR_PATH, LOCAL_DIR, OK_DIR } from '@inkeep/open-knowledge-core';+import {+ CONFIG_SCHEMA_MAJOR_PATH,+ LOCAL_DIR,+ OK_DIR,+ OK_LEGACY_MACHINE_LOCAL_ROOT_DIRS,+ OK_MACHINE_LOCAL_ROOT_FILES,+ WORKTREES_DIRNAME,+} from '@inkeep/open-knowledge-core';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import {tracedMkdirSync,tracedRenameSync,@@ -12,7 +19,18 @@ import { assertNotHomeProjectRoot } from './home-project-root.ts';export const CONFIG_FILENAME = 'config.yml';-function assertNotSymlink(filePath: string, label: string): void {+export class SymlinkRefusedError extends Error {+ constructor(filePath: string, label: string) {+ super(+ `Refusing to follow symlink at ${label} (${filePath}). ` ++ `An untrusted upstream may have committed this symlink to redirect writes outside the project. ` ++ `Remove the symlink and re-run.`,+ );+ this.name = 'SymlinkRefusedError';+ }+}++export function assertNotSymlink(filePath: string, label: string): void {let lst: ReturnType<typeof lstatSync>;try {lst = lstatSync(filePath);@@ -21,11 +39,7 @@ function assertNotSymlink(filePath: string, label: string): void {throw err;}if (lst.isSymbolicLink()) {- throw new Error(- `Refusing to follow symlink at ${label} (${filePath}). ` +- `An untrusted upstream may have committed this symlink to redirect writes outside the project. ` +- `Remove the symlink and re-run.`,- );+ throw new SymlinkRefusedError(filePath, label);}}@@ -120,7 +134,7 @@ function writeIfMissing(filePath: string, content: string, label: string): boolereturn true;}-function ensureGitignoreEntries(+export function ensureGitignoreEntries(filePath: string,scaffoldContent: string,label = '.ok/.gitignore',@@ -186,7 +200,7 @@ export function removeProjectSkillGitignoreBlock(projectDir: string): 'removed'kept.push(line);}const next = kept.join('\n');- const tmpPath = `${filePath}.tmp.${randomUUID()}`;+ const tmpPath = atomicTempPath(filePath);try {tracedWriteFileSync(tmpPath, next, 'utf-8');tracedRenameSync(tmpPath, filePath);@@ -208,17 +222,18 @@ ${LOCAL_DIR}/# selector — per-machine checkouts, never committed (WORKTREES_PARENT_DIR in# @inkeep/open-knowledge-core). Worktree creation also appends this path to# .git/info/exclude so projects whose committed rule predates it stay clean.-worktrees/+${WORKTREES_DIRNAME}/++# Machine-local directories left behind by projects that predate .ok/local/.+# Nothing writes them anymore, but a project that has them keeps them until+# someone deletes them, so they stay ignored.+${OK_LEGACY_MACHINE_LOCAL_ROOT_DIRS.map((name) => `${name}/`).join('\n')}# Per-machine runtime state at the .ok/ root. Contains PII (principal email,# UUID), hostnames, and absolute filesystem paths — never commit. The only# file at .ok/ root that SHOULD be committed is \`config.yml\` (project# configuration), which is explicitly NOT in this ignore list.-principal.json-state.json-server.lock-sync-state.json-last-spawn-error.log+${OK_MACHINE_LOCAL_ROOT_FILES.join('\n')}`;export const OK_OKIGNORE_TEMPLATE = `# .okignore — paths to exclude from the OpenKnowledge document index.diff --git a/packages/server/src/lint/fs-safety.ts b/packages/server/src/lint/fs-safety.tsindex 5bf0b9e65..eddd6f39b 100644--- a/packages/server/src/lint/fs-safety.ts+++ b/packages/server/src/lint/fs-safety.ts@@ -1,4 +1,5 @@import { isAbsolute, relative } from 'node:path';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import { tracedRenameSync, tracedUnlinkSync, tracedWriteFileSync } from '../fs-traced.ts';export function isInside(path: string, root: string): boolean {@@ -7,7 +8,7 @@ export function isInside(path: string, root: string): boolean {}export function writeFileAtomic(file: string, content: string): void {- const tmp = `${file}.tmp.${process.pid}.${Date.now()}`;+ const tmp = atomicTempPath(file);try {tracedWriteFileSync(tmp, content, 'utf-8');tracedRenameSync(tmp, file);diff --git a/packages/server/src/metrics.ts b/packages/server/src/metrics.tsindex 2c812b292..6f565f816 100644--- a/packages/server/src/metrics.ts+++ b/packages/server/src/metrics.ts@@ -52,6 +52,8 @@ export interface ReconciliationMetrics {collabSocketEconnresetCount: number;collabMessageTooLargeCount: number;shadowMigrationLegacyRefsDeleted: number;+ shadowExcludeIndexEntriesDropped: number;+ shadowExcludeIndexSweepFailures: number;effectDiffCaptureFailures: number;agentPresenceMutationErrors: number;agentWriteCalls: number;@@ -155,6 +157,8 @@ const counters: ReconciliationMetrics = {collabSocketEconnresetCount: 0,collabMessageTooLargeCount: 0,shadowMigrationLegacyRefsDeleted: 0,+ shadowExcludeIndexEntriesDropped: 0,+ shadowExcludeIndexSweepFailures: 0,effectDiffCaptureFailures: 0,agentPresenceMutationErrors: 0,agentWriteCalls: 0,@@ -549,6 +553,14 @@ export function incrementShadowMigrationLegacyRefsDeleted(count: number): void {counters.shadowMigrationLegacyRefsDeleted += count;}+export function incrementShadowExcludeIndexEntriesDropped(count: number): void {+ counters.shadowExcludeIndexEntriesDropped += count;+}++export function incrementShadowExcludeIndexSweepFailures(): void {+ counters.shadowExcludeIndexSweepFailures++;+}+export function incrementEffectDiffCaptureFailures(): void {counters.effectDiffCaptureFailures++;}@@ -622,6 +634,8 @@ export function resetMetrics(): void {counters.collabSocketEconnresetCount = 0;counters.collabMessageTooLargeCount = 0;counters.shadowMigrationLegacyRefsDeleted = 0;+ counters.shadowExcludeIndexEntriesDropped = 0;+ counters.shadowExcludeIndexSweepFailures = 0;counters.effectDiffCaptureFailures = 0;counters.agentPresenceMutationErrors = 0;counters.agentWriteCalls = 0;diff --git a/packages/server/src/persistence.ts b/packages/server/src/persistence.tsindex 6ae74bd00..22c104738 100644--- a/packages/server/src/persistence.ts+++ b/packages/server/src/persistence.ts@@ -18,6 +18,7 @@ import {prependFrontmatter,stripFrontmatter,} from '@inkeep/open-knowledge-core';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import {composeCommitSubject,formatOkActor,@@ -1675,7 +1676,7 @@ export function createPersistenceExtension(options?: PersistenceOptions): Persisreturn;}- const tmpPath = `${canonicalPath}.tmp.${crypto.randomUUID()}`;+ const tmpPath = atomicTempPath(canonicalPath);try {if (process.env.NODE_ENV === 'test' && process.env.OK_TEST_STORE_FAULT === documentName) {const faultErr = new Error(diff --git a/packages/server/src/server-factory.test.ts b/packages/server/src/server-factory.test.tsindex 3e256ac02..cc200b1b1 100644--- a/packages/server/src/server-factory.test.ts+++ b/packages/server/src/server-factory.test.ts@@ -7,6 +7,7 @@ import {renameSync,rmSync,statSync,+ symlinkSync,unlinkSync,writeFileSync,} from 'node:fs';@@ -865,6 +866,29 @@ describe('createServer() degraded signal', () => {await srv.destroy();});+ test('shadow exclude write refusal — degraded includes "shadow-excludes"', async () => {+ mkdirSync(resolve(testProjectDir, '.git', 'ok', 'info'), { recursive: true });+ symlinkSync(+ resolve(testProjectDir, 'planted-exclude-target'),+ resolve(testProjectDir, '.git', 'ok', 'info', 'exclude'),+ );++ const contentDir = mkdtempSync(resolve(testProjectDir, 'content-'));+ const srv = createServer({+ contentDir,+ projectDir: testProjectDir,+ quiet: true,+ });++ try {+ await srv.ready;+ expect(srv.degraded).toContain('shadow-excludes');+ expect(srv.degraded.filter((s) => s === 'shadow-excludes')).toHaveLength(1);+ } finally {+ await srv.destroy();+ }+ });+test.each([{ failure: 'file' as const, label: 'file-watcher' },{ failure: 'head' as const, label: 'head-watcher' },diff --git a/packages/server/src/server-factory.ts b/packages/server/src/server-factory.tsindex cba799ed5..47158d6d8 100644--- a/packages/server/src/server-factory.ts+++ b/packages/server/src/server-factory.ts@@ -258,6 +258,7 @@ import {configureShadowGc,destroyShadowRepo,initShadowRepo,+ isShadowExcludesDegraded,OK_GENERATOR_WRITER,type ParkableDoc,parkBranch,@@ -3063,6 +3064,9 @@ export function createServer(options: ServerOptions): ServerInstance {{ gitDir: shadowRef.current.gitDir },`[server] history repo initialized at ${shadowRef.current.gitDir}`,);+ if (isShadowExcludesDegraded(shadowRef.current) && !degraded.includes('shadow-excludes')) {+ degraded.push('shadow-excludes');+ }} catch (e) {log.error({ err: e }, '[server] history repo init failed');degraded.push('shadow-repo');@@ -3097,6 +3101,12 @@ export function createServer(options: ServerOptions): ServerInstance {log.warn({}, '[server] history repo appears corrupted — reinitializing');try {shadowRef.current = await initShadowRepo(projectDir, { deferGcConfig: true });+ if (+ isShadowExcludesDegraded(shadowRef.current) &&+ !degraded.includes('shadow-excludes')+ ) {+ degraded.push('shadow-excludes');+ }} catch (e2) {log.error({ err: e2 }, '[server] history repo reinit failed');shadowRef.current = undefined;diff --git a/packages/server/src/shadow-repo.test.ts b/packages/server/src/shadow-repo.test.tsindex 09e47b412..8b0208124 100644--- a/packages/server/src/shadow-repo.test.ts+++ b/packages/server/src/shadow-repo.test.ts@@ -1,8 +1,10 @@-import { existsSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';+import { execFileSync } from 'node:child_process';+import { existsSync, mkdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';import { mkdtemp, rm } from 'node:fs/promises';import { tmpdir } from 'node:os';import { resolve } from 'node:path';import { setTimeout as wait } from 'node:timers/promises';+import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import {CHECKPOINT_KIND_REGISTRY,CHECKPOINT_KINDS,@@ -13,15 +15,19 @@ import {} from '@inkeep/open-knowledge-core/shadow-repo-layout';import simpleGit from 'simple-git';import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';+import { applySkillWrite } from './content/skills-write.ts';import { getLogger } from './logger.ts';+import { getMetrics, resetMetrics } from './metrics.ts';import {buildWipTree,commitUpstreamImport,commitWip,DEFAULT_CHECKPOINT_RETENTION,+ FANOUT_INDEX_NAME,GIT_UPSTREAM_WRITER,type InMemoryCheckpointParams,initShadowRepo,+ isShadowExcludesDegraded,listRescueCheckpoints,type ParkableDoc,parkBranch,@@ -37,6 +43,19 @@ import {type WriterIdentity,} from './shadow-repo';+const fsTracedRecorder = vi.hoisted(() => ({ renameSources: [] as string[] }));++vi.mock('./fs-traced.ts', async (importOriginal) => {+ const actual = await importOriginal<typeof import('./fs-traced.ts')>();+ return {+ ...actual,+ tracedRenameSync: (from: string, to: string): void => {+ fsTracedRecorder.renameSources.push(from);+ actual.tracedRenameSync(from, to);+ },+ };+});+let tmpDir: string;beforeEach(async () => {@@ -1871,3 +1890,563 @@ describe('sweepLegacyShadowRefs (US-018, D35, NFR-6)', () => {expect(deleted).toBe(0);});});++const STOCK_GIT_EXCLUDE = `# git ls-files --others --exclude-from=.git/info/exclude+# Lines that start with '#' are comments.+`;++describe('shadow repo excludes OpenKnowledge machine-local state', () => {+ let projectRoot: string;+ let shadow: ShadowHandle;++ const writer: WriterIdentity = {+ id: 'human-ada',+ name: 'Ada Lovelace',+ email: '[email protected]',+ };++ beforeEach(async () => {+ projectRoot = resolve(tmpDir, 'project');+ mkdirSync(resolve(projectRoot, '.ok/local/cache/main'), { recursive: true });+ mkdirSync(resolve(projectRoot, '.ok/worktrees/feature'), { recursive: true });++ const git = simpleGit(projectRoot);+ await git.init();+ await git.raw('config', 'user.name', 'Test');+ await git.raw('config', 'user.email', '[email protected]');++ writeFileSync(resolve(projectRoot, 'intro.md'), '# Hello\n');+ writeFileSync(resolve(projectRoot, '.ok/config.yml'), 'content:\n dir: .\n');+ writeFileSync(+ resolve(projectRoot, '.ok/local/principal.json'),+ '{"email":"[email protected]"}\n',+ );+ writeFileSync(resolve(projectRoot, '.ok/local/server.lock'), '{"pid":1}\n');+ writeFileSync(resolve(projectRoot, '.ok/local/cache/main/backlinks.json'), '{}\n');+ writeFileSync(resolve(projectRoot, '.ok/worktrees/feature/scratch.md'), '# scratch\n');++ shadow = await initShadowRepo(projectRoot);+ });++ async function seedStaleWipRef(handle: ShadowHandle, writerId: string): Promise<void> {+ const excludeFile = resolve(handle.gitDir, 'info/exclude');+ const realExclude = existsSync(excludeFile) ? readFileSync(excludeFile, 'utf-8') : '';+ const seedIndex = resolve(handle.gitDir, `index-stale-seed-${writerId}`);+ rmSync(seedIndex, { force: true });+ writeFileSync(excludeFile, STOCK_GIT_EXCLUDE);+ const gitEnv = { ...process.env, GIT_DIR: handle.gitDir, GIT_INDEX_FILE: seedIndex };+ try {+ execFileSync('git', ['add', '-A', '.'], {+ cwd: handle.workTree,+ env: { ...gitEnv, GIT_WORK_TREE: handle.workTree },+ });+ const tree = execFileSync('git', ['write-tree'], {+ cwd: handle.workTree,+ env: gitEnv,+ encoding: 'utf-8',+ }).trim();+ const commit = execFileSync('git', ['commit-tree', tree, '-m', 'WIP: pre-upgrade'], {+ cwd: handle.workTree,+ env: gitEnv,+ encoding: 'utf-8',+ }).trim();+ execFileSync('git', ['update-ref', `refs/wip/main/${writerId}`, commit], {+ cwd: handle.workTree,+ env: gitEnv,+ });+ } finally {+ writeFileSync(excludeFile, realExclude);+ rmSync(seedIndex, { force: true });+ }+ }++ function seedStaleFanoutIndex(handle: ShadowHandle): string {+ const excludeFile = resolve(handle.gitDir, 'info/exclude');+ const realExclude = existsSync(excludeFile) ? readFileSync(excludeFile, 'utf-8') : '';+ const fanoutIndex = resolve(handle.gitDir, FANOUT_INDEX_NAME);+ rmSync(fanoutIndex, { force: true });+ writeFileSync(excludeFile, STOCK_GIT_EXCLUDE);+ try {+ execFileSync('git', ['add', '-A', '.'], {+ cwd: handle.workTree,+ env: {+ ...process.env,+ GIT_DIR: handle.gitDir,+ GIT_WORK_TREE: handle.workTree,+ GIT_INDEX_FILE: fanoutIndex,+ },+ });+ } finally {+ writeFileSync(excludeFile, realExclude);+ }+ return fanoutIndex;+ }++ function indexPaths(handle: ShadowHandle, indexFile: string): string[] {+ return execFileSync('git', ['ls-files'], {+ cwd: handle.workTree,+ env: { ...process.env, GIT_DIR: handle.gitDir, GIT_INDEX_FILE: indexFile },+ encoding: 'utf-8',+ })+ .split('\n')+ .map((path) => path.trim())+ .filter((path) => path.length > 0);+ }++ async function wipTreePaths(handle: ShadowHandle, writerId: string): Promise<string[]> {+ const sg = shadowGit(handle);+ return (await sg.raw('ls-tree', '-r', '--name-only', `refs/wip/main/${writerId}`))+ .split('\n')+ .map((path) => path.trim())+ .filter((path) => path.length > 0);+ }++ async function checkpointTreePaths(ref: string): Promise<string[]> {+ const sg = shadowGit(shadow);+ return (await sg.raw('ls-tree', '-r', '--name-only', ref))+ .split('\n')+ .map((path) => path.trim())+ .filter((path) => path.length > 0);+ }++ test('the shadow does not see .ok/local or .ok/worktrees as content', async () => {+ const sg = shadowGit(shadow);+ const status = await sg.raw('status', '--porcelain', '--untracked-files=all');+ const paths = status+ .split('\n')+ .map((line) => line.slice(3).trim())+ .filter((p) => p.length > 0);++ expect(paths).toContain('intro.md');+ expect(paths).toContain('.ok/config.yml');+ expect(paths.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ expect(paths.filter((p) => p.startsWith('.ok/worktrees/'))).toEqual([]);+ });++ test('a checkpoint keeps machine-local state out of version history', async () => {+ const result = await saveVersion(shadow, '.', [writer]);+ const sg = shadowGit(shadow);+ const tree = (await sg.raw('ls-tree', '-r', '--name-only', result.checkpointRef))+ .split('\n')+ .map((p) => p.trim())+ .filter((p) => p.length > 0);++ expect(tree).toContain('intro.md');+ expect(tree).toContain('.ok/config.yml');+ expect(tree.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ expect(tree.filter((p) => p.startsWith('.ok/worktrees/'))).toEqual([]);+ });++ test('a nested content dir keeps its own .ok/local out of the staged tree', async () => {+ mkdirSync(resolve(projectRoot, 'docs/.ok/local'), { recursive: true });+ writeFileSync(resolve(projectRoot, 'docs/note.md'), '# note\n');+ writeFileSync(resolve(projectRoot, 'docs/.ok/local/comments.json'), '{}\n');++ const result = await saveVersion(shadow, 'docs', [writer]);+ const sg = shadowGit(shadow);+ const tree = (await sg.raw('ls-tree', '-r', '--name-only', result.checkpointRef))+ .split('\n')+ .map((p) => p.trim())+ .filter((p) => p.length > 0);++ expect(tree).toContain('docs/note.md');+ expect(tree.filter((p) => p.startsWith('docs/.ok/local/'))).toEqual([]);+ });++ test('legacy machine-local files at the .ok root stay out of a checkpoint', async () => {+ const legacyRootFiles = [+ 'principal.json',+ 'state.json',+ 'server.lock',+ 'ui.lock',+ 'sync-state.json',+ 'conflicts.json',+ 'last-spawn-error.log',+ ];+ const legacyRootDirs = ['local', 'worktrees', 'cache', 'tmp'];+ for (const name of legacyRootFiles) {+ writeFileSync(resolve(projectRoot, '.ok', name), '{"legacy":true}\n');+ }+ for (const name of legacyRootDirs) {+ mkdirSync(resolve(projectRoot, '.ok', name), { recursive: true });+ writeFileSync(resolve(projectRoot, '.ok', name, 'residue.json'), '{}\n');+ }++ const result = await saveVersion(shadow, '.', [writer]);+ const sg = shadowGit(shadow);+ const tree = (await sg.raw('ls-tree', '-r', '--name-only', result.checkpointRef))+ .split('\n')+ .map((p) => p.trim())+ .filter((p) => p.length > 0);++ expect(tree).toContain('.ok/config.yml');+ expect(tree).toContain('intro.md');+ for (const name of legacyRootFiles) {+ expect(tree).not.toContain(`.ok/${name}`);+ }+ for (const name of legacyRootDirs) {+ expect(tree.filter((p) => p.startsWith(`.ok/${name}/`))).toEqual([]);+ }+ });++ test('an in-flight document flush is not staged into a checkpoint', async () => {+ const inFlight = atomicTempPath(resolve(projectRoot, 'intro.md'));+ writeFileSync(inFlight, '# half written\n');++ const result = await saveVersion(shadow, '.', [writer]);+ const sg = shadowGit(shadow);+ const tree = (await sg.raw('ls-tree', '-r', '--name-only', result.checkpointRef))+ .split('\n')+ .map((p) => p.trim())+ .filter((p) => p.length > 0);++ expect(tree).toContain('intro.md');+ expect(tree.filter((p) => p.includes('.tmp.'))).toEqual([]);+ });++ test('an in-flight skill write is not staged into a checkpoint', async () => {+ const skillsRoot = resolve(projectRoot, '.ok/skills');+ const skillFile = resolve(skillsRoot, 'trip-log/SKILL.md');+ fsTracedRecorder.renameSources.length = 0;++ const written = applySkillWrite({+ skillsRoot,+ name: 'trip-log',+ body: '# Steps\n\nDo the thing.',+ frontmatter: { name: 'trip-log', description: 'Use when logging a fishing trip.' },+ });+ expect(written.ok).toBe(true);++ const inFlight = fsTracedRecorder.renameSources.find((from) =>+ from.startsWith(`${skillFile}.`),+ );+ expect(inFlight).toBeDefined();+ writeFileSync(inFlight as string, '# half written\n');++ const result = await saveVersion(shadow, '.', [writer]);+ const sg = shadowGit(shadow);+ const tree = (await sg.raw('ls-tree', '-r', '--name-only', result.checkpointRef))+ .split('\n')+ .map((p) => p.trim())+ .filter((p) => p.length > 0);++ expect(tree).toContain('.ok/skills/trip-log/SKILL.md');+ expect(tree.filter((p) => p.startsWith('.ok/skills/trip-log/SKILL.md.'))).toEqual([]);+ });++ test('a WIP commit drops machine-local paths a previous build had staged', async () => {+ const agentWriter: WriterIdentity = {+ id: 'agent-cursor',+ name: 'cursor-agent',+ email: '[email protected]',+ };+ await seedStaleWipRef(shadow, agentWriter.id);+ expect(await wipTreePaths(shadow, agentWriter.id)).toContain('.ok/local/principal.json');++ const reinit = await initShadowRepo(projectRoot);+ writeFileSync(resolve(projectRoot, 'intro.md'), '# Hello again\n');+ await commitWip(reinit, agentWriter, '.', 'WIP: post-upgrade');+ const freshTree = await wipTreePaths(reinit, agentWriter.id);++ expect(freshTree).toContain('intro.md');+ expect(freshTree.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ });++ test('paths that only share a prefix with a machine-local name stay in a checkpoint', async () => {+ mkdirSync(resolve(projectRoot, 'docs/.ok'), { recursive: true });+ writeFileSync(resolve(projectRoot, '.ok/state.json'), '{"machine":true}\n');+ writeFileSync(resolve(projectRoot, '.ok/state.jsonl'), '{"content":true}\n');+ writeFileSync(resolve(projectRoot, '.ok/principal.json'), '{"machine":true}\n');+ writeFileSync(resolve(projectRoot, '.ok/principal.json.bak'), '{"content":true}\n');+ writeFileSync(resolve(projectRoot, 'docs/.ok/server.lock'), '{"machine":true}\n');+ writeFileSync(resolve(projectRoot, 'docs/.ok/server.lock.disabled'), '{"content":true}\n');++ const result = await saveVersion(shadow, '.', [writer]);+ const tree = await checkpointTreePaths(result.checkpointRef);++ expect(tree).toContain('.ok/state.jsonl');+ expect(tree).toContain('.ok/principal.json.bak');+ expect(tree).toContain('docs/.ok/server.lock.disabled');+ expect(tree).not.toContain('.ok/state.json');+ expect(tree).not.toContain('.ok/principal.json');+ expect(tree).not.toContain('docs/.ok/server.lock');+ });++ test('a staged path whose name begins with a dash is still dropped', async () => {+ const agentWriter: WriterIdentity = {+ id: 'agent-dash',+ name: 'dash-agent',+ email: '[email protected]',+ };+ const leadingDash = `-lead.md${atomicTempPath('x').slice(1)}`;+ writeFileSync(resolve(projectRoot, leadingDash), '# half written\n');+ await seedStaleWipRef(shadow, agentWriter.id);+ expect(await wipTreePaths(shadow, agentWriter.id)).toContain(leadingDash);++ const reinit = await initShadowRepo(projectRoot);+ await commitWip(reinit, agentWriter, '.', 'WIP: post-upgrade');+ const freshTree = await wipTreePaths(reinit, agentWriter.id);++ expect(freshTree).toContain('intro.md');+ expect(freshTree).not.toContain(leadingDash);+ });++ test('a contaminated index larger than the argv ceiling is still repaired', async () => {+ const agentWriter: WriterIdentity = {+ id: 'agent-bulk',+ name: 'bulk-agent',+ email: '[email protected]',+ };+ const leaf = 'q'.repeat(160);+ const threadsDir = resolve(projectRoot, '.ok/local/threads');+ mkdirSync(threadsDir, { recursive: true });+ const bulk = Array.from(+ { length: 8_000 },+ (_unused, i) => `${String(i).padStart(6, '0')}-${leaf}.ndjson`,+ );+ for (const name of bulk) writeFileSync(resolve(threadsDir, name), '{}\n');+ const argvBytes = bulk.reduce((acc, name) => acc + `.ok/local/threads/${name}`.length + 1, 0);+ expect(argvBytes).toBeGreaterThan(1_500_000);++ await seedStaleWipRef(shadow, agentWriter.id);+ const staleTree = await wipTreePaths(shadow, agentWriter.id);+ expect(staleTree.filter((p) => p.startsWith('.ok/local/threads/'))).toHaveLength(8_000);++ const reinit = await initShadowRepo(projectRoot);+ await commitWip(reinit, agentWriter, '.', 'WIP: post-upgrade');+ const freshTree = await wipTreePaths(reinit, agentWriter.id);++ expect(freshTree).toContain('intro.md');+ expect(freshTree.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ });++ test('a repair the index sweep cannot run does not abort initialization, but degrades the shadow', async () => {+ writeFileSync(resolve(shadow.gitDir, FANOUT_INDEX_NAME), 'GARBAGE-NOT-AN-INDEX-FILE');+ resetMetrics();+ const warnSpy = vi.spyOn(getLogger('shadow-repo'), 'warn');++ try {+ const reinit = await initShadowRepo(projectRoot);++ expect(existsSync(resolve(reinit.gitDir, 'info/exclude'))).toBe(true);+ expect(isShadowExcludesDegraded(reinit)).toBe(true);+ expect(getMetrics().shadowExcludeIndexSweepFailures).toBe(1);+ expect(getMetrics().shadowExcludeIndexEntriesDropped).toBe(0);+ const warnings = warnSpy.mock.calls.map((call) => String(call[1] ?? ''));+ expect(+ warnings.some((message) =>+ message.includes(+ '[shadow-repo] could not read the index to clear machine-local entries — none were dropped and an unknown number are still staged',+ ),+ ),+ ).toBe(true);+ } finally {+ warnSpy.mockRestore();+ }+ });++ test('an index sweep blocked partway reports how many machine-local entries it left staged', async () => {+ const fanoutIndex = seedStaleFanoutIndex(shadow);+ const machineLocalStaged = indexPaths(shadow, fanoutIndex).filter(+ (path) => path.startsWith('.ok/local/') || path.startsWith('.ok/worktrees/'),+ );+ expect(machineLocalStaged.length).toBeGreaterThan(0);+ writeFileSync(`${fanoutIndex}.lock`, '');+ resetMetrics();+ const warnSpy = vi.spyOn(getLogger('shadow-repo'), 'warn');+ const infoSpy = vi.spyOn(getLogger('shadow-repo'), 'info');++ try {+ const reinit = await initShadowRepo(projectRoot);++ expect(isShadowExcludesDegraded(reinit)).toBe(true);+ expect(getMetrics().shadowExcludeIndexSweepFailures).toBe(1);+ expect(getMetrics().shadowExcludeIndexEntriesDropped).toBe(0);+ expect(+ indexPaths(reinit, fanoutIndex).filter(+ (path) => path.startsWith('.ok/local/') || path.startsWith('.ok/worktrees/'),+ ),+ ).toEqual(machineLocalStaged);+ const warnings = warnSpy.mock.calls.map((call) => String(call[1] ?? ''));+ expect(+ warnings.some((message) =>+ message.includes(`dropped 0, left ${machineLocalStaged.length} still staged`),+ ),+ ).toBe(true);+ const infos = infoSpy.mock.calls.map((call) => String(call[1] ?? ''));+ expect(+ infos.some((message) =>+ message.includes(+ '[shadow-repo] dropped machine-local entries a previous build had staged',+ ),+ ),+ ).toBe(false);+ } finally {+ warnSpy.mockRestore();+ infoSpy.mockRestore();+ }+ });++ test('a checkpoint stays clean even when the shadow exclude file is gone', async () => {+ writeFileSync(resolve(shadow.gitDir, 'info/exclude'), STOCK_GIT_EXCLUDE);++ const result = await saveVersion(shadow, '.', [writer]);+ const tree = await checkpointTreePaths(result.checkpointRef);++ expect(tree).toContain('intro.md');+ expect(tree).toContain('.ok/config.yml');+ expect(tree.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ expect(tree.filter((p) => p.startsWith('.ok/worktrees/'))).toEqual([]);+ });++ test('a shadow whose exclude file cannot be written reports itself degraded', async () => {+ const excludeFile = resolve(shadow.gitDir, 'info/exclude');+ rmSync(excludeFile, { force: true });+ symlinkSync(resolve(tmpDir, 'planted-exclude'), excludeFile);++ const reinit = await initShadowRepo(projectRoot);++ expect(isShadowExcludesDegraded(reinit)).toBe(true);+ });++ test('a symlink at the exclude path is refused at error level, not warned as I/O', async () => {+ const excludeFile = resolve(shadow.gitDir, 'info/exclude');+ rmSync(excludeFile, { force: true });+ symlinkSync(resolve(tmpDir, 'planted-exclude'), excludeFile);+ const errorSpy = vi.spyOn(getLogger('shadow-repo'), 'error');+ const warnSpy = vi.spyOn(getLogger('shadow-repo'), 'warn');++ try {+ await initShadowRepo(projectRoot);++ const errors = errorSpy.mock.calls.map((call) => String(call[1] ?? ''));+ const warnings = warnSpy.mock.calls.map((call) => String(call[1] ?? ''));+ expect(+ errors.some((message) =>+ message.includes('[shadow-repo] refused to write the shadow exclude file'),+ ),+ ).toBe(true);+ expect(+ warnings.some((message) =>+ message.includes('[shadow-repo] could not write the shadow exclude file'),+ ),+ ).toBe(false);+ } finally {+ errorSpy.mockRestore();+ warnSpy.mockRestore();+ }+ });++ test('a symlinked shadow info directory pointing at a live directory is refused, not followed', async () => {+ const infoDir = resolve(shadow.gitDir, 'info');+ const plantedInfo = resolve(tmpDir, 'planted-info-live');+ mkdirSync(plantedInfo, { recursive: true });+ rmSync(infoDir, { recursive: true, force: true });+ symlinkSync(plantedInfo, infoDir);+ const errorSpy = vi.spyOn(getLogger('shadow-repo'), 'error');+ const warnSpy = vi.spyOn(getLogger('shadow-repo'), 'warn');++ try {+ const reinit = await initShadowRepo(projectRoot);++ expect(isShadowExcludesDegraded(reinit)).toBe(true);+ expect(existsSync(resolve(plantedInfo, 'exclude'))).toBe(false);+ const errors = errorSpy.mock.calls.map((call) => String(call[1] ?? ''));+ const warnings = warnSpy.mock.calls.map((call) => String(call[1] ?? ''));+ expect(+ errors.some((message) =>+ message.includes('[shadow-repo] refused to write the shadow exclude file'),+ ),+ ).toBe(true);+ expect(+ warnings.some((message) =>+ message.includes('[shadow-repo] could not write the shadow exclude file'),+ ),+ ).toBe(false);+ } finally {+ errorSpy.mockRestore();+ warnSpy.mockRestore();+ }+ });++ test('a dangling symlink at the shadow info directory is refused, not warned as I/O', async () => {+ const infoDir = resolve(shadow.gitDir, 'info');+ rmSync(infoDir, { recursive: true, force: true });+ symlinkSync(resolve(tmpDir, 'planted-info-absent'), infoDir);+ const errorSpy = vi.spyOn(getLogger('shadow-repo'), 'error');+ const warnSpy = vi.spyOn(getLogger('shadow-repo'), 'warn');++ try {+ const reinit = await initShadowRepo(projectRoot);++ expect(isShadowExcludesDegraded(reinit)).toBe(true);+ const errors = errorSpy.mock.calls.map((call) => String(call[1] ?? ''));+ const warnings = warnSpy.mock.calls.map((call) => String(call[1] ?? ''));+ expect(+ errors.some((message) =>+ message.includes('[shadow-repo] refused to write the shadow exclude file'),+ ),+ ).toBe(true);+ expect(+ warnings.some((message) =>+ message.includes('[shadow-repo] could not write the shadow exclude file'),+ ),+ ).toBe(false);+ } finally {+ errorSpy.mockRestore();+ warnSpy.mockRestore();+ }+ });++ test('the index repair reports its drop count through the metrics surface', async () => {+ const agentWriter: WriterIdentity = {+ id: 'agent-metrics',+ name: 'metrics-agent',+ email: '[email protected]',+ };+ await seedStaleWipRef(shadow, agentWriter.id);+ const reinit = await initShadowRepo(projectRoot);+ resetMetrics();++ await commitWip(reinit, agentWriter, '.', 'WIP: post-upgrade');++ expect(getMetrics().shadowExcludeIndexEntriesDropped).toBeGreaterThan(0);+ });++ test('re-initializing drops a fan-out index that already staged machine-local state', async () => {+ const fanoutIndex = seedStaleFanoutIndex(shadow);+ expect(indexPaths(shadow, fanoutIndex)).toContain('.ok/local/principal.json');++ const reinit = await initShadowRepo(projectRoot);+ const freshTree = await buildWipTree(reinit, '.');+ const sgFresh = shadowGit(reinit);+ const freshNames = (await sgFresh.raw('ls-tree', '-r', '--name-only', freshTree))+ .split('\n')+ .map((p) => p.trim())+ .filter((p) => p.length > 0);++ expect(freshNames).toContain('intro.md');+ expect(freshNames.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ });++ test('a fan-out WIP tree stays clean even when the shadow exclude file is gone', async () => {+ writeFileSync(resolve(shadow.gitDir, 'info/exclude'), STOCK_GIT_EXCLUDE);++ const tree = await buildWipTree(shadow, '.');+ const names = await checkpointTreePaths(tree);++ expect(names).toContain('intro.md');+ expect(names.filter((p) => p.startsWith('.ok/local/'))).toEqual([]);+ expect(names.filter((p) => p.startsWith('.ok/worktrees/'))).toEqual([]);+ });++ test('re-initializing an existing shadow restores the exclusions', async () => {+ writeFileSync(resolve(shadow.gitDir, 'info/exclude'), STOCK_GIT_EXCLUDE);++ const reinit = await initShadowRepo(projectRoot);+ const sg = shadowGit(reinit);+ const status = await sg.raw('status', '--porcelain', '--untracked-files=all');++ expect(status).not.toContain('.ok/local/');+ });+});diff --git a/packages/server/src/shadow-repo.ts b/packages/server/src/shadow-repo.tsindex aeb15ebac..a853c4892 100644--- a/packages/server/src/shadow-repo.ts+++ b/packages/server/src/shadow-repo.ts@@ -1,7 +1,14 @@import { randomUUID } from 'node:crypto';import { existsSync, readdirSync, rmSync } from 'node:fs';import { resolve } from 'node:path';-import { pathspecArgs, SYSTEM_WRITER_DISPLAY_NAMES } from '@inkeep/open-knowledge-core';+import {+ OK_DIR,+ OK_MACHINE_LOCAL_ROOT_DIRS,+ OK_MACHINE_LOCAL_ROOT_FILES,+ pathspecArgs,+ SYSTEM_WRITER_DISPLAY_NAMES,+} from '@inkeep/open-knowledge-core';+import { ATOMIC_TEMP_GLOB } from '@inkeep/open-knowledge-core/server';import {type AutoConsolidationTrigger,CHECKPOINT_KIND_REGISTRY,@@ -24,8 +31,13 @@ import simpleGit from 'simple-git';import { resolveCheckpointChainAnchors } from './checkpoint-chain.ts';import { tracedMkdirSync, tracedRenameSync, tracedWriteFileSync } from './fs-traced.ts';import { listTreeLongEntries } from './git-paths.ts';+import { assertNotSymlink, ensureGitignoreEntries, SymlinkRefusedError } from './init-project.ts';import { getLogger } from './logger.ts';-import { incrementShadowMigrationLegacyRefsDeleted } from './metrics.ts';+import {+ incrementShadowExcludeIndexEntriesDropped,+ incrementShadowExcludeIndexSweepFailures,+ incrementShadowMigrationLegacyRefsDeleted,+} from './metrics.ts';import { acquireLock, releaseLock } from './shadow-lock.ts';import { releaseShadowOpGate, shadowOpGateFor } from './shadow-op-gate.ts';import { withSpan } from './telemetry.ts';@@ -70,6 +82,126 @@ const CORPUS_STAGE_GIT_TIMEOUT_MS = (() => {const SHADOW_GC_AUTO = 512;+const MACHINE_LOCAL_DIRS = OK_MACHINE_LOCAL_ROOT_DIRS.map((name) => `${OK_DIR}/${name}/`);++const MACHINE_LOCAL_FILES = OK_MACHINE_LOCAL_ROOT_FILES.map((name) => `${OK_DIR}/${name}`);++const SHADOW_EXCLUDE_PATTERNS: readonly string[] = [+ ...MACHINE_LOCAL_DIRS.map((dir) => `**/${dir}`),+ ...MACHINE_LOCAL_FILES.map((file) => `**/${file}`),+ ATOMIC_TEMP_GLOB,+];++const SHADOW_EXCLUDE_CONTENT = `# OpenKnowledge machine-local state and in-flight atomic writes: never part of a version, never staged.+${SHADOW_EXCLUDE_PATTERNS.join('\n')}+`;++const EXCLUDE_PATTERN_ARGS = SHADOW_EXCLUDE_PATTERNS.flatMap((pattern) => ['-x', pattern]);++const UPDATE_INDEX_ARGV_BUDGET_BYTES = 8000;++const shadowExcludesWritten = new Map<string, boolean>();++export function isShadowExcludesDegraded(shadow: ShadowHandle): boolean {+ return shadowExcludesWritten.get(shadow.gitDir) === false;+}++function chunkByArgvBudget(paths: readonly string[], budgetBytes: number): string[][] {+ const chunks: string[][] = [];+ let current: string[] = [];+ let used = 0;+ for (const path of paths) {+ const cost = Buffer.byteLength(path, 'utf-8') + 1;+ if (current.length > 0 && used + cost > budgetBytes) {+ chunks.push(current);+ current = [];+ used = 0;+ }+ current.push(path);+ used += cost;+ }+ if (current.length > 0) chunks.push(current);+ return chunks;+}++interface IndexSweepResult {+ failed: boolean;+}++async function dropExcludedIndexEntries(+ sg: ReturnType<typeof shadowGit>,+ shadow: ShadowHandle,+ indexFile: string,+): Promise<IndexSweepResult> {+ const env = { GIT_DIR: shadow.gitDir, GIT_INDEX_FILE: indexFile };+ let dropped = 0;+ let listedCount: number | undefined;+ try {+ const listed = await sg.env(env).raw('ls-files', '-z', '-i', '-c', ...EXCLUDE_PATTERN_ARGS);+ const stale = listed.split('\0').filter((p) => p.length > 0);+ listedCount = stale.length;+ for (const chunk of chunkByArgvBudget(stale, UPDATE_INDEX_ARGV_BUDGET_BYTES)) {+ await sg.env(env).raw('update-index', '--force-remove', '--', ...chunk);+ dropped += chunk.length;+ }+ if (dropped > 0) {+ incrementShadowExcludeIndexEntriesDropped(dropped);+ log.info(+ { count: dropped },+ '[shadow-repo] dropped machine-local entries a previous build had staged',+ );+ }+ return { failed: false };+ } catch (e) {+ const leftStaged = listedCount === undefined ? undefined : listedCount - dropped;+ incrementShadowExcludeIndexSweepFailures();+ log.warn(+ { err: e, dropped, leftStaged: leftStaged ?? 'unknown' },+ leftStaged === undefined+ ? '[shadow-repo] could not read the index to clear machine-local entries — none were dropped and an unknown number are still staged (non-fatal)'+ : `[shadow-repo] could not finish clearing machine-local entries from the index — dropped ${dropped}, left ${leftStaged} still staged (non-fatal)`,+ );+ return { failed: true };+ }+}++async function ensureShadowExcludes(shadow: ShadowHandle): Promise<void> {+ const infoDir = resolve(shadow.gitDir, 'info');+ const excludeFile = resolve(infoDir, 'exclude');+ try {+ assertNotSymlink(infoDir, 'shadow info/');+ tracedMkdirSync(infoDir, { recursive: true });+ const outcome = ensureGitignoreEntries(+ excludeFile,+ SHADOW_EXCLUDE_CONTENT,+ 'shadow info/exclude',+ );+ shadowExcludesWritten.set(shadow.gitDir, true);+ log.info({ outcome, excludeFile }, `[shadow-repo] shadow exclude file ${outcome}`);+ } catch (e) {+ shadowExcludesWritten.set(shadow.gitDir, false);+ if (e instanceof SymlinkRefusedError) {+ log.error(+ { err: e, excludeFile },+ '[shadow-repo] refused to write the shadow exclude file: a symlink is planted on a component of the shadow info/exclude path',+ );+ } else {+ log.warn(+ { err: e, excludeFile },+ '[shadow-repo] could not write the shadow exclude file — machine-local state stays out of versions only through the index sweep',+ );+ }+ }+ const fanoutIndex = resolve(shadow.gitDir, FANOUT_INDEX_NAME);+ if (!existsSync(fanoutIndex)) return;+ const sweep = await dropExcludedIndexEntries(+ shadowGit(shadow, { timeoutMs: CORPUS_STAGE_GIT_TIMEOUT_MS }),+ shadow,+ fanoutIndex,+ );+ if (sweep.failed) shadowExcludesWritten.set(shadow.gitDir, false);+}+export function shadowGit(shadow: ShadowHandle, opts?: { timeoutMs?: number }) {return simpleGit({baseDir: shadow.workTree,@@ -169,6 +301,8 @@ export async function initShadowRepo(const handle: ShadowHandle = { gitDir: shadowDir, workTree: projectRoot };+ await ensureShadowExcludes(handle);+if (!opts?.deferGcConfig) {try {await configureShadowGc(handle);@@ -317,6 +451,7 @@ async function commitWipInner(GIT_INDEX_FILE: tmpIndex,}).raw('add', ...pathspecArgs([gitPathspec]));+ await dropExcludedIndexEntries(sg, shadow, tmpIndex);const treeSha = (await sg.env({ GIT_DIR: shadow.gitDir, GIT_INDEX_FILE: tmpIndex }).raw('write-tree')).trim();@@ -394,6 +529,7 @@ async function buildWipTreeWithIndex(GIT_INDEX_FILE: indexFile,}).raw('add', ...pathspecArgs([gitPathspec]));+ await dropExcludedIndexEntries(sg, shadow, indexFile);return (await sg.env({ GIT_DIR: shadow.gitDir, GIT_INDEX_FILE: indexFile }).raw('write-tree')).trim();@@ -1445,6 +1581,13 @@ async function saveVersionInner(GIT_INDEX_FILE: shadowTmpIndex,}).raw('add', ...pathspecArgs([gitPathspec]));+ await dropExcludedIndexEntries(+ shadowGit(shadow, {+ timeoutMs: Math.max(options?.timeoutMs ?? 0, CORPUS_STAGE_GIT_TIMEOUT_MS),+ }),+ shadow,+ shadowTmpIndex,+ );const shadowTreeSha = (await sg.env({ GIT_DIR: shadow.gitDir, GIT_INDEX_FILE: shadowTmpIndex }).raw('write-tree')).trim();