A self-organizing Obsidian vault that gives AI coding agents persistent memory.
TypeScript ai-agentsclaude-codecodex-cligemini-cliknowledge-managementobsidianobsidian-vaultpersistent-memorysecond-braintemplate
fix: run remember's near-duplicate check only against memories the caller can see (#387)
Brenno Ferrarid51ffe7e24ba
diff --git a/.claude/scripts/lib/mcp-server.ts b/.claude/scripts/lib/mcp-server.tsindex 33d900f..96bd662 100644--- a/.claude/scripts/lib/mcp-server.ts+++ b/.claude/scripts/lib/mcp-server.ts@@ -533,56 +533,80 @@ export function createHandlers(deps: ServerDeps): Handlers {// comparison; every OTHER memory stays in it, so a correction that also// duplicates a third note is still refused, which is the check working.//- // Only what recall serves this caller (#364). A title can name a memory- // scoped to another project, or one recall already withholds; marking it- // would withhold it from its own project's callers, whose only replacement- // is out of their reach, and naming its file here would show this caller a+ // Only what recall serves this caller is marked or named (#364, #375). A+ // memory scoped to another project, or one recall already withholds, is+ // never marked and never named: naming its file would show this caller a// memory recall never would. Nor is it counted: a count would say whether// another project holds that title, so a claim with no match in reach is// simply not matched, which is the truth from where the caller stands.+ // The near-duplicate refusal below is the one accepted exception: it says+ // that near-identical content exists where the new memory would be read,+ // without naming it, because keeping a correction intact outweighs that+ // hint (#375). It is audited.+ // Building `served` itself needs every memory, since a stale copy's+ // replacement may be out of reach.const served = servedAfterSupersession(digests.map((d) => ({ rel: d.rel, full: d.full, title: d.title, body: d.body, facets: d })),who,);- const supers = resolveSupersedes(- args.supersedes,- digests.filter((d) => served.has(d.rel)),- );+ const inReach = digests.filter((d) => served.has(d.rel));+ const supers = resolveSupersedes(args.supersedes, inReach);const superseded = new Set(supers.matched.map((m) => m.rel));// Near-duplicate suppression, facet-gated so two projects can each hold- // their own copy of the same lesson.+ // their own copy of the same lesson. A duplicate is judged by where the+ // NEW memory would be read, so it is compared against every memory that+ // shares a facet with it, in the caller's reach or not (#375): otherwise+ // a caller could file a second copy into a project it can't see, or+ // restate a claim whose correction it can't see and undo that correction.+ // Reach decides only what the refusal may name.if (args.force !== true) {const { duplicates } = findSimilar(v.value,digests.filter((d) => !superseded.has(d.rel)),);- if (duplicates.length) {- const d = duplicates[0]!;- // When the caller DID pass supersedes and NONE of them resolved,- // repeating "pass supersedes" is an instruction they already followed- // and cannot follow successfully — the title they named does not exist.- // Say which ones did not resolve, and point at the exact-match rule,- // instead of sending them round the loop a second time.+ const nameable = duplicates.find((dup) => served.has(dup.entry.rel));+ const unseen = duplicates.some((dup) => !served.has(dup.entry.rel));+ // When the caller DID pass supersedes and NONE of them resolved,+ // repeating "pass supersedes" is an instruction they already followed+ // and cannot follow successfully — the title they named does not exist.+ // Say which ones did not resolve, and point at the exact-match rule,+ // instead of sending them round the loop a second time.+ const unresolved =+ supers.unmatched.length && supers.matched.length === 0+ ? [+ "",+ `Supersedes did not match any memory: ${supers.unmatched.join(", ")}.`,+ "Titles are matched exactly, case-insensitively — check the spelling against a recall result.",+ ]+ : [];+ if (duplicates.length && nameable === undefined) {+ audit("refused", { tool: "remember", reason: "near-duplicate out of reach" });+ return [+ "Not recorded: a near-identical memory already exists where this one would be read,",+ "outside what recall serves you, so it isn't named here.",+ "If this genuinely differs, pass force: true.",+ ...unresolved,+ ].join("\n");+ }+ if (nameable !== undefined) {+ const d = nameable;// If at least one title DID match, keep the original "pass supersedes"// advice for the colliding memory — a typo among several targets must// not hide the hint that still applies to the collision.- const advice =- supers.unmatched.length && supers.matched.length === 0- ? [- "If this genuinely differs, pass force: true.",- "",- `Supersedes did not match any memory: ${supers.unmatched.join(", ")}.`,- "Titles are matched exactly, case-insensitively — check the spelling against a recall result.",- ]- : [- "If this genuinely differs, pass force: true. If it CORRECTS that memory,",- "pass supersedes: [\"<its exact title>\"] instead — the old one is kept and back-linked.",- ];+ const advice = unresolved.length+ ? ["If this genuinely differs, pass force: true.", ...unresolved]+ : [+ "If this genuinely differs, pass force: true. If it CORRECTS that memory,",+ "pass supersedes: [\"<its exact title>\"] instead — the old one is kept and back-linked.",+ ];return [`Not recorded: a near-identical memory already exists (${Math.round(d.score * 100)}% similar).`,` ${d.entry.title}`,` ${d.entry.rel}`,+ // Said, so that following the advice above doesn't run into a+ // second refusal the caller was never told about.+ ...(unseen ? ["Another near-identical memory, outside what recall serves you, also matches."] : []),"",...advice,].join("\n");diff --git a/.claude/scripts/lib/mcp-tools.ts b/.claude/scripts/lib/mcp-tools.tsindex 857bc48..e0358b5 100644--- a/.claude/scripts/lib/mcp-tools.ts+++ b/.claude/scripts/lib/mcp-tools.ts@@ -224,7 +224,7 @@ export const TOOLS: readonly ToolDef[] = [force: {type: "boolean",description:- "Write even if a near-identical memory already exists. Default false — you will be told what it collided with.",+ "Write even if a near-identical memory already exists. Default false — you will be told what it collided with, or, if that memory is outside what recall serves you, only that one exists.",},dry_run: { type: "boolean", description: "Preview without writing (default false)." },},diff --git a/.claude/scripts/tests/memory-supersede-same-title.test.ts b/.claude/scripts/tests/memory-supersede-same-title.test.tsindex e817027..d24c85f 100644--- a/.claude/scripts/tests/memory-supersede-same-title.test.ts+++ b/.claude/scripts/tests/memory-supersede-same-title.test.ts@@ -1,5 +1,6 @@/**- * Supersession under a shared title (#364).+ * What `remember` may mark, compare and name, and for which caller: supersession+ * under a shared title (#364), and the near-duplicate check's reach (#375).** A correction that keeps its title is the normal path, so a title can name* several memories. Resolving a `supersedes` claim to the first of them marked@@ -9,8 +10,8 @@* coincidence is visible to the caller rather than retired silently.** Driven through the real `tools/call` dispatch, as mcp-refusal-audit does.- * Every call passes `force: true`: the near-duplicate check is not what these- * tests are about.+ * Calls pass `force: true` by default, so only the near-duplicate tests, which+ * pass `force: false`, meet that check.*/import { test, describe } from "node:test";@@ -27,7 +28,11 @@ import { rmTemp } from "./_helpers.ts";const TITLE = "tokens expire after one hour";const BODY = "A body written for a session that will not have this context, long enough to clear the minimum.";-function harness(vaultRoot: string): { remember: (at: Date, args: Record<string, unknown>) => Promise<string> } {+function harness(vaultRoot: string): {+ remember: (at: Date, args: Record<string, unknown>) => Promise<string>;+ audited: Array<{ action: string; detail: Record<string, unknown> }>;+} {+ const audited: Array<{ action: string; detail: Record<string, unknown> }> = [];let clock = new Date(2026, 5, 1);const session = {roots: [{ uri: "file:///elsewhere/atlas", name: "atlas" }],@@ -40,7 +45,7 @@ function harness(vaultRoot: string): { remember: (at: Date, args: Record<string,policy: { roots: ["brain"], never_expose: [] },session,qmd: () => ({}),- audit: () => {},+ audit: (action: string, detail: Record<string, unknown> = {}) => audited.push({ action, detail }),reindex: () => true,now: () => clock,} as never);@@ -52,11 +57,15 @@ function harness(vaultRoot: string): { remember: (at: Date, args: Record<string,})) as { content: { text: string }[] };return r.content[0]!.text;};- return { remember };+ return { remember, audited };}const recorded = (reply: string): string => /^Recorded: (.+)$/m.exec(reply)?.[1] ?? "";+/** Rescope a written memory to the harbor project, out of the atlas caller's reach. */+const scopeToHarbor = (file: string): void =>+ writeFileSync(file, readFileSync(file, "utf8").replace(/^scope: .*$/m, "scope: project").replace(/^projects: .*$/m, 'projects: ["harbor"]'));+describe("remember marks every memory a shared title names", () => {test("a third same-title correction retires the live wrong one, and the chain reaches only where its end does", async () => {const dir = mkdtempSync(join(tmpdir(), "sup-chain-"));@@ -134,7 +143,7 @@ describe("remember marks every memory a shared title names", () => {const { remember } = harness(dir);const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));const full = join(dir, theirs);- writeFileSync(full, readFileSync(full, "utf8").replace(/^scope: .*$/m, "scope: project").replace(/^projects: .*$/m, 'projects: ["harbor"]'));+ scopeToHarbor(full);const reply = await remember(new Date(2026, 6, 1), { scope: "general", supersedes: [TITLE] });assert.ok(!reply.includes(theirs), "the reply does not name a memory outside the caller's reach");@@ -171,7 +180,7 @@ describe("remember marks every memory a shared title names", () => {const withheld = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));const harborFix = recorded(await remember(new Date(2026, 5, 2), { scope: "general", title: "tokens expire after one hour (harbor)" }));const fixFull = join(dir, harborFix);- writeFileSync(fixFull, readFileSync(fixFull, "utf8").replace(/^scope: .*$/m, "scope: project").replace(/^projects: .*$/m, 'projects: ["harbor"]'));+ scopeToHarbor(fixFull);const full = join(dir, withheld);writeFileSync(full, readFileSync(full, "utf8").replace(/^scope: general$/m, 'scope: general\nsuperseded_by: ["tokens expire after one hour (harbor)"]'));@@ -183,3 +192,146 @@ describe("remember marks every memory a shared title names", () => {}});});++/**+ * The near-duplicate check judges a duplicate by where the NEW memory would be+ * read, in the caller's reach or not, and reach decides only what the refusal+ * names (#375). Comparing only against the caller's reach let a caller file a+ * second copy into a project it can't see, or restate a claim whose+ * correction it can't see and so undo that correction.+ */+describe("remember's near-duplicate check: compared where it would be read, named only in reach", () => {+ const RESTATED = `${BODY} Restated.`;++ test("a general write duplicating another project's memory is refused, without naming it", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-general-"));+ try {+ const { remember } = harness(dir);+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));+ scopeToHarbor(join(dir, theirs));++ const reply = await remember(new Date(2026, 6, 1), { scope: "general", force: false });+ assert.match(reply, /^Not recorded: a near-identical memory already exists where this one would be read/m);+ assert.ok(!reply.includes(theirs), "the reply does not name a memory outside the caller's reach");+ assert.ok(!reply.includes(TITLE), "nor its title");+ } finally {+ rmTemp(dir);+ }+ });++ test("a write scoped into another project is compared there, so it can't add a second copy", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-into-"));+ try {+ const { remember } = harness(dir);+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));+ scopeToHarbor(join(dir, theirs));++ const reply = await remember(new Date(2026, 6, 1), { scope: "project", projects: ["harbor"], force: false });+ assert.match(reply, /^Not recorded: a near-identical memory already exists where this one would be read/m);+ assert.ok(!reply.includes(theirs));+ } finally {+ rmTemp(dir);+ }+ });++ test("restating a claim whose correction is out of reach is refused, so the correction holds", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-retired-"));+ try {+ const { remember } = harness(dir);+ // A, general, corrected by a harbor-only memory: withheld from atlas. The+ // correction shares no wording with the restatement, so only A can match.+ const FIX_TITLE = "session keys rotate on a fixed weekly schedule";+ const retired = recorded(await remember(new Date(2026, 5, 1), { scope: "general", body: RESTATED }));+ const fix = recorded(+ await remember(new Date(2026, 5, 2), {+ scope: "general",+ title: FIX_TITLE,+ body: "Rotation runs every Monday from the scheduler, so a key older than seven days is a scheduler fault to investigate.",+ }),+ );+ scopeToHarbor(join(dir, fix));+ const full = join(dir, retired);+ writeFileSync(full, readFileSync(full, "utf8").replace(/^scope: general$/m, `scope: general\nsuperseded_by: ["${FIX_TITLE}"]`));++ const reply = await remember(new Date(2026, 6, 1), { scope: "general", body: RESTATED, force: false });+ assert.match(reply, /^Not recorded: a near-identical memory already exists where this one would be read/m);+ assert.ok(!reply.includes(retired), "the withheld memory is not named either");+ } finally {+ rmTemp(dir);+ }+ });++ test("a write whose facets another project's copy doesn't share is recorded: two projects may each hold one", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-own-"));+ try {+ const { remember } = harness(dir);+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));+ scopeToHarbor(join(dir, theirs));++ const reply = await remember(new Date(2026, 6, 1), { scope: "project", projects: ["atlas"], force: false });+ assert.match(reply, /^Recorded: /m);+ } finally {+ rmTemp(dir);+ }+ });++ test("a near-identical memory the caller can see still refuses the write, and is named (the control)", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-control-"));+ try {+ const { remember } = harness(dir);+ const mine = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));+ const reply = await remember(new Date(2026, 6, 1), { scope: "general", force: false });+ assert.match(reply, /near-identical/);+ assert.ok(reply.includes(mine), "a memory in reach is named");+ } finally {+ rmTemp(dir);+ }+ });++ test("the generic refusal is audited, and keeps the note that supersedes matched nothing", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-audit-"));+ try {+ const { remember, audited } = harness(dir);+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));+ scopeToHarbor(join(dir, theirs));++ const reply = await remember(new Date(2026, 6, 1), { scope: "general", supersedes: ["no such title"], force: false });+ assert.match(reply, /where this one would be read/);+ assert.match(reply, /Supersedes did not match any memory: no such title/);+ assert.ok(audited.some((a) => a.action === "refused" && a.detail["reason"] === "near-duplicate out of reach"));+ } finally {+ rmTemp(dir);+ }+ });++ test("a named refusal also says when a near-identical memory out of reach matches", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-mixed-"));+ try {+ const { remember } = harness(dir);+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));+ scopeToHarbor(join(dir, theirs));+ const mine = recorded(await remember(new Date(2026, 5, 2), { scope: "general", force: true }));++ const reply = await remember(new Date(2026, 6, 1), { scope: "general", force: false });+ assert.ok(reply.includes(mine), "the one in reach is named");+ assert.ok(!reply.includes(theirs), "the other is not");+ assert.match(reply, /Another near-identical memory, outside what recall serves you, also matches/);+ } finally {+ rmTemp(dir);+ }+ });++ test("a same-title correction chain records each correction without force", async () => {+ const dir = mkdtempSync(join(tmpdir(), "neardup-chain-"));+ try {+ const { remember } = harness(dir);+ await remember(new Date(2026, 5, 1), { scope: "general" });+ const second = await remember(new Date(2026, 6, 1), { scope: "general", supersedes: [TITLE], force: false });+ assert.match(second, /^Recorded: /m, second);+ const third = await remember(new Date(2026, 7, 1), { scope: "general", supersedes: [TITLE], force: false });+ assert.match(third, /^Recorded: /m, third);+ } finally {+ rmTemp(dir);+ }+ });+});diff --git a/ARCHITECTURE.md b/ARCHITECTURE.mdindex 2905eb7..3a65d26 100644--- a/ARCHITECTURE.md+++ b/ARCHITECTURE.md@@ -737,9 +737,9 @@ flowchart TBVault -->|no| Neut["neutralise dangling wikilinks<br/>in title AND body"]Neut --> Val["validateMemory<br/>epistemic contract"]Val -->|fails| Report["refused, with reasons"]- Val --> Sup0["resolve supersedes — the named targets<br/>drop out of the comparison"]- Sup0 --> Dup{"near-duplicate against the rest?<br/>facet-gated"}- Dup -->|"yes, and force is not set"| Collide["reports what it collided with —<br/>suggests supersedes, or names the<br/>supersedes titles that matched nothing"]+ Val --> Sup0["resolve supersedes, in the caller's reach —<br/>the named targets drop out of the comparison"]+ Sup0 --> Dup{"near-duplicate against the rest?<br/>facet-gated, wherever the new<br/>memory would be read"}+ Dup -->|"yes, and force is not set"| Collide["reports what it collided with, if the caller<br/>can see it — suggests supersedes, or names the<br/>supersedes titles that matched nothing;<br/>out of reach, refuses without naming it"]Dup -->|no| Links["resolve links — emitted only if the target exists"]Links --> Write["claimFile: atomic exclusive create"]Write --> Sup["mark superseded memories,<br/>kept and back-linked"]