Skip to content

A self-organizing Obsidian vault that gives AI coding agents persistent memory.

fix: run remember's near-duplicate check only against memories the caller can see (#387)

Brenno Ferrarid51ffe7e24ba
diff --git a/.claude/scripts/lib/mcp-server.ts b/.claude/scripts/lib/mcp-server.ts
index 33d900f..96bd662 100644
--- a/.claude/scripts/lib/mcp-server.ts
+++ b/.claude/scripts/lib/mcp-server.ts
@@ -533,56 +533,80 @@ export function createHandlers(deps: ServerDeps): Handlers {
// comparison; every OTHER memory stays in it, so a correction that also
// duplicates a third note is still refused, which is the check working.
//
- // Only what recall serves this caller (#364). A title can name a memory
- // scoped to another project, or one recall already withholds; marking it
- // would withhold it from its own project's callers, whose only replacement
- // is out of their reach, and naming its file here would show this caller a
+ // Only what recall serves this caller is marked or named (#364, #375). A
+ // memory scoped to another project, or one recall already withholds, is
+ // never marked and never named: naming its file would show this caller a
// memory recall never would. Nor is it counted: a count would say whether
// another project holds that title, so a claim with no match in reach is
// simply not matched, which is the truth from where the caller stands.
+ // The near-duplicate refusal below is the one accepted exception: it says
+ // that near-identical content exists where the new memory would be read,
+ // without naming it, because keeping a correction intact outweighs that
+ // hint (#375). It is audited.
+ // Building `served` itself needs every memory, since a stale copy's
+ // replacement may be out of reach.
const served = servedAfterSupersession(
digests.map((d) => ({ rel: d.rel, full: d.full, title: d.title, body: d.body, facets: d })),
who,
);
- const supers = resolveSupersedes(
- args.supersedes,
- digests.filter((d) => served.has(d.rel)),
- );
+ const inReach = digests.filter((d) => served.has(d.rel));
+ const supers = resolveSupersedes(args.supersedes, inReach);
const superseded = new Set(supers.matched.map((m) => m.rel));
// Near-duplicate suppression, facet-gated so two projects can each hold
- // their own copy of the same lesson.
+ // their own copy of the same lesson. A duplicate is judged by where the
+ // NEW memory would be read, so it is compared against every memory that
+ // shares a facet with it, in the caller's reach or not (#375): otherwise
+ // a caller could file a second copy into a project it can't see, or
+ // restate a claim whose correction it can't see and undo that correction.
+ // Reach decides only what the refusal may name.
if (args.force !== true) {
const { duplicates } = findSimilar(
v.value,
digests.filter((d) => !superseded.has(d.rel)),
);
- if (duplicates.length) {
- const d = duplicates[0]!;
- // When the caller DID pass supersedes and NONE of them resolved,
- // repeating "pass supersedes" is an instruction they already followed
- // and cannot follow successfully — the title they named does not exist.
- // Say which ones did not resolve, and point at the exact-match rule,
- // instead of sending them round the loop a second time.
+ const nameable = duplicates.find((dup) => served.has(dup.entry.rel));
+ const unseen = duplicates.some((dup) => !served.has(dup.entry.rel));
+ // When the caller DID pass supersedes and NONE of them resolved,
+ // repeating "pass supersedes" is an instruction they already followed
+ // and cannot follow successfully — the title they named does not exist.
+ // Say which ones did not resolve, and point at the exact-match rule,
+ // instead of sending them round the loop a second time.
+ const unresolved =
+ supers.unmatched.length && supers.matched.length === 0
+ ? [
+ "",
+ `Supersedes did not match any memory: ${supers.unmatched.join(", ")}.`,
+ "Titles are matched exactly, case-insensitively — check the spelling against a recall result.",
+ ]
+ : [];
+ if (duplicates.length && nameable === undefined) {
+ audit("refused", { tool: "remember", reason: "near-duplicate out of reach" });
+ return [
+ "Not recorded: a near-identical memory already exists where this one would be read,",
+ "outside what recall serves you, so it isn't named here.",
+ "If this genuinely differs, pass force: true.",
+ ...unresolved,
+ ].join("\n");
+ }
+ if (nameable !== undefined) {
+ const d = nameable;
// If at least one title DID match, keep the original "pass supersedes"
// advice for the colliding memory — a typo among several targets must
// not hide the hint that still applies to the collision.
- const advice =
- supers.unmatched.length && supers.matched.length === 0
- ? [
- "If this genuinely differs, pass force: true.",
- "",
- `Supersedes did not match any memory: ${supers.unmatched.join(", ")}.`,
- "Titles are matched exactly, case-insensitively — check the spelling against a recall result.",
- ]
- : [
- "If this genuinely differs, pass force: true. If it CORRECTS that memory,",
- "pass supersedes: [\"<its exact title>\"] instead — the old one is kept and back-linked.",
- ];
+ const advice = unresolved.length
+ ? ["If this genuinely differs, pass force: true.", ...unresolved]
+ : [
+ "If this genuinely differs, pass force: true. If it CORRECTS that memory,",
+ "pass supersedes: [\"<its exact title>\"] instead — the old one is kept and back-linked.",
+ ];
return [
`Not recorded: a near-identical memory already exists (${Math.round(d.score * 100)}% similar).`,
` ${d.entry.title}`,
` ${d.entry.rel}`,
+ // Said, so that following the advice above doesn't run into a
+ // second refusal the caller was never told about.
+ ...(unseen ? ["Another near-identical memory, outside what recall serves you, also matches."] : []),
"",
...advice,
].join("\n");
diff --git a/.claude/scripts/lib/mcp-tools.ts b/.claude/scripts/lib/mcp-tools.ts
index 857bc48..e0358b5 100644
--- a/.claude/scripts/lib/mcp-tools.ts
+++ b/.claude/scripts/lib/mcp-tools.ts
@@ -224,7 +224,7 @@ export const TOOLS: readonly ToolDef[] = [
force: {
type: "boolean",
description:
- "Write even if a near-identical memory already exists. Default false — you will be told what it collided with.",
+ "Write even if a near-identical memory already exists. Default false — you will be told what it collided with, or, if that memory is outside what recall serves you, only that one exists.",
},
dry_run: { type: "boolean", description: "Preview without writing (default false)." },
},
diff --git a/.claude/scripts/tests/memory-supersede-same-title.test.ts b/.claude/scripts/tests/memory-supersede-same-title.test.ts
index e817027..d24c85f 100644
--- a/.claude/scripts/tests/memory-supersede-same-title.test.ts
+++ b/.claude/scripts/tests/memory-supersede-same-title.test.ts
@@ -1,5 +1,6 @@
/**
- * Supersession under a shared title (#364).
+ * What `remember` may mark, compare and name, and for which caller: supersession
+ * under a shared title (#364), and the near-duplicate check's reach (#375).
*
* A correction that keeps its title is the normal path, so a title can name
* several memories. Resolving a `supersedes` claim to the first of them marked
@@ -9,8 +10,8 @@
* coincidence is visible to the caller rather than retired silently.
*
* Driven through the real `tools/call` dispatch, as mcp-refusal-audit does.
- * Every call passes `force: true`: the near-duplicate check is not what these
- * tests are about.
+ * Calls pass `force: true` by default, so only the near-duplicate tests, which
+ * pass `force: false`, meet that check.
*/
import { test, describe } from "node:test";
@@ -27,7 +28,11 @@ import { rmTemp } from "./_helpers.ts";
const TITLE = "tokens expire after one hour";
const BODY = "A body written for a session that will not have this context, long enough to clear the minimum.";
-function harness(vaultRoot: string): { remember: (at: Date, args: Record<string, unknown>) => Promise<string> } {
+function harness(vaultRoot: string): {
+ remember: (at: Date, args: Record<string, unknown>) => Promise<string>;
+ audited: Array<{ action: string; detail: Record<string, unknown> }>;
+} {
+ const audited: Array<{ action: string; detail: Record<string, unknown> }> = [];
let clock = new Date(2026, 5, 1);
const session = {
roots: [{ uri: "file:///elsewhere/atlas", name: "atlas" }],
@@ -40,7 +45,7 @@ function harness(vaultRoot: string): { remember: (at: Date, args: Record<string,
policy: { roots: ["brain"], never_expose: [] },
session,
qmd: () => ({}),
- audit: () => {},
+ audit: (action: string, detail: Record<string, unknown> = {}) => audited.push({ action, detail }),
reindex: () => true,
now: () => clock,
} as never);
@@ -52,11 +57,15 @@ function harness(vaultRoot: string): { remember: (at: Date, args: Record<string,
})) as { content: { text: string }[] };
return r.content[0]!.text;
};
- return { remember };
+ return { remember, audited };
}
const recorded = (reply: string): string => /^Recorded: (.+)$/m.exec(reply)?.[1] ?? "";
+/** Rescope a written memory to the harbor project, out of the atlas caller's reach. */
+const scopeToHarbor = (file: string): void =>
+ writeFileSync(file, readFileSync(file, "utf8").replace(/^scope: .*$/m, "scope: project").replace(/^projects: .*$/m, 'projects: ["harbor"]'));
+
describe("remember marks every memory a shared title names", () => {
test("a third same-title correction retires the live wrong one, and the chain reaches only where its end does", async () => {
const dir = mkdtempSync(join(tmpdir(), "sup-chain-"));
@@ -134,7 +143,7 @@ describe("remember marks every memory a shared title names", () => {
const { remember } = harness(dir);
const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
const full = join(dir, theirs);
- writeFileSync(full, readFileSync(full, "utf8").replace(/^scope: .*$/m, "scope: project").replace(/^projects: .*$/m, 'projects: ["harbor"]'));
+ scopeToHarbor(full);
const reply = await remember(new Date(2026, 6, 1), { scope: "general", supersedes: [TITLE] });
assert.ok(!reply.includes(theirs), "the reply does not name a memory outside the caller's reach");
@@ -171,7 +180,7 @@ describe("remember marks every memory a shared title names", () => {
const withheld = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
const harborFix = recorded(await remember(new Date(2026, 5, 2), { scope: "general", title: "tokens expire after one hour (harbor)" }));
const fixFull = join(dir, harborFix);
- writeFileSync(fixFull, readFileSync(fixFull, "utf8").replace(/^scope: .*$/m, "scope: project").replace(/^projects: .*$/m, 'projects: ["harbor"]'));
+ scopeToHarbor(fixFull);
const full = join(dir, withheld);
writeFileSync(full, readFileSync(full, "utf8").replace(/^scope: general$/m, 'scope: general\nsuperseded_by: ["tokens expire after one hour (harbor)"]'));
@@ -183,3 +192,146 @@ describe("remember marks every memory a shared title names", () => {
}
});
});
+
+/**
+ * The near-duplicate check judges a duplicate by where the NEW memory would be
+ * read, in the caller's reach or not, and reach decides only what the refusal
+ * names (#375). Comparing only against the caller's reach let a caller file a
+ * second copy into a project it can't see, or restate a claim whose
+ * correction it can't see and so undo that correction.
+ */
+describe("remember's near-duplicate check: compared where it would be read, named only in reach", () => {
+ const RESTATED = `${BODY} Restated.`;
+
+ test("a general write duplicating another project's memory is refused, without naming it", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-general-"));
+ try {
+ const { remember } = harness(dir);
+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
+ scopeToHarbor(join(dir, theirs));
+
+ const reply = await remember(new Date(2026, 6, 1), { scope: "general", force: false });
+ assert.match(reply, /^Not recorded: a near-identical memory already exists where this one would be read/m);
+ assert.ok(!reply.includes(theirs), "the reply does not name a memory outside the caller's reach");
+ assert.ok(!reply.includes(TITLE), "nor its title");
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("a write scoped into another project is compared there, so it can't add a second copy", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-into-"));
+ try {
+ const { remember } = harness(dir);
+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
+ scopeToHarbor(join(dir, theirs));
+
+ const reply = await remember(new Date(2026, 6, 1), { scope: "project", projects: ["harbor"], force: false });
+ assert.match(reply, /^Not recorded: a near-identical memory already exists where this one would be read/m);
+ assert.ok(!reply.includes(theirs));
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("restating a claim whose correction is out of reach is refused, so the correction holds", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-retired-"));
+ try {
+ const { remember } = harness(dir);
+ // A, general, corrected by a harbor-only memory: withheld from atlas. The
+ // correction shares no wording with the restatement, so only A can match.
+ const FIX_TITLE = "session keys rotate on a fixed weekly schedule";
+ const retired = recorded(await remember(new Date(2026, 5, 1), { scope: "general", body: RESTATED }));
+ const fix = recorded(
+ await remember(new Date(2026, 5, 2), {
+ scope: "general",
+ title: FIX_TITLE,
+ body: "Rotation runs every Monday from the scheduler, so a key older than seven days is a scheduler fault to investigate.",
+ }),
+ );
+ scopeToHarbor(join(dir, fix));
+ const full = join(dir, retired);
+ writeFileSync(full, readFileSync(full, "utf8").replace(/^scope: general$/m, `scope: general\nsuperseded_by: ["${FIX_TITLE}"]`));
+
+ const reply = await remember(new Date(2026, 6, 1), { scope: "general", body: RESTATED, force: false });
+ assert.match(reply, /^Not recorded: a near-identical memory already exists where this one would be read/m);
+ assert.ok(!reply.includes(retired), "the withheld memory is not named either");
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("a write whose facets another project's copy doesn't share is recorded: two projects may each hold one", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-own-"));
+ try {
+ const { remember } = harness(dir);
+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
+ scopeToHarbor(join(dir, theirs));
+
+ const reply = await remember(new Date(2026, 6, 1), { scope: "project", projects: ["atlas"], force: false });
+ assert.match(reply, /^Recorded: /m);
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("a near-identical memory the caller can see still refuses the write, and is named (the control)", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-control-"));
+ try {
+ const { remember } = harness(dir);
+ const mine = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
+ const reply = await remember(new Date(2026, 6, 1), { scope: "general", force: false });
+ assert.match(reply, /near-identical/);
+ assert.ok(reply.includes(mine), "a memory in reach is named");
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("the generic refusal is audited, and keeps the note that supersedes matched nothing", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-audit-"));
+ try {
+ const { remember, audited } = harness(dir);
+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
+ scopeToHarbor(join(dir, theirs));
+
+ const reply = await remember(new Date(2026, 6, 1), { scope: "general", supersedes: ["no such title"], force: false });
+ assert.match(reply, /where this one would be read/);
+ assert.match(reply, /Supersedes did not match any memory: no such title/);
+ assert.ok(audited.some((a) => a.action === "refused" && a.detail["reason"] === "near-duplicate out of reach"));
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("a named refusal also says when a near-identical memory out of reach matches", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-mixed-"));
+ try {
+ const { remember } = harness(dir);
+ const theirs = recorded(await remember(new Date(2026, 5, 1), { scope: "general" }));
+ scopeToHarbor(join(dir, theirs));
+ const mine = recorded(await remember(new Date(2026, 5, 2), { scope: "general", force: true }));
+
+ const reply = await remember(new Date(2026, 6, 1), { scope: "general", force: false });
+ assert.ok(reply.includes(mine), "the one in reach is named");
+ assert.ok(!reply.includes(theirs), "the other is not");
+ assert.match(reply, /Another near-identical memory, outside what recall serves you, also matches/);
+ } finally {
+ rmTemp(dir);
+ }
+ });
+
+ test("a same-title correction chain records each correction without force", async () => {
+ const dir = mkdtempSync(join(tmpdir(), "neardup-chain-"));
+ try {
+ const { remember } = harness(dir);
+ await remember(new Date(2026, 5, 1), { scope: "general" });
+ const second = await remember(new Date(2026, 6, 1), { scope: "general", supersedes: [TITLE], force: false });
+ assert.match(second, /^Recorded: /m, second);
+ const third = await remember(new Date(2026, 7, 1), { scope: "general", supersedes: [TITLE], force: false });
+ assert.match(third, /^Recorded: /m, third);
+ } finally {
+ rmTemp(dir);
+ }
+ });
+});
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 2905eb7..3a65d26 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -737,9 +737,9 @@ flowchart TB
Vault -->|no| Neut["neutralise dangling wikilinks<br/>in title AND body"]
Neut --> Val["validateMemory<br/>epistemic contract"]
Val -->|fails| Report["refused, with reasons"]
- Val --> Sup0["resolve supersedes — the named targets<br/>drop out of the comparison"]
- Sup0 --> Dup{"near-duplicate against the rest?<br/>facet-gated"}
- Dup -->|"yes, and force is not set"| Collide["reports what it collided with —<br/>suggests supersedes, or names the<br/>supersedes titles that matched nothing"]
+ Val --> Sup0["resolve supersedes, in the caller's reach —<br/>the named targets drop out of the comparison"]
+ Sup0 --> Dup{"near-duplicate against the rest?<br/>facet-gated, wherever the new<br/>memory would be read"}
+ Dup -->|"yes, and force is not set"| Collide["reports what it collided with, if the caller<br/>can see it — suggests supersedes, or names the<br/>supersedes titles that matched nothing;<br/>out of reach, refuses without naming it"]
Dup -->|no| Links["resolve links — emitted only if the target exists"]
Links --> Write["claimFile: atomic exclusive create"]
Write --> Sup["mark superseded memories,<br/>kept and back-linked"]