Self-hosted MCP server + dashboard giving Claude Code, Cursor & Hermes agents shared long-term memory over a git-backed markdown vault. Agent-native, Obsidian-compatible second brain — markdown RAG, no vector DB.
TypeScript agent-memoryai-agentsanthropicclaude-codecursorgitknowledge-baseknowledge-graphllmmarkdownmcpmcp-servermodel-context-protocolnote-takingobsidianpkmragsecond-brainself-hostedzettelkasten
feat(search): rank by authority, not just relevance
Ronalds Palacisddf8d32f320f
diff --git a/app/api/search/route.ts b/app/api/search/route.tsindex 2774dc0..bb0ff83 100644--- a/app/api/search/route.ts+++ b/app/api/search/route.ts@@ -3,6 +3,9 @@ import { searchNotes } from "@/lib/vault/store";export const dynamic = "force-dynamic";export async function GET(req: Request) {- const q = new URL(req.url).searchParams.get("q") ?? "";- return Response.json({ results: searchNotes(q) });+ const params = new URL(req.url).searchParams;+ const q = params.get("q") ?? "";+ // Humans browsing the dashboard should be able to find archived notes; they still rank+ // far below live ones. Agents (MCP) get them excluded by default.+ return Response.json({ results: searchNotes(q, { includeArchive: params.get("archive") !== "false" }) });}diff --git a/lib/config.ts b/lib/config.tsindex 5be6743..6663d74 100644--- a/lib/config.ts+++ b/lib/config.ts@@ -29,6 +29,17 @@ export const VAULT_IGNORE = new Set(.filter(Boolean),);+/**+ * Top-level folders whose notes are historical: demoted in search and excluded by default.+ * Purely a ranking hint — the notes stay readable, and nothing is hidden from brain_read.+ */+export const ARCHIVE_FOLDERS = new Set(+ (process.env.ARCHIVE_FOLDERS ?? "archive,archives,_archive,trash")+ .split(",")+ .map((s) => s.trim().toLowerCase())+ .filter(Boolean),+);+/** Bearer token required to call the MCP endpoint. */export const MCP_TOKEN = process.env.MCP_TOKEN ?? "";diff --git a/lib/mcp/tools.ts b/lib/mcp/tools.tsindex dc6679d..0e21ba3 100644--- a/lib/mcp/tools.ts+++ b/lib/mcp/tools.ts@@ -4,9 +4,13 @@ import {getNote,getTree,listNotes,+ listRecent,readVaultFile,searchNotes,+ vaultConventions,} from "@/lib/vault/store";+import { authorityOf } from "@/lib/vault/authority";+import { extractSection, listHeadings } from "@/lib/vault/parse";import {appendNote,createFolder,@@ -55,42 +59,110 @@ export const TOOLS: Tool[] = [{name: "brain_schema",description:- "Return the vault's SCHEMA.md — folder taxonomy, frontmatter conventions, wikilink model, and the write protocol. READ THIS FIRST before writing notes.",+ "READ THIS FIRST, before searching or writing. Returns the vault's SCHEMA.md (folder taxonomy, frontmatter conventions, wikilink model, write protocol) AND a live description of this specific vault: its folders, the `status:` values actually in use, how search ranks notes by authority, and any integrity warnings. Vaults differ — never assume conventions, read them here.",inputSchema: { type: "object", properties: {} },- handler: () => readVaultFile("SCHEMA.md") ?? "(no SCHEMA.md in this vault)",+ handler: () => ({+ schema: readVaultFile("SCHEMA.md") ?? "(no SCHEMA.md in this vault)",+ conventions: vaultConventions(),+ }),},{name: "brain_search",- description: "Full-text + fuzzy search across all notes. Returns ranked results (path, title, folder).",+ description:+ "Keyword search across the vault. Returns ranked hits with `path`, `title`, `folder`, `status`, `snippet`, and `authority`.\n\n" ++ "IMPORTANT — ranking is by keyword relevance, NOT by truth. A superseded document repeats the query words just as often as the live one, so it can outrank it. Every hit carries an `authority` field: `authoritative` (source of truth — prefer it), `current`, `provisional` (draft/proposed — never quote as settled), `superseded`, `archived`. Rank order is a suggestion; `authority` is the signal.\n\n" ++ "Before quoting a price, guarantee, contract term, or any other single-valued fact, open the `authoritative` note and read it. If two notes disagree on such a fact, that is a defect in the vault — report it rather than averaging them.\n\n" ++ "Archived notes are excluded unless `includeArchive` is true.",inputSchema: {type: "object",- properties: { query: s("search query"), limit: { type: "number", description: "max results (default 20)" } },+ properties: {+ query: s("search query — a few keywords beat a full sentence"),+ limit: { type: "number", description: "max results (default 20)" },+ folder: s("restrict to one top-level folder, e.g. 'decisions'"),+ includeArchive: { type: "boolean", description: "include historical notes in archive folders (default false)" },+ },required: ["query"],},- handler: ({ query, limit }) => searchNotes(String(query ?? ""), typeof limit === "number" ? limit : 20),+ handler: ({ query, limit, folder, includeArchive }) =>+ searchNotes(String(query ?? ""), {+ limit: typeof limit === "number" ? limit : 20,+ folder: folder ? String(folder) : undefined,+ includeArchive: includeArchive === true,+ }),},{name: "brain_read",- description: "Read a note's full markdown (frontmatter + body) plus its backlinks. Path is vault-relative, e.g. 'clients/mks/mks.md'.",- inputSchema: { type: "object", properties: { path: s("vault-relative path") }, required: ["path"] },- handler: ({ path }) => {+ description:+ "Read a note: full markdown (frontmatter + body), its `authority`, and its backlinks. Path is vault-relative, e.g. 'clients/mks/mks.md'.\n\n" ++ "Pass `section` to read just one heading's content instead of the whole file — cheaper on long documents. If the heading isn't found you get the list of available headings back.\n\n" ++ "Always check the returned `authority` before acting on the content: `archived` and `superseded` notes are history, not instructions.",+ inputSchema: {+ type: "object",+ properties: {+ path: s("vault-relative path"),+ section: s("optional heading to extract, e.g. 'Pricing' — matches case-insensitively by prefix"),+ },+ required: ["path"],+ },+ handler: ({ path, section }) => {const n = getNote(String(path));if (!n) return { error: "not found", path };- return {+ const base = {path: n.path,title: n.title,+ status: n.status,+ authority: authorityOf(n),+ tags: n.tags,frontmatter: n.frontmatter,- content: n.raw,backlinks: getBacklinks(n.path).map((b) => b.path),};+ if (!section) return { ...base, content: n.raw };++ const found = extractSection(n.body, String(section));+ if (found === null) {+ return { ...base, error: `section "${section}" not found`, sections: listHeadings(n.body) };+ }+ return { ...base, section: String(section), content: found, sections: listHeadings(n.body) };},},{name: "brain_list",- description: "List all notes with metadata (path, title, folder, type, tags, status). Use to discover what exists.",+ description:+ "List every note with metadata (path, title, folder, type, tags, status, authority). Use to discover what exists. `authority` tells you which notes are source-of-truth and which are history.",inputSchema: { type: "object", properties: {} },handler: () =>- listNotes().map((n) => ({ path: n.path, title: n.title, folder: n.folder, type: n.type, tags: n.tags, status: n.status })),+ listNotes().map((n) => ({+ path: n.path,+ title: n.title,+ folder: n.folder,+ type: n.type,+ tags: n.tags,+ status: n.status,+ authority: authorityOf(n),+ })),+ },+ {+ name: "brain_recent",+ description:+ "Notes changed most recently, newest first. Use to catch up on what humans or other agents have done since you last looked.",+ inputSchema: {+ type: "object",+ properties: {+ since: s("optional ISO date/time — only notes modified at or after this"),+ limit: { type: "number", description: "max results (default 20)" },+ },+ },+ handler: ({ since, limit }) => {+ const t = since ? Date.parse(String(since)) : NaN;+ return listRecent(Number.isNaN(t) ? undefined : t, typeof limit === "number" ? limit : 20).map((n) => ({+ path: n.path,+ title: n.title,+ folder: n.folder,+ status: n.status,+ authority: authorityOf(n),+ modified: new Date(n.mtimeMs).toISOString(),+ }));+ },},{name: "brain_tree",@@ -154,7 +226,8 @@ export const TOOLS: Tool[] = [},{name: "brain_move",- description: "Move or rename a note.",+ description:+ "Move or rename a note. This is how you retire something: when a note stops being true, move it to an archive folder (see brain_schema → conventions.ranking.archiveFolders) rather than deleting it. Archived notes are demoted in search and excluded by default, so they stop misleading agents while the reasoning trail survives. Leave a pointer in the replacement note saying what superseded what.",inputSchema: { type: "object", properties: { from: s("current path"), to: s("new path") }, required: ["from", "to"] },handler: async ({ from, to }) => ({ ok: true, path: await moveNote(String(from), String(to)) }),},@@ -173,7 +246,8 @@ export const TOOLS: Tool[] = [},{name: "brain_delete",- description: "Delete a note from the working copy (recoverable via git history).",+ description:+ "Delete a note (recoverable via git history). Prefer brain_move into an archive folder — deleting destroys the reasoning trail, archiving only removes it from search. Delete when the note is wrong or duplicated, archive when it is merely no longer true.",inputSchema: { type: "object", properties: { path: s("vault-relative path") }, required: ["path"] },handler: async ({ path }) => {await deleteNote(String(path));diff --git a/lib/vault/authority.ts b/lib/vault/authority.tsnew file mode 100644index 0000000..074e447--- /dev/null+++ b/lib/vault/authority.ts@@ -0,0 +1,80 @@+import { ARCHIVE_FOLDERS } from "@/lib/config";++/**+ * Authority — how much a note should be trusted, independent of how well it matches a query.+ *+ * Search engines rank by *resemblance*. They cannot tell a live contract from a dead one:+ * a superseded price list mentions "price" just as often as the real one, so it can easily+ * outrank it. Authority is the missing axis. It is derived entirely from the files+ * (path + frontmatter), never stored separately, so the index stays disposable.+ */+export type Authority = "authoritative" | "current" | "provisional" | "superseded" | "archived";++/** Status/tag words that mark a note as the source of truth for whatever it covers. */+const AUTHORITATIVE = ["locked", "canonical", "source-of-truth", "authoritative"];+/** Words that mark a note as no longer true. */+const SUPERSEDED = ["superseded", "deprecated", "obsolete", "replaced", "retired", "dead"];+/** Words that mark a note as not yet decided. A `proposed` decision is not a decision. */+const PROVISIONAL = ["draft", "proposed", "exploring", "tentative", "wip", "idea"];++/** Multiplier applied to a note's search score. Files are truth; this only reorders hits. */+const WEIGHT: Record<Authority, number> = {+ authoritative: 3.5,+ current: 1,+ provisional: 0.55,+ superseded: 0.15,+ archived: 0.08,+};++/** One-line explanation per class, surfaced to agents via brain_schema. */+export const AUTHORITY_MEANING: Record<Authority, string> = {+ authoritative: "Source of truth for its subject. Quote this over anything that disagrees.",+ current: "Live, but not declared authoritative. Fine for context.",+ provisional: "Not decided yet (draft/proposed). Never quote as settled.",+ superseded: "Explicitly replaced. Historical only.",+ archived: "Lives in an archive folder. Historical only — never quote as current.",+};++/** Match `needle` as a whole token inside `hay` (so "draft" hits "draft-for-approval"). */+function hasWord(hay: string, needle: string): boolean {+ const esc = needle.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");+ return new RegExp(`(^|[^a-z0-9])${esc}([^a-z0-9]|$)`, "i").test(hay);+}++/** True when the note's top-level folder is an archive folder. */+export function isArchivedPath(relPath: string): boolean {+ return ARCHIVE_FOLDERS.has(relPath.split("/")[0].toLowerCase());+}++/**+ * Classify a note. Order matters: a locked note inside `archive/` is still archived,+ * and an explicitly superseded note is dead however loudly it calls itself canonical.+ */+export function authorityOf(meta: { path: string; status?: string; tags?: string[] }): Authority {+ if (isArchivedPath(meta.path)) return "archived";+ const hay = [meta.status ?? "", ...(meta.tags ?? [])].join(" ").toLowerCase();+ if (SUPERSEDED.some((w) => hasWord(hay, w))) return "superseded";+ if (PROVISIONAL.some((w) => hasWord(hay, w))) return "provisional";+ if (AUTHORITATIVE.some((w) => hasWord(hay, w))) return "authoritative";+ return "current";+}++export function weightOf(a: Authority): number {+ return WEIGHT[a];+}++/**+ * The live ranking contract, returned by brain_schema so an agent working against a vault+ * it has never seen can discover the rules instead of guessing them.+ */+export function authorityRules() {+ return {+ summary:+ "Search ranks by keyword relevance, then multiplies by authority. Authority comes from the note's folder and frontmatter — never from the text.",+ archiveFolders: [...ARCHIVE_FOLDERS],+ statusWords: { authoritative: AUTHORITATIVE, superseded: SUPERSEDED, provisional: PROVISIONAL },+ weights: WEIGHT,+ meaning: AUTHORITY_MEANING,+ note: "Words are matched in both `status:` and `tags:`. A note with no status is `current` (weight 1) — vaults that use no conventions rank purely by relevance, exactly as before.",+ };+}diff --git a/lib/vault/parse.ts b/lib/vault/parse.tsindex 8530d5d..5223f54 100644--- a/lib/vault/parse.ts+++ b/lib/vault/parse.ts@@ -50,6 +50,47 @@ export function extractRawLinks(body: string, frontmatter: Record<string, unknowreturn links;}+/** Every markdown heading in a body, in document order, without the leading #s. */+export function listHeadings(body: string): string[] {+ return body+ .split("\n")+ .filter((l) => /^#{1,6}\s+\S/.test(l))+ .map((l) => l.replace(/^#{1,6}\s+/, "").trim());+}++/**+ * Return one section of a note: the matching heading plus everything under it, stopping at+ * the next heading of the same or higher level. Lets an agent read 40 lines of a 1000-line+ * document instead of all of it. Matches case-insensitively, by prefix.+ */+export function extractSection(body: string, heading: string): string | null {+ const lines = body.split("\n");+ const want = heading.replace(/^#+\s*/, "").trim().toLowerCase();+ if (!want) return null;++ let start = -1;+ let level = 0;+ for (let i = 0; i < lines.length; i++) {+ const m = /^(#{1,6})\s+(.*)$/.exec(lines[i]);+ if (!m) continue;+ const text = m[2].trim().toLowerCase();+ if (text === want || text.startsWith(want)) {+ start = i;+ level = m[1].length;+ break;+ }+ }+ if (start === -1) return null;++ const out = [lines[start]];+ for (let i = start + 1; i < lines.length; i++) {+ const m = /^(#{1,6})\s+/.exec(lines[i]);+ if (m && m[1].length <= level) break;+ out.push(lines[i]);+ }+ return out.join("\n").trim();+}+export interface ParsedNote {meta: Omit<NoteMeta, "mtimeMs">;body: string;@@ -78,6 +119,7 @@ export function parseNote(relPath: string, raw: string): ParsedNote {folder: folderOf(relPath),type: typeof data.type === "string" ? data.type : tags[0],tags,+ aliases: normalizeTags(data.aliases),status: data.status != null ? String(data.status) : undefined,created: data.created != null ? String(data.created) : undefined,updated:diff --git a/lib/vault/store.ts b/lib/vault/store.tsindex 1752c57..0a3141a 100644--- a/lib/vault/store.ts+++ b/lib/vault/store.ts@@ -6,15 +6,34 @@ import { VAULT_IGNORE } from "@/lib/config";import { activeVaultDir } from "@/lib/repos";import { scanVault } from "./scan";import { parseNote, stemOf } from "./parse";+import { authorityOf, authorityRules, isArchivedPath, weightOf, type Authority } from "./authority";import type { Graph, GraphEdge, GraphNode, Note, NoteMeta, TreeNode } from "./types";+interface IndexDoc {+ id: string;+ title: string;+ aliases: string;+ tags: string;+ body: string;+ folder: string;+ type: string;+ /** stored, not indexed — used for filtering + authority-weighted ranking */+ status: string;+ authority: Authority;+ mtimeMs: number;+}+interface IndexState {dir: string;notes: Map<string, NoteMeta>;+ /** path -> wikilink target stems. Kept so link/graph rebuilds need no disk IO. */+ linksBySource: Map<string, string[]>;stemToPath: Map<string, string>;+ /** stems claimed by more than one file — wikilinks to these resolve to the first, silently. */+ duplicateStems: Map<string, string[]>;outEdges: Map<string, Set<string>>;inEdges: Map<string, Set<string>>;- search: MiniSearch;+ search: MiniSearch<IndexDoc>;builtAt: number;}@@ -22,52 +41,147 @@ let state: IndexState | null = null;let watcher: FSWatcher | null = null;let watchedDir = "";-function buildState(): IndexState {- const dir = activeVaultDir();- const files = scanVault(dir);- const notes = new Map<string, NoteMeta>();- const stemToPath = new Map<string, string>();- const linksBySource = new Map<string, string[]>();- const docs: Array<{ id: string; title: string; tags: string; body: string; folder: string; type: string }> = [];-- for (const f of files) {- let raw = "";- try {- raw = fs.readFileSync(f.abs, "utf8");- } catch {- continue;- }- const { meta, body, rawLinks } = parseNote(f.rel, raw);- const nm: NoteMeta = { ...meta, mtimeMs: f.mtimeMs };- notes.set(f.rel, nm);- if (!stemToPath.has(nm.slug)) {- stemToPath.set(nm.slug, f.rel);- } else if (process.env.NODE_ENV !== "production") {- console.warn(`[vault] duplicate stem "${nm.slug}" (${stemToPath.get(nm.slug)} vs ${f.rel}) — links resolve to the first.`);+function newIndex(): MiniSearch<IndexDoc> {+ return new MiniSearch<IndexDoc>({+ fields: ["title", "aliases", "tags", "body", "folder", "type"],+ storeFields: ["title", "folder", "type", "status", "authority", "mtimeMs"],+ searchOptions: {+ boost: { title: 3, aliases: 3, tags: 2 },+ prefix: true,+ fuzzy: 0.2,+ // OR, deliberately. With AND, one query word the canonical note happens to lack+ // ("tiers") drops it from the results entirely — and the superseded note that *does*+ // use the word becomes the only answer. Authority weighting can only reorder what+ // survives the combiner, so the combiner must not throw the truth away.+ // Docs matching more terms still score higher; noise is handled by ranking, not exclusion.+ combineWith: "OR",+ },+ });+}++function toDoc(meta: NoteMeta, body: string): IndexDoc {+ return {+ id: meta.path,+ title: meta.title,+ aliases: meta.aliases.join(" "),+ tags: meta.tags.join(" "),+ body,+ folder: meta.folder,+ type: meta.type ?? "",+ status: meta.status ?? "",+ authority: authorityOf(meta),+ mtimeMs: meta.mtimeMs,+ };+}++function readAndParse(+ dir: string,+ rel: string,+ mtimeMs: number,+): { meta: NoteMeta; body: string; stems: string[] } | null {+ let raw: string;+ try {+ raw = fs.readFileSync(path.join(dir, rel), "utf8");+ } catch {+ return null;+ }+ const { meta, body, rawLinks } = parseNote(rel, raw);+ return { meta: { ...meta, mtimeMs }, body, stems: rawLinks.map((l) => stemOf(l.target)) };+}++/**+ * Rebuild stem resolution + the link graph from in-memory maps only.+ * Cheap (no disk IO), so it can run after every single-file change.+ */+function recomputeLinks(s: IndexState): void {+ s.stemToPath = new Map();+ s.duplicateStems = new Map();+ for (const rel of [...s.notes.keys()].sort()) {+ const slug = s.notes.get(rel)!.slug;+ const first = s.stemToPath.get(slug);+ if (first === undefined) {+ s.stemToPath.set(slug, rel);+ } else {+ const dupes = s.duplicateStems.get(slug) ?? [first];+ dupes.push(rel);+ s.duplicateStems.set(slug, dupes);}- linksBySource.set(f.rel, rawLinks.map((l) => stemOf(l.target)));- docs.push({ id: f.rel, title: nm.title, tags: nm.tags.join(" "), body, folder: nm.folder, type: nm.type ?? "" });+ }+ for (const [slug, paths] of s.duplicateStems) {+ console.warn(`[vault] duplicate stem "${slug}" (${paths.join(", ")}) — wikilinks resolve to the first.`);}- const outEdges = new Map<string, Set<string>>();- const inEdges = new Map<string, Set<string>>();- for (const [src, stems] of linksBySource) {+ s.outEdges = new Map();+ s.inEdges = new Map();+ for (const [src, stems] of s.linksBySource) {for (const stem of stems) {- const tgt = stemToPath.get(stem);+ const tgt = s.stemToPath.get(stem);if (!tgt || tgt === src) continue;- (outEdges.get(src) ?? outEdges.set(src, new Set()).get(src)!).add(tgt);- (inEdges.get(tgt) ?? inEdges.set(tgt, new Set()).get(tgt)!).add(src);+ (s.outEdges.get(src) ?? s.outEdges.set(src, new Set()).get(src)!).add(tgt);+ (s.inEdges.get(tgt) ?? s.inEdges.set(tgt, new Set()).get(tgt)!).add(src);}}+}- const search = new MiniSearch({- fields: ["title", "tags", "body", "folder", "type"],- storeFields: ["title", "folder", "type"],- searchOptions: { boost: { title: 3, tags: 2 }, prefix: true, fuzzy: 0.2, combineWith: "AND" },- });- search.addAll(docs);+function buildState(): IndexState {+ const dir = activeVaultDir();+ const s: IndexState = {+ dir,+ notes: new Map(),+ linksBySource: new Map(),+ stemToPath: new Map(),+ duplicateStems: new Map(),+ outEdges: new Map(),+ inEdges: new Map(),+ search: newIndex(),+ builtAt: Date.now(),+ };- return { dir, notes, stemToPath, outEdges, inEdges, search, builtAt: Date.now() };+ const docs: IndexDoc[] = [];+ for (const f of scanVault(dir)) {+ const parsed = readAndParse(dir, f.rel, f.mtimeMs);+ if (!parsed) continue;+ s.notes.set(f.rel, parsed.meta);+ s.linksBySource.set(f.rel, parsed.stems);+ docs.push(toDoc(parsed.meta, parsed.body));+ }+ s.search.addAll(docs);+ recomputeLinks(s);+ return s;+}++/** Add or update one file in the live index. Returns false when the file is gone. */+function upsert(s: IndexState, rel: string): boolean {+ let mtimeMs = 0;+ try {+ mtimeMs = fs.statSync(path.join(s.dir, rel)).mtimeMs;+ } catch {+ return false;+ }+ const parsed = readAndParse(s.dir, rel, mtimeMs);+ if (!parsed) return false;+ if (s.notes.has(rel)) {+ try {+ s.search.discard(rel);+ } catch {+ /* not in the index — fall through to add */+ }+ }+ s.notes.set(rel, parsed.meta);+ s.linksBySource.set(rel, parsed.stems);+ s.search.add(toDoc(parsed.meta, parsed.body));+ return true;+}++function remove(s: IndexState, rel: string): void {+ if (!s.notes.has(rel)) return;+ try {+ s.search.discard(rel);+ } catch {+ /* already gone */+ }+ s.notes.delete(rel);+ s.linksBySource.delete(rel);}function startWatcher(dir: string) {@@ -88,18 +202,41 @@ function startWatcher(dir: string) {return [...VAULT_IGNORE].some((ig) => base === ig || p.includes(`${path.sep}${ig}${path.sep}`));},});++ const pending = new Set<string>();let t: ReturnType<typeof setTimeout> | null = null;- const trigger = () => {- if (t) clearTimeout(t);- t = setTimeout(() => {- try {+ let fullRebuild = false;++ const flush = () => {+ try {+ if (fullRebuild || !state || state.dir !== dir) {state = buildState();- } catch (e) {- console.error("[vault] rebuild failed", e);+ } else {+ for (const rel of pending) if (!upsert(state, rel)) remove(state, rel);+ recomputeLinks(state);}- }, 250);+ } catch (e) {+ console.error("[vault] index update failed", e);+ }+ pending.clear();+ fullRebuild = false;+ };++ const schedule = () => {+ if (t) clearTimeout(t);+ t = setTimeout(flush, 250);+ };+ const onFile = (abs: string) => {+ if (!abs.toLowerCase().endsWith(".md")) return;+ pending.add(path.relative(dir, abs).split(path.sep).join("/"));+ schedule();+ };+ const onDir = () => {+ fullRebuild = true;+ schedule();};- watcher.on("add", trigger).on("change", trigger).on("unlink", trigger).on("addDir", trigger).on("unlinkDir", trigger);++ watcher.on("add", onFile).on("change", onFile).on("unlink", onFile).on("addDir", onDir).on("unlinkDir", onDir);} catch (e) {console.error("[vault] watcher failed to start", e);}@@ -112,16 +249,34 @@ function ensure(): IndexState {return state;}-/** Force a synchronous rebuild (after a write or a workspace switch). */+/** Force a synchronous full rebuild (workspace switch, or after a git pull changed many files). */export function rebuildIndex(): void {state = buildState();startWatcher(state.dir);}+/** Update just these paths (created, edited, moved, or deleted). No full re-scan. */+export function refreshPaths(relPaths: string[]): void {+ const s = ensure();+ for (const rel of relPaths) {+ if (!rel.toLowerCase().endsWith(".md")) continue;+ if (!upsert(s, rel)) remove(s, rel);+ }+ recomputeLinks(s);+}+export function listNotes(): NoteMeta[] {return [...ensure().notes.values()];}+/** Notes ordered by last modification, newest first. Powers "what changed lately". */+export function listRecent(sinceMs?: number, limit = 20): NoteMeta[] {+ return [...ensure().notes.values()]+ .filter((n) => (sinceMs ? n.mtimeMs >= sinceMs : true))+ .sort((a, b) => b.mtimeMs - a.mtimeMs)+ .slice(0, limit);+}+export function getNote(relPath: string): Note | null {const s = ensure();const abs = path.join(s.dir, relPath);@@ -140,19 +295,93 @@ export interface SearchHit {title: string;folder: string;type?: string;+ status?: string;+ /** Trust class, independent of relevance. See lib/vault/authority.ts. */+ authority: Authority;score: number;+ snippet?: string;+}++export interface SearchOpts {+ limit?: number;+ /** Restrict to one top-level folder. */+ folder?: string;+ /** Include notes in archive folders (still ranked far below live ones). Default false. */+ includeArchive?: boolean;+ /** Attach ~200 chars of matching context per hit. Default true. */+ snippets?: boolean;}-export function searchNotes(q: string, limit = 40): SearchHit[] {+/** First body line containing a query term (or the first prose line), trimmed for display. */+function snippetFor(dir: string, rel: string, terms: string[]): string | undefined {+ try {+ const raw = fs.readFileSync(path.join(dir, rel), "utf8");+ const { body } = parseNote(rel, raw);+ const lines = body+ .split("\n")+ .map((l) => l.trim())+ .filter((l) => l.length > 0 && !l.startsWith("#") && !l.startsWith("---"));+ const hit = lines.find((l) => {+ const low = l.toLowerCase();+ return terms.some((t) => low.includes(t));+ });+ const chosen = hit ?? lines[0];+ if (!chosen) return undefined;+ return chosen.length > 220 ? `${chosen.slice(0, 217)}…` : chosen;+ } catch {+ return undefined;+ }+}++/**+ * Keyword search, then reweighted by authority.+ *+ * The ranking is deliberately two-stage: MiniSearch says how well a note *matches*,+ * authorityOf() says how far it can be *trusted*. Without the second stage a superseded+ * price list outranks the locked one, because it repeats the query words just as often.+ */+export function searchNotes(q: string, opts: SearchOpts = {}): SearchHit[] {const s = ensure();if (!q.trim()) return [];- return s.search.search(q).slice(0, limit).map((r) => ({- path: r.id as string,- title: (r as unknown as { title: string }).title,- folder: (r as unknown as { folder: string }).folder,- type: (r as unknown as { type?: string }).type || undefined,- score: r.score,- }));+ const { limit = 20, folder, includeArchive = false, snippets = true } = opts;++ const raw = s.search.search(q, {+ boostDocument: (_id, _term, stored) => weightOf((stored?.authority as Authority) ?? "current"),+ filter: (r) => {+ const stored = r as unknown as { folder: string; authority: Authority };+ if (!includeArchive && stored.authority === "archived") return false;+ if (folder && stored.folder !== folder) return false;+ return true;+ },+ });++ const terms = q+ .toLowerCase()+ .split(/\s+/)+ .map((t) => t.replace(/[^a-z0-9-]/g, ""))+ .filter((t) => t.length >= 3);++ return raw.slice(0, limit).map((r) => {+ const hit = r as unknown as {+ id: string;+ title: string;+ folder: string;+ type?: string;+ status?: string;+ authority: Authority;+ score: number;+ };+ return {+ path: hit.id,+ title: hit.title,+ folder: hit.folder,+ type: hit.type || undefined,+ status: hit.status || undefined,+ authority: hit.authority,+ score: hit.score,+ snippet: snippets ? snippetFor(s.dir, hit.id, terms) : undefined,+ };+ });}export function getBacklinks(relPath: string): NoteMeta[] {@@ -165,6 +394,43 @@ export function getOutlinks(relPath: string): NoteMeta[] {return [...(s.outEdges.get(relPath) ?? [])].map((p) => s.notes.get(p)).filter(Boolean) as NoteMeta[];}+/**+ * What this vault actually looks like + how search will treat it. Returned by brain_schema so+ * an agent meeting an unfamiliar vault discovers its conventions instead of assuming ours.+ */+export function vaultConventions() {+ const s = ensure();+ const folders = new Set<string>();+ const statuses = new Map<string, number>();+ const types = new Set<string>();+ let archived = 0;++ for (const n of s.notes.values()) {+ folders.add(n.folder);+ if (n.type) types.add(n.type);+ if (n.status) statuses.set(n.status, (statuses.get(n.status) ?? 0) + 1);+ if (isArchivedPath(n.path)) archived++;+ }++ return {+ noteCount: s.notes.size,+ archivedCount: archived,+ folders: [...folders].sort(),+ types: [...types].sort(),+ statusesInUse: [...statuses.entries()]+ .sort((a, b) => b[1] - a[1])+ .map(([status, count]) => ({ status, count })),+ ranking: authorityRules(),+ integrity:+ s.duplicateStems.size === 0+ ? { duplicateStems: [] }+ : {+ duplicateStems: [...s.duplicateStems.entries()].map(([stem, paths]) => ({ stem, paths })),+ warning: "Wikilinks to a duplicated stem resolve to the first path only. Rename one.",+ },+ };+}+export function getGraph(folder?: string): Graph {const s = ensure();const inScope = (p: string) => !folder || s.notes.get(p)?.folder === folder;diff --git a/lib/vault/types.ts b/lib/vault/types.tsindex 926e983..1a013bc 100644--- a/lib/vault/types.ts+++ b/lib/vault/types.ts@@ -8,6 +8,8 @@ export interface NoteMeta {folder: string;type?: string;tags: string[];+ /** Alternate names/synonyms for this note, from `aliases:`. Indexed like the title. */+ aliases: string[];status?: string;created?: string;updated?: string;diff --git a/lib/vault/write.ts b/lib/vault/write.tsindex 669b2d0..b22930d 100644--- a/lib/vault/write.ts+++ b/lib/vault/write.ts@@ -2,7 +2,7 @@ import fsp from "node:fs/promises";import path from "node:path";import matter from "gray-matter";import { activeVaultDir } from "@/lib/repos";-import { rebuildIndex } from "./store";+import { refreshPaths } from "./store";import { requestSync } from "@/lib/git";/** Resolve a vault-relative path to an absolute path in the active vault, refusing escapes. */@@ -18,9 +18,10 @@ export function normalizeNotePath(relPath: string): string {return /\.md$/i.test(p) ? p : `${p}.md`;}-function after(p: string) {- rebuildIndex();- requestSync(p);+/** Re-index only what changed, then queue a git commit. `touched` are vault-relative paths. */+function after(message: string, touched: string[]) {+ refreshPaths(touched);+ requestSync(message);}/** Write a note from a raw markdown string (frontmatter included). Used by the editor. */@@ -29,7 +30,7 @@ export async function writeNoteRaw(relPath: string, content: string): Promise<stconst abs = safeAbs(p);await fsp.mkdir(path.dirname(abs), { recursive: true });await fsp.writeFile(abs, content, "utf8");- after(`edit ${p}`);+ after(`edit ${p}`, [p]);return p;}@@ -52,7 +53,7 @@ export async function appendNote(relPath: string, text: string): Promise<string>await fsp.mkdir(path.dirname(abs), { recursive: true });const sep = existing && !existing.endsWith("\n") ? "\n" : "";await fsp.writeFile(abs, `${existing}${sep}${text}\n`, "utf8");- after(`append ${p}`);+ after(`append ${p}`, [p]);return p;}@@ -61,13 +62,13 @@ export async function moveNote(from: string, to: string): Promise<string> {const b = safeAbs(normalizeNotePath(to));await fsp.mkdir(path.dirname(b), { recursive: true });await fsp.rename(a, b);- after(`move ${from} -> ${to}`);+ after(`move ${from} -> ${to}`, [normalizeNotePath(from), normalizeNotePath(to)]);return normalizeNotePath(to);}export async function deleteNote(relPath: string): Promise<void> {await fsp.rm(safeAbs(normalizeNotePath(relPath)));- after(`delete ${relPath}`);+ after(`delete ${relPath}`, [normalizeNotePath(relPath)]);}export async function createFolder(relPath: string): Promise<string> {diff --git a/sample-vault/SCHEMA.md b/sample-vault/SCHEMA.mdindex 1b30c84..44401b7 100644--- a/sample-vault/SCHEMA.md+++ b/sample-vault/SCHEMA.md@@ -3,9 +3,13 @@The contract every agent reads before writing (served via the `brain_schema` MCP tool).The app is taxonomy-agnostic — it renders whatever folders exist; this documents the defaults.+> `brain_schema` returns this file **plus a live report of your actual vault**: its folders, the+> `status:` values in use, the exact ranking weights, and any integrity warnings. An agent never+> has to guess your conventions — it can read them.+## Folders`clients/` · `decisions/` · `projects/` · `people/` · `meetings/` · `docs/` · `research/` ·-`inbox/` (unsorted captures). Slugs are kebab-case. New folders are allowed.+`inbox/` (unsorted captures) · `archive/` (see below). Slugs are kebab-case. New folders are allowed.## Frontmatter (YAML, optional)```yaml@@ -13,17 +17,55 @@ The app is taxonomy-agnostic — it renders whatever folders exist; this documentitle: Human title # graph node label (falls back to filename)type: decision # decision | client | project | person | meeting | doc | researchtags: [alpha, beta]-status: active # free text+status: active # see "Authority" — this drives search ranking+aliases: [win-back, churned] # synonyms; searched like the titlecreated: 2026-01-15related:- "[[projects/engram]]"---```+## Authority — how search decides what to trust++Search ranks by **keyword relevance**, which knows nothing about truth. A superseded price list+uses the word "price" as often as the live one, so it will happily outrank it. Engram fixes this by+multiplying each hit's score by an **authority** weight derived from the note itself — never from+its text. Every search hit and every `brain_read` returns its `authority`.++| authority | Triggered by | Weight | Meaning |+|---|---|---|---|+| `authoritative` | `status:` or a tag containing `locked`, `canonical`, `source-of-truth` | ×3.5 | Source of truth. Quote this over anything that disagrees. |+| `current` | anything else (**including no status at all**) | ×1 | Live. Fine for context. |+| `provisional` | `draft`, `proposed`, `exploring`, `wip`, `tentative`, `idea` | ×0.55 | Not decided. Never quote as settled. |+| `superseded` | `superseded`, `deprecated`, `obsolete`, `replaced`, `retired`, `dead` | ×0.15 | Explicitly replaced. History only. |+| `archived` | note lives under `archive/` (or `archives/`, `_archive/`, `trash/`) | ×0.08 | History only. **Excluded from agent search by default.** |++Words are matched inside `status:` *and* `tags:`, as whole tokens — so `status: draft-for-approval`+counts as `draft`. A vault that uses none of these conventions ranks purely by relevance, exactly as+it would without them: **the feature is opt-in and degrades to nothing.**++Override the archive folder names with the `ARCHIVE_FOLDERS` env var.++### The rule this exists to enforce+**Compose for context; route for facts.** Reading widely across notes to understand a client or a+history is what the graph is for. But a *single-valued fact* — a price, a guarantee, a legal entity,+a contract term — has exactly one owning note. Don't merge sources, don't take the top hit. Open the+`authoritative` note. **If two notes disagree on such a fact, that is a defect in the vault** — report+it rather than averaging them.++### Retiring a note+When something stops being true, **`brain_move` it into `archive/`** rather than deleting it, and add+a pointer in the replacement saying what superseded what. Archiving removes it from search; deleting+destroys the reasoning trail. An archived note with no superseding pointer is worse than a deleted+one — it still looks like an answer.++Mark a note `status: superseded` when it must stay in place (a decision log, where chronology matters).+## Links- `[[note]]` / `[[../path/note|Alias]]` — resolve by filename stem.- `related:` frontmatter arrays are edges too.- Graph: nodes = notes (colored by folder, sized by degree), edges = the links above.+- Two files sharing a basename is a bug: links resolve to the first. `brain_schema` reports these.## Callouts`> [!note]`, `> [!tip]`, `> [!warning]`, `> [!abstract]`, `> [!danger]`, `> [!question]`.