Turbocharge Claude Code, Cursor, Codex, Gemini & every coding agent: faster, cheaper, with contextual understanding specific to your codebase.
TypeScript ai-agentsanthropicclaude-codeclicode-graphcodexcontext-engineeringcursordeveloper-toolsgeminiknowledge-graphllmmcpmcp-serveropen-sourceopenaitree-sitter
Add auto-watch: folders keep syncing into an evolving graph
anirudhkumar-nanonetsb417c4226c22
diff --git a/README.md b/README.mdindex 3eb70ef0..2232b6d1 100644--- a/README.md+++ b/README.md@@ -102,6 +102,12 @@ context-graph ingest ./docs/*.md ./handbook.pdfcontext-graph ingest-dir ./docs # a whole folder, recursivelyecho "some notes" | context-graph ingest-text --title "Notes"+# Keep the graph evolving as a folder changes (see "Auto-watch")+context-graph watch ./docs # register + watch; re-run with no args to resume+context-graph ingest-dir ./docs --watch # ingest now, register for watching+context-graph watch-status # list registered folders+context-graph unwatch ./docs+# Ingest a code repository — as summaries, not raw code (see "Code repositories")context-graph repo .@@ -166,6 +172,9 @@ Tools exposed:| `context_ingest_file` | Ingest files from disk, including **PDFs** (parsed automatically) || `context_ingest_dir` | Ingest a whole **directory** of docs (PDF/MD/TXT by default; pass `extensions` to widen), recursively || `context_ingest_repo` | Ingest a **code repository** as per-file prose summaries (never raw code); incremental across runs |+| `context_watch_dir` | Connect a folder for **auto-watching**: ingest it now and re-ingest files as they change (see "Auto-watch") |+| `context_unwatch_dir` | Disconnect a watched folder (its knowledge stays in the graph) |+| `context_watch_status` | List connected folders and the ingest queue || `context_export` | Write the graph to an interactive **HTML** visualization || `context_sync` | Team sharing (git mode): import + re-merge the shared `graph.jsonl`, then write it back || `context_stats` | Report how much the graph currently holds |@@ -190,6 +199,26 @@ The same replace-on-change behavior now applies to every file ingest: re-ingestiFor a ~30-file repo expect ~30 small summary calls on first run; the graph stays module-scale (a handful of documents) rather than chunk-soup-scale. Agents should still read raw code with their own tools — the graph holds what code can't say: intent, decisions, and gotchas.+## Auto-watch — an evolving graph++Connect a folder once and the graph tracks it: files you add or edit are re-ingested automatically, so the graph evolves with your notes instead of freezing at ingest time.++```bash+context-graph watch ~/notes # catch up on the folder, then watch it live+# … later, in a fresh shell — no args needed, registered folders resume:+context-graph watch+```++How it behaves:++- **Debounced and cheap.** Saves are coalesced (~1.5 s quiet time per file) and unchanged files are skipped by content hash, so a daemon restart over a big unchanged folder costs zero LLM calls. Edited files *replace* their old document rather than piling up copies.+- **Append-only.** Deleting a file never removes knowledge already learned from it — the graph is a memory, not a mirror. The deletion is logged and nothing else happens.+- **Same rules as `ingest-dir`.** Default extensions `.pdf .md .markdown .txt` (override with `--ext`), dot-dirs / `node_modules` / files over 1 MB are ignored.+- **Registered folders** live in `watched-dirs.json` next to the db (machine-local — it never rides the git-synced graph). `ingest-dir --watch` registers while ingesting; `watch-status` / `unwatch` manage the list.+- **One watcher per db.** Run a single `watch` daemon per graph; MCP sessions and the web UI can read concurrently while it runs.++Elsewhere: the `context_watch_dir` MCP tool connects a folder for the life of the agent session (set `CONTEXT_GRAPH_AUTOWATCH=1` on the MCP server to auto-resume registered folders), and `CONTEXT_GRAPH_WATCH=1` does the same for `context-graph-web` — the UI picks up growth live.+## Claude Code hooks`context-graph install-hooks` wires the graph into Claude Code via `.claude/settings.json` (merges with what's already there):diff --git a/package-lock.json b/package-lock.jsonindex 6cb07793..9f4672e6 100644--- a/package-lock.json+++ b/package-lock.json@@ -12,6 +12,7 @@"@huggingface/transformers": "^3.8.1","@modelcontextprotocol/sdk": "^1.29.0","better-sqlite3": "^12.11.1",+ "chokidar": "^4.0.3","commander": "^15.0.0","dotenv": "^17.4.2","openai": "^6.45.0",@@ -1348,6 +1349,21 @@"url": "https://github.com/sponsors/ljharb"}},+ "node_modules/chokidar": {+ "version": "4.0.3",+ "resolved": "https://registry.npmjs.org/chokidar/-/chokidar-4.0.3.tgz",+ "integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==",+ "license": "MIT",+ "dependencies": {+ "readdirp": "^4.0.1"+ },+ "engines": {+ "node": ">= 14.16.0"+ },+ "funding": {+ "url": "https://paulmillr.com/funding/"+ }+ },"node_modules/chownr": {"version": "1.1.4","resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",@@ -2714,6 +2730,19 @@"node": ">= 6"}},+ "node_modules/readdirp": {+ "version": "4.1.2",+ "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-4.1.2.tgz",+ "integrity": "sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg==",+ "license": "MIT",+ "engines": {+ "node": ">= 14.18.0"+ },+ "funding": {+ "type": "individual",+ "url": "https://paulmillr.com/funding/"+ }+ },"node_modules/require-from-string": {"version": "2.0.2","resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",diff --git a/package.json b/package.jsonindex 88e48116..25d381bb 100644--- a/package.json+++ b/package.json@@ -50,6 +50,7 @@"@huggingface/transformers": "^3.8.1","@modelcontextprotocol/sdk": "^1.29.0","better-sqlite3": "^12.11.1",+ "chokidar": "^4.0.3","commander": "^15.0.0","dotenv": "^17.4.2","openai": "^6.45.0",diff --git a/src/cli.ts b/src/cli.tsindex 6ee9bd38..8889cc67 100644--- a/src/cli.ts+++ b/src/cli.ts@@ -17,6 +17,7 @@ import { execFile } from "node:child_process";import { promisify } from "node:util";import { basename, dirname, join, resolve } from "node:path";import { ContextGraphEngine } from "./engine.js";+import { GraphWatcher, type WatchEvent } from "./watch.js";import { resolveConfig } from "./ai/providers.js";import { toHtml, toMermaid } from "./graph/export.js";@@ -116,13 +117,13 @@ program.command("ingest-dir").description("Ingest every supported doc (.pdf .md .txt) in a directory, recursively").argument("<dir>", "directory to ingest")- .action(async (dir: string) => {+ .option("-w, --watch", "also register the directory for auto-watching")+ .action(async (dir: string, opts: { watch?: boolean }) => {const engine = engineFrom();try {const results = await engine.ingestDir(dir);if (results.length === 0) {console.log(`No supported files found under ${dir}`);- return;}for (const r of results) {console.log(@@ -131,6 +132,126 @@ program: `✓ ${r.title} — ${r.chunks} chunks, +${r.nodesCreated} nodes, +${r.edgesCreated} edges`,);}+ if (opts.watch) {+ const w = engine.addWatchedDir(dir);+ console.log(`✓ registered ${w.dir} for auto-watch — run \`context-graph watch\` to start the daemon`);+ }+ } finally {+ await engine.close();+ }+ });++/** One log line per watcher event, mirroring the ingest commands' ✓/•/✗ style. */+function logWatchEvent(event: WatchEvent): void {+ switch (event.type) {+ case "watching":+ console.log(`watching ${event.dir} (${event.files} files to catch up on)`);+ break;+ case "queued":+ if (event.reason !== "initial") console.log(` queued ${event.file} (${event.reason})`);+ break;+ case "ingested":+ console.log(+ event.result.skipped+ ? `• ${event.result.title} — unchanged, skipped`+ : `✓ ${event.result.title} — ${event.result.chunks} chunks, +${event.result.nodesCreated} nodes, +${event.result.edgesCreated} edges`,+ );+ break;+ case "deleted":+ console.log(`• ${event.file} — deleted (knowledge kept, graph is append-only)`);+ break;+ case "error":+ console.error(`✗ ${event.file}: ${event.error}`);+ break;+ }+}++program+ .command("watch")+ .description(+ "Watch folders and keep the graph evolving: new and edited docs are re-ingested automatically. " ++ "With no arguments, resumes previously registered folders. Ctrl-C to stop.",+ )+ .argument("[dirs...]", "directories to watch (registered for future runs unless --no-register)")+ .option("-e, --ext <exts...>", 'extensions to ingest (default: ".pdf" ".md" ".markdown" ".txt")')+ .option("--debounce <ms>", "quiet time after a save before re-ingesting", (v) => parseInt(v, 10), 1500)+ .option("--no-register", "don't persist the given directories to the watch registry")+ .option("--no-initial-scan", "skip the catch-up scan of existing files at startup")+ .action(+ async (+ dirs: string[],+ opts: { ext?: string[]; debounce: number; register: boolean; initialScan: boolean },+ ) => {+ const engine = engineFrom();+ if (opts.register) {+ for (const dir of dirs) engine.addWatchedDir(dir, opts.ext);+ }+ const targets = new Set(dirs.map((d) => resolve(d)));+ for (const w of engine.listWatchedDirs()) targets.add(w.dir);+ if (targets.size === 0) {+ console.error("No watched folders. Run: context-graph watch <dir>");+ await engine.close();+ process.exit(1);+ }++ const watcher = new GraphWatcher(engine, {+ extensions: opts.ext,+ debounceMs: opts.debounce,+ initialScan: opts.initialScan,+ onEvent: logWatchEvent,+ });+ // The daemon owns the engine for its lifetime — closed on signal, not+ // in a finally like the one-shot commands.+ let stopping = false;+ const stop = async () => {+ if (stopping) return;+ stopping = true;+ console.log("\nstopping…");+ await watcher.close();+ await engine.close();+ process.exit(0);+ };+ process.on("SIGINT", () => void stop());+ process.on("SIGTERM", () => void stop());++ for (const dir of targets) await watcher.add(dir);+ await watcher.idle();+ console.log("caught up — watching for changes (Ctrl-C to stop)");+ },+ );++program+ .command("watch-status")+ .description("List folders registered for auto-watching")+ .action(async () => {+ const engine = engineFrom();+ try {+ const dirs = engine.listWatchedDirs();+ if (dirs.length === 0) {+ console.log("No watched folders. Register one with: context-graph watch <dir>");+ return;+ }+ for (const w of dirs) {+ const exts = w.extensions?.join(" ") ?? "default extensions";+ console.log(`${w.dir} (${exts}, since ${w.addedAt.slice(0, 10)})`);+ }+ } finally {+ await engine.close();+ }+ });++program+ .command("unwatch")+ .description("Remove a folder from the auto-watch registry")+ .argument("<dir>", "directory to stop watching")+ .action(async (dir: string) => {+ const engine = engineFrom();+ try {+ console.log(+ engine.removeWatchedDir(dir)+ ? `✓ removed ${resolve(dir)} from the watch registry`+ : `• ${resolve(dir)} was not registered`,+ );} finally {await engine.close();}diff --git a/src/engine.ts b/src/engine.tsindex 2eb9d7d7..73b7ce87 100644--- a/src/engine.ts+++ b/src/engine.ts@@ -1,6 +1,7 @@-import { readFileSync, readdirSync, statSync, writeFileSync, existsSync, mkdirSync } from "node:fs";+import { readFileSync, writeFileSync, existsSync, mkdirSync } from "node:fs";import { basename, dirname, join, relative, resolve, sep } from "node:path";import { extractPdfText, isPdfPath } from "./ingest/pdf.js";+import { DOC_EXTENSIONS, walkDir } from "./ingest/fs.js";import { buildGraphExport, type GraphExport } from "./graph/export.js";import type { GraphStore } from "./graph/store.js";import { SqliteStore } from "./graph/sqlite-store.js";@@ -87,6 +88,19 @@ export interface RepoIngestResult {/** file path → { content hash, cached summary } */type RepoSummaryCache = Record<string, { hash: string; summary: string }>;+/** A folder registered for auto-watching (see `context-graph watch`). */+export interface WatchedDir {+ /** Absolute path of the watched directory. */+ dir: string;+ /** Extensions to ingest; undefined means {@link DOC_EXTENSIONS}. */+ extensions?: string[];+ /** ISO timestamp of when the folder was first registered. */+ addedAt: string;+}++/** absolute dir → registry entry (persisted as watched-dirs.json next to the db) */+type WatchRegistry = Record<string, { extensions?: string[]; addedAt: string }>;+/*** The Context Graph Engine.*@@ -101,6 +115,7 @@ export class ContextGraphEngine {private _extractor?: Extractor;private _summarizer?: Summarizer;private _memRepoCache: RepoSummaryCache = {};+ private _memWatchRegistry: WatchRegistry = {};constructor(config: EngineConfig = {}) {this.cfg = resolveConfig(config);@@ -267,7 +282,7 @@ export class ContextGraphEngine {onProgress?: (info: { index: number; total: number; file: string }) => void;} = {},): Promise<IngestResult[]> {- const exts = opts.extensions ?? [".pdf", ".md", ".markdown", ".txt"];+ const exts = opts.extensions ?? DOC_EXTENSIONS;const files = walkDir(dir).filter((f) => exts.some((e) => f.toLowerCase().endsWith(e)));const results: IngestResult[] = [];for (let i = 0; i < files.length; i++) {@@ -363,6 +378,66 @@ export class ContextGraphEngine {return result;}+ /** Path of the watch registry, or undefined for in-memory graphs. */+ private get watchRegistryPath(): string | undefined {+ if (this.cfg.dbPath === ":memory:") return undefined;+ return join(dirname(this.cfg.dbPath), "watched-dirs.json");+ }++ private loadWatchRegistry(): WatchRegistry {+ const path = this.watchRegistryPath;+ if (!path) return this._memWatchRegistry;+ if (!existsSync(path)) return {};+ try {+ return JSON.parse(readFileSync(path, "utf8")) as WatchRegistry;+ } catch {+ return {};+ }+ }++ private saveWatchRegistry(registry: WatchRegistry): void {+ const path = this.watchRegistryPath;+ if (!path) {+ this._memWatchRegistry = registry;+ return;+ }+ mkdirSync(dirname(path), { recursive: true });+ writeFileSync(path, JSON.stringify(registry, null, 2));+ }++ /** Folders registered for auto-watching, in registration order. */+ listWatchedDirs(): WatchedDir[] {+ return Object.entries(this.loadWatchRegistry()).map(([dir, entry]) => ({ dir, ...entry }));+ }++ /**+ * Register a folder for auto-watching (see `context-graph watch`). The+ * registry lives in `watched-dirs.json` next to the db — machine-local+ * paths deliberately stay out of the git-synced graph. Idempotent; the+ * extensions of an existing entry are updated in place.+ */+ addWatchedDir(dir: string, extensions?: string[]): WatchedDir {+ const abs = resolve(dir);+ const registry = this.loadWatchRegistry();+ const entry = {+ extensions: extensions ?? registry[abs]?.extensions,+ addedAt: registry[abs]?.addedAt ?? new Date().toISOString(),+ };+ registry[abs] = entry;+ this.saveWatchRegistry(registry);+ return { dir: abs, ...entry };+ }++ /** Unregister a watched folder. Returns false if it wasn't registered. */+ removeWatchedDir(dir: string): boolean {+ const abs = resolve(dir);+ const registry = this.loadWatchRegistry();+ if (!(abs in registry)) return false;+ delete registry[abs];+ this.saveWatchRegistry(registry);+ return true;+ }+/** Path of the file-summary cache, or undefined for in-memory graphs. */private get repoCachePath(): string | undefined {if (this.cfg.dbPath === ":memory:") return undefined;@@ -506,46 +581,6 @@ export class ContextGraphEngine {}}-/** Directories that are dependency/build output, never knowledge. */-const SKIP_DIRS = new Set([- "node_modules",- "dist",- "build",- "out",- "target",- "vendor",- "coverage",- "__pycache__",- "venv",-]);--/** Files above this size are generated/vendored in practice, not written docs or code. */-const MAX_FILE_BYTES = 1_000_000;--/**- * Recursively list all files under a directory. Skips dot-directories,- * dependency/build directories (node_modules, dist, …), and files over 1 MB.- */-function walkDir(dir: string): string[] {- const out: string[] = [];- for (const entry of readdirSync(dir, { withFileTypes: true })) {- if (entry.name.startsWith(".")) continue;- const full = join(dir, entry.name);- if (entry.isDirectory()) {- if (SKIP_DIRS.has(entry.name)) continue;- out.push(...walkDir(full));- } else if (entry.isFile()) {- try {- if (statSync(full).size > MAX_FILE_BYTES) continue;- } catch {- continue;- }- out.push(full);- }- }- return out;-}-/** Run `fn` over `items` with at most `limit` in flight at once, preserving order. */async function mapWithConcurrency<T, R>(items: T[],diff --git a/src/index.ts b/src/index.tsindex 7aa5daf4..0892216d 100644--- a/src/index.ts+++ b/src/index.ts@@ -24,8 +24,13 @@ export type {ContributeResult,RepoIngestOptions,RepoIngestResult,+ WatchedDir,} from "./engine.js";+export { DOC_EXTENSIONS } from "./ingest/fs.js";+export { GraphWatcher } from "./watch.js";+export type { WatchOptions, WatchEvent } from "./watch.js";+export type {EngineConfig,ResolvedConfig,diff --git a/src/ingest/fs.ts b/src/ingest/fs.tsnew file mode 100644index 00000000..c58749cf--- /dev/null+++ b/src/ingest/fs.ts@@ -0,0 +1,95 @@+/**+ * Filesystem walking/filtering rules shared by directory ingestion and the+ * auto-watch daemon, so both always agree on which files are knowledge.+ */+import { readdirSync, statSync } from "node:fs";+import { join, relative, sep } from "node:path";++/** Extensions treated as documents by `ingestDir` and the watcher. */+export const DOC_EXTENSIONS = [".pdf", ".md", ".markdown", ".txt"];++/** Directories that are dependency/build output, never knowledge. */+export const SKIP_DIRS = new Set([+ "node_modules",+ "dist",+ "build",+ "out",+ "target",+ "vendor",+ "coverage",+ "__pycache__",+ "venv",+]);++/** Files above this size are generated/vendored in practice, not written docs or code. */+export const MAX_FILE_BYTES = 1_000_000;++/**+ * Recursively list all files under a directory. Skips dot-directories,+ * dependency/build directories (node_modules, dist, …), and files over 1 MB.+ */+export function walkDir(dir: string): string[] {+ const out: string[] = [];+ for (const entry of readdirSync(dir, { withFileTypes: true })) {+ if (entry.name.startsWith(".")) continue;+ const full = join(dir, entry.name);+ if (entry.isDirectory()) {+ if (SKIP_DIRS.has(entry.name)) continue;+ out.push(...walkDir(full));+ } else if (entry.isFile()) {+ try {+ if (statSync(full).size > MAX_FILE_BYTES) continue;+ } catch {+ continue;+ }+ out.push(full);+ }+ }+ return out;+}++/** True if any path segment is a dot-entry or a SKIP_DIRS directory. */+function hasSkippedSegment(path: string): boolean {+ return path+ .split(sep)+ .some((seg) => seg.length > 0 && (seg.startsWith(".") || SKIP_DIRS.has(seg)));+}++/**+ * True if this path passes the same rules {@link walkDir} applies below+ * `root`: a matching extension, no dot-file/dot-dir or SKIP_DIRS segment+ * under the root, and at most 1 MB. The root itself is exempt from the+ * segment rules — watching `~/.notes` is legitimate, just as `walkDir` never+ * inspects the directory it was pointed at.+ */+export function isIngestablePath(+ path: string,+ root: string,+ extensions: string[] = DOC_EXTENSIONS,+): boolean {+ if (hasSkippedSegment(relative(root, path))) return false;+ const lower = path.toLowerCase();+ if (!extensions.some((e) => lower.endsWith(e))) return false;+ try {+ if (statSync(path).size > MAX_FILE_BYTES) return false;+ } catch {+ return false;+ }+ return true;+}++/**+ * Predicate for a file watcher's `ignored` option: true for dot/SKIP_DIRS+ * entries below `root` (so they are never descended into) and for oversized+ * files. Extension filtering is left to the event handler — watchers hand us+ * stats only sometimes, and directories have no extension to test.+ */+export function shouldIgnorePath(+ path: string,+ root: string,+ stats?: { isDirectory(): boolean; size: number },+): boolean {+ if (hasSkippedSegment(relative(root, path))) return true;+ if (stats && !stats.isDirectory() && stats.size > MAX_FILE_BYTES) return true;+ return false;+}diff --git a/src/mcp.ts b/src/mcp.tsindex d5782979..28a37dda 100644--- a/src/mcp.ts+++ b/src/mcp.ts@@ -12,10 +12,32 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"import { z } from "zod";import { writeFileSync, readFileSync, existsSync } from "node:fs";import { ContextGraphEngine } from "./engine.js";+import { GraphWatcher, type WatchEvent } from "./watch.js";import { toHtml } from "./graph/export.js";const engine = new ContextGraphEngine();+// One lazily-created watcher for the life of this MCP process. Folders added+// via context_watch_dir keep updating the graph in the background for as long+// as the client session (and thus this process) lives. context_watch_dir+// temporarily points `progressSink` at its own progress notifications while+// the catch-up scan runs.+let watcher: GraphWatcher | undefined;+let progressSink: ((event: WatchEvent) => void) | undefined;++function getWatcher(): GraphWatcher {+ return (watcher ??= new GraphWatcher(engine, {+ onEvent: (event) => {+ // stderr only — stdout is the JSON-RPC channel.+ if (event.type === "error") console.error(`watch: ✗ ${event.file}: ${event.error}`);+ else if (event.type === "ingested" && !event.result.skipped) {+ console.error(`watch: ✓ ${event.result.title} (+${event.result.nodesCreated} entities)`);+ }+ progressSink?.(event);+ },+ }));+}+const server = new McpServer({name: "context-graph-engine",version: "0.1.0",@@ -211,6 +233,126 @@ server.registerTool(},);+server.registerTool(+ "context_watch_dir",+ {+ title: "Watch a folder (evolving graph)",+ description:+ "Connect a folder to the graph and keep it evolving: existing documents are ingested now, " ++ "and any file added or edited later is re-ingested automatically for as long as this session " ++ "lives. The folder is registered, so `context-graph watch` (or CONTEXT_GRAPH_AUTOWATCH=1) " ++ "resumes it in future sessions. Append-only: deleting a file never removes learned knowledge.",+ inputSchema: {+ dir: z.string().describe("Absolute path to the folder to watch."),+ extensions: z+ .array(z.string())+ .optional()+ .describe('File extensions to include (e.g. [".md", ".rst"]). Default: .pdf .md .markdown .txt'),+ },+ },+ async ({ dir, extensions }, extra) => {+ const w = getWatcher();+ // Extensions are fixed per watcher process; a custom set is recorded in+ // the registry and takes effect when a daemon starts from it.+ engine.addWatchedDir(dir, extensions);++ const progressToken = extra?._meta?.progressToken;+ const lines: string[] = [];+ let ingested = 0;+ let queued = 0;+ progressSink = (event) => {+ if (event.type === "queued") queued++;+ if (event.type === "ingested") {+ ingested++;+ lines.push(+ event.result.skipped+ ? `• ${event.result.title}: unchanged, skipped`+ : `✓ ${event.result.title}: +${event.result.nodesCreated} entities, +${event.result.edgesCreated} relationships`,+ );+ if (progressToken) {+ void extra.sendNotification({+ method: "notifications/progress",+ params: {+ progressToken,+ progress: ingested,+ total: queued,+ message: `Ingesting ${event.file.split("/").pop()} (${ingested}/${queued})`,+ },+ });+ }+ }+ if (event.type === "error") lines.push(`✗ ${event.file}: ${event.error}`);+ };+ try {+ await w.add(dir);+ await w.idle();+ } finally {+ progressSink = undefined;+ }+ lines.push(+ `\nWatching ${dir} — new and edited files will be ingested automatically while this session lives.`,+ );+ return { content: [{ type: "text", text: lines.join("\n") }] };+ },+);++server.registerTool(+ "context_unwatch_dir",+ {+ title: "Stop watching a folder",+ description:+ "Stop auto-ingesting a folder and remove it from the watch registry. Knowledge already " ++ "learned from it stays in the graph.",+ inputSchema: {+ dir: z.string().describe("Absolute path of the watched folder to disconnect."),+ },+ },+ async ({ dir }) => {+ await watcher?.remove(dir);+ const removed = engine.removeWatchedDir(dir);+ return {+ content: [+ {+ type: "text",+ text: removed+ ? `Stopped watching ${dir}. Its knowledge remains in the graph.`+ : `${dir} was not being watched.`,+ },+ ],+ };+ },+);++server.registerTool(+ "context_watch_status",+ {+ title: "Watched folders",+ description:+ "List folders connected for auto-watching: registered folders, which are live in this session, and the ingest queue size.",+ inputSchema: {},+ },+ async () => {+ const registered = engine.listWatchedDirs();+ const live = new Set(watcher?.dirs() ?? []);+ if (registered.length === 0 && live.size === 0) {+ return {+ content: [+ { type: "text", text: "No watched folders. Connect one with context_watch_dir." },+ ],+ };+ }+ const lines = registered.map(+ (wd) => `${live.has(wd.dir) ? "● live" : "○ registered"} ${wd.dir}`,+ );+ for (const dir of live) {+ if (!registered.some((wd) => wd.dir === dir)) lines.push(`● live (unregistered) ${dir}`);+ }+ const pending = watcher?.pendingCount() ?? 0;+ if (pending > 0) lines.push(`\n${pending} file(s) queued for ingestion.`);+ return { content: [{ type: "text", text: lines.join("\n") }] };+ },+);+server.registerTool("context_export",{@@ -296,6 +438,22 @@ async function main(): Promise<void> {await server.connect(transport);// Log to stderr so we don't corrupt the stdio JSON-RPC channel.console.error("context-graph-engine MCP server running on stdio");++ // Opt-in: resume registered folders in the background. Off by default —+ // every client session spawns its own MCP process, and silent LLM spend on+ // file saves should be something the user chose.+ if (process.env.CONTEXT_GRAPH_AUTOWATCH === "1") {+ const dirs = engine.listWatchedDirs();+ if (dirs.length > 0) {+ const w = getWatcher();+ for (const wd of dirs) {+ w.add(wd.dir).catch((err) =>+ console.error(`watch: ✗ ${wd.dir}: ${err instanceof Error ? err.message : String(err)}`),+ );+ }+ console.error(`auto-watching ${dirs.length} registered folder(s)`);+ }+ }}main().catch((err) => {diff --git a/src/watch.ts b/src/watch.tsnew file mode 100644index 00000000..2db7ec0d--- /dev/null+++ b/src/watch.ts@@ -0,0 +1,221 @@+/**+ * Auto-watch: keep the graph evolving as connected folders change.+ *+ * A {@link GraphWatcher} watches document folders and re-ingests files as they+ * are added or edited, so the graph tracks the folder without manual re-runs+ * of `ingest-dir`. Deliberately append-only: deleting a file never removes+ * knowledge already learned from it (the grow-only observation model is+ * untouched); the deletion is surfaced as an event and nothing more.+ *+ * Rapid saves are absorbed by a per-file debounce, and files are ingested by a+ * single serial worker — each ingest already fans out to concurrent LLM+ * extraction internally, so one document at a time keeps spend predictable.+ */+import { watch, type FSWatcher } from "chokidar";+import type { Stats } from "node:fs";+import { resolve, sep } from "node:path";+import type { ContextGraphEngine, IngestResult } from "./engine.js";+import { DOC_EXTENSIONS, isIngestablePath, shouldIgnorePath, walkDir } from "./ingest/fs.js";++export interface WatchOptions {+ /** Extensions to ingest. Default: {@link DOC_EXTENSIONS}. */+ extensions?: string[];+ /** Quiet time after a file's last event before it is ingested. Default 1500ms. */+ debounceMs?: number;+ /** Catch up on the folder's current contents when it is added. Default true. */+ initialScan?: boolean;+ /** Called for every lifecycle event, for progress UIs and logging. */+ onEvent?: (event: WatchEvent) => void;+}++export type WatchEvent =+ | { type: "watching"; dir: string; files: number }+ | { type: "queued"; file: string; reason: "add" | "change" | "initial" }+ | { type: "ingested"; file: string; result: IngestResult }+ | { type: "deleted"; file: string }+ | { type: "error"; file: string; error: string };++/**+ * Watches folders and streams their document changes into an engine.+ *+ * One watcher process per db is the supported mode — concurrent graph writes+ * from multiple processes are safe (WAL) but their merges are not transactional.+ */+export class GraphWatcher {+ private watchers = new Map<string, FSWatcher>();+ private debounces = new Map<string, NodeJS.Timeout>();+ private pending = new Set<string>();+ private inFlight?: string;+ private working = false;+ private closed = false;+ private idleResolvers: Array<() => void> = [];++ constructor(+ private engine: ContextGraphEngine,+ private opts: WatchOptions = {},+ ) {}++ private get extensions(): string[] {+ return this.opts.extensions ?? DOC_EXTENSIONS;+ }++ private get debounceMs(): number {+ return this.opts.debounceMs ?? 1500;+ }++ private emit(event: WatchEvent): void {+ this.opts.onEvent?.(event);+ }++ /** Watch a folder. Resolves once watching is established (and the initial catch-up scan is queued). */+ async add(dir: string): Promise<void> {+ const root = resolve(dir);+ if (this.watchers.has(root)) return;++ const watcher = watch(root, {+ ignored: (path: string, stats?: Stats) => shouldIgnorePath(path, root, stats),+ ignoreInitial: true,+ // Don't ingest a file mid-write: wait for its size to hold still, so a+ // large PDF that takes a moment to save arrives whole.+ awaitWriteFinish: { stabilityThreshold: 500, pollInterval: 100 },+ });+ this.watchers.set(root, watcher);++ const onFile = (reason: "add" | "change") => (path: string) => {+ if (this.closed || !isIngestablePath(path, root, this.extensions)) return;+ this.debounce(path, reason);+ };+ watcher.on("add", onFile("add"));+ watcher.on("change", onFile("change"));+ watcher.on("unlink", (path: string) => {+ // Append-only by design: the file is gone but what it taught us stays.+ this.debounces.delete(path);+ this.pending.delete(path);+ this.emit({ type: "deleted", file: path });+ this.checkIdle();+ });+ watcher.on("error", (err) => {+ this.emit({ type: "error", file: root, error: err instanceof Error ? err.message : String(err) });+ });++ await new Promise<void>((res, rej) => {+ watcher.once("ready", res);+ watcher.once("error", rej);+ });++ let files = 0;+ if (this.opts.initialScan !== false) {+ for (const file of walkDir(root)) {+ if (!isIngestablePath(file, root, this.extensions)) continue;+ files++;+ this.pending.add(file);+ this.emit({ type: "queued", file, reason: "initial" });+ }+ this.kick();+ }+ this.emit({ type: "watching", dir: root, files });+ }++ /** Stop watching a folder and drop its queued (not in-flight) files. */+ async remove(dir: string): Promise<void> {+ const root = resolve(dir);+ const watcher = this.watchers.get(root);+ if (!watcher) return;+ this.watchers.delete(root);+ await watcher.close();+ const under = (path: string) => path === root || path.startsWith(root + sep);+ for (const [path, timer] of this.debounces) {+ if (under(path)) {+ clearTimeout(timer);+ this.debounces.delete(path);+ }+ }+ for (const path of this.pending) {+ if (under(path)) this.pending.delete(path);+ }+ this.checkIdle();+ }++ /** Folders currently being watched. */+ dirs(): string[] {+ return [...this.watchers.keys()];+ }++ /** Files queued or being ingested right now. */+ pendingCount(): number {+ return this.pending.size + this.debounces.size + (this.inFlight ? 1 : 0);+ }++ /** Resolves once every queued and debouncing file has been ingested. */+ idle(): Promise<void> {+ if (this.pendingCount() === 0) return Promise.resolve();+ return new Promise((res) => this.idleResolvers.push(res));+ }++ /** Stop watching everything, drop queued work, and wait out the in-flight ingest. */+ async close(): Promise<void> {+ this.closed = true;+ await Promise.all([...this.watchers.values()].map((w) => w.close()));+ this.watchers.clear();+ for (const timer of this.debounces.values()) clearTimeout(timer);+ this.debounces.clear();+ this.pending.clear();+ await this.idle();+ }++ private debounce(path: string, reason: "add" | "change"): void {+ const existing = this.debounces.get(path);+ if (existing) clearTimeout(existing);+ else this.emit({ type: "queued", file: path, reason });+ this.debounces.set(+ path,+ setTimeout(() => {+ this.debounces.delete(path);+ if (this.closed) return this.checkIdle();+ this.pending.add(path);+ this.kick();+ }, this.debounceMs),+ );+ }++ private checkIdle(): void {+ if (this.pendingCount() > 0) return;+ const resolvers = this.idleResolvers;+ this.idleResolvers = [];+ for (const res of resolvers) res();+ }++ private kick(): void {+ if (this.working) return;+ this.working = true;+ void this.work().finally(() => {+ this.working = false;+ this.checkIdle();+ });+ }++ private async work(): Promise<void> {+ for (let next = this.takeNext(); next; next = this.takeNext()) {+ this.inFlight = next;+ try {+ const result = await this.engine.ingestFile(next);+ this.emit({ type: "ingested", file: next, result });+ } catch (err) {+ // The file vanished between the event and the ingest — that's a delete.+ if ((err as NodeJS.ErrnoException).code === "ENOENT") {+ this.emit({ type: "deleted", file: next });+ } else {+ this.emit({ type: "error", file: next, error: err instanceof Error ? err.message : String(err) });+ }+ } finally {+ this.inFlight = undefined;+ }+ }+ }++ private takeNext(): string | undefined {+ const next = this.pending.values().next().value as string | undefined;+ if (next !== undefined) this.pending.delete(next);+ return next;+ }+}diff --git a/src/web.ts b/src/web.tsindex 6ea58d7b..91f7edeb 100644--- a/src/web.ts+++ b/src/web.ts@@ -22,6 +22,7 @@ import { networkInterfaces } from "node:os";import { resolve, dirname, join } from "node:path";import OpenAI from "openai";import { ContextGraphEngine } from "./engine.js";+import { GraphWatcher } from "./watch.js";import { resolveConfig } from "./ai/providers.js";import { extractPdfTextFromData, isPdfPath } from "./ingest/pdf.js";@@ -262,6 +263,15 @@ const server = createServer(async (req, res) => {return;}+ if (req.method === "GET" && url.pathname === "/api/watched") {+ json(res, 200, {+ registered: engine.listWatchedDirs(),+ live: webWatcher?.dirs() ?? [],+ pending: webWatcher?.pendingCount() ?? 0,+ });+ return;+ }+if (req.method === "POST" && url.pathname === "/api/ask") {const { question } = JSON.parse(await readBody(req)) as { question?: string };if (!question?.trim()) return json(res, 400, { error: "question is required" });@@ -286,10 +296,13 @@ const server = createServer(async (req, res) => {}if (req.method === "POST" && url.pathname === "/api/ingest-dir") {- // Accepts a directory or a single file on the server's disk.- const { dir, extensions } = JSON.parse(await readBody(req)) as {+ // Accepts a directory or a single file on the server's disk. With+ // watch:true the directory is also registered and watched in-process,+ // so future edits keep flowing into the graph while the server runs.+ const { dir, extensions, watch } = JSON.parse(await readBody(req)) as {dir?: string;extensions?: string[];+ watch?: boolean;};if (!dir?.trim()) return json(res, 400, { error: "dir is required" });const abs = resolve(dir);@@ -308,7 +321,17 @@ const server = createServer(async (req, res) => {onProgress: (info) => emit({ type: "progress", ...info }),}): [await engine.ingestFile(abs)];- emit({ type: "done", results, stats: await engine.stats() });+ let watching = false;+ if (watch && isDir) {+ engine.addWatchedDir(abs, extensions);+ // The just-finished ingest makes the watcher's catch-up scan a+ // pure hash-dedup pass — no LLM calls. Don't hold the response.+ void ensureWebWatcher()+ .add(abs)+ .catch((e) => console.error(`watch: ✗ ${abs}: ${e instanceof Error ? e.message : String(e)}`));+ watching = true;+ }+ emit({ type: "done", results, watching, stats: await engine.stats() });} catch (e) {emit({ type: "error", error: e instanceof Error ? e.message : String(e) });}@@ -320,9 +343,13 @@ const server = createServer(async (req, res) => {// One uploaded file at a time (the browser folder picker can't hand us a// server path, so it ships file contents instead). Text files arrive as// `text`; PDFs and other binaries arrive base64-encoded in `dataBase64`.+ // `relPath` (folder/sub/file.md) keys the document when present, so two+ // same-named files in different folders don't replace each other and a+ // synced folder's re-uploads update the right document.// 60 MB cap leaves room for a large PDF plus base64's ~33% overhead.- const { name, text, dataBase64 } = JSON.parse(await readBody(req, 60_000_000)) as {+ const { name, relPath, text, dataBase64 } = JSON.parse(await readBody(req, 60_000_000)) as {name?: string;+ relPath?: string;text?: string;dataBase64?: string;};@@ -337,7 +364,10 @@ const server = createServer(async (req, res) => {}if (!content.trim()) return json(res, 200, { skipped: true, name, empty: true });- const result = await engine.ingest(content, { title: name, source: name });+ const result = await engine.ingest(content, {+ title: name,+ source: relPath?.trim() || name,+ });json(res, 200, { result, stats: await engine.stats() });return;}@@ -359,6 +389,40 @@ const server = createServer(async (req, res) => {}});+// In-process auto-watch. Created lazily: by CONTEXT_GRAPH_WATCH=1 at startup+// (resumes registered folders), or the first time the UI ingests a server+// path with "keep watching" enabled.+let webWatcher: GraphWatcher | undefined;++function ensureWebWatcher(): GraphWatcher {+ return (webWatcher ??= new GraphWatcher(engine, {+ onEvent: (event) => {+ if (event.type === "ingested" && !event.result.skipped) {+ console.log(`watch: ✓ ${event.result.title} (+${event.result.nodesCreated} entities)`);+ } else if (event.type === "deleted") {+ console.log(`watch: • ${event.file} deleted (knowledge kept)`);+ } else if (event.type === "error") {+ console.error(`watch: ✗ ${event.file}: ${event.error}`);+ }+ },+ }));+}++if (process.env.CONTEXT_GRAPH_WATCH === "1") {+ const dirs = engine.listWatchedDirs();+ if (dirs.length > 0) {+ for (const wd of dirs) {+ ensureWebWatcher()+ .add(wd.dir)+ .catch((err) =>+ console.error(`watch: ✗ ${wd.dir}: ${err instanceof Error ? err.message : String(err)}`),+ );+ }+ } else {+ console.log("CONTEXT_GRAPH_WATCH=1 but no folders are registered — run: context-graph watch <dir>");+ }+}+server.listen(PORT, HOST, () => {const p = providerStatus();console.log(`Context Graph web UI → http://localhost:${PORT}`);diff --git a/src/web/ui.html b/src/web/ui.htmlindex 24c5e5d7..6d828ef4 100644--- a/src/web/ui.html+++ b/src/web/ui.html@@ -345,6 +345,24 @@details.server-path .grow-hint code { font-family: var(--mono); font-size: 11.5px; }details.server-path[open] { padding-bottom: var(--s1); }details.server-path > *:not(summary) { margin-top: var(--s2); }+ .watch-opt {+ display: flex; align-items: center; gap: 7px; font-size: 12.5px; color: var(--ink-soft);+ cursor: pointer; user-select: none;+ }+ .watch-opt input { accent-color: var(--verdant); }++ .sync-pill {+ display: flex; align-items: center; gap: 8px; margin-top: var(--s3);+ padding: 8px 12px; border: 1px solid var(--verdant); border-radius: var(--r-sm);+ background: var(--verdant-soft); font-size: 12.5px; color: var(--ink-soft);+ }+ .sync-pill .dot {+ width: 8px; height: 8px; border-radius: 50%; background: var(--verdant); flex: none;+ animation: sync-pulse 2s ease-in-out infinite;+ }+ .sync-pill #folder-sync-text { flex: 1; min-width: 0; }+ .sync-pill .btn { padding: 3px 10px; font-size: 12px; }+ @keyframes sync-pulse { 0%, 100% { opacity: 1; } 50% { opacity: .35; } }.doc-row {display: flex; gap: var(--s2); align-items: baseline; padding: 7px 4px;@@ -488,12 +506,18 @@<input type="file" id="files-input" multiple accept=".pdf,.md,.markdown,.txt" hidden /><input type="file" id="folder-input" webkitdirectory multiple hidden /><div class="picker-row">- <button class="btn quiet" id="choose-folder-btn" type="button">Choose a folder…</button>+ <button class="btn quiet" id="choose-folder-btn" type="button" title="The folder keeps syncing while this tab is open">Connect a folder…</button>+ </div>+ <div id="folder-sync" class="sync-pill hidden">+ <span class="dot"></span>+ <span id="folder-sync-text"></span>+ <button class="btn quiet" id="folder-sync-stop" type="button">Stop</button></div><details class="server-path"><summary>Ingest a path on the server</summary><p class="grow-hint">Reads a file or folder from the server's own disk — in Docker, that's the mounted <code>/docs</code>.</p><input class="dir" id="ingest-input" placeholder="/path/to/docs or /path/to/file.md" />+ <label class="watch-opt"><input type="checkbox" id="watch-check" /> Keep watching — future changes are ingested automatically</label><button class="btn quiet" id="ingest-btn" type="button">Ingest server path</button></details><div id="ingest-progress" class="hidden">@@ -1127,7 +1151,7 @@ $("ingest-btn").addEventListener("click", async () => {const r = await api("/api/ingest-dir", {method: "POST",headers: { "content-type": "application/json" },- body: JSON.stringify({ dir }),+ body: JSON.stringify({ dir, watch: $("watch-check").checked }),});if (!r.ok) {const err = await r.json().catch(() => ({}));@@ -1160,7 +1184,8 @@ $("ingest-btn").addEventListener("click", async () => {status.textContent =`✓ ${fresh.length} document${fresh.length === 1 ? "" : "s"} added` +(skipped ? ` (${skipped} already known, skipped)` : "") +- `, ${nodes} facts written or reinforced`;+ `, ${nodes} facts written or reinforced` ++ (ev.watching ? " — now watching this folder for changes" : "");} else if (ev.type === "error") {throw new Error(ev.error);}@@ -1182,16 +1207,19 @@ $("ingest-btn").addEventListener("click", async () => {const UPLOAD_ACCEPT = /\.(pdf|md|markdown|txt)$/i;// Base64-encode file bytes in chunks (fromCharCode.apply chokes on huge arrays).+// The relative path (from a folder pick or live sync) keys the document on the+// server, so same-named files in different subfolders stay distinct.async function fileToPayload(file) {+ const relPath = file._relPath || file.webkitRelativePath || undefined;if (/\.pdf$/i.test(file.name)) {const bytes = new Uint8Array(await file.arrayBuffer());let binary = "";for (let i = 0; i < bytes.length; i += 0x8000) {binary += String.fromCharCode.apply(null, bytes.subarray(i, i + 0x8000));}- return { name: file.name, dataBase64: btoa(binary) };+ return { name: file.name, relPath, dataBase64: btoa(binary) };}- return { name: file.name, text: await file.text() };+ return { name: file.name, relPath, text: await file.text() };}$("dropzone").addEventListener("click", () => $("files-input").click());@@ -1206,7 +1234,20 @@ $("dropzone").addEventListener("drop", (e) => {if (ingestBusy) return;uploadFiles(Array.from(e.dataTransfer.files || []), "files");});-$("choose-folder-btn").addEventListener("click", () => $("folder-input").click());+$("choose-folder-btn").addEventListener("click", async () => {+ // Chromium: keep a handle to the picked folder and live-sync it while the+ // tab is open. Elsewhere: fall back to the classic one-shot folder upload.+ if (!window.showDirectoryPicker) return $("folder-input").click();+ try {+ const handle = await showDirectoryPicker({ mode: "read" });+ await startFolderSync(handle);+ } catch (e) {+ if (e.name !== "AbortError") {+ $("ingest-status").className = "grow-status err";+ $("ingest-status").textContent = "Could not open the folder: " + e.message;+ }+ }+});$("files-input").addEventListener("change", (e) => uploadPicked(e, "files"));$("folder-input").addEventListener("change", (e) => uploadPicked(e, "folder"));@@ -1276,6 +1317,105 @@ async function uploadFiles(all, kind) {}}+/* ---------- live folder sync (File System Access API, Chromium) ----------+ The picked folder's handle stays with the page: every few seconds it is+ re-scanned and files whose size/mtime moved are re-uploaded, so the graph+ keeps evolving while the tab is open. Deletions are ignored on purpose —+ the graph is append-only. */+const SYNC_INTERVAL_MS = 5000;+const SYNC_SKIP = new Set(["node_modules", "dist", "build", "out", "target", "vendor", "coverage", "__pycache__", "venv"]);+const SYNC_MAX_BYTES = 1_000_000;+let folderSync = null; // { handle, name, state: Map<rel, sig>, timer, busy }++async function collectFolderFiles(handle, name) {+ const files = [];+ const state = new Map();+ async function walk(dir, prefix) {+ for await (const entry of dir.values()) {+ if (entry.name.startsWith(".") || SYNC_SKIP.has(entry.name)) continue;+ if (entry.kind === "directory") {+ await walk(entry, prefix + entry.name + "/");+ } else if (UPLOAD_ACCEPT.test(entry.name)) {+ const f = await entry.getFile();+ if (f.size > SYNC_MAX_BYTES) continue;+ const rel = prefix + entry.name;+ state.set(rel, f.size + ":" + f.lastModified);+ f._relPath = name + "/" + rel;+ f._rel = rel;+ files.push(f);+ }+ }+ }+ await walk(handle, "");+ return { files, state };+}++function renderSyncPill(msg) {+ $("folder-sync-text").textContent = msg;+ $("folder-sync").classList.remove("hidden");+}++async function startFolderSync(handle) {+ stopFolderSync();+ const session = { handle, name: handle.name, state: new Map(), timer: null, busy: false };+ folderSync = session;+ renderSyncPill(`Scanning “${handle.name}”…`);+ const { files, state } = await collectFolderFiles(handle, handle.name);+ session.state = state;+ await uploadFiles(files, "folder"); // the usual progress UI for the first sweep+ if (folderSync !== session) return; // stopped or replaced mid-sweep+ session.timer = setInterval(syncTick, SYNC_INTERVAL_MS);+ renderSyncPill(`Watching “${handle.name}” — ${state.size} files. Edits sync while this tab is open.`);+}++function stopFolderSync() {+ if (!folderSync) return;+ clearInterval(folderSync.timer);+ folderSync = null;+ $("folder-sync").classList.add("hidden");+}+$("folder-sync-stop").addEventListener("click", stopFolderSync);++async function syncTick() {+ const session = folderSync;+ if (!session || session.busy || ingestBusy) return;+ session.busy = true;+ try {+ const { files, state } = await collectFolderFiles(session.handle, session.name);+ const changed = files.filter((f) => session.state.get(f._rel) !== state.get(f._rel));+ session.state = state; // forgets deleted entries too — harmless, append-only+ let synced = 0;+ for (const f of changed) {+ if (folderSync !== session) return; // stopped mid-batch+ const r = await api("/api/ingest-upload", {+ method: "POST",+ headers: { "content-type": "application/json" },+ body: JSON.stringify(await fileToPayload(f)),+ });+ if (r.ok) {+ const data = await r.json();+ if (data.result && !data.result.skipped) synced++;+ }+ }+ if (synced && folderSync === session) {+ refreshStats().catch(() => {});+ refreshGraph({ diff: true }).catch(() => {});+ refreshDocs().catch(() => {});+ const at = new Date().toLocaleTimeString();+ renderSyncPill(`Watching “${session.name}” — synced ${synced} change${synced === 1 ? "" : "s"} at ${at}.`);+ }+ } catch (e) {+ // Permission revoked or the folder went away — stop cleanly, not noisily.+ if (folderSync === session) {+ clearInterval(session.timer);+ folderSync = null;+ renderSyncPill(`Sync stopped: ${e.message}`);+ }+ return;+ }+ session.busy = false;+}+/* ---------- documents ---------- */const DOCS_PREVIEW = 10;let docsExpanded = false;@@ -1329,6 +1469,25 @@ async function refreshDocs() {$("ask-form").requestSubmit();}})();++/* ---------- live updates ----------+ When the server runs with an auto-watcher (CONTEXT_GRAPH_WATCH=1), the graph+ grows on its own as watched folders change. A cheap stats poll notices and+ refreshes the heavier graph/docs views only when the counts actually moved. */+let liveCounts = null;+setInterval(async () => {+ try {+ const s = await refreshStats();+ const counts = `${s.documents}|${s.nodes}|${s.edges}|${s.chunks}`;+ if (liveCounts !== null && counts !== liveCounts) {+ refreshGraph({ diff: true }).catch(() => {});+ refreshDocs().catch(() => {});+ }+ liveCounts = counts;+ } catch {+ /* offline or token prompt pending — try again next tick */+ }+}, 10_000);</script></body></html>diff --git a/test/watch.test.ts b/test/watch.test.tsnew file mode 100644index 00000000..3db97491--- /dev/null+++ b/test/watch.test.ts@@ -0,0 +1,216 @@+/**+ * Auto-watch: the graph evolves as a connected folder changes. All tests run+ * on the offline fake providers; FS events come from real chokidar watchers on+ * temp directories, so generous polling deadlines absorb platform latency.+ */+import { test } from "node:test";+import assert from "node:assert/strict";+import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { ContextGraphEngine, GraphWatcher, type WatchEvent } from "../src/index.js";+import { fakeProviders } from "./helpers.ts";++function engine() {+ return new ContextGraphEngine({ dbPath: ":memory:", ...fakeProviders() });+}++function tempDir(): string {+ return mkdtempSync(join(tmpdir(), "cg-watch-"));+}++/** Poll until `cond` holds — FS events arrive with platform-dependent latency. */+async function waitFor(cond: () => boolean, what: string, ms = 10_000): Promise<void> {+ const start = Date.now();+ while (!cond()) {+ if (Date.now() - start > ms) throw new Error(`timed out waiting for ${what}`);+ await new Promise((r) => setTimeout(r, 50));+ }+}++function collector() {+ const events: WatchEvent[] = [];+ const of = (type: WatchEvent["type"]) => events.filter((e) => e.type === type);+ return { events, of, onEvent: (e: WatchEvent) => events.push(e) };+}++test("initial scan ingests existing files; a second watcher skips them all", async () => {+ const dir = tempDir();+ const e = engine();+ writeFileSync(join(dir, "auth.md"), "[[Auth Service]] ==uses==> [[OAuth]]");+ writeFileSync(join(dir, "deploy.md"), "[[Deploys]] ==run_on==> [[CI]]");++ const first = collector();+ const w1 = new GraphWatcher(e, { debounceMs: 50, onEvent: first.onEvent });+ await w1.add(dir);+ await w1.idle();+ await w1.close();++ assert.equal(first.of("ingested").length, 2);+ assert.ok(first.of("ingested").every((ev) => ev.type === "ingested" && !ev.result.skipped));+ assert.equal((await e.stats()).documents, 2);++ // A fresh watcher over the same unchanged folder is a cheap no-op.+ const second = collector();+ const w2 = new GraphWatcher(e, { debounceMs: 50, onEvent: second.onEvent });+ await w2.add(dir);+ await w2.idle();+ await w2.close();++ assert.equal(second.of("ingested").length, 2);+ assert.ok(second.of("ingested").every((ev) => ev.type === "ingested" && ev.result.skipped));+ assert.equal((await e.stats()).documents, 2);++ await e.close();+ rmSync(dir, { recursive: true, force: true });+});++test("a file created after watching starts is ingested automatically", async () => {+ const dir = tempDir();+ const e = engine();+ const c = collector();+ const w = new GraphWatcher(e, { debounceMs: 50, onEvent: c.onEvent });+ await w.add(dir);++ writeFileSync(join(dir, "new.md"), "[[Billing]] ==bills==> [[Customers]]");+ await waitFor(() => c.of("ingested").length === 1, "the new file to be ingested");++ assert.equal((await e.stats()).documents, 1);+ const [node] = await e.store.findNodesByName("billing");+ assert.ok(node, "entity from the new file reached the graph");++ await w.close();+ await e.close();+ rmSync(dir, { recursive: true, force: true });+});++test("rapid successive saves are debounced into one ingest", async () => {+ const dir = tempDir();+ const e = engine();+ const c = collector();+ const w = new GraphWatcher(e, { debounceMs: 400, onEvent: c.onEvent });+ await w.add(dir);++ const file = join(dir, "notes.md");+ writeFileSync(file, "[[Draft]] v1");+ writeFileSync(file, "[[Draft]] v2");+ writeFileSync(file, "[[Draft]] final version");+ await waitFor(() => c.of("ingested").length >= 1, "the burst of saves to be ingested");+ // Give any straggler event time to (wrongly) trigger a second ingest.+ await new Promise((r) => setTimeout(r, 700));+ await w.idle();++ assert.equal(c.of("ingested").length, 1, "one ingest for the whole burst");+ const docs = await e.store.documentsBySource(file);+ assert.equal(docs.length, 1);++ await w.close();+ await e.close();+ rmSync(dir, { recursive: true, force: true });+});++test("editing a file replaces its document instead of piling up copies", async () => {+ const dir = tempDir();+ const e = engine();+ const c = collector();+ const w = new GraphWatcher(e, { debounceMs: 50, onEvent: c.onEvent });+ const file = join(dir, "auth.md");+ writeFileSync(file, "[[Auth Service]] ==uses==> [[OAuth]]");+ await w.add(dir);+ await w.idle();+ const before = await e.store.documentsBySource(file);+ assert.equal(before.length, 1);++ writeFileSync(file, "[[Auth Service]] ==rotates==> [[Tokens]] every 15 minutes");+ await waitFor(() => c.of("ingested").length === 2, "the edited file to be re-ingested");++ const after = await e.store.documentsBySource(file);+ assert.equal(after.length, 1, "still one document for the source");+ assert.notEqual(after[0].hash, before[0].hash, "the document reflects the new content");+ assert.equal((await e.stats()).documents, 1);++ await w.close();+ await e.close();+ rmSync(dir, { recursive: true, force: true });+});++test("deleting a file keeps its knowledge (append-only)", async () => {+ const dir = tempDir();+ const e = engine();+ const c = collector();+ const w = new GraphWatcher(e, { debounceMs: 50, onEvent: c.onEvent });+ const file = join(dir, "legacy.md");+ writeFileSync(file, "[[Legacy System]] ==replaced_by==> [[New System]]");+ await w.add(dir);+ await w.idle();+ const statsBefore = await e.stats();++ rmSync(file);+ await waitFor(() => c.of("deleted").length === 1, "the deletion to be observed");+ await w.idle();++ assert.deepEqual(await e.stats(), statsBefore, "nothing is pruned on delete");+ assert.equal(c.of("ingested").length, 1, "the delete triggered no ingest");++ await w.close();+ await e.close();+ rmSync(dir, { recursive: true, force: true });+});++test("wrong extensions, oversized files, and skip-dirs are ignored", async () => {+ const dir = tempDir();+ const e = engine();+ const c = collector();+ writeFileSync(join(dir, "keep.md"), "[[Kept]]");+ writeFileSync(join(dir, "skip.log"), "[[Log Noise]]");+ writeFileSync(join(dir, "huge.md"), "x".repeat(1_000_001));+ mkdirSync(join(dir, "node_modules"));+ writeFileSync(join(dir, "node_modules", "readme.md"), "[[Vendored]]");++ const w = new GraphWatcher(e, { debounceMs: 50, onEvent: c.onEvent });+ await w.add(dir);+ await w.idle();++ assert.equal(c.of("ingested").length, 1);+ assert.equal((await e.stats()).documents, 1);+ assert.equal((await e.store.findNodesByName("log noise")).length, 0);+ assert.equal((await e.store.findNodesByName("vendored")).length, 0);++ // Live events under a skip-dir are ignored too.+ writeFileSync(join(dir, "node_modules", "later.md"), "[[Vendored Later]]");+ writeFileSync(join(dir, "later.txt"), "[[Arrived Later]]");+ await waitFor(() => c.of("ingested").length === 2, "only the valid live file to be ingested");+ await w.idle();+ assert.equal((await e.store.findNodesByName("vendored later")).length, 0);++ await w.close();+ await e.close();+ rmSync(dir, { recursive: true, force: true });+});++test("the watch registry survives engine restarts", async () => {+ const home = tempDir();+ const dbPath = join(home, "graph.db");+ const notes = join(home, "notes");+ mkdirSync(notes);++ const e1 = new ContextGraphEngine({ dbPath, ...fakeProviders() });+ const added = e1.addWatchedDir(notes, [".md"]);+ assert.equal(added.dir, notes);+ // Re-registering is idempotent and keeps the original timestamp.+ assert.equal(e1.addWatchedDir(notes).addedAt, added.addedAt);+ await e1.close();++ const e2 = new ContextGraphEngine({ dbPath, ...fakeProviders() });+ const listed = e2.listWatchedDirs();+ assert.equal(listed.length, 1);+ assert.equal(listed[0].dir, notes);+ assert.deepEqual(listed[0].extensions, [".md"]);++ assert.equal(e2.removeWatchedDir(notes), true);+ assert.equal(e2.removeWatchedDir(notes), false);+ assert.equal(e2.listWatchedDirs().length, 0);+ await e2.close();++ rmSync(home, { recursive: true, force: true });+});