Skip to content

Turbocharge Claude Code, Cursor, Codex, Gemini & every coding agent: faster, cheaper, with contextual understanding specific to your codebase.

Add PDF ingestion + MCP file tool + demo PDFs

anirudhkumar-nanonetsed4a34b33402
diff --git a/DEMO.md b/DEMO.md
index 22188b1d..5a8be812 100644
--- a/DEMO.md
+++ b/DEMO.md
@@ -23,7 +23,9 @@ Think "durable, structured memory for a team of agents" — the same integration
| **Storage** (`src/graph/sqlite-store.ts`) | Embedded SQLite (`better-sqlite3`), zero infra; swappable behind a `GraphStore` interface | ✅ Done |
| **Local-first providers** (`src/ai/local.ts`) | In-process embeddings (transformers.js) + local extraction (Ollama) — **runs with no API keys** | ✅ Done |
| **Cloud auto-upgrade** (`src/engine.ts`) | If `OPENROUTER_API_KEY` is set, uses OpenRouter for extraction automatically (any tool-calling model) for higher quality | ✅ Done |
-| **Three access modes** | Library, `context-graph` CLI, and `context-graph-mcp` MCP server for any agent | ✅ Done |
+| **PDF ingestion** (`src/ingest/pdf.ts`) | `.pdf` files parsed to text automatically (pure-JS `unpdf`, no native deps) — via CLI, library, or the `context_ingest_file` MCP tool | ✅ Done |
+| **Three access modes** | Library, `context-graph` CLI, and `context-graph-mcp` MCP server (5 tools) for any agent | ✅ Done |
+| **Demo PDFs** (`examples/demo-docs/`) | 3 detail-rich fictional docs + a generator script | ✅ Done |
| **One-line installer** (`install.sh`) | `curl … | sh` clones, builds, and puts both commands on PATH | ✅ Done |
| Update modes (silent vs. flagged) | Deferred per your call — needs discussion | ⏸ Deferred |
| Delete / decay of stale nodes | In the spec, not yet built | ⬜ Not started |
@@ -56,43 +58,72 @@ No accounts, nothing leaves the machine. Slightly coarser graph and a one-time m
---
-## 4. The demo (60 seconds)
+## 4. The MCP demo (the main event)
-The cleanest thing to show is the **library quickstart** — it's self-contained, uses an in-memory graph (leaves nothing behind), and tells the whole story in one run:
+This is the flow to show: **install → point an MCP client at it → ingest PDFs → ask questions → the facts from the PDFs come back.** Verified end-to-end (OpenRouter extraction + local embeddings).
+There are 3 demo PDFs already generated in `examples/demo-docs/` (fictional "Northwind" company — architecture, billing runbook, onboarding). Regenerate them any time with `node scripts/make-demo-pdfs.mjs`.
+
+### Step 1 — Install
```bash
-npm install
-npm run example
+# the one-liner your manager asked for (once the repo URL is filled in):
+curl -fsSL https://raw.githubusercontent.com/YOUR_ORG/context-graph-engine/main/install.sh | sh
+
+# …or, from this checkout, for today:
+npm install && npm run build && npm link
```
+`npm link` puts `context-graph` and `context-graph-mcp` on your PATH.
+
+### Step 2 — Point your MCP client at it
+Add this to your Claude Code / Cursor MCP config (`.mcp.json` or the client's settings). Set a DB path for the demo graph and your OpenRouter key:
+
+```json
+{
+ "mcpServers": {
+ "context-graph": {
+ "command": "context-graph-mcp",
+ "env": {
+ "CONTEXT_GRAPH_DB": "/absolute/path/to/northwind-demo.db",
+ "OPENROUTER_API_KEY": "sk-or-..."
+ }
+ }
+ }
+}
+```
+Restart the client so it picks up the server. The agent now has 5 tools: `context_read`, `context_contribute`, `context_ingest`, `context_ingest_file`, `context_stats`.
-It walks through, live:
-1. **Ingest** a small "Payments Platform" doc → builds a graph of services and how they relate.
-2. **Read** `"what happens when a charge fails?"` → prints a structured context block (entities + relationships + sources) ready to drop into an agent prompt.
-3. **Contribute** a new learning (`"the Dunning Worker sends a Slack alert to #billing…"`) as if an agent discovered it mid-task.
-4. **Read again** → the new knowledge is now part of the graph.
+### Step 3 — Ingest the PDFs (say this to the agent)
+> "Ingest these files into the context graph:
+> /abs/path/examples/demo-docs/northwind-architecture.pdf,
+> /abs/path/examples/demo-docs/northwind-billing-runbook.pdf,
+> /abs/path/examples/demo-docs/northwind-onboarding.pdf"
-**The money moment:** step 3 doesn't just append text — the learning is extracted, deduped against existing entities, and merged. Re-observed facts get their `confidence` and `observation` count bumped instead of creating duplicates. That's the "gets smarter over time" claim, demonstrated.
+The agent calls `context_ingest_file`. The PDFs are parsed, chunked, embedded, and extracted into the graph. You'll see something like "✓ 3 chunks, +23 entities, +35 relationships" per file. Then ask it to run `context_stats` — ~58 entities, ~70 relationships across 3 documents.
-### CLI version (if you'd rather show the shell)
-```bash
-npm run build
+### Step 4 — Ask questions (the payoff)
+Ask the agent things that only live inside the PDFs. It calls `context_read` and gets the answer back — as structured entities/relationships **and** the exact source passages:
-echo "Our billing worker retries failed charges 3x, then marks the subscription past_due." \
- | node dist/cli.js --db ./demo.db ingest-text --title "Billing"
+- *"How are failed charges retried, and when does an account get suspended?"*
+ → 3 retries with exponential backoff (1h / 6h / 24h) → `past_due` → suspended after 7 days.
+- *"How long do access tokens last and what backs the Auth Service?"*
+ → access tokens expire after 15 minutes; Auth Service is backed by Redis for token revocation.
+- *"Who do I escalate a Stripe outage to?"*
+ → Dana Whitfield (Payments team lead), post in `#billing-incidents`, page PagerDuty service `northwind-billing`.
+- *"What do I do in my first week as a new hire?"*
+ → security training, ship one small prod change, on-call shadow; buddy assigned day one.
-node dist/cli.js --db ./demo.db query "how are failed charges handled?"
+**The point to land:** these facts came from three separate PDFs, and the agent gets a *structured, cross-referenced* answer with the source text as evidence — not a blind vector-snippet dump.
-node dist/cli.js --db ./demo.db contribute \
- "After the final retry the billing worker posts a Slack alert to #billing." --agent oncall
+### Step 5 (optional) — Show it getting smarter
+Tell the agent: *"Contribute this learning: the Dunning Worker also posts to #billing-alerts after the final retry."* It calls `context_contribute`. Ask the billing question again — the new fact is now merged into the graph, and re-stated facts reinforce (bump confidence) rather than duplicate.
-node dist/cli.js --db ./demo.db query "how does the team find out about failed charges?"
-
-node dist/cli.js --db ./demo.db stats
+### Fallback (no MCP client handy) — same thing via CLI
+```bash
+context-graph --db ./northwind-demo.db ingest examples/demo-docs/*.pdf
+context-graph --db ./northwind-demo.db query "how are failed charges retried?"
+context-graph --db ./northwind-demo.db stats
```
-(Delete `demo.db*` afterward to reset.)
-
-### MCP version (if the audience cares about agent integration)
-Point Claude Code / Cursor at the server and the agent gets four tools — `context_read`, `context_contribute`, `context_ingest`, `context_stats`. The pitch: **read context → do the task → contribute what you learned**, on a shared team graph. Config is in the README.
+(Delete `northwind-demo.db*` to reset.)
---
@@ -132,8 +163,14 @@ src/
types.ts ← the data model (nodes, edges, chunks) — read this to grok the domain
merge.ts ← dedup + reinforcement (the "gets smarter" logic)
sqlite-store.ts ← the embedded storage backend
+ ingest/
+ chunker.ts ← paragraph-aware chunking with overlap
+ pdf.ts ← PDF → text extraction (unpdf)
retrieval/retriever.ts ← semantic match + one-hop expansion + prompt rendering
cli.ts ← the CLI
- mcp.ts ← the MCP server
-examples/quickstart.ts ← the demo script from §4
+ mcp.ts ← the MCP server (context_read / _contribute / _ingest / _ingest_file / _stats)
+examples/
+ quickstart.ts ← library quickstart
+ demo-docs/*.pdf ← the 3 demo PDFs used in §4
+scripts/make-demo-pdfs.mjs ← regenerates the demo PDFs
```
diff --git a/README.md b/README.md
index 38df2c96..29194825 100644
--- a/README.md
+++ b/README.md
@@ -84,8 +84,8 @@ npm run example
## CLI
```bash
-# Ingest documents
-context-graph ingest ./docs/*.md
+# Ingest documents (Markdown, text, and PDFs — PDFs are parsed automatically)
+context-graph ingest ./docs/*.md ./handbook.pdf
echo "some notes" | context-graph ingest-text --title "Notes"
# Read the graph
@@ -131,7 +131,8 @@ Tools exposed:
|------|--------------|
| `context_read` | Read relevant context for a query before working |
| `context_contribute` | Write a learning back into the shared graph |
-| `context_ingest` | Ingest a document into the graph |
+| `context_ingest` | Ingest a document (raw text) into the graph |
+| `context_ingest_file` | Ingest files from disk, including **PDFs** (parsed automatically) |
| `context_stats` | Report how much the graph currently holds |
A natural agent workflow: **read context → do the task → contribute what you learned.**
diff --git a/examples/demo-docs/northwind-architecture.pdf b/examples/demo-docs/northwind-architecture.pdf
new file mode 100644
index 00000000..a3ef16af
Binary files /dev/null and b/examples/demo-docs/northwind-architecture.pdf differ
diff --git a/examples/demo-docs/northwind-billing-runbook.pdf b/examples/demo-docs/northwind-billing-runbook.pdf
new file mode 100644
index 00000000..6a16122c
Binary files /dev/null and b/examples/demo-docs/northwind-billing-runbook.pdf differ
diff --git a/examples/demo-docs/northwind-onboarding.pdf b/examples/demo-docs/northwind-onboarding.pdf
new file mode 100644
index 00000000..87ed7150
Binary files /dev/null and b/examples/demo-docs/northwind-onboarding.pdf differ
diff --git a/package-lock.json b/package-lock.json
index 2572e543..9f932034 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -15,6 +15,7 @@
"commander": "^15.0.0",
"dotenv": "^17.4.2",
"openai": "^6.45.0",
+ "unpdf": "^1.6.2",
"zod": "^4.4.3"
},
"bin": {
@@ -24,6 +25,7 @@
"devDependencies": {
"@types/better-sqlite3": "^7.6.13",
"@types/node": "^26.1.0",
+ "pdfkit": "^0.17.2",
"tsx": "^4.22.4",
"typescript": "^6.0.3"
},
@@ -1090,6 +1092,16 @@
"integrity": "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==",
"license": "BSD-3-Clause"
},
+ "node_modules/@swc/helpers": {
+ "version": "0.5.23",
+ "resolved": "https://registry.npmjs.org/@swc/helpers/-/helpers-0.5.23.tgz",
+ "integrity": "sha512-5lSsMOTXURePglDfvuAQUqkGek9Hg2kksOYay2m0+XR++b2NWYL/4sWyuvVBIs8oKnJaxkdi9whaL/sqN13afw==",
+ "dev": true,
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.8.0"
+ }
+ },
"node_modules/@types/better-sqlite3": {
"version": "7.6.13",
"resolved": "https://registry.npmjs.org/@types/better-sqlite3/-/better-sqlite3-7.6.13.tgz",
@@ -1253,6 +1265,26 @@
"deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.",
"license": "MIT"
},
+ "node_modules/brotli": {
+ "version": "1.3.3",
+ "resolved": "https://registry.npmjs.org/brotli/-/brotli-1.3.3.tgz",
+ "integrity": "sha512-oTKjJdShmDuGW94SyyaoQvAjf30dZaHnjJ8uAF+u2/vGJkJbJPJAT1gDiOJP5v1Zb6f9KEyW/1HpuaWIXtGHPg==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "base64-js": "^1.1.2"
+ }
+ },
+ "node_modules/browserify-zlib": {
+ "version": "0.2.0",
+ "resolved": "https://registry.npmjs.org/browserify-zlib/-/browserify-zlib-0.2.0.tgz",
+ "integrity": "sha512-Z942RysHXmJrhqk88FmKBVq/v5tqmSkDz7p54G/MGyjMnCFFnC79XWNbg+Vta8W6Wb2qtSZTSxIGkJrRpCFEiA==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "pako": "~1.0.5"
+ }
+ },
"node_modules/buffer": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
@@ -1321,6 +1353,16 @@
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "ISC"
},
+ "node_modules/clone": {
+ "version": "2.1.2",
+ "resolved": "https://registry.npmjs.org/clone/-/clone-2.1.2.tgz",
+ "integrity": "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.8"
+ }
+ },
"node_modules/commander": {
"version": "15.0.0",
"resolved": "https://registry.npmjs.org/commander/-/commander-15.0.0.tgz",
@@ -1401,6 +1443,13 @@
"node": ">= 8"
}
},
+ "node_modules/crypto-js": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/crypto-js/-/crypto-js-4.2.0.tgz",
+ "integrity": "sha512-KALDyEYgpY+Rlob/iriUtjV6d5Eq+Y191A5g4UqLAi8CyGP9N1+FdVbkc1SxKc2r4YAYqG8JzO2KGL+AizD70Q==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
@@ -1500,6 +1549,13 @@
"integrity": "sha512-T0NIuQpnTvFDATNuHN5roPwSBG83rFsuO+MXXH9/3N1eFbn4wcPjttvjMLEPWJ0RGUYgQE7cGgS3tNxbqCGM7g==",
"license": "MIT"
},
+ "node_modules/dfa": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/dfa/-/dfa-1.2.0.tgz",
+ "integrity": "sha512-ED3jP8saaweFTjeGX8HQPjeC1YYyZs98jGNZx6IiBvxW7JG5v492kamAQB3m2wop07CvU/RQmzcKr6bgcC5D/Q==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/dotenv": {
"version": "17.4.2",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-17.4.2.tgz",
@@ -1801,6 +1857,24 @@
"integrity": "sha512-MI1qs7Lo4Syw0EOzUl0xjs2lsoeqFku44KpngfIduHBYvzm8h2+7K8YMQh1JtVVVrUvhLpNwqVi4DERegUJhPQ==",
"license": "Apache-2.0"
},
+ "node_modules/fontkit": {
+ "version": "2.0.4",
+ "resolved": "https://registry.npmjs.org/fontkit/-/fontkit-2.0.4.tgz",
+ "integrity": "sha512-syetQadaUEDNdxdugga9CpEYVaQIxOwk7GlwZWWZ19//qW4zE5bknOKeMBDYAASwnpaSHKJITRLMF9m1fp3s6g==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "@swc/helpers": "^0.5.12",
+ "brotli": "^1.3.2",
+ "clone": "^2.1.2",
+ "dfa": "^1.2.0",
+ "fast-deep-equal": "^3.1.3",
+ "restructure": "^3.0.0",
+ "tiny-inflate": "^1.0.3",
+ "unicode-properties": "^1.4.0",
+ "unicode-trie": "^2.0.0"
+ }
+ },
"node_modules/forwarded": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz",
@@ -2095,6 +2169,14 @@
"url": "https://github.com/sponsors/panva"
}
},
+ "node_modules/jpeg-exif": {
+ "version": "1.1.4",
+ "resolved": "https://registry.npmjs.org/jpeg-exif/-/jpeg-exif-1.1.4.tgz",
+ "integrity": "sha512-a+bKEcCjtuW5WTdgeXFzswSrdqi0jk4XlEtZlx5A94wCoBpFjfFTbo/Tra5SpNCl/YFZPvcV1dJc+TAYeg6ROQ==",
+ "deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/json-schema-traverse": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz",
@@ -2113,6 +2195,27 @@
"integrity": "sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==",
"license": "ISC"
},
+ "node_modules/linebreak": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmjs.org/linebreak/-/linebreak-1.1.0.tgz",
+ "integrity": "sha512-MHp03UImeVhB7XZtjd0E4n6+3xr5Dq/9xI/5FptGk5FrbDR3zagPa2DS6U8ks/3HjbKWG9Q1M2ufOzxV2qLYSQ==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "base64-js": "0.0.8",
+ "unicode-trie": "^2.0.0"
+ }
+ },
+ "node_modules/linebreak/node_modules/base64-js": {
+ "version": "0.0.8",
+ "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-0.0.8.tgz",
+ "integrity": "sha512-3XSA2cR/h/73EzlXXdU6YNycmYI7+kicTxks4eJg2g39biHR84slg2+des+p7iHYhbRg/udIS4TD53WabcOUkw==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.4"
+ }
+ },
"node_modules/long": {
"version": "5.3.2",
"resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz",
@@ -2391,6 +2494,13 @@
}
}
},
+ "node_modules/pako": {
+ "version": "1.0.11",
+ "resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz",
+ "integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
+ "dev": true,
+ "license": "(MIT AND Zlib)"
+ },
"node_modules/parseurl": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
@@ -2419,6 +2529,20 @@
"url": "https://opencollective.com/express"
}
},
+ "node_modules/pdfkit": {
+ "version": "0.17.2",
+ "resolved": "https://registry.npmjs.org/pdfkit/-/pdfkit-0.17.2.tgz",
+ "integrity": "sha512-UnwF5fXy08f0dnp4jchFYAROKMNTaPqb/xgR8GtCzIcqoTnbOqtp3bwKvO4688oHI6vzEEs8Q6vqqEnC5IUELw==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "crypto-js": "^4.2.0",
+ "fontkit": "^2.0.4",
+ "jpeg-exif": "^1.1.4",
+ "linebreak": "^1.1.0",
+ "png-js": "^1.0.0"
+ }
+ },
"node_modules/pkce-challenge": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/pkce-challenge/-/pkce-challenge-5.0.1.tgz",
@@ -2434,6 +2558,15 @@
"integrity": "sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==",
"license": "MIT"
},
+ "node_modules/png-js": {
+ "version": "1.1.0",
+ "resolved": "https://registry.npmjs.org/png-js/-/png-js-1.1.0.tgz",
+ "integrity": "sha512-PM/uYGzGdNSzqeOgly68+6wKQDL1SY0a/N+OEa/+br6LnHWOAJB0Npiamnodfq3jd2LS/i2fMeOKSAILjA+m5Q==",
+ "dev": true,
+ "dependencies": {
+ "browserify-zlib": "^0.2.0"
+ }
+ },
"node_modules/prebuild-install": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@ -2589,6 +2722,13 @@
"node": ">=0.10.0"
}
},
+ "node_modules/restructure": {
+ "version": "3.0.2",
+ "resolved": "https://registry.npmjs.org/restructure/-/restructure-3.0.2.tgz",
+ "integrity": "sha512-gSfoiOEA0VPE6Tukkrr7I0RBdE0s7H1eFCDBk05l1KIQT1UIKNc5JZy6jdyW6eYH3aR3g5b3PuL77rq0hvwtAw==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/roarr": {
"version": "2.15.4",
"resolved": "https://registry.npmjs.org/roarr/-/roarr-2.15.4.tgz",
@@ -3000,6 +3140,13 @@
"node": ">=18"
}
},
+ "node_modules/tiny-inflate": {
+ "version": "1.0.3",
+ "resolved": "https://registry.npmjs.org/tiny-inflate/-/tiny-inflate-1.0.3.tgz",
+ "integrity": "sha512-pkY1fj1cKHb2seWDy0B16HeWyczlJA9/WW3u3c4z/NiWDsO3DOU5D7nhTLE9CF0yXv/QZFY7sEJmj24dK+Rrqw==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/toidentifier": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
@@ -3013,8 +3160,8 @@
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
- "license": "0BSD",
- "optional": true
+ "devOptional": true,
+ "license": "0BSD"
},
"node_modules/tsx": {
"version": "4.22.4",
@@ -3110,6 +3257,49 @@
"integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==",
"license": "MIT"
},
+ "node_modules/unicode-properties": {
+ "version": "1.4.1",
+ "resolved": "https://registry.npmjs.org/unicode-properties/-/unicode-properties-1.4.1.tgz",
+ "integrity": "sha512-CLjCCLQ6UuMxWnbIylkisbRj31qxHPAurvena/0iwSVbQ2G1VY5/HjV0IRabOEbDHlzZlRdCrD4NhB0JtU40Pg==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "base64-js": "^1.3.0",
+ "unicode-trie": "^2.0.0"
+ }
+ },
+ "node_modules/unicode-trie": {
+ "version": "2.0.0",
+ "resolved": "https://registry.npmjs.org/unicode-trie/-/unicode-trie-2.0.0.tgz",
+ "integrity": "sha512-x7bc76x0bm4prf1VLg79uhAzKw8DVboClSN5VxJuQ+LKDOVEW9CdH+VY7SP+vX7xCYQqzzgQpFqz15zeLvAtZQ==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "pako": "^0.2.5",
+ "tiny-inflate": "^1.0.0"
+ }
+ },
+ "node_modules/unicode-trie/node_modules/pako": {
+ "version": "0.2.9",
+ "resolved": "https://registry.npmjs.org/pako/-/pako-0.2.9.tgz",
+ "integrity": "sha512-NUcwaKxUxWrZLpDG+z/xZaCgQITkA/Dv4V/T6bw7VON6l1Xz/VnrBqrYjZQ12TamKHzITTfOEIYUj48y2KXImA==",
+ "dev": true,
+ "license": "MIT"
+ },
+ "node_modules/unpdf": {
+ "version": "1.6.2",
+ "resolved": "https://registry.npmjs.org/unpdf/-/unpdf-1.6.2.tgz",
+ "integrity": "sha512-zQ80ySoPuPHOsvIoRp/nJyQt8TOUoTh1+WBCGcBvlddQNgKDLRwm0AY3x8Q35I7+kIiRSgqMx+Ma2pl9McIp7A==",
+ "license": "MIT",
+ "peerDependencies": {
+ "@napi-rs/canvas": "^0.1.69"
+ },
+ "peerDependenciesMeta": {
+ "@napi-rs/canvas": {
+ "optional": true
+ }
+ }
+ },
"node_modules/unpipe": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
diff --git a/package.json b/package.json
index 5a357c61..eb48447f 100644
--- a/package.json
+++ b/package.json
@@ -49,11 +49,13 @@
"commander": "^15.0.0",
"dotenv": "^17.4.2",
"openai": "^6.45.0",
+ "unpdf": "^1.6.2",
"zod": "^4.4.3"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.13",
"@types/node": "^26.1.0",
+ "pdfkit": "^0.17.2",
"tsx": "^4.22.4",
"typescript": "^6.0.3"
}
diff --git a/scripts/make-demo-pdfs.mjs b/scripts/make-demo-pdfs.mjs
new file mode 100644
index 00000000..66076606
--- /dev/null
+++ b/scripts/make-demo-pdfs.mjs
@@ -0,0 +1,130 @@
+/**
+ * Generate the demo PDFs used in the MCP walkthrough (see DEMO.md).
+ *
+ * node scripts/make-demo-pdfs.mjs
+ *
+ * Writes 2–3 PDFs full of specific, queryable facts into examples/demo-docs/.
+ * The content is fictional ("Northwind") so nothing here is sensitive; the point
+ * is to have concrete details a query can retrieve back through the graph.
+ */
+import PDFDocument from "pdfkit";
+import { createWriteStream, mkdirSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const OUT_DIR = join(dirname(fileURLToPath(import.meta.url)), "..", "examples", "demo-docs");
+mkdirSync(OUT_DIR, { recursive: true });
+
+/** Render a simple titled document with headings + paragraphs to a PDF file. */
+function writePdf(filename, title, blocks) {
+ return new Promise((resolve, reject) => {
+ const doc = new PDFDocument({ margin: 56, size: "A4" });
+ const path = join(OUT_DIR, filename);
+ const stream = createWriteStream(path);
+ doc.pipe(stream);
+
+ doc.font("Helvetica-Bold").fontSize(20).text(title);
+ doc.moveDown(0.8);
+
+ for (const block of blocks) {
+ if (block.heading) {
+ doc.moveDown(0.4);
+ doc.font("Helvetica-Bold").fontSize(13).text(block.heading);
+ doc.moveDown(0.2);
+ }
+ doc.font("Helvetica").fontSize(11).text(block.body, { align: "left", lineGap: 2 });
+ }
+
+ doc.end();
+ stream.on("finish", () => resolve(path));
+ stream.on("error", reject);
+ });
+}
+
+const docs = [
+ {
+ filename: "northwind-architecture.pdf",
+ title: "Northwind Platform — Architecture Overview",
+ blocks: [
+ {
+ heading: "Services",
+ body:
+ "The Northwind platform is composed of five core services. The Orders Service accepts customer orders and is the system of record for order state. It calls the Inventory Service to reserve stock and the Payments Service to charge the customer. The Payments Service integrates with Stripe as the payment processor. The Notifications Service sends transactional email through SendGrid and SMS through Twilio.",
+ },
+ {
+ heading: "Data stores",
+ body:
+ "The Orders Service stores order data in a PostgreSQL database named orders_db. The Inventory Service uses a separate PostgreSQL database named inventory_db. Both services publish domain events to an Apache Kafka cluster on the topic 'northwind.events'. The Notifications Service consumes those events to decide when to message customers.",
+ },
+ {
+ heading: "Authentication",
+ body:
+ "All services authenticate requests using JWT access tokens issued by the Auth Service. Access tokens expire after 15 minutes; refresh tokens expire after 30 days. The Auth Service is backed by Redis for token revocation lists.",
+ },
+ {
+ heading: "Deployment",
+ body:
+ "Northwind runs on Kubernetes in the AWS us-east-1 region. Each service is deployed as a separate Deployment with a minimum of 3 replicas. Container images are built by GitHub Actions and pushed to Amazon ECR. Production deploys require approval from a member of the Platform team.",
+ },
+ ],
+ },
+ {
+ filename: "northwind-billing-runbook.pdf",
+ title: "Northwind Billing — On-Call Runbook",
+ blocks: [
+ {
+ heading: "Failed charges",
+ body:
+ "When a charge fails, the Payments Service emits a payment_failed event. The Dunning Worker retries the charge up to 3 times with exponential backoff (1 hour, 6 hours, 24 hours). After the third failed retry, the subscription is marked past_due and the Notifications Service emails the customer. If the account remains past_due for 7 days, it is automatically suspended.",
+ },
+ {
+ heading: "Refunds",
+ body:
+ "Refunds are issued through the Payments Service admin API. Refunds above $500 require approval from a Finance team lead. All refunds are logged to the audit_log table in payments_db and reflected in Stripe within 24 hours.",
+ },
+ {
+ heading: "Escalation",
+ body:
+ "The billing on-call rotation is owned by the Payments team. The primary on-call responds within 15 minutes during business hours and within 30 minutes off-hours. Page the on-call via PagerDuty service 'northwind-billing'. For Stripe outages, escalate to the Payments team lead, Dana Whitfield, and post an update in the #billing-incidents Slack channel.",
+ },
+ {
+ heading: "Key metrics",
+ body:
+ "The billing dashboard tracks charge success rate (target: above 97%), average time to first retry, and past_due account count. An alert fires if the charge success rate drops below 95% over any 30-minute window.",
+ },
+ ],
+ },
+ {
+ filename: "northwind-onboarding.pdf",
+ title: "Northwind Engineering — New Hire Onboarding",
+ blocks: [
+ {
+ heading: "Getting access",
+ body:
+ "New engineers request access through the IT portal. Standard access includes GitHub (the northwind-inc org), the AWS developer account, PagerDuty, and Slack. Production database access requires a separate request approved by your team lead and is granted for 90 days at a time.",
+ },
+ {
+ heading: "Local setup",
+ body:
+ "Clone the northwind-inc/platform monorepo. Install Node.js 20 and Docker. Run 'make bootstrap' to start local Postgres, Redis, and Kafka via docker-compose. Copy .env.sample to .env and fill in your personal API keys. Run 'make test' to verify the setup.",
+ },
+ {
+ heading: "Team structure",
+ body:
+ "Engineering has four teams. The Platform team owns infrastructure and CI/CD. The Payments team owns the Payments Service and billing. The Orders team owns the Orders and Inventory services. The Growth team owns the Notifications Service and customer-facing features. The VP of Engineering is Priya Nair.",
+ },
+ {
+ heading: "First week",
+ body:
+ "In your first week you should complete the security training, ship one small change to production, and pair with a teammate on an on-call shadow. Your onboarding buddy is assigned on day one. Ask questions in the #eng-help Slack channel.",
+ },
+ ],
+ },
+];
+
+const written = [];
+for (const d of docs) {
+ written.push(await writePdf(d.filename, d.title, d.blocks));
+}
+console.log("Wrote demo PDFs:");
+for (const p of written) console.log(" " + p);
diff --git a/src/engine.ts b/src/engine.ts
index 10406094..3518c52d 100644
--- a/src/engine.ts
+++ b/src/engine.ts
@@ -1,5 +1,6 @@
import { readFileSync } from "node:fs";
import { basename } from "node:path";
+import { extractPdfText, isPdfPath } from "./ingest/pdf.js";
import type { GraphStore } from "./graph/store.js";
import { SqliteStore } from "./graph/sqlite-store.js";
import type { Embedder, EngineConfig, Extractor, ResolvedConfig } from "./ai/providers.js";
@@ -108,9 +109,12 @@ export class ContextGraphEngine {
return this._extractor;
}
- /** Ingest a document from disk. Title defaults to the file name. */
+ /**
+ * Ingest a document from disk. PDFs (`.pdf`) are parsed to text automatically;
+ * anything else is read as UTF-8. Title defaults to the file name.
+ */
async ingestFile(path: string, opts: IngestOptions = {}): Promise<IngestResult> {
- const text = readFileSync(path, "utf8");
+ const text = isPdfPath(path) ? await extractPdfText(path) : readFileSync(path, "utf8");
return this.ingest(text, { title: opts.title ?? basename(path), source: opts.source ?? path });
}
diff --git a/src/ingest/pdf.ts b/src/ingest/pdf.ts
new file mode 100644
index 00000000..f34feed3
--- /dev/null
+++ b/src/ingest/pdf.ts
@@ -0,0 +1,21 @@
+import { readFile } from "node:fs/promises";
+
+/**
+ * Extract plain text from a PDF file.
+ *
+ * Uses `unpdf` (a pure-JS build of pdf.js) so there are no native binaries to
+ * compile — it works anywhere Node runs, which keeps the `curl | sh` install
+ * story honest. Pages are merged into a single string for chunking.
+ */
+export async function extractPdfText(path: string): Promise<string> {
+ const { extractText, getDocumentProxy } = await import("unpdf");
+ const buffer = await readFile(path);
+ const pdf = await getDocumentProxy(new Uint8Array(buffer));
+ const { text } = await extractText(pdf, { mergePages: true });
+ return Array.isArray(text) ? text.join("\n\n") : text;
+}
+
+/** True if the path looks like a PDF (by extension). */
+export function isPdfPath(path: string): boolean {
+ return /\.pdf$/i.test(path);
+}
diff --git a/src/mcp.ts b/src/mcp.ts
index 91b22ceb..4cf7a390 100644
--- a/src/mcp.ts
+++ b/src/mcp.ts
@@ -86,6 +86,37 @@ server.registerTool(
},
);
+server.registerTool(
+ "context_ingest_file",
+ {
+ title: "Ingest a file (incl. PDFs)",
+ description:
+ "Ingest one or more files from disk into the shared context graph. PDF files are parsed to text automatically. Accepts absolute paths.",
+ inputSchema: {
+ paths: z
+ .array(z.string())
+ .min(1)
+ .describe("Absolute file paths to ingest (e.g. PDFs)."),
+ },
+ },
+ async ({ paths }) => {
+ const lines: string[] = [];
+ for (const path of paths) {
+ try {
+ const r = await engine.ingestFile(path);
+ lines.push(
+ r.skipped
+ ? `• ${r.title}: already ingested, skipped`
+ : `✓ ${r.title}: ${r.chunks} chunks, +${r.nodesCreated} entities (${r.nodesUpdated} reinforced), +${r.edgesCreated} relationships`,
+ );
+ } catch (err) {
+ lines.push(`✗ ${path}: ${err instanceof Error ? err.message : String(err)}`);
+ }
+ }
+ return { content: [{ type: "text", text: lines.join("\n") }] };
+ },
+);
+
server.registerTool(
"context_stats",
{