Turbocharge Claude Code, Cursor, Codex, Gemini & every coding agent: faster, cheaper, with contextual understanding specific to your codebase.
TypeScript ai-agentsanthropicclaude-codeclicode-graphcodexcontext-engineeringcursordeveloper-toolsgeminiknowledge-graphllmmcpmcp-serveropen-sourceopenaitree-sitter
feat(app): keep review pages on disk so posted links survive a restart (#278)
anirudhkumar-nanonets373506d41565
diff --git a/Dockerfile b/Dockerfileindex 36987b7b..bd7aab38 100644--- a/Dockerfile+++ b/Dockerfile@@ -44,10 +44,19 @@ COPY --from=build /app/dist ./dist# --ignore-scripts stops `prepare` from trying to rebuild without tsc.RUN npm prune --omit=dev --ignore-scripts && npm cache clean --force+# The page store's directory exists in the image, owned by node, for one reason:+# Docker initialises a fresh named volume from the mountpoint it covers, so a+# `-v graft-pages:/var/lib/graft/pages` over a path that did NOT exist arrives+# root-owned and the App silently cannot write to it. Unmounted this is still+# worth having — `docker restart` keeps the container's filesystem, so the links+# already posted to pull requests survive that much on their own.+RUN mkdir -p /var/lib/graft/pages && chown node:node /var/lib/graft/pages+# Never root: this process handles untrusted source and has no reason to be able# to write outside its own tree.USER nodeENV PORT=3000+ENV GRAFT_PAGE_DIR=/var/lib/graft/pagesEXPOSE 3000HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||3000)+'/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"diff --git a/docs/github-app.md b/docs/github-app.mdindex ab7af864..7dce891d 100644--- a/docs/github-app.md+++ b/docs/github-app.md@@ -54,6 +54,7 @@ docker run -p 3000:3000 \-e GRAFT_APP_PRIVATE_KEY="$(cat graft.private-key.pem)" \-e GRAFT_WEBHOOK_SECRET=... \-e GRAFT_PUBLIC_URL=https://graft.example.com \+ -v graft-pages:/var/lib/graft/pages \graft-app```@@ -65,6 +66,16 @@ because it is what the comment's link is built from.The process refuses to start if any of those four are missing: a server thatboots without a webhook secret looks healthy and silently rejects every delivery.+The volume is the fifth thing and it is optional — but without it, every link+already posted to a pull request breaks the first time the container is replaced.+The token in a link is derived from the page id rather than stored, so the link+keeps verifying and simply 404s, with nothing in the comment to say why. The image+already points `GRAFT_PAGE_DIR` at `/var/lib/graft/pages` (500 pages of ~50 kB,+so ~25 MB is the ceiling); mount anything durable there and the pages come back+with the process. Set the variable yourself on hosts that are not this image, or+leave it empty to keep the store in memory — it is deliberately not required, as+a server that refuses to boot reviews nothing at all.+### 3. Install it on a repositoryApp settings → Install App → pick the repos. **Installing requires admin on thediff --git a/src/app/main.ts b/src/app/main.tsindex c29998be..381bb7ac 100644--- a/src/app/main.ts+++ b/src/app/main.ts@@ -40,6 +40,12 @@ const { server, queue } = createApp({publicUrl: required("GRAFT_PUBLIC_URL"),port,concurrency: Number(process.env.GRAFT_CONCURRENCY ?? 2),+ // Optional, unlike the four above, and deliberately so: point it at a mounted+ // volume and the pages already linked from pull requests survive a restart;+ // leave it unset and they do not. Neither is worth refusing to boot over — a+ // server that will not start reviews nothing at all, which is strictly worse+ // than one whose old links go stale.+ pageDir: process.env.GRAFT_PAGE_DIR,});/**diff --git a/src/app/pages.ts b/src/app/pages.tsindex 8e2eda46..7728ec08 100644--- a/src/app/pages.ts+++ b/src/app/pages.ts@@ -10,13 +10,25 @@* capability, not a login. Anyone with the link can read that one page until it* expires, which is the same property a GitHub artifact link has, and the link* only ever appears in a comment on the pull request it describes.+ *+ * Pages are mirrored to `dir`, when one is given, because the token is DERIVED+ * from the id rather than stored: a link keeps passing the signature check for as+ * long as the secret lives, so a process that lost its pages answers 404 to a+ * link that looks perfectly valid. That failure is silent at both ends — the+ * comment still sits on the pull request, the reviewer just sees "not found" —+ * and one restart of a container with nothing mounted stranded 29 of them. Disk+ * is the fix; without a directory this behaves exactly as it did before.*/-import { createHmac, timingSafeEqual } from "node:crypto";+import { createHmac, randomBytes, timingSafeEqual } from "node:crypto";+import { mkdirSync, readFileSync, readdirSync, renameSync, unlinkSync, writeFileSync } from "node:fs";+import { join } from "node:path";/** Long enough to review a PR at leisure, short enough that a leaked link dies. */const DEFAULT_TTL_MS = 14 * 24 * 60 * 60 * 1000;/** A page is ~50 kB; a few hundred of them is nothing, and eviction is by age. */const DEFAULT_MAX_PAGES = 500;+/** One file per page, named by its id. The extension also means "fully written". */+const SUFFIX = ".json";export interface StoredPage {html: string;@@ -28,6 +40,8 @@ export interface PageStoreOptions {ttlMs?: number;maxPages?: number;now?: () => number;+ /** Kept across restarts here. Absent: pages live only as long as the process. */+ dir?: string;}export class PageStore {@@ -36,12 +50,15 @@ export class PageStore {private readonly ttlMs: number;private readonly maxPages: number;private readonly now: () => number;+ private readonly dir: string | undefined;constructor(opts: PageStoreOptions) {this.secret = opts.secret;this.ttlMs = opts.ttlMs ?? DEFAULT_TTL_MS;this.maxPages = opts.maxPages ?? DEFAULT_MAX_PAGES;this.now = opts.now ?? Date.now;+ this.dir = opts.dir;+ this.hydrate();}/** `owner/repo#number` → a path segment safe to put in a URL. */@@ -52,7 +69,9 @@ export class PageStore {/** Store (replacing any previous page for the same PR) and return `id` + `token`. */put(owner: string, repo: string, number: number, html: string): { id: string; token: string } {const id = PageStore.idFor(owner, repo, number);- this.pages.set(id, { html, storedMs: this.now() });+ const page: StoredPage = { html, storedMs: this.now() };+ this.pages.set(id, page);+ this.write(id, page);this.evict();return { id, token: this.sign(id) };}@@ -63,12 +82,15 @@ export class PageStore {* Expiry is checked against the STORED time rather than encoded in the token,* so a link cannot outlive the data it points at — and re-reviewing a PR* refreshes both together.+ *+ * Answered from memory even when there is a directory: `id` comes off a URL,+ * and an id that never reaches the filesystem cannot be walked out of it.*/get(id: string, token: string | undefined): string | null {const page = this.pages.get(id);if (!page || !token || !this.valid(id, token)) return null;if (this.now() - page.storedMs > this.ttlMs) {- this.pages.delete(id);+ this.remove(id);return null;}return page.html;@@ -93,11 +115,94 @@ export class PageStore {/** Oldest out first, plus anything past its TTL. */private evict(): void {const cutoff = this.now() - this.ttlMs;- for (const [id, page] of this.pages) if (page.storedMs < cutoff) this.pages.delete(id);+ for (const [id, page] of this.pages) if (page.storedMs < cutoff) this.remove(id);while (this.pages.size > this.maxPages) {const oldest = this.pages.keys().next();if (oldest.done) break;- this.pages.delete(oldest.value);+ this.remove(oldest.value);+ }+ }++ /** Take back what a previous process left on disk. */+ private hydrate(): void {+ const dir = this.dir;+ if (!dir) return;++ let names: string[];+ try {+ mkdirSync(dir, { recursive: true });+ names = readdirSync(dir);+ } catch (err) {+ // Nothing usable there: carry on in memory, which is exactly what every+ // deployment without a directory does anyway.+ console.warn(`page store: ${dir} is unusable, pages will not survive a restart (${reason(err)})`);+ return;+ }++ const loaded: Array<[string, StoredPage]> = [];+ for (const name of names) {+ // The suffix also skips the temp files a write leaves behind when it dies+ // partway, so a half-written page is never a candidate to begin with.+ if (!name.endsWith(SUFFIX)) continue;+ const page = this.read(join(dir, name));+ if (page) loaded.push([name.slice(0, -SUFFIX.length), page]);+ }+ // Oldest first, because eviction takes the front of the Map and readdir order+ // is the filesystem's business rather than a record of what happened when.+ loaded.sort((a, b) => a[1].storedMs - b[1].storedMs);+ for (const [id, page] of loaded) this.pages.set(id, page);+ // A restart is also when pages that expired while the process was down go.+ this.evict();+ }++ /** A stored page, or nothing: truncated, corrupt and gone are one answer here. */+ private read(file: string): StoredPage | null {+ try {+ const parsed: unknown = JSON.parse(readFileSync(file, "utf8"));+ const { html, storedMs } = (parsed ?? {}) as Partial<StoredPage>;+ if (typeof html !== "string" || typeof storedMs !== "number" || !Number.isFinite(storedMs)) return null;+ return { html, storedMs };+ } catch {+ return null;+ }+ }++ /**+ * Temp file, then rename: a reader only ever sees a whole page, and two reviews+ * of the same pull request racing each other leave one of them intact rather+ * than the two interleaved.+ */+ private write(id: string, page: StoredPage): void {+ const dir = this.dir;+ if (!dir) return;++ const tmp = join(dir, `${id}.${randomBytes(6).toString("hex")}.tmp`);+ try {+ mkdirSync(dir, { recursive: true });+ writeFileSync(tmp, JSON.stringify(page));+ renameSync(tmp, join(dir, `${id}${SUFFIX}`));+ } catch (err) {+ // The page is in memory and its link works; it just will not outlive this+ // process. Worth a line in the log and nothing more: throwing here would+ // fail a review that has already done all of its work.+ console.warn(`page store: could not write ${id} (${reason(err)})`);+ try {+ unlinkSync(tmp);+ } catch {+ // Never created, or already renamed into place.+ }+ }+ }++ private remove(id: string): void {+ this.pages.delete(id);+ if (!this.dir) return;+ try {+ unlinkSync(join(this.dir, `${id}${SUFFIX}`));+ } catch {+ // Already gone: deleted underneath us, or it never reached the disk.}}}++const reason = (err: unknown): string => (err instanceof Error ? err.message : String(err));diff --git a/src/app/server.ts b/src/app/server.tsindex db487b76..dcce6a40 100644--- a/src/app/server.ts+++ b/src/app/server.ts@@ -32,6 +32,8 @@ export interface AppConfig extends AppCredentials {webhookSecret: string;/** Public origin, for the links put in comments, e.g. https://graft.example.com */publicUrl: string;+ /** Where pages are kept, so a restart does not strand the links already posted. */+ pageDir?: string;port?: number;concurrency?: number;api?: string;@@ -46,7 +48,7 @@ export function createApp(const fetchImpl = seams.fetch ?? (globalThis.fetch as unknown as Fetch);const review = seams.review ?? reviewPullRequest;const tokens = new InstallationTokens(config, fetchImpl, seams.now ?? Date.now, config.api);- const pages = new PageStore({ secret: config.webhookSecret });+ const pages = new PageStore({ secret: config.webhookSecret, dir: config.pageDir });const origin = config.publicUrl.replace(/\/$/, "");const queue = new WorkQueue<ReviewJob>(diff --git a/test/app-pages.test.ts b/test/app-pages.test.tsnew file mode 100644index 00000000..32497d84--- /dev/null+++ b/test/app-pages.test.ts@@ -0,0 +1,200 @@+/**+ * Pages that outlive the process holding them.+ *+ * The link in a pull request comment carries a token derived from the page id, so+ * it verifies forever and a restart turns it into a silent 404 — which is how 29+ * comments on one repository ended up pointing at nothing. What is asserted here+ * is therefore the restart itself: a second `PageStore` over the same directory+ * has to answer the links the first one handed out, on the same terms (token,+ * TTL, cap), and none of the ways a filesystem can misbehave may reach the+ * caller as an exception.+ */+import { test } from "node:test";+import assert from "node:assert/strict";+import { mkdtempSync, readdirSync, readFileSync, rmSync, unlinkSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { PageStore } from "../src/app/pages.js";++const secret = "webhook-secret";++/** Each test gets its own directory, and the store's is a level below it. */+function scratch(): { root: string; dir: string; cleanup: () => void } {+ const root = mkdtempSync(join(tmpdir(), "graft-pages-"));+ // Nested and absent: the App's volume is mounted empty, so the store has to+ // make its own directory rather than expect one.+ return { root, dir: join(root, "pages"), cleanup: () => rmSync(root, { recursive: true, force: true }) };+}++test("pages: a restart serves the links the previous process handed out", () => {+ const { dir, cleanup } = scratch();+ try {+ const first = new PageStore({ secret, dir });+ const { id, token } = first.put("NanoNets", "assign", 42, "<h1>radius</h1>");++ const restarted = new PageStore({ secret, dir });+ assert.equal(restarted.size, 1, "the page was on disk, not only in the process that made it");+ assert.equal(restarted.get(id, token), "<h1>radius</h1>", "the comment's link still works");+ // The token is the only credential and it is derived, so the restart must not+ // have widened access either.+ assert.equal(restarted.get(id, "nope"), null);+ assert.equal(restarted.get(id, undefined), null);+ assert.equal(restarted.get("NanoNets__assign__43", token), null);+ } finally {+ cleanup();+ }+});++test("pages: TTL is still counted from when the page was stored", () => {+ const { dir, cleanup } = scratch();+ try {+ let now = 1_000;+ const first = new PageStore({ secret, dir, ttlMs: 100, now: () => now });+ const { id, token } = first.put("o", "r", 1, "<html>");++ now += 50;+ assert.equal(new PageStore({ secret, dir, ttlMs: 100, now: () => now }).get(id, token), "<html>");++ now += 51;+ const late = new PageStore({ secret, dir, ttlMs: 100, now: () => now });+ assert.equal(late.get(id, token), null, "expired while the process was down, not on a fresh clock");+ assert.equal(late.size, 0);+ assert.deepEqual(readdirSync(dir), [], "and a restart is when an expired page finally leaves the disk");+ } finally {+ cleanup();+ }+});++test("pages: expiry during a run takes the file with it", () => {+ const { dir, cleanup } = scratch();+ try {+ let now = 1_000;+ const store = new PageStore({ secret, dir, ttlMs: 100, now: () => now });+ const { id, token } = store.put("o", "r", 1, "<html>");+ now += 101;++ assert.equal(store.get(id, token), null);+ assert.equal(store.size, 0);+ assert.deepEqual(readdirSync(dir), [], "otherwise the next restart would resurrect it");+ } finally {+ cleanup();+ }+});++test("pages: maxPages bounds the directory too, oldest out first", () => {+ const { dir, cleanup } = scratch();+ try {+ let now = 1_000;+ const store = new PageStore({ secret, dir, maxPages: 2, now: () => now++ });+ const first = store.put("o", "r", 1, "<one>");+ store.put("o", "r", 2, "<two>");+ store.put("o", "r", 3, "<three>");++ assert.equal(store.size, 2);+ assert.deepEqual(readdirSync(dir).sort(), ["o__r__2.json", "o__r__3.json"]);+ assert.equal(store.get(first.id, first.token), null);++ const restarted = new PageStore({ secret, dir, maxPages: 2, now: () => now });+ assert.equal(restarted.size, 2, "a restart cannot re-import what eviction already dropped");+ } finally {+ cleanup();+ }+});++test("pages: a re-review replaces the page rather than accumulating files", () => {+ const { dir, cleanup } = scratch();+ try {+ let now = 1_000;+ const store = new PageStore({ secret, dir, now: () => now });+ store.put("o", "r", 7, "<first pass>");+ now += 5_000;+ const { id, token } = store.put("o", "r", 7, "<second pass>");++ assert.deepEqual(readdirSync(dir), ["o__r__7.json"], "one file per PR, and no temp files left over");+ const restarted = new PageStore({ secret, dir, now: () => now });+ assert.equal(restarted.get(id, token), "<second pass>");+ } finally {+ cleanup();+ }+});++test("pages: an unreadable file is a missing page, not a thrown request", () => {+ const { dir, cleanup } = scratch();+ try {+ // Made by the store so the directory exists, then vandalised the three ways+ // a real one goes wrong: truncated JSON, valid JSON of the wrong shape, and+ // bytes that are not JSON at all.+ const store = new PageStore({ secret, dir });+ const { id, token } = store.put("o", "r", 1, "<html>");+ writeFileSync(join(dir, "o__r__1.json"), '{"html":"<htm');+ writeFileSync(join(dir, "o__r__2.json"), '{"html":42}');+ writeFileSync(join(dir, "o__r__3.json"), "not json");+ writeFileSync(join(dir, "o__r__4.json.tmp"), JSON.stringify({ html: "<half>", storedMs: 1 }));++ const restarted = new PageStore({ secret, dir });+ assert.equal(restarted.size, 0, "nothing there was a page, including the interrupted write");+ assert.equal(restarted.get(id, token), null, "the caller sees a 404's worth of nothing");+ } finally {+ cleanup();+ }+});++test("pages: a directory it cannot use degrades to memory instead of failing reviews", () => {+ const { root, cleanup } = scratch();+ // A file where the directory should be — a bad mount, in one line.+ const wrong = join(root, "not-a-directory");+ writeFileSync(wrong, "");+ const warned: string[] = [];+ const real = console.warn;+ console.warn = (...args: unknown[]) => void warned.push(args.map(String).join(" "));+ try {+ const store = new PageStore({ secret, dir: wrong });+ const { id, token } = store.put("o", "r", 9, "<served anyway>");++ assert.equal(store.get(id, token), "<served anyway>", "the review still published a working link");+ assert.equal(store.size, 1);+ // Loud enough to find in the logs, because the operator has lost durability+ // and nothing else in the system will say so.+ assert.match(warned.join("\n"), /will not survive a restart/);+ assert.match(warned.join("\n"), /could not write o__r__9/);+ } finally {+ console.warn = real;+ cleanup();+ }+});++test("pages: a file deleted underneath the process does not break the live page", () => {+ const { dir, cleanup } = scratch();+ try {+ const store = new PageStore({ secret, dir });+ const { id, token } = store.put("o", "r", 1, "<html>");+ unlinkSync(join(dir, "o__r__1.json"));++ assert.equal(store.get(id, token), "<html>", "the page is in memory; the disk is a mirror of it");+ // Storing it again has to put the file back, and dropping it again has to+ // survive the file being gone a second time.+ store.put("o", "r", 1, "<html>");+ assert.equal(JSON.parse(readFileSync(join(dir, "o__r__1.json"), "utf8")).html, "<html>");+ unlinkSync(join(dir, "o__r__1.json"));+ assert.equal(new PageStore({ secret, dir }).size, 0);+ } finally {+ cleanup();+ }+});++test("pages: no directory means the old in-memory behaviour, untouched", () => {+ const { dir, cleanup } = scratch();+ try {+ const store = new PageStore({ secret });+ const { id, token } = store.put("o", "r", 1, "<html>");++ assert.equal(store.get(id, token), "<html>");+ assert.equal(store.size, 1);+ assert.equal(new PageStore({ secret }).get(id, token), null, "a restart loses them, as it always did");+ // Nothing was created anywhere: the directory in this test was never given+ // to a store, and the store must not have invented one.+ assert.throws(() => readdirSync(dir));+ } finally {+ cleanup();+ }+});