Skip to content

Turbocharge Claude Code, Cursor, Codex, Gemini & every coding agent: faster, cheaper, with contextual understanding specific to your codebase.

feat(app): keep review pages on disk so posted links survive a restart (#278)

anirudhkumar-nanonets373506d41565
diff --git a/Dockerfile b/Dockerfile
index 36987b7b..bd7aab38 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -44,10 +44,19 @@ COPY --from=build /app/dist ./dist
# --ignore-scripts stops `prepare` from trying to rebuild without tsc.
RUN npm prune --omit=dev --ignore-scripts && npm cache clean --force
+# The page store's directory exists in the image, owned by node, for one reason:
+# Docker initialises a fresh named volume from the mountpoint it covers, so a
+# `-v graft-pages:/var/lib/graft/pages` over a path that did NOT exist arrives
+# root-owned and the App silently cannot write to it. Unmounted this is still
+# worth having — `docker restart` keeps the container's filesystem, so the links
+# already posted to pull requests survive that much on their own.
+RUN mkdir -p /var/lib/graft/pages && chown node:node /var/lib/graft/pages
+
# Never root: this process handles untrusted source and has no reason to be able
# to write outside its own tree.
USER node
ENV PORT=3000
+ENV GRAFT_PAGE_DIR=/var/lib/graft/pages
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s \
CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||3000)+'/healthz').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"
diff --git a/docs/github-app.md b/docs/github-app.md
index ab7af864..7dce891d 100644
--- a/docs/github-app.md
+++ b/docs/github-app.md
@@ -54,6 +54,7 @@ docker run -p 3000:3000 \
-e GRAFT_APP_PRIVATE_KEY="$(cat graft.private-key.pem)" \
-e GRAFT_WEBHOOK_SECRET=... \
-e GRAFT_PUBLIC_URL=https://graft.example.com \
+ -v graft-pages:/var/lib/graft/pages \
graft-app
```
@@ -65,6 +66,16 @@ because it is what the comment's link is built from.
The process refuses to start if any of those four are missing: a server that
boots without a webhook secret looks healthy and silently rejects every delivery.
+The volume is the fifth thing and it is optional — but without it, every link
+already posted to a pull request breaks the first time the container is replaced.
+The token in a link is derived from the page id rather than stored, so the link
+keeps verifying and simply 404s, with nothing in the comment to say why. The image
+already points `GRAFT_PAGE_DIR` at `/var/lib/graft/pages` (500 pages of ~50 kB,
+so ~25 MB is the ceiling); mount anything durable there and the pages come back
+with the process. Set the variable yourself on hosts that are not this image, or
+leave it empty to keep the store in memory — it is deliberately not required, as
+a server that refuses to boot reviews nothing at all.
+
### 3. Install it on a repository
App settings → Install App → pick the repos. **Installing requires admin on the
diff --git a/src/app/main.ts b/src/app/main.ts
index c29998be..381bb7ac 100644
--- a/src/app/main.ts
+++ b/src/app/main.ts
@@ -40,6 +40,12 @@ const { server, queue } = createApp({
publicUrl: required("GRAFT_PUBLIC_URL"),
port,
concurrency: Number(process.env.GRAFT_CONCURRENCY ?? 2),
+ // Optional, unlike the four above, and deliberately so: point it at a mounted
+ // volume and the pages already linked from pull requests survive a restart;
+ // leave it unset and they do not. Neither is worth refusing to boot over — a
+ // server that will not start reviews nothing at all, which is strictly worse
+ // than one whose old links go stale.
+ pageDir: process.env.GRAFT_PAGE_DIR,
});
/**
diff --git a/src/app/pages.ts b/src/app/pages.ts
index 8e2eda46..7728ec08 100644
--- a/src/app/pages.ts
+++ b/src/app/pages.ts
@@ -10,13 +10,25 @@
* capability, not a login. Anyone with the link can read that one page until it
* expires, which is the same property a GitHub artifact link has, and the link
* only ever appears in a comment on the pull request it describes.
+ *
+ * Pages are mirrored to `dir`, when one is given, because the token is DERIVED
+ * from the id rather than stored: a link keeps passing the signature check for as
+ * long as the secret lives, so a process that lost its pages answers 404 to a
+ * link that looks perfectly valid. That failure is silent at both ends — the
+ * comment still sits on the pull request, the reviewer just sees "not found" —
+ * and one restart of a container with nothing mounted stranded 29 of them. Disk
+ * is the fix; without a directory this behaves exactly as it did before.
*/
-import { createHmac, timingSafeEqual } from "node:crypto";
+import { createHmac, randomBytes, timingSafeEqual } from "node:crypto";
+import { mkdirSync, readFileSync, readdirSync, renameSync, unlinkSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
/** Long enough to review a PR at leisure, short enough that a leaked link dies. */
const DEFAULT_TTL_MS = 14 * 24 * 60 * 60 * 1000;
/** A page is ~50 kB; a few hundred of them is nothing, and eviction is by age. */
const DEFAULT_MAX_PAGES = 500;
+/** One file per page, named by its id. The extension also means "fully written". */
+const SUFFIX = ".json";
export interface StoredPage {
html: string;
@@ -28,6 +40,8 @@ export interface PageStoreOptions {
ttlMs?: number;
maxPages?: number;
now?: () => number;
+ /** Kept across restarts here. Absent: pages live only as long as the process. */
+ dir?: string;
}
export class PageStore {
@@ -36,12 +50,15 @@ export class PageStore {
private readonly ttlMs: number;
private readonly maxPages: number;
private readonly now: () => number;
+ private readonly dir: string | undefined;
constructor(opts: PageStoreOptions) {
this.secret = opts.secret;
this.ttlMs = opts.ttlMs ?? DEFAULT_TTL_MS;
this.maxPages = opts.maxPages ?? DEFAULT_MAX_PAGES;
this.now = opts.now ?? Date.now;
+ this.dir = opts.dir;
+ this.hydrate();
}
/** `owner/repo#number` → a path segment safe to put in a URL. */
@@ -52,7 +69,9 @@ export class PageStore {
/** Store (replacing any previous page for the same PR) and return `id` + `token`. */
put(owner: string, repo: string, number: number, html: string): { id: string; token: string } {
const id = PageStore.idFor(owner, repo, number);
- this.pages.set(id, { html, storedMs: this.now() });
+ const page: StoredPage = { html, storedMs: this.now() };
+ this.pages.set(id, page);
+ this.write(id, page);
this.evict();
return { id, token: this.sign(id) };
}
@@ -63,12 +82,15 @@ export class PageStore {
* Expiry is checked against the STORED time rather than encoded in the token,
* so a link cannot outlive the data it points at — and re-reviewing a PR
* refreshes both together.
+ *
+ * Answered from memory even when there is a directory: `id` comes off a URL,
+ * and an id that never reaches the filesystem cannot be walked out of it.
*/
get(id: string, token: string | undefined): string | null {
const page = this.pages.get(id);
if (!page || !token || !this.valid(id, token)) return null;
if (this.now() - page.storedMs > this.ttlMs) {
- this.pages.delete(id);
+ this.remove(id);
return null;
}
return page.html;
@@ -93,11 +115,94 @@ export class PageStore {
/** Oldest out first, plus anything past its TTL. */
private evict(): void {
const cutoff = this.now() - this.ttlMs;
- for (const [id, page] of this.pages) if (page.storedMs < cutoff) this.pages.delete(id);
+ for (const [id, page] of this.pages) if (page.storedMs < cutoff) this.remove(id);
while (this.pages.size > this.maxPages) {
const oldest = this.pages.keys().next();
if (oldest.done) break;
- this.pages.delete(oldest.value);
+ this.remove(oldest.value);
+ }
+ }
+
+ /** Take back what a previous process left on disk. */
+ private hydrate(): void {
+ const dir = this.dir;
+ if (!dir) return;
+
+ let names: string[];
+ try {
+ mkdirSync(dir, { recursive: true });
+ names = readdirSync(dir);
+ } catch (err) {
+ // Nothing usable there: carry on in memory, which is exactly what every
+ // deployment without a directory does anyway.
+ console.warn(`page store: ${dir} is unusable, pages will not survive a restart (${reason(err)})`);
+ return;
+ }
+
+ const loaded: Array<[string, StoredPage]> = [];
+ for (const name of names) {
+ // The suffix also skips the temp files a write leaves behind when it dies
+ // partway, so a half-written page is never a candidate to begin with.
+ if (!name.endsWith(SUFFIX)) continue;
+ const page = this.read(join(dir, name));
+ if (page) loaded.push([name.slice(0, -SUFFIX.length), page]);
+ }
+ // Oldest first, because eviction takes the front of the Map and readdir order
+ // is the filesystem's business rather than a record of what happened when.
+ loaded.sort((a, b) => a[1].storedMs - b[1].storedMs);
+ for (const [id, page] of loaded) this.pages.set(id, page);
+ // A restart is also when pages that expired while the process was down go.
+ this.evict();
+ }
+
+ /** A stored page, or nothing: truncated, corrupt and gone are one answer here. */
+ private read(file: string): StoredPage | null {
+ try {
+ const parsed: unknown = JSON.parse(readFileSync(file, "utf8"));
+ const { html, storedMs } = (parsed ?? {}) as Partial<StoredPage>;
+ if (typeof html !== "string" || typeof storedMs !== "number" || !Number.isFinite(storedMs)) return null;
+ return { html, storedMs };
+ } catch {
+ return null;
+ }
+ }
+
+ /**
+ * Temp file, then rename: a reader only ever sees a whole page, and two reviews
+ * of the same pull request racing each other leave one of them intact rather
+ * than the two interleaved.
+ */
+ private write(id: string, page: StoredPage): void {
+ const dir = this.dir;
+ if (!dir) return;
+
+ const tmp = join(dir, `${id}.${randomBytes(6).toString("hex")}.tmp`);
+ try {
+ mkdirSync(dir, { recursive: true });
+ writeFileSync(tmp, JSON.stringify(page));
+ renameSync(tmp, join(dir, `${id}${SUFFIX}`));
+ } catch (err) {
+ // The page is in memory and its link works; it just will not outlive this
+ // process. Worth a line in the log and nothing more: throwing here would
+ // fail a review that has already done all of its work.
+ console.warn(`page store: could not write ${id} (${reason(err)})`);
+ try {
+ unlinkSync(tmp);
+ } catch {
+ // Never created, or already renamed into place.
+ }
+ }
+ }
+
+ private remove(id: string): void {
+ this.pages.delete(id);
+ if (!this.dir) return;
+ try {
+ unlinkSync(join(this.dir, `${id}${SUFFIX}`));
+ } catch {
+ // Already gone: deleted underneath us, or it never reached the disk.
}
}
}
+
+const reason = (err: unknown): string => (err instanceof Error ? err.message : String(err));
diff --git a/src/app/server.ts b/src/app/server.ts
index db487b76..dcce6a40 100644
--- a/src/app/server.ts
+++ b/src/app/server.ts
@@ -32,6 +32,8 @@ export interface AppConfig extends AppCredentials {
webhookSecret: string;
/** Public origin, for the links put in comments, e.g. https://graft.example.com */
publicUrl: string;
+ /** Where pages are kept, so a restart does not strand the links already posted. */
+ pageDir?: string;
port?: number;
concurrency?: number;
api?: string;
@@ -46,7 +48,7 @@ export function createApp(
const fetchImpl = seams.fetch ?? (globalThis.fetch as unknown as Fetch);
const review = seams.review ?? reviewPullRequest;
const tokens = new InstallationTokens(config, fetchImpl, seams.now ?? Date.now, config.api);
- const pages = new PageStore({ secret: config.webhookSecret });
+ const pages = new PageStore({ secret: config.webhookSecret, dir: config.pageDir });
const origin = config.publicUrl.replace(/\/$/, "");
const queue = new WorkQueue<ReviewJob>(
diff --git a/test/app-pages.test.ts b/test/app-pages.test.ts
new file mode 100644
index 00000000..32497d84
--- /dev/null
+++ b/test/app-pages.test.ts
@@ -0,0 +1,200 @@
+/**
+ * Pages that outlive the process holding them.
+ *
+ * The link in a pull request comment carries a token derived from the page id, so
+ * it verifies forever and a restart turns it into a silent 404 — which is how 29
+ * comments on one repository ended up pointing at nothing. What is asserted here
+ * is therefore the restart itself: a second `PageStore` over the same directory
+ * has to answer the links the first one handed out, on the same terms (token,
+ * TTL, cap), and none of the ways a filesystem can misbehave may reach the
+ * caller as an exception.
+ */
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { mkdtempSync, readdirSync, readFileSync, rmSync, unlinkSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { PageStore } from "../src/app/pages.js";
+
+const secret = "webhook-secret";
+
+/** Each test gets its own directory, and the store's is a level below it. */
+function scratch(): { root: string; dir: string; cleanup: () => void } {
+ const root = mkdtempSync(join(tmpdir(), "graft-pages-"));
+ // Nested and absent: the App's volume is mounted empty, so the store has to
+ // make its own directory rather than expect one.
+ return { root, dir: join(root, "pages"), cleanup: () => rmSync(root, { recursive: true, force: true }) };
+}
+
+test("pages: a restart serves the links the previous process handed out", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ const first = new PageStore({ secret, dir });
+ const { id, token } = first.put("NanoNets", "assign", 42, "<h1>radius</h1>");
+
+ const restarted = new PageStore({ secret, dir });
+ assert.equal(restarted.size, 1, "the page was on disk, not only in the process that made it");
+ assert.equal(restarted.get(id, token), "<h1>radius</h1>", "the comment's link still works");
+ // The token is the only credential and it is derived, so the restart must not
+ // have widened access either.
+ assert.equal(restarted.get(id, "nope"), null);
+ assert.equal(restarted.get(id, undefined), null);
+ assert.equal(restarted.get("NanoNets__assign__43", token), null);
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: TTL is still counted from when the page was stored", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ let now = 1_000;
+ const first = new PageStore({ secret, dir, ttlMs: 100, now: () => now });
+ const { id, token } = first.put("o", "r", 1, "<html>");
+
+ now += 50;
+ assert.equal(new PageStore({ secret, dir, ttlMs: 100, now: () => now }).get(id, token), "<html>");
+
+ now += 51;
+ const late = new PageStore({ secret, dir, ttlMs: 100, now: () => now });
+ assert.equal(late.get(id, token), null, "expired while the process was down, not on a fresh clock");
+ assert.equal(late.size, 0);
+ assert.deepEqual(readdirSync(dir), [], "and a restart is when an expired page finally leaves the disk");
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: expiry during a run takes the file with it", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ let now = 1_000;
+ const store = new PageStore({ secret, dir, ttlMs: 100, now: () => now });
+ const { id, token } = store.put("o", "r", 1, "<html>");
+ now += 101;
+
+ assert.equal(store.get(id, token), null);
+ assert.equal(store.size, 0);
+ assert.deepEqual(readdirSync(dir), [], "otherwise the next restart would resurrect it");
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: maxPages bounds the directory too, oldest out first", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ let now = 1_000;
+ const store = new PageStore({ secret, dir, maxPages: 2, now: () => now++ });
+ const first = store.put("o", "r", 1, "<one>");
+ store.put("o", "r", 2, "<two>");
+ store.put("o", "r", 3, "<three>");
+
+ assert.equal(store.size, 2);
+ assert.deepEqual(readdirSync(dir).sort(), ["o__r__2.json", "o__r__3.json"]);
+ assert.equal(store.get(first.id, first.token), null);
+
+ const restarted = new PageStore({ secret, dir, maxPages: 2, now: () => now });
+ assert.equal(restarted.size, 2, "a restart cannot re-import what eviction already dropped");
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: a re-review replaces the page rather than accumulating files", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ let now = 1_000;
+ const store = new PageStore({ secret, dir, now: () => now });
+ store.put("o", "r", 7, "<first pass>");
+ now += 5_000;
+ const { id, token } = store.put("o", "r", 7, "<second pass>");
+
+ assert.deepEqual(readdirSync(dir), ["o__r__7.json"], "one file per PR, and no temp files left over");
+ const restarted = new PageStore({ secret, dir, now: () => now });
+ assert.equal(restarted.get(id, token), "<second pass>");
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: an unreadable file is a missing page, not a thrown request", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ // Made by the store so the directory exists, then vandalised the three ways
+ // a real one goes wrong: truncated JSON, valid JSON of the wrong shape, and
+ // bytes that are not JSON at all.
+ const store = new PageStore({ secret, dir });
+ const { id, token } = store.put("o", "r", 1, "<html>");
+ writeFileSync(join(dir, "o__r__1.json"), '{"html":"<htm');
+ writeFileSync(join(dir, "o__r__2.json"), '{"html":42}');
+ writeFileSync(join(dir, "o__r__3.json"), "not json");
+ writeFileSync(join(dir, "o__r__4.json.tmp"), JSON.stringify({ html: "<half>", storedMs: 1 }));
+
+ const restarted = new PageStore({ secret, dir });
+ assert.equal(restarted.size, 0, "nothing there was a page, including the interrupted write");
+ assert.equal(restarted.get(id, token), null, "the caller sees a 404's worth of nothing");
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: a directory it cannot use degrades to memory instead of failing reviews", () => {
+ const { root, cleanup } = scratch();
+ // A file where the directory should be — a bad mount, in one line.
+ const wrong = join(root, "not-a-directory");
+ writeFileSync(wrong, "");
+ const warned: string[] = [];
+ const real = console.warn;
+ console.warn = (...args: unknown[]) => void warned.push(args.map(String).join(" "));
+ try {
+ const store = new PageStore({ secret, dir: wrong });
+ const { id, token } = store.put("o", "r", 9, "<served anyway>");
+
+ assert.equal(store.get(id, token), "<served anyway>", "the review still published a working link");
+ assert.equal(store.size, 1);
+ // Loud enough to find in the logs, because the operator has lost durability
+ // and nothing else in the system will say so.
+ assert.match(warned.join("\n"), /will not survive a restart/);
+ assert.match(warned.join("\n"), /could not write o__r__9/);
+ } finally {
+ console.warn = real;
+ cleanup();
+ }
+});
+
+test("pages: a file deleted underneath the process does not break the live page", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ const store = new PageStore({ secret, dir });
+ const { id, token } = store.put("o", "r", 1, "<html>");
+ unlinkSync(join(dir, "o__r__1.json"));
+
+ assert.equal(store.get(id, token), "<html>", "the page is in memory; the disk is a mirror of it");
+ // Storing it again has to put the file back, and dropping it again has to
+ // survive the file being gone a second time.
+ store.put("o", "r", 1, "<html>");
+ assert.equal(JSON.parse(readFileSync(join(dir, "o__r__1.json"), "utf8")).html, "<html>");
+ unlinkSync(join(dir, "o__r__1.json"));
+ assert.equal(new PageStore({ secret, dir }).size, 0);
+ } finally {
+ cleanup();
+ }
+});
+
+test("pages: no directory means the old in-memory behaviour, untouched", () => {
+ const { dir, cleanup } = scratch();
+ try {
+ const store = new PageStore({ secret });
+ const { id, token } = store.put("o", "r", 1, "<html>");
+
+ assert.equal(store.get(id, token), "<html>");
+ assert.equal(store.size, 1);
+ assert.equal(new PageStore({ secret }).get(id, token), null, "a restart loses them, as it always did");
+ // Nothing was created anywhere: the directory in this test was never given
+ // to a store, and the store must not have invented one.
+ assert.throws(() => readdirSync(dir));
+ } finally {
+ cleanup();
+ }
+});