Skip to content

Build your personal knowledge base with Trilium Notes

perf(server): load undici only when a request is actually made

Elian Doranccf66b08b404
diff --git a/.claude/skills/analyzing-backend-bundle/SKILL.md b/.claude/skills/analyzing-backend-bundle/SKILL.md
index 5d065e0fb5..56d38ffff3 100644
--- a/.claude/skills/analyzing-backend-bundle/SKILL.md
+++ b/.claude/skills/analyzing-backend-bundle/SKILL.md
@@ -48,6 +48,9 @@ node .claude/skills/analyzing-backend-bundle/analyze-bundle.mjs why \
# 4. Whole-bundle composition, no boot needed:
node .claude/skills/analyzing-backend-bundle/analyze-bundle.mjs packages --meta <meta.json>
+# 5. After adding any lazy seam — catches the CJS interop break described below:
+node .claude/skills/analyzing-backend-bundle/check-dynamic-imports.mjs apps/server/dist
+
# Memory numbers (RSS/heap post-GC; add --snapshot for a heap snapshot):
env $ENV node --expose-gc .claude/skills/analyzing-backend-bundle/profile-bundle.cjs \
apps/server/dist/main.mjs --port 8123 --snapshot /tmp/s.heapsnapshot
@@ -96,9 +99,33 @@ restoring. (Making buildBackend write per-entry metafiles is the real fix if thi
| Consumed synchronously (script API, sucrase transpile) | Per-call import is impossible — preload conditionally at startup behind the feature's flag (backend scripting is off by default) |
| Pulled by a dep's edge that never runs meaningfully | esbuild `alias`/stub for that one file |
-After any fix: re-run `record` + `startup` and compare the eager total, and check no chunk
-in the new startup set carries characters above U+00FF (`grep -lP '[^\x00-\xFF]'` over the
-loaded chunks; `heap-strings.mjs` shows a ~2x source string when one slips through).
+After any fix: re-run `record` + `startup` and compare the eager total, run
+`check-dynamic-imports.mjs` (see below), and check no chunk in the new startup set carries
+characters above U+00FF (`grep -lP '[^\x00-\xFF]'` over the loaded chunks; `heap-strings.mjs`
+shows a ~2x source string when one slips through).
+
+## The CommonJS interop trap — verify every new seam
+
+**`const { x } = await import("some-cjs-package")` is silently broken in the split ESM
+build.** esbuild cannot know a CommonJS module's named exports, so it emits the chunk with a
+single `default` export; destructuring names off the namespace yields `undefined`, and the
+first call fails with something like "l is not a constructor". Unit tests do not catch it,
+because a `vi.mock("some-cjs-package", …)` supplies whatever names the test asks for.
+
+Write the seam through the interop instead, and give the test mock a matching `default` (the
+real module has one — that is the shape Node's own CJS interop produces):
+
+```ts
+const mod = await import("undici");
+const { Agent, fetch } = mod.default ?? mod; // works in ESM chunks, CJS output, and Node
+```
+
+Seams whose target is **own source or a real ESM package** (unpdf, the agent SDK, the MCP
+SDK, core's own modules) are unaffected — they have genuine named exports. Only CJS
+packages bite. `check-dynamic-imports.mjs` compares, for every dynamic import in the built
+bundle, the names the consumer destructures against the names the target chunk exports; run
+it on a clean build after adding a seam, and confirm the seam works for real (import the
+emitted chunk in a scratch script and call into it) rather than trusting the unit tests.
## Gotchas
diff --git a/.claude/skills/analyzing-backend-bundle/check-dynamic-imports.mjs b/.claude/skills/analyzing-backend-bundle/check-dynamic-imports.mjs
new file mode 100644
index 0000000000..2c2b376eb1
--- /dev/null
+++ b/.claude/skills/analyzing-backend-bundle/check-dynamic-imports.mjs
@@ -0,0 +1,67 @@
+// Catches the CommonJS interop break in a split ESM build: a dynamic import of
+// a CJS package yields a chunk that can only export `default`, so destructuring
+// named exports from it silently gives `undefined` at runtime — a failure unit
+// tests miss whenever they mock the package.
+//
+// node check-dynamic-imports.mjs <dist-dir>
+//
+// Exit code 1 if any consumer destructures a name its target chunk lacks.
+// Run it on a CLEAN build: leftover chunks from an earlier build are still
+// scanned and report stale findings.
+import fs from "node:fs";
+import path from "node:path";
+
+const dist = process.argv[2];
+if (!dist) {
+ console.error("usage: node check-dynamic-imports.mjs <dist-dir>");
+ process.exit(1);
+}
+const chunkDir = path.join(dist, "chunks");
+const files = [path.join(dist, "main.mjs"),
+ ...fs.readdirSync(chunkDir).map(f => path.join(chunkDir, f))];
+
+function exportedNames(file) {
+ const src = fs.readFileSync(file, "utf8");
+ const names = new Set();
+ for (const m of src.matchAll(/export\s*\{([^}]*)\}/g)) {
+ for (const part of m[1].split(",")) {
+ const bits = part.trim().split(/\s+as\s+/);
+ const name = (bits[1] ?? bits[0]).trim();
+ if (name) names.add(name);
+ }
+ }
+ if (/export\s+default\s/.test(src)) names.add("default");
+ return names;
+}
+
+const exportCache = new Map();
+let checked = 0, broken = 0;
+for (const file of files) {
+ const src = fs.readFileSync(file, "utf8");
+ // No declarator anchor: minified output continues a `let` with commas, so
+ // the destructure often has no keyword directly before it.
+ for (const m of src.matchAll(/\{([^}]*)\}\s*=\s*await import\("\.\/([^"]+\.mjs)"\)/g)) {
+ // `[^}]*` starts after the previous `}`, so the capture can include
+ // preceding code; the destructure is whatever follows the last `{`.
+ const inner = m[1].slice(m[1].lastIndexOf("{") + 1);
+ const wanted = inner.split(",").map(s => s.split(":")[0].trim())
+ .filter(n => /^[A-Za-z_$][\w$]*$/.test(n));
+ const target = path.join(chunkDir, path.basename(m[2]));
+ if (!exportCache.has(target)) exportCache.set(target, exportedNames(target));
+ const has = exportCache.get(target);
+ const missing = wanted.filter(n => !has.has(n));
+ checked++;
+ if (missing.length) {
+ broken++;
+ console.log(`BROKEN ${path.basename(file)} -> ${path.basename(m[2])} wants=[${wanted}] exports=[${[...has].slice(0, 5)}]`);
+ }
+ }
+}
+console.log(`\n${checked} destructuring dynamic imports checked, ${broken} broken`);
+if (broken) {
+ console.log("Fix: read the module object through its CJS interop, e.g.\n" +
+ " const mod = await import(\"pkg\");\n" +
+ " const { thing } = mod.default ?? mod;\n" +
+ "and give the package's test mock a matching `default` key.");
+ process.exit(1);
+}
diff --git a/apps/server/src/services/safe_fetch.spec.ts b/apps/server/src/services/safe_fetch.spec.ts
index ba51b0ba23..a0de87b489 100644
--- a/apps/server/src/services/safe_fetch.spec.ts
+++ b/apps/server/src/services/safe_fetch.spec.ts
@@ -29,8 +29,14 @@ const { agentInstances, MockAgent, undiciFetch } = vi.hoisted(() => {
});
// safeFetch calls undici's own `fetch` (not the global one) so that it and the `Agent` it passes as
-// a dispatcher come from the same undici copy — see the comment in safe_fetch.ts.
-vi.mock("undici", () => ({ Agent: MockAgent, fetch: undiciFetch }));
+// a dispatcher come from the same undici copy — see the comment in safe_fetch.ts. undici is
+// CommonJS, so the mock carries the names under `default` as well: that is the only shape a bundled
+// ESM build can expose, and safe_fetch reads it first.
+vi.mock("undici", () => ({
+ default: { Agent: MockAgent, fetch: undiciFetch },
+ Agent: MockAgent,
+ fetch: undiciFetch
+}));
import { safeFetch, validateHostResolution, validateUrl } from "./safe_fetch.js";
diff --git a/apps/server/src/services/safe_fetch.ts b/apps/server/src/services/safe_fetch.ts
index 5f8f15785a..47379545bd 100644
--- a/apps/server/src/services/safe_fetch.ts
+++ b/apps/server/src/services/safe_fetch.ts
@@ -4,7 +4,7 @@ import net from "node:net";
import { ValidationError } from "@triliumnext/core";
import { validateFetchableUrl } from "@triliumnext/core/src/services/request.js";
import ipaddr from "ipaddr.js";
-import { Agent, fetch as undiciFetch, type RequestInit as UndiciRequestInit, type Response as UndiciResponse } from "undici";
+import type { Agent, RequestInit as UndiciRequestInit, Response as UndiciResponse } from "undici";
const FETCH_TIMEOUT_MS = 5000;
const MAX_REDIRECTS = 5;
@@ -213,13 +213,24 @@ async function safeFetch(url: string, options: RequestInit = {}, policy: SafeFet
const { allowPrivateNetwork = false, timeoutMs = FETCH_TIMEOUT_MS, maxRedirects = MAX_REDIRECTS } = policy;
let currentUrl = url;
+ // Imported here rather than at module scope so undici lands in a lazy chunk:
+ // request.ts imports this module during startup, while an outbound request
+ // happens only once a feature that makes one runs.
+ //
+ // undici is CommonJS, so a bundled ESM build can only expose it as `default`
+ // — destructuring the namespace directly yields undefined for every name.
+ // The fallback covers the shapes that do carry the names: CJS output's
+ // interop wrapper, Node's own, and the spec's module mock.
+ const undici = await import("undici");
+ const { Agent: UndiciAgent, fetch: undiciFetch } = undici.default ?? undici;
+
for (let i = 0; i <= maxRedirects; i++) {
const parsed = validateUrl(currentUrl);
const validatedAddresses = await validateHostResolution(parsed.hostname, allowPrivateNetwork);
// Use a custom dispatcher that pins DNS to the validated IPs,
// preventing a second DNS lookup from resolving to a different (private) IP.
- const dispatcher = new Agent({
+ const dispatcher = new UndiciAgent({
connect: {
lookup: createPinnedLookup(validatedAddresses) as never
}