Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(gate): judge every fetch under api/, not only the ones with a hardcoded host

nirholasc6f0e528bbd6
diff --git a/docs/shared-utilities.md b/docs/shared-utilities.md
index b353ef94e..ffc24549f 100644
--- a/docs/shared-utilities.md
+++ b/docs/shared-utilities.md
@@ -321,20 +321,32 @@ npm run check:fetch-timeouts # wired into `npm run gate`
node scripts/check-fetch-timeouts.mjs --json
```
-Fails the gate on any `fetch()` to a literal external host that carries no
-deadline. A fetch with no signal has no deadline at all: undici's defaults run
-to minutes, so one upstream that accepts a connection and then stalls holds the
-request until the platform kills the invocation, and takes the request budget of
-every fallback behind it with it. In production that does not read as "an
-upstream was slow", it reads as a dead endpoint, a dead cron, or a page that
-spins forever.
+Fails the gate on a `fetch()` that carries no deadline. A fetch with no signal
+has no deadline at all: undici's defaults run to minutes, so one upstream that
+accepts a connection and then stalls holds the request until the platform kills
+the invocation, and takes the request budget of every fallback behind it with
+it. In production that does not read as "an upstream was slow", it reads as a
+dead endpoint, a dead cron, or a page that spins forever.
A call counts as bounded when it passes a `signal` or goes through a wrapper
-that supplies one (`fetchUpstream`, `fetchFirst`, `pumpFetchJson`, ...). Only
-the deadline is enforced, deliberately: retries, provider ladders and last-good
-tiers are the right thing to add on top, but which one fits is a judgement call
-per call site, while a deadline never is. Same-origin and relative calls are out
-of scope.
+that supplies one (`fetchUpstream`, `fetchFirst`, `pumpFetchJson`, ...). An init
+object assembled earlier in the function counts too: the checker resolves
+`fetch(url, init)` back to where `init` was built. Only the deadline is
+enforced, deliberately: retries, provider ladders and last-good tiers are the
+right thing to add on top, but which one fits is a judgement call per call site,
+while a deadline never is.
+
+**Scope differs by directory.** In browser code (`src`, `public`, `workers`) the
+rule covers literal external hosts only, since a page's fetch of our own API is
+bounded by the page lifecycle and by `src/api.js`. Under `api/` it covers EVERY
+call, whatever the URL is, because the two shapes that hang in practice are
+invisible to a literal-host scan: a handler calling its own `/api/forge` (a real
+socket between two serverless instances, not a free local call) and a download
+of a provider result URL that only exists at runtime. Three exemptions there,
+each for a call that cannot hang or is already bounded elsewhere: a `data:` URL
+(no socket), a client snippet quoted as a string (it runs in a browser), and a
+wrapper that spreads its caller's `init` (the deadline is the caller's, and a
+second one would silently shorten it).
The checker reads each call's real extent by balancing parentheses rather than
scanning a fixed window, because `signal` is conventionally the last key of an
diff --git a/scripts/check-fetch-timeouts.mjs b/scripts/check-fetch-timeouts.mjs
index a7b064581..7d41936f1 100644
--- a/scripts/check-fetch-timeouts.mjs
+++ b/scripts/check-fetch-timeouts.mjs
@@ -18,9 +18,16 @@
* they are judgement calls per call site; a deadline is not, so only the
* deadline is enforced here.
*
- * Same-origin calls (three.ws, localhost, relative paths) are out of scope: a
- * browser call to our own API is bounded by the page's own lifecycle and by
- * src/api.js, and a server call to itself is a different kind of bug.
+ * In browser code (src, public, workers) that rule is limited to literal
+ * external hosts: a page's own fetch of our API is bounded by the page
+ * lifecycle and by src/api.js.
+ *
+ * Under api/ the rule covers EVERY call, whatever the URL is. Two of the three
+ * hangs this repo has actually paid for were invisible to a literal-host scan:
+ * a handler calling its own /api/forge (a real socket between two serverless
+ * instances, not a free local call) and a download of a provider result URL
+ * that only exists at runtime. A checker that only sees hardcoded hosts stops
+ * exactly where the risk starts.
*
* node scripts/check-fetch-timeouts.mjs # report and exit non-zero on a violation
* node scripts/check-fetch-timeouts.mjs --json # machine-readable
@@ -47,6 +54,16 @@ const SKIP_PATH = /\/vendor\/|node_modules|public\/chat\/assets\/|\.min\.js$|\/d
const OWN_HOST = /three\.ws|localhost|127\.0\.0\.1/;
+// A `data:` URL is decoded in memory: there is no socket, so there is nothing to
+// time out.
+const DATA_URL_RE = /fetch\(\s*[`'"]data:/;
+
+// A wrapper whose init object is spread in from its own caller inherits that
+// caller's deadline, and imposing a second one here would silently shorten it.
+// This is the shape of the pass-through wrappers (ssrf-guard, the Vertex fetch
+// override the AI SDK calls with its own abortSignal, the Solana RPC sender).
+const DELEGATED_INIT_RE = /\{\s*\.\.\.\s*(init|opts|options|requestInit)\b/;
+
function listFiles() {
const cmd = `grep -rl 'fetch(' ${ROOTS.join(' ')} --include=*.js --include=*.mjs 2>/dev/null || true`;
return execSync(cmd, { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 })
@@ -90,7 +107,33 @@ function callExtent(lines, start) {
return `${head}\n${out.join('\n')}`;
}
+// `fetch(url, init)` where the options were assembled further up the function is
+// a normal shape, and reading only the call itself reports it as unbounded. Find
+// where that identifier was built and judge THAT text: an init object carrying a
+// signal bounds the call wherever it was written.
+function initObjectBounded(lines, name) {
+ if (!/^[A-Za-z_$][\w$]*$/.test(name)) return false;
+ const decl = new RegExp(`(?:const|let|var)\\s+${name}\\s*=|\\b${name}\\.signal\\s*=|\\b${name}\\s*=\\s*\\{`);
+ for (let i = 0; i < lines.length; i++) {
+ if (!decl.test(lines[i])) continue;
+ // The declaration plus the statement that follows it: an options object is
+ // often filled in over the next few lines (`init.body = ...`).
+ const stmt = lines.slice(i, Math.min(lines.length, i + 20)).join('\n');
+ if (SIGNAL_RE.test(stmt)) return true;
+ }
+ return false;
+}
+
+// Lines that are quoted source code rather than executable code: some handlers
+// build a client-side snippet as a string array to hand to an agent, and the
+// fetch inside it runs in the reader's browser, not here.
+function isQuotedSource(line) {
+ return /^\s*[`'"]/.test(line);
+}
+
export function scanFile(file, src) {
+ // Server handlers and helpers: every call is in scope, same-origin included.
+ const serverSide = /^api\//.test(file);
const lines = src.split('\n');
const consts = externalConsts(lines);
const out = [];
@@ -105,7 +148,8 @@ export function scanFile(file, src) {
const urlLiteral = call.match(/[`'"]https?:\/\/[^`'"]*/)?.[0] || '';
const inlineExternal = !!urlLiteral && !OWN_HOST.test(urlLiteral);
const constExternal = [...consts].some((c) => new RegExp(`\\b${c}\\b`).test(call));
- if (!inlineExternal && !constExternal) continue;
+ if (!inlineExternal && !constExternal && !serverSide) continue;
+ if (serverSide && (DATA_URL_RE.test(call) || isQuotedSource(line))) continue;
// Read the ACTUAL extent of the call by balancing parentheses from `fetch(`
// rather than guessing a window: an options object can run for thirty
@@ -113,6 +157,9 @@ export function scanFile(file, src) {
// reports a bounded call as unbounded exactly when the options are long.
const win = callExtent(lines, i);
if (BOUNDED_RE.test(win) || SIGNAL_RE.test(win)) continue;
+ if (serverSide && DELEGATED_INIT_RE.test(win)) continue;
+ const initIdent = /fetch\([^,()]+,\s*([A-Za-z_$][\w$]*)\s*\)/.exec(win)?.[1];
+ if (initIdent && initObjectBounded(lines, initIdent)) continue;
out.push({ file, line: i + 1, code: line.trim().slice(0, 100) });
}
return out;
@@ -133,11 +180,11 @@ function main() {
}
if (!violations.length) {
- console.log('check:fetch-timeouts: every external fetch is bounded by a deadline');
+ console.log('check:fetch-timeouts: every external fetch, and every fetch under api/, is bounded by a deadline');
process.exit(0);
}
- console.error(`check:fetch-timeouts: ${violations.length} external fetch(es) with no deadline\n`);
+ console.error(`check:fetch-timeouts: ${violations.length} fetch(es) with no deadline\n`);
for (const v of violations) {
console.error(` ${v.file}:${v.line}`);
console.error(` ${v.code}`);
diff --git a/tests/check-fetch-timeouts.test.js b/tests/check-fetch-timeouts.test.js
index 6b547ef1c..4e9223f1b 100644
--- a/tests/check-fetch-timeouts.test.js
+++ b/tests/check-fetch-timeouts.test.js
@@ -62,3 +62,55 @@ describe('check-fetch-timeouts', () => {
expect(out[0].code).toContain('api.example.com');
});
});
+
+// Under api/ the rule is wider: a handler calling its own origin opens a real
+// socket between two instances, and the URLs that hang most often (a provider's
+// delivery URL, a worker artifact) only exist at runtime, so neither is visible
+// to a scan for literal hosts.
+describe('check-fetch-timeouts under api/', () => {
+ const scanApi = (src) => scanFile('api/thing.js', src);
+
+ it('flags a same-origin self-call with no deadline', () => {
+ const out = scanApi(`const r = await fetch(\`\${origin}/api/forge\`, { method: 'POST' });`);
+ expect(out).toHaveLength(1);
+ });
+
+ it('flags a fetch of a URL only known at runtime', () => {
+ expect(scanApi(`const resp = await fetch(sourceUrl);`)).toHaveLength(1);
+ });
+
+ it('accepts the same call once it is bounded', () => {
+ expect(scanApi(`const resp = await fetch(sourceUrl, { signal: AbortSignal.timeout(9000) });`)).toEqual([]);
+ expect(scanApi(`const resp = await fetchUpstream(sourceUrl, {}, { timeoutMs: 9000 });`)).toEqual([]);
+ });
+
+ it('accepts an init object assembled further up the function', () => {
+ const src = [
+ `const init = { method, headers, signal: AbortSignal.timeout(timeoutMs) };`,
+ `if (body != null) init.body = JSON.stringify(body);`,
+ `const a = 1;`,
+ `const b = 2;`,
+ `const c = 3;`,
+ `const d = 4;`,
+ `const e = 5;`,
+ `const upstream = await fetch(url, init);`,
+ ].join('\n');
+ expect(scanApi(src)).toEqual([]);
+ });
+
+ it('accepts a wrapper that passes its caller\'s init straight through', () => {
+ // The deadline belongs to the caller; imposing a second one here would
+ // silently shorten it.
+ expect(scanApi(`const res = await fetch(url.toString(), { ...init, redirect: 'manual' });`)).toEqual([]);
+ });
+
+ it('ignores a data: URL and a client snippet quoted as a string', () => {
+ expect(scanApi(`const buf = await (await fetch('data:application/octet-stream;base64,' + b64)).arrayBuffer();`)).toEqual([]);
+ expect(scanApi(`\t\t"const _r = await fetch('/api/mcp', {",`)).toEqual([]);
+ });
+
+ it('leaves browser code judged on external hosts only', () => {
+ expect(scanFile('src/page.js', `await fetch('/api/thing');`)).toEqual([]);
+ expect(scanFile('src/page.js', `await fetch(someUrl);`)).toEqual([]);
+ });
+});