Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
feat(gate): judge every fetch under api/, not only the ones with a hardcoded host
nirholasc6f0e528bbd6
diff --git a/docs/shared-utilities.md b/docs/shared-utilities.mdindex b353ef94e..ffc24549f 100644--- a/docs/shared-utilities.md+++ b/docs/shared-utilities.md@@ -321,20 +321,32 @@ npm run check:fetch-timeouts # wired into `npm run gate`node scripts/check-fetch-timeouts.mjs --json```-Fails the gate on any `fetch()` to a literal external host that carries no-deadline. A fetch with no signal has no deadline at all: undici's defaults run-to minutes, so one upstream that accepts a connection and then stalls holds the-request until the platform kills the invocation, and takes the request budget of-every fallback behind it with it. In production that does not read as "an-upstream was slow", it reads as a dead endpoint, a dead cron, or a page that-spins forever.+Fails the gate on a `fetch()` that carries no deadline. A fetch with no signal+has no deadline at all: undici's defaults run to minutes, so one upstream that+accepts a connection and then stalls holds the request until the platform kills+the invocation, and takes the request budget of every fallback behind it with+it. In production that does not read as "an upstream was slow", it reads as a+dead endpoint, a dead cron, or a page that spins forever.A call counts as bounded when it passes a `signal` or goes through a wrapper-that supplies one (`fetchUpstream`, `fetchFirst`, `pumpFetchJson`, ...). Only-the deadline is enforced, deliberately: retries, provider ladders and last-good-tiers are the right thing to add on top, but which one fits is a judgement call-per call site, while a deadline never is. Same-origin and relative calls are out-of scope.+that supplies one (`fetchUpstream`, `fetchFirst`, `pumpFetchJson`, ...). An init+object assembled earlier in the function counts too: the checker resolves+`fetch(url, init)` back to where `init` was built. Only the deadline is+enforced, deliberately: retries, provider ladders and last-good tiers are the+right thing to add on top, but which one fits is a judgement call per call site,+while a deadline never is.++**Scope differs by directory.** In browser code (`src`, `public`, `workers`) the+rule covers literal external hosts only, since a page's fetch of our own API is+bounded by the page lifecycle and by `src/api.js`. Under `api/` it covers EVERY+call, whatever the URL is, because the two shapes that hang in practice are+invisible to a literal-host scan: a handler calling its own `/api/forge` (a real+socket between two serverless instances, not a free local call) and a download+of a provider result URL that only exists at runtime. Three exemptions there,+each for a call that cannot hang or is already bounded elsewhere: a `data:` URL+(no socket), a client snippet quoted as a string (it runs in a browser), and a+wrapper that spreads its caller's `init` (the deadline is the caller's, and a+second one would silently shorten it).The checker reads each call's real extent by balancing parentheses rather thanscanning a fixed window, because `signal` is conventionally the last key of andiff --git a/scripts/check-fetch-timeouts.mjs b/scripts/check-fetch-timeouts.mjsindex a7b064581..7d41936f1 100644--- a/scripts/check-fetch-timeouts.mjs+++ b/scripts/check-fetch-timeouts.mjs@@ -18,9 +18,16 @@* they are judgement calls per call site; a deadline is not, so only the* deadline is enforced here.*- * Same-origin calls (three.ws, localhost, relative paths) are out of scope: a- * browser call to our own API is bounded by the page's own lifecycle and by- * src/api.js, and a server call to itself is a different kind of bug.+ * In browser code (src, public, workers) that rule is limited to literal+ * external hosts: a page's own fetch of our API is bounded by the page+ * lifecycle and by src/api.js.+ *+ * Under api/ the rule covers EVERY call, whatever the URL is. Two of the three+ * hangs this repo has actually paid for were invisible to a literal-host scan:+ * a handler calling its own /api/forge (a real socket between two serverless+ * instances, not a free local call) and a download of a provider result URL+ * that only exists at runtime. A checker that only sees hardcoded hosts stops+ * exactly where the risk starts.** node scripts/check-fetch-timeouts.mjs # report and exit non-zero on a violation* node scripts/check-fetch-timeouts.mjs --json # machine-readable@@ -47,6 +54,16 @@ const SKIP_PATH = /\/vendor\/|node_modules|public\/chat\/assets\/|\.min\.js$|\/dconst OWN_HOST = /three\.ws|localhost|127\.0\.0\.1/;+// A `data:` URL is decoded in memory: there is no socket, so there is nothing to+// time out.+const DATA_URL_RE = /fetch\(\s*[`'"]data:/;++// A wrapper whose init object is spread in from its own caller inherits that+// caller's deadline, and imposing a second one here would silently shorten it.+// This is the shape of the pass-through wrappers (ssrf-guard, the Vertex fetch+// override the AI SDK calls with its own abortSignal, the Solana RPC sender).+const DELEGATED_INIT_RE = /\{\s*\.\.\.\s*(init|opts|options|requestInit)\b/;+function listFiles() {const cmd = `grep -rl 'fetch(' ${ROOTS.join(' ')} --include=*.js --include=*.mjs 2>/dev/null || true`;return execSync(cmd, { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 })@@ -90,7 +107,33 @@ function callExtent(lines, start) {return `${head}\n${out.join('\n')}`;}+// `fetch(url, init)` where the options were assembled further up the function is+// a normal shape, and reading only the call itself reports it as unbounded. Find+// where that identifier was built and judge THAT text: an init object carrying a+// signal bounds the call wherever it was written.+function initObjectBounded(lines, name) {+ if (!/^[A-Za-z_$][\w$]*$/.test(name)) return false;+ const decl = new RegExp(`(?:const|let|var)\\s+${name}\\s*=|\\b${name}\\.signal\\s*=|\\b${name}\\s*=\\s*\\{`);+ for (let i = 0; i < lines.length; i++) {+ if (!decl.test(lines[i])) continue;+ // The declaration plus the statement that follows it: an options object is+ // often filled in over the next few lines (`init.body = ...`).+ const stmt = lines.slice(i, Math.min(lines.length, i + 20)).join('\n');+ if (SIGNAL_RE.test(stmt)) return true;+ }+ return false;+}++// Lines that are quoted source code rather than executable code: some handlers+// build a client-side snippet as a string array to hand to an agent, and the+// fetch inside it runs in the reader's browser, not here.+function isQuotedSource(line) {+ return /^\s*[`'"]/.test(line);+}+export function scanFile(file, src) {+ // Server handlers and helpers: every call is in scope, same-origin included.+ const serverSide = /^api\//.test(file);const lines = src.split('\n');const consts = externalConsts(lines);const out = [];@@ -105,7 +148,8 @@ export function scanFile(file, src) {const urlLiteral = call.match(/[`'"]https?:\/\/[^`'"]*/)?.[0] || '';const inlineExternal = !!urlLiteral && !OWN_HOST.test(urlLiteral);const constExternal = [...consts].some((c) => new RegExp(`\\b${c}\\b`).test(call));- if (!inlineExternal && !constExternal) continue;+ if (!inlineExternal && !constExternal && !serverSide) continue;+ if (serverSide && (DATA_URL_RE.test(call) || isQuotedSource(line))) continue;// Read the ACTUAL extent of the call by balancing parentheses from `fetch(`// rather than guessing a window: an options object can run for thirty@@ -113,6 +157,9 @@ export function scanFile(file, src) {// reports a bounded call as unbounded exactly when the options are long.const win = callExtent(lines, i);if (BOUNDED_RE.test(win) || SIGNAL_RE.test(win)) continue;+ if (serverSide && DELEGATED_INIT_RE.test(win)) continue;+ const initIdent = /fetch\([^,()]+,\s*([A-Za-z_$][\w$]*)\s*\)/.exec(win)?.[1];+ if (initIdent && initObjectBounded(lines, initIdent)) continue;out.push({ file, line: i + 1, code: line.trim().slice(0, 100) });}return out;@@ -133,11 +180,11 @@ function main() {}if (!violations.length) {- console.log('check:fetch-timeouts: every external fetch is bounded by a deadline');+ console.log('check:fetch-timeouts: every external fetch, and every fetch under api/, is bounded by a deadline');process.exit(0);}- console.error(`check:fetch-timeouts: ${violations.length} external fetch(es) with no deadline\n`);+ console.error(`check:fetch-timeouts: ${violations.length} fetch(es) with no deadline\n`);for (const v of violations) {console.error(` ${v.file}:${v.line}`);console.error(` ${v.code}`);diff --git a/tests/check-fetch-timeouts.test.js b/tests/check-fetch-timeouts.test.jsindex 6b547ef1c..4e9223f1b 100644--- a/tests/check-fetch-timeouts.test.js+++ b/tests/check-fetch-timeouts.test.js@@ -62,3 +62,55 @@ describe('check-fetch-timeouts', () => {expect(out[0].code).toContain('api.example.com');});});++// Under api/ the rule is wider: a handler calling its own origin opens a real+// socket between two instances, and the URLs that hang most often (a provider's+// delivery URL, a worker artifact) only exist at runtime, so neither is visible+// to a scan for literal hosts.+describe('check-fetch-timeouts under api/', () => {+ const scanApi = (src) => scanFile('api/thing.js', src);++ it('flags a same-origin self-call with no deadline', () => {+ const out = scanApi(`const r = await fetch(\`\${origin}/api/forge\`, { method: 'POST' });`);+ expect(out).toHaveLength(1);+ });++ it('flags a fetch of a URL only known at runtime', () => {+ expect(scanApi(`const resp = await fetch(sourceUrl);`)).toHaveLength(1);+ });++ it('accepts the same call once it is bounded', () => {+ expect(scanApi(`const resp = await fetch(sourceUrl, { signal: AbortSignal.timeout(9000) });`)).toEqual([]);+ expect(scanApi(`const resp = await fetchUpstream(sourceUrl, {}, { timeoutMs: 9000 });`)).toEqual([]);+ });++ it('accepts an init object assembled further up the function', () => {+ const src = [+ `const init = { method, headers, signal: AbortSignal.timeout(timeoutMs) };`,+ `if (body != null) init.body = JSON.stringify(body);`,+ `const a = 1;`,+ `const b = 2;`,+ `const c = 3;`,+ `const d = 4;`,+ `const e = 5;`,+ `const upstream = await fetch(url, init);`,+ ].join('\n');+ expect(scanApi(src)).toEqual([]);+ });++ it('accepts a wrapper that passes its caller\'s init straight through', () => {+ // The deadline belongs to the caller; imposing a second one here would+ // silently shorten it.+ expect(scanApi(`const res = await fetch(url.toString(), { ...init, redirect: 'manual' });`)).toEqual([]);+ });++ it('ignores a data: URL and a client snippet quoted as a string', () => {+ expect(scanApi(`const buf = await (await fetch('data:application/octet-stream;base64,' + b64)).arrayBuffer();`)).toEqual([]);+ expect(scanApi(`\t\t"const _r = await fetch('/api/mcp', {",`)).toEqual([]);+ });++ it('leaves browser code judged on external hosts only', () => {+ expect(scanFile('src/page.js', `await fetch('/api/thing');`)).toEqual([]);+ expect(scanFile('src/page.js', `await fetch(someUrl);`)).toEqual([]);+ });+});