Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
fix(cdn): stream the r2.dev fallback instead of redirecting to it
nirholas2d29ab72b274
diff --git a/ISSUES.md b/ISSUES.mdindex 0826ff3d3..2dc91696d 100644--- a/ISSUES.md+++ b/ISSUES.md@@ -183,20 +183,21 @@ here; the code-quality items from that pass are not production issues.9. **Live R2 CORS does not match `scripts/set-r2-cors.mjs`** (owner action:one credential). CONFIRMED by measurement, not inference, and RE-CONFIRMED- unchanged on 2026-09-04: `node scripts/set-r2-cors.mjs --probe` reads the+ unchanged on 2026-09-09: `node scripts/set-r2-cors.mjs --probe` reads theenforced policy from outside and exits 1 on this bucket. The probe needs nocredentials of any kind (it discovers the public host from a live listing- endpoint and the upload host from the auth-free `/api/forge-upload`), so- anyone can re-check this in one command.+ endpoint, and the upload host from an auth-free presign route or from an+ explicit `--endpoint=`/`--bucket=`, both non-secret), so anyone can re-check+ this in one command.- Measured again 2026-09-04, every row from raw `curl` as well as the probe:+ Measured again 2026-09-09, every row from raw `curl` as well as the probe:| Surface | Result ||---|---|| Site edge, `three.ws/avatars/*.glb`, foreign origin | PASS, `access-control-allow-origin: *` with `access-control-allow-methods: GET, HEAD, OPTIONS`. Not affected. || Site edge, first-party `three.ws/cdn/<key>`, foreign origin | PASS, `access-control-allow-origin: *`. Serves the same bucket objects, so it is a working route around the row below for any caller that has the object key. |- | Public bucket host `pub-*.r2.dev`, foreign origin GET/HEAD | FAIL. Body returns `200`, but with no `access-control-allow-origin`, so the browser discards it. Allowlisted origins DO get their origin echoed (`Vary: Origin`), which is how the live read rule is known to still be the old allowlist rather than the world-open `*`. Its `OPTIONS` preflight is a bare `403`. |- | Presigned `PUT` preflight on the S3 endpoint | Mixed, unchanged. `204` for `three.ws`, `*.vercel.app` (wildcard confirmed with a synthetic subdomain), `localhost:3000`; `403` for `www.three.ws`, `*.app.github.dev`, `localhost:5173`. |+ | Public bucket host `pub-*.r2.dev`, foreign origin GET/HEAD | FAIL. On a real `200` object, `https://example.org` gets the body with no `access-control-allow-origin`, so the browser discards it. Allowlisted origins DO get their origin echoed (`Vary: Origin`), which is how the live read rule is known to still be the old allowlist rather than the world-open `*`. Its `OPTIONS` preflight is a bare `403`. |+ | Presigned `PUT` preflight on the S3 endpoint | Mixed, unchanged. `204` for `three.ws`, `*.vercel.app` (wildcard confirmed with a synthetic subdomain), `localhost:3000`; `403` for `www.three.ws`, `*.app.github.dev`, `localhost:5173`, `example.org`. |The live policy is one allowlist rule serving both reads and writes(`GET, PUT, HEAD, POST, DELETE`), predating the script's split into a@@ -210,14 +211,18 @@ here; the code-quality items from that pass are not production issues.applies where (docs/media-api.md, docs/character-library.md, both embedtutorials).- Fixed on the way past, 2026-09-04: the site edge advertised- `access-control-allow-methods: GET, HEAD, OPTIONS` on the media routes but- answered `OPTIONS` with the 404 page, because the filesystem phase of- [server/index.mjs](server/index.mjs) serves GET/HEAD only. A simple- cross-origin GET was unaffected (it sends no preflight), but any fetch- carrying a non-safelisted header was blocked. Those routes now answer the- preflight `204` from the headers the route already collected- (tests/server-media-cors-preflight.test.js). Ships with the next deploy.+ Fixed on the way past, 2026-09-04, and now VERIFIED LIVE (2026-09-09): the+ site edge advertised `access-control-allow-methods: GET, HEAD, OPTIONS` on+ the media routes but answered `OPTIONS` with the 404 page, because the+ filesystem phase of [server/index.mjs](server/index.mjs) serves GET/HEAD+ only. A simple cross-origin GET was unaffected (it sends no preflight), but+ any fetch carrying a non-safelisted header was blocked. Those routes now+ answer the preflight `204` from the headers the route already collected+ (tests/server-media-cors-preflight.test.js). Confirmed on the live site:+ `curl -I -X OPTIONS -H 'Origin: https://example.org'+ -H 'Access-Control-Request-Method: GET' -H 'Access-Control-Request-Headers: range'+ https://three.ws/avatars/cesium-man.glb` returns `204` with+ `access-control-allow-origin: *` and `access-control-allow-headers: range`.Blocked on one credential, not on code. The only R2 token reachable fromthis machine (`S3_*` in `.env`, identical to the Cloud Run service env,@@ -235,6 +240,36 @@ here; the code-quality items from that pass are not production issues.of crashing), drop it in `.env.local` as `R2_ACCESS_KEY_ID`/`R2_SECRET_ACCESS_KEY`,then `node scripts/set-r2-cors.mjs` and confirm with `--probe`.+10. **Object storage is DOWN in production: the bucket rejects our credential**+ (owner action: one env var). Found 2026-09-09 while measuring item 9.+ `/api/healthz` reports the `object_storage` subsystem `down` with+ `credentialFault: true` and the bucket's own words: `The request signature+ we calculated does not match the signature you provided.` The access key id+ is accepted, so the SECRET is what is wrong, not the code and not the key.++ Live symptoms, all measured 2026-09-09:++ | Surface | Result |+ |---|---|+ | `POST /api/forge-upload` | `503 storage_unavailable`, so no reference image can be parked and no upload can land. |+ | `GET /api/avatars/library`, `GET /api/objects/library` | `{"total":0}`, empty manifests where the catalog should be. |+ | `forge_generation` subsystem | `down`, 0/30 finished over 6 hours, every failure on the image path. |++ No lane failover routes around it: `/cdn/<key>` falls back to the+ rate-limited public bucket domain, and the forge cannot stage inputs at all.+ On Cloudflare R2 the secret access key is the API token's SHA-256 digest,+ not the token value, and a trailing newline fails identically (`env.js`+ trims, so a padded value can only come from a store read elsewhere).++ `S3_SECRET_ACCESS_KEY` on `three-ws-api` is a Secret Manager reference+ (`secret:s3-secret-access-key:latest`), and reading or rotating it is+ owner-gated from this machine. Owner: re-set that secret to the token's+ SHA-256 digest, then confirm with+ `curl -s https://three.ws/api/healthz | grep -o '"name":"object_storage"[^}]*'`.+ Nothing else needs redeploying. Note this must be fixed BEFORE item 9's+ admin-token step is worth doing: a correct CORS policy on a bucket we+ cannot authenticate to changes nothing users can see.+---## Recently closed (do not re-open without new evidence)diff --git a/scripts/set-r2-cors.mjs b/scripts/set-r2-cors.mjsindex fcb5da5fb..6dbc21acd 100644--- a/scripts/set-r2-cors.mjs+++ b/scripts/set-r2-cors.mjs@@ -16,6 +16,8 @@* node scripts/set-r2-cors.mjs --probe # measure the LIVE policy from outside (no credentials at all)* node scripts/set-r2-cors.mjs --probe --site=https://staging.example # probe another deployment* node scripts/set-r2-cors.mjs --probe --key=thumb/x.png # read a specific object+ * node scripts/set-r2-cors.mjs --probe --endpoint=https://<account>.r2.cloudflarestorage.com --bucket=<name>+ * # measure the write rule when the presign routes are down* node scripts/set-r2-cors.mjs --get # read the live policy (needs an admin token)* node scripts/set-r2-cors.mjs --dry-run # print the policy, don't push* node scripts/set-r2-cors.mjs # apply (idempotent, needs an admin token)@@ -243,32 +245,42 @@ async function probe() {return false;}+ // A missing write host does NOT abort the probe. The `public-read` rule is+ // the one third-party embeds actually hit, it is measurable on its own, and+ // the presign routes that reveal the write host return 503 exactly when+ // object storage is unhealthy. Reporting the half we can measure beats+ // exiting with nothing measured at all.const writeUrl = await probeWriteUrl(site);if (!writeUrl) {- console.error('--probe could not determine the presigned-upload host.');- console.error(`Set S3_ENDPOINT + S3_BUCKET, or check that ${site}/api/forge-upload is reachable.`);- return false;+ console.warn('Note: the presigned-upload host could not be determined, so the');+ console.warn('`browser-upload` (write) rule is NOT measured below and shows as "skip".');+ console.warn(`Pass it explicitly to measure that rule too, no credentials needed:`);+ console.warn(' node scripts/set-r2-cors.mjs --probe --endpoint=https://<account>.r2.cloudflarestorage.com --bucket=<name>');+ console.warn('(both values are non-secret; read them with `node scripts/read-service-env.mjs \'^S3_ENDPOINT$\' --raw`)\n');}- console.log(`Probing ${Bucket || hostLabel(writeUrl)} (read: ${readUrl})\n`);+ console.log(`Probing ${Bucket || hostLabel(writeUrl || readUrl)} (read: ${readUrl})\n`);console.log(`${'ORIGIN'.padEnd(38)} ${'READ'.padEnd(6)} ${'WRITE'.padEnd(6)} EXPECTED`);let ok = true;for (const origin of PROBE_ORIGINS) {const [read, write] = await Promise.all([corsAllowed(readUrl, origin, 'GET'),- corsAllowed(writeUrl, origin, 'PUT'),+ writeUrl ? corsAllowed(writeUrl, origin, 'PUT') : Promise.resolve(null),]);const wantRead = ruleAllows('public-read', origin);const wantWrite = ruleAllows('browser-upload', origin);- const bad = read !== wantRead || write !== wantWrite;+ const bad = read !== wantRead || (write !== null && write !== wantWrite);if (bad) ok = false;const want = `read=${wantRead ? 'yes' : 'no'} write=${wantWrite ? 'yes' : 'no'}${bad ? ' <- DRIFT' : ''}`;- console.log(`${origin.padEnd(38)} ${(read ? 'yes' : 'no').padEnd(6)} ${(write ? 'yes' : 'no').padEnd(6)} ${want}`);+ const writeCell = write === null ? 'skip' : write ? 'yes' : 'no';+ console.log(`${origin.padEnd(38)} ${(read ? 'yes' : 'no').padEnd(6)} ${writeCell.padEnd(6)} ${want}`);}console.log('');- if (ok) {+ if (ok && !writeUrl) {+ console.log('Live READ policy matches POLICY in this script. The write rule was not measured.');+ } else if (ok) {console.log('Live policy matches POLICY in this script.');} else {console.log('Live policy DIFFERS from POLICY in this script. Apply it with an admin token:');@@ -326,28 +338,46 @@ async function discoverPublicAssetUrl(site, publicHost) {return null;}-// The presigned-upload host. From env when credentials are present, else from-// the site's own auth-free presign endpoint, which returns a real upload URL on-// the exact host the browser PUTs to. Only its OPTIONS preflight is ever sent,-// so no object is created.+// The presigned-upload host. Three sources, best first: an explicit+// --endpoint (the S3 endpoint and bucket name are not secrets, so anyone can+// pass them), the same values from env, and finally the site's own auth-free+// presign endpoints, which return a real upload URL on the exact host the+// browser PUTs to. Only an OPTIONS preflight is ever sent, so no object is+// created and no presigned signature is ever used.+//+// Every presign endpoint is tried in turn rather than just the first: they all+// hand back a URL on the one host we want, so any one of them answering is+// enough. That matters because a presign route returns 503 while object storage+// itself is unhealthy, and the write host is still perfectly measurable then.async function probeWriteUrl(site) {- if (process.env.S3_ENDPOINT && Bucket) {- return `${process.env.S3_ENDPOINT.replace(/\/$/, '')}/${Bucket}/${PROBE_WRITE_KEY}`;+ const endpoint = argValue('--endpoint') || process.env.S3_ENDPOINT;+ const bucket = argValue('--bucket') || Bucket;+ if (endpoint && bucket) {+ return `${endpoint.replace(/\/$/, '')}/${bucket}/${PROBE_WRITE_KEY}`;}- try {- const res = await fetch(`${site}/api/forge-upload`, {- method: 'POST',- headers: { 'content-type': 'application/json', 'x-forge-client': 'set-r2-cors-probe' },- body: JSON.stringify({ content_type: 'image/png', size_bytes: 1024 }),- });- if (!res.ok) return null;- const { upload_url: uploadUrl } = await res.json();- if (!uploadUrl) return null;- const u = new URL(uploadUrl);- return `${u.origin}${u.pathname}`;- } catch {- return null;++ const presignRoutes = [+ ['/api/forge-upload', { content_type: 'image/png', size_bytes: 1024 }],+ ['/api/avatar/presign-glb', { content_type: 'model/gltf-binary', size_bytes: 1024 }],+ ['/api/avatar/presign-audio', { content_type: 'audio/mpeg', size_bytes: 1024 }],+ ];+ for (const [path, body] of presignRoutes) {+ try {+ const res = await fetch(`${site}${path}`, {+ method: 'POST',+ headers: { 'content-type': 'application/json', 'x-forge-client': 'set-r2-cors-probe' },+ body: JSON.stringify(body),+ });+ if (!res.ok) continue;+ const { upload_url: uploadUrl } = await res.json();+ if (!uploadUrl) continue;+ const u = new URL(uploadUrl);+ return `${u.origin}${u.pathname}`;+ } catch {+ continue;+ }}+ return null;}function hostLabel(url) {