Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
perf(secrets): migrate plaintext secrets a few at a time instead of one gcloud call per var
nirholasb0f3be9ae7de
diff --git a/CLAUDE.md b/CLAUDE.mdindex 9217dd88b..c6f66fb54 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -31,7 +31,7 @@ Agents stopping mid-task to ask a question is a failure mode the owner explicitl| Blocker | Resolution (do this, don't ask) ||---|---|-| Missing env var / credential | Look in `.env` and `.env.local`, then the Cloud Run service (`gcloud run services describe three-ws-api --region us-central1 --project aerial-vehicle-466722-p5 --format=yaml`), then Secret Manager (`gcloud secrets list`). Never trust `vercel env pull` (returns empty for secrets). Update single vars with `--update-env-vars` (merges); `--set-env-vars` REPLACES the whole set, so never use it for one key. If a credential truly exists nowhere, build the feature fully wired behind the env var, prove it with a mock-free dry run, and list the single missing var in your report. |+| Missing env var / credential | Look in `.env` and `.env.local`, then the Cloud Run service. Since 2026-09-02 every credential on that service is a Secret Manager reference, so `describe` shows `valueFrom` where the value used to be: read one with `node scripts/read-service-env.mjs '^NAME$' --raw`, which resolves literals and references alike (`--names` lists which is which). Never trust `vercel env pull` (returns empty for secrets). Update single vars with `--update-env-vars` (merges); `--set-env-vars` REPLACES the whole set, so never use it for one key. If a credential truly exists nowhere, build the feature fully wired behind the env var, prove it with a mock-free dry run, and list the single missing var in your report. || A QA login for authed page testing | `AUDIT_EMAIL` / `AUDIT_PASSWORD` in `.env` are a real production QA account. If they are missing from `.env` (a fresh clone has no `.env` at all), `npm run audit:web:provision` creates a new one through the real `/register` page and writes both vars back; no server-side code reads them, so the Cloud Run env does not need them. Unauthenticated sweep: `npm run audit:web`. Authed sweep: `npm run audit:web:login` to mint the session, then `npm run audit:web`, which replays it. Details: `docs/ops/page-audit.md`. || GCP access / project facts | Already authenticated in this workspace. Project `aerial-vehicle-466722-p5`, region `us-central1`. Fleet + quota + pre-approved scaling: `docs/ops/gcp-credits-plan.md`. Full production runbook (LB/DNS/TLS/env/rollback): `docs/ops/gcp-production.md`. Logs: `gcloud logging read 'resource.type="cloud_run_revision" resource.labels.service_name="three-ws-api" textPayload:"<term>"' --freshness=24h`. || The database | `DATABASE_URL` in `.env.local` (Neon); `.env` currently holds only the QA audit login, so a script that loads `.env` alone dies on `missing required env var: DATABASE_URL`. Production's authoritative copy is on the Cloud Run service env. Migrations: `npm run db:status` previews pending ones, `npm run db:check` is the deploy gate (exit 4 if pending), and **`npm run db:migrate` APPLIES immediately, with no dry run** (the npm script hardcodes `--apply`). Read `db:status` before you ever run `db:migrate`; it applies every pending migration in `api/_lib/migrations/`, not just yours. If `db:status` reports `[drift]` because an applied migration's *comments* were edited (the banned-dash rule does this), `npm run db:restamp` re-records its hash after recovering the applied bytes from git and confirming not one statement changed; a real statement change is refused and needs a NEW migration. `forge_creations` carries per-generation backend/status/error/prompt: the fastest ground truth for generation issues. |diff --git a/scripts/migrate-plaintext-secrets.mjs b/scripts/migrate-plaintext-secrets.mjsindex b0ddebcb7..211f6713a 100755--- a/scripts/migrate-plaintext-secrets.mjs+++ b/scripts/migrate-plaintext-secrets.mjs@@ -71,6 +71,7 @@ const APPLY = argv.includes('--apply');const VERIFY_ONLY = argv.includes('--verify');const FORCE_NEW_VERSION = argv.includes('--force-new-version');const REUSE = !argv.includes('--no-reuse');+const CONCURRENCY = Math.max(1, Number(optValue('--concurrency', 6)) || 6);function optValue(flag, fallback) {const i = argv.indexOf(flag);@@ -150,6 +151,21 @@ const EXTRA_SECRETS = new Set(['A2A_PAYER_SOLANA_SECRET',]);+// gcloud is a Python process per invocation, so a fan-out has to be bounded or a+// sweep of a few dozen secrets spawns a few dozen interpreters at once and the+// machine starts swapping. Everything here runs through this pool.+async function mapLimit(items, limit, fn) {+ const results = new Array(items.length);+ let next = 0;+ const worker = async () => {+ for (let i = next++; i < items.length; i = next++) {+ results[i] = await fn(items[i], i);+ }+ };+ await Promise.all(Array.from({ length: Math.max(1, Math.min(limit, items.length)) }, worker));+ return results;+}+function fail(msg) {console.error(`\n FAILED: ${msg}\n`);process.exit(1);@@ -278,7 +294,7 @@ async function buildValueIndex() {} catch {return index;}- const digests = await Promise.all(names.map((n) => latestVersionDigest(n)));+ const digests = await mapLimit(names, CONCURRENCY, (n) => latestVersionDigest(n));names.forEach((name, i) => {const d = digests[i];if (!d) return;@@ -458,20 +474,26 @@ async function main() {return;}- const updates = [];- for (const item of credentials) {+ // Each var's secret work is independent, and every step of it is idempotent+ // (an existing secret holding the same value is reused, a repeated IAM binding+ // is a no-op), so this runs a few at a time. Serially, a full sweep of the+ // service spends over an hour in gcloud process startup alone, which is long+ // enough that an interrupted run becomes likely.+ const prepared = await mapLimit(credentials, CONCURRENCY, async (item) => {const result = await prepareSecret(item);if (result.error) {console.log(`\n SKIPPED ${item.name}: ${result.error}`);- continue;+ return null;}const grant = await grantAccessor(item.secretName, before.serviceAccount);console.log(`\n ${item.name}`);console.log(` secret: ${item.secretName}`);console.log(` ${result.action}`);console.log(` ${grant}`);- updates.push(item);- }+ return item;+ });+ // mapLimit preserves input order, so the update reads the way the plan did.+ const updates = prepared.filter(Boolean);if (!updates.length) fail('no var could be prepared; nothing was flipped on the service');diff --git a/scripts/set-r2-cors.mjs b/scripts/set-r2-cors.mjsindex bcde23e0c..fcb5da5fb 100644--- a/scripts/set-r2-cors.mjs+++ b/scripts/set-r2-cors.mjs@@ -29,9 +29,9 @@* - --probe needs NONE of them. With no credentials it discovers the public* host from a live listing endpoint and the presigned-upload host from the* auth-free /api/forge-upload, which is exactly what a browser sees.- * - Production runtime values live on the Cloud Run service, readable with- * `gcloud run services describe three-ws-api --region us-central1- * --project aerial-vehicle-466722-p5 --format=yaml`. Those are the same+ * - Production runtime values live on the Cloud Run service, where credentials+ * are Secret Manager references rather than literals; read one with+ * `node scripts/read-service-env.mjs '^S3_' `. Those are the same* object-scoped keys, so they do not unlock --get either.* - Do NOT use `vercel env pull`: it returns empty for secret-type vars, and* production has not run on Vercel since 2026-07-07.@@ -93,7 +93,7 @@ if (missing.length && !flag('--probe') && !flag('--dry-run')) {// Deploy-time invocation: missing env is not a deploy failure. Local-dev// invocation: surface the hint. Either way, exit 0 so a CI step can chain.console.log(`[set-r2-cors] skipped, missing env: ${missing.join(', ')}`);- console.log('[set-r2-cors] (local: drop R2_ACCOUNT_ID/R2_ACCESS_KEY_ID/R2_SECRET_ACCESS_KEY/R2_BUCKET in .env.local. Production values: gcloud run services describe three-ws-api --region us-central1 --project aerial-vehicle-466722-p5 --format=yaml)');+ console.log("[set-r2-cors] (local: drop R2_ACCOUNT_ID/R2_ACCESS_KEY_ID/R2_SECRET_ACCESS_KEY/R2_BUCKET in .env.local. Production values: node scripts/read-service-env.mjs '^S3_')");console.log('[set-r2-cors] (to measure the LIVE policy without any bucket credentials: node scripts/set-r2-cors.mjs --probe)');process.exit(0);}