Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

docs(owner-actions): re-measure deploy gap at 535 commits, confirm 9 failing pages are still deploy lag

nirholas02b0ada426f4
diff --git a/prompts/finish/_context/production-100-OWNER-ACTIONS.md b/prompts/finish/_context/production-100-OWNER-ACTIONS.md
index 22bbcd2d5..f89968562 100644
--- a/prompts/finish/_context/production-100-OWNER-ACTIONS.md
+++ b/prompts/finish/_context/production-100-OWNER-ACTIONS.md
@@ -9,11 +9,11 @@ it and, if that unblocks a work order, say which one in the owning pack's `produ
Rows carry the date they were added so staleness is visible. Re-verify a row before acting
on it; some resolve themselves as other work lands.
-## Open rows (2026-10-15 (triage-lane deep sweep, second pass): row 1 re-measured, deploy gap still 2 commits against `threews/main`, unchanged from 2026-10-14; two `investigate` findings reclassified into `KNOWN_SIGNATURES` and fixed on `evolve` (NVIDIA NVCF text-to-3d flakiness folded into the existing `three-ws-ai-providers-unavailable-5xx` signature, and a new self-healing `pump-trending-upstreams-502` signature for a confirmed transient double-upstream outage); row 63's duplicate number (PAY_VERDICT_HMAC_KEY, added same day by a concurrent agent) renumbered to 67; rows 24 and 36 re-confirmed unchanged against live `/api/healthz`; 2026-10-15 (order 522-integrate-23 close-out): row 66 added for the x402 outbound-receipts resolved-columns migration (`20261015110000_x402_outbound_resolved.sql`), not yet an active bug since zero outbound-spend rows exist today; 2026-10-15 (triage-lane deep sweep): row 65 added for the payment-session policy-upgrades migration (`20261015100000_payment_session_policy_upgrades.sql`), needed before the `655b5a491` code deploy to avoid a `42703 undefined_column` 500 on every payment-session create/execute call; row 1 re-measured, deploy gap still 2 commits against `threews/main`, unchanged from 2026-10-14; 2026-10-15 (order 518-integrate-19 close-out): row 63 added for setting a dedicated `PAY_VERDICT_HMAC_KEY` on the production Cloud Run service; not an active bug, the signed-verdict feature it covers already works today on a fallback secret, this just gives it its own rotatable key; 2026-10-14 (order 506-integrate-07 close-out): row 62 added for deploying the new `services/pump-claims` Cloud Run service and applying its `pump_fee_claims` migration to production; 2026-10-14 (order 503-integrate-04 close-out): row 61 added for pushing and archiving 7 retired repos (`three-ui`, `crypto-market-data`, `crypto-market-data-ts`, `crypto-data-aggregator`, `memescope-monday-directory`, `robinhood-chain-alert-bot`, `robinhood-chain-trading-bot`); 2026-10-14 (order 501-integrate-02 close-out): row 60 added for approving the `@nirholas/pump-sdk` bump to `^1.36.0` in `packages/avatar-agent-mcp/package.json`; 2026-10-14 (order 500-integrate-01 close-out): row 59 added for pushing every staged standalone mirror now that sync-standalone-repos.mjs stages instead of force-pushing; 2026-10-14 (order 404 second pass): correcting the 2026-10-11 entry below, the `x-mention-reply.js` `chainFor()` fix was not actually carried by merge commit `d77b89178`; checking out that commit's own `evolve` parent showed `chainFor()` still importing `resolveAgentBrain`/`modelChain` (the tool-calling chain shape, no `buildBody`), the exact bug row 34 flagged, still live. The real fix is this session's `evolve` commit `7ba9f9749` ("fix(x-mentions): give public replies a buildBody-shaped provider chain"), switching `chainFor()` to `llm.js`'s own `providerChain()` plus `loadOwnerGrokKey`/`agentDefaultModel`/`MODEL_CATALOG`; re-verified now genuinely passing with the same command the earlier entry cited (`RUN_LLM_TESTS=1 npx vitest run tests/x-mention-reply.llm.test.js`, 16/16 `source: 'model'`, no `buildBody` TypeError). Row 34's premise (the `evolve`/`main` divergence) is still correctly closed, just not for the reason originally stated; no row reopened. Order 060's matching status paragraph corrected the same way. 2026-10-13 (order 095 close-out): row 58 added for the Home Assistant integration's standalone repository, release, and the hacs/default and home-assistant/brands pull requests; 2026-10-13 (order 094 close-out): row 57 added for the WordPress.org/Framer/Webflow account signups and submissions for the three <agent-3d> site-builder embeds; 2026-10-13 (order 092 close-out): row 55 added for the VRoid Hub avatar-import OAuth app registration and migration; 2026-10-13 (triage-lane sweep, fifth pass): row 53 added for the Birdeye compute-units cap behind `pump-price-history-upstreams-502` (confirmed live, a billing/plan decision only); row 1 re-measured, deploy gap still 0 against the pushed `threews/main` ref and the 8 page-404/wrong-canonical findings this pass confirmed as deploy lag; row 47 re-measured, still a no-op (`COINGECKO_API_KEY` still absent); the previously-unclassified `/dist-lib/agent-3d.js` and `/agent-3d/latest/agent-3d.js` 500s ("Response size was too large") are the same bug row 48 already tracks, fixed on `evolve` in `f939d5ef1`, confirmed still green (`npx vitest run tests/lib-bundle-size.test.js tests/whisper-stt.test.js`, 16/16) and awaiting the pending deploy, no new row; 2026-10-13 (triage-lane sweep, fourth pass): row 1 re-measured, deploy gap still 0 against the pushed `threews/main` ref; the `ccv-ai-providers-unavailable-503` finding's Groq/OpenRouter/Vertex angles confirmed already covered by rows 6 and 20, no new row added; 2026-10-13 (order 087 close-out): row 51 added for the LiveKit and Pipecat avatar integrations, both built and verified end to end this session, with the PyPI publishes and the two upstream pull requests left as owner actions; 2026-10-13 (order 085 close-out): row 50 added for publishing the CC0 bulk animation library to production, and the Adobe Mixamo permission letter draft is ready to send as owner order 938; 2026-10-12 (order 084 close-out): row 49 added for the Tripo signed webhook's three linked steps (apply the `forge_webhook_deliveries` migration, set `TRIPO_WEBHOOK_SECRET`, register the dashboard webhook), not an active bug since the secret is unset and polling answers every Tripo job today; 2026-10-12 18:10 UTC (triage-lane sweep, third pass): row 48 added for the dist-lib CDN embed size fix (previously-unclassified `/dist-lib/agent-3d.js` 500s, root-caused and fixed on `evolve`, pure code change awaiting the next deploy); 2026-10-12 17:54 UTC (triage-lane sweep, second pass): row 1's same-day "11 behind" figure corrected to the real deploy gap of 0 (querying `/api/version` live, not `/api/healthz`, which has no commit field), row 47 added for CoinGecko's keyless tier being rate-limited (`COINGECKO_API_KEY` already absent from the service, so the finding's own pre-approved env fix is a no-op); 2026-10-12 (triage-lane sweep): row 1 re-measured against the canonical `threews` remote, 11 commits behind `main` (down from 28, local stale ref corrected), row 45 added for the HF image-provider (fal-ai/nscale) credit exhaustion, a separate pool from row 6; 2026-10-11 (best3d order 070 close-out): rows 43 and 44 added for the `model-partcrafter` and `model-holopart` first deploys; 2026-10-11 (triage-lane sweep): row 42 added for the MCP client-sessions nullable-column migration, an active production bug; row 1 re-measured, 28 commits behind `main` (up from 5); 2026-10-11 (best3d order 069 close-out): rows 40 and 41 added for the `model-rig-nonhuman` and `model-anigen` first deploys; 2026-10-11 (best3d order 068 close-out): row 39 added for the `model-pixal3d` first deploy; 2026-10-11 (best3d order 067 retry): row 38 added for the `model-trellis2` first deploy; 2026-10-11 (order 404 close-out): row 34 deleted, its premise cleared by merge commit `d77b89178` (the `evolve`/`main` reconciliation) which also carried the `x-mention-reply.js` `chainFor()` fix row 34 called out, verified live with `RUN_LLM_TESTS=1 npx vitest run tests/x-mention-reply.llm.test.js` (16/16 `source: 'model'`, no `buildBody` TypeError); 2026-10-11 (scout lane): row 36 added for the Cloudflare R2 flapping degraded/down pattern; 2026-10-11 (even later): row 24 re-measured again after order 400's ring-leak-scan tiering/lock fix shipped on `evolve`; 2026-10-11 (later): row 35 added for the agent mention-policy migration; 2026-10-11 04:16 UTC: rows 1 and 24 re-measured by the deep triage sweep; row 34 added for the evolve/main branch divergence; 2026-10-10: row 33 added for the xAI MCP e2e key; rows 1, 19 and 24 re-measured by the triage sweep; 2026-10-09: rows 1, 2 and 24 re-measured by the triage sweep, row 31 added for the GPU worker redeploy and row 32 for database table retention; row 30 added for the Grok Bot CLI setup rerun; 2026-10-08: row 26 added for the coin-gated MCP catalog regeneration; rows 1 and 2 re-measured by the triage sweep, row 24 added for the paid Solana RPC credits, row 25 added for the MCP connector OAuth deploy; 2026-09-30: row 23 added for the $THREE receiver wallets; 2026-09-29: row 19 re-measured after the agent-sniper deploy; 2026-09-09: row 19 added, and row 1 corrected because its outage premise no longer holds; rows 1, 2, 3, 9, 13, 15 and 16 re-measured 2026-09-04; the former row 18 was cleared that day and its number is not reused; the 2026-09-02 and 2026-08-09 passes are folded in below)
+## Open rows (2026-10-23 (triage-lane deep sweep): row 1 re-measured, deploy gap escalated to 535 commits against `threews/main` (up from 2 on 2026-10-14/15, production unchanged at `45c768d07`); the `/api/chat` 503 storm traced to deploy lag already fixed on HEAD, no new row; one `investigate` finding (`/api/pump/coin` 502, pump.fun Cloudflare anti-bot challenge) given a new self-healing signature (`pump-coin-lookup-upstream-502`) and fixed on `evolve` (`72b600f1c`); rows 2, 3, 4, 5, 6, 19, 23, 24, 36 and 47 re-confirmed unchanged; 2026-10-15 (triage-lane deep sweep, second pass): row 1 re-measured, deploy gap still 2 commits against `threews/main`, unchanged from 2026-10-14; two `investigate` findings reclassified into `KNOWN_SIGNATURES` and fixed on `evolve` (NVIDIA NVCF text-to-3d flakiness folded into the existing `three-ws-ai-providers-unavailable-5xx` signature, and a new self-healing `pump-trending-upstreams-502` signature for a confirmed transient double-upstream outage); row 63's duplicate number (PAY_VERDICT_HMAC_KEY, added same day by a concurrent agent) renumbered to 67; rows 24 and 36 re-confirmed unchanged against live `/api/healthz`; 2026-10-15 (order 522-integrate-23 close-out): row 66 added for the x402 outbound-receipts resolved-columns migration (`20261015110000_x402_outbound_resolved.sql`), not yet an active bug since zero outbound-spend rows exist today; 2026-10-15 (triage-lane deep sweep): row 65 added for the payment-session policy-upgrades migration (`20261015100000_payment_session_policy_upgrades.sql`), needed before the `655b5a491` code deploy to avoid a `42703 undefined_column` 500 on every payment-session create/execute call; row 1 re-measured, deploy gap still 2 commits against `threews/main`, unchanged from 2026-10-14; 2026-10-15 (order 518-integrate-19 close-out): row 63 added for setting a dedicated `PAY_VERDICT_HMAC_KEY` on the production Cloud Run service; not an active bug, the signed-verdict feature it covers already works today on a fallback secret, this just gives it its own rotatable key; 2026-10-14 (order 506-integrate-07 close-out): row 62 added for deploying the new `services/pump-claims` Cloud Run service and applying its `pump_fee_claims` migration to production; 2026-10-14 (order 503-integrate-04 close-out): row 61 added for pushing and archiving 7 retired repos (`three-ui`, `crypto-market-data`, `crypto-market-data-ts`, `crypto-data-aggregator`, `memescope-monday-directory`, `robinhood-chain-alert-bot`, `robinhood-chain-trading-bot`); 2026-10-14 (order 501-integrate-02 close-out): row 60 added for approving the `@nirholas/pump-sdk` bump to `^1.36.0` in `packages/avatar-agent-mcp/package.json`; 2026-10-14 (order 500-integrate-01 close-out): row 59 added for pushing every staged standalone mirror now that sync-standalone-repos.mjs stages instead of force-pushing; 2026-10-14 (order 404 second pass): correcting the 2026-10-11 entry below, the `x-mention-reply.js` `chainFor()` fix was not actually carried by merge commit `d77b89178`; checking out that commit's own `evolve` parent showed `chainFor()` still importing `resolveAgentBrain`/`modelChain` (the tool-calling chain shape, no `buildBody`), the exact bug row 34 flagged, still live. The real fix is this session's `evolve` commit `7ba9f9749` ("fix(x-mentions): give public replies a buildBody-shaped provider chain"), switching `chainFor()` to `llm.js`'s own `providerChain()` plus `loadOwnerGrokKey`/`agentDefaultModel`/`MODEL_CATALOG`; re-verified now genuinely passing with the same command the earlier entry cited (`RUN_LLM_TESTS=1 npx vitest run tests/x-mention-reply.llm.test.js`, 16/16 `source: 'model'`, no `buildBody` TypeError). Row 34's premise (the `evolve`/`main` divergence) is still correctly closed, just not for the reason originally stated; no row reopened. Order 060's matching status paragraph corrected the same way. 2026-10-13 (order 095 close-out): row 58 added for the Home Assistant integration's standalone repository, release, and the hacs/default and home-assistant/brands pull requests; 2026-10-13 (order 094 close-out): row 57 added for the WordPress.org/Framer/Webflow account signups and submissions for the three <agent-3d> site-builder embeds; 2026-10-13 (order 092 close-out): row 55 added for the VRoid Hub avatar-import OAuth app registration and migration; 2026-10-13 (triage-lane sweep, fifth pass): row 53 added for the Birdeye compute-units cap behind `pump-price-history-upstreams-502` (confirmed live, a billing/plan decision only); row 1 re-measured, deploy gap still 0 against the pushed `threews/main` ref and the 8 page-404/wrong-canonical findings this pass confirmed as deploy lag; row 47 re-measured, still a no-op (`COINGECKO_API_KEY` still absent); the previously-unclassified `/dist-lib/agent-3d.js` and `/agent-3d/latest/agent-3d.js` 500s ("Response size was too large") are the same bug row 48 already tracks, fixed on `evolve` in `f939d5ef1`, confirmed still green (`npx vitest run tests/lib-bundle-size.test.js tests/whisper-stt.test.js`, 16/16) and awaiting the pending deploy, no new row; 2026-10-13 (triage-lane sweep, fourth pass): row 1 re-measured, deploy gap still 0 against the pushed `threews/main` ref; the `ccv-ai-providers-unavailable-503` finding's Groq/OpenRouter/Vertex angles confirmed already covered by rows 6 and 20, no new row added; 2026-10-13 (order 087 close-out): row 51 added for the LiveKit and Pipecat avatar integrations, both built and verified end to end this session, with the PyPI publishes and the two upstream pull requests left as owner actions; 2026-10-13 (order 085 close-out): row 50 added for publishing the CC0 bulk animation library to production, and the Adobe Mixamo permission letter draft is ready to send as owner order 938; 2026-10-12 (order 084 close-out): row 49 added for the Tripo signed webhook's three linked steps (apply the `forge_webhook_deliveries` migration, set `TRIPO_WEBHOOK_SECRET`, register the dashboard webhook), not an active bug since the secret is unset and polling answers every Tripo job today; 2026-10-12 18:10 UTC (triage-lane sweep, third pass): row 48 added for the dist-lib CDN embed size fix (previously-unclassified `/dist-lib/agent-3d.js` 500s, root-caused and fixed on `evolve`, pure code change awaiting the next deploy); 2026-10-12 17:54 UTC (triage-lane sweep, second pass): row 1's same-day "11 behind" figure corrected to the real deploy gap of 0 (querying `/api/version` live, not `/api/healthz`, which has no commit field), row 47 added for CoinGecko's keyless tier being rate-limited (`COINGECKO_API_KEY` already absent from the service, so the finding's own pre-approved env fix is a no-op); 2026-10-12 (triage-lane sweep): row 1 re-measured against the canonical `threews` remote, 11 commits behind `main` (down from 28, local stale ref corrected), row 45 added for the HF image-provider (fal-ai/nscale) credit exhaustion, a separate pool from row 6; 2026-10-11 (best3d order 070 close-out): rows 43 and 44 added for the `model-partcrafter` and `model-holopart` first deploys; 2026-10-11 (triage-lane sweep): row 42 added for the MCP client-sessions nullable-column migration, an active production bug; row 1 re-measured, 28 commits behind `main` (up from 5); 2026-10-11 (best3d order 069 close-out): rows 40 and 41 added for the `model-rig-nonhuman` and `model-anigen` first deploys; 2026-10-11 (best3d order 068 close-out): row 39 added for the `model-pixal3d` first deploy; 2026-10-11 (best3d order 067 retry): row 38 added for the `model-trellis2` first deploy; 2026-10-11 (order 404 close-out): row 34 deleted, its premise cleared by merge commit `d77b89178` (the `evolve`/`main` reconciliation) which also carried the `x-mention-reply.js` `chainFor()` fix row 34 called out, verified live with `RUN_LLM_TESTS=1 npx vitest run tests/x-mention-reply.llm.test.js` (16/16 `source: 'model'`, no `buildBody` TypeError); 2026-10-11 (scout lane): row 36 added for the Cloudflare R2 flapping degraded/down pattern; 2026-10-11 (even later): row 24 re-measured again after order 400's ring-leak-scan tiering/lock fix shipped on `evolve`; 2026-10-11 (later): row 35 added for the agent mention-policy migration; 2026-10-11 04:16 UTC: rows 1 and 24 re-measured by the deep triage sweep; row 34 added for the evolve/main branch divergence; 2026-10-10: row 33 added for the xAI MCP e2e key; rows 1, 19 and 24 re-measured by the triage sweep; 2026-10-09: rows 1, 2 and 24 re-measured by the triage sweep, row 31 added for the GPU worker redeploy and row 32 for database table retention; row 30 added for the Grok Bot CLI setup rerun; 2026-10-08: row 26 added for the coin-gated MCP catalog regeneration; rows 1 and 2 re-measured by the triage sweep, row 24 added for the paid Solana RPC credits, row 25 added for the MCP connector OAuth deploy; 2026-09-30: row 23 added for the $THREE receiver wallets; 2026-09-29: row 19 re-measured after the agent-sniper deploy; 2026-09-09: row 19 added, and row 1 corrected because its outage premise no longer holds; rows 1, 2, 3, 9, 13, 15 and 16 re-measured 2026-09-04; the former row 18 was cleared that day and its number is not reused; the 2026-09-02 and 2026-08-09 passes are folded in below)
| # | Action | Size | Unblocks | Detail lives in |
|---|---|---|---|---|
-| 1 | Approve the production deploy (or say "get production working", which is itself the approval). **The outage this row was written for is CLEARED.** Re-measured 2026-09-09 from the live origin: production serves `880bdcef8` on revision `three-ws-api-00420-ljh`, built 2026-09-08 19:05 UTC, and the `ERR_MODULE_NOT_FOUND` on `/app/services/home-relay/src/token.js` is gone: `/api/healthz` answers **HTTP 200** and `/api/home/status` and `/api/home/connections` answer 401 (auth required) rather than 500. What remains is ordinary staleness, 182 commits behind `main`, which is a routine cadence question and no longer a platform-health emergency. The 2026-09-04 text of this row, which described 96 commits and a 500ing healthz, is superseded. **Re-measured 2026-10-08 17:05 UTC:** production serves `fe2a8b24f` on `three-ws-api-00479-d7z`, **146 commits behind `main`, 90 of them fixes** (security, payouts, SSRF and XSS among them). Shipping them clears four live findings at once: the ring sweep's fee falls back to the treasury instead of the dry sponsor (`a9c996e72`, about 30 failed sweeps an hour since 2026-10-03), an EVM testnet identity-index cursor frozen since 2026-09-11 (`21fa42bb5`), two pages and their two docs pages advertised in the sitemap but answering 404 (added in `16f9d07f4`), and `/tty` holding its animation until the end for any client that accepts compression (`03b4f1189`). One migration must go with it: `20261008150000_revenue_direct_settle_backfill.sql` (pending; it matches 0 rows today, so applying it is safe, and the code half is what keeps the payout cron from paying a direct-to-seller sale twice). Commands, in order: `npm run db:status`, `npm run db:migrate`, `npm run deploy:gcp:full`. **Re-measured 2026-10-09 03:05 UTC by the triage sweep:** production still serves image `fe2a8b24f` (now on revision `three-ws-api-00480-mws`, an env-only roll that raised `DB_RETENTION_HIGH_WATER_MB` to 16384), **172 commits behind `main`**, and the `evolve` branch holds 85 more awaiting the runner's merge, six of them this sweep's production fixes: the leak scanner's lease and budget (`0abce6d69`, which ends about eight overlapping 600 to 900 s scans that were burning the Solana RPC quota), `-32020` RPC failover (`4b51a179c`), static 412/416 instead of 500 (`e01e43b3b`), the news knowledge write fix (`833407d4f`, 120 lost stories a day), and the reconstruct sweep's plan-limit close (`85d3b2034`). `20261008150000_revenue_direct_settle_backfill.sql` has since been applied; `npm run db:status` now lists two pending, `20261008190000_oauth_refresh_user_client_index.sql` and `20261008200000_mcp_client_analytics.sql` (rows 27 and 28). The 8 sitemap pages still answering 404 (`/grok`, `/grok-skill.md`, `/docs/grok`, `/docs/grok-bot`, `/dextools`, `/docs/dextools`, `/chart-companion`, `/docs/chart-companion`) are all deploy lag. **Re-measured 2026-10-10 20:06 UTC by the triage sweep:** production serves `5af439bc0` on `three-ws-api-00487-7mt`, built 2026-10-10 05:01 UTC, **45 commits behind `main`** (down from 172 on 2026-10-09; most of the gap already shipped), and the `evolve` branch holds 89 more. All 869 advertised pages, every cron (125) and the fleet (48 services) check out clean; the only pending migrations are rows 27 and 28's two files. **Re-measured 2026-10-11 04:16 UTC by the deep triage sweep:** production serves `3503381b1` on `three-ws-api-00491-fbl`, **112 commits behind `main`** (up from 45 on 2026-10-10). One new page-lag symptom from the gap: `/docs/tutorials/grok-bot-recipes` (added in `71c673f99`, after the running revision) answers 200 with `/docs`'s content instead of its own; this is deploy lag, not a routing bug (verified: `71c673f99` is not an ancestor of `3503381b1`). The fleet (48 services), 125 crons and DB are otherwise clean; the only pending migrations remain rows 27 and 28's two files. **Re-measured 2026-10-11 12:32 UTC by the triage-lane sweep:** production serves `66a5e93bd` on `three-ws-api-00494-wh5`, built 2026-10-11 07:22 UTC, **only 5 commits behind `main`** (down sharply from 112 on 2026-10-11 04:16; the earlier large gap has shipped). The 5 outstanding commits are the desktop OS/shell/widgets feature set and the campus world update (`e33f529d8`, `33b5490c5`, `a48caf816`, `9c47f3ceb`, `86366a55e`). Rows 27 and 28's migrations are now applied; `npm run db:status` shows 2 new pending files, `20261011220000_agent_mention_policy.sql` (tracked by row 35) and `20261011230000_x402_ring_scan_checked_at.sql` (tracked by row 24's narrative), both additive. This sweep also committed two root-cause fixes to `main`-bound code that are NOT yet in this 5-commit gap: `65d6c4be8` (clink-fun MCP stream timeout misclassified as investigate) and `2f2fedf99` (a Pollinations 402 was putting the paid Replicate/TRELLIS lane into cooldown, contributing to `forge_generation: down`); the second one fixes a live production defect, so it is worth prioritizing in the next deploy. **Re-measured 2026-10-11 21:12 UTC by the triage-lane sweep:** production still serves `66a5e93bd` on `three-ws-api-00494-wh5`, **28 commits behind `main`** (`main` now at `9eda7992d`), up from 5 at 12:32 UTC because the desktop OS/shell/widgets/campus set from that measurement has been joined by new work: the x-content partner-spotlight feature (registry, daily scheduler, published IBM and AWS partner articles, the `/partners/benefits` page) and five dependabot bumps (`js-yaml`/`markdownlint-cli2`, `nanoid`, `maplibre-gl`, `torch` in `workers/texture`, `brace-expansion`, `source-map-js`, `moment`, `fast-uri`). `2f2fedf99` (the forge-cooldown fix) is still only on `evolve`, not yet on `main`; it remains worth prioritizing. The deep sweep found exactly one new pending migration, now tracked as row 42; rows 24 and 35's migrations are applied. The `/docs/desktop-widgets` 404 and `/docs/tutorials/grok-bot-recipes` wrong-canonical findings are confirmed deploy lag again (their catalog commits are not ancestors of `66a5e93bd`), not new routing bugs. **Re-measured 2026-10-12 by the triage-lane sweep, against the canonical `threews` remote (the local `main` ref in the `evolve` worktree was stale):** production (`/api/healthz`'s own `commitShort`) serves `c266e2591`, **11 commits behind `threews/main`** at `80e5e35f7`, down sharply from 28. The outstanding set is the Dissolve feature (a GPU particle-storm viewer for any 3D model) with its docs and page-catalog regeneration, a security-hardening batch (`8a56a44ab`: auth, wallet policy, SDP, vault, skill sandbox, MCP bridge URL guard, all with regression tests), a worker fix (`315e96f96`: block internal-host navigation in the screen pool, load text2motion weights with `weights_only`), an accessibility fix (`f772c8708`: icon-only button tooltips, gallery/create/header name-to-text matching), the Grid desktop world (`762269248`, `f35d07d93`, `a60d72dad`) and a new UI-lint tool (`568162d8d`). None of this sweep's own fixes (the Vertex/HF image-lane circuit breaker, the `forge-image-ladder-exhausted-502` and widened `r2-downstream-5xx` triage signatures) are in this 11-commit gap either; they are further back on `evolve`, behind the runner's usual fast-forward-to-`main` cadence. **Corrected 2026-10-12 17:54 UTC by the triage-lane sweep: the same-day "11 commits behind" figure above was a misread.** Querying the live `/api/version` endpoint directly (not `/api/healthz`, which carries no commit field) shows production still serves `771cfef01` on `three-ws-api-00496-78g`, built 2026-10-12 07:08 UTC. A fresh `git fetch threews` puts `threews/main` at `80e5e35f7`, and `git merge-base --is-ancestor threews/main 771cfef01` succeeds with exactly 1 commit between them, so the deployed revision is 1 commit AHEAD of the canonical remote `main`, not behind it. **The real deploy gap (commits on `main` not yet in production) is 0.** The 242 commits the deep sweep reports as "ahead" are `evolve`-branch commits (this worktree's local `main`, one commit ahead of pushed `threews/main`) not yet pushed, including this sweep's own `5afb045cf` (widened the `r2-downstream-5xx` triage signature to cover `/api/render/animate`, `/api/render/glb`, `/api/render/avatar-clip` and `/api/animations/clips/<id>`, all previously-unmatched symptoms of the same R2 flap row 36 already tracks); they are not a production staleness problem. **Re-measured 2026-10-13 by the triage-lane sweep:** production still serves `771cfef01` on `three-ws-api-00496-78g`. A fresh `git fetch threews` puts `threews/main` at `80e5e35f74302c85138bf6cabdad4f2db65b9ecf`, and `git merge-base --is-ancestor threews/main 771cfef01` succeeds, so the deployed revision is still exactly 1 commit AHEAD of the canonical remote `main`. **The real deploy gap remains 0.** The deep sweep's "283 commits ahead" figure is `evolve`-branch backlog (unpushed local commits, including `f939d5ef1`, the dist-lib CDN embed size fix documented in row 48) awaiting the runner's fast-forward, not a production staleness problem. This sweep found no new code-level defect to fix: the dist-lib bug is already root-caused, fixed and tested on `evolve`; the Solana-adjacent degraded subsystems (`agent_index`, `rpc_lanes`, `helius`, `sniper`, `three_token_rail`) and the `ccv-ai-providers-unavailable-503` finding (Groq's 200k tokens/day cap, the GCP billing hold, and the revoked OpenRouter key) are all pre-existing owner items already tracked in rows 23, 24, 19, 2, 20 and 6; none needed a new row. **Re-measured 2026-10-13 (triage-lane sweep, fifth pass):** production still serves `771cfef01` on `three-ws-api-00497-2ss`. `git fetch threews` puts `threews/main` at `80e5e35f74302c85138bf6cabdad4f2db65b9ecf`, and `git merge-base --is-ancestor threews/main 771cfef01` still succeeds, so the deployed revision remains exactly 1 commit AHEAD of `main`. **The real deploy gap stays 0.** The deep sweep's "291 commits ahead" figure is `evolve`-branch backlog (unpushed, includes `f939d5ef1` / row 48 and this pass's new row 53) awaiting the runner's fast-forward, not production staleness. The 8 sitemap-advertised pages answering 404 or wrong-canonical this pass (`/discord-activity`, `/docs/discord-activity`, `/docs/engine-bridges`, `/docs/engine-qa`, `/docs/open-source-credits`, `/docs/voice-framework-integrations`, `/open-source`, `/docs/tutorials/grok-bot-recipes`) are all confirmed deploy lag (`git merge-base --is-ancestor <add-commit> 771cfef01` fails for each); no routing bug. **Re-measured 2026-10-14 by the triage-lane sweep (production health sweep order):** production serves `45c768d07` on `three-ws-api-00500-b8q`. A fresh `git fetch threews` puts `threews/main` at `43a5dba21`, and `git log --oneline 45c768d07..threews/main` lists exactly 2 commits (`43a5dba21`, an x-content proofs refresh, and `3480497ed`, the R2-NotEntitled storage-outage failover fix): **the real deploy gap is 2 commits, both routine, neither urgent.** The deep sweep's 16 failing-page findings (`/discord-activity`, `/docs/engine-bridges`, `/docs/engine-qa`, `/docs/google-adk`, `/docs/open-source-credits`, `/docs/shopify-app`, `/docs/video-to-motion`, `/docs/voice-framework-integrations`, `/docs/vroid-hub-import`, `/docs/wordpress-embed`, `/import/vroid`, `/open-source`, `/video-to-motion` and their docs pages, plus the `/docs/tutorials/grok-bot-recipes` wrong-canonical) are confirmed NOT part of this 2-commit gap: none of their routes exist in `threews/main`'s `data/pages.json` at all (checked directly), so they are unpushed `evolve`-branch backlog (334 commits ahead of `threews/main` in this worktree), the same already-tracked rows 49, 52, 54 and 55 cover (Tripo webhook, Discord Activity, Shopify app, VRoid Hub import). The 5 pending migrations `npm run db:status` lists (`mcp_client_sessions_client_nullable`, `forge_webhook_deliveries`, `discord_activity_rooms`, `shopify_app`, `vroid_hub_source`) are the same set tracked by rows 42, 49, 52, 54 and 55; none applied, consistent with those rows' guidance. The lone unclassified `investigate` finding, a single `HTTP 502 GET /api/v1/coins` on `cryptocurrency-cv` with no matching app-level error, has no isolated root cause distinguishable from the 300-plus already-tracked `ccv-ai-providers-unavailable-503` hits on the same service in the same window; not elevated to a new row. This sweep's one code-level fix, a missing diagnostic log on `/api/agents/:id/alpha/read`'s generic LLM-failure 502 branch (not in the deep sweep's own findings list, found by log analysis; test-first, committed on `evolve`), is orthogonal to this row and does not change the gap count. **Re-measured 2026-10-15 by the triage-lane deep sweep:** production still serves `45c768d07` on `three-ws-api-00500-b8q`. A fresh `git fetch threews` puts `threews/main` at `43a5dba21`, and `git log --oneline 45c768d07..threews/main` still lists exactly the same 2 commits (`43a5dba21`, `3480497ed`): **the real deploy gap remains 2 commits, unchanged from 2026-10-14.** The deep sweep's own "422 commits ahead" figure is `evolve`-branch backlog in this worktree, not production staleness; the same caveat row 1 has carried since 2026-10-12. This pass's 9 failing-page findings (`/docs/vroid-hub-import`, `/docs/wordpress-embed`, `/import/vroid`, `/open-source`, `/video-to-motion`, `/docs/forge-workflows`, `/docs/standalone-repos`, `/docs/team-chat`, `/docs/tutorials/grok-bot-recipes`) are confirmed deploy lag the same way (none of their `data/pages.json` entries predate the deployed build; `git merge-base --is-ancestor` fails for each against `45c768d07`), consistent with rows 49/52/54/55/58. The `deep-db-migrations` probe's 8 pending files match rows already open above; none applied this pass, per the self-unblock playbook's no-dry-run warning on `npm run db:migrate`. Two `investigate` findings got code fixes instead of new rows: `/api/v1/ai/text-to-3d` 502s are the same NVIDIA NVCF free-tier capacity flakiness `three-ws-ai-providers-unavailable-5xx` already covers (rows 6/20), its matcher widened to include that route; `/api/pump/trending` and `/api/v1/pump/trending` 502s are a genuine but self-healing double-upstream outage (Birdeye and the pump.fun board both down in the same window, stale-cache fallback also exhausted), given a new self-healing signature so the monitor stops flagging it as `investigate`. Both fixes are committed on `evolve` (`90628d426`) with passing tests, pending the normal fast-forward to `main`. Rows 24 (`rpc_lanes: degraded, both paid lanes exhausted`) and 36 (`object_storage: down`, R2 "Please enable R2 through the Cloudflare Dashboard") re-confirmed unchanged against live `/api/healthz`; no new rows, no edits needed to either. | one yes | Shipping whatever has landed on `main` since 2026-09-08, now including the security-hardening batch and the forge image-lane breaker fix | [01-ship-readiness.md](../901-production-100-01-ship-readiness.md) |
+| 1 | Approve the production deploy (or say "get production working", which is itself the approval). **The outage this row was written for is CLEARED.** Re-measured 2026-09-09 from the live origin: production serves `880bdcef8` on revision `three-ws-api-00420-ljh`, built 2026-09-08 19:05 UTC, and the `ERR_MODULE_NOT_FOUND` on `/app/services/home-relay/src/token.js` is gone: `/api/healthz` answers **HTTP 200** and `/api/home/status` and `/api/home/connections` answer 401 (auth required) rather than 500. What remains is ordinary staleness, 182 commits behind `main`, which is a routine cadence question and no longer a platform-health emergency. The 2026-09-04 text of this row, which described 96 commits and a 500ing healthz, is superseded. **Re-measured 2026-10-08 17:05 UTC:** production serves `fe2a8b24f` on `three-ws-api-00479-d7z`, **146 commits behind `main`, 90 of them fixes** (security, payouts, SSRF and XSS among them). Shipping them clears four live findings at once: the ring sweep's fee falls back to the treasury instead of the dry sponsor (`a9c996e72`, about 30 failed sweeps an hour since 2026-10-03), an EVM testnet identity-index cursor frozen since 2026-09-11 (`21fa42bb5`), two pages and their two docs pages advertised in the sitemap but answering 404 (added in `16f9d07f4`), and `/tty` holding its animation until the end for any client that accepts compression (`03b4f1189`). One migration must go with it: `20261008150000_revenue_direct_settle_backfill.sql` (pending; it matches 0 rows today, so applying it is safe, and the code half is what keeps the payout cron from paying a direct-to-seller sale twice). Commands, in order: `npm run db:status`, `npm run db:migrate`, `npm run deploy:gcp:full`. **Re-measured 2026-10-09 03:05 UTC by the triage sweep:** production still serves image `fe2a8b24f` (now on revision `three-ws-api-00480-mws`, an env-only roll that raised `DB_RETENTION_HIGH_WATER_MB` to 16384), **172 commits behind `main`**, and the `evolve` branch holds 85 more awaiting the runner's merge, six of them this sweep's production fixes: the leak scanner's lease and budget (`0abce6d69`, which ends about eight overlapping 600 to 900 s scans that were burning the Solana RPC quota), `-32020` RPC failover (`4b51a179c`), static 412/416 instead of 500 (`e01e43b3b`), the news knowledge write fix (`833407d4f`, 120 lost stories a day), and the reconstruct sweep's plan-limit close (`85d3b2034`). `20261008150000_revenue_direct_settle_backfill.sql` has since been applied; `npm run db:status` now lists two pending, `20261008190000_oauth_refresh_user_client_index.sql` and `20261008200000_mcp_client_analytics.sql` (rows 27 and 28). The 8 sitemap pages still answering 404 (`/grok`, `/grok-skill.md`, `/docs/grok`, `/docs/grok-bot`, `/dextools`, `/docs/dextools`, `/chart-companion`, `/docs/chart-companion`) are all deploy lag. **Re-measured 2026-10-10 20:06 UTC by the triage sweep:** production serves `5af439bc0` on `three-ws-api-00487-7mt`, built 2026-10-10 05:01 UTC, **45 commits behind `main`** (down from 172 on 2026-10-09; most of the gap already shipped), and the `evolve` branch holds 89 more. All 869 advertised pages, every cron (125) and the fleet (48 services) check out clean; the only pending migrations are rows 27 and 28's two files. **Re-measured 2026-10-11 04:16 UTC by the deep triage sweep:** production serves `3503381b1` on `three-ws-api-00491-fbl`, **112 commits behind `main`** (up from 45 on 2026-10-10). One new page-lag symptom from the gap: `/docs/tutorials/grok-bot-recipes` (added in `71c673f99`, after the running revision) answers 200 with `/docs`'s content instead of its own; this is deploy lag, not a routing bug (verified: `71c673f99` is not an ancestor of `3503381b1`). The fleet (48 services), 125 crons and DB are otherwise clean; the only pending migrations remain rows 27 and 28's two files. **Re-measured 2026-10-11 12:32 UTC by the triage-lane sweep:** production serves `66a5e93bd` on `three-ws-api-00494-wh5`, built 2026-10-11 07:22 UTC, **only 5 commits behind `main`** (down sharply from 112 on 2026-10-11 04:16; the earlier large gap has shipped). The 5 outstanding commits are the desktop OS/shell/widgets feature set and the campus world update (`e33f529d8`, `33b5490c5`, `a48caf816`, `9c47f3ceb`, `86366a55e`). Rows 27 and 28's migrations are now applied; `npm run db:status` shows 2 new pending files, `20261011220000_agent_mention_policy.sql` (tracked by row 35) and `20261011230000_x402_ring_scan_checked_at.sql` (tracked by row 24's narrative), both additive. This sweep also committed two root-cause fixes to `main`-bound code that are NOT yet in this 5-commit gap: `65d6c4be8` (clink-fun MCP stream timeout misclassified as investigate) and `2f2fedf99` (a Pollinations 402 was putting the paid Replicate/TRELLIS lane into cooldown, contributing to `forge_generation: down`); the second one fixes a live production defect, so it is worth prioritizing in the next deploy. **Re-measured 2026-10-11 21:12 UTC by the triage-lane sweep:** production still serves `66a5e93bd` on `three-ws-api-00494-wh5`, **28 commits behind `main`** (`main` now at `9eda7992d`), up from 5 at 12:32 UTC because the desktop OS/shell/widgets/campus set from that measurement has been joined by new work: the x-content partner-spotlight feature (registry, daily scheduler, published IBM and AWS partner articles, the `/partners/benefits` page) and five dependabot bumps (`js-yaml`/`markdownlint-cli2`, `nanoid`, `maplibre-gl`, `torch` in `workers/texture`, `brace-expansion`, `source-map-js`, `moment`, `fast-uri`). `2f2fedf99` (the forge-cooldown fix) is still only on `evolve`, not yet on `main`; it remains worth prioritizing. The deep sweep found exactly one new pending migration, now tracked as row 42; rows 24 and 35's migrations are applied. The `/docs/desktop-widgets` 404 and `/docs/tutorials/grok-bot-recipes` wrong-canonical findings are confirmed deploy lag again (their catalog commits are not ancestors of `66a5e93bd`), not new routing bugs. **Re-measured 2026-10-12 by the triage-lane sweep, against the canonical `threews` remote (the local `main` ref in the `evolve` worktree was stale):** production (`/api/healthz`'s own `commitShort`) serves `c266e2591`, **11 commits behind `threews/main`** at `80e5e35f7`, down sharply from 28. The outstanding set is the Dissolve feature (a GPU particle-storm viewer for any 3D model) with its docs and page-catalog regeneration, a security-hardening batch (`8a56a44ab`: auth, wallet policy, SDP, vault, skill sandbox, MCP bridge URL guard, all with regression tests), a worker fix (`315e96f96`: block internal-host navigation in the screen pool, load text2motion weights with `weights_only`), an accessibility fix (`f772c8708`: icon-only button tooltips, gallery/create/header name-to-text matching), the Grid desktop world (`762269248`, `f35d07d93`, `a60d72dad`) and a new UI-lint tool (`568162d8d`). None of this sweep's own fixes (the Vertex/HF image-lane circuit breaker, the `forge-image-ladder-exhausted-502` and widened `r2-downstream-5xx` triage signatures) are in this 11-commit gap either; they are further back on `evolve`, behind the runner's usual fast-forward-to-`main` cadence. **Corrected 2026-10-12 17:54 UTC by the triage-lane sweep: the same-day "11 commits behind" figure above was a misread.** Querying the live `/api/version` endpoint directly (not `/api/healthz`, which carries no commit field) shows production still serves `771cfef01` on `three-ws-api-00496-78g`, built 2026-10-12 07:08 UTC. A fresh `git fetch threews` puts `threews/main` at `80e5e35f7`, and `git merge-base --is-ancestor threews/main 771cfef01` succeeds with exactly 1 commit between them, so the deployed revision is 1 commit AHEAD of the canonical remote `main`, not behind it. **The real deploy gap (commits on `main` not yet in production) is 0.** The 242 commits the deep sweep reports as "ahead" are `evolve`-branch commits (this worktree's local `main`, one commit ahead of pushed `threews/main`) not yet pushed, including this sweep's own `5afb045cf` (widened the `r2-downstream-5xx` triage signature to cover `/api/render/animate`, `/api/render/glb`, `/api/render/avatar-clip` and `/api/animations/clips/<id>`, all previously-unmatched symptoms of the same R2 flap row 36 already tracks); they are not a production staleness problem. **Re-measured 2026-10-13 by the triage-lane sweep:** production still serves `771cfef01` on `three-ws-api-00496-78g`. A fresh `git fetch threews` puts `threews/main` at `80e5e35f74302c85138bf6cabdad4f2db65b9ecf`, and `git merge-base --is-ancestor threews/main 771cfef01` succeeds, so the deployed revision is still exactly 1 commit AHEAD of the canonical remote `main`. **The real deploy gap remains 0.** The deep sweep's "283 commits ahead" figure is `evolve`-branch backlog (unpushed local commits, including `f939d5ef1`, the dist-lib CDN embed size fix documented in row 48) awaiting the runner's fast-forward, not a production staleness problem. This sweep found no new code-level defect to fix: the dist-lib bug is already root-caused, fixed and tested on `evolve`; the Solana-adjacent degraded subsystems (`agent_index`, `rpc_lanes`, `helius`, `sniper`, `three_token_rail`) and the `ccv-ai-providers-unavailable-503` finding (Groq's 200k tokens/day cap, the GCP billing hold, and the revoked OpenRouter key) are all pre-existing owner items already tracked in rows 23, 24, 19, 2, 20 and 6; none needed a new row. **Re-measured 2026-10-13 (triage-lane sweep, fifth pass):** production still serves `771cfef01` on `three-ws-api-00497-2ss`. `git fetch threews` puts `threews/main` at `80e5e35f74302c85138bf6cabdad4f2db65b9ecf`, and `git merge-base --is-ancestor threews/main 771cfef01` still succeeds, so the deployed revision remains exactly 1 commit AHEAD of `main`. **The real deploy gap stays 0.** The deep sweep's "291 commits ahead" figure is `evolve`-branch backlog (unpushed, includes `f939d5ef1` / row 48 and this pass's new row 53) awaiting the runner's fast-forward, not production staleness. The 8 sitemap-advertised pages answering 404 or wrong-canonical this pass (`/discord-activity`, `/docs/discord-activity`, `/docs/engine-bridges`, `/docs/engine-qa`, `/docs/open-source-credits`, `/docs/voice-framework-integrations`, `/open-source`, `/docs/tutorials/grok-bot-recipes`) are all confirmed deploy lag (`git merge-base --is-ancestor <add-commit> 771cfef01` fails for each); no routing bug. **Re-measured 2026-10-14 by the triage-lane sweep (production health sweep order):** production serves `45c768d07` on `three-ws-api-00500-b8q`. A fresh `git fetch threews` puts `threews/main` at `43a5dba21`, and `git log --oneline 45c768d07..threews/main` lists exactly 2 commits (`43a5dba21`, an x-content proofs refresh, and `3480497ed`, the R2-NotEntitled storage-outage failover fix): **the real deploy gap is 2 commits, both routine, neither urgent.** The deep sweep's 16 failing-page findings (`/discord-activity`, `/docs/engine-bridges`, `/docs/engine-qa`, `/docs/google-adk`, `/docs/open-source-credits`, `/docs/shopify-app`, `/docs/video-to-motion`, `/docs/voice-framework-integrations`, `/docs/vroid-hub-import`, `/docs/wordpress-embed`, `/import/vroid`, `/open-source`, `/video-to-motion` and their docs pages, plus the `/docs/tutorials/grok-bot-recipes` wrong-canonical) are confirmed NOT part of this 2-commit gap: none of their routes exist in `threews/main`'s `data/pages.json` at all (checked directly), so they are unpushed `evolve`-branch backlog (334 commits ahead of `threews/main` in this worktree), the same already-tracked rows 49, 52, 54 and 55 cover (Tripo webhook, Discord Activity, Shopify app, VRoid Hub import). The 5 pending migrations `npm run db:status` lists (`mcp_client_sessions_client_nullable`, `forge_webhook_deliveries`, `discord_activity_rooms`, `shopify_app`, `vroid_hub_source`) are the same set tracked by rows 42, 49, 52, 54 and 55; none applied, consistent with those rows' guidance. The lone unclassified `investigate` finding, a single `HTTP 502 GET /api/v1/coins` on `cryptocurrency-cv` with no matching app-level error, has no isolated root cause distinguishable from the 300-plus already-tracked `ccv-ai-providers-unavailable-503` hits on the same service in the same window; not elevated to a new row. This sweep's one code-level fix, a missing diagnostic log on `/api/agents/:id/alpha/read`'s generic LLM-failure 502 branch (not in the deep sweep's own findings list, found by log analysis; test-first, committed on `evolve`), is orthogonal to this row and does not change the gap count. **Re-measured 2026-10-15 by the triage-lane deep sweep:** production still serves `45c768d07` on `three-ws-api-00500-b8q`. A fresh `git fetch threews` puts `threews/main` at `43a5dba21`, and `git log --oneline 45c768d07..threews/main` still lists exactly the same 2 commits (`43a5dba21`, `3480497ed`): **the real deploy gap remains 2 commits, unchanged from 2026-10-14.** The deep sweep's own "422 commits ahead" figure is `evolve`-branch backlog in this worktree, not production staleness; the same caveat row 1 has carried since 2026-10-12. This pass's 9 failing-page findings (`/docs/vroid-hub-import`, `/docs/wordpress-embed`, `/import/vroid`, `/open-source`, `/video-to-motion`, `/docs/forge-workflows`, `/docs/standalone-repos`, `/docs/team-chat`, `/docs/tutorials/grok-bot-recipes`) are confirmed deploy lag the same way (none of their `data/pages.json` entries predate the deployed build; `git merge-base --is-ancestor` fails for each against `45c768d07`), consistent with rows 49/52/54/55/58. The `deep-db-migrations` probe's 8 pending files match rows already open above; none applied this pass, per the self-unblock playbook's no-dry-run warning on `npm run db:migrate`. Two `investigate` findings got code fixes instead of new rows: `/api/v1/ai/text-to-3d` 502s are the same NVIDIA NVCF free-tier capacity flakiness `three-ws-ai-providers-unavailable-5xx` already covers (rows 6/20), its matcher widened to include that route; `/api/pump/trending` and `/api/v1/pump/trending` 502s are a genuine but self-healing double-upstream outage (Birdeye and the pump.fun board both down in the same window, stale-cache fallback also exhausted), given a new self-healing signature so the monitor stops flagging it as `investigate`. Both fixes are committed on `evolve` (`90628d426`) with passing tests, pending the normal fast-forward to `main`. Rows 24 (`rpc_lanes: degraded, both paid lanes exhausted`) and 36 (`object_storage: down`, R2 "Please enable R2 through the Cloudflare Dashboard") re-confirmed unchanged against live `/api/healthz`; no new rows, no edits needed to either. **Re-measured 2026-10-23 by the triage-lane deep sweep:** production still serves `45c768d07` on `three-ws-api-00500-b8q`. A fresh `git fetch threews main` followed by `git log --oneline 45c768d07..threews/main | wc -l` returns **535**, a sharp escalation from the 2-commit gap recorded 2026-10-14/15: the deploy window simply widened while nothing shipped, not a new regression. The `deep-db-migrations` probe lists 11 pending files (`20261011220000_agent_mention_policy.sql`, `20261011230000_x402_ring_scan_checked_at.sql`, `20261015010000_mcp_client_sessions_client_nullable.sql`, `20261015040000_forge_webhook_deliveries.sql`, `20261015050000_discord_activity_rooms.sql`, `20261015060000_shopify_app.sql`, `20261015070000_vroid_hub_source.sql`, `20261015080000_pump_fee_claims.sql`, `20261015090000_agent_coin_earnings_timeline.sql`, `20261015100000_payment_session_policy_upgrades.sql`, `20261015110000_x402_outbound_resolved.sql`), none applied, per the self-unblock playbook's no-dry-run warning on `npm run db:migrate`. This pass's `/api/chat` 503 storm (groq's `FALLBACK_SIBLINGS` pointing at the dead model id `llama-3.3-70b-versatile`) is the same deploy-lag story, confirmed by `git show 45c768d07:api/chat.js`: HEAD already carries the fix (`groq: ['openai/gpt-oss-20b']`), so no new code change is needed, it ships with this row. One `investigate` finding got a code fix instead of a new row: a lone `/api/pump/coin` 502 traced via app logs to pump.fun's own Cloudflare anti-bot challenge page for that mint, with no last-good cache yet for it, same shape as the existing `pump-trending-upstreams-502` signature; given its own self-healing signature (`pump-coin-lookup-upstream-502`) so the monitor stops flagging it, committed on `evolve` (`72b600f1c`) with a passing test. The same 9 failing pages flagged 2026-10-14/15 (`/docs/vroid-hub-import`, `/docs/wordpress-embed`, `/import/vroid`, `/open-source`, `/video-to-motion`, `/docs/forge-workflows`, `/docs/standalone-repos`, `/docs/team-chat`, `/docs/tutorials/grok-bot-recipes`) still fail the same way; re-checked each individually with `git merge-base --is-ancestor <adding-commit> 45c768d07` and all 9 fail the check, confirming deploy lag again, not a new or regressed routing bug. Rows 2, 3, 4, 5, 6, 19, 23, 24, 36 and 47 re-confirmed unchanged against live `/api/healthz` and service env; no new rows, no edits needed to any of them. | one yes | Shipping whatever has landed on `main` since 2026-09-08, now including the security-hardening batch and the forge image-lane breaker fix | [01-ship-readiness.md](../901-production-100-01-ship-readiness.md) |
| 2 | Fund the settle sponsor wallet `WwwuGbqHrwF5RG89KhUbmRWEvjnRH9k5kVM5p7T3WwW`. Healthz reports `x402_settle` down with `cause: sponsor_floor` (settle 5.9%, 3 of 51 paid attempts over 3 hours, 329 `no_solana_accept`) and the sniper worker out of capital (0.2307 SOL to refill its wallets, funding master holds 0.0250 SOL); no config change moves either. Re-read 2026-09-04 08:21 UTC against mainnet: the sponsor holds **0.001507661 SOL** (1,507,661 lamports), still below the settle floor, so the Solana accept stays withdrawn from every 402 challenge. Two months of re-measurement have only ever moved this number down; nothing on this machine can move it up. **Re-measured 2026-10-08 17:05 UTC:** the sponsor holds **651,828 lamports (0.000652 SOL)**, below rent-exempt (890,880), and healthz reads `x402_settle` down at 0.0% (0 of 32 paid attempts in 3 hours, 58 settles refused at the fee-wallet floor). Because the sponsor cannot co-sign, every 402 challenge advertises the Solana accept without `feePayer`, and the ring skips it: 151 `missing_fee_payer` skips in 24 hours on the MCP streaming health probe alone. Last successful settle: 2026-10-03 19:24 UTC. **Row 1's deploy alone restores the ring, no funding needed** (diagnosed 2026-10-08 by read-only simulation): the treasury pays its own sweep fee (5,001 lamports; it holds 0.060 SOL, about 11,800 sweeps) and moves its stranded 2.587 USDC back to the ring payer, which holds 341 atomics today, and ring settles are self-pay. What still needs this wallet: the sponsor-mode callers (the loop's inline entries and the MCP streaming health probe) and the facilitator's `sponsor_fee_unfunded` verify rejects. Clearing the 2,000,000-lamport floor takes at least **1,348,172 lamports**. **Re-measured 2026-10-09 03:05 UTC:** still **651,828 lamports**, unchanged; healthz reads `x402_settle` down at 0.0% (0 of 39 paid attempts in 3 hours, 37 `sweep_broadcast_failed`, 53 settles refused at the fee-wallet floor). The sweep failures are the ones row 1's deploy clears (`a9c996e72`). **Re-measured 2026-10-14 by the triage-lane sweep (production health sweep order):** sponsor still holds **651,828 lamports**, unchanged in five days; nothing on this machine moves it. But `x402_settle` itself is no longer down: healthz now reads `degraded` at **89.8% (149/166 paid attempts, 3 hours)**, and the 17 failures are `http_503×12, rpc_error×4, rpc_preflight_failed×1`, not `sponsor_floor` or `missing_fee_payer`. Row 1's self-pay fix did restore the ring as diagnosed above; the remaining ~10% rail faults trace to row 24's exhausted paid RPC lanes (`rpc_lanes: degraded, all 2 paid lanes exhausted`), a separate billing decision, not this wallet. This wallet is still needed for the sponsor-mode callers row 1's self-pay fix does not cover, so the funding ask stands, just smaller in blast radius than it was. | ~1 SOL | x402 settle recovery, backlog 01 retirement, sniper | [../backlog/PROGRESS.md](backlog-PROGRESS.md), 2026-08-02 and 2026-09-01 entries |
| 3 | Decide the stranded customer funds: customers hold about 0.35 SOL behind the retired encryption key and cannot withdraw. Credit (recommended), contact first, or write off. The brief now exists with the measurement, the impossibility argument, the cost of each option and the exact commands: **[docs/ops/stranded-wallets.md](../../docs/ops/stranded-wallets.md)**. Measured 2026-09-02 from production's own records: the platform side is exactly two bot wallets holding 0.142875505 SOL (the reclaim cron has failed on them ~35,700 times each, most recently today), and one customer agent is confirmed by a `wallet_key_retired` withdraw failure (`GemVS5fT958FKRe5fpgizohUYUKE8cUDueEdmB1bmXnm`, 0.250001 SOL). **Both customer wallets are now named** (2026-09-04): the keyed audit was run against production, so this row no longer waits on anything. `GemVS5fT958FKRe5fpgizohUYUKE8cUDueEdmB1bmXnm` holds 0.250001 SOL and `HPL1LfuTdYDwtzJDzsnrmR2ngrrQwLTQyxJszCC4DHsN` holds 0.100001 SOL, 0.350002 SOL across exactly two wallet-auth accounts. Nine of 735 custodial wallets are sealed; the other five customer ones hold nothing. The brief carries the full table. Every input the decision needs is measured. | decision | Closing the support obligation | [docs/ops/stranded-wallets.md](../../docs/ops/stranded-wallets.md), [../backlog/PROGRESS.md](backlog-PROGRESS.md) |
| 4 | Rotate or explicitly accept three credentials, now that the storage half is done. **The config work shipped 2026-09-02**: the economy master key and 58 other plaintext credentials are Secret Manager references on `three-ws-api-00407-m7c`, zero plaintext credentials remain, healthz verified no worse and the master's signing path verified live. What is left is a judgment only the owner can make. (a) The master key was readable in the service config for some window, so decide whether to rotate it or accept that exposure, knowing who held project viewer access in that period; the exact owner-gated commands are in the runbook. (b) `WALLET_ENCRYPTION_KEY` and `JWT_SECRET` hold the SAME value on production, which passes the dedicated-key guard in `secret-box.js` while defeating what it guards; changing it is a re-key with real stranding risk, so it is not an agent's call. (c) `X402_SEED_SOLANA_SECRET_BASE58` holds the same value as `LAUNCHER_MASTER_SECRET_KEY_B64` though `scripts/wire-master-wallet.mjs` assigns those slots to different wallets; decide which is correct. | decision (+ one fund move if rotating) | 03 is otherwise closed | [docs/ops/wallet-key-migration.md](../../docs/ops/wallet-key-migration.md), 2026-09-02 section |