Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

docs(owner-actions): re-measure the deploy gap and dry sponsor, add the exhausted Solana RPC credits row

nirholasb374c9713459
diff --git a/prompts/finish/_context/production-100-OWNER-ACTIONS.md b/prompts/finish/_context/production-100-OWNER-ACTIONS.md
index 9ed91ed87..ceef7fa88 100644
--- a/prompts/finish/_context/production-100-OWNER-ACTIONS.md
+++ b/prompts/finish/_context/production-100-OWNER-ACTIONS.md
@@ -9,12 +9,12 @@ it and, if that unblocks a work order, say which one in the owning pack's `produ
Rows carry the date they were added so staleness is visible. Re-verify a row before acting
on it; some resolve themselves as other work lands.
-## Open rows (2026-09-30: row 23 added for the $THREE receiver wallets; 2026-09-29: row 19 re-measured after the agent-sniper deploy; 2026-09-09: row 19 added, and row 1 corrected because its outage premise no longer holds; rows 1, 2, 3, 9, 13, 15 and 16 re-measured 2026-09-04; the former row 18 was cleared that day and its number is not reused; the 2026-09-02 and 2026-08-09 passes are folded in below)
+## Open rows (2026-10-08: rows 1 and 2 re-measured by the triage sweep, row 24 added for the paid Solana RPC credits; 2026-09-30: row 23 added for the $THREE receiver wallets; 2026-09-29: row 19 re-measured after the agent-sniper deploy; 2026-09-09: row 19 added, and row 1 corrected because its outage premise no longer holds; rows 1, 2, 3, 9, 13, 15 and 16 re-measured 2026-09-04; the former row 18 was cleared that day and its number is not reused; the 2026-09-02 and 2026-08-09 passes are folded in below)
| # | Action | Size | Unblocks | Detail lives in |
|---|---|---|---|---|
-| 1 | Approve the production deploy (or say "get production working", which is itself the approval). **The outage this row was written for is CLEARED.** Re-measured 2026-09-09 from the live origin: production serves `880bdcef8` on revision `three-ws-api-00420-ljh`, built 2026-09-08 19:05 UTC, and the `ERR_MODULE_NOT_FOUND` on `/app/services/home-relay/src/token.js` is gone: `/api/healthz` answers **HTTP 200** and `/api/home/status` and `/api/home/connections` answer 401 (auth required) rather than 500. What remains is ordinary staleness, 182 commits behind `main`, which is a routine cadence question and no longer a platform-health emergency. The 2026-09-04 text of this row, which described 96 commits and a 500ing healthz, is superseded. | one yes | Shipping whatever has landed on `main` since 2026-09-08 | [01-ship-readiness.md](../901-production-100-01-ship-readiness.md) |
-| 2 | Fund the settle sponsor wallet `WwwuGbqHrwF5RG89KhUbmRWEvjnRH9k5kVM5p7T3WwW`. Healthz reports `x402_settle` down with `cause: sponsor_floor` (settle 5.9%, 3 of 51 paid attempts over 3 hours, 329 `no_solana_accept`) and the sniper worker out of capital (0.2307 SOL to refill its wallets, funding master holds 0.0250 SOL); no config change moves either. Re-read 2026-09-04 08:21 UTC against mainnet: the sponsor holds **0.001507661 SOL** (1,507,661 lamports), still below the settle floor, so the Solana accept stays withdrawn from every 402 challenge. Two months of re-measurement have only ever moved this number down; nothing on this machine can move it up. | ~1 SOL | x402 settle recovery, backlog 01 retirement, sniper | [../backlog/PROGRESS.md](backlog-PROGRESS.md), 2026-08-02 and 2026-09-01 entries |
+| 1 | Approve the production deploy (or say "get production working", which is itself the approval). **The outage this row was written for is CLEARED.** Re-measured 2026-09-09 from the live origin: production serves `880bdcef8` on revision `three-ws-api-00420-ljh`, built 2026-09-08 19:05 UTC, and the `ERR_MODULE_NOT_FOUND` on `/app/services/home-relay/src/token.js` is gone: `/api/healthz` answers **HTTP 200** and `/api/home/status` and `/api/home/connections` answer 401 (auth required) rather than 500. What remains is ordinary staleness, 182 commits behind `main`, which is a routine cadence question and no longer a platform-health emergency. The 2026-09-04 text of this row, which described 96 commits and a 500ing healthz, is superseded. **Re-measured 2026-10-08 17:05 UTC:** production serves `fe2a8b24f` on `three-ws-api-00479-d7z`, **146 commits behind `main`, 90 of them fixes** (security, payouts, SSRF and XSS among them). Shipping them clears four live findings at once: the ring sweep's fee falls back to the treasury instead of the dry sponsor (`a9c996e72`, about 30 failed sweeps an hour since 2026-10-03), an EVM testnet identity-index cursor frozen since 2026-09-11 (`21fa42bb5`), two pages and their two docs pages advertised in the sitemap but answering 404 (added in `16f9d07f4`), and `/tty` holding its animation until the end for any client that accepts compression (`03b4f1189`). One migration must go with it: `20261008150000_revenue_direct_settle_backfill.sql` (pending; it matches 0 rows today, so applying it is safe, and the code half is what keeps the payout cron from paying a direct-to-seller sale twice). Commands, in order: `npm run db:status`, `npm run db:migrate`, `npm run deploy:gcp:full`. | one yes | Shipping whatever has landed on `main` since 2026-09-08 | [01-ship-readiness.md](../901-production-100-01-ship-readiness.md) |
+| 2 | Fund the settle sponsor wallet `WwwuGbqHrwF5RG89KhUbmRWEvjnRH9k5kVM5p7T3WwW`. Healthz reports `x402_settle` down with `cause: sponsor_floor` (settle 5.9%, 3 of 51 paid attempts over 3 hours, 329 `no_solana_accept`) and the sniper worker out of capital (0.2307 SOL to refill its wallets, funding master holds 0.0250 SOL); no config change moves either. Re-read 2026-09-04 08:21 UTC against mainnet: the sponsor holds **0.001507661 SOL** (1,507,661 lamports), still below the settle floor, so the Solana accept stays withdrawn from every 402 challenge. Two months of re-measurement have only ever moved this number down; nothing on this machine can move it up. **Re-measured 2026-10-08 17:05 UTC:** the sponsor holds **651,828 lamports (0.000652 SOL)**, below rent-exempt (890,880), and healthz reads `x402_settle` down at 0.0% (0 of 32 paid attempts in 3 hours, 58 settles refused at the fee-wallet floor). Because the sponsor cannot co-sign, every 402 challenge advertises the Solana accept without `feePayer`, and the ring skips it: 151 `missing_fee_payer` skips in 24 hours on the MCP streaming health probe alone. Row 1's deploy moves the ring sweep's fee onto the treasury (0.060 SOL), but sponsor-mode settles stay withdrawn until this wallet is funded. | ~1 SOL | x402 settle recovery, backlog 01 retirement, sniper | [../backlog/PROGRESS.md](backlog-PROGRESS.md), 2026-08-02 and 2026-09-01 entries |
| 3 | Decide the stranded customer funds: customers hold about 0.35 SOL behind the retired encryption key and cannot withdraw. Credit (recommended), contact first, or write off. The brief now exists with the measurement, the impossibility argument, the cost of each option and the exact commands: **[docs/ops/stranded-wallets.md](../../docs/ops/stranded-wallets.md)**. Measured 2026-09-02 from production's own records: the platform side is exactly two bot wallets holding 0.142875505 SOL (the reclaim cron has failed on them ~35,700 times each, most recently today), and one customer agent is confirmed by a `wallet_key_retired` withdraw failure (`GemVS5fT958FKRe5fpgizohUYUKE8cUDueEdmB1bmXnm`, 0.250001 SOL). **Both customer wallets are now named** (2026-09-04): the keyed audit was run against production, so this row no longer waits on anything. `GemVS5fT958FKRe5fpgizohUYUKE8cUDueEdmB1bmXnm` holds 0.250001 SOL and `HPL1LfuTdYDwtzJDzsnrmR2ngrrQwLTQyxJszCC4DHsN` holds 0.100001 SOL, 0.350002 SOL across exactly two wallet-auth accounts. Nine of 735 custodial wallets are sealed; the other five customer ones hold nothing. The brief carries the full table. Every input the decision needs is measured. | decision | Closing the support obligation | [docs/ops/stranded-wallets.md](../../docs/ops/stranded-wallets.md), [../backlog/PROGRESS.md](backlog-PROGRESS.md) |
| 4 | Rotate or explicitly accept three credentials, now that the storage half is done. **The config work shipped 2026-09-02**: the economy master key and 58 other plaintext credentials are Secret Manager references on `three-ws-api-00407-m7c`, zero plaintext credentials remain, healthz verified no worse and the master's signing path verified live. What is left is a judgment only the owner can make. (a) The master key was readable in the service config for some window, so decide whether to rotate it or accept that exposure, knowing who held project viewer access in that period; the exact owner-gated commands are in the runbook. (b) `WALLET_ENCRYPTION_KEY` and `JWT_SECRET` hold the SAME value on production, which passes the dedicated-key guard in `secret-box.js` while defeating what it guards; changing it is a re-key with real stranding risk, so it is not an agent's call. (c) `X402_SEED_SOLANA_SECRET_BASE58` holds the same value as `LAUNCHER_MASTER_SECRET_KEY_B64` though `scripts/wire-master-wallet.mjs` assigns those slots to different wallets; decide which is correct. | decision (+ one fund move if rotating) | 03 is otherwise closed | [docs/ops/wallet-key-migration.md](../../docs/ops/wallet-key-migration.md), 2026-09-02 section |
| 5 | Accept the Anthropic terms in Vertex Model Garden for `aerial-vehicle-466722-p5`, then an agent flips `VERTEX_CLAUDE_ENABLED=1` after a live probe. | minutes | First-party Claude lane | `docs/ops/llm-lanes.md` |
@@ -35,6 +35,7 @@ on it; some resolve themselves as other work lands.
| 21 | **Decide whether to rotate the lane credentials read on 2026-09-09.** Resolving the LLM chain for the fact-check benchmark printed secret material into an agent session transcript: the `vercel-inference@aerial-vehicle-466722-p5` service-account **private key** (`GCP_SERVICE_ACCOUNT_JSON`, key id `93be907a...`), plus `GROQ_API_KEY`, all five OpenRouter keys, `NVIDIA_API_KEY` and `OPENAI_API_KEY`. `INTERNAL_API_KEY` was used but never printed. Nothing indicates misuse and all of them remain valid; the exposure surface is whoever can read that transcript. Rotate or accept, and note that the OpenAI key is dead anyway (row 6) and OpenRouter key #1 is already revoked. | decision (+ minutes if rotating) | Closing out the exposure | [PROGRESS.md](production-100-PROGRESS.md), 2026-09-09 entry |
| 22 | **Decide whether to arm the three trading growth programs** (added 2026-09-29). Built, tested and disarmed: the big-win X poster (`BIG_WIN_X_ENABLED`, posts to @trythreews), the first-rug softener (`RUG_SOFTENER_ENABLED`, pays $THREE, default cap $25 per claim and $100 per day) and the early-leader bonus (`EARLY_LEADER_ENABLED`, pays $THREE, default 20 slots at $50 and $250 per week). Each is armed on its own. Steps: (1) `npm run db:status`, then `npm run db:migrate` for `20260929120000_growth_program_payouts.sql` (payout programs only); (2) set `THREE_PRIZE_PAYOUT_KEY` (unset on production as of 2026-09-29; the programs deliberately refuse the club treasury fallback, which is the economy master wallet) to a dedicated wallet holding the $THREE to cover the caps (payout programs only); (3) deploy; (4) read the dry run at `/api/ops/growth-programs`; (5) `gcloud run services update three-ws-api --region us-central1 --update-env-vars <FLAG>=1`. Measured 2026-09-29 against production data: nothing would be posted or paid today (no live win cleared the 0.1 SOL size gate, no first trade met the softener bar, no leader has a profitable 30-day record with followers). | one yes per program | Roadmap 917 items 3.8, 12.15, 13.4 | [docs/growth-programs.md](../../docs/growth-programs.md) |
| 23 | **Name the two $THREE receiver wallets** (added 2026-09-30). `/api/healthz` reports `three_token_rail` down and signed-in `POST /api/token/quote` answers `503 treasury_unavailable`, so no $THREE-priced purchase can be priced (Forge paid tiers, spins, scarcity mints, token marketplace sales, copy-trading fees). The quote signing key is done (`three-quote-secret`, wired on `three-ws-api-00466-85g`); only `THREE_TREASURY_WALLET` and `THREE_REWARDS_WALLET` remain. No revision ever held them, so there is nothing to restore. Candidates: treasury `wwwwwDxFWRn7grgr3Esrsg5C6NvDoDHSA4gaCffccrU` (x402 receiver, already holds a $THREE account), rewards `WwwuGbqHrwF5RG89KhUbmRWEvjnRH9k5kVM5p7T3WwW` (the economy master, which `REWARDS_DISTRIBUTOR_SECRET` signs as). The rewards candidate is also the buyback signer, so arming `THREE_BUYBACK_ENABLED` or `THREE_MICROBUY_ENABLED` would sweep the whole rewards pool into the treasury; decide that before arming either lane. Setting the two vars moves no funds by itself: `gcloud run services update three-ws-api --region us-central1 --update-env-vars THREE_TREASURY_WALLET=<treasury>,THREE_REWARDS_WALLET=<rewards>`. | two addresses | The whole $THREE purchase rail | [docs/money-map.md](../../docs/money-map.md#three-split-receivers-inbound-three) |
+| 24 | **Top up the paid Solana RPC plans** (added 2026-10-08). Helius answers `429: max usage reached` on every call: 45,461 such lines on `three-ws-api` in the 24 hours to 2026-10-08 17:10 UTC, the oldest retained one from 2026-09-08. healthz reads `helius` degraded and `rpc_lanes` "all 2 paid lanes exhausted", so Solana traffic runs on throttled free nodes. That is the whole cause of `agent_index` reading down: 172 of the 176 erroring Solana cursors carry `solana rpc 429 @ public.rpc.solanavibestation.com`, the last free lane, and the rest are a free provider's own errors. No config change adds credits, and promoting the reserve lane is the documented wrong fix. Upgrade or top up the Helius plan (and the second paid lane), then confirm `helius` and `rpc_lanes` read ok on `curl -s https://three.ws/api/healthz`. | billing | Solana agent index freshness, balance reads, every on-chain route's latency | [docs/ops/production-log-triage.md](../../docs/ops/production-log-triage.md), "Every paid Solana RPC lane exhausted at once" |
Deleted 2026-09-01: the former row 5 (`WALLET_ENCRYPTION_KEY_PREVIOUS` policy), which
declared itself "delete on reading" and had outlived that by three weeks.