Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

docs(fact-check): record why the mixed run measured but cannot be published

nirholas01009b41672a
diff --git a/prompts/finish/_context/production-100-OWNER-ACTIONS.md b/prompts/finish/_context/production-100-OWNER-ACTIONS.md
index 4fff3f68d..364e79ee1 100644
--- a/prompts/finish/_context/production-100-OWNER-ACTIONS.md
+++ b/prompts/finish/_context/production-100-OWNER-ACTIONS.md
@@ -31,6 +31,8 @@ on it; some resolve themselves as other work lands.
| 16 | Create the `three-ws` GitHub organization and `three-ws/examples` (public), then run the push command `npm run export:satellites` prints. The export now stages 72 files and passes all four smoke stages online (registry, install, links); re-probed 2026-09-03, both the organization and the repository still answer 404. | minutes | roadmap developer-resources retirement | [docs/ops/examples-repo-export.md](../../docs/ops/examples-repo-export.md) |
| 17 | Apple Developer account. The WidgetKit extension, the Mac menu bar app and the iOS target are written and guarded (`npm run check:apple-widget`); the account is what signs them, for a Developer ID Mac build and an App Store iOS build alike. Separately, publish the Android 1.1.0 build (versionCode 2) to Play. | account + minutes | roadmap native-widgets task 4 and its release | [../roadmap/native-widgets.md](../916-roadmap-native-widgets.md) |
| 19 | Deploy the `agent-sniper` worker, then decide separately whether to arm the Risk Officer. Measured 2026-09-09 against Cloud Run and mainnet: the service runs revision `agent-sniper-00033-rg6`, whose image was built **2026-08-11**, while it trades `SNIPER_MODE=live` right now (heartbeat 6s old, 11 strategies, 3,482 positions, one opened 05:10 UTC that morning). **48 files and about 11,000 lines under `workers/agent-sniper/` are committed and not in that image**, including `amm-exit.js`, which is the roadmap's own named blocker: `executor.js`, `positions.js` and `graduation-ride.js` import it at HEAD, so the running image has no code path that exits a position after its coin graduates. The Risk Officer (`risk-officer.js`, migration already applied in production) has therefore never run in production either: `sniper_risk_reviews` holds 7 rows, all from the 2026-09-04 local verification, and the shadow-evidence query in `workers/agent-sniper/README.md` runs green against the live schema and returns zero rows. The deploy is one command, `npm run deploy:sniper`, dry-run clean on 2026-09-09 (gcloud authed, runtime SA and Artifact Registry repo present, both required secrets present); on an existing service it rolls `gcloud run services update --image`, which preserves the running env, so it does **not** demote the fleet to simulate. Only the optional `telegram-alerts-chat-id` secret is missing, which disables Telegram ops alerting. Arming enforcement is a **separate, later** yes: `SNIPER_RISK_OFFICER=enforce` fleet-wide (via `--update-env-vars`, never `--set-env-vars`) or per strategy via `risk_officer_level`, and it should wait until shadow rows exist to justify it. | one yes now, one decision later | The trading Arena copy-trade loop (graduated-position exits), and all shadow evidence for the Risk Officer | [../918-roadmap-pumpfun-trading-arena.md](../918-roadmap-pumpfun-trading-arena.md), [workers/agent-sniper/README.md](../../workers/agent-sniper/README.md) |
+| 20 | **Clear the GCP billing hold on `aerial-vehicle-466722-p5`.** Every Vertex surface answers `403 Lightning dunning decision is deny for project: projects/93741856042`, confirmed 2026-09-09 both locally and in production's own logs (Vertex Imagen falling back at 05:33 UTC). This is not IAM and nothing in code routes around it: it takes out the Vertex Gemini anchor, Imagen, and `groundedSearch`, which is the fact-checker's ONLY working search rung (no Brave/Tavily/Exa/Serper key exists). With it down the chain falls to Wikipedia alone: a benchmark run that day saw 190 sources, 100% Wikipedia, and 15 of 38 claims had zero stance-bearing evidence. It is a billing/payment state on the account, so only the owner can clear it. | minutes, at the billing console | Grounded search, Vertex Gemini, Imagen, and publishing the fact-check accuracy run (04b) | [PROGRESS.md](production-100-PROGRESS.md), 2026-09-09 entry |
+| 21 | **Decide whether to rotate the lane credentials read on 2026-09-09.** Resolving the LLM chain for the fact-check benchmark printed secret material into an agent session transcript: the `vercel-inference@aerial-vehicle-466722-p5` service-account **private key** (`GCP_SERVICE_ACCOUNT_JSON`, key id `93be907a...`), plus `GROQ_API_KEY`, all five OpenRouter keys, `NVIDIA_API_KEY` and `OPENAI_API_KEY`. `INTERNAL_API_KEY` was used but never printed. Nothing indicates misuse and all of them remain valid; the exposure surface is whoever can read that transcript. Rotate or accept, and note that the OpenAI key is dead anyway (row 6) and OpenRouter key #1 is already revoked. | decision (+ minutes if rotating) | Closing out the exposure | [PROGRESS.md](production-100-PROGRESS.md), 2026-09-09 entry |
Deleted 2026-09-01: the former row 5 (`WALLET_ENCRYPTION_KEY_PREVIOUS` policy), which
declared itself "delete on reading" and had outlived that by three weeks.
diff --git a/prompts/finish/_context/production-100-PROGRESS.md b/prompts/finish/_context/production-100-PROGRESS.md
index fd61ae2bf..036febebf 100644
--- a/prompts/finish/_context/production-100-PROGRESS.md
+++ b/prompts/finish/_context/production-100-PROGRESS.md
@@ -599,6 +599,65 @@ whole of the first one and is dry-run proven. (2) Arming enforcement afterwards
real SOL buys and should wait for production shadow rows to justify it. Order 918 stays on
disk because of (1); nothing in it is unbuilt.
+## 2026-09-09: 04b fact-check benchmark (measured; NOT published, and the reason is the owner's)
+
+The `mixed` fix works. It is not publishable, because the search chain underneath it is
+down project-wide.
+
+**The fix is live and reachable.** Production runs `880bdcef8`, which carries both halves of
+the 2026-09-02 change; a live probe of `/api/x402/fact-check` through the service key came
+back with a `partial`-stance source on the first try, a stance the pre-fix rubric could not
+emit at all. The two unit files still pass (49 tests).
+
+**Per-class A/B against the published 2026-08-10 run** (in-process, cache disabled, real
+chain, 2/40 errors = 5%, inside the 10% ceiling, so the degradation guard passed it):
+
+| class | published 2026-08-10 | this run | delta |
+|---|---|---|---|
+| mixed | 0% | 10% | **+10** |
+| insufficient | 40% | 70% | +30 |
+| supported | 50% | 30% | -20 |
+| contradicted | 70% | 10% | -60 |
+| overall | 40% | 30% | -10 |
+
+`mixed` left zero, which is what the order set out to prove: the published run produced the
+`mixed` verdict **zero** times across all 40 claims (it appears nowhere in that confusion
+matrix), and this run produced it 6 times. The calculus is reachable from real evidence now.
+
+**Two classes regressed, so the run was NOT published, per the order's own rule.** The
+diagnosis is not the calculus. Every one of the 190 sources this run saw was Wikipedia, 100%
+of them, because `groundedSearch` answers **403 `Lightning dunning decision is deny for
+project: projects/93741856042`**: a project-wide GCP billing hold that denies every Vertex
+surface. Production hits the same wall (its own logs carry the same denial for Vertex Imagen
+minutes before this was written, and the live probe above returned five Wikipedia pages, one
+of them `Health_effects_of_tea` and one `Insurance`, for a claim about coffee). With no
+grounded rung the evidence is starved: **15 of 38 checked claims had ZERO stance-bearing
+sources** and the mean was 1.08, so `supported` and `contradicted` collapse into
+`insufficient` (6 of 10 each). That is an evidence-supply regression, not a verdict-quality
+one, and publishing 30% as the product's accuracy would describe a chain that exists only
+while the billing hold does. Left the published run alone.
+
+**One real fix shipped: the runner was throttling itself and scoring it as failure.** The
+first attempt was refused as degraded at 7/40 errors (17.5%), and every error was a 429
+sweep across all three groq rungs and all five OpenRouter keys, minutes after the same chain
+answered a probe in 225ms. A 40-claim run is 80 LLM turns; unspaced they land far inside the
+free lanes' per-minute allowances. In-process mode also had no retry, though the remote path
+has had one all along for exactly this reason. Added both (`FACT_CHECK_BENCH_SPACING_MS`,
+default 6000, plus a single 15s-backoff retry): errors fell 17.5% to 5% and the guard passed
+the run. `MAX_ERROR_RATE` was not touched.
+
+Left:
+- **Publishing is gated on the GCP billing hold** (OWNER-ACTIONS row 20). Nothing on this
+ machine can clear a dunning denial. Once it lifts, one command finishes this order:
+ `node scripts/fact-check-benchmark.mjs --in-process --publish` with the service env loaded,
+ then confirm `/fact-check` in a browser. `04b` stays on disk until then.
+- **Credential exposure to declare** (OWNER-ACTIONS row 21). Reading the lane keys printed
+ the `vercel-inference@` service-account private key and the groq / OpenRouter x5 / NVIDIA /
+ OpenAI keys into a session transcript. Rotation is the owner's call.
+- Rows 5 and 6 are re-confirmed live by this session's chain dump: `openrouter#2` still 401s
+ and `openai` still answers `429 billing_not_active`, so two of thirteen rungs are dead
+ weight on every single call.
+
## Retire this file when the campaign is done (required)
This file is shared context rather than a single order, so it outlives the