Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

fix(gcp-triage): classify /api/crypto/bonding 503 as self-healing, not investigate

nirholasa51f61927d2f
diff --git a/docs/ops/production-log-triage.md b/docs/ops/production-log-triage.md
index 089aa0199..fc90444af 100644
--- a/docs/ops/production-log-triage.md
+++ b/docs/ops/production-log-triage.md
@@ -940,6 +940,21 @@ ws error: Unexpected server response: 301
(self-healing). Escalate only if it persists after `rpc_lanes` reads ok;
[api/_lib/crypto-token-holders.js](../../api/_lib/crypto-token-holders.js)
is the composition logic if it does.
+- **Bonding-status symptom (2026-10-15 deep sweep):** `HTTP 503 GET
+ /api/crypto/bonding` answers `upstream_unavailable` when the pump.fun feed
+ is temporarily unreachable, the identical designed backoff contract
+ `getBondingStatus()` documents in its own handler comment ("Never 500:
+ answer 503 + Retry-After so the agent backs off and retries, per the
+ free-API contract"). A single hit with no matching app-level error log line
+ fell through to `investigate` every sweep before this signature was added.
+ The monitor now classifies it `bonding-upstream-unavailable-503`
+ (self-healing). Escalate only if it persists for the same mint after the
+ pump.fun public frontend itself is reachable again;
+ [api/crypto/bonding.js](../../api/crypto/bonding.js) and
+ [api/_lib/pump-bonding.js](../../api/_lib/pump-bonding.js) are the handler
+ and composition logic. Regression-covered in
+ [tests/gcp-triage.test.js](../../tests/gcp-triage.test.js) ("classifies a
+ bonding-curve upstream outage as self-healing").
## 🟡 HTTP timeout with no cron context
diff --git a/scripts/gcp-triage.mjs b/scripts/gcp-triage.mjs
index 1da80b2a9..0d8ee0c78 100644
--- a/scripts/gcp-triage.mjs
+++ b/scripts/gcp-triage.mjs
@@ -332,6 +332,18 @@ const KNOWN_HTTP_SIGNATURES = [
class: 'self-healing',
action: `The free holder-distribution route exhausted Helius and the keyless RPC fallback for a mint with no cached last-good report, so it answered an honest 503 upstream_unavailable rather than a guess. Check healthz rpc_lanes/helius: this fires in bursts while every paid Solana RPC lane is cooling down (see ${RUNBOOK} "Every paid Solana RPC lane exhausted at once"). Investigate only if it persists after rpc_lanes reads ok. ${RUNBOOK} §solana-rpc.`,
},
+ {
+ // 2026-10-15 deep sweep: api/crypto/bonding.js's own handler comment is
+ // explicit ("Never 500: answer 503 + Retry-After so the agent backs off
+ // and retries, per the free-API contract") for the identical
+ // getBondingStatus() upstream_down case the holders route above already
+ // tracks; a single hit with zero matching app-level error log lines is
+ // one pump.fun feed hiccup honestly reported, not a crash.
+ id: 'bonding-upstream-unavailable-503',
+ test: (g) => g.service === 'three-ws-api' && g.status === 503 && g.path === '/api/crypto/bonding',
+ class: 'self-healing',
+ action: `The free bonding-curve status route found the pump.fun feed temporarily unreachable and answered its designed 503 upstream_unavailable + Retry-After rather than a stale guess (api/crypto/bonding.js, api/_lib/pump-bonding.js). Investigate only if it persists for the same mint after the pump.fun public frontend itself is reachable again. ${RUNBOOK} §solana-rpc.`,
+ },
{
id: 'watsonx-unconfigured-503',
test: (g) => g.status === 503 && (g.path === '/api/galaxy' || g.path.startsWith('/api/galaxy/')),
diff --git a/tests/gcp-triage.test.js b/tests/gcp-triage.test.js
index 1bab53a27..5438842a6 100644
--- a/tests/gcp-triage.test.js
+++ b/tests/gcp-triage.test.js
@@ -121,6 +121,17 @@ describe('gcp-triage HTTP classification', () => {
expect(finding.class).toBe('self-healing');
});
+ // 2026-10-15 deep sweep: a single /api/crypto/bonding 503 with zero matching
+ // app-level error log lines fell through to `investigate` every sweep even
+ // though api/crypto/bonding.js's own handler comment documents the 503 as
+ // the designed free-API backoff contract (same upstream_down shape as
+ // holders-rpc-exhausted-503 above, different route).
+ it('classifies a bonding-curve upstream outage as self-healing', () => {
+ const [finding] = buildFindings([request({ status: 503, path: '/api/crypto/bonding' })]);
+ expect(finding.signature).toBe('bonding-upstream-unavailable-503');
+ expect(finding.class).toBe('self-healing');
+ });
+
// 2026-10-12 triage-lane sweep: confirmed live against production that
// /api/render/animate?avatar=<id> and GET /api/animations/clips/<id> both
// 502 with the exact same root cause as the rest of the r2-downstream-5xx