Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

fix(vision): floor the lane timeout so the fallback rung can send at all

nirholasbdea4fc55044
diff --git a/api/_lib/vision.js b/api/_lib/vision.js
index cef48e976..6b9bf8b73 100644
--- a/api/_lib/vision.js
+++ b/api/_lib/vision.js
@@ -44,12 +44,17 @@ import {
vertexGeminiHeaders,
} from './vertex-gemini.js';
-// Free NIM vision lanes, in order. nemotron-nano carries the smallest image
-// token footprint (~281 prompt tokens for a tiny image vs ~1600 for llama-90B);
-// llama-3.2-11b is a different model family, so its failure modes are
-// independent — a real second lane, not a re-roll of the first.
+// Free NIM vision lanes, in order.
+//
+// `nvidia/nemotron-nano-12b-v2-vl` used to lead here for its small image token
+// footprint. NVIDIA retired it: the host answers every request for it with a
+// hard 410 ("has reached its end of life on 2026-08-26T09:00:00Z"), verified
+// against production on 2026-09-08. A 410 never recovers, so leaving it in the
+// list only spent the first, largest slice of every vision request's deadline
+// on a model that cannot answer. Adding a replacement free NIM VLM needs a live
+// NVIDIA_API_KEY to probe the catalog against, which is why this is a removal
+// rather than a swap.
const NVIDIA_VISION_MODELS = [
- 'nvidia/nemotron-nano-12b-v2-vl',
'meta/llama-3.2-11b-vision-instruct',
];
// Paid last-resort tail. gpt-5.4-nano is vision-capable and already priced in
@@ -76,11 +81,11 @@ const OPENAI_VISION_MODEL = 'gpt-5.4-nano';
// remaining budget across the lanes that are still to come guarantees
// every rung a real attempt, which is the only reason a chain exists.
//
-// The two free NIM rungs share one host, so a transport failure or a 429 there
-// is a statement about the HOST, not the model: cooling only the model that
-// happened to be asked would send the very next request to its twin on the same
-// sick host. Model-specific rejections (a 404 for a retired model id, a 400)
-// cool just that lane.
+// Free NIM rungs share one host, so a transport failure or a 429 there is a
+// statement about the HOST, not the model: cooling only the model that happened
+// to be asked would send the very next request to a sibling on the same sick
+// host. Model-specific rejections (a 404 for a retired model id, a 400) cool
+// just that lane, and a 410 parks it for the long window (see below).
const VISION_LANE_COOLDOWN_SECONDS = 45;
// Below this a lane cannot complete a VLM call, so handing it a smaller slice
// only burns budget the next rung could have used.
@@ -132,7 +137,16 @@ export function laneAttemptTimeout(remainingMs, lanesLeft, timeoutMs) {
const share = remainingMs / Math.max(1, lanesLeft);
// The floor may exceed the share when the budget is nearly spent; capping it
// by what is actually left keeps the attempt inside the deadline either way.
- return Math.max(1, Math.min(timeoutMs, remainingMs, Math.max(MIN_LANE_ATTEMPT_MS, share)));
+ //
+ // Math.floor is load-bearing, not cosmetic: this value is handed straight to
+ // AbortSignal.timeout(), which rejects a non-integer delay with
+ // ERR_OUT_OF_RANGE. The division above produces an integer only when the
+ // remaining budget happens to divide evenly by the lane count, so in
+ // production the fallback rung usually threw before it sent a single byte
+ // and was recorded as "unreachable" ("The value of \"delay\" is out of range.
+ // It must be an integer. Received 7924.333333333333"). That silently cost the
+ // chain the very rung it exists for, on every request that reached it.
+ return Math.floor(Math.max(1, Math.min(timeoutMs, remainingMs, Math.max(MIN_LANE_ATTEMPT_MS, share))));
}
/**
@@ -465,7 +479,11 @@ export async function describeImage({
// every rung sharing it. Anything else (a 404 for a retired model id, a
// 400) is specific to this model and cools this lane alone.
const st = upstream.status;
- const authFault = st === 401 || st === 403 || st === 402;
+ // 410 Gone is how NIM reports a retired model id. Unlike a 404 or a 500 it
+ // is a permanent verdict about this model, so it parks the lane for the
+ // long window instead of being re-probed (and re-charged a slice of the
+ // deadline) on every request until someone notices.
+ const authFault = st === 401 || st === 403 || st === 402 || st === 410;
// A 429 is the one verdict that is unambiguously about the HOST and the
// account behind it: every model served there is throttled by the same
// quota, so both the bench and the in-request skip cover every sibling
diff --git a/tests/api/vision-lane-health.test.js b/tests/api/vision-lane-health.test.js
index ef93f3e5a..6867b47df 100644
--- a/tests/api/vision-lane-health.test.js
+++ b/tests/api/vision-lane-health.test.js
@@ -16,7 +16,6 @@ const ENV_KEYS = ['NVIDIA_API_KEY', 'OPENAI_API_KEY', 'GOOGLE_CLOUD_PROJECT'];
const ORIGINAL_ENV = Object.fromEntries(ENV_KEYS.map((k) => [k, process.env[k]]));
const LANE_KEYS = [
- 'vision:nvidia:nvidia/nemotron-nano-12b-v2-vl',
'vision:nvidia:meta/llama-3.2-11b-vision-instruct',
'vision:openai:gpt-5.4-nano',
];
@@ -75,6 +74,23 @@ describe('laneAttemptTimeout', () => {
it('leaves the caller timeout alone when there is no deadline', () => {
expect(laneAttemptTimeout(Infinity, 3, 20_000)).toBe(20_000);
});
+ it('always returns an integer AbortSignal.timeout will accept', () => {
+ // Every case above divides evenly, which is how a fractional return value
+ // shipped: AbortSignal.timeout() throws ERR_OUT_OF_RANGE on a non-integer
+ // delay, so the lane threw before sending a byte and the chain lost its
+ // fallback rung. 23_474/3 is the shape production actually produced.
+ const split = laneAttemptTimeout(23_474, 3, 20_000);
+ expect(Number.isInteger(split)).toBe(true);
+ expect(split).toBe(7_824);
+ expect(() => AbortSignal.timeout(split)).not.toThrow();
+ for (const remaining of [23_474, 9_999, 12_345, 7_001]) {
+ for (const lanes of [1, 2, 3, 4, 7]) {
+ const ms = laneAttemptTimeout(remaining, lanes, 20_000);
+ expect(Number.isInteger(ms)).toBe(true);
+ expect(() => AbortSignal.timeout(ms)).not.toThrow();
+ }
+ }
+ });
});
describe('inlineImageBudget', () => {
@@ -98,8 +114,8 @@ describe('vision lane cooldowns', () => {
const a = await describeImage({ prompt: 'p', imageBase64: 'AAAA', mimeType: 'image/png' });
expect(a.provider).toBe('openai');
// 429 is the one verdict that is unambiguously about the host and its
- // quota, so the sibling rung is skipped inside this very request: one
- // throttled attempt is paid, not one per model the host serves.
+ // quota, so every rung sharing that host is skipped inside this very
+ // request: one throttled attempt is paid, not one per model it serves.
expect(first.filter((c) => c.url.includes('nvidia')).length).toBe(1);
const second = stubFetch([
@@ -112,32 +128,39 @@ describe('vision lane cooldowns', () => {
expect(second[0].url).toContain('openai.com');
});
- it('keeps trying the twin rung on a transport failure, which may be one slow model', async () => {
- let nimCalls = 0;
+ it('fails a transport error over to the backstop rather than giving up', async () => {
const calls = stubFetch([
- ['integrate.api.nvidia.com', () => {
- if (++nimCalls === 1) throw Object.assign(new Error('aborted'), { name: 'TimeoutError' });
- return chatOk('twin answered');
- }],
+ ['integrate.api.nvidia.com', () => { throw Object.assign(new Error('aborted'), { name: 'TimeoutError' }); }],
+ ['api.openai.com', () => chatOk('backstop answered')],
]);
const r = await describeImage({ prompt: 'p', imageBase64: 'AAAA', mimeType: 'image/png' });
- expect(r.text).toBe('twin answered');
- expect(calls.filter((c) => c.url.includes('nvidia')).length).toBe(2);
+ expect(r.text).toBe('backstop answered');
+ expect(r.provider).toBe('openai');
+ // The NIM rung was really attempted before the failover: a chain that skips
+ // its free lane and bills the paid one is the regression worth catching.
+ expect(calls.filter((c) => c.url.includes('nvidia')).length).toBe(1);
});
- it('benches only the failing model on a 5xx, keeping its twin on the same host', async () => {
- let nimCalls = 0;
+ it('parks a 410 lane for the long window instead of re-probing a retired model', async () => {
+ // NVIDIA answers a retired model id with 410 Gone. That verdict never
+ // reverses, so the lane must be benched for the auth-length window, not the
+ // 45s health one, or every request keeps paying it a slice of the deadline.
stubFetch([
- ['integrate.api.nvidia.com', () => (++nimCalls === 1 ? httpErr(500, 'boom') : chatOk('twin'))],
+ ['integrate.api.nvidia.com', () => httpErr(410, "model has reached its end of life")],
+ ['api.openai.com', () => chatOk('backstop')],
]);
const a = await describeImage({ prompt: 'p', imageBase64: 'AAAA', mimeType: 'image/png' });
- expect(a.provider).toBe('nvidia');
+ expect(a.provider).toBe('openai');
- // Next request still leads with a NIM lane, because a model-level 500 is no
- // evidence against the host or against the sibling model.
- const next = stubFetch([['integrate.api.nvidia.com', () => chatOk('ok')]]);
- await describeImage({ prompt: 'p', imageBase64: 'AAAA', mimeType: 'image/png' });
- expect(next[0].url).toContain('integrate.api.nvidia.com');
+ // The retired lane is now behind the healthy one, so the next request does
+ // not lead with it.
+ const next = stubFetch([
+ ['integrate.api.nvidia.com', () => httpErr(410, 'gone')],
+ ['api.openai.com', () => chatOk('backstop again')],
+ ]);
+ const b = await describeImage({ prompt: 'p', imageBase64: 'AAAA', mimeType: 'image/png' });
+ expect(b.provider).toBe('openai');
+ expect(next[0].url).toContain('openai.com');
});
it('clears a lane cooldown as soon as that lane serves a real request', async () => {