Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

fix(voice): pin the TTS lane-outage contract with tests and keep caller errors specific

nirholasc638addcea65
diff --git a/api/tts/synthesize.js b/api/tts/synthesize.js
index 323850e6c..e278bbae2 100644
--- a/api/tts/synthesize.js
+++ b/api/tts/synthesize.js
@@ -82,6 +82,10 @@ const STATUS_BY_CODE = {
*/
function userFacingLaneError(provider, err) {
switch (err?.code) {
+ // Our own validation prose ("text is required"), already written for a
+ // caller and specific in a way a generic sentence would throw away.
+ case 'invalid_argument':
+ return err.message;
case 'invalid_key':
case 'not_configured':
return (
@@ -163,14 +167,15 @@ export default wrap(async (req, res) => {
// the platform key's outage. A 'health' cooldown (throttle, blip) still lets
// an explicit request through, so a recovered lane proves itself immediately.
const laneScopedToPlatform = !(byok && providerId === 'elevenlabs');
+ let laneOutage = null;
if (laneScopedToPlatform) {
- const outage = (await laneOutages([providerId])).get(providerId);
- if (outage === 'auth') {
+ laneOutage = (await laneOutages([providerId])).get(providerId) || null;
+ if (laneOutage === 'auth') {
return error(
res,
503,
'lane_unavailable',
- `${provider.label} is temporarily unavailable: ${laneOutageCopy(outage)}. ` +
+ `${provider.label} is temporarily unavailable: ${laneOutageCopy(laneOutage)}. ` +
'Microsoft Edge and NVIDIA Magpie are free and unaffected.',
{ provider: providerId, retry_with: ['edge', 'nvidia'] },
);
@@ -322,7 +327,9 @@ export default wrap(async (req, res) => {
{ provider: providerId, detail: String(err?.message || '').slice(0, 300) },
);
}
- if (laneScopedToPlatform) await noteLaneHealthy(providerId);
+ // Only a lane that was cooling has anything to clear, so the common path
+ // (a healthy lane serving a clip) pays no cache round trip for the breaker.
+ if (laneOutage) await noteLaneHealthy(providerId);
if (!out.audio?.length) {
await refundMetering();
diff --git a/docs/api-reference.md b/docs/api-reference.md
index 3f7d00b0b..b56def0fc 100644
--- a/docs/api-reference.md
+++ b/docs/api-reference.md
@@ -976,6 +976,19 @@ clip is cached in R2 for 30 days on a hash of the full request; cache hits are
never charged. A short credit balance returns `402 insufficient_credits` with a
`top_up_url`.
+**When a lane is down**
+
+An upstream failure answers with a single sentence a picker can render in
+`error_description`, the vendor's own text in `detail`, and the lane id in
+`provider`. A lane that refuses this deployment's credentials (an expired key, a
+billing hold) is then withheld for a few minutes: `/api/tts/catalog` reports it
+`available: false` with the reason, and a synthesis request aimed at it answers
+`503 lane_unavailable` carrying `retry_with: ["edge", "nvidia"]` instead of
+spending another doomed round trip. The window clears on its own, and a lane
+that serves successfully is restored immediately, so nothing has to be deployed
+or flipped by hand when the upstream problem is fixed. A request carrying your
+own `x-eleven-key` is never gated by the platform key's outage.
+
### ElevenLabs Voice Library
```
diff --git a/docs/voice-lab.md b/docs/voice-lab.md
index b50bec1bd..e8b64f468 100644
--- a/docs/voice-lab.md
+++ b/docs/voice-lab.md
@@ -150,7 +150,8 @@ The agent-voice handler ([`api/agents/_id/voice.js`](../api/agents/_id/voice.js)
- **Synthesis limits.** 1,000 characters per `/api/tts/synthesize` request (500 on the older `/api/tts/eleven`); every platform-key request is metered to credits per the billing section (charged before synthesis, refunded on failure). BYOK requests are unmetered by the platform. Clips are cached in R2 and cache hits are not charged.
- **Audio size.** Clone uploads are capped at ~10 MB.
- **"Add to an agent" on `/voice`** links to the dashboard; the actual assignment happens in the agent editor. The cloned `voice_id` lives in browser `localStorage` until you assign it.
-- **Error copy.** Mic denial, recorder failure, too-short recording, missing name, network failure, and clone failure each map to a specific message; the playground shows "Synthesizing...", then a size, a cached/generated tag, and the billing rung that served it, or an error. A lane's upstream failure is tagged with a code (`invalid_key`, `rate_limited`, `content_blocked`, `provider_unreachable`) that `/api/tts/synthesize` maps to a truthful HTTP status rather than a blanket 502.
+- **Error copy.** Mic denial, recorder failure, too-short recording, missing name, network failure, and clone failure each map to a specific message; the playground shows "Synthesizing...", then a size, a cached/generated tag, and the billing rung that served it, or an error. A lane's upstream failure is tagged with a code (`invalid_key`, `rate_limited`, `content_blocked`, `provider_unreachable`) that `/api/tts/synthesize` maps to a truthful HTTP status rather than a blanket 502. The message the picker renders is a written sentence, never the vendor's raw body: that ships alongside it in `detail` for debugging.
+- **A lane that goes down leaves the menu.** Configuration presence is not health, so what the last real synthesis learned decides what the catalog offers. A lane the provider refuses (expired key, billing hold upstream) is reported `available: false` with the reason for a few minutes, its voices drop out of the picker, and a synthesis aimed at it answers `503 lane_unavailable` with `retry_with: ["edge", "nvidia"]` rather than failing slowly again. The Voice Lab re-reads the catalog the moment a preview hits that 503, so the grid stops offering voices nobody can render while the explanation stays on screen. The window expires on its own and any successful call clears it, so recovery needs no deploy. Your own `x-eleven-key` is never gated by the platform key's outage.
- **Catalog freshness.** The Edge catalog is fetched live from Microsoft and cached 6 hours per instance; the ElevenLabs account catalog 5 minutes. Nothing is a hardcoded snapshot, so a voice Microsoft or ElevenLabs adds shows up on its own.
## Related
diff --git a/tests/tts-lane-outage-api.test.js b/tests/tts-lane-outage-api.test.js
new file mode 100644
index 000000000..4d3c49cb3
--- /dev/null
+++ b/tests/tts-lane-outage-api.test.js
@@ -0,0 +1,190 @@
+// Integration tests for how /api/tts/catalog and /api/tts/synthesize behave when
+// a voice lane is configured but refusing to serve.
+//
+// The failure they pin is real: on 2026-08-15 every Gemini synthesis came back
+// 403 ("Lightning dunning decision is deny for project ...") while the lane
+// reported itself configured, so the Voice Lab listed 30 Gemini voices, sorted
+// them first, and the first Preview a visitor clicked dumped that raw JSON into
+// the page. These assert the three things that has to produce instead: a
+// sentence a human can act on, the lane withheld from the catalog afterwards,
+// and the next caller answered without a second doomed round trip.
+//
+// Only the network is stubbed (upstream Gemini and the Microsoft voice list);
+// the handlers, the router, and the breaker are the real modules.
+
+import { vi, describe, it, expect, beforeEach, afterAll } from 'vitest';
+
+import catalogHandler from '../api/tts/catalog.js';
+import synthesizeHandler from '../api/tts/synthesize.js';
+import { noteLaneHealthy } from '../api/_lib/tts-lane-health.js';
+
+const realFetch = global.fetch;
+const savedEnv = { ...process.env };
+
+afterAll(() => {
+ global.fetch = realFetch;
+ process.env = savedEnv;
+});
+
+// The exact body Google returns for a dunning-denied project.
+const DUNNING_403 = JSON.stringify(
+ {
+ error: {
+ code: 403,
+ message: 'Lightning dunning decision is deny for project: projects/93741856042',
+ status: 'PERMISSION_DENIED',
+ },
+ },
+ null,
+ 2,
+);
+
+const EDGE_VOICES = [
+ {
+ ShortName: 'en-US-AriaNeural',
+ Locale: 'en-US',
+ Gender: 'Female',
+ FriendlyName: 'Microsoft Aria Online (Natural) - English (United States)',
+ VoiceTag: { ContentCategories: ['General'], VoicePersonalities: ['Positive'] },
+ },
+];
+
+function mockRes() {
+ return {
+ statusCode: 200,
+ _headers: {},
+ _body: '',
+ setHeader(k, v) {
+ this._headers[k.toLowerCase()] = v;
+ },
+ getHeader(k) {
+ return this._headers[k.toLowerCase()];
+ },
+ end(b) {
+ this._body = b || '';
+ },
+ get headersSent() {
+ return false;
+ },
+ get writableEnded() {
+ return false;
+ },
+ get json() {
+ try {
+ return JSON.parse(this._body);
+ } catch {
+ return null;
+ }
+ },
+ };
+}
+
+function getReq(url) {
+ return {
+ method: 'GET',
+ url,
+ headers: { accept: 'application/json', origin: 'http://localhost:3000' },
+ socket: { remoteAddress: '127.0.0.1' },
+ };
+}
+
+/** A POST whose body the handler reads off the stream, as a real request does. */
+function postReq(url, body) {
+ const payload = Buffer.from(JSON.stringify(body));
+ return {
+ method: 'POST',
+ url,
+ headers: {
+ accept: 'application/json',
+ origin: 'http://localhost:3000',
+ 'content-type': 'application/json',
+ 'content-length': String(payload.length),
+ },
+ socket: { remoteAddress: '127.0.0.1' },
+ on(event, cb) {
+ if (event === 'data') cb(payload);
+ if (event === 'end') cb();
+ return this;
+ },
+ };
+}
+
+beforeEach(async () => {
+ await noteLaneHealthy('gemini');
+ process.env.GOOGLE_CLOUD_PROJECT = '';
+ process.env.GEMINI_API_KEY = 'test-key';
+ delete process.env.OPENAI_API_KEY;
+ delete process.env.ELEVENLABS_API_KEY;
+ delete process.env.NVIDIA_API_KEY;
+ global.fetch = vi.fn(async (url) => {
+ const href = String(url);
+ if (href.includes('speech.platform.bing.com')) {
+ return { ok: true, status: 200, json: async () => EDGE_VOICES };
+ }
+ if (href.includes('generativelanguage.googleapis.com')) {
+ return { ok: false, status: 403, text: async () => DUNNING_403 };
+ }
+ throw new Error(`unexpected fetch: ${href}`);
+ });
+});
+
+async function catalog() {
+ const res = mockRes();
+ await catalogHandler(getReq('/api/tts/catalog?limit=2000'), res);
+ return res;
+}
+
+async function speak(provider = 'gemini') {
+ const res = mockRes();
+ await synthesizeHandler(
+ postReq('/api/tts/synthesize', { provider, voiceId: 'Kore', text: 'Hello.' }),
+ res,
+ );
+ return res;
+}
+
+describe('a lane that is configured but refusing', () => {
+ it('is offered until it fails, then withheld with the reason attached', async () => {
+ const before = (await catalog()).json;
+ expect(before.providers.find((p) => p.id === 'gemini').available).toBe(true);
+ expect(before.counts.gemini).toBeGreaterThan(0);
+
+ const failed = await speak();
+ expect(failed.statusCode).toBe(502);
+
+ const after = (await catalog()).json;
+ const gemini = after.providers.find((p) => p.id === 'gemini');
+ expect(gemini.available).toBe(false);
+ expect(gemini.reason).toMatch(/Temporarily unavailable/);
+ expect(after.counts.gemini ?? 0).toBe(0);
+ expect(after.voices.some((v) => v.provider === 'gemini')).toBe(false);
+ });
+
+ it('never leaks the raw upstream body into the message a picker renders', async () => {
+ const res = await speak();
+ const { error_description: message, detail } = res.json;
+ expect(message).not.toMatch(/dunning|PERMISSION_DENIED|projects\//);
+ expect(message).not.toContain('\n');
+ expect(message).toMatch(/Microsoft Edge and NVIDIA Magpie are free/);
+ // The diagnostic text survives, it just stops being the headline.
+ expect(detail).toMatch(/403/);
+ });
+
+ it('answers the next caller from the breaker instead of a second doomed call', async () => {
+ await speak();
+ const calls = global.fetch.mock.calls.length;
+
+ const second = await speak();
+ expect(second.statusCode).toBe(503);
+ expect(second.json.error).toBe('lane_unavailable');
+ expect(second.json.retry_with).toEqual(['edge', 'nvidia']);
+ expect(global.fetch.mock.calls.length).toBe(calls);
+ });
+
+ it('leaves every other lane alone', async () => {
+ await speak();
+ const after = (await catalog()).json;
+ expect(after.providers.find((p) => p.id === 'edge').available).toBe(true);
+ expect(after.counts.edge).toBeGreaterThan(0);
+ });
+});