Skip to content

armgit / cindy

Public Imported from GitHub
0

Consider it done. The open-source AI agent that works out of the box · 想到,就能做到。开源、开箱即用的 AI Agent。

fix(desktop): normalize Gateway speech API base

JiangJiahao1651528101c5
diff --git a/apps/desktop/src/main/voice-conversation/__tests__/models.test.ts b/apps/desktop/src/main/voice-conversation/__tests__/models.test.ts
index a5e07b4791..a470fd99cd 100644
--- a/apps/desktop/src/main/voice-conversation/__tests__/models.test.ts
+++ b/apps/desktop/src/main/voice-conversation/__tests__/models.test.ts
@@ -7,6 +7,7 @@ import {
type ProviderRuntimeModelConfig,
} from '@cindy/model-providers';
import { resolveSpeechConnection, speechModels, type SpeechCatalogDeps } from '../models.js';
+import { speechRequest } from '../speechProtocols.js';
import { normalizeVoiceInputSettings } from '../../../shared/voiceInputData.js';
import { voiceModelKey } from '../../../shared/voiceConversation.js';
@@ -87,6 +88,46 @@ it('keeps Gateway entitlement and supported-protocol gates', () => {
expect(speechModels({ ...d, gatewayAllowed: false })).toEqual([]);
});
+// Unit regression: resolveSpeechConnection -> speechRequest must send Gateway TTS
+// to /v1/audio/speech. Live root endpoints returned 404; previous fixtures already
+// included /v1. Cover the root, existing API base and deployment prefix without I/O.
+it.each([
+ ['https://gateway.test', 'https://gateway.test/v1/audio/speech'],
+ ['https://gateway.test/', 'https://gateway.test/v1/audio/speech'],
+ ['https://gateway.test/v1', 'https://gateway.test/v1/audio/speech'],
+ ['https://gateway.test/v1/', 'https://gateway.test/v1/audio/speech'],
+ ['https://gateway.test/prefix/', 'https://gateway.test/prefix/v1/audio/speech'],
+ ['https://gateway.test/prefix/v1/', 'https://gateway.test/prefix/v1/audio/speech'],
+])('uses the Gateway speech API path for %s', (baseUrl, expectedUrl) => {
+ const credentials = { baseUrl, apiKey: 'gateway-key' };
+ const d = deps([
+ { id: 'xd', audioModels: [model('gateway-speech', { nativeApi: 'openai-audio-speech' })] },
+ ] as Catalog['providers']);
+ d.gatewayConnection = () => credentials;
+ const input = { modelId: 'gateway-speech', voice: 'voice-id', text: 'hello' };
+ const connection = resolveSpeechConnection(d, input);
+ const request = speechRequest(connection, input);
+ expect(request.url.href).toBe(expectedUrl);
+ expect(JSON.parse(request.body)).toEqual({
+ model: input.modelId,
+ voice: input.voice,
+ input: input.text,
+ response_format: 'pcm',
+ });
+ expect(connection.headers).toEqual({ authorization: 'Bearer gateway-key' });
+ // The shared server credential endpoint must remain unchanged for chat/media callers.
+ expect(credentials).toEqual({ baseUrl, apiKey: 'gateway-key' });
+});
+
+it('keeps a BYOK speech API prefix without adding the Gateway version path', () => {
+ const d = deps([custom('proxy', 'https://speech-proxy.test/tts-api')]);
+ const input = { modelId: 'tts', voice: 'voice-id', text: 'hello' };
+ const connection = resolveSpeechConnection(d, { ...input, providerId: 'proxy', agent: 'codex' });
+ expect(speechRequest(connection, input).url.href).toBe(
+ 'https://speech-proxy.test/tts-api/audio/speech',
+ );
+});
+
it('distinguishes two user connections with the same upstream ID and uses only the selected runtime key', () => {
const d = deps([custom('a', 'https://a.test/prefix/v1'), custom('b', 'https://b.test/v1')]);
const models = speechModels({ ...d, gatewayAllowed: false });
diff --git a/apps/desktop/src/main/voice-conversation/models.ts b/apps/desktop/src/main/voice-conversation/models.ts
index 57e309df35..f304952257 100644
--- a/apps/desktop/src/main/voice-conversation/models.ts
+++ b/apps/desktop/src/main/voice-conversation/models.ts
@@ -121,7 +121,12 @@ function bindings(deps: SpeechCatalogDeps, selection?: SpeechModelSelection): Bi
)
continue;
const connection = deps.gatewayConnection();
- if (!connection.apiKey || !apiBase(connection.baseUrl)) continue;
+ const base = apiBase(connection.baseUrl);
+ if (!connection.apiKey || !base) continue;
+ // Server credentials carry the deployment base shared by chat/media.
+ // Version only Gateway TTS; BYOK keeps its explicitly configured API prefix.
+ if (!base.pathname.endsWith('/v1'))
+ base.pathname = `${base.pathname.replace(/\/+$/, '')}/v1`;
result.push({
model: {
id: media.id,
@@ -132,7 +137,7 @@ function bindings(deps: SpeechCatalogDeps, selection?: SpeechModelSelection): Bi
defaultVoice: referenceSpeechVoices(media.id, 'openai')[0] ?? '',
},
connection: {
- baseUrl: connection.baseUrl,
+ baseUrl: base.toString().replace(/\/+$/, ''),
protocol: 'openai',
headers: { authorization: `Bearer ${connection.apiKey}` },
},
diff --git a/docs/product-rules/companion-voice-mode.md b/docs/product-rules/companion-voice-mode.md
index db98af958b..28b53c2ad6 100644
--- a/docs/product-rules/companion-voice-mode.md
+++ b/docs/product-rules/companion-voice-mode.md
@@ -36,7 +36,7 @@
- 当前仅支持 Desktop、本机伙伴一对一。群聊、Mobile、远程控制的麦克风、原生 Realtime speech-to-speech 不在此实现内。
- 新 IPC 只接受可信应用顶层 Renderer,按窗口、账号代次、callId 和 canonical 任务绑定。callId 在发起请求前生成,使结束按钮也能取消连接过程;它不是授权凭证。接口不加入 device-link 或插件白名单。输入静音控制仍绑定当前窗口与 callId,输入每次重开分配新代次;旧代次 PCM 和识别回调不能进入新输入,结束中的迟到麦克风或 ASR 分配仍须释放。模型权限在开始和每次 TTS 请求、读取时复核。
- ASR 适配器额外提供按上游 utterance/item ID 标识的分段事件,现有累积 `AsrEvent` 合同保持不变。连接在 150 秒轮换,沿用 voice-server 的 dictation 协议;切换期间最多缓存 5 秒输入,不能无声丢弃或重复发送前文。
-- Gateway 执行合同为 OpenAI 兼容 `POST <下发的 baseUrl>/audio/speech`,body 包含 `model`、`voice`、`input`、`response_format: pcm`。PCM 固定为该协议规定的 24kHz、单声道、16bit little-endian。显式声明其他 native API 或 `null` 未验证 API 的 Cindy AI 模型不进入选择器;未声明协议的 Gateway 新模型沿用此端点合同。下发 `mode` 只代表能力分类,不证明具体音色、音频格式或端点可用。
+- Gateway 执行合同为 OpenAI 兼容 `POST <语音 API base>/audio/speech`。服务端下发的是与密钥配对的推理部署地址:仅在本地 Cindy AI TTS 连接中补齐末尾 `/v1`,已有 `/v1` 不重复添加,保留部署路径前缀;不回写聊天、图像等共用的 endpoint,也不改 BYOK 配置的 API base。body 包含 `model`、`voice`、`input`、`response_format: pcm`。PCM 固定为该协议规定的 24kHz、单声道、16bit little-endian。显式声明其他 native API 或 `null` 未验证 API 的 Cindy AI 模型不进入选择器;未声明协议的 Gateway 新模型沿用此端点合同。下发 `mode` 只代表能力分类,不证明具体音色、音频格式或端点可用。
- 凭证只在 Main:Gateway 使用当前账号配对的 baseUrl / 密钥,BYOK 复用对应连接、对应 runtime 的 safeStorage 密钥和固定请求头。排除原生订阅 OAuth、远程或共享供应商;per-model 地址不得跨越凭证所属连接的 origin。开始时固定连接,每次合成与音频读取复核资格及路由/密钥是否变化;变化则停止该语音请求。禁止自动切换到其他供应商、系统朗读或另一条凭证。响应按 Renderer 播放速度读取,播放只预排约 300ms;中断丢弃旧请求和迟到音频。原始音频只在内存,不新增录音文件或对话音频归档。
- 初始实现的声音活动判断是带迟滞的音量检测,配合浏览器回声消除与降噪。它不是语义 VAD,不能保证所有扬声器、蓝牙或噪声环境都能准确分辨插话。