Skip to content

armgit / cindy

Public Imported from GitHub
0

Consider it done. The open-source AI agent that works out of the box · 想到,就能做到。开源、开箱即用的 AI Agent。

fix(im): project rich answers to readable outbound text

nanaed1da855869e
diff --git a/apps/desktop/src/main/__tests__/messagePersistBroadcaster.test.ts b/apps/desktop/src/main/__tests__/messagePersistBroadcaster.test.ts
index a9d9573e32..47e56a837f 100644
--- a/apps/desktop/src/main/__tests__/messagePersistBroadcaster.test.ts
+++ b/apps/desktop/src/main/__tests__/messagePersistBroadcaster.test.ts
@@ -215,6 +215,19 @@ beforeEach(() => {
clearSessionPersistState(SESSION);
});
+// #6884, unit: IM display projection must not change history. Existing text
+// persistence cases had no component fence; the DB boundary here is an in-memory spy.
+it('persists the original rich answer for Cindy history', async () => {
+ const source = '```cindy-ui\n' + JSON.stringify({
+ version: 1, seq: 1, op: 'create', id: 'answer', kind: 'text', props: { text: 'Readable answer' },
+ }) + '\n```';
+ onAssistantTextEvent(SESSION, { text: source, isFinal: true }, null);
+ await drainPersistQueue();
+ expect(createMessage).toHaveBeenCalledWith(SESSION, expect.objectContaining({
+ role: 'assistant', content: source,
+ }), expect.anything());
+});
+
describe('browser proxy credential persistence', () => {
const proxyServer = 'http://user%40example.test:p%3Ass%2Fword%40%[email protected]:12323';
diff --git a/apps/desktop/src/main/im/shared/__tests__/turnRunnerSendOutcome.test.ts b/apps/desktop/src/main/im/shared/__tests__/turnRunnerSendOutcome.test.ts
index 887577d616..367409981d 100644
--- a/apps/desktop/src/main/im/shared/__tests__/turnRunnerSendOutcome.test.ts
+++ b/apps/desktop/src/main/im/shared/__tests__/turnRunnerSendOutcome.test.ts
@@ -3034,6 +3034,99 @@ describe('turnRunner send outcome policy (feishu adapter characterization)', ()
}
});
+ // #6884: exercise the real runner's outbound boundary. Existing cases only used
+ // plain text; one component fence used to reach the transport as protocol JSON.
+ // Unit tier: in-memory session/transport, no external sends or database writes.
+ describe('rich answers at the IM boundary', () => {
+ const rich = '```cindy-ui\n' + JSON.stringify({
+ version: 1, seq: 1, op: 'create', id: 'answer', kind: 'heading', props: { text: 'Readable answer' },
+ }) + '\n```';
+
+ it.each(['rich-card', 'chunked-text', 'start-failed', 'terminal-error'] as const)(
+ 'projects component output for %s while leaving the session event intact', async (kind) => {
+ const previousOutput = fakeAdapter.output;
+ const commitFinal = vi.fn<(payload: { text: string }) => Promise<void>>(async () => undefined);
+ const replace = vi.fn();
+ const finalize = vi.fn();
+ mocks.feishuIm.startStreamingText.mockResolvedValue({
+ messageId: 'rich', append: vi.fn(), replace, finalize, close: vi.fn(),
+ });
+ if (kind === 'start-failed') mocks.feishuIm.startStreamingText.mockRejectedValue(new Error('offline'));
+ if (kind === 'chunked-text') fakeAdapter.output = {
+ kind: 'chunked-text', im: mocks.feishuIm as unknown as ChannelIM, commitFinal,
+ };
+ try {
+ const h = setupSession(async () => ({ accepted: true }));
+ const observed = vi.fn();
+ h.session.onEvent(observed);
+ await runDefaultTurn();
+ const source = `Before\n\n${rich}\n\nAfter`;
+ const event: AgentEvent = { type: 'text', data: { text: source, isFinal: true } };
+ h.emit(event);
+ await flushMicrotasks();
+ h.emit(kind === 'terminal-error'
+ ? { type: 'error', data: { message: 'process exited', isTerminal: true } }
+ : { type: 'done', data: {} });
+ await waitForAssertion(() => {
+ const text = kind === 'chunked-text' ? commitFinal.mock.calls[0]?.[0].text
+ : kind === 'start-failed' ? mocks.feishuIm.sendText.mock.calls.at(-1)?.[1]
+ : finalize.mock.calls[0]?.[0];
+ expect(text).toContain('### Readable answer');
+ expect(text).toContain('Before');
+ expect(text).toContain('After');
+ expect(text).not.toContain('cindy-ui');
+ expect(text).not.toContain('"op"');
+ if (kind === 'terminal-error') expect(text).toContain('process exited');
+ });
+ expect(observed).toHaveBeenCalledWith(event);
+ expect(event.data).toEqual({ text: source, isFinal: true });
+ expect(mocks.persistAssistantMessage).not.toHaveBeenCalled();
+ } finally {
+ fakeAdapter.output = previousOutput;
+ }
+ },
+ );
+
+ it('waits on an incomplete component instead of streaming raw JSON, then shows readable content', async () => {
+ const replace = vi.fn();
+ const finalize = vi.fn();
+ mocks.feishuIm.startStreamingText.mockResolvedValue({
+ messageId: 'rich', append: vi.fn(), replace, finalize, close: vi.fn(),
+ });
+ const h = setupSession(async () => ({ accepted: true }));
+ await runDefaultTurn();
+ h.emit({ type: 'text', data: { text: '```cindy-ui\n{"version":1,"seq":1', isFinal: false } });
+ await waitForAssertion(() => expect(replace).toHaveBeenCalledWith(''));
+ h.emit({ type: 'text', data: { text: rich, isFinal: true } });
+ await waitForAssertion(() => expect(replace).toHaveBeenLastCalledWith('### Readable answer'));
+ h.emit({ type: 'done', data: {} });
+ await waitForAssertion(() => expect(finalize).toHaveBeenCalledWith('### Readable answer'));
+ expect(JSON.stringify(replace.mock.calls)).not.toContain('cindy-ui');
+ });
+
+ it.each([
+ ['```cindy-ui\n{"version":', 'Some content could not be displayed.', false],
+ ['```cindy-ui\ninvalid\n```', 'Some content could not be displayed.', false],
+ ['```cindy-ui\n{"version":', /Some content could not be displayed\.[\s\S]*process exited/, true],
+ ['Before\n\n```cindy-ui\ninvalid\n```\n\nAfter', /Before\s+After/, false],
+ ['# Markdown\n\n```json\n{"op":"create"}\n```', '# Markdown\n\n```json\n{"op":"create"}\n```', false],
+ ] as const)('keeps final output readable without changing ordinary code blocks (%s)', async (source, expected, failed) => {
+ setMainLocale('en');
+ const finalize = vi.fn();
+ mocks.feishuIm.startStreamingText.mockResolvedValue({
+ messageId: 'rich', append: vi.fn(), replace: vi.fn(), finalize, close: vi.fn(),
+ });
+ const h = setupSession(async () => ({ accepted: true }));
+ await runDefaultTurn();
+ h.emit({ type: 'text', data: { text: source, isFinal: true } });
+ h.emit(failed ? { type: 'error', data: { message: 'process exited', isTerminal: true } }
+ : { type: 'done', data: {} });
+ await waitForAssertion(() => expect(finalize).toHaveBeenCalledWith(
+ typeof expected === 'string' ? expected : expect.stringMatching(expected),
+ ));
+ });
+ });
+
it('commits one terminal payload for a chunked-text output driver', async () => {
const previousOutput = fakeAdapter.output;
const beginReply = vi.fn(async () => undefined);
diff --git a/apps/desktop/src/main/im/shared/__tests__/turnRunnerThreadRouting.test.ts b/apps/desktop/src/main/im/shared/__tests__/turnRunnerThreadRouting.test.ts
index 7b5fdd1ca1..fe70317232 100644
--- a/apps/desktop/src/main/im/shared/__tests__/turnRunnerThreadRouting.test.ts
+++ b/apps/desktop/src/main/im/shared/__tests__/turnRunnerThreadRouting.test.ts
@@ -248,7 +248,7 @@ function streamingHandleStub() {
messageId: 'C1|9.9',
append: vi.fn(),
replace: vi.fn(),
- finalize: vi.fn(async (): Promise<void> => undefined),
+ finalize: vi.fn<(text: string) => Promise<void>>(async () => undefined),
addExtraImageAbsPath: vi.fn(),
close: vi.fn(),
};
@@ -1104,6 +1104,43 @@ describe('turnRunner 自动任务转播(scheduler turn → 远程控制 thread)'
const withOrigin = (e: Partial<AgentEvent>): AgentEvent =>
({ ...e, turnOrigin: schedulerOrigin }) as AgentEvent;
+ // #6884, unit: the same component fence must be projected for background
+ // streaming/final delivery. Existing relay tests only covered plain Markdown.
+ it.each(['rich-card', 'chunked-text'] as const)('projects background rich answers for %s', async (kind) => {
+ const stub = streamingHandleStub();
+ mocks.slackIm.startStreamingText.mockResolvedValue(stub);
+ const previousOutput = fakeAdapter.output;
+ if (kind === 'chunked-text') fakeAdapter.output = {
+ kind: 'chunked-text', im: mocks.slackIm as unknown as ChannelIM, commitFinal: vi.fn(async () => undefined),
+ };
+ await runner.disposeAllSessions();
+ runner = createTurnRunner(fakeAdapter, fakeRepo, fakeCards);
+ try {
+ const h = await attachAndIdle();
+ const source = '```cindy-ui\n' + JSON.stringify({
+ version: 1, seq: 1, op: 'create', id: 'result', kind: 'text', props: { text: 'Readable background result' },
+ }) + '\n```';
+ h.emit(withOrigin({ type: 'text', data: { text: '```cindy-ui\n{"version":', isFinal: false } }));
+ await new Promise((resolve) => setTimeout(resolve, 0));
+ expect(JSON.stringify(stub.replace.mock.calls)).not.toContain('cindy-ui');
+ h.emit(withOrigin({ type: 'text', data: { text: source, isFinal: true } }));
+ if (kind === 'rich-card') {
+ await vi.waitFor(() => expect(stub.replace).toHaveBeenCalledWith(expect.stringContaining('Readable background result')));
+ }
+ h.emit(withOrigin({ type: 'done', data: {} }));
+ await vi.waitFor(() => {
+ const text = kind === 'chunked-text' ? mocks.slackIm.sendMarkdownText.mock.calls.at(-1)?.[1]
+ : stub.finalize.mock.calls.at(-1)?.[0];
+ expect(text).toContain('Readable background result');
+ expect(text).not.toContain('cindy-ui');
+ expect(text).not.toContain('"op"');
+ });
+ } finally {
+ await runner.disposeAllSessions();
+ fakeAdapter.output = previousOutput;
+ }
+ });
+
it('successful background work without a reply removes the progress card and its title', async () => {
const stub = streamingHandleStub();
mocks.slackIm.startStreamingText.mockResolvedValue(stub);
diff --git a/apps/desktop/src/main/im/shared/turnRunner.ts b/apps/desktop/src/main/im/shared/turnRunner.ts
index 503e0695e9..e4b8551b9d 100644
--- a/apps/desktop/src/main/im/shared/turnRunner.ts
+++ b/apps/desktop/src/main/im/shared/turnRunner.ts
@@ -59,6 +59,7 @@ const GROUP_APPROVAL_OWNER_DM_NOTE =
import { eq } from 'drizzle-orm';
import { stripInternalWebCitations } from '@cindy/maker-shared/internal-citation';
+import { hasRichAnswer, richAnswerText } from '@cindy/maker-shared/rich-answer';
import {
isProductTurnDoneEvent,
isTurnContinuationBoundaryEvent,
@@ -77,6 +78,7 @@ import { hydrateSessionProvider } from '../../maker-host/session-provider-store'
import { setSessionEffort, setSessionFastMode } from '../../maker-host/session-effort-store';
import { hasCustomProviderKey } from '../../maker-host/provider-route';
import { createLogger } from '../../logger';
+import { t } from '../../i18n';
import { resolveSafe as resolveXdtImageUrl } from '../../imageCacheStore';
import { resolveSafe as resolveCindyMediaUrl } from '../../cindy-media/blobStore';
import { beginHeadlessGhostSetupTurn } from '../../mcp-integrations/ghostSetupInteractionSurface.js';
@@ -2928,19 +2930,25 @@ export function createTurnRunner(
* 消息只留干净正文(error 收口不置 done — 保留过程区, 用户能看到死在哪步)。
* 纯文本快答没有 tool_use, renderActivity 返回空串, 视图与旧行为逐字一致。
*/
- function composeStreamingView(turn: TurnState): string {
+ function composeStreamingView(turn: TurnState, final = turn.done): string {
const rawBody = turn.outputCardPrefix
? turn.outputCardPrefix + turn.presenter.wholeText()
: turn.presenter.wholeText();
- // External-channel safeguard: maker-core normally strips these tokens,
- // while this boundary also protects old continuations and future adapters.
- const body = stripInternalWebCitations(rawBody);
+ const body = composeOutboundBody(rawBody, final);
if (turn.done) return body;
// 过程区在上、正文在下的合成骨架与官方 bot 共用(presenter.composeRunning →
// composeProgressView); 纯文本快答无 tool_use 时过程区为空, 视图与旧行为逐字一致。
return turn.presenter.composeRunning(body);
}
+ /** Project only the outbound copy; persistence keeps the original component events. */
+ function composeOutboundBody(source: string, final: boolean): string {
+ const body = stripInternalWebCitations(richAnswerText(source));
+ // A partial component can legitimately project to nothing. Never fall back
+ // to its protocol source; wait while streaming and explain an unreadable final.
+ return final && !body.trim() && hasRichAnswer(source) ? t('richAnswer.incomplete') : body;
+ }
+
/** tool_use → 过程区时间线推进 + 卡片刷新(渠道 handle 自带节流兜底)。 */
function handleToolUseEvent(turn: TurnState, event: AgentEvent): void {
if (!turn.presenter.applyToolUse(event)) return;
@@ -2993,7 +3001,7 @@ export function createTurnRunner(
/** 转播卡正文:运行中 = 头 + 步骤时间线 + 正文;收口 = 头 + 正文(去步骤)。 */
function composeTranspondView(state: SessionState, t: ScheduledTranspond, final: boolean): string {
const header = t.header;
- const publicBody = stripInternalWebCitations(t.buffer);
+ const publicBody = composeOutboundBody(t.buffer, final);
// Rich transports resolve managed refs themselves. A remote session's refs
// must stay text, never reach those host-local file/media resolvers.
const body = state.makerSession.remoteHostId
@@ -3864,7 +3872,7 @@ export function createTurnRunner(
} : undefined,
);
if (turn?.streamingHandle) {
- const view = composeStreamingView(turn);
+ const view = composeStreamingView(turn, true);
const body = view ? `${view}\n\n❌ 错误:${msg}` : `❌ 错误:${msg}`;
await finalizeTurnStream(turn, body);
} else {
diff --git a/docs/product-rules/telegram-bot-parity.md b/docs/product-rules/telegram-bot-parity.md
index b1e06afc88..c3011dbfdd 100644
--- a/docs/product-rules/telegram-bot-parity.md
+++ b/docs/product-rules/telegram-bot-parity.md
@@ -299,6 +299,7 @@ hook 侧既有的逐轮 `[渠道说明]`,两者措辞不同但都向模型说
| 2i | **群内回复触发与任务归属两边各写一套** | 官方 bot 的服务端正本用 `TelegramMessageRoute.botAuthored === true` 区分“消息与任务有关”和“消息由 Cindy 发出、可通过回复召唤”:回复曾触发任务的普通用户消息不会误触发,明确 `@cindyapp_bot` 仍召唤当前发言者自己的 Cindy;回复同一 principal 的 Cindy 输出可续用其历史群 lane,回复其他 principal 的 Cindy 输出只带引用上下文,任务归当前发言者,不继承原任务的 principal、设备、会话或权限。普通群与 forum topic 都适用;`/session`、interaction、取消和 reaction 继续严格校验 owner。个人 bot 则在 `packages/lizi-im/src/telegram/inbound.ts` 的 `detectGroupTrigger` 独立用 `reply_to_message.from.id === botId` 判定回复触发,且采用本地单 owner 模型;两侧没有共享代码或数据,不能因为当前触发语义相近就跳过双路核对。**纯 @(只发一句 `@bot`、没有正文)两侧同口径都算召唤**:剥离召唤标记后 `userText`/事件正文保持空串,召唤及附件投递情况作为独立事实进入模型,不回填裸 `@` 或虚构请求。个人侧通过 `invoked` 保留纯召唤,即使随附媒体下载失败、超限或不支持也继续处理;仅有不可用媒体且没有召唤、引用或正文时仍直接提示。个人侧提及判据只认平台实体——`mention` 按 username、`text_mention` 按 user id,@ 在句中、句尾都触发;显示名召唤是唯一的字面匹配兜底 | 本客户端改动统一个人 bot 入站事实及 Hook 原文存档;官方 bot 的输入组装需配套版本更新,不能把客户端实现完成视为官方已上线。两侧保持同语义,仍各自维护触发与归属判据;以后修改群回复行为必须同时核对两条路径 |
| 2j | **入站音视频与「先发媒体、再单独 @」:个人 bot 已补齐,官方待核** | 个人:`packages/lizi-im/src/telegram/inbound.ts` 的 `collectMedia` 把 video / video_note / voice / audio 及 `video/*`、`audio/*` 文档经 `host.media.cacheMedia` 存入 cindy-media 媒体总仓(可回收暂存,消息落库时转为附件引用;不写冻结的 `cc-agent/telegram-media`),以 file block(可读路径)交给模型,Agent 自行抽帧或转写;总仓不收的格式或 host 未提供 `cacheMedia` 时标 video / audio 不支持;Bot API 只能下载 20MB 以内,超出标 `oversize`,均作为未提供原因告知模型。群里先发媒体、再单独 @bot 时,`TelegramIM` 用收到时记下的 file_id(内存池,200 条 / 24 小时,重启清空,受保护群不入池;不随轮次出池,已受理轮次由群窗口游标防重复,同一轮已随附件送达的相册成员与被引消息排除)补下载本轮群上下文实际纳入那些消息的附件(`groupWindowCore` 的 `messageIds`),名额与飞书群上下文同口径(图片 6、文件 4,最新优先,先占名额再下载),全响应(ambient)轮次不补,经 `contextAttachments` 只进模型消息、不落库(总仓内保持可回收缓存),`<group_chat_attachments>` 块写明已提供与未提供的原因。官方:服务端入站对音视频与群历史媒体的处理**未核** PDF 等非媒体文件经 `host.files` 暂存后移入按账号隔离的 `im-attachments/telegram/sessions/<任务ID>/`(随任务删除清理),与飞书、Discord、企业微信同一实现,不再写 `cc-agent/telegram-media`。| 待核官方侧;若服务端仍丢弃音视频,按同一语义跨仓补齐 |
| 2k | **全响应(ambient)消息先经辅助模型判「该不该插话」:个人 bot 有、官方没有** | 个人:`im/shared/messageHandler.ts` 在 ack 与提前落库**之前**调 adapter 的 `ambientTurnGate`,Telegram 适配器把群窗口前缀(只读不 commit)、发言人行、被引消息与机器人身份(设置页人格 / 伙伴名字 + 简介)交给 `im/shared/ambientReplyJudge.ts`,走共享辅助模型链(与伙伴群聊分工判定同一条,单候选 5 秒 / 总预算 8 秒,只答 REPLY / SKIP)。SKIP 的消息不落库、不起轮次、游标不动(下次真正开口时仍是背景);REPLY / 判不出来(没有辅助模型、超时、输出不合规)照旧起完整轮次,主模型仍可 NO_REPLY,所以第 2b / 2f 行的哨兵与失败收口语义不变。官方:服务端在 `controller.ts` 打 ambient 标后直接下发整轮,没有前置判定 | 待判:官方侧要省同样的钱得在服务端接同类判定;在那之前登记为缺口。个人侧判定器失败一律放行,不得把「判不出来」当 SKIP |
+| 2l | **富答案出站:个人已投影,官方待核(#6884)** | 个人 `im/shared/turnRunner.ts` 在普通轮次与后台转播的统一出站层复用 `richAnswerText`,把 `cindy-ui` 转为可读文本,覆盖流式卡片、文本终稿、建卡失败后的文本补发及错误收尾。流式投影为空时等待后续内容,组件终稿仍无可读内容时用本地化提示,绝不回退协议原文;历史保留原始组件。回归见 `turnRunnerSendOutcome.test.ts`、`turnRunnerThreadRouting.test.ts`、`messagePersistBroadcaster.test.ts` | 官方 hook 是独立架构,本次未改,其富答案出站能力待单独核验,不计为两侧已同源 |
| 3 | **终稿必达只有官方有** | 官方侧终稿先落盘、失败重试到送达或有界放弃(`xindong/cindy-server#348`)。个人 bot 的 `streamingText.finalize` 是进程内尽力而为,桌面进程挂掉那条终稿就没了。**两侧的「有界」各有一条明确边界、且不在同一层**:官方路径经桌面账本 `hook-control/requestLedger.ts`,客户端侧的投递时效是 `HOOK_TERMINAL_DELIVERY_TTL_MS ≈ 24h`,**规则无条件**——过线的终稿一律不再发出,不论是谁在要。覆盖的出口:持久出箱 `listPending`、ACK 退避重发、离线内存缓冲、ACK 缓冲的两个消费分支(`onConnected` 入口一次性清扫,含能力降级回落),以及 **server 显式重投**——那一支只回放 `task.ack` 后返回,不发终稿、也不把记录改回 `pending`。(重投一度有过 `origin='server-request'` 豁免,后被删除:它在持久记录里没有位置,每条路径都要手工传播,连续三轮 review 各找出一条漏掉的。)server 侧只丢掉一份它自己也已放弃发布的终稿——服务端 `OUTBOX_MAX_ATTEMPTS × OUTBOX_MAX_DELAY_MS ≈ 24h` 的发布放弃线与客户端刻意取同一个数,而结果总比请求更晚,所以它索取一份过线结果时自身 outbox 早已过放弃点;X 侧入口还另有 `x-hook-server` 的 `onMention` 陈旧守卫。**个人 bot 不经过这个账本**,既没有落盘也没有这条时效,它的边界就是「进程活着就尽力,挂了就没了」。官方协商 `telegram-final-ops-v1` 由桌面端发布终稿后**仍走这本账**:发布前先把不带 `clientFinal` 的 `turn.end` 写进出箱,崩溃重放时服务端删掉已落地的客户端终稿段并自己重发,所以官方这条保障不降级 | 待判:个人侧是否需要等价保障,还是接受「桌面挂了本来就没人在跑」。**注意两侧要判的不是同一件事**:官方已定「超过 24h 的终稿不再主动补发」,个人待定的是「要不要先落盘」 |
| 4 | 受保护群内容的隐私边界 | 个人侧已做(出站回流 fail-closed,任一分片带保护标即整条不回流)。官方侧是否等价**待核** | 待核 |
| 5 | 相册失败逐张回落 | 两侧都有实现,判据是否等价**待核** | 待核 |