A local-first multi-agent app development platform, built on PenguinHarness
feat(web,ui): 点开回复里的图片,多了两个按钮直接让 Agent 动手
lmliheng970c33d8cb86
diff --git a/changelog/unreleased/2026-10-30-reply-picture-actions.md b/changelog/unreleased/2026-10-30-reply-picture-actions.mdnew file mode 100644index 00000000..5e0544aa--- /dev/null+++ b/changelog/unreleased/2026-10-30-reply-picture-actions.md@@ -0,0 +1,68 @@+# A picture in a reply now offers the two things the Agent can do with it++- **Date:** 2026-10-30+- **Type:** feature+- **Scope:** `web`, `ui`+- **Breaking:** no++[中文版](2026-10-30-reply-picture-actions.zh.md)++Opening a picture in a conversation used to offer saving it and stepping through the batch, nothing+else: the reader could see the picture, and to have anything made from it they went back to the+composer and typed its path by hand. The lightbox's capsule bar now carries two more buttons —+**Split into layers** and **Redraw from this one** — with the picture's own path filled in, so+nobody copies it out of the transcript.++Neither action is a request this App makes against an endpoint of its own: the repository has none,+and the model, the credential and the file all live on the Agent's side, so an action is **a+sentence handed to the Agent**. Splitting into layers is one Seedream call with fixed options+(`--layers`), so it goes out at once as a message of the user's own and the reply carries the base+image plus every transparent layer PNG. What a redraw should change is the user's to say, so that+one only puts the instruction — the picture named as its reference image — in the composer, and+pressing Send stays their move: generation is billed per image, and no tap read as "redo this"+should buy one.++An Agent without the image Skill (`seedream`, installable from `@adelie/use-seedream`) is offered+neither button. A control that can only fail is worse than no control — the rule the bar started+with.++## Details++- `packages/ui/src/components/overlays/lightbox/lightbox.tsx`: `LightboxAction` (`key` / `label` /+ `onSelect`) becomes an optional `actions` on `Lightbox` and `ZoomableImage`, drawn in the capsule+ bar after the download. Selecting one closes the overlay and then runs it — an action usually+ lands on the page underneath (a sentence in the composer, a turn for the Agent), and doing that+ under a scrim leaves the reader looking at the picture they just acted on.+- `packages/ui/src/components/content/prose/prose.tsx`: `ProseImagesProvider` takes an optional+ `actions` resolver (`ProseImageActionsResolver`), shaped like the image and link resolvers beside+ it, and `MdImage` hands its answer to the lightbox. It is asked with the src **as the Markdown+ wrote it**, not with the URL the image resolver produced: the surface above recognizes a file by+ that src, and a file route's URL is nothing it can use.+- `packages/web/src/lib/image-actions.ts` (new): `hasSeedreamSkill` and `replyImageActionBuilder`.+ Pure logic — both actions are wired here, with the words passed in from the active dictionary, so+ each language stays in one place.+- `packages/web/src/features/chat/workspace-refs.tsx`: the conversation gains a resolver answering+ what can be done with a picture by Workspace path, sorted through the same `resolveReplyImage` as+ the fetch (one path sorted twice is two chances to disagree).+- `packages/web/src/features/chat/chat-page.tsx`: the Agent's Skill list is already loaded for the+ composer, and it is what decides whether the buttons appear at all. Split into layers posts+ straight through `api.postTask(..., { queueIfBusy: true })` without disturbing the draft being+ typed; the redraw goes through `ComposerControl.fillPrompt` with `seedream` pinned, so Send+ builds the same `[use_skills]` message the one-click action would have posted.+- `packages/web/src/features/chat/message-stream.tsx`: the render context carries `imageActions`+ down to the Markdown.+- `packages/web/src/lib/strings{,-en}.ts`: `chat.imageAction` — the two buttons' words and the+ sentence each carries (the instruction IS the message, so it follows the interface's language).+- Tests: `packages/web/test/image-actions.test.ts` (new: the Skill gate, both buttons' names and+ destinations, the `[use_skills]` wrapping, the path carried through verbatim),+ `packages/ui/test/lightbox.test.ts` (the buttons in the bar, an unchanged bar when nothing is+ offered, close-then-run), `packages/ui/test/prose.test.ts` (the resolver is asked about the src as+ written), `packages/web/e2e/reply-picture-actions.spec.mjs` (new — a real browser: an Agent+ without the Skill gets no buttons, and once it has them a click closes the overlay and leaves the+ instruction, path and all, in the composer, while the layer split goes out as a message carrying+ the Skill's own `[use_skills]` mark).++## Compatibility++- No API, on-disk or protocol change. An Agent without the image Skill, and every Markdown surface+ that is not a conversation, behaves exactly as before: a picture is still only enlarged and saved.diff --git a/changelog/unreleased/2026-10-30-reply-picture-actions.zh.md b/changelog/unreleased/2026-10-30-reply-picture-actions.zh.mdnew file mode 100644index 00000000..9b3f0f39--- /dev/null+++ b/changelog/unreleased/2026-10-30-reply-picture-actions.zh.md@@ -0,0 +1,51 @@+# 回复里的图片点开后多了两个动作:拆图层,照这张改一版++- **日期:** 2026-10-30+- **类型:** feature+- **范围:** `web`、`ui`+- **破坏性:** 无++[English](2026-10-30-reply-picture-actions.md)++对话里点开一张图,此前只有「下载」和翻页:图已经能看了,想拿它再生成点什么都得回输入框里手打+一遍那张图的路径。现在放大层的胶囊条上会多出两个按钮 —— **拆图层** 与 **照这张改一版** —— 图片+自己的路径由界面填好,用户不用再抄一遍。++两个动作都不走 App 自己的接口:这一仓没有图像端点,模型、密钥和文件都在 Agent 那一侧,所以动作+其实是**交给 Agent 的一句话**。拆图层是一次参数确定的 Seedream 调用(`--layers`),点了就直接作为+用户的消息发出去,回复里会有底图和每一层透明 PNG;改一版改成什么样得由用户说,所以它只把带参考图+的指令填进输入框,按下发送仍是用户的事 —— 生成按张计费,不该由一次「重做」的点击替人买单。++Agent 没装出图技能(`@adelie/use-seedream` 带来的 `seedream`)时两个按钮都不出现:只能失败的按钮比+没有按钮更糟,这也是胶囊条一开始就立下的规矩。++## 细节++- `packages/ui/src/components/overlays/lightbox/lightbox.tsx`:`LightboxAction`(`key` / `label` /+ `onSelect`)成为 `Lightbox` 与 `ZoomableImage` 的可选 `actions`,画在胶囊条里「下载」之后。选中+ 先关覆盖层再执行 —— 动作多半要落在下面的页面上(输入框里的一段话、给 Agent 的一轮对话),隔着遮+ 罩做等于让用户看回自己刚点过的那张图。+- `packages/ui/src/components/content/prose/prose.tsx`:`ProseImagesProvider` 多一个可选+ `actions` 解析器(`ProseImageActionsResolver`),与旁边的图片/链接解析器同形,`MdImage` 把它交给+ 放大层。传下去的是 **Markdown 写下的 src**,不是解析器换出来的 URL —— 上层按 src 认文件,文件路+ 由那一串 URL 它认不出来。+- `packages/web/src/lib/image-actions.ts`(新):`hasSeedreamSkill`、`replyImageActionBuilder`。+ 纯逻辑,两个动作的接线都在这里,词典里的措辞由调用方传入,所以中英各自一处。+- `packages/web/src/features/chat/workspace-refs.tsx`:会话多一个按工作区路径回答「这张图能做什么」的+ 解析器,与取文件走同一个 `resolveReplyImage`(同一条路径分拣两次,是两次各说各话的机会)。+- `packages/web/src/features/chat/chat-page.tsx`:技能清单本来就在(输入框的技能选择用同一份),够+ 判断要不要出按钮。拆图层用 `api.postTask(..., { queueIfBusy: true })` 直接发,不动用户正在写的+ 草稿;改一版走 `ComposerControl.fillPrompt`,并钉上 `seedream`,与拆图层发出的是同一条+ `[use_skills]` 消息。+- `packages/web/src/features/chat/message-stream.tsx`:渲染上下文多 `imageActions`,透给 Markdown。+- `packages/web/src/lib/strings{,-en}.ts`:`chat.imageAction` —— 两个按钮的字,以及各自那句话+ (提示词就是消息本身,所以跟着界面语言走)。+- 测试:`packages/web/test/image-actions.test.ts`(新:技能门禁、两个动作的名字与去向、+ `[use_skills]` 包装、路径原样带上)、`packages/ui/test/lightbox.test.ts`(条上的按钮、没有动作时+ 条子一个像素不变、先关再执行)、`packages/ui/test/prose.test.ts`(解析器拿到的是写下的 src)、+ `packages/web/e2e/reply-picture-actions.spec.mjs`(新,真浏览器:没装技能的 Agent 不出按钮,装上之后点了按钮,覆盖层关掉、输入框里是带路径的那句话,拆图层则作为一条带技能标记的消息发出去)。++## 兼容++- 接口、磁盘格式、线上协议都没变。没装出图技能的 Agent、以及一切非会话的 Markdown 界面,行为与+ 从前完全一致:图片照旧只可放大与下载。diff --git a/packages/ui/src/components/content/prose/prose.tsx b/packages/ui/src/components/content/prose/prose.tsxindex c892b14c..3922d7d4 100644--- a/packages/ui/src/components/content/prose/prose.tsx+++ b/packages/ui/src/components/content/prose/prose.tsx@@ -28,7 +28,9 @@* A picture is a plain `<img>` — unless a `ProseImagesProvider` above turns `zoomable` on, which is* what a conversation does: there a click opens the picture in the lightbox, because a 2000px* screenshot fitted to the reading column is too small to read and there was no way to see it whole- * short of the Files panel. Every other surface keeps the still picture a README expects.+ * short of the Files panel. Every other surface keeps the still picture a README expects. The same+ * provider may bring a second resolver saying what can be done with the picture once it is open —+ * the conversation's Agent having an image tool, say — and those actions ride to the lightbox.** Two optional props open the renderer to a surface with its own nodes — the channel message body,* which keeps `@mentions` as chips — without a second copy of the pipeline drifting from this one.@@ -41,6 +43,7 @@ import ReactMarkdown from "react-markdown";import type { Components, ExtraProps, Options } from "react-markdown";import { CodeBlock } from "../code-block/code-block";import { ZoomableImage } from "../../overlays/lightbox/lightbox";+import type { LightboxAction } from "../../overlays/lightbox/lightbox";import { NO_REHYPE_PLUGINS, REHYPE_PLUGINS, REMARK_PLUGINS } from "./markdown-plugins";import "katex/dist/katex.min.css";import "./prose.css";@@ -119,6 +122,23 @@ export type ProseImageResolver = (src: string, alt: string) => { src: string } |const ProseImagesContext = createContext<ProseImageResolver | null>(null);+/**+ * What a surface above can do with one picture, from the src and alt the Markdown wrote — the+ * same pair the image resolver gets, so a surface that has already resolved a src to a file can+ * offer actions about that file. `null` (or an empty list) means nothing beyond the lightbox's+ * own download and stepping.+ *+ * A different question from the resolver's, which says where the picture is FETCHED from: a+ * surface may serve the file and still have nothing to offer, and the App's own origin serves no+ * file at all while the actions are about one. Kept apart so each answer stays honest.+ */+export type ProseImageActionsResolver = (+ src: string,+ alt: string,+) => readonly LightboxAction[] | null;++const ProseImageActionsContext = createContext<ProseImageActionsResolver | null>(null);+/** Whether a click on a picture below opens it in the lightbox; off unless a provider turns it on. */const ProseImageZoomContext = createContext(false);@@ -145,15 +165,25 @@ const ProseInLinkContext = createContext(false);export function ProseImagesProvider({resolve,zoomable = false,+ actions = null,children,}: {resolve: ProseImageResolver | null;zoomable?: boolean;+ /**+ * What can be done with the pictures below, on top of the lightbox's download (see+ * {@link ProseImageActionsResolver}). Only a surface that mounts the lightbox reads it; it+ * costs a surface that leaves it out nothing, and it too must keep its identity while its+ * inputs do.+ */+ actions?: ProseImageActionsResolver | null;children?: ReactNode;}): ReactElement {return (<ProseImageZoomContext.Provider value={zoomable}>- <ProseImagesContext.Provider value={resolve}>{children}</ProseImagesContext.Provider>+ <ProseImageActionsContext.Provider value={actions}>+ <ProseImagesContext.Provider value={resolve}>{children}</ProseImagesContext.Provider>+ </ProseImageActionsContext.Provider></ProseImageZoomContext.Provider>);}@@ -197,12 +227,14 @@ function MdLink({** With `zoomable` above, the picture becomes a {@link ZoomableImage} and a click opens it whole —* which is the difference between seeing that a screenshot is a screenshot and reading what is in- * it. Two cases keep the still `<img>`: an empty src, where there is nothing to enlarge (the- * lightbox would open on the browser's broken-image placeholder), and a picture the author wrapped- * in a link (`[](…)`) — there the click is the link's, and a button in an anchor- * would fire both (measured: the button's own handler, then the anchor's navigation), so the- * picture would open a tab and an overlay at once. An author who made the picture a link meant the- * link.+ * it. A provider that also offered actions hands them to that lightbox; the picture itself stays+ * exactly the thumbnail it was, because an action row on every picture in a long transcript would+ * be a gallery of buttons rather than a conversation. Two cases keep the still `<img>`: an empty+ * src, where there is nothing to enlarge (the lightbox would open on the browser's broken-image+ * placeholder), and a picture the author wrapped in a link (`[](…)`) — there the+ * click is the link's, and a button in an anchor would fire both (measured: the button's own+ * handler, then the anchor's navigation), so the picture would open a tab and an overlay at once.+ * An author who made the picture a link meant the link.*/function MdImage({node: _node,@@ -212,12 +244,24 @@ function MdImage({}: ComponentPropsWithoutRef<"img"> & ExtraProps) {const resolve = useContext(ProseImagesContext);const zoomable = useContext(ProseImageZoomContext);+ const actions = useContext(ProseImageActionsContext);const inLink = useContext(ProseInLinkContext);const written = typeof src === "string" ? src : "";const resolved = written === "" ? null : (resolve?.(written, alt ?? "") ?? null);const href = resolved?.src ?? written;if (zoomable && !inLink && href !== "") {- return <ZoomableImage src={href} alt={alt ?? ""} lazy {...imageProps} />;+ // Asked with the src as WRITTEN, not the href the resolver produced: the surface above is the+ // one that knows what that src names, and a file route's URL says nothing a caller can use.+ const picture = actions?.(written, alt ?? "") ?? null;+ return (+ <ZoomableImage+ src={href}+ alt={alt ?? ""}+ lazy+ {...(picture !== null && picture.length > 0 ? { actions: picture } : {})}+ {...imageProps}+ />+ );}return (<img {...imageProps} {...(href === "" ? {} : { src: href })} alt={alt ?? ""} loading="lazy" />diff --git a/packages/ui/src/components/overlays/lightbox/lightbox.tsx b/packages/ui/src/components/overlays/lightbox/lightbox.tsxindex 5fc1669d..2df799d7 100644--- a/packages/ui/src/components/overlays/lightbox/lightbox.tsx+++ b/packages/ui/src/components/overlays/lightbox/lightbox.tsx@@ -12,8 +12,11 @@* inside a dialog closes alone and leaves the dialog up.** The bar carries only what the page can really do: save the file (a plain `download` link, no- * request of its own) and move within the batch. Editing actions a generation service would back- * have no endpoint behind them here, and a row of buttons that cannot work is worse than none.+ * request of its own), move within the batch, and whatever the caller's `actions` add. The+ * package itself knows of no generation service, so it offers none — a row of buttons that cannot+ * work is worse than none — but a surface that does know of one (a conversation whose Agent has an+ * image tool) hands its own words and its own work in, and the bar is where they belong: the+ * gesture starts from the picture, at its own size, in one place rather than two.** The close glyph and the size badge hang in the picture's top corners while the picture leaves* room for them; on a picture smaller than either chip they are set beside it instead — the size@@ -34,6 +37,26 @@ import type { LightboxImage } from "./image-board";export type { LightboxImage };+/**+ * One thing the surface that opened the lightbox can do with the picture, beside the download.+ * The lightbox draws the button and owns nothing else about it: the caller writes the word on it,+ * decides in its own language, and does the work — because whether there is anything to do, and+ * what, depends on the page rather than on the picture.+ *+ * `onSelect` runs after the overlay has closed. That order is the point: the action usually puts+ * something in the page underneath (a prompt in the composer, a request to the Agent), and doing+ * it under a scrim leaves the reader looking at the picture they just acted on instead of at the+ * result.+ */+export interface LightboxAction {+ /** Identity of the action across renders. */+ key: string;+ /** The word on the button, in the caller's language. */+ label: string;+ /** What the action does, run once the overlay is gone. */+ onSelect: () => void;+}+export interface LightboxProps {open: boolean;src: string;@@ -48,9 +71,14 @@ export interface LightboxProps {* download.*/images?: readonly LightboxImage[];+ /**+ * Extra things this page can do with the picture, drawn in the bar after the download. Omitted+ * or empty, the bar is exactly the download and the stepping (see {@link LightboxAction}).+ */+ actions?: readonly LightboxAction[];}-export function Lightbox({ open, src, alt, onClose, closeLabel, images }: LightboxProps) {+export function Lightbox({ open, src, alt, onClose, closeLabel, images, actions }: LightboxProps) {const strings = useUiStrings();// Closing resets the step, so the next opening starts on the caller's own picture rather than on// whichever one the reader last looked at.@@ -162,7 +190,8 @@ export function Lightbox({ open, src, alt, onClose, closeLabel, images }: Lightb</div>)}- {/* The capsule bar: step within the batch on the left, save on the right. */}+ {/* The capsule bar: step within the batch on the left, save on the right, and whatever the+ page itself can do with the picture beyond saving it. */}<div className="anim-pop flex items-center gap-1 rounded-full border border-line bg-surface px-1.5 py-1">{multi && (<>@@ -195,6 +224,22 @@ export function Lightbox({ open, src, alt, onClose, closeLabel, images }: Lightb<DownloadIcon /><span>{strings.download}</span></a>+ {/* The page's own actions. The overlay closes first, and the action then runs against the+ page the picture came from — `close()` also drops the step, so reopening starts on the+ caller's picture again. */}+ {actions?.map((action) => (+ <button+ key={action.key}+ type="button"+ onClick={() => {+ close();+ action.onSelect();+ }}+ className="flex h-7 items-center rounded-full px-2 text-xs text-fg transition-colors duration-150 hover:bg-surface-muted"+ >+ {action.label}+ </button>+ ))}</div></div>,document.body,@@ -209,6 +254,7 @@ export function ZoomableImage({closeLabel,lazy = false,group,+ actions,...imgProps}: {src: string;@@ -221,6 +267,8 @@ export function ZoomableImage({lazy?: boolean;/** The batch this picture belongs to, when the caller has one (a message's attachments). */group?: readonly LightboxImage[];+ /** What this page can do with the picture, drawn in the lightbox's bar (see {@link LightboxAction}). */+ actions?: readonly LightboxAction[];} & Omit<ComponentPropsWithoutRef<"img">, "src" | "alt" | "className" | "loading">) {const [open, setOpen] = useState(false);return (@@ -244,6 +292,7 @@ export function ZoomableImage({onClose={() => setOpen(false)}closeLabel={closeLabel}{...(group !== undefined ? { images: group } : {})}+ {...(actions !== undefined ? { actions } : {})}/></>);diff --git a/packages/ui/test/lightbox.test.ts b/packages/ui/test/lightbox.test.tsindex 6d303047..de796e93 100644--- a/packages/ui/test/lightbox.test.ts+++ b/packages/ui/test/lightbox.test.ts@@ -177,6 +177,32 @@ describe("Lightbox's bar", () => {});});+describe("Lightbox's actions", () => {+ const actions = [+ { key: "layers", label: "拆图层", onSelect: noop },+ { key: "redraw", label: "照这张改一版", onSelect: noop },+ ] as const;++ it("draws the page's own actions in the bar, in the caller's words", () => {+ const html = renderStatic(+ createElement(Lightbox, { open: true, src: "/a.png", alt: "Chart", onClose: noop, actions }),+ );+ const labels = [...html.matchAll(/<button[^>]*>([^<]*)<\/button>/g)].map((m) => m[1]);+ expect(labels).toEqual(["拆图层", "照这张改一版"]);+ });++ it("leaves the bar exactly as it was when the page offers nothing", () => {+ const html = lightbox();+ expect(html).toContain(">Download</span>");+ expect([...html.matchAll(/<button\b/g)]).toHaveLength(1); // the close glyph, and nothing else+ });++ it("closes the overlay before running the action, so the page underneath acts in the open", () => {+ const text = readFileSync(join(SRC_DIR, "components/overlays/lightbox/lightbox.tsx"), "utf8");+ expect(text).toMatch(/close\(\);\s*\n\s*action\.onSelect\(\);/);+ });+});+describe("Lightbox with a batch", () => {it("opens on the picture that was clicked, not on the batch's first", () => {const [shown] = images(batchOpened());diff --git a/packages/ui/test/prose.test.ts b/packages/ui/test/prose.test.tsindex a8b9112e..6a97ce72 100644--- a/packages/ui/test/prose.test.ts+++ b/packages/ui/test/prose.test.ts@@ -181,6 +181,24 @@ describe("Md images", () => {expect(html).not.toContain("src=");});+ it("ask the actions resolver about the src as written, not the one the resolver produced", () => {+ // What the surface above can do with the picture depends on what its src NAMES — a Workspace+ // path — whereas the URL it renders is that surface's own private spelling (a file route).+ const seen: [string, string][] = [];+ renderStatic(+ createElement(ProseImagesProvider, {+ resolve: () => ({ src: "/api/sessions/s1/files/content?path=a.png" }),+ zoomable: true,+ actions: (src, alt) => {+ seen.push([src, alt]);+ return null;+ },+ children: createElement(Md, { text: "" }),+ }),+ );+ expect(seen).toEqual([["a.png", "截图"]]);+ });+it("leave a picture the author wrapped in a link to that link", () => {const html = renderStatic(createElement(ProseImagesProvider, {diff --git a/packages/web/e2e/reply-picture-actions.spec.mjs b/packages/web/e2e/reply-picture-actions.spec.mjsnew file mode 100644index 00000000..11bf1b40--- /dev/null+++ b/packages/web/e2e/reply-picture-actions.spec.mjs@@ -0,0 +1,137 @@+/**+ * A reply's picture offers what the Agent can do with it.+ *+ * Opening a picture in a conversation used to offer saving it and stepping through the batch. The+ * lightbox now also carries whatever the conversation hands it (`actions`), and for a reply that+ * is the two things the image tool can do from a Workspace file: split it into layers, or redraw+ * from it with the instruction left for the user to finish. Both are sentences to the Agent — the+ * App has no image endpoint of its own — so what this spec checks in a real browser is exactly the+ * two ends that only a browser can show: the buttons appear only once the Agent really has the+ * Skill, and a click closes the overlay and lands on the page underneath with the picture's own+ * path already in it.+ *+ * Seeded through the API against the mock model (mock-llm.mjs, branch "image reply test"), with a+ * real PNG in the Session's Workspace first — the picture has to load for its thumbnail to be the+ * button the click goes to.+ */+import fs from "node:fs";+import os from "node:os";+import path from "node:path";+import { test, expect } from "@playwright/test";+import { provisionAndLogin } from "./auth.mjs";++const BASE = process.env.BASE_URL;+const MOCK = process.env.MOCK_URL;+const U = "replypictureaction";+const P = "password123";++/** A 2×2 PNG, the same one reply-image.spec uses: real enough that a browser decodes it. */+const PNG_2X2 =+ "iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAYAAABytg0kAAAAFklEQVQIW2P8z8Dwn4GBgYGJAQ0AAB6nAgHkPj0eAAAAAElFTkSuQmCC";++/** The Agent this spec drives: created without plugins, so it starts with no image Skill at all. */+const AGENT = "pic_actions_agent";++test("a reply picture's actions need the Agent's image Skill, and act on the page", async ({+ page,+}) => {+ await provisionAndLogin(page.request, U, P);+ const projects = await (await page.request.get(`${BASE}/api/projects`)).json();+ const projectId = projects.projects[0].projectId;++ const put = await page.request.put(`${BASE}/api/projects/${projectId}/models`, {+ data: {+ defaultModel: { provider: "custom", modelId: "claude-4-8" },+ models: [+ {+ provider: "custom",+ modelId: "claude-4-8",+ apiKey: "sk-mock",+ baseUrl: MOCK,+ contextWindow: 200000,+ },+ ],+ },+ });+ expect(put.ok(), "put models").toBeTruthy();++ // A plain Agent — no plugins, therefore no `seedream` Skill — and a Workspace already holding+ // the picture the reply will name.+ const created = await page.request.post(`${BASE}/api/projects/${projectId}/agents`, {+ data: { agentId: AGENT },+ });+ expect(created.ok() || created.status() === 409, await created.text()).toBeTruthy();++ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), "e2e-reply-picture-actions-"));+ fs.writeFileSync(path.join(workspace, "shot.png"), Buffer.from(PNG_2X2, "base64"));++ const sess = await (+ await page.request.post(`${BASE}/api/projects/${projectId}/agents/${AGENT}/sessions`, {+ data: { provider: "custom", modelId: "claude-4-8", workspace },+ })+ ).json();+ const sessionId = sess.session.sessionId;++ await page.goto(`${BASE}/chat/${sessionId}`);+ const composer = page.getByPlaceholder(/输入消息/);+ await composer.waitFor();+ await composer.fill("image reply test, please");+ await page.getByRole("button", { name: "发送" }).click();++ const pictures = page.locator(".md-body img");+ await expect(pictures).toHaveCount(3, { timeout: 30_000 });+ const picture = pictures.nth(0);+ await expect+ .poll(() => picture.evaluate((img) => img.complete && img.naturalWidth > 0), {+ timeout: 15_000,+ })+ .toBe(true);++ // —— Without the Skill: the bar is exactly the download, the picture opens all the same ——+ const scrim = page.locator(".ui-scrim");+ await picture.click();+ await expect(scrim).toHaveCount(1);+ await expect(scrim.getByRole("link", { name: "下载" })).toBeVisible();+ await expect(scrim.getByRole("button", { name: "拆图层" })).toHaveCount(0);+ await expect(scrim.getByRole("button", { name: "照这张改一版" })).toHaveCount(0);+ await page.keyboard.press("Escape");+ await expect(scrim).toHaveCount(0);++ // —— Install the image tool on that Agent (the library's `use-seedream`, which ships the+ // `seedream` Skill) and reload: the two buttons are there ——+ const install = await page.request.post(+ `${BASE}/api/projects/${projectId}/agents/${AGENT}/plugins`,+ { data: { names: ["use-seedream"] } },+ );+ expect(install.ok(), await install.text()).toBeTruthy();+ const skills = await (+ await page.request.get(`${BASE}/api/projects/${projectId}/agents/${AGENT}/skills`)+ ).json();+ expect(skills.skills.map((s) => s.name)).toContain("seedream");++ await page.reload();+ await picture.waitFor();+ await picture.click();+ await expect(scrim).toHaveCount(1);+ const redraw = scrim.getByRole("button", { name: "照这张改一版" });+ await expect(redraw).toBeVisible();++ // —— Redraw stops at the composer: the instruction carries the picture's own Workspace path,+ // the overlay is gone, and nothing was posted ——+ await redraw.click();+ await expect(scrim).toHaveCount(0);+ await expect(composer).toHaveValue(+ "用 seedream 技能,以工作区里的 `shot.png` 作为参考图改一版:",+ );++ // —— Layer split goes out at once, as a message of the user's own, still naming the Skill ——+ await composer.fill("");+ await picture.click();+ await expect(scrim).toHaveCount(1);+ await scrim.getByRole("button", { name: "拆图层" }).click();+ await expect(scrim).toHaveCount(0);+ await expect(page.getByText("使用技能:seedream", { exact: false })).toBeVisible({+ timeout: 15_000,+ });+ await expect(page.getByText("拆成图层(--layers)", { exact: false })).toBeVisible();+});diff --git a/packages/web/src/features/chat/chat-page.tsx b/packages/web/src/features/chat/chat-page.tsxindex 7bc803e1..a3f39edd 100644--- a/packages/web/src/features/chat/chat-page.tsx+++ b/packages/web/src/features/chat/chat-page.tsx@@ -102,6 +102,7 @@ import { ChatInput } from "./chat-input";import type { ComposerControl } from "./chat-input";import { steerVideoFields } from "../../lib/video-input";import type { ComposerVideo } from "../../lib/video-input";+import { SEEDREAM_SKILL, replyImageActionBuilder } from "../../lib/image-actions";import { onPluginConfigSaved } from "../../lib/plugin-config-event";import { approvalModeChoices } from "./approval-mode";import type { ComposerReference } from "../../lib/workspace-tree";@@ -1580,6 +1581,50 @@ export function ChatPage() {composerRef.current?.addReference(reference);}, []);+ /**+ * A reply picture's actions (lib/image-actions.ts), offered only while this Session's Agent has+ * the image Skill installed — the list is already here for the composer's own Skill picker, and+ * an Agent without it could only fail, so the picture keeps the bar it has always had.+ *+ * Nothing is posted by the browser against an image endpoint: the App has none. "Split into+ * layers" is one deterministic Seedream call, so it goes out as a turn of the user's own; the+ * redraw stops at the composer, because what changes is the prompt and generation is billed per+ * image. Both leave whatever the user had typed alone — the turn is posted directly rather than+ * through the composer's Send, and only the redraw (a composed prompt, like the schedules+ * panel's and the draft screen's) takes the text body, which is what that gesture asks for.+ */+ const askWithPicture = useCallback(+ async (message: string) => {+ if (!selected) return;+ try {+ // queueIfBusy: an action tapped while a turn is running joins the queue rather than+ // failing on a busy Session.+ await api.postTask(selected.sessionId, {+ input: [{ type: "text", text: message }],+ queueIfBusy: true,+ });+ } catch (e) {+ toastError(apiErrorText(e, { modelId: selected.modelId }));+ }+ },+ [selected, toastError],+ );+ const composeImageRedraw = useCallback((prompt: string) => {+ // The Skill rides with the prompt, so pressing Send builds the same [use_skills] message the+ // one-click action would have posted.+ composerRef.current?.fillPrompt(prompt, [SEEDREAM_SKILL]);+ }, []);+ const imageActions = useMemo(+ () =>+ replyImageActionBuilder({+ skills: agentSkills,+ words: S.chat.imageAction,+ ask: (message) => void askWithPicture(message),+ compose: composeImageRedraw,+ }),+ [agentSkills, askWithPicture, composeImageRedraw],+ );+// A fresh Session row from the server, applied wherever the page reads the row from: the// paged list, and the direct lookup's copy for a row the list does not hold (see// resolveRoutedSession).@@ -2008,6 +2053,7 @@ export function ChatPage() {},onOpenFile: openWorkspaceFile,...(workspaceFileUrl === undefined ? {} : { workspaceFileUrl }),+ ...(imageActions === undefined ? {} : { imageActions }),onOpenSubagent: (sessionId, origin) => {// Chip click: the agents tab focused on that child (the focus chain ends with the// child's own id), with the graph pinned to this chip's Task.diff --git a/packages/web/src/features/chat/message-stream.tsx b/packages/web/src/features/chat/message-stream.tsxindex 30db7448..bdb98fe1 100644--- a/packages/web/src/features/chat/message-stream.tsx+++ b/packages/web/src/features/chat/message-stream.tsx@@ -10,6 +10,7 @@import { useEffect, useLayoutEffect, useRef, useState } from "react";import type { ReactNode, RefObject } from "react";import { EmptyState, Spinner } from "@adelie/ui";+import type { LightboxAction } from "@adelie/ui";import { S } from "../../lib/strings";import type { ChatItem } from "../../lib/omni/stream-model";import type { MemoryChangeRow } from "../../lib/omni/memory-changes";@@ -70,6 +71,13 @@ export interface StreamRenderContext {* WorkspaceRefsProvider).*/workspaceFileUrl?: (path: string) => string;+ /**+ * What this conversation can do with a picture a reply points at, given the picture's+ * Workspace-relative path — the lightbox's action row (lib/image-actions.ts). Absent, a picture+ * opens with the download and the stepping alone. Must be stable, for the same reason+ * `workspaceFileUrl` must: every picture in the transcript reads it through context.+ */+ imageActions?: (path: string) => readonly LightboxAction[] | null;/** Opens the subagents panel focused on this child session (subagent chip click); `origin` is the ctx.origin at the chip's render level — the child's ancestor chain, excluding its own id. */onOpenSubagent?: (sessionId: string, origin: string[]) => void;/** Opens the Memory panel on its list (memory-changes card header); the button doesn't render if this isn't wired up. */@@ -550,6 +558,7 @@ export function MessageStream({workspace={ctx.workspace ?? null}onOpenFile={ctx.onOpenFile}fileUrl={ctx.workspaceFileUrl}+ imageActions={ctx.imageActions}><MessageItems items={items} ctx={ctx} /></WorkspaceRefsProvider>diff --git a/packages/web/src/features/chat/workspace-refs.tsx b/packages/web/src/features/chat/workspace-refs.tsxindex 51c9fee4..ae860c77 100644--- a/packages/web/src/features/chat/workspace-refs.tsx+++ b/packages/web/src/features/chat/workspace-refs.tsx@@ -15,11 +15,21 @@* `zoomable` is unconditional: in a conversation a picture is something to look at, so a click* opens it whole whatever its src resolved to — the resolver decides where the picture is fetched* from, not whether the reader may enlarge it.+ *+ * `imageActions` is asked with the same src, on the same terms, and answers what this conversation+ * can do with the FILE that src names (lib/image-actions.ts). One path is sorted once and used for+ * both questions: the picture the lightbox shows and the file an action would name have to be the+ * same file, and two copies of `resolveReplyImage`'s rules are two chances to disagree.*/import { useMemo } from "react";import type { ReactNode } from "react";import { ProseImagesProvider, ProseLinksProvider } from "@adelie/ui";-import type { ProseImageResolver, ProseLinkResolver } from "@adelie/ui";+import type {+ LightboxAction,+ ProseImageActionsResolver,+ ProseImageResolver,+ ProseLinkResolver,+} from "@adelie/ui";import { replyLinkBehavior } from "../../lib/reply-link";import { resolveReplyImage } from "../../lib/reply-image";@@ -27,12 +37,20 @@ export function WorkspaceRefsProvider({workspace,onOpenFile,fileUrl,+ imageActions,children,}: {workspace: string | null;onOpenFile: ((path: string) => void) | undefined;/** A URL an `<img>` can fetch for a Workspace-relative path. */fileUrl: ((path: string) => string) | undefined;+ /**+ * What this conversation can do with the picture at a Workspace-relative path, or `undefined`+ * when it can do nothing beyond the lightbox's own download. Kept a Workspace path rather than+ * a src: the caller that knows the Agent's Skills has no business resolving a src, and this is+ * where the src is understood.+ */+ imageActions: ((path: string) => readonly LightboxAction[] | null) | undefined;children: ReactNode;}) {const resolveLink = useMemo<ProseLinkResolver | null>(@@ -54,9 +72,21 @@ export function WorkspaceRefsProvider({},[workspace, fileUrl],);+ const resolveImageActions = useMemo<ProseImageActionsResolver | null>(+ () =>+ imageActions === undefined+ ? null+ : (src) => {+ // Only a Workspace file: the Agent reads what is on its disk, and a picture fetched+ // from elsewhere would have to be downloaded first (which is a different feature).+ const image = resolveReplyImage(src, workspace);+ return image.kind === "file" ? imageActions(image.path) : null;+ },+ [workspace, imageActions],+ );return (<ProseLinksProvider resolve={resolveLink}>- <ProseImagesProvider resolve={resolveImage} zoomable>+ <ProseImagesProvider resolve={resolveImage} zoomable actions={resolveImageActions}>{children}</ProseImagesProvider></ProseLinksProvider>diff --git a/packages/web/src/lib/image-actions.ts b/packages/web/src/lib/image-actions.tsnew file mode 100644index 00000000..0f98739a--- /dev/null+++ b/packages/web/src/lib/image-actions.ts@@ -0,0 +1,88 @@+/**+ * What a conversation offers for a picture a reply points at, once the picture is open in the+ * lightbox: two things, whatever the Agent's image tool can actually do from a file it is handed.+ *+ * The App has no image endpoint of its own — the model call, the credential and the picture all+ * live on the Agent's side — so neither action is a request the browser makes. Both are+ * instructions to the Agent, which is why this module builds text rather than URLs:+ *+ * - **Split into layers** is one Seedream call with `--layers`, deterministic in its options, so+ * the turn is posted straight away (`ask`) and the answer is a directory of transparent PNGs.+ * It goes out wrapped in a `[use_skills]` block, exactly as the composer wraps a selection, so+ * the Skill runs whether or not the model would have reached for it by itself.+ * - **Redraw from this one** is not deterministic: what changes is the prompt, and the prompt is+ * the user's. It stops at the composer (`compose`), prefilled with the reference picture pinned+ * as the seedream Skill — generation is billed per image, and nobody should buy one by tapping+ * a button they read only as "redo this".+ *+ * Nothing here without the Skill: an Agent that cannot run `seedream` is offered neither button,+ * because a control that can only fail is worse than no control (the same rule the lightbox's bar+ * follows, see the package's lightbox.tsx). The words and the instructions arrive from the active+ * dictionary, so this module stays pure and both languages stay in one place.+ */+import { buildSkillsMessage } from "@adelie/core/markers";+import type { LightboxAction } from "@adelie/ui";++/**+ * The Skill directory name the image tool ships as — what an Agent lists once+ * `@adelie/use-seedream` is installed on it, and what `[use_skills]` names. A name, not a kind:+ * the actions read the Agent's installed list, not a plugin table, because the Skill is what the+ * turn will actually invoke.+ */+export const SEEDREAM_SKILL = "seedream";++/** Whether an Agent can run the image tool at all, from its installed Skill list. */+export function hasSeedreamSkill(skills: readonly { name: string }[]): boolean {+ return skills.some((skill) => skill.name === SEEDREAM_SKILL);+}++/** What the two buttons are called, and what each one carries, in the interface's language. */+export interface ReplyImageWords {+ /** The button that asks for the layer split. */+ layers: string;+ /** The button that puts the redraw instruction in the composer. */+ redraw: string;+ /** The layer split's instruction, for the picture at this Workspace-relative path. */+ layersPrompt: (path: string) => string;+ /** The redraw instruction, ending where the user writes what to change. */+ redrawPrompt: (path: string) => string;+}++/** How the conversation acts on an action, given as two callbacks so this module posts nothing itself. */+export interface ReplyImageActionWiring {+ /** The Session Agent's installed Skills (what decides whether there is anything to offer). */+ skills: readonly { name: string }[];+ /** The active dictionary's words for this feature (`S.chat.imageAction`). */+ words: ReplyImageWords;+ /** Sends an instruction to the Agent as a message of the user's own. */+ ask: (message: string) => void;+ /** Puts an instruction in the composer, to be finished and sent by hand. */+ compose: (prompt: string) => void;+}++/**+ * The resolver a conversation hands to its Markdown (`ProseImagesProvider`'s `actions`), or+ * `undefined` when the Agent has no image tool: `undefined` is what the provider takes to mean+ * "nothing extra", and it keeps the lightbox exactly the bar it was before this existed.+ *+ * Stable per wiring, like the image and link resolvers beside it: a new function identity+ * re-renders every picture in the transcript (memoized message bodies included).+ */+export function replyImageActionBuilder(+ wiring: ReplyImageActionWiring,+): ((path: string) => readonly LightboxAction[] | null) | undefined {+ if (!hasSeedreamSkill(wiring.skills)) return undefined;+ const { words } = wiring;+ return (path: string) => [+ {+ key: "seedream-layers",+ label: words.layers,+ onSelect: () => wiring.ask(buildSkillsMessage([SEEDREAM_SKILL], words.layersPrompt(path))),+ },+ {+ key: "seedream-redraw",+ label: words.redraw,+ onSelect: () => wiring.compose(words.redrawPrompt(path)),+ },+ ];+}diff --git a/packages/web/src/lib/strings-en.ts b/packages/web/src/lib/strings-en.tsindex 5fbfcf3a..a21e1e9a 100644--- a/packages/web/src/lib/strings-en.ts+++ b/packages/web/src/lib/strings-en.ts@@ -3524,6 +3524,20 @@ export const en: Strings = {openPreview: "Click to preview",showMoreFiles: (n: number) => `Show ${n} more ${n === 1 ? "file" : "files"}`,showLess: "Show less",+ /**+ * What a conversation can do with a picture a reply points at, once the picture is open in the+ * lightbox: the two buttons' words, and the instruction each one carries (see+ * lib/image-actions.ts). The instruction IS the message — the App has no image endpoint of its+ * own — and `path` is the picture as the reply named it, relative to the Workspace.+ */+ imageAction: {+ layers: "Split into layers",+ redraw: "Redraw from this one",+ layersPrompt: (path: string) =>+ `Using the seedream skill, split \`${path}\` in the Workspace into layers (--layers), writing the base image and every transparent layer PNG back into the Workspace.`,+ redrawPrompt: (path: string) =>+ `Using the seedream skill, make another version of \`${path}\` in the Workspace, with it as the reference image: `,+ },memoryChangesTitle: (n: number) => `${n} memory ${n === 1 ? "update" : "updates"}`,memoryScopeWorkspace: (key: string) => `Workspace memory (${key})`,memoryOpWrite: "Wrote",diff --git a/packages/web/src/lib/strings.ts b/packages/web/src/lib/strings.tsindex f369eda0..70a0b2b8 100644--- a/packages/web/src/lib/strings.ts+++ b/packages/web/src/lib/strings.ts@@ -3572,6 +3572,22 @@ export const zh = {openPreview: "点击预览",showMoreFiles: (n: number) => `显示其余 ${n} 个文件`,showLess: "收起",+ /**+ * What a conversation can do with a picture a reply points at, once the picture is open in the+ * lightbox: the two buttons' words, and the instruction each one carries. The instruction is+ * the message itself — the App has no image endpoint of its own, so an action is a sentence to+ * the Agent rather than a request the browser makes (see lib/image-actions.ts). `path` is the+ * picture as the reply named it, relative to the Workspace, and the Skill is named so the turn+ * does not depend on the model reaching for it by itself.+ */+ imageAction: {+ layers: "拆图层",+ redraw: "照这张改一版",+ layersPrompt: (path: string) =>+ `用 seedream 技能把工作区里的 \`${path}\` 拆成图层(--layers),底图和每层透明 PNG 都写回工作区。`,+ redrawPrompt: (path: string) =>+ `用 seedream 技能,以工作区里的 \`${path}\` 作为参考图改一版:`,+ },/** Memory-change card below the file summary and the Memory side panel: titles, scope/op tooltips, collapsed row. */memoryChangesTitle: (n: number) => `${n} 条记忆更新`,memoryScopeWorkspace: (key: string) => `工作区记忆(${key})`,diff --git a/packages/web/test/image-actions.test.ts b/packages/web/test/image-actions.test.tsnew file mode 100644index 00000000..fc537681--- /dev/null+++ b/packages/web/test/image-actions.test.ts@@ -0,0 +1,89 @@+/**+ * A reply picture's actions (lib/image-actions.ts): what a conversation offers for the picture a+ * reply points at, once it is open in the lightbox.+ *+ * The App posts nothing against an image endpoint — it has none, and the model, the credential+ * and the file all live on the Agent's side — so an action is a sentence to the Agent, and this+ * module is where the sentence is built. The two rules worth pinning: nothing is offered to an+ * Agent that has no image Skill (a button that can only fail is worse than none), and the+ * layer split goes out inside a `[use_skills]` block, so the turn runs the Skill whether or not+ * the model would have reached for it.+ */+import { describe, expect, it, vi } from "vitest";+import {+ SEEDREAM_SKILL,+ hasSeedreamSkill,+ replyImageActionBuilder,+} from "../src/lib/image-actions";+import type { ReplyImageActionWiring } from "../src/lib/image-actions";++const WORDS = {+ layers: "拆图层",+ redraw: "照这张改一版",+ layersPrompt: (path: string) => `拆 ${path}`,+ redrawPrompt: (path: string) => `改 ${path}:`,+};++/** A wiring that records what each action did, with the picture named `shot.png`. */+function wiring(overrides: Partial<ReplyImageActionWiring> = {}) {+ const ask = vi.fn();+ const compose = vi.fn();+ const builder = replyImageActionBuilder({+ skills: [{ name: SEEDREAM_SKILL }],+ words: WORDS,+ ask,+ compose,+ ...overrides,+ });+ return { builder, ask, compose };+}++describe("hasSeedreamSkill", () => {+ it("finds the image Skill by the name the Agent lists it under", () => {+ expect(hasSeedreamSkill([{ name: "web-design" }, { name: "seedream" }])).toBe(true);+ });++ it("is not fooled by a neighbouring name", () => {+ expect(hasSeedreamSkill([{ name: "seedream-layers" }, { name: "seedance" }])).toBe(false);+ expect(hasSeedreamSkill([])).toBe(false);+ });+});++describe("replyImageActionBuilder", () => {+ it("offers nothing at all to an Agent without the Skill", () => {+ const { builder } = wiring({ skills: [{ name: "software-engineering" }] });+ expect(builder).toBeUndefined();+ });++ it("answers each picture with the two buttons, in the interface's words", () => {+ const { builder } = wiring();+ expect(builder?.("shot.png")?.map((a) => [a.key, a.label])).toEqual([+ ["seedream-layers", "拆图层"],+ ["seedream-redraw", "照这张改一版"],+ ]);+ });++ it("runs the layer split as the user's own turn, with the Skill pinned", () => {+ const { builder, ask, compose } = wiring();+ builder?.("shots/cat.png")?.[0]?.onSelect();+ expect(ask).toHaveBeenCalledWith(+ "[use_skills]\nskills: seedream\n[/use_skills]\n\n拆 shots/cat.png",+ );+ // The split is deterministic in its options and goes out; the redraw waits for the user.+ expect(compose).not.toHaveBeenCalled();+ });++ it("stops the redraw at the composer, leaving the send to the user", () => {+ const { builder, ask, compose } = wiring();+ builder?.("shots/cat.png")?.[1]?.onSelect();+ expect(compose).toHaveBeenCalledWith("改 shots/cat.png:");+ // Nothing is posted: generation is billed per image, and what changes is the prompt.+ expect(ask).not.toHaveBeenCalled();+ });++ it("carries the picture's path to each action, never a URL", () => {+ const { builder, compose } = wiring();+ builder?.("中文 名.png")?.[1]?.onSelect();+ expect(compose).toHaveBeenCalledWith("改 中文 名.png:");+ });+});diff --git a/packages/web/test/md.test.ts b/packages/web/test/md.test.tsindex f6c9bc3c..2cb45d61 100644--- a/packages/web/test/md.test.ts+++ b/packages/web/test/md.test.ts@@ -31,7 +31,8 @@ const WS = "/home/user/workspaces/tmp-1";/*** Renders inside a conversation's Workspace, the way MessageStream wraps its items. `onOpenFile`* is passed through as given — undefined included — since that is how a stream with no file- * opener wired renders.+ * opener wired renders. No picture actions either: this file is about links and srcs, and+ * image-actions covers what a picture offers once it is open.*/const renderWithOpener = (text: string,@@ -45,6 +46,7 @@ const renderWithOpener = (// A conversation serves a picture out of its own file route; reply-image.test.ts covers// how a src is sorted, this file covers that the src reaching the DOM is that one.fileUrl: (path) => `/api/sessions/s1/files/raw?path=${encodeURIComponent(path)}`,+ imageActions: undefined,children: createElement(Md, { text, streaming }),}),);