AI media harness for planning, generating, and editing locally
TypeScript
Adapt H3 prompts for still image composition
Bitti484c52bc37d8
diff --git a/docs/architecture.md b/docs/architecture.mdindex 1ad025a..1d0c3b3 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -31,7 +31,7 @@ An agent can answer, ask the user a question, or propose a compact JSONL command## MiniMax H3 prompt contract-Base prompts use the official three-field shape:+Video base prompts use the official three-field shape:```textintegrated_multimodal_description: [Shot 1] ...@@ -43,6 +43,8 @@ non_diegetic_music: ...Shots are sequential, later shots carry increasing cut timestamps, camera moves combine motion type with meaningful amplitude and speed, and dialogue uses stable speaker IDs. Keyframe jobs prepend the appropriate first/last-frame alignment instruction. Full-reference jobs additionally define stable subject/picture/video/audio labels and retention relationships. The editor stores both the human creative brief and the compiled H3 prompt so either can be revised.+Image projects use Slopus's still-image adaptation of the [H3 visual prompt guidance](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md), rather than a published image-specific schema. `imageScenePrompt` places medium, look, aesthetics, lighting and camera/art details before one composition. It describes nested visual elements, full-image placement percentages, palettes and verbatim quoted lettering. `compileImagePrompt` appends selected visual references, numbering `<Picture N>` in the exact order of the submitted image paths. It adds no shot markers, timestamps, camera motion or audio sections, and preserves authored descriptions. Debug Prompt and generation use this same compiler. The native request sets `stillImage: true` and `frames: 1`.+## Generation boundaryThe slopfab C API is loaded dynamically by the Rust backend. It resolves requests immediately, runs asynchronously, permits only one active generation per process, reports progress from a worker thread, and returns raw planar RGB frames plus interleaved audio. Slopus therefore owns:diff --git a/src-tauri/assets/agent-image.md b/src-tauri/assets/agent-image.mdindex 49e03e3..43e4fa1 100644--- a/src-tauri/assets/agent-image.md+++ b/src-tauri/assets/agent-image.md@@ -1,5 +1,7 @@This is an IMAGE project. Apply these image-specific instructions instead of video planning instructions. The model is MiniMax H3 in native still-image mode. There is no timeline, soundtrack, scene sequence, or Python worker in this workflow. Use the Agent, Editor and References tabs.+Describe a single visible composition: subjects, pose, spatial relationships, background, lighting, materials and appearance. Write camera/lens details as framing, perspective, focal length and depth of field. For action subjects, describe the moment depicted in the image. Do not add shot markers, timestamps, camera movements, dialogue delivery or audio sections. Put visible lettering verbatim in text nodes, including its language and line breaks. Slopus compiles the hierarchy and style into visual prose and adds the selected picture references; do not paste a video prompt template into descriptions.+Prefer small, targeted commands for edits to the image hierarchy. Unmentioned fields and unrelated nodes are preserved. Every edit response is JSONL with a final {"op":"commit","summary":"..."} line, just like other project commands.- {"op":"image.configure","prompt"?:string,"background"?:string,"style"?:{"mode"?:"photo"|"art","aesthetics"?:string,"lighting"?:string,"medium"?:string,"detail"?:string},"steps"?:integer,"seed"?:integer,"referenceIds"?:string[]}. Updates only supplied fields; prompt edits the root description. Style merges supplied fields. An empty string clears a description, [] clears references. Use project.set for resolution and aspectRatio.diff --git a/src/components/ImageProject.integration.test.tsx b/src/components/ImageProject.integration.test.tsxindex 9f92e1f..5c2a670 100644--- a/src/components/ImageProject.integration.test.tsx+++ b/src/components/ImageProject.integration.test.tsx@@ -39,7 +39,8 @@ it("shows the full image prompt at the end of the inspector only when debug is econst prompt = within(screen.getByRole("dialog", { name: "Debug Prompt" })).getByLabelText("The compiled MiniMax H3 prompt");for (const line of imageScenePrompt(project.config.imageScene!).split("\n")) expect(prompt.textContent).toContain(line);expect(prompt.textContent).toContain('Render the exact text "TO THE MOON".');- expect(prompt.textContent).toContain("Visual style: Watercolor.\nA silver rocket Use <Picture 1>, <Picture 2> as visual references for this subject.\nPeaceful and bright");+ expect(prompt.textContent).toContain("Visual style: Watercolor.");+ expect(prompt.textContent).toContain("Use <Picture 1>, <Picture 2> as visual guidance for Rocket. A silver rocket.\nAdditional visual guidance: Peaceful and bright.");expect(prompt.textContent).not.toContain("Do not include this reference");fireEvent.keyDown(window, { key: "Escape" });expect(screen.queryByRole("dialog", { name: "Debug Prompt" })).not.toBeInTheDocument();diff --git a/src/components/workspace/ImageEditor.test.tsx b/src/components/workspace/ImageEditor.test.tsxindex cbd1921..785ad7b 100644--- a/src/components/workspace/ImageEditor.test.tsx+++ b/src/components/workspace/ImageEditor.test.tsx@@ -46,7 +46,7 @@ it("edits ordered style chips with suggestions, custom tags, removal, undo, andfireEvent.keyDown(window, { key: "Escape" });fireEvent.click(screen.getByRole("button", { name: "Add Camera / lens tag" }));fireEvent.click(screen.getByRole("button", { name: "35mm" }));- expect(imageScenePrompt(current().imageScene!)).toContain("Style: Photograph. Minimal poster, intricate, custom finish. Sunrise, golden hour. Screen print, 35mm.");+ expect(imageScenePrompt(current().imageScene!)).toContain("Medium: Photograph.\nAesthetics: Minimal poster, intricate, custom finish.\nLighting: Sunrise, golden hour.\nCamera and lens: Screen print, 35mm.");fireEvent.keyDown(window, { key: "Escape" });fireEvent.click(screen.getByRole("button", { name: "Add Aesthetics tag" }));expect(screen.getByRole("button", { name: "photorealistic" })).toBeInTheDocument();diff --git a/src/lib/imagePrompt.test.ts b/src/lib/imagePrompt.test.tsnew file mode 100644index 0000000..53985fe--- /dev/null+++ b/src/lib/imagePrompt.test.ts@@ -0,0 +1,79 @@+import { describe, expect, it } from "vitest";+import { compileImagePrompt } from "./imagePrompt";+import { addImageNode, imageScenePrompt } from "./imageScene";+import { createProjectConfig, parseProjectConfig } from "./project";+import fixture from "../../fixtures/project-v1-image.json";++const photo = () => createProjectConfig({ name: "Portrait", generationType: "image", prompt: "A runner suspended mid-stride", aspectRatio: "1:1", resolution: "768p", targetDurationSeconds: 60 });+describe("MiniMax H3 still-image prompts", () => {+ it("places photo style before a single visual composition without video scaffolding", () => {+ const config = photo();+ config.settings.defaultLook = "watercolor";+ config.imageScene!.style = { mode: "photo", aesthetics: "editorial, high contrast", lighting: "golden hour", medium: "Ignored art medium", detail: "85mm, f/1.4" };+ const snapshot = structuredClone(config);+ expect(compileImagePrompt(config).prompt).toBe([+ "Create one still image with a single composition.",+ "Medium: Photograph.",+ "Visual style: Watercolor.",+ "Aesthetics: editorial, high contrast.",+ "Lighting: golden hour.",+ "Camera and lens: 85mm, f/1.4.",+ "", "A runner suspended mid-stride.",+ ].join("\n"));+ expect(compileImagePrompt(config).prompt).not.toMatch(/\[Shot|overall_soundscape|non_diegetic_music|integrated_multimodal_description|00:|root:|Ignored art medium/);+ expect(config).toEqual(snapshot);+ });++ it("preserves nested composition, placement and palettes even for undescribed groups", () => {+ const config = parseProjectConfig(fixture);+ config.imageScene!.nodes[1].description = "";+ const prompt = compileImagePrompt(config).prompt;+ expect(prompt).toContain("Medium: Illustration.");+ expect(prompt).toContain("Art style: Screen print.");+ expect(prompt).toContain("Overall color palette: #204060.");+ expect(prompt).toContain("Background: Ocean at dawn.");+ expect(prompt).toContain("A grouped arrangement. Position in the full image: left 30%, top 10%, width 40%, height 80%. Its composition includes:\n Visible lettering. Bold red lettering.");+ expect(prompt).toContain('Render the exact text "LIFT OFF". Position in the full image: left 35%, top 15%, width 30%, height 10%. Color palette: #FF0000.');+ expect(prompt).not.toMatch(/group:|text:|group-rocket|Camera and lens/);+ });++ it("keeps exact lettering and authored content rather than escaping or stripping it", () => {+ const config = photo();+ const scene = addImageNode(config.imageScene!, "image-root", "text");+ const text = '营业中\nSay "hello" — C:\\Art';+ scene.nodes.at(-1)!.text = text;+ scene.nodes.at(-1)!.description = "Hand-lettered sign";+ expect(imageScenePrompt(scene)).toContain(`Render the exact text "${text}".`);+ expect(imageScenePrompt(scene)).not.toContain("\\n");+ });++ it("numbers pictures in selected payload order and treats text references as visual guidance", () => {+ const config = photo();+ config.references = [+ { id: "runner", kind: "image", name: "Runner", description: "Red jersey", relativePath: "references/runner.png", images: [{ id: "detail", name: "Detail", relativePath: "references/detail.png", sourcePath: null }], intendedUse: [], createdAt: config.createdAt },+ { id: "palette", kind: "text", name: "Palette", description: "", content: "Warm ochre and red", intendedUse: [], createdAt: config.createdAt },+ { id: "track", kind: "image", name: "Track", description: "A gravel track", relativePath: "references/track.png", intendedUse: [], createdAt: config.createdAt },+ { id: "unused", kind: "text", name: "Unused", description: "Exclude these words", intendedUse: [], createdAt: config.createdAt },+ ];+ config.imageScene!.referenceIds = ["track", "palette", "runner"];+ const result = compileImagePrompt(config);+ expect(result.references.map((reference) => reference.id)).toEqual(["track", "palette", "runner"]);+ expect(result.prompt).toContain([+ "Visual references for this still image:",+ "Use <Picture 1> as visual guidance for Track. A gravel track.",+ "Additional visual guidance: Warm ochre and red.",+ "Use <Picture 2>, <Picture 3> as visual guidance for Runner. Red jersey.",+ ].join("\n"));+ expect(result.prompt).not.toMatch(/Exclude these words|<Video|<Audio|first frame|last frame/);+ config.imageScene!.referenceIds = ["missing"];+ expect(() => compileImagePrompt(config)).toThrow("no longer exists");+ });++ it("uses the brief for legacy image documents and leaves genuinely empty prompts empty", () => {+ const config = photo();+ config.imageScene = null;+ expect(compileImagePrompt(config).prompt).toContain("A runner suspended mid-stride.");+ config.brief.prompt = "";+ expect(compileImagePrompt(config).prompt).toBe("");+ });+});diff --git a/src/lib/imagePrompt.ts b/src/lib/imagePrompt.tsindex b892eb0..c849c30 100644--- a/src/lib/imagePrompt.ts+++ b/src/lib/imagePrompt.ts@@ -14,12 +14,17 @@ export function compileImagePrompt(config: ProjectConfig) {let picture = 0;const referencePrompts = references.map((reference) => {const labels = referenceImages(reference).map(() => `<Picture ${++picture}>`);- return `${referenceDefinition(reference)}${labels.length ? ` Use ${labels.join(", ")} as visual references for this subject.` : ""}`;+ const definition = referenceDefinition(reference);+ const content = definition || reference.name;+ const description = /[.!?。!?]$/.test(content) ? content : `${content}.`;+ return labels.length+ ? `Use ${labels.join(", ")} as visual guidance for ${reference.name}. ${description}`+ : `Additional visual guidance: ${description}`;});if (picture > 9) throw new Error("MiniMax H3 supports at most nine reference images per generation.");const look = SHOT_TAG_GROUPS.find((group) => group.id === "visualStyle")?.options.find((option) => option.id === config.settings.defaultLook)?.label;return {- prompt: [imageScenePrompt(scene), ...(look ? [`Visual style: ${look}.`] : []), ...referencePrompts].join("\n"),+ prompt: [imageScenePrompt(scene, look), ...(referencePrompts.length ? [`Visual references for this still image:\n${referencePrompts.join("\n")}`] : [])].filter(Boolean).join("\n\n"),references,};}diff --git a/src/lib/imageScene.ts b/src/lib/imageScene.tsindex 9e8b681..35c596d 100644--- a/src/lib/imageScene.ts+++ b/src/lib/imageScene.ts@@ -73,19 +73,39 @@ export function resizeImageNode(scene: ImageScene, id: string, box: ImageBox): Ireturn { ...node, box: { x, y, width: Math.max(1, Math.min(1000 - x, Math.round(node.box.width * box.width / original.width))), height: Math.max(1, Math.min(1000 - y, Math.round(node.box.height * box.height / original.height))) } };}) });}-/** H3 accepts prose, not ImgFab's Ideogram-specific caption JSON. Coordinates- * are composition guidance in the prompt, not a guarantee of exact placement. */-export function imageScenePrompt(scene: ImageScene): string {- const lines: string[] = [];+const sentence = (text: string) => /[.!?。!?]$/.test(text.trim()) ? text.trim() : `${text.trim()}.`;++/** Slopus's still-image adaptation of H3's visual prose guidance. No generated+ * shot markers, timing or audio sections; boxes refer to the full image. */+export function imageScenePrompt(scene: ImageScene, look?: string): string {+ const root = scene.nodes.find((node) => node.kind === "root")!;+ const hasContent = scene.background.trim() || scene.nodes.some((node) => node.description.trim() || node.colors.length || (node.kind === "text" && node.text) || (node.kind !== "root" && node.box));+ if (!hasContent) return "";+ const style = [+ `Medium: ${sentence(scene.style.mode === "photo" ? "Photograph" : scene.style.medium || "Artwork")}`,+ ...(look?.trim() ? [`Visual style: ${sentence(look)}`] : []),+ ...(scene.style.aesthetics.trim() ? [`Aesthetics: ${sentence(scene.style.aesthetics)}`] : []),+ ...(scene.style.lighting.trim() ? [`Lighting: ${sentence(scene.style.lighting)}`] : []),+ ...(scene.style.detail.trim() ? [`${scene.style.mode === "photo" ? "Camera and lens" : "Art style"}: ${sentence(scene.style.detail)}`] : []),+ ];+ const placement = (node: ImageNode) => node.box ? `Position in the full image: left ${node.box.x / 10}%, top ${node.box.y / 10}%, width ${node.box.width / 10}%, height ${node.box.height / 10}%.` : "";+ const lines: string[] = [+ ...(root.description.trim() ? [sentence(root.description)] : []),+ ...(root.colors.length ? [`Overall color palette: ${root.colors.join(", ")}.`] : []),+ ...(root.box ? [placement(root)] : []),+ ...(scene.background.trim() ? [`Background: ${sentence(scene.background)}`] : []),+ ];const visit = (node: ImageNode, depth: number) => {- const placement = node.box ? ` Placement: left ${node.box.x / 10}%, top ${node.box.y / 10}%, width ${node.box.width / 10}%, height ${node.box.height / 10}% of the image.` : "";- const text = node.kind === "text" && node.text ? ` Render the exact text ${JSON.stringify(node.text)}.` : "";- const colors = node.colors.length ? ` Colors: ${node.colors.join(", ")}.` : "";- if (node.description.trim() || text || colors) lines.push(`${" ".repeat(depth)}${node.kind}: ${node.description.trim()}${text}${placement}${colors}`);- scene.nodes.filter((child) => child.parentId === node.id).forEach((child) => visit(child, depth + 1));+ const children = scene.nodes.filter((child) => child.parentId === node.id);+ const description = node.description.trim() ? sentence(node.description) : "";+ // Quote visible lettering verbatim, including its language and line breaks.+ const text = node.kind === "text" && node.text ? `Render the exact text "${node.text}".` : "";+ const colors = node.colors.length ? `Color palette: ${node.colors.join(", ")}.` : "";+ const subject = node.kind === "group" ? "A grouped arrangement." : node.kind === "text" ? "Visible lettering." : node.kind === "background" ? "A background element." : "A depicted object.";+ const detail = [subject, description, text, placement(node), colors, ...(children.length ? ["Its composition includes:"] : [])].filter(Boolean).join(" ");+ lines.push(`${" ".repeat(depth)}${detail}`);+ children.forEach((child) => visit(child, depth + 1));};- scene.nodes.filter((node) => node.kind === "root").forEach((node) => visit(node, 0));- if (scene.background.trim()) lines.push(`Background: ${scene.background.trim()}`);- const style = [scene.style.mode === "photo" ? "Photograph" : scene.style.medium || "Artwork", scene.style.aesthetics, scene.style.lighting, scene.style.detail].filter(Boolean).join(". ");- return lines.length ? `Create a single still image.\n${lines.join("\n")}\nStyle: ${style}.` : "";+ scene.nodes.filter((node) => node.parentId === root.id).forEach((node) => visit(node, 0));+ return `Create one still image with a single composition.\n${style.join("\n")}\n\n${lines.join("\n")}`;}diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.tsindex d8f20d8..ed0021a 100644--- a/src/lib/workQueue.test.ts+++ b/src/lib/workQueue.test.ts@@ -67,7 +67,8 @@ describe("image generation work", () => {await waitFor(() => expect(enqueueSlopfabGeneration).toHaveBeenCalledOnce());const request = vi.mocked(enqueueSlopfabGeneration).mock.calls[0][0];expect(request.prompt).toBe(preview);- expect(request.prompt).toContain("Visual style: Watercolor.\nTurquoise water Use <Picture 1> as visual references for this subject.");+ expect(request.prompt).toContain("Visual style: Watercolor.");+ expect(request.prompt).toContain("Use <Picture 1> as visual guidance for Ocean. Turquoise water.");expect(request.referencePaths).toEqual(["C:/Image/references/ocean.png"]);await queue.cancel(request.jobId);});