Skip to content

AI media harness for planning, generating, and editing locally

TypeScript

Gate image controls by generator model type

Bittic2e152d35173
diff --git a/docs/architecture.md b/docs/architecture.md
index b120e1c..898434d 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -43,7 +43,7 @@ non_diegetic_music: ...
Shots are sequential, later shots carry increasing cut timestamps, camera moves combine motion type with meaningful amplitude and speed, and dialogue uses stable speaker IDs. Keyframe jobs prepend the appropriate first/last-frame alignment instruction. Full-reference jobs additionally define stable subject/picture/video/audio labels and retention relationships. The editor stores both the human creative brief and the compiled H3 prompt so either can be revised.
-Image projects retain the [H3 base prompt structure](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md): `integrated_multimodal_description` begins with `[Shot 1]`, visual style and one still composition. With selected references, `compileImagePrompt` uses the [six-section reference format](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md): `subject_definitions`, `summary`, `retention_analysis`, `detailed_description`, `overall_soundscape`, `non_diegetic_music`. Reference-mode style precedes `[Shot 1]`. Stable `<Subject N>` labels identify referenced content; `<Picture N>` sources follow the exact submitted image-path order. Refmods provide conditioning without consuming picture labels. Both audio fields are `N/A` for the silent still-image workflow. The compiler preserves nested visual elements and verbatim quoted lettering, omitting bounding-box coordinates and color palettes from prompts; it adds no timestamps, subsequent shots or camera motion. Debug Prompt and generation use this same compiler. This applies the published video formats to native still-image requests, which set `stillImage: true` and `frames: 1`.
+Image projects retain the [H3 base prompt structure](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md): `integrated_multimodal_description` begins with `[Shot 1]`, visual style and one still composition. With selected references, `compileImagePrompt` uses the [six-section reference format](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md): `subject_definitions`, `summary`, `retention_analysis`, `detailed_description`, `overall_soundscape`, `non_diegetic_music`. Reference-mode style precedes `[Shot 1]`. Stable `<Subject N>` labels identify referenced content; `<Picture N>` sources follow the exact submitted image-path order. Refmods provide conditioning without consuming picture labels. Both audio fields are `N/A` for the silent still-image workflow. The compiler preserves nested visual elements and verbatim quoted lettering, omitting bounding-box coordinates and color palettes from prompts; it adds no timestamps, subsequent shots or camera motion. Debug Prompt and generation use this same compiler. Generator templates carry an extensible `modelType` identifier; current and legacy templates default to `minimax-h3`. The image inspector hides Placement and Color palette for this type, while canvas box drawing remains available. This applies the published video formats to native still-image requests, which set `stillImage: true` and `frames: 1`.
Image-root edits also use all six reference sections, even without selected references. The original source is submitted as the first reference image and defined as the standalone `<Picture 1>` composition anchor. The summary uses `[keyframe completion]` (plus `reference generation` for additional subjects), and retention marks the edited keyframe `partially_preserved`. Additional reference pictures start at 2; the source consumes one of H3's nine image-reference slots. Each inpainting step uses the previous result's pixels for its edit source while retaining the original picture as the visual anchor. Debug Prompt and saved generation snapshots contain these exact per-edit prompts.
diff --git a/src/components/ImageProject.integration.test.tsx b/src/components/ImageProject.integration.test.tsx
index 2b77703..39a5d67 100644
--- a/src/components/ImageProject.integration.test.tsx
+++ b/src/components/ImageProject.integration.test.tsx
@@ -28,7 +28,7 @@ it.each([false, true])("scopes generation controls to each image and queues othe
vi.spyOn(runtime, "resolveSlopfabPlan").mockResolvedValue({ alignedFrames: 1, canvasWidth: 768, canvasHeight: 768 } as Awaited<ReturnType<typeof runtime.resolveSlopfabPlan>>);
const enqueue = vi.spyOn(runtime, "enqueueSlopfabGeneration").mockResolvedValue(undefined);
const cancel = vi.spyOn(runtime, "cancelSlopfabGeneration").mockResolvedValue(true);
- saveGeneratorTemplateSettings({ defaultTemplateId: "test", templates: [{ id: "test", name: "Test", defaultSteps: 20, attention: "sage2", paths: { ...EMPTY_ENGINE_SETTINGS, transformer: "D:/h3.safetensors" } }] });
+ saveGeneratorTemplateSettings({ defaultTemplateId: "test", templates: [{ id: "test", name: "Test", modelType: "minimax-h3", defaultSteps: 20, attention: "sage2", paths: { ...EMPTY_ENGINE_SETTINGS, transformer: "D:/h3.safetensors" } }] });
const project = record();
project.config.assets = ["First", "Second", "Third"].map((name) => ({ id: name, name, kind: "image", imageDraft,
relativePath: `media/generated/${name}.jpg`, mimeType: "image/jpeg", createdAt: project.config.createdAt,
diff --git a/src/components/workspace/ImageEditor.test.tsx b/src/components/workspace/ImageEditor.test.tsx
index 7205f3a..df12859 100644
--- a/src/components/workspace/ImageEditor.test.tsx
+++ b/src/components/workspace/ImageEditor.test.tsx
@@ -8,7 +8,7 @@ import { ImageEditor } from "./ImageEditor";
import { choose, comboValue } from "./comboTestUtils";
import { imageGenerationSnapshotSchema, parseProjectConfig, type ProjectConfig } from "../../lib/project";
import { imageGenerationSnapshot } from "../../lib/imageHistory";
-import { defaultGeneratorTemplate, saveGeneratorTemplateSettings } from "../../lib/settings";
+import { createGeneratorTemplate, defaultGeneratorTemplate, saveGeneratorTemplateSettings } from "../../lib/settings";
import { imageScenePrompt } from "../../lib/imageScene";
import { compileImageEdits } from "../../lib/imageEditing";
import { invoke } from "@tauri-apps/api/core";
@@ -32,6 +32,28 @@ function setup(initial = parseProjectConfig(fixture)) {
/* Undo lives in the title bar; inside the editor it is Ctrl+Z. */
const undoImageEdit = () => fireEvent.keyDown(screen.getByRole("tree", { name: "Image nodes" }), { key: "z", code: "KeyZ", ctrlKey: true });
+it("hides placement and palette controls by model type while preserving boxes and palettes", () => {
+ const h3 = createGeneratorTemplate("Renamed H3");
+ const future = createGeneratorTemplate("Future generator", "future-model");
+ saveGeneratorTemplateSettings({ templates: [h3, future], defaultTemplateId: h3.id, catalogVersion: 9 });
+ const current = setup();
+ const before = structuredClone(current().imageScene!.nodes);
+ expect(screen.queryByText("Color palette")).not.toBeInTheDocument();
+ fireEvent.click(screen.getByRole("button", { name: "Title" }));
+ expect(screen.queryByText("Placement (0–1000)")).not.toBeInTheDocument();
+ expect(screen.queryByText("Color palette")).not.toBeInTheDocument();
+ expect(screen.queryByLabelText("Explicit placement")).not.toBeInTheDocument();
+ expect(screen.getByRole("radio", { name: "Draw object" })).toBeEnabled();
+ expect(screen.getByLabelText("Image placement canvas").querySelectorAll("rect[data-node]").length).toBeGreaterThan(0);
+ choose("Generator", "Future generator");
+ expect(screen.getByText("Placement (0–1000)")).toBeInTheDocument();
+ expect(screen.getByText("Color palette")).toBeInTheDocument();
+ choose("Generator", "Renamed H3");
+ expect(screen.queryByText("Placement (0–1000)")).not.toBeInTheDocument();
+ expect(screen.queryByText("Color palette")).not.toBeInTheDocument();
+ expect(current().imageScene!.nodes).toEqual(before);
+});
+
it("adds image files in edit mode and saves whole-image and drawn edits with separate references", async () => {
vi.spyOn(persistence, "isTauri").mockReturnValue(true);
vi.spyOn(persistence, "readMediaFileUrl").mockResolvedValue(null);
@@ -69,6 +91,8 @@ it("adds image files in edit mode and saves whole-image and drawn edits with sep
expect(screen.getByRole("radio", { name: "Draw edit" })).toHaveAttribute("aria-checked", "true");
pointer("pointerDown", 100, 80); pointer("pointerMove", 400, 320); pointer("pointerUp", 400, 320);
expect(current().imageScene!.nodes[1]).toMatchObject({ name: "Edit", box: { x: 100, y: 100, width: 300, height: 300 } });
+ expect(screen.queryByText("Placement (0–1000)")).not.toBeInTheDocument();
+ expect(screen.queryByText("Color palette")).not.toBeInTheDocument();
expect(promptValue(screen.getByLabelText("Edit prompt"))).toBe("");
typePrompt(screen.getByLabelText("Edit prompt"), "Add a red @[ref:Balloon]");
const edits = compileImageEdits(current()).edits;
diff --git a/src/components/workspace/ImageEditor.tsx b/src/components/workspace/ImageEditor.tsx
index 632aba4..1b27bb4 100644
--- a/src/components/workspace/ImageEditor.tsx
+++ b/src/components/workspace/ImageEditor.tsx
@@ -8,7 +8,7 @@ import { compileImagePrompt } from "../../lib/imagePrompt";
import { createEmptyImage, imageFamily, removeImageAsset, restoreGeneratedImage, saveImageDraft } from "../../lib/imageHistory";
import { isTauri } from "../../lib/persistence";
import { PROJECT_RESOLUTIONS, referenceTypeLabel, type ProjectConfig } from "../../lib/project";
-import { defaultGeneratorTemplate, loadDebugOptionsEnabled, loadGeneratorTemplateSettings, subscribeDebugOptions, subscribeGeneratorTemplates, templateNeedsDownload, type GeneratorTemplate } from "../../lib/settings";
+import { defaultGeneratorTemplate, loadDebugOptionsEnabled, loadGeneratorTemplateSettings, MINIMAX_H3_MODEL_TYPE, subscribeDebugOptions, subscribeGeneratorTemplates, templateNeedsDownload, type GeneratorTemplate } from "../../lib/settings";
import { isWorkActive, type WorkItem } from "../../lib/workQueue";
import type { ConfigUpdate } from "./TimelineView";
import { ReferenceImage } from "./ReferenceImage";
@@ -60,6 +60,7 @@ export function ImageEditor({ config, folderPath, onChange: changeConfig, onGene
const [templateId, setTemplateId] = useState(() => localStorage.getItem("slopus.image-generator-template.v1") ?? defaultGeneratorTemplate().id);
const imageTemplates = templates.templates.filter((template) => template.mode !== "animate");
const template = imageTemplates.find((candidate) => candidate.id === templateId) ?? imageTemplates.find((candidate) => !templateNeedsDownload(candidate)) ?? imageTemplates[0];
+ const isMiniMaxH3 = !template || template.modelType === MINIMAX_H3_MODEL_TYPE;
const onChange = (update: ConfigUpdate) => changeConfig((current) => {
const next = typeof update === "function" ? update(current) : update;
const selecting = next.imageScene?.outputAssetId && next.imageScene.outputAssetId !== current.imageScene?.outputAssetId;
@@ -562,7 +563,7 @@ export function ImageEditor({ config, folderPath, onChange: changeConfig, onGene
{boxes && scene.nodes.filter((node) => node.box).map((node) => { const box = pointer.current?.id === node.id && draftBox ? draftBox : node.box!; return <span key={node.id} className="image-box-label" style={{ left: `${box.x / 10}%`, top: `${box.y / 10}%` }}>{node.name}</span>; })}
</div></div>
{(error || work?.error) && <InfoBar className="image-error" severity="error" title="Couldn’t update the image" message={error ?? work?.error ?? ""} onClose={error ? () => setError(null) : undefined} />}
- <div className="image-status" role="status">{active && <ProgressBar value={work.progress * 100} aria-label="Image generation progress" />}<span>{active ? work.detail : imageRoot ? editPlan?.error ?? `${width} × ${height} · ${editPlan?.edits.length} edits in hierarchy order` : !isTauri() ? "Image generation is available in the desktop app." : `${width} × ${height} · Placement boxes guide the prompt`}</span></div>
+ <div className="image-status" role="status">{active && <ProgressBar value={work.progress * 100} aria-label="Image generation progress" />}<span>{active ? work.detail : imageRoot ? editPlan?.error ?? `${width} × ${height} · ${editPlan?.edits.length} edits in hierarchy order` : !isTauri() ? "Image generation is available in the desktop app." : `${width} × ${height}${isMiniMaxH3 ? "" : " · Placement boxes guide the prompt"}`}</span></div>
<ImageBar ref={imageResults} images={images} selectedId={scene.outputAssetId} folderPath={folderPath} onSelect={selectImage}
onMenu={(id, x, y, inFamily) => { setContextMenu(null); setImageMenu({ id, x, y, inFamily }); }} />
</section>
@@ -605,13 +606,13 @@ export function ImageEditor({ config, folderPath, onChange: changeConfig, onGene
<TagEditor key={scene.style.mode} label={scene.style.mode === "photo" ? "Camera / lens" : "Art style"} value={scene.style.detail} suggestions={scene.style.mode === "photo" ? styleSuggestions.PhotoSuggestions : styleSuggestions.ArtSuggestions} onChange={(detail) => commit({ ...scene, style: { ...scene.style, detail } })} />
</PropSection>
</>}
- {selected.kind !== "root" && <PropSection title="Placement (0–1000)" persistKey="image.placement">
+ {!isMiniMaxH3 && selected.kind !== "root" && <PropSection title="Placement (0–1000)" persistKey="image.placement">
<label className="image-check"><input type="checkbox" checked={Boolean(selected.box)} onChange={(event) => patchNode({ box: event.target.checked ? { x: 250, y: 250, width: 500, height: 500 } : null })} />Explicit placement</label>
{selected.box && (["x", "y", "width", "height"] as const).map((field) => <PropRow key={field} label={field === "x" ? "X" : field === "y" ? "Y" : field === "width" ? "Width" : "Height"} htmlFor={rootField(field)}>
<input id={rootField(field)} className="text-field" type="number" min={field === "x" || field === "y" ? 0 : 1} max="1000" value={selected.box![field]} onChange={(event) => { const value = Number(event.target.value); const box = { ...selected.box!, [field]: value }; if (Number.isFinite(value) && box.x >= 0 && box.y >= 0 && box.width > 0 && box.height > 0 && box.x + box.width <= 1000 && box.y + box.height <= 1000) commit(resizeImageNode(scene, selected.id, box)); }} />
</PropRow>)}
</PropSection>}
- {(!imageRoot || selected.colors.length > 0) && <PropSection title="Color palette" persistKey="image.palette">
+ {!isMiniMaxH3 && (!imageRoot || selected.colors.length > 0) && <PropSection title="Color palette" persistKey="image.palette">
<div className="image-palette">{selected.colors.map((color, index) => <div key={index}><input aria-label={`Palette color ${index + 1}`} type="color" value={color} onChange={(event) => patchNode({ colors: selected.colors.map((old, i) => i === index ? event.target.value : old) })} /><button className="icon-button image-pane-button" aria-label={`Remove color ${index + 1}`} data-tooltip="Remove color" onClick={() => patchNode({ colors: selected.colors.filter((_, i) => i !== index) })}><Delete14 aria-hidden="true" /></button></div>)}<button className="secondary-button" disabled={selected.colors.length >= (selected.kind === "root" ? 16 : 5)} onClick={() => patchNode({ colors: [...selected.colors, "#808080"] })}><Add14 aria-hidden="true" />Color</button></div>
</PropSection>}
</div>
diff --git a/src/lib/settings.test.ts b/src/lib/settings.test.ts
index 209f368..6ba9bbd 100644
--- a/src/lib/settings.test.ts
+++ b/src/lib/settings.test.ts
@@ -25,6 +25,20 @@ import {
describe("generator templates", () => {
beforeEach(() => localStorage.clear());
+ it("assigns MiniMax H3 to current and legacy generators and preserves future model types", () => {
+ expect(createGeneratorTemplate().modelType).toBe("minimax-h3");
+ expect(loadGeneratorTemplateSettings().templates.every((template) => template.modelType === "minimax-h3")).toBe(true);
+ for (const modelType of [undefined, null, "", 123]) {
+ localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({
+ templates: [{ id: "legacy", name: "Renamed generator", modelType }], defaultTemplateId: "legacy", catalogVersion: 9,
+ }));
+ expect(loadGeneratorTemplateSettings().templates[0].modelType).toBe("minimax-h3");
+ }
+ const future = createGeneratorTemplate("Future generator", "future-model");
+ saveGeneratorTemplateSettings({ templates: [future], defaultTemplateId: future.id, catalogVersion: 9 });
+ expect(loadGeneratorTemplateSettings().templates[0]).toMatchObject(future);
+ });
+
it("persists MotionCache per template and replaces stale runtime options", () => {
const template = createGeneratorTemplate();
expect(template.motionCache ?? false).toBe(false);
diff --git a/src/lib/settings.ts b/src/lib/settings.ts
index f581c60..1634c74 100644
--- a/src/lib/settings.ts
+++ b/src/lib/settings.ts
@@ -67,9 +67,14 @@ function migratedStorageItem(key: string, legacyKey: string): string | null {
return legacy;
}
+export const MINIMAX_H3_MODEL_TYPE = "minimax-h3";
+
export interface GeneratorTemplate {
id: string;
name: string;
+ /** Stable model-family identifier, independent of template name and mode.
+ * Open to new identifiers as other model families are added. */
+ modelType: string;
defaultSteps: number;
attention: AttentionMode;
motionCache?: boolean;
@@ -164,10 +169,11 @@ const pathsFrom = (value: unknown): EngineSettings => {
return result;
};
-export function createGeneratorTemplate(name = "New template"): GeneratorTemplate {
+export function createGeneratorTemplate(name = "New template", modelType = MINIMAX_H3_MODEL_TYPE): GeneratorTemplate {
return {
id: `generator-template-${crypto.randomUUID()}`,
name,
+ modelType,
defaultSteps: DEFAULT_GENERATION_STEPS,
attention: "sage2",
paths: copyPaths(EMPTY_ENGINE_SETTINGS),
@@ -187,7 +193,7 @@ export function minimaxOriginalTemplate(): GeneratorTemplate {
{ url: paths.transformer, gpuModel: "", minVramGb: 21 },
{ url: "https://huggingface.co/koongrizzly/MiniMax_H3_int4_W4A8_ConvRot_Pruned/resolve/main/diffusion_models/minimax_h3_ref2va_hybrid_b20-49_pruned_w4a8_mixed.safetensors", gpuModel: "", minVramGb: 0 },
];
- return { id: "minimax-h3-original", name: "First/Last Frame", defaultSteps: 20, attention: "sage2", paths, sources };
+ return { id: "minimax-h3-original", name: "First/Last Frame", modelType: MINIMAX_H3_MODEL_TYPE, defaultSteps: 20, attention: "sage2", paths, sources };
}
function minimaxVariantTemplate(id: string, name: string, transformer: string, defaultSteps = 20): GeneratorTemplate {
@@ -236,7 +242,7 @@ export function viggleAnimateTemplate(): GeneratorTemplate {
}
const initialTemplateSettings = (paths = EMPTY_ENGINE_SETTINGS): GeneratorTemplateSettings => ({
- templates: [{ id: "default", name: "Default", defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(paths) }, minimaxOriginalTemplate(), minimaxReferencesTemplate(), minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate()],
+ templates: [{ id: "default", name: "Default", modelType: MINIMAX_H3_MODEL_TYPE, defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(paths) }, minimaxOriginalTemplate(), minimaxReferencesTemplate(), minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate()],
defaultTemplateId: "default",
catalogVersion: 9,
});
@@ -253,6 +259,7 @@ const normalizeTemplateSettings = (value: unknown): GeneratorTemplateSettings |
if (!id || ids.has(id)) return [];
ids.add(id);
const name = typeof candidate.name === "string" && candidate.name.trim() ? candidate.name.trim() : "Untitled template";
+ const modelType = typeof candidate.modelType === "string" && candidate.modelType.trim() ? candidate.modelType.trim() : MINIMAX_H3_MODEL_TYPE;
const defaultSteps = typeof candidate.defaultSteps === "number"
&& Number.isInteger(candidate.defaultSteps)
&& candidate.defaultSteps >= 2
@@ -285,7 +292,7 @@ const normalizeTemplateSettings = (value: unknown): GeneratorTemplateSettings |
url: file.url.trim(), ...(file.role === "promptEmbedding" ? { role: "promptEmbedding" as const } : {}),
...(typeof file.downloadedPath === "string" && file.downloadedPath.trim() && !isDownloadUrl(file.downloadedPath) ? { downloadedPath: file.downloadedPath } : {}) }];
}) : [];
- return [{ id, name, defaultSteps, attention, paths, sources,
+ return [{ id, name, modelType, defaultSteps, attention, paths, sources,
...(typeof candidate.motionCache === "boolean" ? { motionCache: candidate.motionCache } : {}),
...(candidate.additionalSafetensors !== undefined ? { additionalSafetensors } : {}),
...(candidate.mode === "animate" || candidate.mode === "prompt" ? { mode: candidate.mode } : {}),
@@ -434,7 +441,7 @@ export function saveGeneratorTemplateSettings(settings: GeneratorTemplateSetting
export function defaultGeneratorTemplate(settings = loadGeneratorTemplateSettings()): GeneratorTemplate {
return settings.templates.find((template) => template.id === settings.defaultTemplateId && !templateNeedsDownload(template))
?? settings.templates.find((template) => !templateNeedsDownload(template))
- ?? { id: "", name: "No downloaded generators", defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(EMPTY_ENGINE_SETTINGS) };
+ ?? { id: "", name: "No downloaded generators", modelType: MINIMAX_H3_MODEL_TYPE, defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(EMPTY_ENGINE_SETTINGS) };
}
export function loadDefaultGenerationSteps(): number {
diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.ts
index ded926d..04185d6 100644
--- a/src/lib/workQueue.test.ts
+++ b/src/lib/workQueue.test.ts
@@ -56,7 +56,7 @@ const finish = async (queue: WorkQueue, id: string) => {
};
describe("image generation work", () => {
- const template: GeneratorTemplate = { id: "image-test", name: "MiniMax H3", defaultSteps: 20, attention: "sage2", paths: { ...EMPTY_ENGINE_SETTINGS, transformer: "C:/h3.safetensors" } };
+ const template: GeneratorTemplate = { id: "image-test", name: "MiniMax H3", modelType: "minimax-h3", defaultSteps: 20, attention: "sage2", paths: { ...EMPTY_ENGINE_SETTINGS, transformer: "C:/h3.safetensors" } };
const imageProject = (): ProjectRecord => ({ folderPath: "C:/Image", config: createProjectConfig({ name: "Poster", prompt: "An ocean poster", generationType: "image", aspectRatio: "1:1", resolution: "768p", targetDurationSeconds: 60 }) });
it.each([false, true])("queues distinct images in order without changing the viewed image (draft: %s)", async (imageDraft) => {
const { queue } = setup();