Skip to content

AI media harness for planning, generating, and editing locally

TypeScript

Show live previews in shot cards and images

Bittic2401cc80db3
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 703441c..7a30b7f 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -8,7 +8,7 @@ Changes since 0.3.2.
- **LoRA controls.** Enable MiniMax H3 LoRAs globally, drag the library to set application order, and add scene overrides in Generation. Global adapters remain active and appear read only in scenes. Step counts and sampling recipes follow the effective stack, with conflicting recipes blocked.
-- **Generation previews.** Download TAEH3 from Other weights to play silent MiniMax H3 previews in scene cards during generation, locally or on a worker. A Settings switch controls previews; final videos use the generator's selected VAE.
+- **Generation previews.** MiniMax H3 video previews autoplay and loop inside shot cards with the usual play/pause and pop-out controls. Low-resolution image previews update the image canvas and thumbnail during generation. Download TAEH3 from Other weights and enable generation previews in Settings; previews work locally or on updated workers, while final output uses the generator's selected VAE.
- **Pika H3 Flex.** Added FL2VA and Ref2VA LoRA downloads with Pika's fixed six-step re-noising recipe and video/audio sigma shifts of 12/3.
diff --git a/docs/slopfab-compatibility.md b/docs/slopfab-compatibility.md
index 5f0e65e..1f1ea24 100644
--- a/docs/slopfab-compatibility.md
+++ b/docs/slopfab-compatibility.md
@@ -2,20 +2,24 @@
## Bundled runtime (API 1.46.0)
-Windows and Linux runtimes use SlopFab commit `700affae5b01e4e09af4ecb15289bfd43952e1a5`
+Windows and Linux runtimes use SlopFab commit `2beebd6be5e0c69411e7670e53a789cab9109b6c`
from `D:\Projects\slopfab`, with CUDA and Vulkan enabled and FFmpeg disabled.
This update keeps Keystone and delayed Capstone context inside sampling: the
original front latent is restored before output, and Capstone adds one clean
video latent only during model calls. It does not replace generated end frames
or add hidden tail frames. Saved joined output preserves the previous clip.
-TAEH3 generation previews and saved-latent upscaling remain supported.
+TAEH3 now also decodes single-frame MiniMax image previews on CUDA and Vulkan.
+Video previews autoplay and loop inside the existing shot card player; image
+previews update the image canvas and thumbnail. Both use the generation previews
+setting and require the optional TAEH3 weights. Saved-latent upscaling remains supported.
-LAN protocol is 24. Single Shot automatically shortens its preferred four-update
+LAN protocol is 25 so workers also support still-image previews. Single Shot automatically shortens its preferred four-update
Capstone delay when the resolved video schedule is shorter, without changing the
LoRA recipe. Restart Slopus and update workers to load the new runtime.
Both platform builds pass C API and embedded-tokenizer checks. Slopus core tests
-pass against the Windows DLL. Linux retains the Fibo text-renderer linker
+pass against the Windows DLL. Single-frame TAEH3 decoding matches the upstream
+causal first-frame reference exactly on CUDA and Vulkan. Linux retains the Fibo text-renderer linker
workaround documented below.
## Single Shot
diff --git a/lib/slopfab/libslopfab.so b/lib/slopfab/libslopfab.so
index e146071..80b15e5 100644
--- a/lib/slopfab/libslopfab.so
+++ b/lib/slopfab/libslopfab.so
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
-oid sha256:1f3d0c2a1c8f2a30f02364e481f488fa671506e9806789c104de50da1ca55fbc
+oid sha256:4b5ce49ad603b6867f52f819def6d8282adbeb40a38314785d7bbdb3483b32c3
size 23539272
diff --git a/lib/slopfab/slopfab.dll b/lib/slopfab/slopfab.dll
index b34d221..f7cd859 100644
--- a/lib/slopfab/slopfab.dll
+++ b/lib/slopfab/slopfab.dll
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
-oid sha256:4115966f368b66417bca1fb91d91509e33a05952b4f8e794417918382b0152c4
-size 20017152
+oid sha256:5d3897316ff119acfcf1e93d58ebcc2728b9d7f12fa6cac26197744ed27b6d5b
+size 20016640
diff --git a/src-tauri/core/src/slopfab/generation.rs b/src-tauri/core/src/slopfab/generation.rs
index 190bfb0..23a728d 100644
--- a/src-tauri/core/src/slopfab/generation.rs
+++ b/src-tauri/core/src/slopfab/generation.rs
@@ -240,7 +240,7 @@ fn run_single_generation(
planned_steps: plan.num_model_evaluations,
};
let preview = item.configuration.preview_vae.as_ref()
- .filter(|path| path.is_file() && item.configuration.model_type == "minimax-h3" && !item.request.still_image && item.request.music.is_none())
+ .filter(|path| path.is_file() && item.configuration.model_type == "minimax-h3" && item.request.music.is_none())
.map(|path| (path.clone(), super::events::preview_sink(item.events.clone(), item.request.job_id.clone())));
let mut generation = request.start_with_preview(Some(progress_sink(context)), preview)?;
diagnostics::debug(
diff --git a/src-tauri/core/src/worker/protocol.rs b/src-tauri/core/src/worker/protocol.rs
index 0c08181..3292ee5 100644
--- a/src-tauri/core/src/worker/protocol.rs
+++ b/src-tauri/core/src/worker/protocol.rs
@@ -28,7 +28,8 @@ pub const SERVICE_TYPE: &str = "_slopus-worker._tcp.local.";
// v21 keeps Single Shot's middle fresh; v20 workers default to a repeated source prior.
// v23 carries Single Shot's persistent context archive; older workers ignore Keystone.
// v24 adapts capstone locking to the resolved video schedule, including short LoRA grids.
-pub const PROTOCOL_VERSION: u32 = 24;
+// v25 emits MiniMax still-image previews as well as video previews.
+pub const PROTOCOL_VERSION: u32 = 25;
/// Streaming SeedVR2 restoration; model paths are resolved on the worker.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -383,7 +384,7 @@ mod tests {
let single = restored.single_shot.unwrap();
assert_eq!(single.ending_prompt, "The subject waves goodbye");
assert!(single.ending_path.is_none());
- assert_eq!(PROTOCOL_VERSION, 24);
+ assert_eq!(PROTOCOL_VERSION, 25);
}
#[test]
diff --git a/src/components/workspace/GenerationPreviewPlayer.tsx b/src/components/workspace/GenerationPreviewPlayer.tsx
deleted file mode 100644
index 79a39fe..0000000
--- a/src/components/workspace/GenerationPreviewPlayer.tsx
+++ /dev/null
@@ -1,34 +0,0 @@
-import { useEffect, useRef, useState } from "react";
-import { encodeGenerationPreview, type GenerationPreview } from "../../lib/generationPreview";
-
-export function GenerationPreviewPlayer({ preview }: { preview: GenerationPreview }) {
- const [source, setSource] = useState<{ url: string; step: number }>();
- const [failed, setFailed] = useState(false);
- const video = useRef<HTMLVideoElement>(null);
- const url = useRef<string>();
- const position = useRef({ time: 0, paused: true });
- useEffect(() => {
- const controller = new AbortController();
- void encodeGenerationPreview(preview, controller.signal).then((blob) => {
- if (controller.signal.aborted) return;
- const next = URL.createObjectURL(blob);
- position.current = { time: video.current?.currentTime ?? 0, paused: video.current?.paused ?? true };
- if (url.current) URL.revokeObjectURL(url.current);
- url.current = next;
- setSource({ url: next, step: preview.step });
- setFailed(false);
- }).catch(() => { if (!controller.signal.aborted) setFailed(true); });
- return () => controller.abort();
- }, [preview]);
- useEffect(() => () => { if (url.current) URL.revokeObjectURL(url.current); }, []);
- return <div className="generation-preview" aria-label="Generation preview">
- {source ? <video ref={video} src={source.url} controls muted loop playsInline preload="auto" aria-label="Play generation preview"
- onLoadedMetadata={() => {
- const element = video.current;
- if (!element) return;
- element.currentTime = Math.min(position.current.time, Math.max(0, element.duration - .01));
- if (!position.current.paused) void element.play().catch(() => {});
- }} /> : <img src={`data:image/jpeg;base64,${preview.frames[0]}`} alt="Generation preview" />}
- <span>{failed && !source ? "Preview playback unavailable" : `Generation preview · Step ${(source?.step ?? preview.step) + 1}`}</span>
- </div>;
-}
diff --git a/src/components/workspace/GenerationPreviewPlayer.test.tsx b/src/components/workspace/GenerationPreviewSource.test.tsx
similarity index 61%
rename from src/components/workspace/GenerationPreviewPlayer.test.tsx
rename to src/components/workspace/GenerationPreviewSource.test.tsx
index 9d56b66..970d077 100644
--- a/src/components/workspace/GenerationPreviewPlayer.test.tsx
+++ b/src/components/workspace/GenerationPreviewSource.test.tsx
@@ -2,25 +2,33 @@
import "@testing-library/jest-dom/vitest";
import { cleanup, render, screen, waitFor } from "@testing-library/react";
import { afterEach, expect, it, vi } from "vitest";
+import { useContext } from "react";
import { encodeGenerationPreview, type GenerationPreview } from "../../lib/generationPreview";
-import { GenerationPreviewPlayer } from "./GenerationPreviewPlayer";
+import { GenerationPreviewContext, GenerationPreviewSource } from "./GenerationPreviewSource";
vi.mock("../../lib/generationPreview", () => ({ encodeGenerationPreview: vi.fn() }));
afterEach(() => { cleanup(); vi.restoreAllMocks(); vi.unstubAllGlobals(); });
const preview: GenerationPreview = { jobId: "job", width: 2, height: 2, fps: 4, step: 0, frames: ["/9j/AA=="] };
+function Consumer() {
+ const source = useContext(GenerationPreviewContext);
+ return source?.url ? <video aria-label="Preview" src={source.url} /> : source ? <img src={`data:image/jpeg;base64,${source.preview.frames[0]}`} /> : null;
+}
+function Preview({ preview }: { preview?: GenerationPreview }) {
+ return <GenerationPreviewSource preview={preview}><Consumer /></GenerationPreviewSource>;
+}
it("replaces playable previews, ignores stale encoding and releases URLs on unmount", async () => {
const create = vi.fn().mockReturnValueOnce("blob:first").mockReturnValueOnce("blob:second");
const revoke = vi.fn();
vi.stubGlobal("URL", { createObjectURL: create, revokeObjectURL: revoke });
vi.mocked(encodeGenerationPreview).mockResolvedValueOnce(new Blob());
- const view = render(<GenerationPreviewPlayer preview={preview} />);
- await waitFor(() => expect(screen.getByLabelText("Play generation preview")).toHaveAttribute("src", "blob:first"));
+ const view = render(<Preview preview={preview} />);
+ await waitFor(() => expect(screen.getByLabelText("Preview")).toHaveAttribute("src", "blob:first"));
let stale!: (blob: Blob) => void;
vi.mocked(encodeGenerationPreview).mockImplementationOnce(() => new Promise((resolve) => { stale = resolve; }));
- view.rerender(<GenerationPreviewPlayer preview={{ ...preview, step: 1 }} />);
+ view.rerender(<Preview preview={{ ...preview, step: 1 }} />);
vi.mocked(encodeGenerationPreview).mockResolvedValueOnce(new Blob());
- view.rerender(<GenerationPreviewPlayer preview={{ ...preview, step: 2 }} />);
- await waitFor(() => expect(screen.getByLabelText("Play generation preview")).toHaveAttribute("src", "blob:second"));
+ view.rerender(<Preview preview={{ ...preview, step: 2 }} />);
+ await waitFor(() => expect(screen.getByLabelText("Preview")).toHaveAttribute("src", "blob:second"));
stale(new Blob());
await Promise.resolve();
expect(create).toHaveBeenCalledTimes(2);
@@ -31,7 +39,8 @@ it("replaces playable previews, ignores stale encoding and releases URLs on unmo
it("keeps a still preview when this webview cannot encode previews", async () => {
vi.mocked(encodeGenerationPreview).mockRejectedValue(new Error("unsupported"));
- render(<GenerationPreviewPlayer preview={preview} />);
- expect(await screen.findByText("Preview playback unavailable")).toBeInTheDocument();
+ const view = render(<Preview preview={preview} />);
expect(screen.getByRole("img")).toHaveAttribute("src", "data:image/jpeg;base64,/9j/AA==");
+ view.rerender(<Preview />);
+ expect(screen.queryByRole("img")).toBeNull();
});
diff --git a/src/components/workspace/GenerationPreviewSource.tsx b/src/components/workspace/GenerationPreviewSource.tsx
new file mode 100644
index 0000000..1fb1465
--- /dev/null
+++ b/src/components/workspace/GenerationPreviewSource.tsx
@@ -0,0 +1,20 @@
+import { createContext, useEffect, useState, type ReactNode } from "react";
+import { encodeGenerationPreview, type GenerationPreview } from "../../lib/generationPreview";
+
+export const GenerationPreviewContext = createContext<{ preview: GenerationPreview; url?: string } | undefined>(undefined);
+
+/** Encode once per scene and share the transient clip across its shot cards. */
+export function GenerationPreviewSource({ preview, children }: { preview?: GenerationPreview; children: ReactNode }) {
+ const [source, setSource] = useState<{ preview: GenerationPreview; url: string }>();
+ useEffect(() => {
+ if (!preview) { setSource(undefined); return; }
+ const controller = new AbortController();
+ void encodeGenerationPreview(preview, controller.signal).then((blob) => {
+ if (!controller.signal.aborted) setSource({ preview, url: URL.createObjectURL(blob) });
+ }).catch(() => { /* Keep the previous clip, or show this step's JPEG. */ });
+ return () => controller.abort();
+ }, [preview]);
+ useEffect(() => () => { if (source) URL.revokeObjectURL(source.url); }, [source]);
+ const current = source?.preview.jobId === preview?.jobId ? source : undefined;
+ return <GenerationPreviewContext.Provider value={preview ? current ?? { preview } : undefined}>{children}</GenerationPreviewContext.Provider>;
+}
diff --git a/src/components/workspace/ImageBar.tsx b/src/components/workspace/ImageBar.tsx
index 544d1d0..aaeed4b 100644
--- a/src/components/workspace/ImageBar.tsx
+++ b/src/components/workspace/ImageBar.tsx
@@ -43,6 +43,7 @@ export const ImageBar = forwardRef<HTMLDivElement | null, {
const shown = inFamily ? family : primaries;
const pressed = inFamily ? selectedId : root;
const childCount = (id: string) => images.filter((asset) => asset.id !== id && imageFamilyRoot(images, asset.id) === id).length;
+ const previews = new Map(workItems.filter(isWorkActive).filter((work) => work.preview).map((work) => [work.imageDraftId ?? work.imageAssetId ?? work.id, work.preview!]));
const pending = new Map(workItems.filter(isWorkActive).map((work) => [work.imageDraftId ?? work.imageAssetId ?? work.id, work.status === "queued" ? "Queued" : "Generating"]));
const previousCount = useRef(shown.length);
@@ -129,7 +130,7 @@ export const ImageBar = forwardRef<HTMLDivElement | null, {
onMenu(asset.id, bounds.left, bounds.bottom, inFamily);
})}
onClick={() => { setShowAll(false); onSelect(asset.id); }}>
- {!pending.has(asset.id) && (asset.relativePath || asset.sourcePath) ? <ReferenceImage folderPath={folderPath} relativePath={asset.relativePath} sourcePath={asset.sourcePath} alt={asset.name} /> : <Image24 aria-hidden="true" />}
+ {previews.has(asset.id) ? <img src={`data:image/jpeg;base64,${previews.get(asset.id)!.frames[0]}`} alt={`${asset.name} generation preview`} draggable={false} /> : !pending.has(asset.id) && (asset.relativePath || asset.sourcePath) ? <ReferenceImage folderPath={folderPath} relativePath={asset.relativePath} sourcePath={asset.sourcePath} alt={asset.name} /> : <Image24 aria-hidden="true" />}
{!inFamily && childCount(asset.id) > 0 && <span className="image-family-badge" aria-hidden="true">{childCount(asset.id) + 1}</span>}
{pending.has(asset.id) ? <span className="image-draft-badge">{pending.get(asset.id)}</span> : asset.imageDraft && <span className="image-draft-badge"><Edit11 aria-hidden="true" />{asset.imageGeneration?.scene.rootType === "image" ? "Editing" : "Draft"}</span>}
</button>)}
diff --git a/src/components/workspace/ImageEditor.test.tsx b/src/components/workspace/ImageEditor.test.tsx
index 9993134..5522151 100644
--- a/src/components/workspace/ImageEditor.test.tsx
+++ b/src/components/workspace/ImageEditor.test.tsx
@@ -35,6 +35,22 @@ function setup(initial = parseProjectConfig(fixture), workItems: WorkItem[] = []
return () => latest;
}
+it("updates a low-resolution image preview in the canvas and image card, then removes it on completion", () => {
+ const config = parseProjectConfig(fixture);
+ config.imageScene = createImageScene("A forest");
+ const pending = saveImageDraft(config, "default", true, "pending");
+ const preview = { jobId: "job", width: 128, height: 128, fps: 24, step: 1, frames: ["/9j/AA=="] };
+ const work = { id: "job", imageDraftId: "pending", status: "generating", progress: 0.25, detail: "Generating", preview } as WorkItem;
+ const props = { config: pending, folderPath: "D:/Images", onChange: vi.fn(), onGenerate: vi.fn(), onCancel: vi.fn() };
+ const view = render(<ImageEditor {...props} workItems={[work]} />);
+ expect(screen.getByAltText("Image generation preview")).toHaveAttribute("src", "data:image/jpeg;base64,/9j/AA==");
+ expect(screen.getAllByAltText(/generation preview/)).toHaveLength(2);
+ view.rerender(<ImageEditor {...props} workItems={[{ ...work, preview: { ...preview, step: 2, frames: ["/9j/BB=="] } }]} />);
+ expect(screen.getByAltText("Image generation preview")).toHaveAttribute("src", "data:image/jpeg;base64,/9j/BB==");
+ view.rerender(<ImageEditor {...props} workItems={[{ ...work, status: "completed" }]} />);
+ expect(screen.queryAllByAltText(/generation preview/)).toHaveLength(0);
+});
+
it.each(["queued", "generating"] as const)("shows a separate empty %s entry without progress on the original", async (status) => {
vi.spyOn(persistence, "readMediaFileUrl").mockImplementation(async (_folder, location) => `blob:${location.relativePath}`);
vi.stubGlobal("URL", class extends URL { static revokeObjectURL = vi.fn(); });
diff --git a/src/components/workspace/ImageEditor.tsx b/src/components/workspace/ImageEditor.tsx
index f97906f..88210a5 100644
--- a/src/components/workspace/ImageEditor.tsx
+++ b/src/components/workspace/ImageEditor.tsx
@@ -200,6 +200,7 @@ export function ImageEditor({ config, folderPath, onChange: changeConfig, onGene
};
const work = workItems.filter((item) => (item.imageDraftId ?? item.imageAssetId ?? item.id) === scene.outputAssetId).at(-1);
const active = work && isWorkActive(work);
+ const generationPreview = active ? work.preview : undefined;
const output = config.assets.find((asset) => asset.id === scene.outputAssetId);
const templateSnapshot = output?.imageDraft ? output.imageGeneration?.template : undefined;
const runningTemplate = active ? templateSnapshot : undefined;
@@ -766,7 +767,7 @@ export function ImageEditor({ config, folderPath, onChange: changeConfig, onGene
onPointerUpCapture={endPan} onPointerCancelCapture={endPan} onLostPointerCapture={endPan}
onAuxClick={(event) => { if (event.button === 1) event.preventDefault(); }}>
<div className="image-frame" style={{ "--image-ratio": width / height, "--image-zoom": view.zoom, "--image-pan-x": `${view.x}px`, "--image-pan-y": `${view.y}px` } as CSSProperties}>
- {active && output?.imageDraft && !(imageRoot && scene.sourceImage) ? <div className="image-empty" role="status"><Image32 aria-hidden="true" /><strong>{work.status === "queued" ? "Queued" : "Generating"}</strong><span>The generated image will appear here.</span></div> : showResult ? <ReferenceImage key={output!.id} folderPath={folderPath} relativePath={output!.relativePath} sourcePath={output!.sourcePath} alt={output!.name} /> : imageRoot && scene.sourceImage ? <ReferenceImage folderPath={folderPath} relativePath={scene.sourceImage.relativePath} alt={scene.sourceImage.name} /> : output && !imageRoot && (output.relativePath || output.sourcePath) ? <ReferenceImage key={output.id} folderPath={folderPath} relativePath={output.relativePath} sourcePath={output.sourcePath} alt={output.name} onMeasured={!output.imageGeneration ? (size) => measureImportedImage(output.id, size) : undefined} /> : <div className="image-empty"><Image32 aria-hidden="true" /><strong>{templateSnapshot ? output!.name : imageRoot ? "Add an image to edit" : "Compose your image"}</strong><span>{templateSnapshot ? active ? "The generated image will appear here." : "No generated image yet." : imageRoot ? "Choose Add image file in the image bar menu, then enter an edit prompt." : "Add objects, text and groups, then describe them in the inspector."}</span></div>}
+ {generationPreview ? <img src={`data:image/jpeg;base64,${generationPreview.frames[0]}`} alt="Image generation preview" draggable={false} /> : active && output?.imageDraft && !(imageRoot && scene.sourceImage) ? <div className="image-empty" role="status"><Image32 aria-hidden="true" /><strong>{work.status === "queued" ? "Queued" : "Generating"}</strong><span>The generated image will appear here.</span></div> : showResult ? <ReferenceImage key={output!.id} folderPath={folderPath} relativePath={output!.relativePath} sourcePath={output!.sourcePath} alt={output!.name} /> : imageRoot && scene.sourceImage ? <ReferenceImage folderPath={folderPath} relativePath={scene.sourceImage.relativePath} alt={scene.sourceImage.name} /> : output && !imageRoot && (output.relativePath || output.sourcePath) ? <ReferenceImage key={output.id} folderPath={folderPath} relativePath={output.relativePath} sourcePath={output.sourcePath} alt={output.name} onMeasured={!output.imageGeneration ? (size) => measureImportedImage(output.id, size) : undefined} /> : <div className="image-empty"><Image32 aria-hidden="true" /><strong>{templateSnapshot ? output!.name : imageRoot ? "Add an image to edit" : "Compose your image"}</strong><span>{templateSnapshot ? active ? "The generated image will appear here." : "No generated image yet." : imageRoot ? "Choose Add image file in the image bar menu, then enter an edit prompt." : "Add objects, text and groups, then describe them in the inspector."}</span></div>}
<svg className={`image-overlay ${drawKind ? "drawing" : ""} ${drawKind === "lasso" && lassoModifier ? `lasso-${lassoModifier}` : ""}`} viewBox="0 0 1000 1000" preserveAspectRatio="none" aria-label="Image placement canvas" onPointerEnter={(event) => { if (drawKind === "lasso") updateLassoModifier(event); }} onPointerDown={pointerDown} onPointerMove={pointerMove} onPointerUp={pointerUp} onPointerCancel={() => { pointer.current = null; setDraftBox(null); setDraftLasso(null); }}>
{boxes && scene.nodes.filter((node) => node.box).map((node) => {
const box = pointer.current?.id === node.id && draftBox ? draftBox : node.box!;
diff --git a/src/components/workspace/SceneBoard.tsx b/src/components/workspace/SceneBoard.tsx
index e247f49..3ce3381 100644
--- a/src/components/workspace/SceneBoard.tsx
+++ b/src/components/workspace/SceneBoard.tsx
@@ -1,5 +1,5 @@
import type { GenerationPreview } from "../../lib/generationPreview";
-import { GenerationPreviewPlayer } from "./GenerationPreviewPlayer";
+import { GenerationPreviewSource } from "./GenerationPreviewSource";
import { SceneHistoryBar } from "./SceneHistoryBar";
import { scenePreviewJob, scenePreviewAsset } from "../../lib/sceneHistory";
import { Copy16, Add16, ArrowDown16, ArrowUp16, ChevronDown12, ChevronRight12, Delete16, Grip16, MoveTo16, OpenExternal16, Rename16, Stop14, Stop16, Wand16 } from "../ui/icons";
@@ -407,8 +407,7 @@ export function SceneBoard({
>{cancellable ? <Stop14 /> : <Wand16 />}</HeaderButton>
</header>
- {!isCollapsed && generationPreviews[job.id] && <GenerationPreviewPlayer key={generationPreviews[job.id].jobId} preview={generationPreviews[job.id]} />}
- {!isCollapsed && <div className="scene-section__content"><ol
+ {!isCollapsed && <GenerationPreviewSource key={generationPreviews[job.id]?.jobId ?? job.id} preview={density === "tiles" ? generationPreviews[job.id] : undefined}><div className="scene-section__content"><ol
className={`shot-grid${shotDrop?.beforeShotId === null ? " shot-grid--drop-end" : ""}`}
aria-label={`Shots in ${job.title}`}
onKeyDown={(event) => gridKeys(event, job)}
@@ -475,7 +474,7 @@ export function SceneBoard({
})}
</ol>
<SceneHistoryBar job={job} assets={assets} folderPath={folderPath} onSelect={onHistorySelect} onMakePrimary={onMakePrimary} onRemove={onRemoveGeneration} onRemoveDraft={onRemoveDraft} />
- </div>}
+ </div></GenerationPreviewSource>}
</section>;
})}
{menu.element}
diff --git a/src/components/workspace/ShotPopout.tsx b/src/components/workspace/ShotPopout.tsx
index 7fb2dd4..a35abeb 100644
--- a/src/components/workspace/ShotPopout.tsx
+++ b/src/components/workspace/ShotPopout.tsx
@@ -5,8 +5,8 @@ import { Dismiss16, Pause20, Play20Filled } from "../ui/icons";
export const SHOT_PLAY_EVENT = "slopus:play-generated-shot";
/** A persistent shot player: reaching the cut leaves the frame open for replay. */
-export function ShotPopout({ folderPath, relativePath, start, end, poster, title, onClose }: {
- folderPath: string; relativePath: string; start: number; end: number; poster: string; title: string; onClose: () => void;
+export function ShotPopout({ folderPath, relativePath, sourceUrl, loop = false, start, end, poster, title, onClose }: {
+ folderPath: string; relativePath?: string; sourceUrl?: string; loop?: boolean; start: number; end: number; poster: string; title: string; onClose: () => void;
}) {
const id = useId();
const frame = useRef<HTMLDivElement>(null);
@@ -16,6 +16,8 @@ export function ShotPopout({ folderPath, relativePath, start, end, poster, title
const epoch = useRef(0);
const timer = useRef<number>();
const finished = useRef(false);
+ const userPaused = useRef(false);
+ const position = useRef(start);
const [url, setUrl] = useState<string | null>(null);
const [playing, setPlaying] = useState(false);
const [error, setError] = useState<string | null>(null);
@@ -36,15 +38,15 @@ export function ShotPopout({ folderPath, relativePath, start, end, poster, title
await media.play();
if (!active.current || request !== epoch.current) return;
setPlaying(true);
- timer.current = window.setTimeout(() => pause(true), Math.max(0, (cut - media.currentTime) * 1000 / (media.playbackRate || 1)));
+ timer.current = window.setTimeout(() => {
+ if (loop) { finished.current = true; void play(); } else pause(true);
+ }, Math.max(0, (cut - media.currentTime) * 1000 / (media.playbackRate || 1)));
} catch {
if (active.current && request === epoch.current) { setPlaying(false); setError("Could not play this shot. Try replaying it."); }
}
};
useEffect(() => {
active.current = true;
- let owned: string | null = null;
- let disposed = false;
const focused = document.activeElement as HTMLElement | null;
const board = frame.current?.closest(".generator-board-viewport")?.querySelector<HTMLElement>(".generator-board");
const wasInert = board?.hasAttribute("inert");
@@ -57,39 +59,51 @@ export function ShotPopout({ folderPath, relativePath, start, end, poster, title
document.addEventListener("pointerdown", outside, true);
document.addEventListener("keydown", escape, true);
window.addEventListener(SHOT_PLAY_EVENT, other);
- void readMediaFileUrl(folderPath, { relativePath }, "video/mp4").then(value => {
- if (disposed) { if (value) URL.revokeObjectURL(value); return; }
- owned = value; setUrl(value);
- if (!value) setError("This shot's video file is unavailable.");
- }).catch(() => { if (!disposed) setError("Could not open this shot's video file."); });
const media = video.current;
return () => {
- disposed = true; active.current = false; epoch.current++; clearTimer();
+ active.current = false; epoch.current++; clearTimer();
media?.pause();
document.removeEventListener("pointerdown", outside, true);
document.removeEventListener("keydown", escape, true);
window.removeEventListener(SHOT_PLAY_EVENT, other);
if (board && !wasInert) board.removeAttribute("inert");
- if (owned) URL.revokeObjectURL(owned);
if (focused?.isConnected) focused.focus();
};
}, [folderPath, relativePath, id]);
+ useEffect(() => {
+ let disposed = false;
+ let owned: string | null = null;
+ if (sourceUrl) setUrl(sourceUrl);
+ else if (relativePath) void readMediaFileUrl(folderPath, { relativePath }, "video/mp4").then(value => {
+ if (disposed) { if (value) URL.revokeObjectURL(value); return; }
+ owned = value; setUrl(value);
+ if (!value) setError("This shot's video file is unavailable.");
+ }).catch(() => { if (!disposed) setError("Could not open this shot's video file."); });
+ return () => { disposed = true; clearTimer(); if (owned) URL.revokeObjectURL(owned); };
+ }, [folderPath, relativePath, sourceUrl]);
return <div className="shot-popout" onClick={event => { event.stopPropagation(); if (event.target === event.currentTarget) onClose(); }}>
<div ref={frame} className="shot-popout__frame" role="dialog" aria-label={`${title} preview`} tabIndex={-1}
style={{ "--shot-aspect": aspect } as CSSProperties}>
- <video ref={video} src={url ?? undefined} poster={poster} playsInline
+ <video ref={video} src={url ?? undefined} poster={poster} playsInline muted={loop} loop={loop}
onLoadedMetadata={event => {
if (!url) return;
const media = event.currentTarget;
if (media.videoWidth > 0 && media.videoHeight > 0) setAspect(media.videoWidth / media.videoHeight);
- media.currentTime = start;
- void play();
+ media.currentTime = loop ? Math.max(start, Math.min(position.current, end - .01)) : start;
+ if (!userPaused.current) void play();
+ }}
+ onTimeUpdate={event => {
+ const media = event.currentTarget;
+ if (loop && media.currentTime < start) media.currentTime = start;
+ position.current = media.currentTime;
+ if (media.currentTime > start && media.currentTime >= end - 0.02) {
+ if (loop && !userPaused.current) { finished.current = true; void play(); } else pause(true);
+ }
}}
- onTimeUpdate={event => { if (event.currentTarget.currentTime >= end - 0.02) pause(true); }}
- onEnded={() => pause(true)} onError={() => { if (url) { pause(); setError("This shot's video could not be played."); } }} />
+ onEnded={() => { if (loop && !userPaused.current) { finished.current = true; void play(); } else pause(true); }} onError={() => { if (url) { pause(); setError("This shot's video could not be played."); } }} />
<button type="button" className="shot-popout__close" aria-label="Close shot preview" onClick={onClose}><Dismiss16 /></button>
<button type="button" className="shot-popout__play" disabled={!url} aria-label={`${playing ? "Pause" : "Replay"} ${title}`}
- onClick={() => { if (playing) pause(); else void play(); }}>{playing ? <Pause20 /> : <Play20Filled />}</button>
+ onClick={() => { userPaused.current = playing; if (playing) pause(); else void play(); }}>{playing ? <Pause20 /> : <Play20Filled />}</button>
{error ? <p className="shot-popout__message" role="alert">{error}</p> : !url && <p className="shot-popout__message" role="status">Opening video…</p>}
</div>
</div>;
diff --git a/src/components/workspace/ShotThumbnail.test.tsx b/src/components/workspace/ShotThumbnail.test.tsx
index 4cd6c72..15ca141 100644
--- a/src/components/workspace/ShotThumbnail.test.tsx
+++ b/src/components/workspace/ShotThumbnail.test.tsx
@@ -4,6 +4,7 @@ import { cleanup, fireEvent, render, screen, waitFor, within } from "@testing-li
import { afterEach, describe, expect, it, vi } from "vitest";
import { createProjectConfig, parseProjectConfig, sceneShots, type GenerationJob } from "../../lib/project";
import { ShotThumbnail, shotPoster } from "./ShotThumbnail";
+import { GenerationPreviewContext } from "./GenerationPreviewSource";
vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));
@@ -100,6 +101,55 @@ const rendered = (folder: string): GenerationJob => {
};
describe("a shot's picture", () => {
+ it("keeps a short generation phase within its available frames for a later shot", async () => {
+ const folder = "C:/Short preview", job = { ...rendered(folder), status: "generating" as const };
+ const preview = { jobId: "work", width: 256, height: 144, fps: 4, step: 1, frames: ["/9j/AA=="] };
+ await withFakeFile({ seconds: .25 }, async () => {
+ const view = render(<GenerationPreviewContext.Provider value={{ preview, url: "blob:short-preview" }}>
+ <ShotThumbnail folderPath={folder} job={job} seconds={4} endSeconds={8} shotNumber={2} />
+ </GenerationPreviewContext.Provider>);
+ await screen.findByRole("button", { name: "Pause shot 2 of First scene" });
+ const video = document.querySelector("video")!;
+ expect(video.currentTime).toBe(0);
+ video.currentTime = .25; fireEvent.timeUpdate(video);
+ expect(video.currentTime).toBe(0);
+ view.unmount();
+ });
+ });
+ it("autoplays and loops generation in the card with the finished shot's controls", async () => {
+ const folder = "C:/Live preview", job = { ...rendered(folder), status: "generating" as const };
+ const preview = { jobId: "work", width: 256, height: 144, fps: 4, step: 1, frames: Array(36).fill("/9j/AA==") };
+ await withFakeFile({ seconds: 9 }, async seen => {
+ const contents = (url: string) => <GenerationPreviewContext.Provider value={{ preview, url }}>
+ <div className="generator-board-viewport"><div className="generator-board">
+ <ShotThumbnail folderPath={folder} job={job} seconds={2} endSeconds={4} sourceOffsetSeconds={10} shotNumber={1} />
+ </div></div>
+ </GenerationPreviewContext.Provider>;
+ const view = render(contents("blob:preview1"));
+ await screen.findByRole("button", { name: "Pause shot 1 of First scene" });
+ const video = document.querySelector(".shot-thumb video") as HTMLVideoElement;
+ expect(video.autoplay).toBe(true); expect(video.loop).toBe(true); expect(video.muted).toBe(true);
+ expect(video.currentTime).toBe(2); expect(seen.reads()).toBe(0);
+ video.currentTime = 4; fireEvent.timeUpdate(video);
+ expect(video.currentTime).toBe(2);
+ await waitFor(() => expect(screen.getByRole("button", { name: "Pause shot 1 of First scene" })).not.toBeNull());
+ // jsdom has no playback engine to update its read-only paused property.
+ Object.defineProperty(video, "paused", { configurable: true, value: false });
+ fireEvent.click(screen.getByRole("button", { name: "Pause shot 1 of First scene" }));
+ expect(screen.getByRole("button", { name: "Play shot 1 of First scene" })).not.toBeNull();
+ view.rerender(contents("blob:preview2"));
+ fireEvent.loadedMetadata(video);
+ expect(video.autoplay).toBe(false);
+ expect(screen.getByRole("button", { name: "Play shot 1 of First scene" })).not.toBeNull();
+ fireEvent.click(screen.getByRole("button", { name: "Pop out shot 1 of First scene" }));
+ const dialog = await screen.findByRole("dialog");
+ await waitFor(() => expect(dialog.querySelector("video")?.getAttribute("src")).toBe("blob:preview2"));
+ expect(dialog.querySelector("video")?.loop).toBe(true);
+ expect(seen.reads()).toBe(0);
+ fireEvent.keyDown(document, { key: "Escape" });
+ view.unmount();
+ });
+ });
it("waits for the first presented frame before drawing the thumbnail", async () => {
const prototype = HTMLVideoElement.prototype;
const originalRequest = prototype.requestVideoFrameCallback, originalCancel = prototype.cancelVideoFrameCallback;
diff --git a/src/components/workspace/ShotThumbnail.tsx b/src/components/workspace/ShotThumbnail.tsx
index 7f8fff2..a0c034d 100644
--- a/src/components/workspace/ShotThumbnail.tsx
+++ b/src/components/workspace/ShotThumbnail.tsx
@@ -1,5 +1,6 @@
import { Clock18, Film18, OpenExternal16, Pause20, Play20Filled, Prohibited18, Spinner18, Spinner20, Warning18, Warning20 } from "../ui/icons";
-import { useEffect, useRef, useState, type ReactNode } from "react";
+import { useContext, useEffect, useRef, useState, type ReactNode } from "react";
+import { GenerationPreviewContext } from "./GenerationPreviewSource";
import { createPortal } from "react-dom";
import { ShotPopout, SHOT_PLAY_EVENT } from "./ShotPopout";
import { readMediaFileUrl } from "../../lib/persistence";
@@ -300,9 +301,12 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
/** Opens this shot in the editor when playback begins. */
onPlay?: () => void;
}) {
+ const live = useContext(GenerationPreviewContext);
+ const generation = !cancelling && ["queued", "generating", "ready"].includes(job.status) ? live : undefined;
const relativePath = job.status === "completed" ? job.outputRelativePath : null;
- const at = seconds + sourceOffsetSeconds;
- const endAt = endSeconds + sourceOffsetSeconds;
+ const at = generation ? Math.min(seconds, (generation.preview.frames.length - 1) / generation.preview.fps) : seconds + sourceOffsetSeconds;
+ const endAt = generation ? Math.min(endSeconds, generation.preview.frames.length / generation.preview.fps) : endSeconds + sourceOffsetSeconds;
+ const previewPosition = useRef({ paused: false, time: at });
const playbackId = `${job.id}@${seconds}`;
const [result, setResult] = useState<PosterResult>(() =>
relativePath ? { poster: POSTERS.get(posterKey(fileKey(folderPath, relativePath), at)) ?? null, failed: false } : NOTHING);
@@ -326,6 +330,11 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
const finishPlayback = () => {
clearCutTimer();
+ if (generation?.url && videoRef.current && !previewPosition.current.paused) {
+ videoRef.current.currentTime = at;
+ void playToCut(videoRef.current);
+ return;
+ }
videoRef.current?.pause();
setPlaying(false);
setPlaybackUrl(null);
@@ -364,11 +373,14 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
if ((event as CustomEvent<string>).detail === playbackId) return;
requestRef.current += 1;
setLoadingPlayback(false);
+ previewPosition.current.paused = true;
finishPlayback();
};
window.addEventListener(PLAY_EVENT, stopOtherShot);
return () => window.removeEventListener(PLAY_EVENT, stopOtherShot);
- }, [playbackId]);
+ }, [playbackId, generation?.url]);
+
+ useEffect(() => () => { requestRef.current += 1; clearCutTimer(); }, [generation?.url]);
useEffect(() => () => {
requestRef.current += 1;
@@ -387,20 +399,25 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
const stopAtCut = () => {
const video = videoRef.current;
- if (!video || video.currentTime + 0.02 < endAt) return;
+ if (video && generation?.url && video.currentTime < at) video.currentTime = at;
+ if (video && generation) previewPosition.current.time = video.currentTime;
+ if (!video || video.currentTime <= at || video.currentTime + 0.02 < endAt) return;
finishPlayback();
};
const togglePlayback = async (event: React.MouseEvent<HTMLButtonElement>) => {
event.stopPropagation();
- if (!relativePath || loadingPlayback) return;
+ if ((!relativePath && !generation?.url) || loadingPlayback) return;
const video = videoRef.current;
if (video) {
if (!video.paused) {
+ requestRef.current += 1;
+ previewPosition.current.paused = true;
clearCutTimer();
video.pause();
setPlaying(false);
} else {
+ previewPosition.current.paused = false;
onPlay?.();
window.dispatchEvent(new CustomEvent<string>(PLAY_EVENT, { detail: playbackId }));
if (video.currentTime < at || video.currentTime >= endAt) video.currentTime = at;
@@ -429,9 +446,9 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
}
};
- if (result.poster) {
- const poster = result.poster;
- const shape = POSTER_SHAPES.get(poster);
+ if (result.poster || generation) {
+ const poster = generation ? `data:image/jpeg;base64,${generation.preview.frames[Math.min(generation.preview.frames.length - 1, Math.floor(at * generation.preview.fps))]}` : result.poster!;
+ const shape = generation ? `${generation.preview.width} / ${generation.preview.height}` : POSTER_SHAPES.get(poster);
const action = playFailed ? "Playback unavailable" : loadingPlayback ? "Opening shot" : playing ? "Pause" : "Play";
return <div className="shot-thumb-frame" style={shape ? { aspectRatio: shape } : undefined}><button
type="button"
@@ -442,14 +459,17 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
data-tooltip={`${action} this shot only (${seconds.toFixed(1)}s–${endSeconds.toFixed(1)}s)`}
onClick={(event) => void togglePlayback(event)}
>
- {playbackUrl
+ {generation?.url || playbackUrl
? <video
ref={videoRef}
- src={playbackUrl}
+ src={generation?.url ?? playbackUrl!}
playsInline
+ muted={!!generation}
+ autoPlay={!!generation && !previewPosition.current.paused}
+ loop={!!generation}
onLoadedMetadata={(event) => {
- event.currentTarget.currentTime = Math.min(at, Math.max(0, event.currentTarget.duration - 0.01));
- void playToCut(event.currentTarget);
+ event.currentTarget.currentTime = Math.min(generation ? Math.max(at, Math.min(previewPosition.current.time, endAt - .01)) : at, Math.max(0, event.currentTarget.duration - 0.01));
+ if (!generation || !previewPosition.current.paused) void playToCut(event.currentTarget);
}}
onTimeUpdate={stopAtCut}
onEnded={stopAtCut}
@@ -470,7 +490,7 @@ export function ShotThumbnail({ folderPath, job, seconds, endSeconds = sceneDura
const host = event.currentTarget.closest<HTMLElement>(".generator-board-viewport");
if (host) { onPlay?.(); setPopoutHost(host); }
}}><OpenExternal16 /></button>
- {popoutHost && relativePath && createPortal(<ShotPopout folderPath={folderPath} relativePath={relativePath} start={at} end={endAt}
+ {popoutHost && (relativePath || generation?.url) && createPortal(<ShotPopout folderPath={folderPath} relativePath={relativePath ?? undefined} sourceUrl={generation?.url} loop={!!generation} start={at} end={endAt}
poster={poster} title={`shot ${shotNumber} of ${job.title}`} onClose={() => { setPopoutHost(null); popoutButton.current?.focus(); }} />, popoutHost)}
</div>;
}
diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.ts
index 64338ec..0d0294a 100644
--- a/src/lib/workQueue.test.ts
+++ b/src/lib/workQueue.test.ts
@@ -670,6 +670,24 @@ describe("agent scene generation", () => {
describe("image generation work", () => {
const template: GeneratorTemplate = { id: "image-test", name: "MiniMax H3", modelType: "minimax-h3", defaultSteps: 20, attention: "sage2", paths: { ...EMPTY_ENGINE_SETTINGS, transformer: "C:/h3.safetensors" } };
const imageProject = (): ProjectRecord => ({ folderPath: "C:/Image", config: createProjectConfig({ name: "Poster", prompt: "An ocean poster", generationType: "image", aspectRatio: "1:1", resolution: "768p", targetDurationSeconds: 60 }) });
+ it.each(["minimax-h3", "moss-image"] as const)("accepts single-frame image previews only for MiniMax (%s)", async modelType => {
+ const { queue } = setup();
+ const session = queue.project(imageProject());
+ const id = queue.enqueueImage(session, { ...template, modelType });
+ await waitFor(() => expect(queue.getSnapshot().find(item => item.id === id)?.status).toBe("generating"));
+ const preview = { jobId: id, width: 2, height: 2, fps: 24, step: 1, frames: ["/9j/AA=="] };
+ emit("slopfab-preview", { ...preview, frames: [...preview.frames, ...preview.frames] });
+ expect(queue.getSnapshot().find(item => item.id === id)?.preview).toBeUndefined();
+ emit("slopfab-preview", preview);
+ expect(queue.getSnapshot().find(item => item.id === id)?.preview).toEqual(modelType === "minimax-h3" ? preview : undefined);
+ saveGenerationPreviewsEnabled(false);
+ expect(queue.getSnapshot().find(item => item.id === id)?.preview).toBeUndefined();
+ saveGenerationPreviewsEnabled(true);
+ emit("slopfab-preview", preview);
+ await queue.cancel(id!);
+ expect(queue.getSnapshot().find(item => item.id === id)?.preview).toBeUndefined();
+ emit("slopfab-job", { jobId: id, state: "cancelled", detail: "Cancelled" });
+ });
it.each([-1, 42, Number.MAX_SAFE_INTEGER])("queues a batch of independent images with unique seeds starting at %s", async (seed) => {
const { queue } = setup();
const record = imageProject();
diff --git a/src/lib/workQueue.ts b/src/lib/workQueue.ts
index 6edf371..55f3742 100644
--- a/src/lib/workQueue.ts
+++ b/src/lib/workQueue.ts
@@ -723,7 +723,9 @@ export class WorkQueue {
if (!loadGenerationPreviewsEnabled() || !validGenerationPreview(event)) return;
const work = this.nativeWork(event.jobId);
const item = this.items.find((candidate) => candidate.id === work?.id);
- if (!work || !item || work.cancelled || item.cancelling || item.kind || work.upscaleSource
+ if (!work || !item || work.cancelled || item.cancelling || (item.kind && item.kind !== "image") || work.upscaleSource
+ || (work.config.providerSettings.slopfab?.options.modelType ?? "minimax-h3") !== "minimax-h3"
+ || (work.image && event.frames.length !== 1)
|| !["preparing", "generating"].includes(item.status)) return;
this.patch(work.id, { preview: event });
}
diff --git a/src/styles/generator.css b/src/styles/generator.css
index f1b8903..91578fe 100644
--- a/src/styles/generator.css
+++ b/src/styles/generator.css
@@ -1032,9 +1032,6 @@
/* Takes sit beside tiles, or in a wrapping row below the shot list. */
.scene-section__content { display: grid; grid-template-columns: minmax(0, 1fr) 88px; align-items: start; gap: var(--space-2); }
-.generation-preview { display: grid; gap: var(--space-1); width: min(100%, 360px); margin-bottom: var(--space-2); }
-.generation-preview video, .generation-preview img { display: block; width: 100%; max-height: 260px; object-fit: contain; background: #000; border-radius: var(--radius-lg); }
-.generation-preview > span { font-size: 12px; color: var(--text-secondary); }
.scene-section__content > .shot-grid { min-width: 0; grid-template-columns: repeat(auto-fill, minmax(0, min(var(--shot-tile-width), 100%))); }
.scene-board--list .scene-section__content { grid-template-columns: minmax(0, 1fr); gap: 0; }
.scene-board--list .scene-section__content > .shot-grid { grid-template-columns: minmax(0, 1fr); }