AI media harness for planning, generating, and editing locally
TypeScript
Support video references in generation
Bittid568eae57c5f
diff --git a/docs/video-references.md b/docs/video-references.mdnew file mode 100644index 0000000..f9cfbd6--- /dev/null+++ b/docs/video-references.md@@ -0,0 +1,32 @@+# Video references++In **References**, choose **Import video**, or **Add video** on an existing+reference. MP4, M4V, and MOV containers are accepted; the computer's WebCodecs+decoder must support the video codec. Video files remain at their original+locations. The project stores their paths and clip settings.++Set the clip's start and duration, and choose whether to include its sound.+Each clip must last 2–15 seconds. A generation can use up to three video clips+totaling 15 seconds, alongside up to nine images. Cite the reference in a shot+just like an image reference. Use CUDA and a Ref2VA transformer, such as the+**References** generator. Soundtracks also require audio VAE encoder weights.++The prompt compiler numbers pictures and videos independently, in the order+the native request receives each kind. Video trim settings and soundtrack+choices are included in generation snapshots so changing them marks the+result as needing regeneration.++The work queue reads and demuxes the selected source using mp4box.js, decodes+the interval with WebCodecs, and uploads frames through binary Tauri IPC.+Frames are limited to the model's 24 fps cadence and a 768-pixel longest edge.+Only a small decoder batch is retained in the webview. The current Slopfab+API accepts host pixels, so this boundary requires readback; it does not+provide GPU texture interop. Slopfab copies pixels into its reference handle+and retains immutable snapshots when requests attach them. Optional mono or+stereo sound is decoded by the webview, trimmed, and sent as interleaved+float32 PCM. No FFmpeg is used.++Prepared native handles exist only during queued work. Completion, failure,+and cancellation release them; handles and decoded media are never written+into project JSON. Old DLLs fail with an explicit update message when their+video-reference symbols are absent.diff --git a/src-tauri/src/agent.rs b/src-tauri/src/agent.rsindex 75e92b3..8c26158 100644--- a/src-tauri/src/agent.rs+++ b/src-tauri/src/agent.rs@@ -71,6 +71,7 @@ Example:When creating or rewriting scenes, plan reusable visual references before writing the shots:- Inventory every recurring visible character, location, product, important prop, vehicle, creature, or other identity whose look must remain consistent. Reuse a matching project reference when one already exists; otherwise add a top-level text reference before adding the scenes.- A new ref.add command uses a unique stable id, a clear name, complete text when the user supplied enough detail, and the appropriate use value (such as "character", "animal", "location", or "product"). Slopus creates it as a text reference and supplies its timestamp. Commands cannot invent paths or image entries; only existing project JSON may describe real files.+- Existing video references can guide movement, camera work, or appearance. Cite their reference IDs with the same @[ref:<reference-id>] tokens; Slopus supplies the <Video N> labels and the saved clip/soundtrack settings. A scene may use up to three video clips totaling 15 seconds. Do not invent video paths or add raw <Video N> labels to shot action.- A character reference must establish the character's stable identity in enough physical detail to reproduce them: apparent age, build, face, hair, distinguishing features, clothing, footwear, accessories, and the colours/materials of the outfit when relevant. Keep momentary action, pose, expression, and camera direction in the shot instead.- A location reference establishes persistent architecture, layout, materials, palette, fixtures, and lighting anchors. A product or prop reference establishes persistent shape, proportions, materials, colours, markings, and branding supplied by the user. Do not fabricate brand details.- Every shot that visibly contains one of these subjects must cite the same reference in its action with the exact token @[ref:<reference-id>]. Slopus derives the scene's referenceIds from these tokens. Reuse the same id across shots and scenes; do not re-describe or rename the subject independently in each shot.@@ -1865,6 +1866,7 @@ mod tests {relative_path: None,source_path: None,images: Vec::new(),+ video: None,intended_use: vec!["character".into()],subcategory: None,icon_relative_path: None,diff --git a/src-tauri/src/agent_commands.rs b/src-tauri/src/agent_commands.rsindex 6fb1ad6..265419c 100644--- a/src-tauri/src/agent_commands.rs+++ b/src-tauri/src/agent_commands.rs@@ -453,6 +453,7 @@ fn apply_command(relative_path: None,source_path: None,images: Vec::new(),+ video: None,intended_use: intended_use.clone(),subcategory: None,icon_relative_path: None,diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rsindex 82960e6..30b200d 100644--- a/src-tauri/src/lib.rs+++ b/src-tauri/src/lib.rs@@ -210,6 +210,8 @@ struct ReusableReference {/// readable for projects made when an image was a separate reference kind.#[serde(default, skip_serializing_if = "Vec::is_empty")]images: Vec<ReferenceImage>,+ #[serde(default, skip_serializing_if = "Option::is_none")]+ video: Option<ReferenceVideoOptions>,#[serde(default)]intended_use: Vec<String>,#[serde(default, skip_serializing_if = "Option::is_none")]@@ -230,6 +232,14 @@ struct ReferenceImage {source_path: Option<String>,}+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+struct ReferenceVideoOptions {+ start_seconds: f64,+ duration_seconds: f64,+ include_audio: bool,+}+/// One shot inside a scene. Mirrors `sceneShotSchema` in src/lib/project.ts.////// `startSeconds` and `action` are REQUIRED on the zod side and are plain@@ -810,6 +820,12 @@ fn validate_and_normalize_config(mut config: ProjectConfig) -> Result<ProjectCon));}let mut image_ids = BTreeSet::new();+ if let Some(video) = &reference.video {+ if reference.kind != "video" || !video.start_seconds.is_finite() || video.start_seconds < 0.0+ || !video.duration_seconds.is_finite() || !(2.0..=15.0).contains(&video.duration_seconds) {+ return Err(format!("Reference '{}' needs a nonnegative clip start and a duration of 2 to 15 seconds.", reference.id));+ }+ }for image in &mut reference.images {if image.id.trim().is_empty() || image.name.trim().is_empty() {return Err(format!(@@ -2134,6 +2150,7 @@ fn create_project_at_with_references(relative_path: imported.relative_path,source_path: imported.source_path,images: Vec::new(),+ video: None,intended_use: vec!["style".into()],subcategory: None,icon_relative_path: None,@@ -2311,6 +2328,52 @@ fn cancel_agent_turn(state: tauri::State<'_, agent::AgentRuntime>, request_id: Sstate.cancel(&request_id)}+#[tauri::command]+fn choose_reference_video(app: AppHandle, folder_path: String) -> Result<Option<String>, String> {+ let root = project_root(&folder_path)?;+ let picked = app.dialog().file().set_title("Add a video reference")+ .add_filter("MP4 video", &["mp4", "m4v", "mov"]).blocking_pick_file();+ let Some(picked) = picked else { return Ok(None); };+ let path = picked.into_path().map_err(|error| format!("Could not access selected video: {error}"))?;+ if !path.is_file() { return Err("The selected video does not exist.".into()); }+ picked_external_source_path(&path, &root).map(Some)+}++#[tauri::command]+fn create_reference_video(duration_seconds: f64, config: ProjectConfig) -> Result<String, String> {+ let config = validate_and_normalize_config(config)?;+ slopfab::create_reference_video(duration_seconds, &config.provider_settings)+}++fn reference_video_header<'a>(request: &'a tauri::ipc::Request<'_>, name: &str) -> Result<&'a str, String> {+ request.headers().get(name).and_then(|value| value.to_str().ok())+ .ok_or_else(|| format!("Missing or invalid {name} header."))+}++#[tauri::command]+fn append_reference_video(request: tauri::ipc::Request<'_>) -> Result<(), String> {+ let tauri::ipc::InvokeBody::Raw(bytes) = request.body() else { return Err("Reference frames require a binary body.".into()); };+ let id = reference_video_header(&request, "x-reference-id")?;+ let width = reference_video_header(&request, "x-reference-width")?.parse().map_err(|_| "Invalid frame width.")?;+ let height = reference_video_header(&request, "x-reference-height")?.parse().map_err(|_| "Invalid frame height.")?;+ let timestamp = reference_video_header(&request, "x-reference-time")?.parse().map_err(|_| "Invalid frame timestamp.")?;+ slopfab::append_reference_video(id, bytes, width, height, timestamp)+}++#[tauri::command]+fn set_reference_video_audio(request: tauri::ipc::Request<'_>) -> Result<(), String> {+ let tauri::ipc::InvokeBody::Raw(bytes) = request.body() else { return Err("Reference audio requires a binary body.".into()); };+ let id = reference_video_header(&request, "x-reference-id")?;+ let channels = reference_video_header(&request, "x-reference-channels")?.parse().map_err(|_| "Invalid audio channels.")?;+ let rate = reference_video_header(&request, "x-reference-rate")?.parse().map_err(|_| "Invalid sample rate.")?;+ slopfab::set_reference_video_audio(id, bytes, channels, rate)+}++#[tauri::command]+fn release_reference_videos(ids: Vec<String>) -> Result<(), String> {+ slopfab::release_reference_videos(&ids)+}+#[tauri::command]fn resolve_slopfab_plan(request: slopfab::GenerationRequest,@@ -2319,7 +2382,7 @@ fn resolve_slopfab_plan(let context = serde_json::json!({"jobId": request.job_id, "frames": request.frames, "steps": request.steps,"canvasWidth": request.canvas_width, "canvasHeight": request.canvas_height,- "referenceCount": request.reference_paths.len(), "inputChars": request.prompt.chars().count(),+ "referenceCount": request.reference_count(), "inputChars": request.prompt.chars().count(),});diagnostics::info("slopfab",@@ -2858,6 +2921,11 @@ pub fn run() {choose_initial_reference_images,choose_reference_image,choose_reference_images,+ choose_reference_video,+ create_reference_video,+ append_reference_video,+ set_reference_video_audio,+ release_reference_videos,import_media_files,read_project_file,read_external_media_file,@@ -4275,6 +4343,7 @@ mod tests {relative_path: None,source_path: None,images: Vec::new(),+ video: None,intended_use: Vec::new(),subcategory: None,icon_relative_path: None,@@ -4286,6 +4355,24 @@ mod tests {);}+ #[test]+ fn video_reference_clip_settings_survive_save_and_reopen() {+ let mut config = fixture();+ let reference = &mut config.references[0];+ reference.kind = "video".into();+ reference.relative_path = Some("media/reference.mp4".into());+ reference.source_path = None;+ reference.video = Some(ReferenceVideoOptions { start_seconds: 3.5, duration_seconds: 4.0, include_audio: false });+ let config = validate_and_normalize_config(config).unwrap();+ let value = serde_json::to_value(&config).unwrap();+ assert_eq!(value["references"][0]["video"]["startSeconds"], 3.5);+ let reopened: ProjectConfig = serde_json::from_value(value).unwrap();+ assert_eq!(reopened.references[0].video, config.references[0].video);+ let mut invalid = reopened;+ invalid.references[0].video.as_mut().unwrap().duration_seconds = 16.0;+ assert!(validate_and_normalize_config(invalid).is_err());+ }+#[test]fn scene_frame_settings_survive_save_and_reopen_and_validate_images() {let mut config = fixture();diff --git a/src-tauri/src/slopfab.rs b/src-tauri/src/slopfab.rsindex d082d5a..edf0cc1 100644--- a/src-tauri/src/slopfab.rs+++ b/src-tauri/src/slopfab.rs@@ -125,6 +125,55 @@ pub struct GenerationRequest {pub canvas_height: i32,#[serde(default)]pub reference_paths: Vec<String>,+ #[serde(default)]+ pub reference_video_ids: Vec<String>,+}++impl GenerationRequest {+ pub fn reference_count(&self) -> usize {+ self.reference_paths.len() + self.reference_video_ids.len()+ }+}++// A video owns its DLL and input buffers. Commands serialize mutations; the+// C API retains immutable snapshots when a plan/generation attaches a video.+static REFERENCE_VIDEOS: std::sync::LazyLock<Mutex<HashMap<String, ffi::ReferenceVideoHandle>>> =+ std::sync::LazyLock::new(Default::default);+static NEXT_REFERENCE_VIDEO: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);++pub fn create_reference_video(duration: f64, settings: &BTreeMap<String, ProviderSetting>) -> Result<String, String> {+ let configuration = Configuration::from_settings(settings);+ let api = ffi::Api::load(&configuration.dll_path)?;+ api.version()?;+ if configuration.platform(detect_platform(&api)) == ComputePlatform::Vulkan {+ return Err("Video references require CUDA. Select CUDA in Generator settings and use a Ref2VA transformer.".into());+ }+ let video = ffi::ReferenceVideoHandle::new(api, duration, configuration.dll_path)?;+ let mut videos = REFERENCE_VIDEOS.lock().map_err(|_| "Reference video lock failed.")?;+ if videos.len() >= 3 { return Err("At most three video references can be prepared at once.".into()); }+ let id = NEXT_REFERENCE_VIDEO.fetch_add(1, Ordering::Relaxed).to_string();+ videos.insert(id.clone(), video);+ Ok(id)+}++pub fn append_reference_video(id: &str, bytes: &[u8], width: i32, height: i32, timestamp: f64) -> Result<(), String> {+ REFERENCE_VIDEOS.lock().map_err(|_| "Reference video lock failed.")?+ .get_mut(id).ok_or("The prepared reference video no longer exists.")?+ .append(bytes, width, height, timestamp)+}++pub fn set_reference_video_audio(id: &str, bytes: &[u8], channels: i32, sample_rate: i32) -> Result<(), String> {+ if bytes.len() % 4 != 0 { return Err("Reference audio must contain complete float32 samples.".into()); }+ let samples: Vec<f32> = bytes.chunks_exact(4).map(|chunk| f32::from_le_bytes(chunk.try_into().unwrap())).collect();+ REFERENCE_VIDEOS.lock().map_err(|_| "Reference video lock failed.")?+ .get_mut(id).ok_or("The prepared reference video no longer exists.")?+ .set_audio(&samples, channels, sample_rate)+}++pub fn release_reference_videos(ids: &[String]) -> Result<(), String> {+ let mut videos = REFERENCE_VIDEOS.lock().map_err(|_| "Reference video lock failed.")?;+ for id in ids { videos.remove(id); }+ Ok(())}#[derive(Debug, Clone, Serialize)]@@ -216,7 +265,7 @@ impl Default for SlopfabRuntime {"steps": item.request.steps,"canvasWidth": item.request.canvas_width,"canvasHeight": item.request.canvas_height,- "referenceCount": item.request.reference_paths.len(),+ "referenceCount": item.request.reference_count(),"randomSeed": item.request.seed == -1,}));let result = run_generation(&item);@@ -311,7 +360,7 @@ impl SlopfabRuntime {"steps": request.steps,"canvasWidth": request.canvas_width,"canvasHeight": request.canvas_height,- "referenceCount": request.reference_paths.len(),+ "referenceCount": request.reference_count(),"inputChars": request.prompt.chars().count(),}),);@@ -567,6 +616,12 @@ pub fn resolve_plan(}fn validate_generation_controls(request: &GenerationRequest) -> Result<(), String> {+ if request.reference_paths.len() > 9 || request.reference_video_ids.len() > 3 || request.reference_count() > 12 {+ return Err("Use at most nine images and three videos per generation.".into());+ }+ if request.still_image && !request.reference_video_ids.is_empty() {+ return Err("Video references cannot be used for still-image generation.".into());+ }if request.steps < 2 {return Err("Generation step count must be at least 2.".into());}@@ -619,8 +674,18 @@ fn configure_request(api.set_model(handle, *id, path)?;}}- for path in &request.reference_paths {- api.add_reference(handle, Path::new(path))?;+ }+ for path in &request.reference_paths {+ api.add_reference(handle, Path::new(path))?;+ }+ if !request.reference_video_ids.is_empty() {+ if platform == ComputePlatform::Vulkan {+ return Err("Video references require CUDA and a Ref2VA transformer.".into());+ }+ let videos = REFERENCE_VIDEOS.lock().map_err(|_| "Reference video lock failed.")?;+ for id in &request.reference_video_ids {+ videos.get(id).ok_or("The prepared reference video no longer exists. Please retry generation.")?+ .attach(handle, &configuration.dll_path)?;}}Ok(())@@ -687,7 +752,7 @@ fn run_generation(item: &QueueItem) -> Result<(OutputMetadata, rendered::Rendereframes: item.request.frames,canvas_width: item.request.canvas_width,canvas_height: item.request.canvas_height,- reference_count: item.request.reference_paths.len(),+ reference_count: item.request.reference_count(),timing_profile: timing_profile.clone(),planned_steps: (item.request.steps - 1).max(0),});@@ -760,7 +825,7 @@ fn run_generation(item: &QueueItem) -> Result<(OutputMetadata, rendered::Rendereframes: output.frames, width: output.width, height: output.height, fps: output.fps,audio_channels: output.audio_channels, audio_sample_rate: output.audio_sample_rate,audio_samples: output.audio_float_count,- reference_count: item.request.reference_paths.len(),+ reference_count: item.request.reference_count(),timing_profile,seconds_conditioning: output.seconds_conditioning,seconds_denoise: output.seconds_denoise,@@ -851,6 +916,7 @@ fn write_reference_icon(canvas_width: crate::reference_icons::RENDER_SIZE as i32,canvas_height: crate::reference_icons::RENDER_SIZE as i32,reference_paths: Vec::new(),+ reference_video_ids: Vec::new(),};let handle = RequestHandle::new(api)?;configure_request(api, handle.0, &request, configuration, platform, true)?;@@ -1012,6 +1078,62 @@ mod ffi {};pub enum Request {}+ enum ReferenceVideo {}+ type VideoCreate = unsafe extern "C" fn(f64, *mut *mut ReferenceVideo) -> i32;+ type VideoDestroy = unsafe extern "C" fn(*mut ReferenceVideo);+ type VideoAppend = unsafe extern "C" fn(*mut ReferenceVideo, *const u8, usize, i32, i32, usize, f64) -> i32;+ type VideoAudio = unsafe extern "C" fn(*mut ReferenceVideo, *const f32, usize, i32, i32, f64) -> i32;+ type VideoAttach = unsafe extern "C" fn(*mut Request, *const ReferenceVideo) -> i32;++ pub struct ReferenceVideoHandle {+ api: Api,+ handle: *mut ReferenceVideo,+ destroy: VideoDestroy,+ append_rgba: VideoAppend,+ audio: VideoAudio,+ attach_video: VideoAttach,+ dll_path: std::path::PathBuf,+ frames: usize,+ }++ // The registry mutex serializes access. The DLL handle and all copied input+ // buffers outlive every call, and attached snapshots own their references.+ unsafe impl Send for ReferenceVideoHandle {}++ impl ReferenceVideoHandle {+ pub fn new(api: Api, duration: f64, dll_path: std::path::PathBuf) -> Result<Self, String> {+ unsafe {+ let missing = |error| format!("This slopfab.dll does not support video references. Update the runtime: {error}");+ let create = *api._library.get::<VideoCreate>(b"slopfab_reference_video_create\0").map_err(missing)?;+ let destroy = *api._library.get::<VideoDestroy>(b"slopfab_reference_video_destroy\0").map_err(missing)?;+ let append_rgba = *api._library.get::<VideoAppend>(b"slopfab_reference_video_append_rgba8\0").map_err(missing)?;+ let audio = *api._library.get::<VideoAudio>(b"slopfab_reference_video_set_audio_f32\0").map_err(missing)?;+ let attach_video = *api._library.get::<VideoAttach>(b"slopfab_request_add_reference_video\0").map_err(missing)?;+ let mut handle = std::ptr::null_mut();+ api.error(create(duration, &mut handle))?;+ if handle.is_null() { return Err("slopfab returned an empty reference video handle.".into()); }+ Ok(Self { api, handle, destroy, append_rgba, audio, attach_video, dll_path, frames: 0 })+ }+ }+ pub fn append(&mut self, bytes: &[u8], width: i32, height: i32, timestamp: f64) -> Result<(), String> {+ if width <= 0 || height <= 0 || width > 1024 || height > 1024 || self.frames >= 361 {+ return Err("Reference video preparation exceeds the frame or dimension limit.".into());+ }+ self.api.error(unsafe { (self.append_rgba)(self.handle, bytes.as_ptr(), bytes.len(), width, height, width as usize * 4, timestamp) })?;+ self.frames += 1;+ Ok(())+ }+ pub fn set_audio(&mut self, samples: &[f32], channels: i32, sample_rate: i32) -> Result<(), String> {+ self.api.error(unsafe { (self.audio)(self.handle, samples.as_ptr(), samples.len(), channels, sample_rate, 0.0) })+ }+ pub fn attach(&self, request: *mut Request, dll_path: &Path) -> Result<(), String> {+ if dll_path != self.dll_path { return Err("Reference video and generation must use the same slopfab runtime.".into()); }+ self.api.error(unsafe { (self.attach_video)(request, self.handle) })+ }+ }+ impl Drop for ReferenceVideoHandle {+ fn drop(&mut self) { unsafe { (self.destroy)(self.handle) }; }+ }pub fn has_cuda_device() -> bool {// CUDA's driver API uses the system calling convention on Windows.@@ -1510,6 +1632,30 @@ mod ffi {#[cfg(test)]mod tests {use super::*;+ #[test]+ fn video_reference_c_api_copies_inputs_and_retains_attached_snapshot() {+ let path = default_dll_path();+ let api = ffi::Api::load(&path).unwrap();+ let request = RequestHandle::new(&api).unwrap();+ api.set_prompt(request.0, "A scene using <Video 1>.").unwrap();+ api.set_frames(request.0, 120).unwrap();+ api.set_resolution(request.0, 736, 416).unwrap();+ let mut video = ffi::ReferenceVideoHandle::new(ffi::Api::load(&path).unwrap(), 2.0, path.clone()).unwrap();+ let pixels = vec![127_u8; 64 * 64 * 4];+ video.append(&pixels, 64, 64, 0.0).unwrap();+ assert!(video.append(&pixels, 64, 64, 0.0).is_err());+ assert!(video.append(&pixels[..10], 64, 64, 0.5).is_err());+ video.append(&pixels, 64, 64, 1.0).unwrap();+ video.set_audio(&vec![0.0; 32_000 * 2], 1, 32_000).unwrap();+ video.attach(request.0, &path).unwrap();+ drop(video);+ drop(pixels);+ // Plan resolution reads the retained video/audio snapshot without a+ // neural model or a CUDA allocation, even after the host releases it.+ let plan = api.resolve(request.0).unwrap();+ assert!(plan.aligned_frames > 0);+ }+#[test]fn absent_dll_is_an_explicit_disabled_state() {let mut settings = BTreeMap::new();diff --git a/src/components/workspace/GeneratorView.test.tsx b/src/components/workspace/GeneratorView.test.tsxindex f0cd3e4..3655e9d 100644--- a/src/components/workspace/GeneratorView.test.tsx+++ b/src/components/workspace/GeneratorView.test.tsx@@ -160,6 +160,21 @@ describe("Generator scene controls", () => {expect(screen.queryByRole("dialog", { name: "Debug Prompt" })).not.toBeInTheDocument();});+ it("sends a cited video with its trim and soundtrack choice in the generation snapshot", async () => {+ const initial = project();+ initial.references = [{ id: "motion", kind: "video", name: "Motion", description: "", sourcePath: "C:/motion.mp4", intendedUse: [], createdAt: initial.createdAt,+ video: { startSeconds: 2, durationSeconds: 3, includeAudio: false } }];+ initial.generationJobs[0].referenceIds = ["motion"];+ initial.generationJobs[0].shots![0].action = "Follow @[ref:motion].";+ const state = setup(parseProjectConfig(initial));+ fireEvent.click(within(screen.getByRole("region", { name: "First scene" })).getByRole("button", { name: "Generate" }));+ await waitFor(() => expect(state.latest().generationJobs[0].generationSnapshot).toEqual(expect.any(String)));+ const snapshot = JSON.parse(state.latest().generationJobs[0].generationSnapshot!);+ expect(snapshot.referenceVideos).toEqual([{ name: "Motion", sourcePath: "C:/motion.mp4", startSeconds: 2, durationSeconds: 3, includeAudio: false }]);+ expect(snapshot.referencePaths).toEqual([]);+ expect(snapshot.prompt).toContain("<Video 1>");+ });+it("offers no inferred Look and sends a selected image as the scene's first frame", async () => {const initial = project();initial.references = [{diff --git a/src/components/workspace/GeneratorView.tsx b/src/components/workspace/GeneratorView.tsxindex e5f92f0..7136ef6 100644--- a/src/components/workspace/GeneratorView.tsx+++ b/src/components/workspace/GeneratorView.tsx@@ -1,4 +1,5 @@import type { GenerationSubmission } from "../../lib/workQueue";+import { usableVideoReferences } from "../../lib/project";import { ChevronDown, Plus, Square, Trash2, WandSparkles } from "lucide-react";import { invoke } from "@tauri-apps/api/core";import { useEffect, useMemo, useRef, useState, useSyncExternalStore } from "react";@@ -306,6 +307,12 @@ export function GeneratorView({ config, folderPath, runtime = null, generationCoreferencePaths: usableReferenceImages(bound).map((image) => projectItemPath(folderPath, image) ?? "").filter((path) => path.length > 0),+ referenceVideos: usableVideoReferences(bound).map((reference) => ({+ name: reference.name, relativePath: reference.relativePath, sourcePath: reference.sourcePath,+ startSeconds: reference.video?.startSeconds ?? 0,+ durationSeconds: reference.video?.durationSeconds ?? 2,+ includeAudio: reference.video?.includeAudio ?? true,+ })),};};diff --git a/src/components/workspace/ReferenceVideo.tsx b/src/components/workspace/ReferenceVideo.tsxnew file mode 100644index 0000000..0e64d8c--- /dev/null+++ b/src/components/workspace/ReferenceVideo.tsx@@ -0,0 +1,25 @@+import { useEffect, useState } from "react";+import { readMediaFileUrl } from "../../lib/persistence";+import type { ProjectReference } from "../../lib/project";++export function ReferenceVideo({ folderPath, reference }: { folderPath: string; reference: ProjectReference }) {+ const [url, setUrl] = useState<string | null>(null);+ const [error, setError] = useState<string | null>(null);+ useEffect(() => {+ let disposed = false;+ let owned: string | null = null;+ setUrl(null); setError(null);+ void readMediaFileUrl(folderPath, reference, "video/mp4").then((value) => {+ if (disposed) { if (value) URL.revokeObjectURL(value); return; }+ owned = value; setUrl(value);+ }).catch((reason: unknown) => { if (!disposed) setError(String(reason)); });+ return () => { disposed = true; if (owned) URL.revokeObjectURL(owned); };+ }, [folderPath, reference.sourcePath, reference.relativePath]);+ if (error) return <p role="alert">{error}</p>;+ if (!url) return <p>Loading video preview…</p>;+ return <video className="reference-video-preview" key={`${url}:${reference.video?.startSeconds ?? 0}`}+ src={url} controls preload="metadata" aria-label={`${reference.name} video preview`}+ muted={reference.video?.includeAudio === false}+ onLoadedMetadata={(event) => { event.currentTarget.currentTime = reference.video?.startSeconds ?? 0; }}+ onError={() => setError("This computer cannot preview this video codec.")} />;+}diff --git a/src/components/workspace/ReferencesView.tsx b/src/components/workspace/ReferencesView.tsxindex 0b37c74..4000fac 100644--- a/src/components/workspace/ReferencesView.tsx+++ b/src/components/workspace/ReferencesView.tsx@@ -1,6 +1,6 @@-import { BookOpen, ChevronRight, FileText, ImagePlus, Plus, RefreshCw, Search, Trash2, Users, X } from "lucide-react";+import { BookOpen, ChevronRight, FileText, ImagePlus, Plus, RefreshCw, Search, Trash2, Users, Video, X } from "lucide-react";import { invoke } from "@tauri-apps/api/core";-import { useEffect, useMemo, useState, useSyncExternalStore } from "react";+import { useEffect, useMemo, useRef, useState, useSyncExternalStore } from "react";import { isReferenceDescribed, referenceImages, type ProjectConfig, type ProjectReference, type ProjectReferenceImage } from "../../lib/project";import {composeLocationPrompt,@@ -17,6 +17,8 @@ import {import { LOCATION_SETTING_GROUPS } from "../../lib/expanded-reference-options";import { isTauri } from "../../lib/persistence";import { ReferenceImage } from "./ReferenceImage";+import { ReferenceVideo } from "./ReferenceVideo";+import { inspectReferenceVideo } from "../../lib/referenceVideo";import { DebugPromptDialog } from "./DebugPromptDialog";import { referenceIconPrompt } from "../../lib/referenceIcons";import { loadDebugOptionsEnabled, subscribeDebugOptions } from "../../lib/settings";@@ -26,10 +28,25 @@ import { loadDebugOptionsEnabled, subscribeDebugOptions } from "../../lib/settin* reference, and neither the card nor the inspector does anything with it. */const referenceKindLabel = (reference: ProjectReference) => {const count = referenceImages(reference).length;- if (count === 0) return reference.kind === "video" ? "Video clip" : reference.kind === "audio" ? "Sound" : "Text definition";+ if (reference.kind === "video") return `Video clip${count ? ` + ${count} image${count === 1 ? "" : "s"}` : ""}`;+ if (count === 0) return reference.kind === "audio" ? "Sound" : "Text definition";return `${count} image${count === 1 ? "" : "s"}${isReferenceDescribed(reference) ? " + text" : ""}`;};+function ClipSecondsInput({ label, value, min, max = Number.MAX_SAFE_INTEGER, onChange }: {+ label: string; value: number; min: number; max?: number; onChange: (value: number) => void;+}) {+ const [draft, setDraft] = useState(String(value));+ useEffect(() => setDraft(String(value)), [value]);+ return <label>{label}<input type="number" min={min} max={max} step={0.1} value={draft}+ onChange={(event) => setDraft(event.target.value)} onKeyDown={(event) => { if (event.key === "Enter") event.currentTarget.blur(); }}+ onBlur={() => {+ const parsed = Number(draft);+ const next = draft.trim() && Number.isFinite(parsed) ? Math.max(min, Math.min(max, parsed)) : value;+ setDraft(String(next)); onChange(next);+ }} /></label>;+}+export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon, pendingIconIds = new Set<string>(), onOpenGenerator }: {config: ProjectConfig;folderPath: string;@@ -39,10 +56,13 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,onOpenGenerator?: (jobId: string) => void;}) {const [selectedId, setSelectedId] = useState<string | undefined>(config.references[0]?.id);+ const configRef = useRef(config);+ configRef.current = config;const debugEnabled = useSyncExternalStore(subscribeDebugOptions, loadDebugOptionsEnabled);const [showDebugIconPrompt, setShowDebugIconPrompt] = useState(false);useEffect(() => setShowDebugIconPrompt(false), [selectedId, debugEnabled]);const [importError, setImportError] = useState<string | null>(null);+ const [importingVideo, setImportingVideo] = useState(false);const [iconError, setIconError] = useState<string | null>(null);const [presetDialog, setPresetDialog] = useState(false);const [pickerType, setPickerType] = useState<PresetReferenceType>("character");@@ -58,7 +78,10 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,: undefined, [selected]);const selectedPresetIcon = selectedImages.length === 0 ? selectedPreset?.icon : undefined;const jobs = useMemo(() => config.generationJobs.filter((job) => job.referenceIds.includes(selectedId ?? "")), [config.generationJobs, selectedId]);- const update = (id: string, patch: Partial<ProjectReference>) => onChange({ ...config, references: config.references.map((ref) => ref.id === id ? { ...ref, ...patch } : ref) });+ const update = (id: string, patch: Partial<ProjectReference>) => {+ const current = configRef.current;+ onChange({ ...current, references: current.references.map((ref) => ref.id === id ? { ...ref, ...patch } : ref) });+ };// Starts empty on purpose. Seeding it with the instruction text meant an// unedited definition shipped "Describe the traits…" to the model as if the// user had written it. The guidance lives in the field's placeholder instead.@@ -106,6 +129,29 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,})) });setSelectedId(config.references.find((ref) => ref.id !== selected.id)?.id);};+ const addVideo = async (attach = false) => {+ setImportError(null);+ if (!isTauri()) { setImportError("Video import is available in the desktop app."); return; }+ setImportingVideo(true);+ try {+ const sourcePath = await invoke<string | null>("choose_reference_video", { folderPath });+ if (!sourcePath) return;+ const options = await inspectReferenceVideo(folderPath, sourcePath);+ const video = { startSeconds: 0, ...options };+ const current = configRef.current;+ if (attach && selected) {+ const target = current.references.find((reference) => reference.id === selected.id);+ if (!target) throw new Error("The reference was removed while the video was importing.");+ update(target.id, { kind: "video", relativePath: null, sourcePath, video, images: referenceImages(target) });+ } else {+ const id = `ref-${crypto.randomUUID()}`;+ const name = sourcePath.split(/[\\/]/).pop()!.replace(/\.[^.]+$/, "");+ onChange({ ...current, references: [{ id, kind: "video", name, description: "", sourcePath, video, intendedUse: [], createdAt: new Date().toISOString() }, ...current.references] });+ setSelectedId(id);+ }+ } catch (reason) { setImportError(reason instanceof Error ? reason.message : String(reason)); }+ finally { setImportingVideo(false); }+ };const createEmptyReference = () => {addTextReference(`New ${referenceTypeLabel(pickerType).toLocaleLowerCase()}`, "", pickerType, pickerSubcategory === "all" ? "" : pickerSubcategory);setPresetDialog(false);@@ -132,6 +178,7 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,<h1>References</h1><p>{config.references.length === 1 ? "1 reference" : `${config.references.length} references`}</p></div>+ <button className="secondary-button" disabled={importingVideo} onClick={() => void addVideo()}><Video size={16} /> {importingVideo ? "Importing video…" : "Import video"}</button></header><section className="reference-library" aria-labelledby="reference-library-heading">{/* Both regions used to start at h3, so the outline jumped h1 → h3. */}@@ -148,15 +195,15 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,? <span className="reference-art reference-art--photo"><ReferenceImage folderPath={folderPath} relativePath={ref.iconRelativePath} alt={`${ref.name} icon`} /></span>: presetIcon? <span className="reference-art reference-art--photo"><img src={presetIcon} alt="" /></span>- : <span className="reference-copy-art"><FileText size={26} /></span>}+ : <span className="reference-copy-art">{ref.kind === "video" ? <Video size={26} /> : <FileText size={26} />}</span>}<span className="reference-card__body"><span><b>{ref.name}</b>{(images.length > 0 || ref.kind === "video" || ref.kind === "audio") && <small>{referenceKindLabel(ref)}</small>}</span>{isReferenceDescribed(ref)? <p>{ref.description}</p>- : <p className="reference-card__incomplete">{images.length > 0+ : <p className="reference-card__incomplete">{ref.kind === "video" ? "The clip is sent as a reference. Add a prompt to describe what to keep." : images.length > 0? "Not described yet — the picture is sent, but nothing tells the engine what to keep.": "Not described yet — it won’t be used until you add a definition."}</p>}<span className="use-tags"><i>{referenceTypeLabel(referenceType(ref))}</i></span></span></button>;})}- <button className="reference-add-card" onClick={openNewReference}><span><Plus size={22} /></span><b>Add a reference</b><small>Import images or write a definition</small></button>+ <button className="reference-add-card" onClick={openNewReference}><span><Plus size={22} /></span><b>Add a reference</b><small>Add images, video, or a definition</small></button></div></section></main>@@ -174,7 +221,18 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,</> : <h2>Reference details</h2>}</header><div className="reference-inspector__scroll">{selected ? <>- <div className={`reference-detail-art${selectedImages.length > 0 || selected.iconRelativePath || selectedPresetIcon ? " reference-detail-art--photo" : " reference-detail-art--text"}${selectedPresetIcon || selected.iconRelativePath ? " reference-detail-art--preset" : ""}`}>+ {selected.kind === "video" && <section className="reference-video" aria-label="Video reference">+ <ReferenceVideo folderPath={folderPath} reference={selected} />+ <p>Use 2–15 seconds per clip, up to 3 clips totaling 15 seconds. Requires CUDA and a Ref2VA generator.</p>+ <ClipSecondsInput key={`${selected.id}-start`} label="Clip start (seconds)" min={0} value={selected.video?.startSeconds ?? 0}+ onChange={(startSeconds) => update(selected.id, { video: { durationSeconds: 2, includeAudio: true, ...selected.video, startSeconds } })} />+ <ClipSecondsInput key={`${selected.id}-duration`} label="Clip duration (seconds)" min={2} max={15} value={selected.video?.durationSeconds ?? 2}+ onChange={(durationSeconds) => update(selected.id, { video: { startSeconds: 0, includeAudio: true, ...selected.video, durationSeconds } })} />+ <label><input type="checkbox" checked={selected.video?.includeAudio ?? true}+ onChange={(event) => update(selected.id, { video: { startSeconds: 0, durationSeconds: 2, ...selected.video, includeAudio: event.target.checked } })} /> Include sound</label>+ <button className="secondary-button" onClick={() => update(selected.id, { kind: "text", sourcePath: null, relativePath: null, video: undefined })}><Trash2 size={16} /> Remove video</button>+ </section>}+ {(selected.kind !== "video" || selectedImages.length > 0) && <div className={`reference-detail-art${selectedImages.length > 0 || selected.iconRelativePath || selectedPresetIcon ? " reference-detail-art--photo" : " reference-detail-art--text"}${selectedPresetIcon || selected.iconRelativePath ? " reference-detail-art--preset" : ""}`}>{selectedImages.length > 0? <div className="reference-detail-images">{selectedImages.map((image) => <ReferenceImage key={image.id} folderPath={folderPath} relativePath={image.relativePath} sourcePath={image.sourcePath} alt={image.name} />)}</div>: selected.iconRelativePath@@ -198,11 +256,12 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,catch (reason) { setIconError(reason instanceof Error ? reason.message : String(reason)); }}}><RefreshCw size={14} aria-hidden="true" /></button>}- </div>+ </div>}<div className="reference-fields"><label><span>Prompt</span><textarea value={selected.description} placeholder="Describe what should stay consistent — the traits, materials, colours, or wardrobe Slopus should preserve across shots." onChange={(event) => update(selected.id, { description: event.target.value, content: event.target.value || null, subcategory: selectedSubcategory })} /></label><div className="reference-fields__actions"><button className="secondary-button" onClick={() => void addImages()}><ImagePlus size={16} /> Add images</button>+ <button className="secondary-button" disabled={importingVideo} onClick={() => void addVideo(true)}><Video size={16} /> {selected.kind === "video" ? "Replace video" : "Add video"}</button></div></div>{selectedLocation && <section className="reference-location-settings" aria-label="Location settings">@@ -272,7 +331,7 @@ export function ReferencesView({ config, folderPath, onChange, onRegenerateIcon,</section></div></div></div>}- {importError && <div className="toast" role="alert"><strong>Couldn’t add image</strong><span>{importError}</span><button onClick={() => setImportError(null)}>Dismiss</button></div>}+ {importError && <div className="toast" role="alert"><strong>Couldn’t add reference</strong><span>{importError}</span><button onClick={() => setImportError(null)}>Dismiss</button></div>}{iconError && <div className="toast" role="alert"><strong>Couldn’t regenerate icon</strong><span>{iconError}</span><button onClick={() => setIconError(null)}>Dismiss</button></div>}</div>;}diff --git a/src/components/workspace/ReferencesView.video.test.tsx b/src/components/workspace/ReferencesView.video.test.tsxnew file mode 100644index 0000000..a81c90a--- /dev/null+++ b/src/components/workspace/ReferencesView.video.test.tsx@@ -0,0 +1,41 @@+// @vitest-environment jsdom+import "@testing-library/jest-dom/vitest";+import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react";+import { invoke } from "@tauri-apps/api/core";+import { useState } from "react";+import { afterEach, expect, it, vi } from "vitest";+import { createProjectConfig, parseProjectConfig } from "../../lib/project";+import { inspectReferenceVideo } from "../../lib/referenceVideo";+import { ReferencesView } from "./ReferencesView";++vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));+vi.mock("../../lib/referenceVideo", () => ({ inspectReferenceVideo: vi.fn() }));+vi.mock("./ReferenceVideo", () => ({ ReferenceVideo: () => <div>Video preview</div> }));+afterEach(() => { cleanup(); vi.resetAllMocks(); delete (window as unknown as Record<string, unknown>).__TAURI_INTERNALS__; });++it("imports a video, edits its saved trim and soundtrack, then removes its attachment", async () => {+ Object.assign(window, { __TAURI_INTERNALS__: {} });+ vi.mocked(invoke).mockResolvedValue("C:/motion.mp4");+ vi.mocked(inspectReferenceVideo).mockResolvedValue({ durationSeconds: 10, includeAudio: true });+ let latest = createProjectConfig({ name: "Video", prompt: "", aspectRatio: "16:9", resolution: "1080p", targetDurationSeconds: 20 });+ function Harness() {+ const [config, setConfig] = useState(latest); latest = config;+ return <ReferencesView folderPath="C:/project" config={config} onChange={setConfig} />;+ }+ render(<Harness />);+ fireEvent.click(screen.getByRole("button", { name: "Import video" }));+ await screen.findByText("Video preview");+ expect(invoke).toHaveBeenCalledWith("choose_reference_video", { folderPath: "C:/project" });+ fireEvent.change(screen.getByLabelText("Clip start (seconds)"), { target: { value: "2" } });+ fireEvent.blur(screen.getByLabelText("Clip start (seconds)"));+ fireEvent.change(screen.getByLabelText("Clip duration (seconds)"), { target: { value: "4" } });+ fireEvent.blur(screen.getByLabelText("Clip duration (seconds)"));+ fireEvent.click(screen.getByLabelText("Include sound"));+ expect(parseProjectConfig(JSON.parse(JSON.stringify(latest))).references[0]).toMatchObject({+ kind: "video", sourcePath: "C:/motion.mp4", video: { startSeconds: 2, durationSeconds: 4, includeAudio: false },+ });+ fireEvent.click(screen.getByRole("button", { name: "Remove video" }));+ await waitFor(() => expect(latest.references[0].kind).toBe("text"));+ expect(latest.references[0].video).toBeUndefined();+ expect(latest.references[0].sourcePath).toBeNull();+});diff --git a/src/lib/exportPipeline.ts b/src/lib/exportPipeline.tsindex 1555e76..c74cc0f 100644--- a/src/lib/exportPipeline.ts+++ b/src/lib/exportPipeline.ts@@ -432,6 +432,8 @@ export async function probeCompositor(): Promise<CompositorProbe> {export interface DemuxedSource {config: VideoDecoderConfig;+ durationSeconds?: number;+ hasAudio?: boolean;samples: Array<{ data: Uint8Array; timestampUs: number; durationUs: number; key: boolean }>;}@@ -523,6 +525,8 @@ export async function demux(bytes: ArrayBuffer, name: string): Promise<DemuxedSo});return {+ durationSeconds: track.duration / track.timescale,+ hasAudio: state.movie.audioTracks.length > 0,config: {codec: track.codec,codedWidth: track.video?.width ?? track.track_width,diff --git a/src/lib/project.ts b/src/lib/project.tsindex 4b0190e..e07d3a8 100644--- a/src/lib/project.ts+++ b/src/lib/project.ts@@ -265,6 +265,11 @@ export const projectReferenceSchema = z.object({// written definition can carry any number of images, and an image-only// reference is simply a definition whose description is still blank.images: z.array(projectReferenceImageSchema).optional(),+ video: z.object({+ startSeconds: z.number().finite().min(0),+ durationSeconds: z.number().finite().min(2).max(15),+ includeAudio: z.boolean(),+ }).optional(),intendedUse: z.array(z.enum(["character", "animal", "product", "location", "style", "audio"])).default([]),subcategory: z.string().optional(),// Generated library artwork is not an image conditioning attachment.@@ -272,6 +277,9 @@ export const projectReferenceSchema = z.object({createdAt: isoDateSchema,}).superRefine((reference, context) => {checkOneLocation(reference, context, `Reference '${reference.id}'`);+ if (reference.video && reference.kind !== "video") {+ context.addIssue({ code: z.ZodIssueCode.custom, path: ["video"], message: "Clip settings require a video reference." });+ }const imageIds = new Set<string>();for (const image of reference.images ?? []) {if (imageIds.has(image.id)) {@@ -631,7 +639,7 @@ export function isReferenceDescribed(reference: ProjectReference): boolean {* An image qualifies on its file alone: it can be sent to the engine and cited* as <Picture N> with nothing written about it. */export function isReferenceUsable(reference: ProjectReference): boolean {- return referenceImages(reference).length > 0 || isReferenceDescribed(reference);+ return referenceImages(reference).length > 0 || (reference.kind === "video" && Boolean(reference.sourcePath || reference.relativePath)) || isReferenceDescribed(reference);}/** Reference guide §2.1: `<Subject N>` is VISIBLE content. A reference tagged@@ -646,7 +654,7 @@ export function isReferenceUsable(reference: ProjectReference): boolean {* meets references saved before that. A reference carrying any other tag, or* no tag at all, is visible content as before. */export function isVisualReference(reference: ProjectReference): boolean {- return reference.intendedUse.length === 0 || reference.intendedUse.some((use) => use !== "audio");+ return reference.kind === "video" || reference.intendedUse.length === 0 || reference.intendedUse.some((use) => use !== "audio");}/* ---------------------------------------------------------------------------@@ -719,8 +727,8 @@ const DEFAULT_MUSIC = "N/A";const referenceLabel = (reference: ProjectReference, index: number) => `<Subject ${index + 1}>`;-/** Image references are the only ones that can be sent to the engine as assets,- * so they carry the <Picture N> numbering. That numbering MUST match the order+/** Image references carry the <Picture N> numbering, independently of videos.+ * That numbering MUST match the order* of `reference_paths` in the generation request: slopfab.rs iterates the array* and calls add_reference sequentially, so index 0 is <Picture 1>.*@@ -741,6 +749,11 @@ export function usableReferenceImages(references: ProjectReference[]): ProjectRereturn usableImageReferences(references).flatMap(referenceImages);}+/** Video numbering is independent of pictures and follows native insertion order. */+export function usableVideoReferences(references: ProjectReference[]): ProjectReference[] {+ return references.filter((reference) => reference.kind === "video" && Boolean(reference.sourcePath || reference.relativePath)).filter(isVisualReference);+}+/** Opening and closing pictures come first, then bound references in project* order. Frame anchors use their first image, even on a multi-image reference;* the same anchor selected for both ends is sent only once. */@@ -854,6 +867,7 @@ export interface SceneGenerationInput {canvasWidth: number;canvasHeight: number;referencePaths: readonly string[];+ referenceVideos?: readonly { name: string; relativePath?: string | null; sourcePath?: string | null; startSeconds: number; durationSeconds?: number; includeAudio: boolean }[];previousSceneId?: string;}@@ -874,6 +888,7 @@ export function sceneGenerationSnapshot(job: GenerationJob, input: SceneGeneraticanvasWidth: input.canvasWidth,canvasHeight: input.canvasHeight,referencePaths: input.referencePaths,+ ...(input.referenceVideos?.length ? { referenceVideos: input.referenceVideos } : {}),...(input.previousSceneId ? { previousSceneId: input.previousSceneId } : {}),shots,});@@ -1042,6 +1057,7 @@ export function compileScenePromptSegments(scene: ScenePrompt, references: Projeconst imageReferenceIds = usableImageReferences(references).flatMap((reference) =>referenceImages(reference).map(() => reference.id));const pictureNumbers = new Map<string, number[]>();+ const videoNumbers = new Map(usableVideoReferences(references).map((reference, index) => [reference.id, index + 1]));imageReferenceIds.forEach((referenceId, index) => pictureNumbers.set(referenceId, [...(pictureNumbers.get(referenceId) ?? []), index + 1]));const listOf = (values: string[]): string => values.length === 1? values[0]@@ -1051,9 +1067,8 @@ export function compileScenePromptSegments(scene: ScenePrompt, references: Proje// cited INSIDE its <Subject N> definition. A scene's explicit first frame is// the exception: it describes composition, so it is emitted as a standalone// <Picture N> entry instead of pretending to be a subject.- // No <Audio N> is emitted either: §2.4 defines it as an actual audio asset,- // and every reference in this app is text or image. An "audio" intendedUse- // is a note about desired sound, not a signal to copy.+ // Video citations follow their own native insertion order. An "audio"+ // intendedUse remains a legacy text tag, not a standalone audio attachment.// The identity slot carries ONLY what the user wrote. `name` is a library// label, not a description: for an imported image it is the source file's// stem, so using it here shipped "<Subject 1> is IMG_4821, shown in@@ -1065,10 +1080,11 @@ export function compileScenePromptSegments(scene: ScenePrompt, references: Projeconst definitions = usable.flatMap((reference, index): PromptSegment[] => {const detail = referenceDefinition(reference).replace(/\s+/g, " ");const pictures = pictureNumbers.get(reference.id) ?? [];+ const video = videoNumbers.get(reference.id);const label = referenceLabel(reference, index);const lead = index === 0 ? "" : "\n";- if (pictures.length === 0) return [frame(`${lead}${label}: `), own(detail), ...addedStop(detail)];- const pictureList = listOf(pictures.map((picture) => `<Picture ${picture}>`));+ if (pictures.length === 0 && !video) return [frame(`${lead}${label}: `), own(detail), ...addedStop(detail)];+ const pictureList = listOf([...pictures.map((picture) => `<Picture ${picture}>`), ...(video ? [`<Video ${video}>`] : [])]);return [frame(`${lead}${label} is the content shown in ${pictureList}.`),...(detail ? [frame(" "), own(detail), ...addedStop(detail)] : []),diff --git a/src/lib/project.video.test.ts b/src/lib/project.video.test.tsnew file mode 100644index 0000000..730a7a9--- /dev/null+++ b/src/lib/project.video.test.ts@@ -0,0 +1,36 @@+import { expect, it } from "vitest";+import { compileMiniMaxH3Prompt, createDraftGenerationJob, isReferenceUsable, projectReferenceSchema, sceneGenerationSnapshot, usableReferenceImages, usableVideoReferences, type ProjectReference } from "./project";++const reference = (id: string, kind: ProjectReference["kind"], description = ""): ProjectReference => ({+ id, kind, name: id, description, intendedUse: [], createdAt: "2026-09-12T00:00:00.000Z",+ ...(kind !== "text" ? { sourcePath: `C:/references/${id}.${kind === "video" ? "mp4" : "png"}` } : {}),+});++it("keeps image and video numbering independent and stable in a mixed prompt", () => {+ const references = [reference("motion", "video"), reference("hero", "image"), reference("style", "text", "Ink drawing"), reference("camera", "video")];+ references[0].images = [{ id: "still", name: "Still", sourcePath: "C:/still.png" }];+ expect(isReferenceUsable(references[0])).toBe(true);+ expect(usableVideoReferences(references).map((item) => item.id)).toEqual(["motion", "camera"]);+ expect(usableReferenceImages(references).map((item) => item.sourcePath)).toEqual(["C:/still.png", "C:/references/hero.png"]);+ const prompt = compileMiniMaxH3Prompt("Follow the movement.", references);+ expect(prompt).toContain("<Subject 1> is the content shown in <Picture 1> and <Video 1>.");+ expect(prompt).toContain("<Subject 2> is the content shown in <Picture 2>.");+ expect(prompt).toContain("<Subject 4> is the content shown in <Video 2>.");+});++it("round trips clip options and rejects invalid persisted ranges", () => {+ const video = { ...reference("motion", "video"), video: { startSeconds: 3, durationSeconds: 5, includeAudio: false } };+ expect(projectReferenceSchema.parse(JSON.parse(JSON.stringify(video))).video).toEqual(video.video);+ expect(projectReferenceSchema.safeParse({ ...video, video: { ...video.video, durationSeconds: 16 } }).success).toBe(false);+ expect(projectReferenceSchema.safeParse({ ...video, video: { ...video.video, startSeconds: -1 } }).success).toBe(false);+});++it("records video paths and trims in generation snapshots without temporary handles", () => {+ const request = { prompt: "Motion", frames: 120, steps: 12, seed: 1, canvasWidth: 736, canvasHeight: 416, referencePaths: [],+ referenceVideos: [{ name: "Motion", sourcePath: "C:/motion.mp4", startSeconds: 3, durationSeconds: 5, includeAudio: false }], referenceVideoIds: ["temporary"] };+ const job = createDraftGenerationJob("Motion");+ const snapshot = JSON.parse(sceneGenerationSnapshot(job, request));+ expect(snapshot.referenceVideos).toEqual(request.referenceVideos);+ expect(snapshot.referenceVideoIds).toBeUndefined();+ expect(sceneGenerationSnapshot(job, { ...request, referenceVideos: [{ ...request.referenceVideos[0], startSeconds: 4 }] })).not.toBe(JSON.stringify(snapshot));+});diff --git a/src/lib/referenceVideo.test.ts b/src/lib/referenceVideo.test.tsnew file mode 100644index 0000000..73be448--- /dev/null+++ b/src/lib/referenceVideo.test.ts@@ -0,0 +1,90 @@+import { afterEach, beforeEach, expect, it, vi } from "vitest";+import { invoke } from "@tauri-apps/api/core";+import { demux, type DemuxedSource } from "./exportPipeline";+import { prepareReferenceVideos, referenceSoundtrack, referenceVideoRange, uploadReferenceVideoFrames } from "./referenceVideo";+import { createProjectConfig } from "./project";++vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));+vi.mock("./persistence", () => ({ readMediaFileBytes: vi.fn(async () => new ArrayBuffer(0)) }));+vi.mock("./exportPipeline", () => ({ demux: vi.fn() }));+beforeEach(() => vi.mocked(invoke).mockResolvedValue("video-1"));+afterEach(() => { vi.unstubAllGlobals(); vi.resetAllMocks(); });++const input = { name: "Movement", sourcePath: "C:/movement.mp4", startSeconds: 1, durationSeconds: 2, includeAudio: false };+const source: DemuxedSource = { config: { codec: "avc1" }, durationSeconds: 4, hasAudio: false,+ samples: [0, 500_000, 1_000_000, 1_500_000, 2_000_000, 2_500_000, 3_000_000].map((timestampUs) => ({+ timestampUs, durationUs: 500_000, key: timestampUs === 0, data: new Uint8Array(1),+ })) };++function decoderMocks() {+ const frames: Array<{ close: ReturnType<typeof vi.fn> }> = [];+ const make = (timestamp: number) => {+ const frame = { timestamp, displayWidth: 1920, displayHeight: 1080, close: vi.fn(), clone: () => make(timestamp) };+ frames.push(frame); return frame;+ };+ const close = vi.fn();+ vi.stubGlobal("EncodedVideoChunk", class { constructor(public init: EncodedVideoChunkInit) {} });+ vi.stubGlobal("VideoDecoder", class {+ static async isConfigSupported(config: VideoDecoderConfig) { return { supported: true, config }; }+ state = "configured"; decodeQueueSize = 0;+ constructor(private init: VideoDecoderInit) {}+ configure() {}+ decode(chunk: { init: EncodedVideoChunkInit }) { this.init.output(make(chunk.init.timestamp) as unknown as VideoFrame); }+ async flush() {}+ close = close;+ });+ vi.stubGlobal("OffscreenCanvas", class {+ constructor(public width: number, public height: number) {}+ getContext() { return { drawImage: vi.fn(), getImageData: () => ({ data: new Uint8ClampedArray(this.width * this.height * 4) }) }; }+ });+ return { frames, close };+}++it("validates trim bounds and uses at most 15 seconds for an unspecified clip", () => {+ expect(referenceVideoRange(input, 4)).toEqual({ start: 1, duration: 2 });+ expect(referenceVideoRange({ ...input, durationSeconds: undefined }, 30).duration).toBe(15);+ expect(() => referenceVideoRange({ ...input, durationSeconds: 1 }, 4)).toThrow("2 to 15");+ expect(() => referenceVideoRange(input, 2)).toThrow("beyond");+ expect(() => referenceVideoRange({ ...input, startSeconds: NaN }, 4)).toThrow();+});++it("uploads the trimmed interval at clip-relative times and closes every frame", async () => {+ const { frames, close } = decoderMocks();+ await uploadReferenceVideoFrames(source, "video-1", 1, 2, () => false);+ const calls = vi.mocked(invoke).mock.calls.filter(([command]) => command === "append_reference_video");+ expect(calls.map((call) => (call[2]?.headers as Record<string, string>)["x-reference-time"])).toEqual(["0", "0.5", "1", "1.5"]);+ expect(calls[0][2]?.headers).toMatchObject({ "x-reference-width": "768", "x-reference-height": "432" });+ expect(calls.every((call) => call[1] instanceof Uint8Array)).toBe(true);+ expect(frames.every((frame) => frame.close.mock.calls.length === 1)).toBe(true);+ expect(close).toHaveBeenCalledOnce();+});++it("releases native inputs and decoder frames after an upload failure", async () => {+ const { frames, close } = decoderMocks();+ vi.mocked(demux).mockResolvedValue(source);+ vi.mocked(invoke).mockImplementation(async (command) => {+ if (command === "append_reference_video") throw new Error("upload failed");+ return "video-1";+ });+ const config = createProjectConfig({ name: "Test", prompt: "", aspectRatio: "16:9", resolution: "1080p", targetDurationSeconds: 20 });+ const request = { jobId: "job", prompt: "", frames: 120, steps: 12, seed: 1, canvasWidth: 736, canvasHeight: 416, referencePaths: [], referenceVideos: [input] };+ await expect(prepareReferenceVideos("C:/project", request, config, () => false, () => undefined)).rejects.toThrow("upload failed");+ expect(invoke).toHaveBeenCalledWith("release_reference_videos", { ids: ["video-1"] });+ expect(frames.every((frame) => frame.close.mock.calls.length === 1)).toBe(true);+ expect(close).toHaveBeenCalledOnce();+});++it("stops decoding promptly when preparation is cancelled", async () => {+ const { frames, close } = decoderMocks();+ let cancelled = false;+ vi.mocked(invoke).mockImplementation(async () => { cancelled = true; });+ await expect(uploadReferenceVideoFrames(source, "video-1", 1, 2, () => cancelled)).rejects.toThrow("cancelled");+ expect(frames.every((frame) => frame.close.mock.calls.length === 1)).toBe(true);+ expect(close).toHaveBeenCalledOnce();+});++it("trims and interleaves stereo sound without exceeding the video duration", () => {+ const audio = { sampleRate: 2, numberOfChannels: 2, length: 8,+ getChannelData: (channel: number) => new Float32Array(channel ? [0, 0, -0.5, -2, 0.5, 0, 0, 0] : [0, 0, 0.5, 2, -0.5, 0, 0, 0]) } as AudioBuffer;+ expect(Array.from(referenceSoundtrack(audio, 1, 2))).toEqual([0.5, -0.5, 1, -1, -0.5, 0.5, 0, 0]);+});diff --git a/src/lib/referenceVideo.ts b/src/lib/referenceVideo.tsnew file mode 100644index 0000000..a0611e2--- /dev/null+++ b/src/lib/referenceVideo.ts@@ -0,0 +1,175 @@+import { invoke } from "@tauri-apps/api/core";+import { demux, type DemuxedSource } from "./exportPipeline";+import { readMediaFileBytes } from "./persistence";+import type { ProjectConfig } from "./project";+import type { SlopfabGenerationRequest } from "./runtime";++type VideoInput = NonNullable<SlopfabGenerationRequest["referenceVideos"]>[number];+const cancelledError = () => new Error("Video reference preparation cancelled.");++export function referenceVideoRange(input: VideoInput, sourceDuration: number) {+ const start = input.startSeconds;+ const duration = input.durationSeconds ?? Math.min(15, sourceDuration - start);+ if (!Number.isFinite(start) || start < 0 || !Number.isFinite(duration) || duration < 2 || duration > 15) {+ throw new Error(`${input.name}: choose a clip lasting 2 to 15 seconds with a nonnegative start.`);+ }+ if (!Number.isFinite(sourceDuration) || start + duration > sourceDuration + 0.001) {+ throw new Error(`${input.name}: the selected clip extends beyond the source video.`);+ }+ return { start, duration };+}++export function sourceVideoDuration(source: DemuxedSource): number {+ const first = source.samples.reduce((value, sample) => Math.min(value, sample.timestampUs), Infinity);+ const last = source.samples.reduce((value, sample) => Math.max(value, sample.timestampUs + sample.durationUs), -Infinity);+ return source.durationSeconds ?? (last - first) / 1_000_000;+}++export async function inspectReferenceVideo(folderPath: string, sourcePath: string) {+ const source = await demux(await readMediaFileBytes(folderPath, { sourcePath }), "Video reference");+ const duration = sourceVideoDuration(source);+ referenceVideoRange({ name: "Video reference", startSeconds: 0, includeAudio: true }, duration);+ return { durationSeconds: Math.min(15, duration), includeAudio: Boolean(source.hasAudio) };+}++/** Decode only the selected interval. At most a small decoder batch and one+ * retained frame are alive; binary IPC avoids JSON/base64 copies of pixels.+ * Slopfab consumes host RGB, so readback is confined to this ingestion step. */+export async function uploadReferenceVideoFrames(source: DemuxedSource, id: string, start: number, duration: number, cancelled: () => boolean) {+ if (typeof VideoDecoder === "undefined") throw new Error("This webview does not support WebCodecs video decoding.");+ const support = await VideoDecoder.isConfigSupported({ ...source.config, hardwareAcceleration: "prefer-hardware" });+ if (!support.supported) throw new Error(`This computer cannot decode reference codec ${source.config.codec}.`);+ const origin = source.samples.reduce((value, sample) => Math.min(value, sample.timestampUs), Infinity);+ const beginUs = origin + Math.round(start * 1_000_000);+ const endUs = beginUs + Math.round(duration * 1_000_000);+ let first = 0;+ for (let index = 0; index < source.samples.length; index++) {+ if (source.samples[index].key && source.samples[index].timestampUs <= beginUs) first = index;+ }+ // Include the rest of the last GOP for reordered/B frames at the trim edge.+ let last = source.samples.length;+ for (let index = first + 1; index < source.samples.length; index++) {+ if (source.samples[index].key && source.samples[index].timestampUs >= endUs) { last = index; break; }+ }+ const state: { previous: VideoFrame | null; error: unknown; pending: number; uploaded: number; slot: number; canvas: OffscreenCanvas | null } =+ { previous: null, error: null, pending: 0, uploaded: 0, slot: -1, canvas: null };+ let chain = Promise.resolve();+ const check = () => { if (cancelled()) throw cancelledError(); if (state.error) throw state.error; };+ const upload = async (frame: VideoFrame, seconds: number) => {+ check();+ const slot = Math.round(seconds * 24);+ if (slot <= state.slot || slot >= Math.round(duration * 24)) return;+ if (!state.canvas) {+ const aspect = frame.displayWidth / frame.displayHeight;+ if (aspect < 0.25 || aspect > 4) throw new Error("Video reference aspect ratio must be between 1:4 and 4:1.");+ const scale = Math.min(1, 768 / Math.max(frame.displayWidth, frame.displayHeight));+ state.canvas = new OffscreenCanvas(Math.max(1, Math.round(frame.displayWidth * scale)), Math.max(1, Math.round(frame.displayHeight * scale)));+ }+ const canvas = state.canvas;+ const context = canvas.getContext("2d", { willReadFrequently: true });+ if (!context) throw new Error("Could not create the video reference pixel buffer.");+ context.drawImage(frame, 0, 0, canvas.width, canvas.height);+ const pixels = context.getImageData(0, 0, canvas.width, canvas.height).data;+ await invoke("append_reference_video", new Uint8Array(pixels.buffer), { headers: {+ "x-reference-id": id, "x-reference-width": String(canvas.width), "x-reference-height": String(canvas.height),+ "x-reference-time": String(state.uploaded === 0 ? 0 : seconds),+ } });+ state.slot = slot;+ state.uploaded += 1;+ };+ const consume = async (frame: VideoFrame) => {+ check();+ if (frame.timestamp <= beginUs) {+ state.previous?.close();+ state.previous = frame.clone();+ return;+ }+ if (!state.uploaded && state.previous) {+ await upload(state.previous, 0);+ state.previous.close(); state.previous = null;+ }+ if (frame.timestamp < endUs) await upload(frame, state.uploaded ? (frame.timestamp - beginUs) / 1_000_000 : 0);+ };+ const decoder = new VideoDecoder({+ output: (frame) => {+ state.pending += 1;+ chain = chain.then(() => consume(frame)).catch((error: unknown) => { state.error ??= error; })+ .finally(() => { frame.close(); state.pending -= 1; });+ },+ error: (error) => { state.error = error; },+ });+ try {+ decoder.configure(support.config ?? source.config);+ for (let index = first; index < last; index++) {+ check();+ const sample = source.samples[index];+ decoder.decode(new EncodedVideoChunk({ type: sample.key ? "key" : "delta", timestamp: sample.timestampUs, duration: sample.durationUs, data: sample.data }));+ while (decoder.decodeQueueSize > 8 || state.pending > 8) {+ await new Promise((resolve) => setTimeout(resolve, 0)); check();+ }+ }+ await decoder.flush();+ await chain;+ check();+ if (!state.uploaded && state.previous) await upload(state.previous, 0);+ if (!state.uploaded) throw new Error("The selected reference interval contains no decodable frames.");+ } finally {+ if (decoder.state !== "closed") decoder.close();+ await chain;+ state.previous?.close();+ }+}++export function referenceSoundtrack(buffer: AudioBuffer, start: number, duration: number): Float32Array {+ if (buffer.numberOfChannels < 1 || buffer.numberOfChannels > 2) throw new Error("Video reference soundtracks must be mono or stereo. Turn off Include sound for this clip or use a mono/stereo source.");+ const offset = Math.round(start * buffer.sampleRate);+ const length = Math.min(Math.floor(duration * buffer.sampleRate), buffer.length - offset);+ if (length <= 0) throw new Error("The selected video interval contains no soundtrack samples. Turn off Include sound for this clip.");+ const pcm = new Float32Array(length * buffer.numberOfChannels);+ for (let channel = 0; channel < buffer.numberOfChannels; channel++) {+ const samples = buffer.getChannelData(channel);+ for (let index = 0; index < length; index++) pcm[index * buffer.numberOfChannels + channel] = Math.max(-1, Math.min(1, samples[offset + index]));+ }+ return pcm;+}++export async function releaseReferenceVideos(ids: string[]) {+ if (ids.length) await invoke("release_reference_videos", { ids });+}++export async function prepareReferenceVideos(folderPath: string, request: SlopfabGenerationRequest, config: ProjectConfig, cancelled: () => boolean, report: (detail: string) => void): Promise<string[]> {+ const inputs = request.referenceVideos ?? [];+ if (inputs.length > 3 || request.referencePaths.length > 9) throw new Error("Use at most nine images and three video references per generation.");+ const ids: string[] = [];+ let totalDuration = 0;+ try {+ for (const input of inputs) {+ if (cancelled()) throw cancelledError();+ report(`Preparing video reference: ${input.name}`);+ if (!/\.(mp4|m4v|mov)$/i.test(input.sourcePath ?? input.relativePath ?? "")) throw new Error(`${input.name}: video references currently require an MP4 or MOV file.`);+ const bytes = await readMediaFileBytes(folderPath, input);+ const source = await demux(bytes, input.name);+ const { start, duration } = referenceVideoRange(input, sourceVideoDuration(source));+ totalDuration += duration;+ if (totalDuration > 15 + 1e-9) throw new Error("Video references must total no more than 15 seconds. Shorten the clips in References.");+ if (cancelled()) throw cancelledError();+ const id = await invoke<string>("create_reference_video", { durationSeconds: duration, config });+ ids.push(id);+ await uploadReferenceVideoFrames(source, id, start, duration, cancelled);+ if (input.includeAudio && source.hasAudio) {+ report(`Preparing soundtrack: ${input.name}`);+ const context = new OfflineAudioContext(2, 1, 48_000);+ const decoded = await context.decodeAudioData(bytes.slice(0));+ if (cancelled()) throw cancelledError();+ const pcm = referenceSoundtrack(decoded, start, duration);+ await invoke("set_reference_video_audio", new Uint8Array(pcm.buffer), { headers: {+ "x-reference-id": id, "x-reference-channels": String(decoded.numberOfChannels), "x-reference-rate": String(decoded.sampleRate),+ } });+ }+ }+ return ids;+ } catch (error) {+ await releaseReferenceVideos(ids);+ throw error;+ }+}diff --git a/src/lib/runtime.ts b/src/lib/runtime.tsindex e96ad25..96546f7 100644--- a/src/lib/runtime.ts+++ b/src/lib/runtime.ts@@ -88,6 +88,9 @@ export interface SlopfabGenerationRequest {canvasWidth: number;canvasHeight: number;referencePaths: string[];+ referenceVideos?: Array<{ name: string; relativePath?: string | null; sourcePath?: string | null; startSeconds: number; durationSeconds?: number; includeAudio: boolean }>;+ /** Temporary native handles, populated by the work queue and never saved. */+ referenceVideoIds?: string[];/** Resolved by the app queue before native submission. */previousSceneId?: string;}diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.tsindex f86a722..8e5cebe 100644--- a/src/lib/workQueue.test.ts+++ b/src/lib/workQueue.test.ts@@ -10,6 +10,7 @@ import { cancelSlopfabGeneration, enqueueSlopfabGeneration, resolveSlopfabPlan }import { saveEngineSettings, EMPTY_ENGINE_SETTINGS } from "./settings";import { WorkQueue, type GenerationSubmission } from "./workQueue";import { saveSceneLastFrame } from "./sceneLastFrame";+import { prepareReferenceVideos, releaseReferenceVideos } from "./referenceVideo";import { downloadTemplateWeights, getWeightDownloadState } from "./weightDownloads";vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));@@ -17,6 +18,7 @@ vi.mock("@tauri-apps/api/event", () => ({ listen: vi.fn() }));vi.mock("./generatedVideo", () => ({ saveGeneratedScene: vi.fn(), releaseRendered: vi.fn(async () => true) }));vi.mock("./timelineThumbnails", () => ({ purgeTimelineThumbnails: vi.fn(async () => undefined) }));vi.mock("./sceneLastFrame", () => ({ saveSceneLastFrame: vi.fn(async () => "cache/scene-last/work-new.png") }));+vi.mock("./referenceVideo", () => ({ prepareReferenceVideos: vi.fn(async () => ["video-1"]), releaseReferenceVideos: vi.fn(async () => undefined) }));vi.mock("./runtime", () => ({ cancelSlopfabGeneration: vi.fn(), enqueueSlopfabGeneration: vi.fn(), resolveSlopfabPlan: vi.fn() }));const handlers = new Map<string, (event: { payload: unknown }) => void>();const stops: (() => void)[] = [];@@ -51,6 +53,30 @@ const finish = async (queue: WorkQueue, id: string) => {};describe("application work queue", () => {+ it("prepares video inputs for both planning and generation and releases them after completion", async () => {+ const { queue, first } = setup();+ const next = submission(first);+ next.request.referenceVideos = [{ name: "Motion", sourcePath: "C:/motion.mp4", startSeconds: 1, durationSeconds: 2, includeAudio: true }];+ queue.enqueue(first, [next]);+ await waitFor(() => expect(enqueueSlopfabGeneration).toHaveBeenCalledOnce());+ expect(prepareReferenceVideos).toHaveBeenCalledOnce();+ expect(vi.mocked(resolveSlopfabPlan).mock.calls[0][0].referenceVideoIds).toEqual(["video-1"]);+ expect(vi.mocked(enqueueSlopfabGeneration).mock.calls[0][0].referenceVideoIds).toEqual(["video-1"]);+ await finish(queue, queue.getSnapshot()[0].id);+ await waitFor(() => expect(releaseReferenceVideos).toHaveBeenCalledWith(["video-1"]));+ });++ it("releases prepared video inputs when planning fails", async () => {+ const { queue, first } = setup();+ const next = submission(first);+ next.request.referenceVideos = [{ name: "Motion", sourcePath: "C:/motion.mp4", startSeconds: 0, durationSeconds: 2, includeAudio: false }];+ vi.mocked(resolveSlopfabPlan).mockRejectedValueOnce(new Error("Reference limit exceeded"));+ queue.enqueue(first, [next]);+ await waitFor(() => expect(queue.getSnapshot()[0].status).toBe("failed"));+ expect(releaseReferenceVideos).toHaveBeenCalledWith(["video-1"]);+ expect(enqueueSlopfabGeneration).not.toHaveBeenCalled();+ });+it("resolves a linked first frame after the previous scene finishes encoding and saving", async () => {const { queue, first } = setup();first.update((config) => ({ ...config, generationJobs: [config.generationJobs[0], { ...config.generationJobs[0], id: "linked", usePreviousSceneLastFrame: true }] }));diff --git a/src/lib/workQueue.ts b/src/lib/workQueue.tsindex e106aa0..dac793e 100644--- a/src/lib/workQueue.ts+++ b/src/lib/workQueue.ts@@ -10,6 +10,7 @@ import { cancelSlopfabGeneration, enqueueSlopfabGeneration, resolveSlopfabPlan,import { withEngineSettings } from "./settings";import { purgeTimelineThumbnails } from "./timelineThumbnails";import { ReferenceIconWork } from "./referenceIconWork";+import { prepareReferenceVideos, releaseReferenceVideos } from "./referenceVideo";export type WorkStatus = "queued" | "preparing" | "generating" | "encoding" | "completed" | "failed" | "cancelled";export interface GenerationSubmission { job: GenerationJob; request: SlopfabGenerationRequest; snapshot: string }@@ -215,6 +216,11 @@ export class WorkQueue {}await purgeTimelineThumbnails(next.folderPath, work.sceneId).catch(() => undefined);if (work.cancelled) continue;+ if (work.request.referenceVideos?.length) {+ work.request.referenceVideoIds = await prepareReferenceVideos(next.folderPath, work.request, work.config,+ () => work.cancelled, (detail) => this.patch(work.id, { detail }));+ }+ if (work.cancelled) continue;const plan = await resolveSlopfabPlan(work.request, work.config);if (work.cancelled) continue;this.patch(work.id, { status: "generating", detail: `Generating ${plan.alignedFrames} frames at ${plan.canvasWidth} × ${plan.canvasHeight}` });@@ -224,6 +230,11 @@ export class WorkQueue {if (work.cancelled) await this.requestNativeCancellation(work);await work.done;} catch (reason) { this.fail(work, reason); }+ finally {+ await releaseReferenceVideos(work.request.referenceVideoIds ?? []).catch((reason) =>+ writeDiagnostic("error", "work-queue", "references.release_failed", describeDiagnosticError(reason), { workId: work.id }));+ delete work.request.referenceVideoIds;+ }}} finally { this.pumping = false; }}diff --git a/src/styles/references.css b/src/styles/references.cssindex b6ee16b..0ce28c5 100644--- a/src/styles/references.css+++ b/src/styles/references.css@@ -578,6 +578,37 @@font: inherit;}+.reference-video {+ display: grid;+ gap: var(--space-3);+ padding: var(--space-4);+}++.reference-video-preview {+ display: block;+ width: 100%;+ max-height: 280px;+ background: var(--field-bg);+ border-radius: var(--radius-sm);+}++.reference-video p {+ color: var(--text-muted);+ font-size: var(--text-xs);+}++.reference-video label {+ display: flex;+ align-items: center;+ gap: var(--space-2);+}++.reference-video input[type="number"] {+ min-width: 0;+ width: 80px;+ margin-left: auto;+}+.reference-fields__actions {display: flex;flex-wrap: wrap;