AI media harness for planning, generating, and editing locally
TypeScript
Add music generation workspace
Bittif360fd1e85d6
diff --git a/README.md b/README.mdindex 905ff03..c506a1c 100644--- a/README.md+++ b/README.md@@ -18,7 +18,7 @@----Slopus combines an AI assistant, local image and video generation, reusable references and a timeline editor. SlopFab powers generation without Python; WebCodecs and WebGPU handle media processing without bundled FFmpeg.+Slopus combines an AI assistant, local image, video and music generation, reusable references and a timeline editor. SlopFab powers generation without Python; WebCodecs and WebGPU handle media processing without bundled FFmpeg.@@ -27,12 +27,13 @@ Slopus combines an AI assistant, local image and video generation, reusable refe- **Agent.** Plan scenes, generate media, inspect frames and edit projects with Claude Code, Codex, OpenRouter or a local OpenAI-compatible model. Toggle the pane with Ctrl+Shift+A.- **Video.** Generate Prompt, First & last frame and other [scene types](docs/scene-types.md), with shot timing, dialogue, camera and sound controls. Keep up to four takes and an editable Draft; use **Make Primary** to choose the timeline video.- **Images.** Generate with MiniMax H3, MOSS Image or Fibo 2.0. Use whole-image, box, lasso and Zoom edits where supported, with drafts and versions in the image bar.+- **Music.** Generate songs with YuE2 or MiniMax Music3, edit lyrics and style, browse grid or list views, and play, reference or add songs to the timeline.- **References.** Reuse descriptions, images, trimmed video clips and [H3 RefMods](docs/refmods.md) across scenes.- **Generators.** Manage model weights, supported LoRAs, attention and generation effects. Share [generator templates](docs/generator-weights.md#sharing-generator-templates) as `.slop` bundles or run jobs on LAN workers.- **Timeline.** Arrange video and audio with trimming, splitting, snapping, waveforms, transitions, chroma key and GPU effects.- **Motions.** Ask the agent to create JavaScript animations, edit elements and parameters, and render them on the timeline or as reference videos.- **Export.** Save MP4 video with H.264, VP9 or AV1 where encoders are available, or export images as JPG or PNG. Exports run in the background.-- **Project editing.** Timeline, Video, Image, References and Export views share Save, Undo and Redo. Projects and media are stored locally; agent requests go to the selected provider. API keys stay outside project files.+- **Project editing.** Timeline, Video, Image, Music, References and Export views share Save, Undo and Redo. Projects and media are stored locally; agent requests go to the selected provider. API keys stay outside project files.## Setup@@ -70,3 +71,5 @@ Tauri 2, Rust, React, TypeScript, WebCodecs, WebGPU, `mp4box.js`, `mp4-muxer` anMiniMax H3 model weights are licensed separately under the **MiniMax H3 COMMUNITY LICENSE AGREEMENT**.Fibo 2.0 is released under CC BY-NC 4.0 for non-commercial use. Commercial use is subject to an agreement with BRIA.++[YuE2](https://huggingface.co/Comfy-Org/YuE2) uses CC BY-NC 4.0; [MiniMax Music3](https://huggingface.co/Comfy-Org/MiniMax-Music-3) uses Apache 2.0.diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.mdindex c4d46cd..cf09e8d 100644--- a/RELEASE_NOTES.md+++ b/RELEASE_NOTES.md@@ -4,6 +4,7 @@ Changes since 0.3.2.## What's new+- **Music.** Generate with YuE2 or MiniMax Music3, browse songs in grid or list view, use bottom playback controls, and create references or timeline audio.- **Motions.** Create JavaScript animations with the agent, edit elements and parameters, and export them on the timeline or as reference videos.- **Scene history.** Keep up to four takes and an editable Draft, restore take settings, choose the primary timeline video, and duplicate scenes.- **Agent frame inspection.** Agents can view timeline frames, scenes, takes, references, generated images and Motions at a requested time.@@ -21,4 +22,4 @@ Changes since 0.3.2.- Image edits preserve source images, prompts and settings; queued results get separate entries, and whole-image edits disable feathering.- Imported images now expose steps, seed and resolution; new images use generator step defaults, and renamed image roots retain their names.- Generation effects reuse VAE fingerprint checks to avoid repeated calibration work.-- Updated SlopFab to API 1.36 and LAN protocol 17. Update workers alongside the app.+- Updated SlopFab to API 1.38 and LAN protocol 18. Update workers alongside the app.diff --git a/docs/slopfab-compatibility.md b/docs/slopfab-compatibility.mdindex b4504f6..8aeba40 100644--- a/docs/slopfab-compatibility.md+++ b/docs/slopfab-compatibility.md@@ -1,5 +1,16 @@# Slopfab compatibility+## Music (API 1.38)++Bundled Windows and Linux runtimes support YuE2 and MiniMax Music3 on CUDA and+Vulkan, with FFmpeg disabled. LAN protocol 18 carries music settings and+audio-only results. Update workers alongside the app.++Music requests use `slopfab_request_set_music_settings` with style in the prompt,+model-specific controls and portable Vulkan arithmetic. They bypass H3 video+settings. Stereo samples are saved as 24-bit PCM WAV in `media/generated` and+appear in Music and timeline media. Song settings persist in `musicSongs`.+## Generation effects (API 1.31)Scene and image root inspectors provide **Effects > Add effect** for native@@ -45,7 +56,7 @@ with model weights were not evaluated in this integration.## MOSS image generation (API 1.30+)-The bundled Windows and Linux libraries expose SlopFab API 1.36.0, including+The bundled Windows and Linux libraries expose SlopFab API 1.38.0, includingthe MOSS model-family setter introduced in 1.30. Both builds disable FFmpeg.MOSS uses CUDA/Flash attention, the image VAE slot, and a separate Qwen3.5tokenizer file. It returns one RGB frame through the existing image save path.diff --git a/src-tauri/core/src/models.rs b/src-tauri/core/src/models.rsindex 3b3dc31..d31cb8f 100644--- a/src-tauri/core/src/models.rs+++ b/src-tauri/core/src/models.rs@@ -8,6 +8,7 @@ pub struct GeneratorFeatures {pub label: String,pub image_generation: bool,pub video_generation: bool,+ pub music_generation: bool,pub image_editing: bool,pub image_references: bool,pub image_outpainting: bool,diff --git a/src-tauri/core/src/rendered.rs b/src-tauri/core/src/rendered.rsindex 3794891..003143c 100644--- a/src-tauri/core/src/rendered.rs+++ b/src-tauri/core/src/rendered.rs@@ -237,7 +237,8 @@ pub fn from_source(.and_then(|value| value.checked_mul(height as usize)).and_then(|value| value.checked_mul(channels as usize)).ok_or_else(|| "The render's dimensions do not fit in memory.".to_string())?;- if frame_count == 0 || width == 0 || height == 0 || channels != 3 {+ let audio_only = frame_count == 0 && width == 0 && height == 0 && video_float_count == 0 && !audio.is_empty();+ if !audio_only && (frame_count == 0 || width == 0 || height == 0 || channels != 3) {return Err(format!("The render returned {frame_count} frames of {width}x{height}x{channels}."));@@ -416,6 +417,18 @@ mod tests {use super::*;use std::sync::{atomic::{AtomicBool, Ordering}, Arc};+ #[test]+ fn audio_only_results_keep_samples_without_requesting_video_frames() {+ let dropped = Arc::new(AtomicBool::new(false));+ let result = from_source("music", Box::new(TrackedStill { dropped: dropped.clone(), fail: true }),+ vec![0.25, -0.25], 0, 0, 0, 0, 0, 0.0, 2, 48000).unwrap();+ assert!(result.frame(0).unwrap().is_none());+ assert_eq!(result.audio_bytes(), [0.25_f32.to_le_bytes(), (-0.25_f32).to_le_bytes()].concat());+ assert_eq!(result.summary().audio_samples, 2);+ drop(result);+ assert!(dropped.load(Ordering::Acquire));+ }+struct TrackedStill {dropped: Arc<AtomicBool>,fail: bool,diff --git a/src-tauri/core/src/slopfab/config.rs b/src-tauri/core/src/slopfab/config.rsindex 140a213..125e765 100644--- a/src-tauri/core/src/slopfab/config.rs+++ b/src-tauri/core/src/slopfab/config.rs@@ -123,6 +123,7 @@ impl Configuration {}pub(super) fn validate_inputs(&self, request: &GenerationRequest) -> Result<(), String> {+ if request.music.is_some() { return super::music::validate(request, self); }let features = crate::models::generator_features(&self.model_type);if let Some(negative) = &request.negative_prompt {if negative.contains('\0') { return Err("Negative prompt cannot contain null characters.".into()); }diff --git a/src-tauri/core/src/slopfab/ffi/mod.rs b/src-tauri/core/src/slopfab/ffi/mod.rsindex 3a768f2..8ddfb54 100644--- a/src-tauri/core/src/slopfab/ffi/mod.rs+++ b/src-tauri/core/src/slopfab/ffi/mod.rs@@ -237,6 +237,12 @@ impl Api {self.check(r)?;self.inner.set_image_guidance(r.pointer.as_ptr(), scale, negative_prompt)}+ pub fn set_music_settings(&self, r: &RequestHandle, settings: &str) -> Result<(), String> {+ self.check(r)?; self.inner.set_music_settings(r.pointer.as_ptr(), settings)+ }+ pub fn set_vulkan_portable(&self, r: &RequestHandle) -> Result<(), String> {+ self.check(r)?; self.inner.set_vulkan_portable(r.pointer.as_ptr())+ }pub fn set_model_family(&self, r: &RequestHandle, family: &str) -> Result<(), String> {self.check(r)?;self.inner.set_model_family(r.pointer.as_ptr(), family)diff --git a/src-tauri/core/src/slopfab/ffi/raw/mod.rs b/src-tauri/core/src/slopfab/ffi/raw/mod.rsindex e400f62..36d261a 100644--- a/src-tauri/core/src/slopfab/ffi/raw/mod.rs+++ b/src-tauri/core/src/slopfab/ffi/raw/mod.rs@@ -80,6 +80,8 @@ type ExportRefmod = unsafe extern "C" fn(*const Request, *const c_char, *const ctype SetImageGuidance = unsafe extern "C" fn(*mut Request, f32, *const c_char, *const i32, i32) -> i32;pub struct Api {+ set_music_settings: Option<unsafe extern "C" fn(*mut Request, *const c_char) -> i32>,+ set_vulkan_arithmetic: Option<unsafe extern "C" fn(*mut Request, i32) -> i32>,set_image_guidance: Option<SetImageGuidance>,_library: Library,capi_version: unsafe extern "C" fn() -> u32,@@ -247,6 +249,8 @@ impl Api {unsafe extern "C" fn(*mut Request, u64) -> i32),set_image_guidance: library.get::<SetImageGuidance>(b"slopfab_request_set_image_guidance\0").ok().map(|symbol| *symbol),+ set_music_settings: library.get::<unsafe extern "C" fn(*mut Request, *const c_char) -> i32>(b"slopfab_request_set_music_settings\0").ok().map(|symbol| *symbol),+ set_vulkan_arithmetic: library.get::<unsafe extern "C" fn(*mut Request, i32) -> i32>(b"slopfab_request_set_vulkan_arithmetic\0").ok().map(|symbol| *symbol),set_model_family: library.get::<unsafe extern "C" fn(*mut Request, *const c_char) -> i32>(b"slopfab_request_set_model_family\0").ok().map(|symbol| *symbol),@@ -601,6 +605,15 @@ impl Api {let skip_layers = [22_i32];self.error(unsafe { set(r, scale, negative.as_ptr(), skip_layers.as_ptr(), 1) })}+ pub fn set_music_settings(&self, r: *mut Request, settings: &str) -> Result<(), String> {+ let set = self.set_music_settings.ok_or("Music requires SlopFab API 1.38 or newer. Update the app and workers.")?;+ let json = CString::new(settings).map_err(|e| e.to_string())?;+ self.error(unsafe { set(r, json.as_ptr()) })+ }+ pub fn set_vulkan_portable(&self, r: *mut Request) -> Result<(), String> {+ let set = self.set_vulkan_arithmetic.ok_or("Music requires SlopFab API 1.38 or newer.")?;+ self.error(unsafe { set(r, 1) })+ }pub fn set_model_family(&self, r: *mut Request, family: &str) -> Result<(), String> {let set = self.set_model_family.ok_or("This model requires a newer SlopFab runtime. Update the machine running generation.")?;let value = CString::new(family).map_err(|_| "Invalid model family.")?;diff --git a/src-tauri/core/src/slopfab/generation.rs b/src-tauri/core/src/slopfab/generation.rsindex 62221c7..64f8764 100644--- a/src-tauri/core/src/slopfab/generation.rs+++ b/src-tauri/core/src/slopfab/generation.rs@@ -174,7 +174,7 @@ fn run_single_generation(}fn collect_output(item: &QueueItem, output: ffi::Output, source: Box<dyn rendered::FrameSource>, timing_profile: String) -> Result<(OutputMetadata, rendered::RenderedVideo), String> {- let (_, frames) = scene_frame_window(&item.request, output.frames)?;+ let frames = if item.request.music.is_some() { 0 } else { scene_frame_window(&item.request, output.frames)?.1 };// The VAE decoded both sides of the boundary together. Keep that complete// output, including the source-side pixels and soundtrack near the join.let audio = output.audio;@@ -191,12 +191,14 @@ fn collect_output(item: &QueueItem, output: ffi::Output, source: Box<dyn rendereseconds_total: output.seconds_total,steps_computed: output.steps_computed,steps_skipped: output.steps_skipped,- duration_seconds: if output.fps > 0.0 {+ duration_seconds: if item.request.music.is_some() && output.audio_channels > 0 && output.audio_sample_rate > 0 {+ audio.len() as f64 / output.audio_channels as f64 / output.audio_sample_rate as f64+ } else if output.fps > 0.0 {frames as f64 / output.fps} else {0.0},- boundary: "Latents are saved in the project. Decoded buffers remain owned by slopfab; the webview encodes the full joined video and audio on demand.",+ boundary: if item.request.music.is_some() { "Music samples are ready to save as WAV." } else { "Latents are saved in the project. Decoded buffers remain owned by slopfab; the webview encodes the full joined video and audio on demand." },};let pictures = rendered::from_source(&item.request.job_id,diff --git a/src-tauri/core/src/slopfab/generation_tests.rs b/src-tauri/core/src/slopfab/generation_tests.rsindex 2814201..d0694e7 100644--- a/src-tauri/core/src/slopfab/generation_tests.rs+++ b/src-tauri/core/src/slopfab/generation_tests.rs@@ -2,6 +2,32 @@ use super::*;use crate::slopfab::{config::Configuration, events::NativeEvent, references::ReferenceVideos, types::{GenerationRequest, ProgressEvent}};use std::sync::{Arc, Mutex, atomic::AtomicBool};+#[test]+#[ignore = "requires local music weights and a GPU; set SLOPUS_MUSIC_WEIGHTS"]+fn music_gpu_smoke() {+ use crate::settings::{ProviderOption, ProviderSetting};+ let root = std::path::PathBuf::from(std::env::var("SLOPUS_MUSIC_WEIGHTS").unwrap());+ for family in ["yue2", "minimax_music3"] {+ let paths = if family == "yue2" { vec![("transformer", "yue2_3b_int8_convrot.safetensors")] } else {+ vec![("transformer", "minimax_music3_dit_int8_convrot.safetensors"), ("textEncoder", "text_encoder/minimax_music3_text_encoder_pruned_int8_convrot.safetensors"), ("audioVae", "minimax_music3_dav.safetensors")]+ };+ let mut options = std::collections::BTreeMap::from([("modelType".into(), ProviderOption::String(family.into()))]);+ for (key, file) in paths { options.insert(key.into(), ProviderOption::String(root.join(file).to_string_lossy().into_owned())); }+ let settings = std::collections::BTreeMap::from([("slopfab".into(), ProviderSetting { enabled: true, model: None, options })]);+ let request = GenerationRequest { job_id: format!("smoke-{family}"), prompt: "Quiet solo piano, instrumental".into(), steps: 2, seed: 42,+ music: Some(crate::slopfab::music::MusicSettings { duration: 2.0, planning: if family == "yue2" { "off" } else { "full" }.into(), ..Default::default() }), ..Default::default() };+ let item = QueueItem { request, configuration: Configuration::from_settings(&settings), references: ReferenceVideos::default(),+ events: Arc::new(|_| {}), cancel: Arc::new(AtomicBool::new(false)) };+ let (metadata, output) = run_generation(&item).unwrap();+ assert_eq!(metadata.frames, 0);+ assert_eq!(metadata.audio_channels, 2);+ println!("{family}: {} seconds, {} Hz stereo", metadata.duration_seconds, metadata.audio_sample_rate);+ assert!((1.9..=2.1).contains(&metadata.duration_seconds));+ assert!(output.frame(0).unwrap().is_none());+ assert!(!output.audio_bytes().is_empty());+ }+}+struct Pixels(u8);impl rendered::FrameSource for Pixels {fn frame_rgba(&self, _: u32, _: u32, _: u32) -> Result<Vec<u8>, String> { Ok(vec![self.0, 20, 30, 255]) }diff --git a/src-tauri/core/src/slopfab/h3.rs b/src-tauri/core/src/slopfab/h3.rsindex f89dbcf..a93df59 100644--- a/src-tauri/core/src/slopfab/h3.rs+++ b/src-tauri/core/src/slopfab/h3.rs@@ -17,6 +17,7 @@ pub(super) fn configure_request(purpose: RequestPurpose,references: &ReferenceVideos,) -> Result<(), String> {+ if request.music.is_some() { return super::music::configure(api, handle, request, configuration, platform); }configuration.validate_inputs(request)?;if crate::models::generator_features(&configuration.model_type).cuda_only {if platform == ComputePlatform::Vulkan {diff --git a/src-tauri/core/src/slopfab/mod.rs b/src-tauri/core/src/slopfab/mod.rsindex 5426fc9..a91e2d5 100644--- a/src-tauri/core/src/slopfab/mod.rs+++ b/src-tauri/core/src/slopfab/mod.rs@@ -18,6 +18,7 @@ mod generation;mod lcs;mod lcs_fingerprint;mod h3;+pub mod music;mod icons;mod planning;mod platform;diff --git a/src-tauri/core/src/slopfab/music.rs b/src-tauri/core/src/slopfab/music.rsnew file mode 100644index 0000000..13a3f0e--- /dev/null+++ b/src-tauri/core/src/slopfab/music.rs@@ -0,0 +1,86 @@+//! Audio-only music requests. Do not apply H3 video or sampling defaults.+use super::{config::Configuration, ffi::{Api, RequestHandle}, platform::ComputePlatform, types::GenerationRequest};+use serde::{Deserialize, Serialize};++#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]+#[serde(default, deny_unknown_fields)]+pub struct MusicSettings {+ pub lyrics: String,+ pub duration: f64,+ pub planning: String,+ pub abc: String,+ pub abc_max_tokens: i32,+ pub temperature: f64,+ pub top_p: f64,+ pub top_k: i32,+ pub repetition_penalty: f64,+ pub semantic_guidance: f64,+ pub flow_guidance: f64,+}+impl Default for MusicSettings {+ fn default() -> Self { Self { lyrics: String::new(), duration: 30.0, planning: "full".into(), abc: String::new(),+ abc_max_tokens: 4096, temperature: 1.0, top_p: 0.95, top_k: 0, repetition_penalty: 1.2,+ semantic_guidance: 0.0, flow_guidance: 0.0 } }+}+impl MusicSettings {+ pub fn validate(&self) -> Result<(), String> {+ if !self.duration.is_finite() || !(2.0..=300.0).contains(&self.duration)+ || !matches!(self.planning.as_str(), "full" | "melody" | "off")+ || !(32..=4096).contains(&self.abc_max_tokens) || !(0..=1024).contains(&self.top_k)+ || !self.temperature.is_finite() || !(0.0..=10.0).contains(&self.temperature)+ || !self.top_p.is_finite() || self.top_p <= 0.0 || self.top_p > 1.0+ || !self.repetition_penalty.is_finite() || !(1.0..=10.0).contains(&self.repetition_penalty)+ || !self.semantic_guidance.is_finite() || !(0.0..=10.0).contains(&self.semantic_guidance)+ || (self.semantic_guidance > 0.0 && self.semantic_guidance < 1.0)+ || !self.flow_guidance.is_finite() || !(0.0..=10.0).contains(&self.flow_guidance)+ || self.lyrics.chars().count() > 30000 || self.abc.chars().count() > 30000+ || self.lyrics.contains('\0') || self.abc.contains('\0') {+ return Err("Invalid music settings. Use a duration from 2 to 300 seconds and supported sampling ranges.".into());+ }+ if self.planning == "off" && !self.abc.is_empty() { return Err("ABC requires full or melody planning.".into()); }+ Ok(())+ }+}++pub(super) fn validate(request: &GenerationRequest, config: &Configuration) -> Result<(), String> {+ let settings = request.music.as_ref().ok_or("Music settings are missing.")?;+ settings.validate()?;+ if !matches!(config.model_type.as_str(), "yue2" | "minimax_music3") { return Err("Choose a music generator.".into()); }+ if request.prompt.trim().is_empty() || request.prompt.contains('\0') || request.prompt.chars().count() > 10000+ || !(2..=100).contains(&request.steps) || request.seed < -1 { return Err("Music needs a style prompt, 2–100 steps and a valid seed.".into()); }+ if request.still_image || request.image_edit.is_some() || request.reference_count() != 0 || !request.refmods.is_empty()+ || request.latent_upscale || request.latent_bridge.is_some() || request.continuation_path.is_some()+ || request.continuation_relative_path.is_some() || request.video_transition.is_some() || request.save_latents_path.is_some()+ || request.audio_steps.is_some() || request.video_sigma_shift.is_some() || request.audio_sigma_shift.is_some()+ || request.guidance_scale.is_some() || request.negative_prompt.is_some() || request.generation_effects.is_some()+ || config.animate || config.motion_cache || config.prompt_embedding.is_some()+ || !config.loras.as_ref().map_err(Clone::clone)?.is_empty() {+ return Err("Music does not support image/video controls, references, LoRAs or generation effects.".into());+ }+ let defaults = MusicSettings::default();+ if config.model_type == "minimax_music3" && (settings.planning != defaults.planning || !settings.abc.is_empty()+ || settings.abc_max_tokens != defaults.abc_max_tokens || settings.temperature != defaults.temperature+ || settings.top_p != defaults.top_p || settings.repetition_penalty != defaults.repetition_penalty) {+ return Err("MiniMax Music3 does not support YuE2 planning and sampling controls.".into());+ }+ if config.model_type == "yue2" && settings.flow_guidance != 0.0 { return Err("Flow guidance is available for MiniMax Music3 only.".into()); }+ Ok(())+}++pub(super) fn configure(api: &Api, handle: &RequestHandle, request: &GenerationRequest, config: &Configuration, platform: ComputePlatform) -> Result<(), String> {+ validate(request, config)?;+ api.set_model_family(handle, &config.model_type)?;+ api.set_prompt(handle, &request.prompt)?;+ api.set_music_settings(handle, &serde_json::to_string(request.music.as_ref().unwrap()).map_err(|e| e.to_string())?)?;+ api.set_steps(handle, request.steps)?;+ api.set_seed(handle, super::planning::generation_seed(request.seed)?)?;+ api.set_inference_backend(handle, platform.backend())?;+ if platform == ComputePlatform::Vulkan { api.set_vulkan_portable(handle)?; }+ api.set_attention(handle, "flash2")?;+ for (id, _, path) in &config.models {+ if (*id == 0 || config.model_type == "minimax_music3" && (*id == 1 || *id == 4)) && path.is_some() {+ api.set_model(handle, *id, path.as_ref().unwrap())?;+ }+ }+ Ok(())+}diff --git a/src-tauri/core/src/slopfab/planning.rs b/src-tauri/core/src/slopfab/planning.rsindex 81ae8f6..4e534f0 100644--- a/src-tauri/core/src/slopfab/planning.rs+++ b/src-tauri/core/src/slopfab/planning.rs@@ -40,12 +40,12 @@ pub fn resolve_plan()?;let plan = api.resolve(&handle)?;let description = api.describe(&handle)?;- let (_, frames) = scene_frame_window(request, plan.aligned_frames)?;+ let frames = if request.music.is_some() { 0 } else { scene_frame_window(request, plan.aligned_frames)?.1 };Ok(ResolvedPlan {canvas_width: if request.latent_upscale { request.canvas_width } else { plan.canvas_width },canvas_height: if request.latent_upscale { request.canvas_height } else { plan.canvas_height },aligned_frames: frames,- duration_seconds: plan.duration_seconds * frames as f64 / plan.aligned_frames as f64,+ duration_seconds: if request.music.is_some() { plan.duration_seconds } else { plan.duration_seconds * frames as f64 / plan.aligned_frames as f64 },model_evaluations: plan.num_model_evaluations * request.image_edit.as_ref().map_or(1, |edit| edit.edits.len() as i32),sequence_rows_without_text: plan.sequence_rows_without_text,latent_frames: plan.latent_frames,@@ -57,6 +57,7 @@ pub fn resolve_plan(}pub(super) fn validate_generation_controls(request: &GenerationRequest) -> Result<(), String> {+ if let Some(music) = &request.music { return music.validate(); }if let Some(scale) = request.guidance_scale { crate::models::validate_cfg_scale(f64::from(scale))?; }if let Some(effects) = &request.generation_effects { effects.validate()?; }super::continuation::validate(request)?;diff --git a/src-tauri/core/src/slopfab/status.rs b/src-tauri/core/src/slopfab/status.rsindex aece519..c9f68ef 100644--- a/src-tauri/core/src/slopfab/status.rs+++ b/src-tauri/core/src/slopfab/status.rs@@ -6,7 +6,11 @@ fn model_statuses(configuration: &Configuration, available: impl Fn(&Path) -> bolet mut models = configuration.models.iter()- .filter(|(id, _, _)| crate::models::generator_features(&configuration.model_type).video_generation || *id != 4)+ .filter(|(id, _, _)| match configuration.model_type.as_str() {+ "yue2" => *id == 0,+ "minimax_music3" => matches!(id, 0 | 1 | 4),+ _ => crate::models::generator_features(&configuration.model_type).video_generation || *id != 4,+ }).filter(|(id, _, _)| !configuration.animate || (*id != 1 && *id != 2)).map(|(_, id, path)| ModelStatus {id,diff --git a/src-tauri/core/src/slopfab/tests.rs b/src-tauri/core/src/slopfab/tests.rsindex bbf00a5..a918042 100644--- a/src-tauri/core/src/slopfab/tests.rs+++ b/src-tauri/core/src/slopfab/tests.rs@@ -5,6 +5,31 @@ use super::{use crate::settings::{ProviderOption, ProviderSetting};use std::collections::BTreeMap;+#[test]+fn music_plans_on_cuda_and_vulkan_without_video_controls() {+ let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../lib/slopfab").join(super::DLL_FILE_NAME);+ let api = ffi::Api::load(&path).unwrap();+ for family in ["yue2", "minimax_music3"] {+ let mut configuration = Configuration::from_settings(&BTreeMap::new());+ configuration.model_type = family.into();+ let mut request = GenerationRequest { prompt: "Quiet piano".into(), steps: 2, seed: 42,+ music: Some(music::MusicSettings { duration: 2.0, ..Default::default() }), ..Default::default() };+ for backend in [ComputePlatform::Cuda13, ComputePlatform::Vulkan] {+ let handle = RequestHandle::new(&api).unwrap();+ configure_request(&api, &handle, &request, &configuration, backend, RequestPurpose::Plan, &ReferenceVideos::default()).unwrap();+ let plan = api.resolve(&handle).unwrap();+ assert!(plan.duration_seconds >= 2.0);+ assert_eq!(plan.canvas_width, 0);+ assert_eq!(plan.canvas_height, 0);+ }+ request.reference_paths.push("reference.wav".into());+ assert!(configuration.validate_inputs(&request).unwrap_err().contains("references"));+ request.reference_paths.clear();+ request.music.as_mut().unwrap().duration = f64::NAN;+ assert!(configuration.validate_inputs(&request).is_err());+ }+}+#[test]fn moss_capabilities_reject_video_edits_and_references() {let mut configuration = Configuration::from_settings(&BTreeMap::new());diff --git a/src-tauri/core/src/slopfab/types.rs b/src-tauri/core/src/slopfab/types.rsindex 7fe874e..c237cf6 100644--- a/src-tauri/core/src/slopfab/types.rs+++ b/src-tauri/core/src/slopfab/types.rs@@ -54,6 +54,8 @@ pub struct LatentBridgeRequest {#[derive(Debug, Clone, Default, Deserialize, Serialize)]#[serde(rename_all = "camelCase")]pub struct GenerationRequest {+ #[serde(default, skip_serializing_if = "Option::is_none")]+ pub music: Option<super::music::MusicSettings>,#[serde(default, skip_serializing_if = "Option::is_none")]pub negative_prompt: Option<String>,#[serde(default, skip_serializing_if = "Option::is_none")]diff --git a/src-tauri/core/src/worker/protocol.rs b/src-tauri/core/src/worker/protocol.rsindex f08e48c..c59eaca 100644--- a/src-tauri/core/src/worker/protocol.rs+++ b/src-tauri/core/src/worker/protocol.rs@@ -23,7 +23,7 @@ pub const SERVICE_TYPE: &str = "_slopus-worker._tcp.local.";// v6 added custom upscale weights.// v5 added overlap locking.// v4 added SeedVR2 export streams. v3 added inverted image masks.-pub const PROTOCOL_VERSION: u32 = 17;+pub const PROTOCOL_VERSION: u32 = 18;/// Streaming SeedVR2 restoration; model paths are resolved on the worker.#[derive(Debug, Clone, Serialize, Deserialize)]diff --git a/src-tauri/core/src/worker/server.rs b/src-tauri/core/src/worker/server.rsindex 669d8a8..ee80540 100644--- a/src-tauri/core/src/worker/server.rs+++ b/src-tauri/core/src/worker/server.rs@@ -661,7 +661,7 @@ impl Worker {}self.prune_jobs();let (mut request, settings) = self.materialize(start.job)?;- let latents = if start.save_latents && !request.still_image {+ let latents = if start.save_latents && !request.still_image && request.music.is_none() {let directory = self.data.join("jobs").join(id);let _ = fs::remove_dir_all(&directory);fs::create_dir_all(&directory).map_err(|error| error.to_string())?;diff --git a/src-tauri/src/app.rs b/src-tauri/src/app.rsindex 7e3dc1e..a2216e9 100644--- a/src-tauri/src/app.rs+++ b/src-tauri/src/app.rs@@ -179,6 +179,7 @@ pub fn run() {.plugin(tauri_plugin_process::init()).invoke_handler(tauri::generate_handler![commands::artifacts::save_generated_image,+ commands::artifacts::save_generated_music,commands::character_sheet::save_character_sheet_view,commands::character_sheet::combine_character_sheet,commands::extend_image::prepare_extend_image,diff --git a/src-tauri/src/commands/artifacts.rs b/src-tauri/src/commands/artifacts.rsindex 68bcdd6..db6a915 100644--- a/src-tauri/src/commands/artifacts.rs+++ b/src-tauri/src/commands/artifacts.rs@@ -146,6 +146,62 @@ pub(crate) fn generated_summary(job_id: String) -> Option<rendered::RenderedSummrendered::summary(&job_id)}+/// Save audio-only generations as PCM WAV without a media subprocess.+#[tauri::command]+pub(crate) async fn save_generated_music(folder_path: String, job_id: String) -> Result<serde_json::Value, String> {+ tauri::async_runtime::spawn_blocking(move || {+ let summary = rendered::summary(&job_id).ok_or("Generated music is no longer available.")?;+ let samples = rendered::audio(&job_id).ok_or("Generated music has no audio.")?;+ let wav = music_wav(&samples, summary.audio_channels, summary.audio_sample_rate)?;+ let root = ProjectRoot::open(&folder_path)?;+ let stem = generated_file_stem(&job_id)?;+ let relative_path = format!("media/generated/{stem}.wav");+ crate::storage::atomic::write_atomically(&root.directory("media/generated")?.join(format!("{stem}.wav")), &wav)?;+ Ok(serde_json::json!({ "relativePath": relative_path,+ "durationMs": (samples.len() as f64 / 4.0 / f64::from(summary.audio_channels) / f64::from(summary.audio_sample_rate) * 1000.0).round() as u64 }))+ }).await.map_err(|e| e.to_string())?+}++fn music_wav(samples: &[u8], channels: u32, rate: u32) -> Result<Vec<u8>, String> {+ if !(1..=2).contains(&channels) || !(8000..=192000).contains(&rate) || samples.is_empty()+ || samples.len() % (4 * channels as usize) != 0 { return Err("Invalid generated audio format.".into()); }+ let size = u32::try_from(samples.len() / 4 * 3).map_err(|_| "Generated audio is too large.")?;+ let riff_size = size.checked_add(36).ok_or("Generated audio is too large.")?;+ let mut wav = Vec::with_capacity(size as usize + 44);+ wav.extend_from_slice(b"RIFF"); wav.extend_from_slice(&riff_size.to_le_bytes()); wav.extend_from_slice(b"WAVEfmt ");+ wav.extend_from_slice(&16_u32.to_le_bytes()); wav.extend_from_slice(&1_u16.to_le_bytes());+ wav.extend_from_slice(&(channels as u16).to_le_bytes()); wav.extend_from_slice(&rate.to_le_bytes());+ wav.extend_from_slice(&(rate * channels * 3).to_le_bytes()); wav.extend_from_slice(&((channels * 3) as u16).to_le_bytes());+ wav.extend_from_slice(&24_u16.to_le_bytes()); wav.extend_from_slice(b"data"); wav.extend_from_slice(&size.to_le_bytes());+ for bytes in samples.chunks_exact(4) {+ let sample = f32::from_le_bytes(bytes.try_into().unwrap());+ if !sample.is_finite() { return Err("Generated audio contains non-finite samples.".into()); }+ let pcm = (sample.clamp(-1.0, 1.0) * 8_388_607.0).round() as i32;+ wav.extend_from_slice(&pcm.to_le_bytes()[..3]);+ }+ Ok(wav)+}++#[cfg(test)]+mod music_tests {+ use super::music_wav;+ #[test]+ fn music_wav_encodes_stereo_pcm_and_rejects_invalid_samples() {+ let samples: Vec<u8> = [0_f32, 1.0, -1.0, 0.5].into_iter().flat_map(f32::to_le_bytes).collect();+ let wav = music_wav(&samples, 2, 48000).unwrap();+ assert_eq!(&wav[..4], b"RIFF");+ assert_eq!(&wav[8..16], b"WAVEfmt ");+ assert_eq!(u16::from_le_bytes(wav[22..24].try_into().unwrap()), 2);+ assert_eq!(u32::from_le_bytes(wav[24..28].try_into().unwrap()), 48000);+ assert_eq!(u16::from_le_bytes(wav[34..36].try_into().unwrap()), 24);+ assert_eq!(&wav[44..50], &[0, 0, 0, 255, 255, 127]);+ assert_eq!(wav.len(), 56);+ assert!(music_wav(&[0; 3], 2, 48000).is_err());+ assert!(music_wav(&f32::NAN.to_le_bytes(), 1, 48000).is_err());+ assert!(music_wav(&samples, 0, 48000).is_err());+ }+}+/// Save only a true 256x256 still-image result, directly from the DLL's frame/// buffer. Icon artwork is stored separately from conditioning attachments.#[tauri::command]diff --git a/src-tauri/src/commands/generation.rs b/src-tauri/src/commands/generation.rsindex 278f707..0da080f 100644--- a/src-tauri/src/commands/generation.rs+++ b/src-tauri/src/commands/generation.rs@@ -249,7 +249,7 @@ pub(crate) async fn enqueue_slopfab_generation(blocking(move || {prepare_continuation_path(&mut request, Some(&folder_path))?;prepare_image_edit_path(&mut request, Some(&folder_path))?;- if !request.still_image {+ if !request.still_image && request.music.is_none() {request.save_latents_path = Some(generated_latent_destination(&folder_path, &request.job_id)?);}let job_id = request.job_id.clone();diff --git a/src-tauri/src/project/mod.rs b/src-tauri/src/project/mod.rsindex 462d892..1324ff1 100644--- a/src-tauri/src/project/mod.rs+++ b/src-tauri/src/project/mod.rs@@ -17,3 +17,4 @@ pub(crate) mod commands;pub(crate) mod reference_tokens;pub(crate) mod motion;+pub(crate) mod music;diff --git a/src-tauri/src/project/model.rs b/src-tauri/src/project/model.rsindex 0489d04..c610c36 100644--- a/src-tauri/src/project/model.rs+++ b/src-tauri/src/project/model.rs@@ -18,6 +18,8 @@ pub(crate) enum GenerationType {#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]#[serde(rename_all = "camelCase")]pub(crate) struct ProjectConfig {+ #[serde(default, skip_serializing_if = "Vec::is_empty")]+ pub(crate) music_songs: Vec<super::music::MusicSong>,pub(crate) schema_version: u32,#[serde(default)]pub(crate) generation_type: GenerationType,diff --git a/src-tauri/src/project/music.rs b/src-tauri/src/project/music.rsnew file mode 100644index 0000000..2d0e459--- /dev/null+++ b/src-tauri/src/project/music.rs@@ -0,0 +1,31 @@+use serde::{Deserialize, Serialize};+use slopus_core::slopfab::music::MusicSettings;++#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub(crate) struct MusicSong {+ pub id: String,+ pub title: String,+ #[serde(default)] pub style: String,+ pub generator_id: String,+ pub settings: MusicSettings,+ pub steps: i32,+ pub seed: i64,+ pub status: String,+ pub asset_id: Option<String>,+ pub error: Option<String>,+ pub created_at: String,+}++pub(crate) fn validate(config: &super::ProjectConfig) -> Result<(), String> {+ let mut ids = std::collections::HashSet::new();+ for song in &config.music_songs {+ if song.id.is_empty() || !ids.insert(&song.id) || song.title.trim().is_empty() || song.title.chars().count() > 200+ || song.style.chars().count() > 10000 || !(2..=100).contains(&song.steps) || song.seed < -1+ || !matches!(song.status.as_str(), "draft" | "queued" | "generating" | "completed" | "failed" | "cancelled") {+ return Err("Invalid music song.".into());+ }+ song.settings.validate()?;+ }+ Ok(())+}diff --git a/src-tauri/src/project/validation/mod.rs b/src-tauri/src/project/validation/mod.rsindex 22e325b..33bfd65 100644--- a/src-tauri/src/project/validation/mod.rs+++ b/src-tauri/src/project/validation/mod.rs@@ -21,6 +21,7 @@ pub(crate) fn validate_and_normalize_config(pub(crate) fn validate_checked(mut config: ProjectConfig,) -> Result<ProjectConfig, issue::ValidationIssue> {+ super::music::validate(&config).map_err(|message| issue::ValidationIssue::new("project.invalid", Some(config.id.clone()), "musicSongs", message))?;if let Some(scene) = &config.image_scene {scene.validate().map_err(|message| {issue::ValidationIssue::new(diff --git a/src-tauri/src/tests/schema.rs b/src-tauri/src/tests/schema.rsindex 218831a..48dc651 100644--- a/src-tauri/src/tests/schema.rs+++ b/src-tauri/src/tests/schema.rs@@ -1,5 +1,22 @@use super::*;+#[test]+fn music_settings_survive_project_validation_and_reopen() {+ let mut config = created_fixture();+ config.music_songs.push(serde_json::from_value(serde_json::json!({+ "id": "song-1", "title": "Evening", "style": "Quiet jazz", "generatorId": "yue2",+ "settings": { "lyrics": "[Verse]\nHome again", "duration": 30 }, "steps": 32, "seed": -1,+ "status": "draft", "createdAt": "2026-10-25T00:00:00Z"+ })).unwrap());+ let saved = validate_and_normalize_config(config).unwrap();+ let reopened: ProjectConfig = serde_json::from_slice(&serde_json::to_vec(&saved).unwrap()).unwrap();+ assert_eq!(reopened.music_songs, saved.music_songs);+ assert_eq!(reopened.music_songs[0].settings.lyrics, "[Verse]\nHome again");+ let mut invalid = reopened;+ invalid.music_songs[0].steps = 1000;+ assert!(validate_and_normalize_config(invalid).is_err());+}+#[test]fn rust_never_emits_a_null_the_frontend_schema_refuses() {// The cross-layer direction that went unchecked for nine rounds: Rustdiff --git a/src/App.tsx b/src/App.tsxindex f14e791..98eb7c7 100644--- a/src/App.tsx+++ b/src/App.tsx@@ -230,6 +230,8 @@ function App() {if (item.kind === "image") {const imageId = [item.imageDraftId, item.id, item.imageAssetId].find((id) => record.config.assets.some((asset) => asset.kind === "image" && asset.id === id));navigation = { view: "editor", imageId: imageId ?? undefined };+ } else if (item.kind === "music") {+ navigation = { view: "music", songId: item.sceneId };} else if (item.kind === "refmod" || item.kind === "reference-icons") {navigation = { view: "references", referenceId: item.sceneId };} else navigation = { view: "generator", sceneId: item.sceneId };diff --git a/src/components/ImageProject.integration.test.tsx b/src/components/ImageProject.integration.test.tsxindex e6734ff..a651a56 100644--- a/src/components/ImageProject.integration.test.tsx+++ b/src/components/ImageProject.integration.test.tsx@@ -312,11 +312,11 @@ it("creates one project without a separate image or video type", async () => {expect(submit.mock.calls[0][0]).not.toHaveProperty("generationType");});-it("opens a legacy image project on Image with all five tabs and saves composition edits", async () => {+it("opens a legacy image project on Image with all six tabs and saves composition edits", async () => {const save = vi.fn(async () => undefined);render(<ProjectWorkspace project={record()} onBack={vi.fn()} onSave={save} />);const navigation = within(screen.getByRole("tablist", { name: "Project views" }));- expect(navigation.getAllByRole("tab").map((tab) => tab.textContent)).toEqual(["Timeline", "Video", "Image", "References", "Export"]);+ expect(navigation.getAllByRole("tab").map((tab) => tab.textContent)).toEqual(["Timeline", "Video", "Image", "Music", "References", "Export"]);expect(navigation.getByRole("tab", { name: "Image" })).toHaveAttribute("aria-selected", "true");fireEvent.contextMenu(screen.getByRole("button", { name: "Image" }));fireEvent.click(screen.getByRole("menuitem", { name: "New Text" }));diff --git a/src/components/ProjectWorkspace.test.ts b/src/components/ProjectWorkspace.test.tsindex 524893d..7ab00e7 100644--- a/src/components/ProjectWorkspace.test.ts+++ b/src/components/ProjectWorkspace.test.ts@@ -128,7 +128,7 @@ describe("project workspace timecode", () => {// The views are tabs in the title bar; the agent is not one of them.const tabs = screen.getByRole("tablist", { name: "Project views" });- expect(within(tabs).getAllByRole("tab").map((tab) => tab.textContent?.trim())).toEqual(["Timeline", "Video", "Image", "References", "Export"]);+ expect(within(tabs).getAllByRole("tab").map((tab) => tab.textContent?.trim())).toEqual(["Timeline", "Video", "Image", "Music", "References", "Export"]);expect(within(tabs).getByRole("tab", { name: "Timeline" })).toHaveAttribute("aria-selected", "true");const panel = document.getElementById(within(tabs).getByRole("tab", { name: "Timeline" }).getAttribute("aria-controls")!);expect(panel).toHaveAttribute("role", "tabpanel");diff --git a/src/components/ProjectWorkspace.tsx b/src/components/ProjectWorkspace.tsxindex fc6a90e..cd73002 100644--- a/src/components/ProjectWorkspace.tsx+++ b/src/components/ProjectWorkspace.tsx@@ -1,4 +1,6 @@import { Agent16, Agent16Filled, Back16, ChevronDown12, Download16, Download16Filled, Film16, Film16Filled, Image16, Image16Filled, Redo16, References16, References16Filled, Save16, Sparkle16, Sparkle16Filled, Undo16 } from "./ui/icons";+import { MusicView } from "./workspace/MusicView";+import { Audio16 } from "./ui/icons";import { ImageEditor } from "./workspace/ImageEditor";import { useEffect, useId, useMemo, useRef, useState, useSyncExternalStore, type ReactNode } from "react";import type { GenerationJob, ProjectConfig, ProjectRecord } from "../lib/project";@@ -24,9 +26,9 @@ import { outputDimensions, videoDurationMs } from "../lib/export";/** "agent" is not a view any more — the agent is a docked pane — but opening* a project "on the agent" still means something: the pane opens with it. */-export type ProjectView = "timeline" | "generator" | "references" | "agent" | "export" | "editor";+export type ProjectView = "music" | "timeline" | "generator" | "references" | "agent" | "export" | "editor";type ShownView = Exclude<ProjectView, "agent">;-export interface WorkspaceNavigation { view: ShownView; sceneId?: string; imageId?: string; referenceId?: string; exportKind?: "video" | "image" }+export interface WorkspaceNavigation { view: ShownView; sceneId?: string; imageId?: string; songId?: string; referenceId?: string; exportKind?: "video" | "image" }/** The scene badge includes native generation and the final encoding step. */export const isGenerationOngoing = (job: GenerationJob) =>@@ -67,7 +69,7 @@ const writeAgentPane = (open: boolean) => {};const VIEW_LABELS: Record<ShownView, string> = {- timeline: "Timeline", generator: "Video", references: "References", export: "Export", editor: "Image",+ music: "Music", timeline: "Timeline", generator: "Video", references: "References", export: "Export", editor: "Image",};/* The project window.@@ -144,6 +146,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,const [savingProjectSettings, setSavingProjectSettings] = useState(false);const [selectedGenerationJobId, setSelectedGenerationJobId] = useState<string | undefined>(project.config.generationJobs[0]?.id);const [imageNavigation, setImageNavigation] = useState<{ id: string }>();+ const [musicNavigation, setMusicNavigation] = useState<{ id: string }>();const [referenceNavigation, setReferenceNavigation] = useState<{ id: string }>();useEffect(() => {if (!navigation) return;@@ -151,6 +154,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,if (navigation.view === "export") setMediaMode(navigation.exportKind ?? "video");if (navigation.sceneId) setSelectedGenerationJobId(navigation.sceneId);if (navigation.imageId) setImageNavigation({ id: navigation.imageId });+ if (navigation.songId) setMusicNavigation({ id: navigation.songId });if (navigation.referenceId) setReferenceNavigation({ id: navigation.referenceId });onNavigated?.();}, [navigation]);@@ -194,7 +198,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,const lengthMs = useMemo(() => videoDurationMs(config), [config]);const sequenceFormat = `${frameWidth}×${frameHeight} · ${config.settings.frameRate} fps · ${formatSequenceLength(lengthMs)}`;- const views: ShownView[] = ["timeline", "generator", "editor", "references", "export"];+ const views: ShownView[] = ["timeline", "generator", "editor", "music", "references", "export"];const running = config.generationJobs.filter(isGenerationOngoing).length;const toggleAgent = () => {const open = !agentOpen;@@ -208,7 +212,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,useShortcut("Ctrl+S", () => { if (dirty && !saving) void save(); }, { enabled: idle, allowInInput: true });useShortcut("Ctrl+Z", () => { session.undo(); }, { enabled: idle });useShortcut(["Ctrl+Y", "Ctrl+Shift+Z"], () => { session.redo(); }, { enabled: idle });- useShortcut(["Ctrl+1", "Ctrl+2", "Ctrl+3", "Ctrl+4", "Ctrl+5"], (event) => {+ useShortcut(["Ctrl+1", "Ctrl+2", "Ctrl+3", "Ctrl+4", "Ctrl+5", "Ctrl+6"], (event) => {const next = views[Number(event.code?.replace("Digit", "") || event.key) - 1];if (!next) return false;setView(next);@@ -217,7 +221,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,useShortcut("Ctrl+Shift+A", toggleAgent, { enabled: idle, allowInInput: true });useShortcut("Alt+ArrowLeft", goBack, { enabled: idle });- const viewIcon = (id: ShownView, selected: boolean) => id === "timeline" ? (selected ? <Film16Filled /> : <Film16 />)+ const viewIcon = (id: ShownView, selected: boolean) => id === "music" ? <Audio16 /> : id === "timeline" ? (selected ? <Film16Filled /> : <Film16 />): id === "generator" ? (selected ? <Sparkle16Filled /> : <Sparkle16 />): id === "references" ? (selected ? <References16Filled /> : <References16 />): id === "export" ? (selected ? <Download16Filled /> : <Download16 />)@@ -286,6 +290,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,<ProjectStatusSlot.Provider value={statusSlot}><div className={`project-body${agentOpen ? " project-body--agent" : ""}`} style={agentPane.style}><div id={`${panelId}-view`} role="tabpanel" aria-label={VIEW_LABELS[view]} className={`project-content project-content--${view}`}>+ {view === "music" && <MusicView openSong={musicNavigation} onSongOpened={() => setMusicNavigation(undefined)} config={config} folderPath={project.folderPath} onChange={changeConfig} onGenerate={(id, template) => queue.enqueueMusic(session, id, template)} onCancel={(id) => void queue.cancel(id)} workItems={projectItems.filter((item) => item.kind === "music")} />}{view === "editor" && <ImageEditor openImage={imageNavigation} onImageOpened={() => setImageNavigation(undefined)} config={config} folderPath={project.folderPath} onChange={changeConfig} onGenerate={(template, count) => queue.enqueueImage(session, template, count)} onGenerateCharacterSheet={(template, sourceId, options) => queue.enqueueCharacterSheet(session, template, sourceId, options)} onGenerateExtend={(template, sourceId, options) => queue.enqueueExtendImage(session, template, sourceId, options)} onCancel={(id) => queue.cancel(id)} workItems={projectItems.filter((item) => item.kind === "image")} onOpenGenerator={(jobId) => { setSelectedGenerationJobId(jobId); setView("generator"); }} onOpenReferences={() => setView("references")} />}{view === "timeline" && <TimelineView config={config} openSceneId={selectedGenerationJobId} folderPath={project.folderPath} generationCompletionTimes={generationCompletionTimes} onChange={changeConfig} onMeasured={recordMeasurement} onOpenGenerator={(jobId) => { setSelectedGenerationJobId(jobId); setView("generator"); }} />}{view === "generator" && <GeneratorView onGenerate={(submissions) => queue.enqueue(session, submissions)} onCancelGeneration={(ids) => queue.cancelScenes(session, ids)} cancellingJobIds={cancellingJobIds} config={config} folderPath={project.folderPath} generationCompletionTimes={generationCompletionTimes} runtime={runtime?.slopfab ?? null} onRuntimeChange={onGeneratorRuntimeChange} onChange={changeConfig} selectedJobId={selectedGenerationJobId} onSelectedJobChange={setSelectedGenerationJobId} onOpenTimeline={() => setView("timeline")} />}@@ -303,7 +308,7 @@ export function ProjectWorkspace({ project, initialView, runtime = null, onBack,<aside ref={agentPaneRef} id={`${panelId}-agent`} className="project-agent-pane" aria-label="Agent" hidden={!agentOpen}><AgentDockmode={mediaMode}- context={view === "editor" ? "this image composition" : view === "timeline" ? "the edit" : view === "generator" ? "this generation queue" : view === "references" ? "project references" : "this export"}+ context={view === "music" ? "the music library" : view === "editor" ? "this image composition" : view === "timeline" ? "the edit" : view === "generator" ? "this generation queue" : view === "references" ? "project references" : "this export"}record={{ ...project, config }}providers={runtime?.providers ?? CHECKING_PROVIDERS}onClose={() => { setAgentOpen(false); agentToggle.current?.focus(); }}diff --git a/src/components/SettingsView.test.tsx b/src/components/SettingsView.test.tsxindex d7d7d5e..8c6b7c5 100644--- a/src/components/SettingsView.test.tsx+++ b/src/components/SettingsView.test.tsx@@ -77,7 +77,7 @@ describe("the settings screen", () => {saveLoras([{ id: "future-adapter", name: "MOSS adapter", modelType: "moss", path: "D:/moss-adapter.safetensors" }]);open(null);const models = screen.getByRole("list", { name: "Generator models" });- expect(within(models).getAllByRole("button").map((button) => button.getAttribute("aria-label"))).toEqual(["MiniMax H3", "MOSS", "Fibo 2.0", "Other"]);+ expect(within(models).getAllByRole("button").map((button) => button.getAttribute("aria-label"))).toEqual(["MiniMax H3", "MOSS", "Fibo 2.0", "YuE2", "MiniMax Music3", "Other"]);expect(screen.queryByRole("button", { name: "Edit Default generator" })).toBeNull();fireEvent.click(within(models).getByRole("button", { name: "MiniMax H3" }));expect(screen.getByRole("button", { name: "Edit Default generator" })).toBeTruthy();diff --git a/src/components/SettingsView.tsx b/src/components/SettingsView.tsxindex b5b6880..388b152 100644--- a/src/components/SettingsView.tsx+++ b/src/components/SettingsView.tsx@@ -8,7 +8,7 @@ import { useAgentPane } from "../lib/useAgentPane";import { useShortcut } from "../lib/commands";import { executeGeneratorCommands, type GeneratorCommand } from "../lib/agentGenerators";import {- Add20, Agent16, Agent16Filled, Agent20, Back16, Check14, Check16, Check20, ChevronRight20, Delete16, Diagnostics16,+ Add20, Agent16, Agent16Filled, Agent20, Audio22, Back16, Check14, Check16, Check20, ChevronRight20, Delete16, Diagnostics16,Diagnostics16Filled, Download16, Download20, Error16, Error20, Flash20, Folder20, Gauge20, Gpu20, Image20, ModelFile20, More16,Options20, Palette16, Palette16Filled, Palette20, Refresh16, Refresh16Filled, Rename20, Reset16, Sparkle20, Spinner16,Spinner20, Steps20, TextFile20, Video16, Video16Filled, Video20, Worker16, Worker16Filled,@@ -579,7 +579,7 @@ export function SettingsView({ onClose, updates, initialTab = "engine", titleBar<ComboBox aria-label="Generator model" value={selectedTemplate.modelType} options={[...GENERATOR_MODELS,...(!GENERATOR_MODELS.some(({ value }) => value === selectedTemplate.modelType) ? [{ value: selectedTemplate.modelType, label: selectedTemplate.modelType }] : []),- ]} onChange={(modelType) => updateTemplate({ modelType, editingModel: generatorModel(modelType).imageEditing, ...(modelType === MOSS_MODEL_TYPE || modelType === FIBO_MODEL_TYPE ? { mode: "prompt", attention: "flash2", defaultSteps: modelType === FIBO_MODEL_TYPE ? 4 : 50, motionCache: false, loras: [], audioSigmaShift: undefined, ...(modelType === FIBO_MODEL_TYPE ? { videoSigmaShift: undefined } : {}) } : {}) })} />+ ]} onChange={(modelType) => updateTemplate({ modelType, editingModel: generatorModel(modelType).imageEditing, ...(modelType === MOSS_MODEL_TYPE || modelType === FIBO_MODEL_TYPE || generatorModel(modelType).musicGeneration ? { mode: "prompt", attention: "flash2", defaultSteps: modelType === "yue2" ? 32 : modelType === "minimax_music3" ? 30 : modelType === FIBO_MODEL_TYPE ? 4 : 50, motionCache: false, loras: [], audioSigmaShift: undefined, ...(modelType === FIBO_MODEL_TYPE || generatorModel(modelType).musicGeneration ? { videoSigmaShift: undefined } : {}) } : {}) })} /></SettingsCard>{generatorModel(selectedTemplate.modelType).videoGeneration && <SettingsCard icon={<Video20 />} header="Mode"><ComboBox aria-label="Generator mode" value={selectedTemplate.mode ?? "prompt"} onChange={(value) => updateTemplate({ mode: value === "animate" ? "animate" : "prompt" })}@@ -615,7 +615,7 @@ export function SettingsView({ onClose, updates, initialTab = "engine", titleBar<SettingsGroup heading="Performance"><SettingsCard icon={<Gauge20 />} header="Attention" description={selectedTemplate.attention === "sol" ? "Sol attention requires a CUDA GPU on the machine running generation." : undefined}><ComboBox aria-label="Generator attention" value={selectedTemplate.attention} onChange={(value) => updateTemplate({ attention: value as AttentionMode })}- options={generatorModel(selectedTemplate.modelType).cudaOnly ? [{ value: "flash2", label: "Flash attention" }] : [{ value: "exact", label: "Exact attention" }, { value: "flash2", label: "Flash attention" }, { value: "sage2", label: "Sage attention" }, { value: "sol", label: "Sol attention" }]} />+ options={generatorModel(selectedTemplate.modelType).cudaOnly || generatorModel(selectedTemplate.modelType).musicGeneration ? [{ value: "flash2", label: "Flash attention" }] : [{ value: "exact", label: "Exact attention" }, { value: "flash2", label: "Flash attention" }, { value: "sage2", label: "Sage attention" }, { value: "sol", label: "Sol attention" }]} /></SettingsCard>{generatorModel(selectedTemplate.modelType).motionCache && <SettingsCard icon={<Flash20 />} header="Enable MotionCache" description={<span id="motion-cache-help">{selectedTemplate.mode === "animate" ? "Unavailable in Animate mode" : "Reuses similar denoising results to reduce computation. May affect detail and motion."}</span>}><ToggleSwitch aria-label="Enable MotionCache" aria-describedby="motion-cache-help"@@ -648,7 +648,7 @@ export function SettingsView({ onClose, updates, initialTab = "engine", titleBar<ReferenceIconSetting templates={templateSettings} /><div className="settings-card-list" role="list" aria-label="Generator models">{[...new Set([...GENERATOR_MODELS.map(({ value }) => value), ...templateSettings.templates.map(({ modelType }) => modelType)]), "other"].map((model) =>- <OpenableCard key={model} icon={model === "other" ? <ModelFile20 /> : !generatorModel(model).videoGeneration ? <Image20 /> : <Video20 />} header={modelPageLabel(model)}+ <OpenableCard key={model} icon={model === "other" ? <ModelFile20 /> : generatorModel(model).musicGeneration ? <Audio22 /> : !generatorModel(model).videoGeneration ? <Image20 /> : <Video20 />} header={modelPageLabel(model)}openLabel={modelPageLabel(model)} openRef={`model:${model}`} description={model === "other" ? "Other weights" : "Generators and LoRAs"} onOpen={() => setModelPage(model)} />)}</div></> : modelPage === "other" ? <OtherWeightsSettings desktop={desktop} /> : <>diff --git a/src/components/WorkQueuePanel.tsx b/src/components/WorkQueuePanel.tsxindex a27074f..32fa816 100644--- a/src/components/WorkQueuePanel.tsx+++ b/src/components/WorkQueuePanel.tsx@@ -14,7 +14,7 @@ import { Flyout, ProgressBar, ProgressRing } from "./ui";it runs, a 3px bar. The render settings a row was queued with sit in itstooltip rather than on a third line. */-const settingsLine = (item: WorkItem) => item.kind === "refmod" ? "Safetensors refmod" : `${item.settings.canvasWidth} × ${item.settings.canvasHeight} · ${item.kind === "reference-icons" ? "JPG icons" : item.kind === "image" ? "JPG image" : `${(item.settings.frames / GENERATION_FRAME_RATE).toFixed(1)}s`} · ${item.settings.steps} steps${item.settings.audioSteps !== undefined ? ` · ${item.settings.audioSteps} audio steps` : ""} · ${item.settings.seed === -1 ? "Random seed" : `Seed ${item.settings.seed}`}`;+const settingsLine = (item: WorkItem) => item.kind === "music" ? `Music · ${item.settings.steps} steps · Seed ${item.settings.seed}` : item.kind === "refmod" ? "Safetensors refmod" : `${item.settings.canvasWidth} × ${item.settings.canvasHeight} · ${item.kind === "reference-icons" ? "JPG icons" : item.kind === "image" ? "JPG image" : `${(item.settings.frames / GENERATION_FRAME_RATE).toFixed(1)}s`} · ${item.settings.steps} steps${item.settings.audioSteps !== undefined ? ` · ${item.settings.audioSteps} audio steps` : ""} · ${item.settings.seed === -1 ? "Random seed" : `Seed ${item.settings.seed}`}`;function StateGlyph({ item }: { item: WorkItem }) {if (item.status === "completed") return <Check16 />;diff --git a/src/components/workspace/MusicView.test.tsx b/src/components/workspace/MusicView.test.tsxnew file mode 100644index 0000000..cfa8713--- /dev/null+++ b/src/components/workspace/MusicView.test.tsx@@ -0,0 +1,59 @@+// @vitest-environment jsdom+import "@testing-library/jest-dom/vitest";+import { afterEach, beforeEach, expect, it, vi } from "vitest";+import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react";+import { useState } from "react";+import { MusicView } from "./MusicView";+import { createProjectConfig, type ProjectConfig } from "../../lib/project";+import { newSong } from "../../lib/music";+import { readMediaFileUrl } from "../../lib/persistence";+import { loadGeneratorTemplateSettings, saveGeneratorTemplateSettings } from "../../lib/settings";++vi.mock("../../lib/persistence", async (original) => ({ ...await original<typeof import("../../lib/persistence")>(), isTauri: () => true, readMediaFileUrl: vi.fn(async () => "blob:song") }));+let latest: ProjectConfig;+const generate = vi.fn();+const createConfig = () => createProjectConfig({ name: "Music", prompt: "", aspectRatio: "16:9", resolution: "416p", targetDurationSeconds: 30 });+function Harness({ initial = createConfig() }: { initial?: ProjectConfig }) {+ const [config, setConfig] = useState(initial); latest = config;+ return <MusicView config={config} folderPath="C:/Music" onChange={setConfig} onGenerate={generate} onCancel={() => {}} workItems={[]} />;+}+beforeEach(() => {+ vi.clearAllMocks(); localStorage.clear();+ const settings = loadGeneratorTemplateSettings();+ saveGeneratorTemplateSettings({ ...settings, templates: settings.templates.map((template) => template.id === "yue2" ? { ...template, paths: { ...template.paths, transformer: "C:/weights/yue2.safetensors" }, sources: {} } : template) });+ vi.spyOn(HTMLMediaElement.prototype, "pause").mockImplementation(() => {});+ vi.spyOn(HTMLMediaElement.prototype, "play").mockResolvedValue();+ vi.stubGlobal("URL", Object.assign(URL, { revokeObjectURL: vi.fn() }));+});+afterEach(() => { cleanup(); vi.restoreAllMocks(); vi.unstubAllGlobals(); });++it("adds songs, edits lyrics and style, switches layout and queues the selected song", () => {+ render(<Harness />);+ fireEvent.click(screen.getAllByRole("button", { name: "New song" })[0]);+ fireEvent.change(screen.getByLabelText("Song title"), { target: { value: "Night drive" } });+ fireEvent.change(screen.getByLabelText("Music style"), { target: { value: "Synth pop" } });+ fireEvent.change(screen.getByLabelText("Music lyrics"), { target: { value: "[Verse]\nCity lights" } });+ expect(latest.musicSongs![0]).toMatchObject({ title: "Night drive", style: "Synth pop", settings: { lyrics: "[Verse]\nCity lights" } });+ fireEvent.click(screen.getByRole("button", { name: "List view" }));+ expect(screen.getByRole("list", { name: "Songs" })).toHaveClass("music-songs--list");+ fireEvent.click(screen.getByRole("button", { name: "Generate music" }));+ expect(generate).toHaveBeenCalledWith(latest.musicSongs![0].id, expect.objectContaining({ modelType: "yue2" }));+});++it("plays generated audio and creates a reference for the context-clicked song", async () => {+ const first = { ...newSong(), title: "First", status: "completed" as const, assetId: "song-a" };+ const second = { ...newSong(), title: "Second", status: "completed" as const, assetId: "song-b" };+ const config = { ...createConfig(), musicSongs: [first, second], assets: [first, second].map((song) => ({ id: song.assetId, kind: "audio" as const, name: song.title, relativePath: `media/${song.assetId}.wav`, durationMs: 30000, mimeType: "audio/wav", createdAt: song.createdAt })) };+ const { container } = render(<Harness initial={config} />);+ fireEvent.click(screen.getByRole("button", { name: "Play First" }));+ await waitFor(() => expect(container.querySelector("audio")).toHaveAttribute("src", "blob:song"));+ fireEvent.canPlay(container.querySelector("audio")!);+ expect(HTMLMediaElement.prototype.play).toHaveBeenCalled();+ fireEvent.play(container.querySelector("audio")!);+ expect(screen.getByRole("button", { name: "Pause music" })).toBeEnabled();+ fireEvent.click(screen.getByRole("button", { name: "Select Second" }));+ expect(readMediaFileUrl).toHaveBeenCalledTimes(1);+ fireEvent.contextMenu(screen.getByRole("button", { name: "Select First" }));+ fireEvent.click(screen.getByRole("menuitem", { name: "Create Reference" }));+ expect(latest.references.at(-1)).toMatchObject({ kind: "audio", name: "First", relativePath: "media/song-a.wav", audio: { startSeconds: 0, durationSeconds: 15 } });+});diff --git a/src/components/workspace/MusicView.tsx b/src/components/workspace/MusicView.tsxnew file mode 100644index 0000000..349b0f4--- /dev/null+++ b/src/components/workspace/MusicView.tsx@@ -0,0 +1,197 @@+import { useEffect, useRef, useState } from "react";+import type { ProjectAsset, ProjectConfig } from "../../lib/project";+import { isTauri, readMediaFileUrl } from "../../lib/persistence";+import { loadGeneratorTemplateSettings, subscribeGeneratorTemplates, templateSupportsMusic, templateUsable, generatorPathFields, type GeneratorTemplate } from "../../lib/settings";+import { musicSettingsForModel, newSong, songReference, updateSong, type MusicSettings, type MusicSong } from "../../lib/music";+import { isWorkActive, type WorkItem } from "../../lib/workQueue";+import { ComboBox, PropRow, PropSection, Slider, Splitter, useContextMenu, usePaneSize } from "../ui";+import { Add16, Audio16, GoToEnd16, GoToStart16, GridView16, ListView16, Pause16, Play16, References16, Speaker14 } from "../ui/icons";+import { generatorOptions } from "../generatorOptions";+import { SliderNumberInput, GENERATION_STEPS_RANGE } from "./SliderNumberInput";+import type { ConfigUpdate } from "./TimelineView";++const time = (seconds: number) => `${Math.floor(seconds / 60)}:${Math.floor(seconds % 60).toString().padStart(2, "0")}`;++export function MusicView({ config, folderPath, onChange, onGenerate, onCancel, workItems, openSong, onSongOpened }: {+ openSong?: { id: string }; onSongOpened?: () => void;+ config: ProjectConfig; folderPath: string; onChange: (update: ConfigUpdate, key?: string) => void;+ onGenerate: (songId: string, template: GeneratorTemplate) => void; onCancel: (id: string) => void;+ workItems: readonly WorkItem[];+}) {+ const [templates, setTemplates] = useState(() => loadGeneratorTemplateSettings().templates.filter(templateSupportsMusic));+ useEffect(() => subscribeGeneratorTemplates(() => setTemplates(loadGeneratorTemplateSettings().templates.filter(templateSupportsMusic))), []);+ const [selectedId, setSelectedId] = useState(config.musicSongs?.[0]?.id);+ useEffect(() => { if (openSong) { setSelectedId(openSong.id); onSongOpened?.(); } }, [openSong]);+ const selected = (config.musicSongs ?? []).find((song) => song.id === selectedId) ?? config.musicSongs?.[0];+ const [newGeneratorId, setNewGeneratorId] = useState(templates[0]?.id ?? "");+ const template = templates.find((item) => item.id === (selected?.generatorId ?? newGeneratorId)) ?? templates[0];+ const [view, setView] = useState<"grid" | "list">(() => localStorage.getItem("slopus.music.view") === "list" ? "list" : "grid");+ const inspector = usePaneSize("music.inspector", 340, { min: 280, max: 560 });+ const menu = useContextMenu();+ const [error, setError] = useState<string | null>(null);+ const [notice, setNotice] = useState<string | null>(null);+ const activeWork = selected && workItems.find((item) => item.sceneId === selected.id && isWorkActive(item));+ const busy = Boolean(activeWork);+ const ready = template && templateUsable(template) && generatorPathFields(template).every((field) => !field.required || template.paths[field.id].trim() || template.sources?.[field.id]?.length);+ const edit = (patch: Partial<MusicSong>) => {+ if (selected && !busy) onChange((current) => updateSong(current, selected.id, patch), `music:${selected.id}`);+ };+ const setting = (patch: Partial<MusicSettings>) => { if (selected) edit({ settings: { ...selected.settings, ...patch } }); };+ const add = () => {+ const song = newSong(template?.id, Math.min(100, template?.defaultSteps ?? 32));+ onChange((current) => ({ ...current, musicSongs: [...(current.musicSongs ?? []), song] }));+ setSelectedId(song.id); setError(null);+ };+ const chooseGenerator = (id: string) => {+ const next = templates.find((item) => item.id === id);+ setNewGeneratorId(id);+ if (next && selected) edit({ generatorId: id, steps: Math.min(100, next.defaultSteps), settings: musicSettingsForModel(selected.settings, next.modelType) });+ };+ const generate = () => {+ if (!selected || !template) return;+ setError(null);+ try { onGenerate(selected.id, template); } catch (reason) { setError(String(reason)); }+ };++ const audio = useRef<HTMLAudioElement>(null);+ const [playerId, setPlayerId] = useState<string | null>(null);+ const playerAsset = config.assets.find((asset) => asset.id === playerId);+ const [url, setUrl] = useState<string | null>(null);+ const [playing, setPlaying] = useState(false);+ const [position, setPosition] = useState(0);+ const [length, setLength] = useState(0);+ const [volume, setVolume] = useState(0.8);+ const playWhenReady = useRef(false);+ const [playbackError, setPlaybackError] = useState<string | null>(null);+ const playable = (config.musicSongs ?? []).flatMap((song) => {+ const asset = config.assets.find((item) => item.id === song.assetId && item.kind === "audio" && item.relativePath);+ return asset ? [{ song, asset }] : [];+ });+ useEffect(() => {+ let disposed = false, owned: string | null = null;+ audio.current?.pause(); setPlaying(false); setPosition(0); setLength(0); setUrl(null); setPlaybackError(null);+ if (playerAsset) void readMediaFileUrl(folderPath, playerAsset, "audio/wav").then((value) => {+ if (disposed) { if (value) URL.revokeObjectURL(value); return; }+ owned = value; setUrl(value);+ if (!value) { playWhenReady.current = false; setPlaybackError("This song's audio file is unavailable."); }+ }).catch((reason) => { if (!disposed) { playWhenReady.current = false; setPlaybackError(String(reason)); } });+ return () => { disposed = true; if (owned) URL.revokeObjectURL(owned); };+ }, [folderPath, playerAsset?.id, playerAsset?.relativePath]);+ useEffect(() => { if (audio.current) audio.current.volume = volume; }, [volume, url]);+ const play = async () => {+ try { await audio.current?.play(); } catch (reason) { setPlaybackError(`Could not play song: ${String(reason)}`); setPlaying(false); }+ };+ const playAsset = (asset: ProjectAsset) => {+ setPlaybackError(null);+ if (asset.id === playerId && url) void play();+ else { playWhenReady.current = true; setPlayerId(asset.id); }+ };+ const move = (direction: number) => {+ if (!playable.length) return;+ const index = playable.findIndex(({ asset }) => asset.id === playerId);+ playAsset(playable[(Math.max(0, index) + direction + playable.length) % playable.length].asset);+ };+ const toggle = () => {+ if (playing) { audio.current?.pause(); return; }+ const asset = playerAsset ?? playable.find(({ song }) => song.id === selected?.id)?.asset ?? playable[0]?.asset;+ if (asset) playAsset(asset);+ };+ const context = (event: React.MouseEvent<HTMLElement> | React.KeyboardEvent<HTMLElement>, song: MusicSong) => {+ const asset = config.assets.find((item) => item.id === song.assetId);+ menu.open(event, [{ label: "Create Reference", icon: <References16 />, disabled: !asset?.relativePath || (asset.durationMs ?? 0) < 2000,+ onSelect: () => {+ try {+ const reference = songReference(song, asset!);+ onChange((current) => ({ ...current, references: [...current.references, reference] }));+ setNotice(`Added ${song.title} to References.`);+ } catch (reason) { setError(String(reason)); }+ } }], { "aria-label": `${song.title} actions` });+ };++ return <div className="music-view" style={inspector.style}>+ <div className="music-toolbar" role="toolbar" aria-label="Music tools">+ <ComboBox aria-label="Music generator" value={template?.id ?? ""} options={generatorOptions(templates)} onChange={chooseGenerator} disabled={busy} />+ <button type="button" className="secondary-button" onClick={add}><Add16 />New song</button>+ <div className="music-toolbar__views" role="group" aria-label="Song view">+ {(["grid", "list"] as const).map((mode) => <button type="button" className="icon-button" key={mode} aria-label={`${mode === "grid" ? "Grid" : "List"} view`} aria-pressed={view === mode}+ onClick={() => { setView(mode); localStorage.setItem("slopus.music.view", mode); }}>{mode === "grid" ? <GridView16 /> : <ListView16 />}</button>)}+ </div>+ </div>+ <div className="music-body">+ <div className="music-library">+ <div className="music-library__heading"><h2>Your music</h2><span>{(config.musicSongs?.length ?? 0)} songs</span></div>+ {notice && <p role="status">{notice}</p>}+ {error && <p role="alert">{error}</p>}+ {!(config.musicSongs?.length ?? 0) ? <div className="music-empty"><Audio16 /><h3>Create your first song</h3><p>Add a song, describe its style and write lyrics, or leave lyrics empty for an instrumental.</p><button type="button" className="primary-button" onClick={add}><Add16 />New song</button></div> :+ <div className={`music-songs music-songs--${view}`} role="list" aria-label="Songs">+ {(config.musicSongs ?? []).map((song, index) => {+ const asset = config.assets.find((item) => item.id === song.assetId);+ const work = workItems.find((item) => item.sceneId === song.id && isWorkActive(item));+ return <div className={`music-song${selected?.id === song.id ? " music-song--selected" : ""}`} role="listitem" key={song.id}+ onContextMenu={(event) => context(event, song)}>+ <button type="button" className="music-song__select" aria-label={`Select ${song.title}`} aria-pressed={selected?.id === song.id}+ onClick={() => setSelectedId(song.id)} onDoubleClick={() => { if (asset) playAsset(asset); }}+ onKeyDown={(event) => { if (event.key === "ContextMenu" || event.shiftKey && event.key === "F10") context(event, song); }}>+ <span className="music-song__art" aria-hidden="true"><Audio16 /><span>{String(index + 1).padStart(2, "0")}</span></span>+ <span className="music-song__info"><strong>{song.title}</strong><span>{song.style || "Add a style to get started"}</span></span>+ <span className="music-song__status">{work ? `${work.detail} · ${Math.round(work.progress * 100)}%` : song.status === "completed" && asset ? time((asset.durationMs ?? 0) / 1000) : song.status}</span>+ </button>+ <button type="button" className="icon-button music-song__play" aria-label={`Play ${song.title}`} disabled={!asset?.relativePath} onClick={() => asset && playAsset(asset)}><Play16 /></button>+ </div>;+ })}+ </div>}+ </div>+ <Splitter {...inspector.splitterProps} reverse aria-label="Resize music inspector" aria-controls="music-inspector" />+ <aside id="music-inspector" className="music-inspector" aria-label="Song settings">+ {selected ? <>+ <div className="music-inspector__fields">+ <PropSection title="Song">+ <PropRow label="Title"><input className="text-field" aria-label="Song title" value={selected.title} maxLength={200} disabled={busy} onChange={(event) => edit({ title: event.target.value || "New song" })} /></PropRow>+ <label className="music-prompt">Style<textarea className="text-field" aria-label="Music style" value={selected.style} maxLength={10000} rows={3} disabled={busy} placeholder="Genre, mood, instruments and vocals…" onChange={(event) => edit({ style: event.target.value })} /></label>+ <label className="music-prompt">Lyrics<textarea className="text-field music-lyrics" aria-label="Music lyrics" value={selected.settings.lyrics} maxLength={30000} rows={9} disabled={busy} placeholder="[Verse]… Leave empty for an instrumental." onChange={(event) => setting({ lyrics: event.target.value })} /></label>+ </PropSection>+ <PropSection title="Generation" persistKey="music.generation">+ <PropRow label="Duration"><SliderNumberInput aria-label="Song duration" minimum={2} maximum={300} step={1} suffix="s" value={selected.settings.duration} disabled={busy} onCommit={(duration) => setting({ duration })} /></PropRow>+ <PropRow label="Steps"><SliderNumberInput aria-label="Music steps" {...GENERATION_STEPS_RANGE} value={selected.steps} disabled={busy} onCommit={(steps) => edit({ steps })} /></PropRow>+ <PropRow label="Seed" tooltip="Use -1 for a random seed."><input className="text-field" aria-label="Music seed" type="number" min={-1} max={Number.MAX_SAFE_INTEGER} step={1} value={selected.seed} disabled={busy} onChange={(event) => { const seed = event.target.valueAsNumber; if (Number.isSafeInteger(seed) && seed >= -1) edit({ seed }); }} /></PropRow>+ </PropSection>+ <PropSection title="Advanced" persistKey="music.advanced" defaultOpen={false}>+ {template?.modelType === "yue2" && <PropRow label="Planning"><ComboBox aria-label="Music planning" value={selected.settings.planning} disabled={busy} options={[{ value: "full", label: "Full (melody and chords)" }, { value: "melody", label: "Melody" }, { value: "off", label: "Off" }]} onChange={(planning) => setting({ planning: planning as MusicSettings["planning"], ...(planning === "off" ? { abc: "" } : {}) })} /></PropRow>}+ {([+ ["top_k", "Top K", 0, 1024, 1], ["semantic_guidance", "Guidance", 0, 5, 1],+ ...(template?.modelType === "yue2" ? [["temperature", "Temperature", 0, 2, 0.05], ["top_p", "Top P", 0.05, 1, 0.05], ["repetition_penalty", "Repetition penalty", 1, 2, 0.05]] : [["flow_guidance", "Flow guidance", 0, 5, 0.1]]),+ ] as [keyof MusicSettings, string, number, number, number][]).map(([key, label, minimum, maximum, step]) => <PropRow key={key} label={label} tooltip={key === "top_k" || key.includes("guidance") ? "Zero uses the model default." : undefined}>+ <SliderNumberInput aria-label={label} minimum={minimum} maximum={maximum} step={step} integer={key === "top_k"} value={Number(selected.settings[key])} disabled={busy} onCommit={(value) => setting({ [key]: key === "semantic_guidance" && value > 0 && value < 1 ? 1 : value })} /></PropRow>)}+ {template?.modelType === "yue2" && selected.settings.planning !== "off" && <>+ <PropRow label="Planning tokens"><SliderNumberInput aria-label="Planning tokens" minimum={32} maximum={4096} step={32} integer value={selected.settings.abc_max_tokens} disabled={busy} onCommit={(abc_max_tokens) => setting({ abc_max_tokens })} /></PropRow>+ <label className="music-prompt">ABC (optional)<textarea className="text-field" aria-label="Music ABC" value={selected.settings.abc} maxLength={30000} rows={3} disabled={busy} onChange={(event) => setting({ abc: event.target.value })} /></label>+ </>}+ </PropSection>+ </div>+ <div className="music-inspector__actions">+ {selected.error && <p role="alert">{selected.error}</p>}+ {!ready && <p>Download or select music model weights in Settings → Generators.</p>}+ {!isTauri() && <p>Generate music in the desktop app.</p>}+ {activeWork ? <><progress value={activeWork.progress} max={1} aria-label="Music generation progress" /><button type="button" className="secondary-button inspector-action" disabled={activeWork.cancelling || activeWork.status === "encoding"} onClick={() => onCancel(activeWork.id)}>{activeWork.cancelling ? "Cancelling…" : "Cancel generation"}</button></> :+ <button type="button" className="primary-button inspector-action" disabled={!ready || !isTauri() || !selected.style.trim()} onClick={generate}>Generate music</button>}+ </div>+ </> : <p className="music-inspector__empty">Add a song to edit its lyrics, style and generation settings.</p>}+ </aside>+ </div>+ <div className="music-player" aria-label="Music player">+ <div className="music-player__now"><Audio16 /><div><strong>{playerAsset?.name ?? "No song playing"}</strong><span>{playing ? "Playing" : playerAsset ? "Paused" : "Choose a generated song"}</span></div></div>+ <div className="music-player__transport">+ <div className="music-player__buttons"><button type="button" className="icon-button" aria-label="Previous song" disabled={!playable.length} onClick={() => move(-1)}><GoToStart16 /></button><button type="button" className="music-player__toggle primary-button" aria-label={playing ? "Pause music" : "Play music"} disabled={!playable.length && !playerAsset} onClick={toggle}>{playing ? <Pause16 /> : <Play16 />}</button><button type="button" className="icon-button" aria-label="Next song" disabled={!playable.length} onClick={() => move(1)}><GoToEnd16 /></button></div>+ <div className="music-player__seek"><span>{time(position)}</span><Slider aria-label="Song playback position" min={0} max={length || 1} step={0.1} value={position} disabled={!url || !length} onChange={(value) => { if (audio.current) audio.current.currentTime = value; setPosition(value); }} /><span>{time(length)}</span></div>+ </div>+ <div className="music-player__volume"><Speaker14 /><Slider aria-label="Music volume" min={0} max={1} step={0.01} value={volume} onChange={setVolume} /></div>+ {playbackError && <p className="music-player__error" role="alert">{playbackError}</p>}+ <audio ref={audio} src={url ?? undefined} preload="metadata" onPlay={() => setPlaying(true)} onPause={() => setPlaying(false)}+ onLoadedMetadata={(event) => { const duration = event.currentTarget.duration; setLength(Number.isFinite(duration) ? duration : 0); }}+ onCanPlay={() => { if (playWhenReady.current) { playWhenReady.current = false; void play(); } }}+ onTimeUpdate={(event) => setPosition(event.currentTarget.currentTime)} onEnded={() => { setPlaying(false); const index = playable.findIndex(({ asset }) => asset.id === playerId); if (index >= 0 && index + 1 < playable.length) playAsset(playable[index + 1].asset); }}+ onError={() => { if (url) { playWhenReady.current = false; setPlaying(false); setPlaybackError("This song could not be played."); } }} />+ </div>+ {menu.element}+ </div>;+}diff --git a/src/lib/generatorModels.json b/src/lib/generatorModels.jsonindex 38add48..9c8541c 100644--- a/src/lib/generatorModels.json+++ b/src/lib/generatorModels.json@@ -20,7 +20,8 @@"cudaOnly": false,"imageCompositionCreation": false,"negativePrompt": false,- "negativePromptDisabledReason": null+ "negativePromptDisabledReason": null,+ "musicGeneration": false},"moss": {"label": "MOSS Image",@@ -45,7 +46,8 @@"cudaOnly": true,"imageCompositionCreation": true,"negativePrompt": true,- "negativePromptDisabledReason": null+ "negativePromptDisabledReason": null,+ "musicGeneration": false},"fibo2": {"label": "Fibo 2.0",@@ -68,6 +70,55 @@"cudaOnly": true,"imageCompositionCreation": true,"negativePrompt": false,- "negativePromptDisabledReason": "Fibo 2.0 Turbo uses only the positive prompt. Slopfab does not support negative prompts for this model."+ "negativePromptDisabledReason": "Fibo 2.0 Turbo uses only the positive prompt. Slopfab does not support negative prompts for this model.",+ "musicGeneration": false+ },+ "yue2": {+ "label": "YuE2",+ "description": "Music generation on CUDA or Vulkan",+ "imageGeneration": false,+ "videoGeneration": false,+ "imageEditing": false,+ "imageReferences": false,+ "loras": false,+ "motionCache": false,+ "generationEffects": false,+ "cfg": null,+ "imageOutpainting": false,+ "characterSheets": false,+ "requiresImageInput": false,+ "editImageReferences": false,+ "maxImageReferences": 0,+ "refmods": false,+ "sigmaShifts": false,+ "cudaOnly": false,+ "imageCompositionCreation": false,+ "negativePrompt": false,+ "negativePromptDisabledReason": null,+ "musicGeneration": true+ },+ "minimax_music3": {+ "label": "MiniMax Music3",+ "description": "Music generation on CUDA or Vulkan",+ "imageGeneration": false,+ "videoGeneration": false,+ "imageEditing": false,+ "imageReferences": false,+ "loras": false,+ "motionCache": false,+ "generationEffects": false,+ "cfg": null,+ "imageOutpainting": false,+ "characterSheets": false,+ "requiresImageInput": false,+ "editImageReferences": false,+ "maxImageReferences": 0,+ "refmods": false,+ "sigmaShifts": false,+ "cudaOnly": false,+ "imageCompositionCreation": false,+ "negativePrompt": false,+ "negativePromptDisabledReason": null,+ "musicGeneration": true}}diff --git a/src/lib/music.test.ts b/src/lib/music.test.tsnew file mode 100644index 0000000..db7c79d--- /dev/null+++ b/src/lib/music.test.ts@@ -0,0 +1,22 @@+import { describe, expect, it } from "vitest";+import { createProjectConfig, parseProjectConfig, type ProjectAsset } from "./project";+import { musicSettingsForModel, musicSettingsSchema, newSong, songReference } from "./music";++describe("music projects", () => {+ it("round-trips songs and preserves older projects without songs", () => {+ const config = createProjectConfig({ name: "Music", prompt: "", aspectRatio: "16:9", resolution: "416p", targetDurationSeconds: 30 });+ expect(parseProjectConfig(JSON.parse(JSON.stringify(config))).musicSongs).toBeUndefined();+ const song = { ...newSong(), style: "Jazz", settings: { ...musicSettingsSchema.parse({}), lyrics: "[Verse]\nHome again" } };+ expect(parseProjectConfig(JSON.parse(JSON.stringify({ ...config, musicSongs: [song] })))).toMatchObject({ musicSongs: [song] });+ });+ it("removes incompatible controls when switching generators", () => {+ const source = musicSettingsSchema.parse({ lyrics: "Sing", duration: 12, planning: "off", temperature: 0.7, top_p: 0.8, top_k: 80, semantic_guidance: 2, flow_guidance: 1.8 });+ expect(musicSettingsForModel(source, "minimax_music3")).toMatchObject({ lyrics: "Sing", duration: 12, planning: "full", temperature: 1, top_p: 0.95, top_k: 80, semantic_guidance: 2, flow_guidance: 1.8 });+ expect(musicSettingsForModel(source, "yue2").flow_guidance).toBe(0);+ });+ it("creates an audio reference trimmed to the supported length", () => {+ const asset: ProjectAsset = { id: "audio", kind: "audio", name: "Song", relativePath: "media/generated/song.wav", durationMs: 30000, mimeType: "audio/wav", createdAt: new Date().toISOString() };+ expect(songReference(newSong(), asset)).toMatchObject({ kind: "audio", relativePath: asset.relativePath, intendedUse: ["audio"], audio: { startSeconds: 0, durationSeconds: 15 } });+ expect(() => songReference(newSong(), { ...asset, durationMs: 1000 })).toThrow(/two seconds/);+ });+});diff --git a/src/lib/music.ts b/src/lib/music.tsnew file mode 100644index 0000000..3a8ee2f--- /dev/null+++ b/src/lib/music.ts@@ -0,0 +1,51 @@+import { z } from "zod";+import type { ProjectAsset, ProjectConfig, ProjectReference } from "./project";++export const musicSettingsSchema = z.object({+ lyrics: z.string().max(30000).default(""),+ duration: z.number().finite().min(2).max(300).default(30),+ planning: z.enum(["full", "melody", "off"]).default("full"),+ abc: z.string().max(30000).default(""),+ abc_max_tokens: z.number().int().min(32).max(4096).default(4096),+ temperature: z.number().finite().min(0).max(10).default(1),+ top_p: z.number().finite().gt(0).max(1).default(0.95),+ top_k: z.number().int().min(0).max(1024).default(0),+ repetition_penalty: z.number().finite().min(1).max(10).default(1.2),+ semantic_guidance: z.number().finite().min(0).max(10).default(0),+ flow_guidance: z.number().finite().min(0).max(10).default(0),+}).refine((value) => value.semantic_guidance === 0 || value.semantic_guidance >= 1, "Semantic guidance must be zero (automatic) or at least one.")+ .refine((value) => value.planning !== "off" || !value.abc, "ABC requires full or melody planning.");+export const musicSongSchema = z.object({+ id: z.string().min(1), title: z.string().min(1).max(200),+ style: z.string().max(10000).default(""),+ generatorId: z.string().default("yue2"),+ settings: musicSettingsSchema.default({}),+ steps: z.number().int().min(2).max(100).default(32),+ seed: z.number().int().min(-1).max(Number.MAX_SAFE_INTEGER).default(-1),+ status: z.enum(["draft", "queued", "generating", "completed", "failed", "cancelled"]).default("draft"),+ assetId: z.string().nullish(), error: z.string().nullish(),+ createdAt: z.string(),+});+export type MusicSong = z.infer<typeof musicSongSchema>;+export type MusicSettings = z.infer<typeof musicSettingsSchema>;++export function newSong(generatorId = "yue2", steps = 32): MusicSong {+ return musicSongSchema.parse({ id: `song-${crypto.randomUUID()}`, title: "New song", generatorId, steps, createdAt: new Date().toISOString() });+}++export function musicSettingsForModel(settings: MusicSettings, modelType: string): MusicSettings {+ const defaults = musicSettingsSchema.parse({});+ return modelType === "minimax_music3" ? { ...defaults, lyrics: settings.lyrics, duration: settings.duration,+ top_k: settings.top_k, semantic_guidance: settings.semantic_guidance, flow_guidance: settings.flow_guidance } : { ...settings, flow_guidance: 0 };+}++export function songReference(song: MusicSong, asset: ProjectAsset): ProjectReference {+ if (asset.kind !== "audio" || !asset.relativePath || !asset.durationMs || asset.durationMs < 2000) throw new Error("Generate a song at least two seconds long before creating a reference.");+ return { id: `ref-${crypto.randomUUID()}`, kind: "audio", name: song.title, description: song.style,+ relativePath: asset.relativePath, intendedUse: ["audio"], createdAt: new Date().toISOString(),+ audio: { startSeconds: 0, durationSeconds: Math.min(15, asset.durationMs / 1000) } };+}++export function updateSong(config: ProjectConfig, id: string, patch: Partial<MusicSong>): ProjectConfig {+ return { ...config, musicSongs: (config.musicSongs ?? []).map((song) => song.id === id ? { ...song, ...patch } : song) };+}diff --git a/src/lib/project.ts b/src/lib/project.tsindex c1d2445..2fc25b8 100644--- a/src/lib/project.ts+++ b/src/lib/project.ts@@ -3,6 +3,7 @@ import { generationEffectsSchema, activeGenerationEffects, type GenerationEffectimport { MIN_SIGMA_SHIFT, MAX_SIGMA_SHIFT } from "./sampling";import { z } from "zod";import { backdropDirection, backdropShotDirection, backdropSummary, type BackdropColor } from "./backdrop";+import { musicSongSchema } from "./music";import { createImageScene, imageSceneReferenceIds, imageSceneSchema } from "./imageScene";import { effectEnabledSchema, effectSchemas, isEffectOn } from "./effectSettings";import { normalizeShotTagSelection, shotTagClauses, SHOT_TAG_ID_PATTERN, type ShotTagClauses, type ShotTagSelection } from "./shot-tags";@@ -693,6 +694,7 @@ export const projectConfigSchema = z.object({schemaVersion: z.literal(CURRENT_SCHEMA_VERSION),generationType: generationTypeSchema.default("video"),imageScene: imageSceneSchema.nullish(),+ musicSongs: z.array(musicSongSchema).optional(),imageSettings: z.object({resolution: resolutionSchema,aspectRatio: aspectRatioSchema,diff --git a/src/lib/runtime.ts b/src/lib/runtime.tsindex ba6c43e..14d7222 100644--- a/src/lib/runtime.ts+++ b/src/lib/runtime.ts@@ -96,6 +96,7 @@ export interface AgentTurnResponse {}export interface SlopfabGenerationRequest {+ music?: import("./music").MusicSettings;jobId: string;videoTransition?: "extend" | "bridge";latentBridge?: import("./project").SceneGenerationInput["latentBridge"];diff --git a/src/lib/settings.test.ts b/src/lib/settings.test.tsindex 308f46f..7e2bef4 100644--- a/src/lib/settings.test.ts+++ b/src/lib/settings.test.ts@@ -4,7 +4,7 @@ import { beforeEach, describe, expect, it } from "vitest";import { TURBO_LORA, VIGGLE_ANIMATE_LORA, loadLoras, saveLoras } from "./loras";import {agentEndpointProviderSettings,- selectedImageGeneratorTemplate, createGeneratorTemplate, mossImageTemplate, fiboImageTemplate, fiboBaseTemplate, templateSupportsEditing, templateSupportsOutpainting, defaultGeneratorTemplate, generatorPathFields, templateSupportsVideo,+ selectedImageGeneratorTemplate, createGeneratorTemplate, musicTemplates, mossImageTemplate, fiboImageTemplate, fiboBaseTemplate, templateSupportsEditing, templateSupportsOutpainting, defaultGeneratorTemplate, generatorPathFields, templateSupportsVideo,engineProviderSetting,loadInferenceBackend,saveInferenceBackend,@@ -47,7 +47,7 @@ describe("generator templates", () => {const custom = createGeneratorTemplate("Custom");saveGeneratorTemplateSettings({ templates: [custom], defaultTemplateId: custom.id, catalogVersion: 13 });const migrated = loadGeneratorTemplateSettings();- expect(migrated.templates).toEqual([expect.objectContaining(custom), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(migrated.templates).toEqual([expect.objectContaining(custom), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(migrated.defaultTemplateId).toBe(custom.id);expect(loadGeneratorTemplateSettings()).toEqual(migrated);saveGeneratorTemplateSettings({ ...migrated, templates: [custom] });@@ -75,7 +75,7 @@ describe("generator templates", () => {const custom = createGeneratorTemplate("Custom");saveGeneratorTemplateSettings({ templates: [custom], defaultTemplateId: custom.id, catalogVersion: 9 });const migrated = loadGeneratorTemplateSettings();- expect(migrated.templates).toEqual([expect.objectContaining(custom), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(migrated.templates).toEqual([expect.objectContaining(custom), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(migrated.defaultTemplateId).toBe(custom.id);expect(loadGeneratorTemplateSettings()).toEqual(migrated);saveGeneratorTemplateSettings({ ...migrated, templates: [custom] });@@ -99,7 +99,7 @@ describe("generator templates", () => {it("assigns MiniMax H3 to current and legacy generators and preserves future model types", () => {expect(createGeneratorTemplate().modelType).toBe("minimax-h3");- expect(loadGeneratorTemplateSettings().templates.filter((template) => ! ["moss-image", "fibo2-edit-turbo", "fibo2-base"].includes(template.id)).every((template) => template.modelType === "minimax-h3")).toBe(true);+ expect(loadGeneratorTemplateSettings().templates.filter((template) => ! ["moss-image", "fibo2-edit-turbo", "fibo2-base", "yue2", "minimax_music3"].includes(template.id)).every((template) => template.modelType === "minimax-h3")).toBe(true);for (const modelType of [undefined, null, "", 123]) {localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({templates: [{ id: "legacy", name: "Renamed generator", modelType }], defaultTemplateId: "legacy", catalogVersion: 9,@@ -130,7 +130,7 @@ describe("generator templates", () => {localStorage.setItem("slopus.engine-paths.v1", JSON.stringify({ transformer: "D:\\Models\\main.safetensors" }));const settings = loadGeneratorTemplateSettings();expect(settings.defaultTemplateId).toBe("default");- expect(settings.templates).toHaveLength(9);+ expect(settings.templates).toHaveLength(11);expect(settings.templates[1].name).toBe("First/Last Frame");expect(settings.templates[0]).toMatchObject({id: "default",@@ -198,7 +198,7 @@ describe("generator templates", () => {paths: { ...original.paths, transformer },sources: { ...original.sources, transformer: [{ ...original.sources!.transformer![0], url: transformer }, { url: "https://huggingface.co/koongrizzly/MiniMax_H3_int4_W4A8_ConvRot_Pruned/resolve/main/diffusion_models/minimax_h3_ref2va_hybrid_b20-49_pruned_w4a8_mixed.safetensors", gpuModel: "", minVramGb: 0 }] },});- expect(loadGeneratorTemplateSettings().templates.map(({ name }) => name)).toEqual(["Default", "First/Last Frame", "References", "FastH3 Omni", "Singularity", "Animate", "MOSS Image", "Fibo 2.0 Edit Turbo", "Fibo 2.0 Base"]);+ expect(loadGeneratorTemplateSettings().templates.map(({ name }) => name)).toEqual(["Default", "First/Last Frame", "References", "FastH3 Omni", "Singularity", "Animate", "MOSS Image", "Fibo 2.0 Edit Turbo", "Fibo 2.0 Base", "YuE2", "MiniMax Music3"]);});it.each(["Minimax H3 Original", "My custom generator"])("migrates the saved %s template without changing its configuration", (name) => {@@ -215,7 +215,7 @@ describe("generator templates", () => {expect(settings.templates[2]).toEqual(minimaxFastTemplate());expect(settings.templates[3]).toEqual(minimaxSingularityTemplate());expect(settings.defaultTemplateId).toBe(template.id);- expect(settings.catalogVersion).toBe(15);+ expect(settings.catalogVersion).toBe(16);expect(loadGeneratorTemplateSettings()).toEqual(settings);saveGeneratorTemplateSettings({ ...settings, templates: [settings.templates[0]] });expect(loadGeneratorTemplateSettings().templates).toHaveLength(1);@@ -225,7 +225,7 @@ describe("generator templates", () => {const references = minimaxReferencesTemplate();references.paths.transformer = "D:/Models/custom.safetensors";localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [references], catalogVersion: 3 }));- expect(loadGeneratorTemplateSettings().templates).toEqual([references, minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(loadGeneratorTemplateSettings().templates).toEqual([references, minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);});it("adds Singularity to the previous catalog once while preserving saved templates", () => {@@ -234,9 +234,9 @@ describe("generator templates", () => {custom.paths.transformer = "D:/Models/custom.safetensors";localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [custom], defaultTemplateId: custom.id, catalogVersion: 4 }));const settings = loadGeneratorTemplateSettings();- expect(settings.templates).toEqual([expect.objectContaining(custom), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(settings.templates).toEqual([expect.objectContaining(custom), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(settings.defaultTemplateId).toBe(custom.id);- expect(settings.catalogVersion).toBe(15);+ expect(settings.catalogVersion).toBe(16);expect(loadGeneratorTemplateSettings()).toEqual(settings);saveGeneratorTemplateSettings({ ...settings, templates: [custom] });expect(loadGeneratorTemplateSettings().templates).toEqual([expect.objectContaining(custom)]);@@ -247,7 +247,7 @@ describe("generator templates", () => {template.defaultSteps = 8;template.paths.transformer = "D:/Models/singularity.safetensors";localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 4 }));- expect(loadGeneratorTemplateSettings().templates).toEqual([template, viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(loadGeneratorTemplateSettings().templates).toEqual([template, viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);});it("adds Turbo to saved Singularity once and preserves other adapters and settings", () => {@@ -257,7 +257,7 @@ describe("generator templates", () => {template.loras = [{ loraId: "custom", enabled: false, strength: 0.5 }];localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], defaultTemplateId: template.id, catalogVersion: 5 }));const settings = loadGeneratorTemplateSettings();- expect(settings.templates).toEqual([{ ...template, loras: [...template.loras, { loraId: TURBO_LORA.id, enabled: true, strength: 1 }] }, viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(settings.templates).toEqual([{ ...template, loras: [...template.loras, { loraId: TURBO_LORA.id, enabled: true, strength: 1 }] }, viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(settings.defaultTemplateId).toBe("");expect(loadGeneratorTemplateSettings()).toEqual(settings);saveGeneratorTemplateSettings({ ...settings, templates: [template] });@@ -268,7 +268,7 @@ describe("generator templates", () => {const template = minimaxSingularityTemplate();template.loras = [{ loraId: TURBO_LORA.id, enabled: false, strength: 0.3 }];localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 5 }));- expect(loadGeneratorTemplateSettings().templates).toEqual([template, viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(loadGeneratorTemplateSettings().templates).toEqual([template, viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);});it("adds FastH3 Omni with eight steps and its model URLs", () => {@@ -300,8 +300,8 @@ describe("generator templates", () => {}localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 8 }));const settings = loadGeneratorTemplateSettings();- expect(settings.templates).toEqual([minimaxFastTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);- expect(settings.catalogVersion).toBe(15);+ expect(settings.templates).toEqual([minimaxFastTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);+ expect(settings.catalogVersion).toBe(16);expect(templateNeedsDownload(settings.templates[0])).toBe(true);expect(loadGeneratorTemplateSettings()).toEqual(settings);});@@ -316,8 +316,8 @@ describe("generator templates", () => {const otherPaths = { ...template.paths };localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 10 }));const settings = loadGeneratorTemplateSettings();- expect(settings.catalogVersion).toBe(15);- expect(settings.templates).toHaveLength(3);+ expect(settings.catalogVersion).toBe(16);+ expect(settings.templates).toHaveLength(5);expect(settings.templates[0]).toMatchObject({ id: template.id, name: "FastH3 Omni", defaultSteps: 8,paths: { ...otherPaths, transformer: minimaxFastTemplate().paths.transformer },sources: { transformer: [{ url: minimaxFastTemplate().paths.transformer, gpuModel: "RTX 5090", minVramGb: 24 }] } });@@ -344,7 +344,7 @@ describe("generator templates", () => {expect(migrated.paths.transformer).toBe(selection === "url" || selection === "downloaded" ? expectedUrl : previousPath);expect(migrated.name).toBe(template.name);expect(migrated.defaultSteps).toBe(12);- expect(settings.templates.slice(1)).toEqual([references, fast, fiboImageTemplate(), fiboBaseTemplate()]);+ expect(settings.templates.slice(1)).toEqual([references, fast, fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(loadGeneratorTemplateSettings()).toEqual(settings);});@@ -372,7 +372,7 @@ describe("generator templates", () => {expect(migrated.sources!.transformer).toContainEqual({ url: "https://example.com/custom.safetensors", gpuModel: "Custom GPU", minVramGb: 8 });expect(migrated.paths.transformer).toBe(selection === "custom" ? "D:/custom.safetensors" : selection === "cached omni" ? "D:/weights/omni.safetensors" : omniUrl);expect(settings.defaultTemplateId).toBe(selection === "custom" || selection === "cached omni" ? template.id : "");- expect(settings.templates.slice(1)).toEqual([original, references, fiboImageTemplate(), fiboBaseTemplate()]);+ expect(settings.templates.slice(1)).toEqual([original, references, fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(loadGeneratorTemplateSettings()).toEqual(settings);});@@ -383,11 +383,11 @@ describe("generator templates", () => {template.paths.transformer = "D:/custom/model.safetensors";template.sources!.transformer = [{ url: "https://example.com/custom.safetensors", downloadedPath: template.paths.transformer, gpuModel: "", minVramGb: 0 }];localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 10 }));- expect(loadGeneratorTemplateSettings().templates).toEqual([template, fiboImageTemplate(), fiboBaseTemplate()]);+ expect(loadGeneratorTemplateSettings().templates).toEqual([template, fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);const other = createGeneratorTemplate("Custom");localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [other], defaultTemplateId: other.id, catalogVersion: 10 }));const settings = loadGeneratorTemplateSettings();- expect(settings.templates).toEqual([expect.objectContaining(other), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(settings.templates).toEqual([expect.objectContaining(other), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);expect(settings.defaultTemplateId).toBe(other.id);});@@ -399,10 +399,10 @@ describe("generator templates", () => {template.sources = { ...template.sources, videoVae: [{ url: template.paths.videoVae, gpuModel: "", minVramGb: 0 }] };delete template.sources.transformer;localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 8 }));- expect(loadGeneratorTemplateSettings().templates).toEqual([template, mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(loadGeneratorTemplateSettings().templates).toEqual([template, mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);const original = minimaxOriginalTemplate();localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [original], catalogVersion: 8 }));- expect(loadGeneratorTemplateSettings().templates).toEqual([original, mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()]);+ expect(loadGeneratorTemplateSettings().templates).toEqual([original, mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()]);});it.each(["textEncoder", "transformer"] as const)("upgrades saved Minimax %s variants once and preserves downloaded weights", (field) => {@@ -412,7 +412,7 @@ describe("generator templates", () => {template.paths[field] = downloadedPath;localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 1 }));const settings = loadGeneratorTemplateSettings();- expect(settings.catalogVersion).toBe(15);+ expect(settings.catalogVersion).toBe(16);expect(settings.templates[0].paths[field]).toBe(downloadedPath);expect(settings.templates[0].sources![field]).toEqual(minimaxOriginalTemplate().sources![field]!.map((source, index) =>index === 0 ? { ...source, downloadedPath } : source));@@ -427,7 +427,7 @@ describe("generator templates", () => {expect(loadGeneratorTemplateSettings().templates[0].sources![field]).toEqual(template.sources![field]);const custom = createGeneratorTemplate();localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [custom], catalogVersion: 1 }));- expect(loadGeneratorTemplateSettings().templates.map(({ id }) => id)).toEqual([custom.id, "minimax-h3-references", "minimax-h3-fast", "minimax-h3-singularity", "viggle-animate", "moss-image", "fibo2-edit-turbo", "fibo2-base"]);+ expect(loadGeneratorTemplateSettings().templates.map(({ id }) => id)).toEqual([custom.id, "minimax-h3-references", "minimax-h3-fast", "minimax-h3-singularity", "viggle-animate", "moss-image", "fibo2-edit-turbo", "fibo2-base", "yue2", "minimax_music3"]);});it.each(["url", "downloaded", "original", "custom", "cached original"])("removes the incompatible encoder from saved templates using %s", (selection) => {@@ -447,7 +447,7 @@ describe("generator templates", () => {localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], defaultTemplateId: template.id, catalogVersion: 2 }));const settings = loadGeneratorTemplateSettings();const migrated = settings.templates[0];- expect(settings.catalogVersion).toBe(15);+ expect(settings.catalogVersion).toBe(16);expect(migrated.sources!.textEncoder).toEqual([{ url: originalUrl, gpuModel: "", minVramGb: 0, ...(selection === "cached original" ? { downloadedPath: cachedOriginal } : {}) },custom,@@ -528,7 +528,7 @@ it("adds Base once and persists per-template editing through runtime options", (edit.defaultSteps = 8;saveGeneratorTemplateSettings({ templates: [edit], defaultTemplateId: "", catalogVersion: 14 });const migrated = loadGeneratorTemplateSettings();- expect(migrated.templates).toEqual([edit, fiboBaseTemplate()]);+ expect(migrated.templates).toEqual([edit, fiboBaseTemplate(), ...musicTemplates()]);expect(loadGeneratorTemplateSettings()).toEqual(migrated);const base = migrated.templates[1];expect(base.name).toBe("Fibo 2.0 Base");diff --git a/src/lib/settings.ts b/src/lib/settings.tsindex 9644857..8de1f0e 100644--- a/src/lib/settings.ts+++ b/src/lib/settings.ts@@ -75,6 +75,7 @@ function migratedStorageItem(key: string, legacyKey: string): string | null {export const MINIMAX_H3_MODEL_TYPE = "minimax-h3";export const MOSS_MODEL_TYPE = "moss";export const FIBO_MODEL_TYPE = "fibo2";+export const templateSupportsMusic = (template: Pick<GeneratorTemplate, "modelType">): boolean => generatorModel(template.modelType).musicGeneration;export const templateSupportsVideo = (template: Pick<GeneratorTemplate, "modelType">): boolean => generatorModel(template.modelType).videoGeneration;export const templateSupportsImages = (template: Pick<GeneratorTemplate, "modelType" | "mode">): boolean => generatorModel(template.modelType).imageGeneration && template.mode !== "animate";@@ -119,6 +120,9 @@ export interface WeightSource {}export function generatorPathFields(template: Pick<GeneratorTemplate, "mode"> & { modelType?: string }): EnginePathField[] {+ if (template.modelType === "yue2" || template.modelType === "minimax_music3") return ENGINE_PATH_FIELDS.filter(({ id }) =>+ id === "transformer" || template.modelType === "minimax_music3" && (id === "textEncoder" || id === "audioVae")).map((field) =>+ field.id === "transformer" ? { ...field, hint: "The main model that generates music from your style and lyrics." } : field);if (template.modelType === MOSS_MODEL_TYPE || template.modelType === FIBO_MODEL_TYPE) return ENGINE_PATH_FIELDS.filter(({ id }) => id !== "audioVae").map((field) => {if (field.id === "transformer") return { ...field, hint: "The main model that turns your description into an image." };if (field.id === "videoVae") return { ...field, label: "Image VAE weights", hint: "Decodes the model's internal picture into an image." };@@ -238,12 +242,25 @@ export function createGeneratorTemplate(name = "New template", modelType = MINIMname,modelType,editingModel: generatorModel(modelType).imageEditing,- defaultSteps: modelType === MOSS_MODEL_TYPE ? 50 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS,- attention: generatorModel(modelType).cudaOnly ? "flash2" : "sage2",+ defaultSteps: modelType === "yue2" ? 32 : modelType === "minimax_music3" ? 30 : modelType === MOSS_MODEL_TYPE ? 50 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS,+ attention: generatorModel(modelType).cudaOnly || generatorModel(modelType).musicGeneration ? "flash2" : "sage2",paths: copyPaths(EMPTY_ENGINE_SETTINGS),};}+export function musicTemplates(): GeneratorTemplate[] {+ const root = "https://huggingface.co/Comfy-Org/MiniMax-Music-3/resolve/main/";+ return (["yue2", "minimax_music3"] as const).map((modelType) => {+ const paths = { ...EMPTY_ENGINE_SETTINGS, ...(modelType === "yue2" ? {+ transformer: "https://huggingface.co/Comfy-Org/YuE2/resolve/main/checkpoints/yue2_3b_int8_convrot.safetensors",+ } : { transformer: root + "diffusion_models/minimax_music3_dit_int8_convrot.safetensors",+ textEncoder: root + "text_encoders/minimax_music3_text_encoder_pruned_int8_convrot.safetensors",+ audioVae: root + "vae/minimax_music3_dav.safetensors" }) };+ return { ...createGeneratorTemplate(generatorModel(modelType).label, modelType), id: modelType, paths,+ sources: Object.fromEntries(Object.entries(paths).filter(([, url]) => url).map(([id, url]) => [id, [{ url, gpuModel: "", minVramGb: 0 }]])) };+ });+}+export function mossImageTemplate(): GeneratorTemplate {const root = "https://huggingface.co/Comfy-Org/MOSS-Image/resolve/main/";const paths: EngineSettings = {@@ -347,9 +364,9 @@ export function viggleAnimateTemplate(): GeneratorTemplate {}const initialTemplateSettings = (paths = EMPTY_ENGINE_SETTINGS): GeneratorTemplateSettings => ({- templates: [{ id: "default", name: "Default", modelType: MINIMAX_H3_MODEL_TYPE, defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(paths) }, minimaxOriginalTemplate(), minimaxReferencesTemplate(), minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate()],+ templates: [{ id: "default", name: "Default", modelType: MINIMAX_H3_MODEL_TYPE, defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(paths) }, minimaxOriginalTemplate(), minimaxReferencesTemplate(), minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates()],defaultTemplateId: "default",- catalogVersion: 15,+ catalogVersion: 16,});const normalizeTemplateSettings = (value: unknown): GeneratorTemplateSettings | null => {@@ -370,7 +387,7 @@ const normalizeTemplateSettings = (value: unknown): GeneratorTemplateSettings |&& candidate.defaultSteps >= 2&& candidate.defaultSteps <= MAX_GENERATION_STEPS? candidate.defaultSteps- : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS;+ : modelType === "yue2" ? 32 : modelType === "minimax_music3" ? 30 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS;const attention = candidate.attention === "exact" || candidate.attention === "flash2" || candidate.attention === "sol" ? candidate.attention : "sage2";const paths = pathsFrom(candidate.paths);const sources: GeneratorTemplate["sources"] = {};@@ -412,7 +429,7 @@ const normalizeTemplateSettings = (value: unknown): GeneratorTemplateSettings |templates,defaultTemplateId: templates.find((template) => template.id === requestedDefault && templateUsable(template))?.id?? templates.find(templateUsable)?.id ?? "",- catalogVersion: typeof record.catalogVersion === "number" && [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15].includes(record.catalogVersion) ? record.catalogVersion : 0,+ catalogVersion: typeof record.catalogVersion === "number" && [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16].includes(record.catalogVersion) ? record.catalogVersion : 0,};};@@ -606,6 +623,11 @@ export function loadGeneratorTemplateSettings(): GeneratorTemplateSettings {normalized = normalizeTemplateSettings({ ...normalized, catalogVersion: 15 })!;localStorage.setItem(GENERATOR_TEMPLATES_KEY, JSON.stringify(normalized));}+ if ((normalized.catalogVersion ?? 0) < 16) {+ for (const template of musicTemplates()) if (!normalized.templates.some(({ id }) => id === template.id)) normalized.templates.push(template);+ normalized = normalizeTemplateSettings({ ...normalized, catalogVersion: 16 })!;+ localStorage.setItem(GENERATOR_TEMPLATES_KEY, JSON.stringify(normalized));+ }return normalized;}}@@ -668,7 +690,7 @@ export function engineProviderSetting(settings: EngineSettings, base?: ProviderS// A LAN worker receives URLs for weights it should download itself.const remote = remoteWorkerSelected();const options: ProviderSetting["options"] = { ...(base?.options ?? {}), attention, inferenceBackend: loadInferenceBackend() };- const imageOnly = !generatorModel(shifts.modelType).videoGeneration;+ const pathFields = generatorPathFields({ modelType: shifts.modelType, mode });options.modelType = shifts.modelType ?? MINIMAX_H3_MODEL_TYPE;options.editingModel = templateSupportsEditing({ modelType: options.modelType, editingModel: shifts.editingModel, mode });// Always replace project-carried adapter paths with this machine's selection.@@ -687,10 +709,10 @@ export function engineProviderSetting(settings: EngineSettings, base?: ProviderSdelete options.workerWeightSources;delete options.latentUpscaler;const latentUpscaler = latentUpscalerPath();- if (latentUpscaler) options.latentUpscaler = latentUpscaler;+ if (latentUpscaler && !generatorModel(shifts.modelType).musicGeneration) options.latentUpscaler = latentUpscaler;if (remote && sources) {options.workerWeightSources = JSON.stringify(Object.fromEntries(Object.entries(sources)- .filter(([role, entries]) => entries?.length && !(imageOnly && role === "audioVae") && !(mode === "animate" && (role === "textEncoder" || role === "tokenizer")))+ .filter(([role, entries]) => entries?.length && pathFields.some(({ id }) => id === role)).map(([role, entries]) => [role, entries!.map(({ url, gpuModel, minVramGb }) => ({ url, gpuModel, minVramGb }))])));}if (motionCache && mode !== "animate") options.motionCache = true;@@ -713,7 +735,7 @@ export function engineProviderSetting(settings: EngineSettings, base?: ProviderSdelete options.motionCache;}for (const field of ENGINE_PATH_FIELDS) {- if ((imageOnly && field.id === "audioVae") || (mode === "animate" && (field.id === "textEncoder" || field.id === "tokenizer"))) {+ if (!pathFields.some(({ id }) => id === field.id)) {delete options[field.id];continue;}diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.tsindex 069ee99..b6a77f7 100644--- a/src/lib/workQueue.test.ts+++ b/src/lib/workQueue.test.ts@@ -20,6 +20,8 @@ import { compileImagePrompt } from "./imagePrompt";import { addImageNode, createImageEditScene } from "./imageScene";import { createEmptyImage, imageGenerationSnapshot, makeImagePrimary, restoreGeneratedImage } from "./imageHistory";import { sceneGenerationRequest } from "./sceneGeneration";+import { newSong } from "./music";+import { musicTemplates } from "./settings";import { bridgeGapFrames, currentLongShotArchives } from "./longShot";vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));@@ -63,6 +65,42 @@ const finish = async (queue: WorkQueue, id: string) => {await waitFor(() => expect(queue.getSnapshot().find((item) => item.id === id)?.status).toBe("completed"));};+it("queues an immutable music request and saves its audio into timeline media", async () => {+ const { queue, first, saved } = setup();+ const song = { ...newSong(), title: "Evening", style: "Soft jazz", seed: 42 };+ first.update((config) => ({ ...config, musicSongs: [song] }));+ const template = musicTemplates()[0];+ template.paths.transformer = "C:/weights/yue2.safetensors";+ template.sources = {};+ vi.mocked(invoke).mockResolvedValue({ relativePath: "media/generated/song.wav", durationMs: 30000 });+ queue.enqueueMusic(first, song.id, template);+ queue.enqueueMusic(first, song.id, template);+ first.update((config) => ({ ...config, musicSongs: config.musicSongs!.map((item) => ({ ...item, style: "Changed later" })) }));+ await waitFor(() => expect(enqueueSlopfabGeneration).toHaveBeenCalledOnce());+ const request = vi.mocked(enqueueSlopfabGeneration).mock.calls[0][0];+ expect(request).toMatchObject({ prompt: "Soft jazz", seed: 42, steps: 32, music: { duration: 30 }, referencePaths: [] });+ expect(request).not.toHaveProperty("saveLatentsPath");+ const id = queue.getSnapshot()[0].id;+ await finish(queue, id);+ expect(invoke).toHaveBeenCalledWith("save_generated_music", { folderPath: first.record.folderPath, jobId: id });+ expect(first.getSnapshot().config.musicSongs![0]).toMatchObject({ status: "completed", assetId: id, style: "Changed later" });+ expect(first.getSnapshot().config.assets).toContainEqual(expect.objectContaining({ id, kind: "audio", name: "Evening", relativePath: "media/generated/song.wav", durationMs: 30000 }));+ expect(saved).toHaveBeenCalled();+ expect(releaseRendered).toHaveBeenCalledWith(id);+ expect(saveGeneratedScene).not.toHaveBeenCalled();+});++it("cancels a queued song without altering another song", async () => {+ const { queue, first } = setup();+ const songs = [newSong(), newSong()].map((song) => ({ ...song, style: "Jazz" }));+ first.update((config) => ({ ...config, musicSongs: songs }));+ for (const song of songs) queue.enqueueMusic(first, song.id, musicTemplates()[0]);+ const id = queue.getSnapshot()[1].id;+ await queue.cancel(id);+ expect(first.getSnapshot().config.musicSongs![1].status).toBe("cancelled");+ expect(first.getSnapshot().config.musicSongs![0].status).not.toBe("cancelled");+});+it("releases submitted snapshots after each generation while keeping completed results available", async () => {const { queue, first } = setup();const pending = (queue as unknown as { work: Map<string, unknown> }).work;diff --git a/src/lib/workQueue.ts b/src/lib/workQueue.tsindex 43b9ac5..26ff922 100644--- a/src/lib/workQueue.ts+++ b/src/lib/workQueue.ts@@ -1,4 +1,6 @@import { completeSceneGeneration, sceneInput } from "./sceneHistory";+import { musicSongSchema, musicSettingsForModel, updateSong, type MusicSong } from "./music";+import { templateSupportsMusic } from "./settings";import { imageCfgScale } from "./generatorModels";import { activeGenerationEffects } from "./generationEffects";import { imageSettings } from "./imageSettings";@@ -46,7 +48,7 @@ function imageGenerationSeed(seed: number): number {export interface WorkItem {imageAssetId?: string | null;imageDraftId?: string;- kind?: "reference-icons" | "image" | "refmod";+ kind?: "reference-icons" | "image" | "refmod" | "music";id: string;projectKey: string;folderPath: string;@@ -64,6 +66,7 @@ export interface WorkItem {settings: { frames: number; steps: number; audioSteps?: number; videoSigmaShift?: number; audioSigmaShift?: number; latentUpscale?: boolean; seed: number; canvasWidth: number; canvasHeight: number };}interface PendingWork {+ music?: MusicSong;sceneInput?: SceneInput;extend?: { sourceId: string; name: string; options: ExtendOptions };characterSheet?: {@@ -150,7 +153,8 @@ export class WorkQueue {if (!session) {// The queue belongs to this app run. A persisted in-flight status from a// previous run has no worker behind it and must be restartable.- const config = isTauri() ? { ...record.config, generationJobs: record.config.generationJobs.map((job) =>+ const config = isTauri() ? { ...record.config, musicSongs: (record.config.musicSongs ?? []).map((song) =>+ ["queued", "generating"].includes(song.status) ? { ...song, status: "failed" as const, error: "Generation was interrupted when Slopus closed. Generate again to restart." } : song), generationJobs: record.config.generationJobs.map((job) =>["queued", "generating", "ready"].includes(job.status) ? { ...job, status: "failed" as const, stage: "failed" as const, error: "Generation was interrupted when Slopus closed. Generate again to restart." } : job) } : record.config;session = new ProjectSession({ ...record, config }, this.writer);this.projects.set(key, session);@@ -275,6 +279,30 @@ export class WorkQueue {});}+ enqueueMusic(session: ProjectSession, songId: string, template: GeneratorTemplate) {+ if (!isTauri()) throw new Error("Music generation requires the desktop app and model weights.");+ if (!templateSupportsMusic(template)) throw new Error("Choose a music generator.");+ const current = session.getSnapshot().config;+ const song = musicSongSchema.parse((current.musicSongs ?? []).find((item) => item.id === songId));+ if (!song.style.trim()) throw new Error("Describe the song's style before generating.");+ const projectKey = projectQueueKey(session.record);+ if (this.items.some((item) => item.projectKey === projectKey && item.kind === "music" && item.sceneId === songId && isWorkActive(item))) return;+ const id = `music-${crypto.randomUUID()}`;+ const captured = { ...structuredClone(song), generatorId: template.id, seed: imageGenerationSeed(song.seed) };+ const config = structuredClone({ ...current, providerSettings: { ...current.providerSettings,+ slopfab: engineProviderSetting(template.paths, current.providerSettings.slopfab, "flash2", [], "prompt", [], false, template.sources, { modelType: template.modelType }) } });+ const request: SlopfabGenerationRequest = { jobId: id, prompt: song.style, steps: song.steps, seed: captured.seed,+ music: musicSettingsForModel(song.settings, template.modelType), frames: 0, canvasWidth: 0, canvasHeight: 0, referencePaths: [] };+ let finish!: () => void;+ const done = new Promise<void>((resolve) => { finish = resolve; });+ this.work.set(id, { id, music: captured, session, sceneId: songId, request, config, snapshot: JSON.stringify(captured), cancelled: false, submitted: false, done, finish });+ session.update((value) => updateSong(value, songId, { status: "queued", error: null, generatorId: template.id }));+ this.items = [...this.items, { id, kind: "music", projectKey, folderPath: session.record.folderPath, projectName: current.name,+ sceneId: songId, title: song.title, submittedAt: new Date().toISOString(), status: "queued", progress: 0,+ detail: "Waiting to generate music", error: null, completionAt: null, cancelling: false, needsSave: false,+ settings: { frames: 0, steps: song.steps, seed: captured.seed, canvasWidth: 0, canvasHeight: 0 } }];+ this.publish(); void this.pump();+ }enqueueImage(session: ProjectSession, template: GeneratorTemplate, count = 1) {if (!Number.isInteger(count) || count < 1 || count > MAX_IMAGE_BATCH_COUNT) throw new Error(`Count must be a whole number from 1 to ${MAX_IMAGE_BATCH_COUNT}.`);if (!isTauri()) throw new Error("Image generation requires the desktop app and downloaded model weights.");@@ -453,6 +481,12 @@ export class WorkQueue {this.publish(); this.icons.yieldToVideo(); void this.pump();}private updateScene(work: PendingWork, patch: Partial<GenerationJob>, dirty = true) {+ if (work.music) {+ const id = work.sceneId;+ const status = patch.status === "ready" ? "generating" : patch.status;+ work.session.update((current) => updateSong(current, id, { ...(status ? { status } : {}), ...(patch.error !== undefined ? { error: patch.error } : {}) }), dirty);+ return;+ }if (work.image || work.refmod) return;// The undo replay log retains this callback. Do not retain the submitted// project snapshot with every progress update.@@ -523,14 +557,14 @@ export class WorkQueue {work.snapshot = sceneGenerationSnapshot(captured, work.request);this.updateScene(work, { generationSnapshot: work.snapshot });}- if (!work.image) await purgeTimelineThumbnails(next.folderPath, work.sceneId).catch(() => undefined);+ if (!work.image && !work.music) await purgeTimelineThumbnails(next.folderPath, work.sceneId).catch(() => undefined);if (work.cancelled) continue;await this.prepareReferences(work, next.folderPath);if (work.cancelled) continue;const plan = await resolveSlopfabPlan(work.request, work.config, next.folderPath);if (work.cancelled) continue;work.outputFrames = plan.alignedFrames;- this.patch(work.id, { status: "generating", detail: `Generating ${work.image ? "image" : `${plan.alignedFrames} frames`} at ${plan.canvasWidth} × ${plan.canvasHeight}` });+ this.patch(work.id, { status: "generating", detail: work.music ? "Generating music" : `Generating ${work.image ? "image" : `${plan.alignedFrames} frames`} at ${plan.canvasWidth} × ${plan.canvasHeight}` });this.updateScene(work, { status: "generating", stage: "preparing" }, false);await enqueueSlopfabGeneration(work.request, work.config, next.folderPath);work.submitted = true;@@ -591,7 +625,7 @@ export class WorkQueue {try { await cancelSlopfabGeneration(work.request.jobId); }catch (reason) {work.cancelled = false;- this.patch(work.id, { cancelling: false, detail: work.image ? "Generating image" : "Generating video", error: `Could not cancel: ${describeDiagnosticError(reason)}` });+ this.patch(work.id, { cancelling: false, detail: work.music ? "Generating music" : work.image ? "Generating image" : "Generating video", error: `Could not cancel: ${describeDiagnosticError(reason)}` });}}cancelScenes(session: ProjectSession, sceneIds: string[]) {@@ -642,7 +676,7 @@ export class WorkQueue {return;}const progress = Math.max(item.progress, event.totalSteps > 0 ? Math.min(0.88, 0.12 + Math.max(0, event.step) / event.totalSteps * 0.76) : event.stage === "delivering" ? 0.88 : 0.08);- this.patch(work.id, { status: "generating", progress, completionAt: this.timing.update(event), detail: item.cancelling ? "Cancelling generation" : event.stage === "upscaling" ? "Upscaling video latents" : work.request.imageEdit ? `Applying image edits (${Math.min(work.request.imageEdit.edits.length, Math.floor(event.step / Math.max(1, event.totalSteps) * work.request.imageEdit.edits.length) + 1)} of ${work.request.imageEdit.edits.length})` : event.stage === "starting" || event.stage === "transformerLoad" ? "Loading MiniMax H3" : work.image ? "Generating image" : "Generating video" });+ this.patch(work.id, { status: "generating", progress, completionAt: this.timing.update(event), detail: item.cancelling ? "Cancelling generation" : event.stage === "upscaling" ? "Upscaling video latents" : work.request.imageEdit ? `Applying image edits (${Math.min(work.request.imageEdit.edits.length, Math.floor(event.step / Math.max(1, event.totalSteps) * work.request.imageEdit.edits.length) + 1)} of ${work.request.imageEdit.edits.length})` : work.music ? "Generating music" : event.stage === "starting" || event.stage === "transformerLoad" ? "Loading MiniMax H3" : work.image ? "Generating image" : "Generating video" });this.updateScene(work, { status: "generating", stage: event.stage === "starting" || event.stage === "transformerLoad" ? "preparing" : "generating", progress }, false);}private event(event: JobEvent) {@@ -727,6 +761,7 @@ export class WorkQueue {}}private async encode(work: PendingWork) {+ if (work.music) { await this.saveMusic(work); return; }if (work.image) { await this.saveImage(work); return; }this.patch(work.id, { status: "encoding", progress: 0.9, detail: "Saving video", completionAt: null });this.updateScene(work, { status: "ready", stage: "encoding", progress: 0.9, error: null }, false);@@ -770,9 +805,25 @@ export class WorkQueue {const item = this.items.find((candidate) => candidate.id === id);const session = item && this.projects.get(item.projectKey);if (!item?.needsSave || !session) return;- try { await session.save(); this.patch(id, { status: "completed", needsSave: false, error: null, detail: item.kind === "image" ? "Image saved" : "Video saved" }); }+ try { await session.save(); this.patch(id, { status: "completed", needsSave: false, error: null, detail: item.kind === "music" ? "Music saved" : item.kind === "image" ? "Image saved" : "Video saved" }); }catch (reason) { this.patch(id, { error: describeDiagnosticError(reason) }); }}+ private async saveMusic(work: PendingWork) {+ this.patch(work.id, { status: "encoding", progress: 0.9, detail: "Saving music", completionAt: null });+ try {+ const saved = await invoke<{ relativePath: string; durationMs: number }>("save_generated_music", { folderPath: work.session.record.folderPath, jobId: work.id });+ const song = work.music!, id = work.id;+ const asset: ProjectAsset = { id, kind: "audio", name: song.title, ...saved, mimeType: "audio/wav", hasAudio: true, createdAt: new Date().toISOString() };+ work.session.update((current) => ({ ...updateSong(current, song.id, { status: "completed", assetId: id, error: null }), assets: [...current.assets.filter((item) => item.id !== id), asset] }));+ try {+ await work.session.save();+ this.patch(id, { status: "completed", progress: 1, detail: "Music saved" });+ } catch (reason) {+ this.patch(id, { status: "failed", progress: 1, needsSave: true, detail: "Music created; project save failed", error: describeDiagnosticError(reason) });+ }+ } catch (reason) { this.fail(work, reason); }+ finally { await releaseRendered(work.id); work.finish(); }+ }private async saveImage(work: PendingWork) {this.patch(work.id, { status: "encoding", progress: 0.9, detail: "Saving image", completionAt: null });try {diff --git a/src/styles/index.css b/src/styles/index.cssindex ce0f7e1..446a12e 100644--- a/src/styles/index.css+++ b/src/styles/index.css@@ -24,6 +24,7 @@@import "./workspace.css";@import "./agent-dock.css";@import "./generator.css";+@import "./music.css";@import "./timeline.css";@import "./references.css";@import "./export.css";diff --git a/src/styles/music.css b/src/styles/music.cssnew file mode 100644index 0000000..d3a9a4b--- /dev/null+++ b/src/styles/music.css@@ -0,0 +1,55 @@+.music-view { display: grid; grid-template-rows: auto minmax(0, 1fr) auto; height: 100%; min-height: 0; min-width: 0; }+.music-toolbar { display: flex; align-items: center; gap: var(--space-2); padding: var(--space-2) var(--space-3); border-bottom: 1px solid var(--line); }+.music-toolbar > .ui-combo { width: 240px; max-width: 50%; }+.music-toolbar__views { display: flex; gap: var(--space-1); margin-left: auto; }+.music-toolbar__views [aria-pressed="true"] { background: var(--fill-subtle-hover); color: var(--accent); }+.music-body { display: grid; grid-template-columns: minmax(180px, 1fr) auto minmax(280px, var(--pane-music-inspector)); min-height: 0; }+.music-library { min-width: 0; overflow: auto; padding: var(--space-4); }+.music-library__heading { display: flex; align-items: center; justify-content: space-between; gap: var(--space-2); margin-bottom: var(--space-4); }+.music-library__heading h2 { margin: 0; font-size: var(--text-lg); }+.music-library__heading > span, .music-song__info > span, .music-song__status, .music-player__now span { color: var(--text-secondary); font-size: var(--text-xs); }+.music-songs { display: grid; gap: var(--space-3); }+.music-songs--grid { grid-template-columns: repeat(auto-fill, minmax(170px, 1fr)); }+.music-song { position: relative; min-width: 0; border: 1px solid var(--line); border-radius: var(--radius-lg); background: var(--surface-pane); }+.music-song--selected { border-color: var(--accent); }+.music-song__select { display: flex; flex-direction: column; gap: var(--space-2); width: 100%; height: 100%; padding: var(--space-3); border: 0; border-radius: inherit; text-align: left; color: var(--text-primary); background: transparent; }+.music-song__select:hover { background: var(--fill-subtle-hover); }+.music-song__art { display: flex; align-items: center; justify-content: center; gap: var(--space-3); width: 100%; aspect-ratio: 1; border-radius: var(--radius-sm); background: var(--fill-subtle-hover); color: var(--accent); }+.music-song__art svg { width: 48px; height: 48px; }+.music-song__art > span { font-size: var(--text-lg); font-variant-numeric: tabular-nums; }+.music-song__info { display: flex; flex-direction: column; gap: var(--space-1); min-width: 0; width: 100%; }+.music-song__info > * { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }+.music-song__status { padding-right: 28px; text-transform: capitalize; }+.music-song__play { position: absolute; right: var(--space-2); bottom: var(--space-2); }+.music-songs--list { gap: var(--space-1); }+.music-songs--list .music-song__select { flex-direction: row; align-items: center; padding-right: 48px; }+.music-songs--list .music-song__art { width: 44px; height: 44px; aspect-ratio: auto; flex: none; }+.music-songs--list .music-song__art svg { width: 20px; height: 20px; }+.music-songs--list .music-song__art > span { display: none; }+.music-songs--list .music-song__info { flex: 1; }+.music-songs--list .music-song__status { padding: 0; max-width: 30%; }+.music-songs--list .music-song__play { top: 50%; bottom: auto; transform: translateY(-50%); }+.music-empty { display: flex; flex-direction: column; align-items: center; justify-content: center; min-height: 260px; padding: var(--space-4); text-align: center; }+.music-empty > svg { width: 48px; height: 48px; color: var(--text-secondary); }+.music-empty p { max-width: 340px; color: var(--text-secondary); }+.music-inspector { display: flex; flex-direction: column; min-width: 0; min-height: 0; background: var(--surface-pane); }+.music-inspector__fields { overflow-y: auto; min-height: 0; flex: 1; }+.music-prompt { display: flex; flex-direction: column; gap: var(--space-1); color: var(--text-secondary); margin-top: var(--space-2); }+.music-prompt textarea { width: 100%; resize: vertical; }+.music-inspector__actions { display: grid; gap: var(--space-2); padding: var(--space-3); border-top: 1px solid var(--line); }+.music-inspector__actions p, .music-inspector__empty { color: var(--text-secondary); font-size: var(--text-xs); }+.music-inspector__empty { padding: var(--space-3); }+.music-inspector__actions progress { width: 100%; }+.music-player { display: grid; grid-template-columns: minmax(100px, 1fr) minmax(200px, 2fr) minmax(90px, 1fr); align-items: center; gap: var(--space-4); padding: var(--space-3) var(--space-4); border-top: 1px solid var(--line); background: var(--surface-pane); }+.music-player__now { display: flex; align-items: center; gap: var(--space-3); min-width: 0; }+.music-player__now > div { display: grid; gap: var(--space-1); min-width: 0; }+.music-player__now strong { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }+.music-player__transport { display: grid; gap: var(--space-2); }+.music-player__buttons { display: flex; justify-content: center; align-items: center; gap: var(--space-3); }+.music-player__toggle { min-width: 40px; }+.music-player__seek { display: flex; align-items: center; gap: var(--space-2); font-size: var(--text-xs); font-variant-numeric: tabular-nums; }+.music-player__seek .ui-slider { flex: 1; min-width: 0; }+.music-player__volume { display: flex; align-items: center; justify-self: end; gap: var(--space-2); width: min(130px, 100%); }+.music-player__volume .ui-slider { width: 100%; min-width: 0; }+.music-player__error { grid-column: 1 / -1; margin: 0; color: var(--text-secondary); }+.project-content--music { overflow: hidden; }