Skip to content

AI media harness for planning, generating, and editing locally

TypeScript

Default YuE2 to six DPM++ steps

Bitti24bf2100ab3c
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 83a7a0f..ac3d8cd 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -21,6 +21,7 @@ Changes since 0.3.2.
## Fixes and improvements
+- YuE2 now defaults to six flow steps with DPM++ 2M. The previous Midpoint solver remains available in music's Advanced settings, and saved generator templates using the old 32-step default upgrade once to six steps.
- On first launch after updating, each recent project gets a one-time cleanup of orphaned generated media, latents and caches. Other projects receive the cleanup when first opened; files still referenced by the project are preserved.
- Saving now deletes unused files from replaced video takes and removed video, image and music generations, including latents and derived caches. Files still used by timeline clips, references or other generations are preserved; saved deletions cannot be undone.
- Fixed a Windows file-sharing error when Minimax reads a cropped image-edit reference.
diff --git a/lib/slopfab/libslopfab.so b/lib/slopfab/libslopfab.so
index 15a38a4..472abc1 100644
--- a/lib/slopfab/libslopfab.so
+++ b/lib/slopfab/libslopfab.so
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
-oid sha256:c9528eda973597e56582f008d9c193f8249ef3a96f2b12753949bebd5c730c1a
-size 24094792
+oid sha256:ee8955ce5acf5961db2be0b5053f62f2eab97ab6572b1c29d6dd6f342a49a20e
+size 24120392
diff --git a/lib/slopfab/slopfab.dll b/lib/slopfab/slopfab.dll
index 19c760b..f4513f2 100644
--- a/lib/slopfab/slopfab.dll
+++ b/lib/slopfab/slopfab.dll
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
-oid sha256:d867c9ccf300fd3e2adb119033eb8ca2f56431ac5883c419895818065177fe03
-size 20457984
+oid sha256:bbea018ab44dfa15db947cb5688a580671a2d74fd1a5d7006ec8ac006857586d
+size 20469248
diff --git a/src-tauri/assets/agent-music.md b/src-tauri/assets/agent-music.md
index 4449cf2..65b84c6 100644
--- a/src-tauri/assets/agent-music.md
+++ b/src-tauri/assets/agent-music.md
@@ -5,9 +5,9 @@ Music is available in every open project. Use the same application-owned command
Authoring (project JSONL commands, or generation.prepare)
--------------------------------------------------------
Return one command per line followed by {"op":"commit","summary":"..."}, exactly like other project edits. Do not use a kind:commands wrapper. Inside generation.prepare, use an array of commands without a commit line.
-- music.add: {op,id,title,style,settings?,folderId?,steps?,seed?}. Unique song id; defaults: 32 steps, seed -1, root folder. folderId must exist in musicFolders. settings accepts MusicSettings below. No output/status/path fields are writable.
+- music.add: {op,id,title,style,settings?,folderId?,steps?,seed?}. Unique song id; defaults: 6 steps, seed -1, root folder. folderId must exist in musicFolders. settings accepts MusicSettings below. No output/status/path fields are writable.
- music.set: {op,id,title?,style?,settings?,folderId?,steps?,seed?}. Omitted fields stay unchanged. folderId:null moves to root. settings REPLACES settings; read music.get first and carry forward values to keep. Rejects edits while queued/generating. Editing completed song inputs preserves its audio and generation snapshot.
-- MusicSettings: lyrics (up to 30000 characters), duration (2–300 seconds, default 30), planning (full/melody/off, default full), abc (default empty), abc_max_tokens (32–4096, default 4096), temperature (0–10, default 1), top_p (>0–1, default .95), top_k (0–1024, default 0), repetition_penalty (1–10, default 1.2), semantic_guidance (0 automatic or 1–10, default 0), flow_guidance (0–10, default 0). Unspecified settings use defaults. steps: integer 2–100; seed: -1 random or nonnegative safe integer. Style must be nonempty to generate. Use blank lyrics for instrumental music.
+- MusicSettings: lyrics (up to 30000 characters), duration (2–300 seconds, default 30), planning (full/melody/off, default full), abc (default empty), abc_max_tokens (32–4096, default 4096), temperature (0–10, default 1), top_p (>0–1, default .95), top_k (0–1024, default 0), repetition_penalty (1–10, default 1.2), semantic_guidance (0 automatic or 1–10, default 0), flow_guidance (0–10, default 0). flow_sampler: default, dpmpp_2m or midpoint. YuE2 resolves default to DPM++ 2M; MiniMax Music3 requires default. Unspecified settings use defaults. steps: integer 2–100; seed: -1 random or nonnegative safe integer. Style must be nonempty to generate. Use blank lyrics for instrumental music.
Generation
----------
diff --git a/src-tauri/core/src/slopfab/music.rs b/src-tauri/core/src/slopfab/music.rs
index 13a3f0e..af6cf0b 100644
--- a/src-tauri/core/src/slopfab/music.rs
+++ b/src-tauri/core/src/slopfab/music.rs
@@ -2,6 +2,9 @@
use super::{config::Configuration, ffi::{Api, RequestHandle}, platform::ComputePlatform, types::GenerationRequest};
use serde::{Deserialize, Serialize};
+pub const YUE2_DEFAULT_STEPS: i32 = 6;
+fn default_flow_sampler(value: &String) -> bool { value == "default" }
+
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(default, deny_unknown_fields)]
pub struct MusicSettings {
@@ -16,16 +19,19 @@ pub struct MusicSettings {
pub repetition_penalty: f64,
pub semantic_guidance: f64,
pub flow_guidance: f64,
+ #[serde(skip_serializing_if = "default_flow_sampler")]
+ pub flow_sampler: String,
}
impl Default for MusicSettings {
fn default() -> Self { Self { lyrics: String::new(), duration: 30.0, planning: "full".into(), abc: String::new(),
abc_max_tokens: 4096, temperature: 1.0, top_p: 0.95, top_k: 0, repetition_penalty: 1.2,
- semantic_guidance: 0.0, flow_guidance: 0.0 } }
+ semantic_guidance: 0.0, flow_guidance: 0.0, flow_sampler: "default".into() } }
}
impl MusicSettings {
pub fn validate(&self) -> Result<(), String> {
if !self.duration.is_finite() || !(2.0..=300.0).contains(&self.duration)
|| !matches!(self.planning.as_str(), "full" | "melody" | "off")
+ || !matches!(self.flow_sampler.as_str(), "default" | "midpoint" | "dpmpp_2m")
|| !(32..=4096).contains(&self.abc_max_tokens) || !(0..=1024).contains(&self.top_k)
|| !self.temperature.is_finite() || !(0.0..=10.0).contains(&self.temperature)
|| !self.top_p.is_finite() || self.top_p <= 0.0 || self.top_p > 1.0
@@ -60,7 +66,7 @@ pub(super) fn validate(request: &GenerationRequest, config: &Configuration) -> R
let defaults = MusicSettings::default();
if config.model_type == "minimax_music3" && (settings.planning != defaults.planning || !settings.abc.is_empty()
|| settings.abc_max_tokens != defaults.abc_max_tokens || settings.temperature != defaults.temperature
- || settings.top_p != defaults.top_p || settings.repetition_penalty != defaults.repetition_penalty) {
+ || settings.top_p != defaults.top_p || settings.repetition_penalty != defaults.repetition_penalty || settings.flow_sampler != "default") {
return Err("MiniMax Music3 does not support YuE2 planning and sampling controls.".into());
}
if config.model_type == "yue2" && settings.flow_guidance != 0.0 { return Err("Flow guidance is available for MiniMax Music3 only.".into()); }
@@ -71,7 +77,9 @@ pub(super) fn configure(api: &Api, handle: &RequestHandle, request: &GenerationR
validate(request, config)?;
api.set_model_family(handle, &config.model_type)?;
api.set_prompt(handle, &request.prompt)?;
- api.set_music_settings(handle, &serde_json::to_string(request.music.as_ref().unwrap()).map_err(|e| e.to_string())?)?;
+ let mut settings = request.music.as_ref().unwrap().clone();
+ if config.model_type == "yue2" && settings.flow_sampler == "default" { settings.flow_sampler = "dpmpp_2m".into(); }
+ api.set_music_settings(handle, &serde_json::to_string(&settings).map_err(|e| e.to_string())?)?;
api.set_steps(handle, request.steps)?;
api.set_seed(handle, super::planning::generation_seed(request.seed)?)?;
api.set_inference_backend(handle, platform.backend())?;
diff --git a/src-tauri/core/src/slopfab/tests.rs b/src-tauri/core/src/slopfab/tests.rs
index b68197b..9c37020 100644
--- a/src-tauri/core/src/slopfab/tests.rs
+++ b/src-tauri/core/src/slopfab/tests.rs
@@ -30,6 +30,32 @@ fn music_plans_on_cuda_and_vulkan_without_video_controls() {
}
}
+#[test]
+fn yue_fast_flow_sampler_reaches_native_planning_and_generation() {
+ let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../lib/slopfab").join(super::DLL_FILE_NAME);
+ let api = ffi::Api::load(&path).unwrap();
+ let mut config = Configuration::from_settings(&BTreeMap::new());
+ config.model_type = "yue2".into();
+ for backend in [ComputePlatform::Cuda13, ComputePlatform::Vulkan] {
+ for purpose in [RequestPurpose::Plan, RequestPurpose::Generate] {
+ for (sampler, resolved, steps, evaluations) in [("default", "dpmpp_2m", 6, 6), ("dpmpp_2m", "dpmpp_2m", 6, 6), ("midpoint", "midpoint", 32, 64)] {
+ let request = GenerationRequest { prompt: "Quiet piano".into(), steps, seed: 42,
+ music: Some(music::MusicSettings { duration: 2.0, flow_sampler: sampler.into(), ..Default::default() }), ..Default::default() };
+ let handle = RequestHandle::new(&api).unwrap();
+ configure_request(&api, &handle, &request, &config, backend, purpose, &ReferenceVideos::default()).unwrap();
+ let description = api.describe(&handle).unwrap();
+ assert!(description.contains(&format!("{steps} ({resolved})")), "{description}");
+ assert_eq!(api.resolve(&handle).unwrap().num_model_evaluations, evaluations);
+ }
+ }
+ }
+ config.model_type = "minimax_music3".into();
+ let request = GenerationRequest { prompt: "Quiet piano".into(), steps: 30, seed: 42,
+ music: Some(music::MusicSettings { flow_sampler: "dpmpp_2m".into(), ..Default::default() }), ..Default::default() };
+ assert!(config.validate_inputs(&request).unwrap_err().contains("MiniMax Music3"));
+ assert!(serde_json::from_value::<music::MusicSettings>(serde_json::json!({"flow_sampler":"dpm_2"})).unwrap().validate().is_err());
+}
+
#[test]
fn sampler_and_scheduler_options_reach_planning_and_generation_and_reset() {
let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../lib/slopfab").join(super::DLL_FILE_NAME);
diff --git a/src-tauri/src/project/commands/music.rs b/src-tauri/src/project/commands/music.rs
index 0c63176..4fb3fed 100644
--- a/src-tauri/src/project/commands/music.rs
+++ b/src-tauri/src/project/commands/music.rs
@@ -16,7 +16,7 @@ pub(super) fn apply(project: &mut ProjectConfig, command: &ProjectCommand, times
ProjectCommand::MusicAdd { id, title, style, settings, folder_id, steps, seed } => {
if id.trim().is_empty() || project.music_songs.iter().any(|s| s.id == *id) { return Err("Song requires a unique id.".into()); }
project.music_songs.push(MusicSong { id: id.clone(), title: title.clone(), style: style.clone(), settings: settings.clone(),
- folder_id: folder_id.clone(), steps: steps.unwrap_or(32), seed: seed.unwrap_or(-1), generator_id: String::new(),
+ folder_id: folder_id.clone(), steps: steps.unwrap_or(slopus_core::slopfab::music::YUE2_DEFAULT_STEPS), seed: seed.unwrap_or(-1), generator_id: String::new(),
status: "draft".into(), asset_id: None, cover_relative_path: None, error: None, generation: None, created_at: timestamp.into() });
}
ProjectCommand::MusicSet { id, title, style, settings, folder_id, steps, seed } => {
@@ -159,7 +159,7 @@ mod tests {
let p: ProjectConfig = serde_json::from_str(include_str!("../../../../fixtures/project-v1-complete.json")).unwrap();
let added = run(&p,json!([{"op":"music.add","id":"song","title":"Intro","style":"Jazz","settings":{"lyrics":"Hello"}}])).unwrap();
assert_eq!(added.music_songs[0].settings.lyrics,"Hello");
- assert_eq!(added.music_songs[0].steps,32);
+ assert_eq!(added.music_songs[0].steps,6);
assert!(run(&added,json!([{"op":"music.set","id":"song","title":"Changed"},{"op":"music.set","id":"song","steps":1000}])).is_err());
assert_eq!(added.music_songs[0].title,"Intro");
assert!(run(&p,json!([{"op":"music.add","id":"song","title":"Intro","style":"Jazz","folderId":"missing"}])).is_err());
diff --git a/src/components/SettingsView.tsx b/src/components/SettingsView.tsx
index c182a1c..4c5acc1 100644
--- a/src/components/SettingsView.tsx
+++ b/src/components/SettingsView.tsx
@@ -602,7 +602,7 @@ export function SettingsView({ onClose, updates, initialTab = "engine", titleBar
<ComboBox aria-label="Generator model" value={selectedTemplate.modelType} options={[
...GENERATOR_MODELS,
...(!GENERATOR_MODELS.some(({ value }) => value === selectedTemplate.modelType) ? [{ value: selectedTemplate.modelType, label: selectedTemplate.modelType }] : []),
- ]} onChange={(modelType) => updateTemplate({ modelType, editingModel: generatorModel(modelType).imageEditing, kreaVariant: undefined, sampler: undefined, scheduler: undefined, ...(modelType === MOSS_MODEL_TYPE || modelType === FIBO_MODEL_TYPE || modelType === KREA_MODEL_TYPE || generatorModel(modelType).musicGeneration ? { mode: "prompt", attention: "flash2", defaultSteps: modelType === KREA_MODEL_TYPE ? 8 : modelType === "yue2" ? 32 : modelType === "minimax_music3" ? 30 : modelType === FIBO_MODEL_TYPE ? 4 : 50, motionCache: false, loras: [], audioSigmaShift: undefined, ...(modelType === FIBO_MODEL_TYPE || generatorModel(modelType).musicGeneration ? { videoSigmaShift: undefined } : {}) } : {}) })} />
+ ]} onChange={(modelType) => updateTemplate({ modelType, editingModel: generatorModel(modelType).imageEditing, kreaVariant: undefined, sampler: undefined, scheduler: undefined, ...(modelType === MOSS_MODEL_TYPE || modelType === FIBO_MODEL_TYPE || modelType === KREA_MODEL_TYPE || generatorModel(modelType).musicGeneration ? { mode: "prompt", attention: "flash2", defaultSteps: modelType === KREA_MODEL_TYPE ? 8 : modelType === "yue2" ? 6 : modelType === "minimax_music3" ? 30 : modelType === FIBO_MODEL_TYPE ? 4 : 50, motionCache: false, loras: [], audioSigmaShift: undefined, ...(modelType === FIBO_MODEL_TYPE || generatorModel(modelType).musicGeneration ? { videoSigmaShift: undefined } : {}) } : {}) })} />
</SettingsCard>
{selectedTemplate.modelType === KREA_MODEL_TYPE && <SettingsCard icon={<Steps20 />} header="Sampling defaults" description="Match the recipe to your transformer weights. Raw uses CFG 3.5 and a resolution-dependent sigma shift; Turbo uses CFG 0.">
<ComboBox aria-label="Krea sampling defaults" value={selectedTemplate.kreaVariant ?? "turbo"}
diff --git a/src/components/workspace/MusicView.test.tsx b/src/components/workspace/MusicView.test.tsx
index 40256dd..be67181 100644
--- a/src/components/workspace/MusicView.test.tsx
+++ b/src/components/workspace/MusicView.test.tsx
@@ -4,6 +4,7 @@ import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { cleanup, fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { useState } from "react";
import { MusicView } from "./MusicView";
+import { choose, comboValue } from "./comboTestUtils";
import { createProjectConfig, type ProjectConfig } from "../../lib/project";
import { newMusicFolder, newSong, newSongVariation } from "../../lib/music";
import { readMediaFileUrl } from "../../lib/persistence";
@@ -54,6 +55,19 @@ it.each(["Grid", "List"])("clears song selection on empty %s space and allows se
expect(screen.getByLabelText("Song title")).toHaveValue("First");
});
+it("starts YuE2 at six steps and lets the song select the Midpoint baseline", () => {
+ render(<Harness />);
+ fireEvent.click(within(screen.getByRole("toolbar", { name: "Music tools" })).getByRole("button", { name: "New song" }));
+ expect(latest.musicSongs![0]).toMatchObject({ steps: 6, settings: { flow_sampler: "dpmpp_2m" } });
+ fireEvent.click(screen.getByRole("button", { name: "Advanced" }));
+ expect(comboValue(screen.getByRole("combobox", { name: "Music flow sampler" }))).toBe("dpmpp_2m");
+ choose("Music flow sampler", "Midpoint");
+ expect(latest.musicSongs![0].settings.flow_sampler).toBe("midpoint");
+ const saved = latest;
+ cleanup(); render(<Harness initial={saved} />);
+ expect(latest.musicSongs![0].settings.flow_sampler).toBe("midpoint");
+});
+
it("generates artwork for the context-menu song and displays saved covers in both views", async () => {
const songs = [{ ...newSong(), title: "First" }, { ...newSong(), title: "Second", coverRelativePath: "media/generated/cover.jpg" }];
render(<Harness initial={{ ...createConfig(), musicSongs: songs }} />);
diff --git a/src/components/workspace/MusicView.tsx b/src/components/workspace/MusicView.tsx
index 49cb7ca..d625d84 100644
--- a/src/components/workspace/MusicView.tsx
+++ b/src/components/workspace/MusicView.tsx
@@ -3,7 +3,7 @@ import { loadActiveGeneratorId, saveActiveGeneratorId, selectedGeneratorTemplate
import type { ProjectAsset, ProjectConfig } from "../../lib/project";
import { isTauri, readMediaFileUrl } from "../../lib/persistence";
import { loadGeneratorTemplateSettings, subscribeGeneratorTemplates, templateSupportsMusic, templateUsable, generatorPathFields, type GeneratorTemplate } from "../../lib/settings";
-import { musicSettingsForModel, moveSong, newMusicFolder, newSong, newSongVariation, removeMusicFolder, songReference, updateSong, type MusicSettings, type MusicSong } from "../../lib/music";
+import { YUE2_DEFAULT_STEPS, musicSettingsForModel, moveSong, newMusicFolder, newSong, newSongVariation, removeMusicFolder, songReference, updateSong, type MusicSettings, type MusicSong } from "../../lib/music";
import { isWorkActive, type WorkItem } from "../../lib/workQueue";
import { ComboBox, CommandBarButton, InfoBar, ProgressBar, ProgressRing, PropRow, PropSection, Splitter, useContextMenu, usePaneSize, type MenuEntry } from "../ui";
import { Add16, Audio16, Delete16, Edit16, FolderAdd16, FolderOpen16, GridView16, ListView16, Pause16, Play16, References16, Sparkle16, Stop14 } from "../ui/icons";
@@ -79,7 +79,8 @@ export function MusicView({ config, folderPath, onChange, onGenerate, onGenerate
};
const setting = (patch: Partial<MusicSettings>) => { if (selected) edit({ settings: { ...selected.settings, ...patch } }); };
const add = () => {
- const song = newSong(template?.id, Math.min(100, template?.defaultSteps ?? 32), currentFolderId);
+ const song = newSong(template?.id, Math.min(100, template?.defaultSteps ?? YUE2_DEFAULT_STEPS), currentFolderId);
+ if (template) song.settings = musicSettingsForModel(song.settings, template.modelType);
onChange((current) => ({ ...current, musicSongs: [...(current.musicSongs ?? []), song] }));
setSelectedId(song.id); setError(null);
};
@@ -315,6 +316,9 @@ export function MusicView({ config, folderPath, onChange, onGenerate, onGenerate
<PropRow label="Seed" tooltip="Use -1 for a random seed."><input className="text-field" aria-label="Music seed" type="number" min={-1} max={Number.MAX_SAFE_INTEGER} step={1} value={selected.seed} disabled={busy} onChange={(event) => { const seed = event.target.valueAsNumber; if (Number.isSafeInteger(seed) && seed >= -1) edit({ seed }); }} /></PropRow>
</PropSection>
<PropSection title="Advanced" persistKey="music.advanced" defaultOpen={false}>
+ {template?.modelType === "yue2" && <PropRow label="Flow sampler"><ComboBox aria-label="Music flow sampler" value={musicSettingsForModel(selected.settings, "yue2").flow_sampler} disabled={busy}
+ options={[{ value: "dpmpp_2m", label: "DPM++ 2M" }, { value: "midpoint", label: "Midpoint" }]}
+ onChange={(flow_sampler) => setting({ flow_sampler: flow_sampler as MusicSettings["flow_sampler"] })} /></PropRow>}
{template?.modelType === "yue2" && <PropRow label="Planning"><ComboBox aria-label="Music planning" value={selected.settings.planning} disabled={busy} options={[{ value: "full", label: "Full (melody and chords)" }, { value: "melody", label: "Melody" }, { value: "off", label: "Off" }]} onChange={(planning) => setting({ planning: planning as MusicSettings["planning"], ...(planning === "off" ? { abc: "" } : {}) })} /></PropRow>}
{([
["top_k", "Top K", 0, 1024, 1], ["semantic_guidance", "Guidance", 0, 5, 1],
diff --git a/src/lib/music.test.ts b/src/lib/music.test.ts
index 22486df..0becc2c 100644
--- a/src/lib/music.test.ts
+++ b/src/lib/music.test.ts
@@ -3,6 +3,16 @@ import { createProjectConfig, parseProjectConfig, type ProjectAsset } from "./pr
import { albumCoverPrompt, moveSong, musicFolderLabel, musicSettingsForModel, musicSettingsSchema, newMusicFolder, newSong, songReference } from "./music";
describe("music projects", () => {
+ it("defaults new YuE2 songs to six DPM++ 2M steps and preserves explicit baselines", () => {
+ expect(newSong()).toMatchObject({ steps: 6, settings: { flow_sampler: "dpmpp_2m" } });
+ expect(newSong("minimax_music3")).toMatchObject({ steps: 30, settings: { flow_sampler: "default" } });
+ expect(newSong("yue2", 32).steps).toBe(32);
+ expect(musicSettingsForModel(musicSettingsSchema.parse({}), "yue2").flow_sampler).toBe("dpmpp_2m");
+ const baseline = musicSettingsSchema.parse({ flow_sampler: "midpoint" });
+ expect(musicSettingsForModel(baseline, "yue2").flow_sampler).toBe("midpoint");
+ expect(musicSettingsForModel(baseline, "minimax_music3").flow_sampler).toBe("default");
+ expect(musicSettingsSchema.safeParse({ flow_sampler: "dpm_2" }).success).toBe(false);
+ });
it("persists folders and song moves without moving or rewriting generated audio", () => {
const parent = newMusicFolder("Jazz", null), child = newMusicFolder("Piano", parent.id);
const song = { ...newSong("yue2", 32, parent.id), assetId: "audio-1" };
diff --git a/src/lib/music.ts b/src/lib/music.ts
index af4b7a6..cabce4e 100644
--- a/src/lib/music.ts
+++ b/src/lib/music.ts
@@ -1,6 +1,8 @@
import { z } from "zod";
import type { ProjectAsset, ProjectConfig, ProjectReference } from "./project";
+export const YUE2_DEFAULT_STEPS = 6;
+
export const musicSettingsSchema = z.object({
lyrics: z.string().max(30000).default(""),
duration: z.number().finite().min(2).max(300).default(30),
@@ -13,6 +15,7 @@ export const musicSettingsSchema = z.object({
repetition_penalty: z.number().finite().min(1).max(10).default(1.2),
semantic_guidance: z.number().finite().min(0).max(10).default(0),
flow_guidance: z.number().finite().min(0).max(10).default(0),
+ flow_sampler: z.enum(["default", "midpoint", "dpmpp_2m"]).default("default"),
}).refine((value) => value.semantic_guidance === 0 || value.semantic_guidance >= 1, "Semantic guidance must be zero (automatic) or at least one.")
.refine((value) => value.planning !== "off" || !value.abc, "ABC requires full or melody planning.");
export const musicSongSchema = z.object({
@@ -21,7 +24,7 @@ export const musicSongSchema = z.object({
style: z.string().max(10000).default(""),
generatorId: z.string().default("yue2"),
settings: musicSettingsSchema.default({}),
- steps: z.number().int().min(2).max(100).default(32),
+ steps: z.number().int().min(2).max(100).default(YUE2_DEFAULT_STEPS),
seed: z.number().int().min(-1).max(Number.MAX_SAFE_INTEGER).default(-1),
status: z.enum(["draft", "queued", "generating", "completed", "failed", "cancelled"]).default("draft"),
assetId: z.string().nullish(), error: z.string().nullish(),
@@ -48,8 +51,10 @@ export function albumCoverPrompt(song: MusicSong, modelType: string): string {
export const musicFolderSchema = z.object({ id: z.string().min(1), name: z.string().trim().min(1).max(200), parentId: z.string().min(1).nullish() });
export type MusicFolder = z.infer<typeof musicFolderSchema>;
-export function newSong(generatorId = "yue2", steps = 32, folderId?: string | null): MusicSong {
- return musicSongSchema.parse({ id: `song-${crypto.randomUUID()}`, title: "New song", generatorId, steps, ...(folderId ? { folderId } : {}), createdAt: new Date().toISOString() });
+export function newSong(generatorId = "yue2", steps = generatorId === "minimax_music3" ? 30 : YUE2_DEFAULT_STEPS, folderId?: string | null): MusicSong {
+ return musicSongSchema.parse({ id: `song-${crypto.randomUUID()}`, title: "New song", generatorId, steps,
+ settings: { flow_sampler: generatorId === "minimax_music3" ? "default" : "dpmpp_2m" },
+ ...(folderId ? { folderId } : {}), createdAt: new Date().toISOString() });
}
export function newSongVariation(source: MusicSong, songs: readonly MusicSong[]): MusicSong {
@@ -95,7 +100,8 @@ export function musicFolderLabel(folders: readonly MusicFolder[], id: string): s
export function musicSettingsForModel(settings: MusicSettings, modelType: string): MusicSettings {
const defaults = musicSettingsSchema.parse({});
return modelType === "minimax_music3" ? { ...defaults, lyrics: settings.lyrics, duration: settings.duration,
- top_k: settings.top_k, semantic_guidance: settings.semantic_guidance, flow_guidance: settings.flow_guidance } : { ...settings, flow_guidance: 0 };
+ top_k: settings.top_k, semantic_guidance: settings.semantic_guidance, flow_guidance: settings.flow_guidance }
+ : { ...settings, flow_guidance: 0, flow_sampler: settings.flow_sampler === "midpoint" ? "midpoint" : "dpmpp_2m" };
}
export function songReference(song: MusicSong, asset: ProjectAsset): ProjectReference {
diff --git a/src/lib/settings.test.ts b/src/lib/settings.test.ts
index 4532592..079ff66 100644
--- a/src/lib/settings.test.ts
+++ b/src/lib/settings.test.ts
@@ -78,7 +78,7 @@ describe("generator templates", () => {
singularity.sources!.transformer = [singularity.sources!.transformer![0]];
saveGeneratorTemplateSettings({ templates: [original, fast, singularity], defaultTemplateId: "", catalogVersion: 20 });
const migrated = loadGeneratorTemplateSettings();
- expect(migrated.catalogVersion).toBe(21);
+ expect(migrated.catalogVersion).toBe(22);
expect(migrated.templates.map(({ id }) => id)).toEqual([original.id, fast.id, singularity.id, "minimax-h3-singularity-v2"]);
expect(migrated.templates[0]).toMatchObject({ name: original.name, defaultSteps: 27, paths: { transformer: cached },
sources: { transformer: [original.sources!.transformer![0], { ...original.sources!.transformer![1], minVramGb: 13 }, custom, minimaxOriginalTemplate().sources!.transformer![2]] } });
@@ -168,7 +168,7 @@ describe("generator templates", () => {
sources: { ...template.sources, transformer: [{ ...int8, downloadedPath: cached }, customFp16, custom] } };
saveGeneratorTemplateSettings({ templates: [previous], defaultTemplateId: previous.id, catalogVersion: 18 });
const upgraded = loadGeneratorTemplateSettings();
- expect(upgraded.catalogVersion).toBe(21);
+ expect(upgraded.catalogVersion).toBe(22);
expect(upgraded.templates[0]).toMatchObject({ name: "My music", paths: { transformer: cached },
sources: { transformer: [fp32, { ...int8, downloadedPath: cached }, customFp16, custom] } });
expect(loadGeneratorTemplateSettings()).toEqual(upgraded);
@@ -182,6 +182,18 @@ describe("generator templates", () => {
expect(loadGeneratorTemplateSettings().templates[0].sources!.transformer).toEqual([custom]);
});
+ it("upgrades YuE2's old step default once and leaves custom steps and Music3 unchanged", () => {
+ const yue = { ...musicTemplates()[0], defaultSteps: 32 };
+ const custom = { ...yue, id: "custom-yue", defaultSteps: 12 };
+ const music3 = musicTemplates()[1];
+ saveGeneratorTemplateSettings({ templates: [yue, custom, music3], defaultTemplateId: yue.id, catalogVersion: 21 });
+ const updated = loadGeneratorTemplateSettings();
+ expect(updated.catalogVersion).toBe(22);
+ expect(updated.templates.map(template => template.defaultSteps)).toEqual([6, 12, 30]);
+ saveGeneratorTemplateSettings({ ...updated, templates: updated.templates.map(template => ({ ...template, defaultSteps: 32 })) });
+ expect(loadGeneratorTemplateSettings().templates.every(template => template.defaultSteps === 32)).toBe(true);
+ });
+
it("upgrades existing YuE2 sources once while preserving downloaded and custom choices", () => {
const template = musicTemplates()[0];
const int8 = template.sources!.transformer![1];
@@ -190,7 +202,7 @@ describe("generator templates", () => {
sources: { transformer: [{ ...int8, downloadedPath: cached }] } };
saveGeneratorTemplateSettings({ templates: [previous], defaultTemplateId: previous.id, catalogVersion: 16 });
const upgraded = loadGeneratorTemplateSettings();
- expect(upgraded.catalogVersion).toBe(21);
+ expect(upgraded.catalogVersion).toBe(22);
expect(upgraded.templates[0]).toMatchObject({ name: "My music", paths: { transformer: cached },
sources: { transformer: [template.sources!.transformer![0], { ...int8, downloadedPath: cached }] } });
expect(loadGeneratorTemplateSettings()).toEqual(upgraded);
@@ -395,7 +407,7 @@ describe("generator templates", () => {
expect(generatorOwned(settings.templates[2])).toEqual(generatorOwned(minimaxFastTemplate()));
expect(generatorOwned(settings.templates[3])).toEqual(generatorOwned(minimaxSingularityTemplate()));
expect(settings.defaultTemplateId).toBe(template.id);
- expect(settings.catalogVersion).toBe(21);
+ expect(settings.catalogVersion).toBe(22);
expect(loadGeneratorTemplateSettings()).toEqual(settings);
saveGeneratorTemplateSettings({ ...settings, templates: [settings.templates[0]] });
expect(loadGeneratorTemplateSettings().templates).toHaveLength(1);
@@ -416,7 +428,7 @@ describe("generator templates", () => {
const settings = loadGeneratorTemplateSettings();
expect(generatorOwned(settings.templates)).toEqual(generatorOwned([custom, minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates(), ...kreaImageTemplates(), minimaxSingularityV2Template()]));
expect(settings.defaultTemplateId).toBe(custom.id);
- expect(settings.catalogVersion).toBe(21);
+ expect(settings.catalogVersion).toBe(22);
expect(loadGeneratorTemplateSettings()).toEqual(settings);
saveGeneratorTemplateSettings({ ...settings, templates: [custom] });
expect(generatorOwned(loadGeneratorTemplateSettings().templates)).toEqual(generatorOwned([custom]));
@@ -481,7 +493,7 @@ describe("generator templates", () => {
localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 8 }));
const settings = loadGeneratorTemplateSettings();
expect(generatorOwned(settings.templates)).toEqual(generatorOwned([minimaxFastTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates(), ...kreaImageTemplates(), minimaxSingularityV2Template()]));
- expect(settings.catalogVersion).toBe(21);
+ expect(settings.catalogVersion).toBe(22);
expect(templateNeedsDownload(settings.templates[0])).toBe(true);
expect(loadGeneratorTemplateSettings()).toEqual(settings);
});
@@ -496,7 +508,7 @@ describe("generator templates", () => {
const otherPaths = { ...template.paths };
localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 10 }));
const settings = loadGeneratorTemplateSettings();
- expect(settings.catalogVersion).toBe(21);
+ expect(settings.catalogVersion).toBe(22);
expect(settings.templates).toHaveLength(8);
expect(settings.templates[0]).toMatchObject({ id: template.id, name: "FastH3 Omni", defaultSteps: 8,
paths: { ...otherPaths, transformer: minimaxFastTemplate().paths.transformer },
@@ -592,7 +604,7 @@ describe("generator templates", () => {
template.paths[field] = downloadedPath;
localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], catalogVersion: 1 }));
const settings = loadGeneratorTemplateSettings();
- expect(settings.catalogVersion).toBe(21);
+ expect(settings.catalogVersion).toBe(22);
expect(settings.templates[0].paths[field]).toBe(downloadedPath);
expect(settings.templates[0].sources![field]).toEqual(minimaxOriginalTemplate().sources![field]!.map((source, index) =>
index === 0 ? { ...source, downloadedPath } : source));
@@ -627,7 +639,7 @@ describe("generator templates", () => {
localStorage.setItem("slopus.generator-templates.v1", JSON.stringify({ templates: [template], defaultTemplateId: template.id, catalogVersion: 2 }));
const settings = loadGeneratorTemplateSettings();
const migrated = settings.templates[0];
- expect(settings.catalogVersion).toBe(21);
+ expect(settings.catalogVersion).toBe(22);
expect(migrated.sources!.textEncoder).toEqual([
{ url: originalUrl, gpuModel: "", minVramGb: 16, ...(selection === "cached original" ? { downloadedPath: cachedOriginal } : {}) },
custom,
diff --git a/src/lib/settings.ts b/src/lib/settings.ts
index 69b41de..011686e 100644
--- a/src/lib/settings.ts
+++ b/src/lib/settings.ts
@@ -15,6 +15,7 @@
import { isSigmaShift, isSampler, isScheduler, type SigmaShifts, type SamplingChoices, type Sampler, type Scheduler } from "./sampling";
import { initialH3Components, normalizeH3Library, resolveH3Components, type H3Components } from "./h3Components";
import { generatorModel } from "./generatorModels";
+import { YUE2_DEFAULT_STEPS } from "./music";
export { generatorModel, GENERATOR_MODELS } from "./generatorModels";
import { latentUpscalerPath, otherWeightPaths, TAEH3_URL } from "./upscalers";
import { loadGenerationPreviewsEnabled } from "./generationPreviewSettings";
@@ -139,6 +140,7 @@ export function templateSamplingOptions(template: Partial<GeneratorTemplate>, ov
const reason = preset ? "The active LoRA recipe controls the sampler and scheduler."
: omni ? "FastH3 Omni requires Euler and its trained schedule."
: template.mode === "animate" ? "Animate requires Euler. Adapters may also require a fixed schedule."
+ : template.modelType === "yue2" ? "Choose the flow sampler in the song's Advanced settings."
: model.musicGeneration ? "This music model uses its built-in solver and schedule." : undefined;
return { samplers, schedulers, reason };
}
@@ -282,7 +284,7 @@ export function createGeneratorTemplate(name = "New template", modelType = MINIM
name,
modelType,
editingModel: generatorModel(modelType).imageEditing,
- defaultSteps: modelType === KREA_MODEL_TYPE ? 8 : modelType === "yue2" ? 32 : modelType === "minimax_music3" ? 30 : modelType === MOSS_MODEL_TYPE ? 50 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS,
+ defaultSteps: modelType === KREA_MODEL_TYPE ? 8 : modelType === "yue2" ? YUE2_DEFAULT_STEPS : modelType === "minimax_music3" ? 30 : modelType === MOSS_MODEL_TYPE ? 50 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS,
attention: modelType === KREA_MODEL_TYPE || generatorModel(modelType).cudaOnly || generatorModel(modelType).musicGeneration ? "flash2" : "sage2",
paths: copyPaths(EMPTY_ENGINE_SETTINGS),
};
@@ -449,7 +451,7 @@ export function viggleAnimateTemplate(): GeneratorTemplate {
const initialTemplateSettings = (paths = EMPTY_ENGINE_SETTINGS): GeneratorTemplateSettings => normalizeTemplateSettings({
templates: [{ id: "default", name: "Default", modelType: MINIMAX_H3_MODEL_TYPE, defaultSteps: DEFAULT_GENERATION_STEPS, attention: "sage2", paths: copyPaths(paths) }, minimaxOriginalTemplate(), minimaxReferencesTemplate(), minimaxFastTemplate(), minimaxSingularityTemplate(), viggleAnimateTemplate(), mossImageTemplate(), fiboImageTemplate(), fiboBaseTemplate(), ...musicTemplates(), ...kreaImageTemplates(), minimaxSingularityV2Template()],
defaultTemplateId: "default",
- catalogVersion: 21,
+ catalogVersion: 22,
})!;
const normalizeTemplateSettings = (value: unknown, shareComponents = true): GeneratorTemplateSettings | null => {
@@ -470,7 +472,7 @@ const normalizeTemplateSettings = (value: unknown, shareComponents = true): Gene
&& candidate.defaultSteps >= 2
&& candidate.defaultSteps <= MAX_GENERATION_STEPS
? candidate.defaultSteps
- : modelType === KREA_MODEL_TYPE ? candidate.kreaVariant === "raw" ? 52 : 8 : modelType === "yue2" ? 32 : modelType === "minimax_music3" ? 30 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS;
+ : modelType === KREA_MODEL_TYPE ? candidate.kreaVariant === "raw" ? 52 : 8 : modelType === "yue2" ? YUE2_DEFAULT_STEPS : modelType === "minimax_music3" ? 30 : modelType === FIBO_MODEL_TYPE ? 4 : DEFAULT_GENERATION_STEPS;
const attention = candidate.attention === "exact" || candidate.attention === "flash2" || candidate.attention === "sol" ? candidate.attention : "sage2";
const paths = pathsFrom(candidate.paths);
const sources: GeneratorTemplate["sources"] = {};
@@ -519,7 +521,7 @@ const normalizeTemplateSettings = (value: unknown, shareComponents = true): Gene
...shared,
defaultTemplateId: !shareComponents ? requestedDefault : shared.templates.find((template) => template.id === requestedDefault && templateUsable(template))?.id
?? shared.templates.find(templateUsable)?.id ?? "",
- catalogVersion: typeof record.catalogVersion === "number" && [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21].includes(record.catalogVersion) ? record.catalogVersion : 0,
+ catalogVersion: typeof record.catalogVersion === "number" && [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22].includes(record.catalogVersion) ? record.catalogVersion : 0,
};
};
@@ -795,6 +797,12 @@ export function loadGeneratorTemplateSettings(): GeneratorTemplateSettings {
normalized = normalizeTemplateSettings({ ...normalized, catalogVersion: 21 }, false)!;
localStorage.setItem(GENERATOR_TEMPLATES_KEY, JSON.stringify(normalized));
}
+ if ((normalized.catalogVersion ?? 0) < 22) {
+ normalized.templates = normalized.templates.map(template => template.modelType === "yue2" && template.defaultSteps === 32
+ ? { ...template, defaultSteps: YUE2_DEFAULT_STEPS } : template);
+ normalized = normalizeTemplateSettings({ ...normalized, catalogVersion: 22 }, false)!;
+ localStorage.setItem(GENERATOR_TEMPLATES_KEY, JSON.stringify(normalized));
+ }
const shared = normalizeTemplateSettings(normalized)!;
if (!normalized.h3Components) localStorage.setItem(GENERATOR_TEMPLATES_KEY, JSON.stringify(shared));
return shared;
diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.ts
index a6072c8..ddfd41d 100644
--- a/src/lib/workQueue.test.ts
+++ b/src/lib/workQueue.test.ts
@@ -450,19 +450,40 @@ it("queues an immutable music request and saves its audio into timeline media",
first.update((config) => ({ ...config, musicSongs: config.musicSongs!.map((item) => ({ ...item, style: "Changed later" })) }));
await waitFor(() => expect(enqueueSlopfabGeneration).toHaveBeenCalledOnce());
const request = vi.mocked(enqueueSlopfabGeneration).mock.calls[0][0];
- expect(request).toMatchObject({ prompt: "Soft jazz", seed: 42, steps: 32, music: { duration: 30 }, referencePaths: [] });
+ expect(request).toMatchObject({ prompt: "Soft jazz", seed: 42, steps: 6, music: { duration: 30, flow_sampler: "dpmpp_2m" }, referencePaths: [] });
expect(request).not.toHaveProperty("saveLatentsPath");
const id = queue.getSnapshot()[0].id;
await finish(queue, id);
expect(invoke).toHaveBeenCalledWith("save_generated_music", { folderPath: first.record.folderPath, jobId: id });
expect(first.getSnapshot().config.musicSongs![0]).toMatchObject({ status: "completed", assetId: id, style: "Changed later",
- generation: { style: "Soft jazz", seed: 42, steps: 32, generatorId: template.id, settings: { lyrics: "", duration: 30 } } });
+ generation: { style: "Soft jazz", seed: 42, steps: 6, generatorId: template.id, settings: { lyrics: "", duration: 30, flow_sampler: "dpmpp_2m" } } });
expect(first.getSnapshot().config.assets).toContainEqual(expect.objectContaining({ id, kind: "audio", name: "Evening", relativePath: "media/generated/song.wav", durationMs: 30000 }));
expect(saved).toHaveBeenCalled();
expect(releaseRendered).toHaveBeenCalledWith(id);
expect(saveGeneratedScene).not.toHaveBeenCalled();
});
+it.each(["yue2", "minimax_music3"])("records the actual flow sampler sent for %s", async modelType => {
+ const { queue, first } = setup();
+ const song = { ...newSong(), style: "Jazz", steps: 32 };
+ song.settings.flow_sampler = "midpoint";
+ first.update(config => ({ ...config, musicSongs: [song] }));
+ const template = musicTemplates().find(template => template.modelType === modelType)!;
+ template.sources = {};
+ template.paths.transformer = "C:/weights/music.safetensors";
+ if (modelType === "minimax_music3") {
+ template.paths.textEncoder = "C:/weights/text.safetensors";
+ template.paths.audioVae = "C:/weights/vae.safetensors";
+ }
+ vi.mocked(invoke).mockResolvedValue({ relativePath: "media/generated/song.wav", durationMs: 30000 });
+ queue.enqueueMusic(first, song.id, template);
+ await waitFor(() => expect(enqueueSlopfabGeneration).toHaveBeenCalledOnce());
+ const expected = modelType === "yue2" ? "midpoint" : "default";
+ expect(vi.mocked(enqueueSlopfabGeneration).mock.calls[0][0]).toMatchObject({ steps: 32, music: { flow_sampler: expected } });
+ await finish(queue, queue.getSnapshot()[0].id);
+ expect(first.getSnapshot().config.musicSongs![0].generation?.settings.flow_sampler).toBe(expected);
+});
+
it("cancels a queued song without altering another song", async () => {
const { queue, first } = setup();
const songs = [newSong(), newSong()].map((song) => ({ ...song, style: "Jazz" }));
diff --git a/src/lib/workQueue.ts b/src/lib/workQueue.ts
index 83fa58a..bdaae0f 100644
--- a/src/lib/workQueue.ts
+++ b/src/lib/workQueue.ts
@@ -333,11 +333,11 @@ export class WorkQueue {
const song = source.assetId || source.status === "completed" ? newSongVariation(source, current.musicSongs ?? []) : source;
songId = song.id;
const id = `music-${crypto.randomUUID()}`;
- const captured = { ...structuredClone(song), generatorId: template.id, seed: imageGenerationSeed(song.seed) };
+ const captured = { ...structuredClone(song), generatorId: template.id, seed: imageGenerationSeed(song.seed), settings: musicSettingsForModel(song.settings, template.modelType) };
const config = structuredClone({ ...current, providerSettings: { ...current.providerSettings,
slopfab: engineProviderSetting(template.paths, current.providerSettings.slopfab, "flash2", [], "prompt", [], false, template.sources, { modelType: template.modelType }) } });
const request: SlopfabGenerationRequest = { jobId: id, prompt: song.style, steps: song.steps, seed: captured.seed,
- music: musicSettingsForModel(song.settings, template.modelType), frames: 0, canvasWidth: 0, canvasHeight: 0, referencePaths: [] };
+ music: captured.settings, frames: 0, canvasWidth: 0, canvasHeight: 0, referencePaths: [] };
let finish!: () => void;
const done = new Promise<void>((resolve) => { finish = resolve; });
this.work.set(id, { id, music: captured, session, sceneId: songId, request, config, snapshot: JSON.stringify(captured), cancelled: false, submitted: false, done, finish });