Skip to content

AI media harness for planning, generating, and editing locally

TypeScript

Add image roots and sequential inpainting

Bitti2e5bd0ca11b8
diff --git a/README.md b/README.md
index d4a7c7d..138e316 100644
--- a/README.md
+++ b/README.md
@@ -23,7 +23,9 @@ Slopus brings an AI assistant, a scene generator, reusable visual references, an
New projects can be **Image projects** or **Video projects**. Existing projects open as Video projects. Both live in folders with a `slopus.json` document.
-Image projects use **Agent**, **Editor**, and **References** tabs. In the Editor, build a hierarchy of objects, text and groups on the left, arrange their placement boxes in the center, and edit descriptions, style, colors and generation settings in the right inspector. Choose a MiniMax H3 generator template above the canvas and press **Generate**. Generation uses the native SlopFab runtime without Python. Full-resolution JPEGs are saved in `media/generated/`, remain available in the image history, and can be exported from the canvas toolbar. Placement boxes guide the model through the prompt; they are not exact masks.
+Image projects use **Agent**, **Editor**, and **References** tabs. In the Editor, build a hierarchy of objects, text and groups on the left, arrange their placement boxes in the center, and edit descriptions, style, colors and generation settings in the right inspector. Choose a MiniMax H3 generator above the canvas and press **Generate**. Generation uses the native SlopFab runtime without Python. Full-resolution images are saved in `media/generated/`, remain available in the image history, and can be exported as JPEG or PNG using **Export** beside **Save**.
+
+The inspector offers **Prompt root** for a new composition and **Image root** to open an existing PNG or JPEG. You can also right-click a generated thumbnail and choose **Edit** to start an image root from it. Add Object nodes with descriptions and placement boxes, then Generate. In image-root mode, SlopFab applies each edit in hierarchy order using exact bounding-box inpainting, preserving pixels outside each box. The final PNG keeps the source resolution and becomes a clean image root for further edits. In prompt-root mode, placement boxes guide the prompt rather than acting as masks.
The Agent can add, edit, reorder, reparent, duplicate and remove image nodes, and adjust prompts, style, placement, palettes, references, steps and seeds. Edits are transactional and preserve generated images. See the [image command guide](src-tauri/assets/agent-image.md) for the command vocabulary and examples.
diff --git a/src-tauri/assets/agent-image.md b/src-tauri/assets/agent-image.md
index db99f97..7395651 100644
--- a/src-tauri/assets/agent-image.md
+++ b/src-tauri/assets/agent-image.md
@@ -4,6 +4,8 @@ Describe a single visible composition: subjects, pose, spatial relationships, ba
Prefer small, targeted commands for edits to the image hierarchy. Unmentioned fields and unrelated nodes are preserved. Every edit response is JSONL with a final {"op":"commit","summary":"..."} line, just like other project commands.
+When imageScene.rootType is "image", sourceImage is the existing image being edited. The root has no authored text prompt. Add Object nodes describing changes, each with an explicit bounding box in the usual 0–1000 full-image coordinates. Each described node is inpainted separately, in depth-first hierarchy order, using the previous edit's result. Empty groups only organize edits. Use image.node.move to change execution order. Root style/background and project resolution do not affect these edits; source dimensions are preserved. The user chooses the source with Open image or a generated thumbnail's Edit action; commands cannot set rootType or sourceImage, and image.set preserves them. Successful generation produces a clean image root on the final result, with no pending edits. Do not put edit instructions in the root.
+
- {"op":"image.configure","prompt"?:string,"background"?:string,"style"?:{"mode"?:"photo"|"art","aesthetics"?:string,"lighting"?:string,"medium"?:string,"detail"?:string},"steps"?:integer,"seed"?:integer,"referenceIds"?:string[]}. Updates only supplied fields; prompt edits the root description. Style merges supplied fields. An empty string clears a description, [] clears references. Use project.set for resolution and aspectRatio.
- {"op":"image.node.add","id":string,"parent":existing-node-id,"kind":"object"|"text"|"group"|"background","name":string,"description"?:string,"text"?:string,"box"?:box|null,"colors"?:string[],"before"?:sibling-id|null}. Adds a node under parent. Optional text and description default to empty, colors to [], and box to null (automatic placement). before inserts before an existing child of parent; omit it or use null to append.
- {"op":"image.node.set","id":existing-node-id,"name"?:string,"description"?:string,"text"?:string,"box"?:box|null,"colors"?:string[]}. Edits only supplied fields. Setting box moves/resizes the node and transforms its descendants' boxes with it. box:null clears only this node's explicit placement; omission leaves placement unchanged. [] clears its palette. Root description is the image prompt.
diff --git a/src-tauri/src/app.rs b/src-tauri/src/app.rs
index d6b1d5a..21ec3bc 100644
--- a/src-tauri/src/app.rs
+++ b/src-tauri/src/app.rs
@@ -85,6 +85,7 @@ pub fn run() {
.plugin(tauri_plugin_process::init())
.invoke_handler(tauri::generate_handler![
commands::artifacts::save_generated_image,
+ commands::artifacts::open_image_source,
commands::artifacts::export_generated_image,
app_updater_enabled,
weights::download_weight,
diff --git a/src-tauri/src/commands/artifacts.rs b/src-tauri/src/commands/artifacts.rs
index 228a79a..4785748 100644
--- a/src-tauri/src/commands/artifacts.rs
+++ b/src-tauri/src/commands/artifacts.rs
@@ -137,6 +137,7 @@ pub(crate) fn save_reference_icon(folder_path: String, job_id: String) -> Result
pub(crate) fn save_generated_image(
folder_path: String,
job_id: String,
+ format: Option<String>,
) -> Result<GeneratedImageFile, String> {
let summary =
rendered::summary(&job_id).ok_or("The generated image is no longer in memory.")?;
@@ -144,7 +145,46 @@ pub(crate) fn save_generated_image(
return Err("Expected a single still image.".into());
}
let rgba = rendered::frame(&job_id, 0)?.ok_or("The generated image has no pixels.")?;
- write_generated_image_frame(&folder_path, &job_id, summary.width, summary.height, &rgba)
+ if format.as_deref() == Some("png") {
+ write_generated_png_frame(&folder_path, &job_id, summary.width, summary.height, &rgba)
+ } else if format.is_none() || format.as_deref() == Some("jpg") {
+ write_generated_image_frame(&folder_path, &job_id, summary.width, summary.height, &rgba)
+ } else { Err("Unsupported generated image format.".into()) }
+}
+
+pub(crate) fn write_generated_png_frame(folder: &str, id: &str, width: u32, height: u32, rgba: &[u8]) -> Result<GeneratedImageFile, String> {
+ use image::ImageEncoder;
+ if width == 0 || height == 0 || width > 8192 || height > 8192 || rgba.len() != width as usize * height as usize * 4 {
+ return Err("Invalid PNG image dimensions or pixels.".into());
+ }
+ let root = ProjectRoot::open(folder)?;
+ let stem = generated_file_stem(id)?;
+ let mut bytes = Vec::new();
+ image::codecs::png::PngEncoder::new(&mut bytes).write_image(rgba, width, height, image::ExtendedColorType::Rgba8).map_err(|error| error.to_string())?;
+ crate::storage::atomic::write_atomically(&root.directory("media/generated")?.join(format!("{stem}.png")), &bytes)?;
+ Ok(GeneratedImageFile { relative_path: format!("media/generated/{stem}.png"), width: width as u16, height: height as u16 })
+}
+
+#[tauri::command]
+pub(crate) async fn open_image_source(app: AppHandle, folder_path: String) -> Result<Option<crate::project::image::ImageSource>, String> {
+ tauri::async_runtime::spawn_blocking(move || {
+ use tauri_plugin_dialog::DialogExt;
+ let Some(selected) = app.dialog().file().set_title("Open image to edit").add_filter("Images", &["png", "jpg", "jpeg"]).blocking_pick_file() else { return Ok(None); };
+ let path = selected.into_path().map_err(|_| "Choose a local image file.")?;
+ let (width, height) = image::image_dimensions(&path).map_err(|error| format!("Could not read image: {error}"))?;
+ if !(1..=8192).contains(&width) || !(1..=8192).contains(&height) { return Err("Image editing supports up to 8192 pixels per axis.".into()); }
+ let bytes = fs::read(&path).map_err(|error| error.to_string())?;
+ let extension = match image::guess_format(&bytes).map_err(|error| error.to_string())? {
+ image::ImageFormat::Png => "png", image::ImageFormat::Jpeg => "jpg", _ => return Err("Choose a PNG or JPEG image.".into()),
+ };
+ let mut random = [0_u8; 16];
+ getrandom::fill(&mut random).map_err(|error| error.to_string())?;
+ let id = random.iter().map(|byte| format!("{byte:02x}")).collect::<String>();
+ let root = ProjectRoot::open(&folder_path)?;
+ let filename = format!("image-source-{id}.{extension}");
+ crate::storage::atomic::write_atomically(&root.directory("media/imported")?.join(&filename), &bytes)?;
+ Ok(Some(crate::project::image::ImageSource { relative_path: format!("media/imported/{filename}"), name: path.file_name().unwrap_or_default().to_string_lossy().into_owned(), width, height }))
+ }).await.map_err(|error| error.to_string())?
}
#[derive(Debug, serde::Serialize)]
@@ -194,9 +234,9 @@ pub(crate) async fn export_generated_image(
let root = ProjectRoot::open(&folder_path)?;
let source = root.existing(&relative_path)?;
if !relative_path.starts_with("media/generated/")
- || source.extension().and_then(|s| s.to_str()) != Some("jpg")
+ || !matches!(source.extension().and_then(|s| s.to_str()), Some("jpg" | "jpeg" | "png"))
{
- return Err("Choose a generated JPEG image to export.".into());
+ return Err("Choose a generated JPEG or PNG image to export.".into());
}
let Some(destination) = app
.dialog()
@@ -239,13 +279,17 @@ pub(crate) fn export_image_file(
return Err("Choose a .jpg, .jpeg or .png filename for the exported image.".into());
}
let bytes = fs::read(source).map_err(|error| format!("Could not read image: {error}"))?;
- let bytes = if extension == "png" {
- let image = image::load_from_memory_with_format(&bytes, image::ImageFormat::Jpeg)
+ let source_format = image::guess_format(&bytes).map_err(|error| format!("Could not read image format: {error}"))?;
+ let target_format = if extension == "png" { image::ImageFormat::Png } else { image::ImageFormat::Jpeg };
+ let bytes = if source_format != target_format {
+ let image = image::load_from_memory_with_format(&bytes, source_format)
.map_err(|error| format!("Could not decode image: {error}"))?;
let mut encoded = std::io::Cursor::new(Vec::new());
- image
- .write_to(&mut encoded, image::ImageFormat::Png)
- .map_err(|error| format!("Could not encode PNG: {error}"))?;
+ if target_format == image::ImageFormat::Jpeg {
+ image::codecs::jpeg::JpegEncoder::new_with_quality(&mut encoded, 95).encode_image(&image.to_rgb8()).map_err(|error| error.to_string())?;
+ } else {
+ image.write_to(&mut encoded, target_format).map_err(|error| format!("Could not encode PNG: {error}"))?;
+ }
encoded.into_inner()
} else {
bytes
diff --git a/src-tauri/src/commands/generation.rs b/src-tauri/src/commands/generation.rs
index 042f5e2..709b8fe 100644
--- a/src-tauri/src/commands/generation.rs
+++ b/src-tauri/src/commands/generation.rs
@@ -84,6 +84,7 @@ pub(crate) fn resolve_slopfab_plan(
folder_path: Option<String>,
) -> Result<slopfab::ResolvedPlan, String> {
prepare_continuation_path(&mut request, folder_path.as_deref())?;
+ prepare_image_edit_path(&mut request, folder_path.as_deref())?;
let context = serde_json::json!({
"jobId": request.job_id, "frames": request.frames, "steps": request.steps,
"canvasWidth": request.canvas_width, "canvasHeight": request.canvas_height,
@@ -123,6 +124,7 @@ pub(crate) fn enqueue_slopfab_generation(
folder_path: String,
) -> Result<(), String> {
prepare_continuation_path(&mut request, Some(&folder_path))?;
+ prepare_image_edit_path(&mut request, Some(&folder_path))?;
if !request.still_image {
request.save_latents_path = Some(generated_latent_destination(&folder_path, &request.job_id)?);
}
@@ -152,6 +154,19 @@ pub(crate) fn enqueue_slopfab_generation(
result
}
+fn prepare_image_edit_path(request: &mut slopfab::GenerationRequest, folder_path: Option<&str>) -> Result<(), String> {
+ if let Some(edit) = &request.image_edit {
+ let root = crate::project::paths::ProjectRoot::open(folder_path.ok_or("Image edits require a project folder.")?)?;
+ let path = root.existing(&edit.source_relative_path)?;
+ let (width, height) = ::image::image_dimensions(&path).map_err(|error| format!("Could not read source image: {error}"))?;
+ if width != request.canvas_width as u32 || height != request.canvas_height as u32 {
+ return Err("Image edit dimensions do not match the source file.".into());
+ }
+ request.image_edit_path = Some(path);
+ }
+ Ok(())
+}
+
#[tauri::command]
pub(crate) fn cancel_slopfab_generation(
state: tauri::State<'_, slopfab::SlopfabRuntime>,
diff --git a/src-tauri/src/project/commands/images.rs b/src-tauri/src/project/commands/images.rs
index 6ab60c7..23e23a2 100644
--- a/src-tauri/src/project/commands/images.rs
+++ b/src-tauri/src/project/commands/images.rs
@@ -23,6 +23,8 @@ pub struct ImageStylePatch {
fn new_scene(prompt: &str) -> ImageScene {
ImageScene {
+ root_type: None,
+ source_image: None,
nodes: vec![ImageNode {
id: "image-root".into(),
parent_id: None,
@@ -150,6 +152,8 @@ pub(super) fn apply(
reference_ids,
} => {
scene = ImageScene {
+ root_type: scene.root_type,
+ source_image: scene.source_image,
nodes: nodes.clone(),
background: background.clone(),
style: style.clone(),
diff --git a/src-tauri/src/project/image.rs b/src-tauri/src/project/image.rs
index b3810df..faef53e 100644
--- a/src-tauri/src/project/image.rs
+++ b/src-tauri/src/project/image.rs
@@ -17,6 +17,10 @@ pub(crate) struct ImageGenerationSnapshot {
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct ImageScene {
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub root_type: Option<String>,
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub source_image: Option<ImageSource>,
pub nodes: Vec<ImageNode>,
pub background: String,
pub style: ImageStyle,
@@ -27,6 +31,14 @@ pub struct ImageScene {
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
+pub struct ImageSource {
+ pub relative_path: String,
+ pub name: String,
+ pub width: u32,
+ pub height: u32,
+}
+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct ImageNode {
pub id: String,
pub parent_id: Option<String>,
@@ -57,6 +69,15 @@ pub struct ImageStyle {
impl ImageScene {
pub fn validate(&self) -> Result<(), String> {
+ if self.root_type.as_deref().is_some_and(|kind| !matches!(kind, "prompt" | "image")) {
+ return Err("Invalid image root type.".into());
+ }
+ if let Some(source) = &self.source_image {
+ crate::project::paths::normalize_project_path(&source.relative_path)?;
+ if source.name.is_empty() || !(1..=8192).contains(&source.width) || !(1..=8192).contains(&source.height) {
+ return Err("Source image dimensions must be between 1 and 8192 pixels.".into());
+ }
+ }
if self.nodes.is_empty()
|| self.nodes.len() > 500
|| !(2..=1000).contains(&self.steps)
diff --git a/src-tauri/src/slopfab/ffi/mod.rs b/src-tauri/src/slopfab/ffi/mod.rs
index ca62b74..0fed5cb 100644
--- a/src-tauri/src/slopfab/ffi/mod.rs
+++ b/src-tauri/src/slopfab/ffi/mod.rs
@@ -97,6 +97,14 @@ impl Api {
self.check(r)?;
self.inner.set_still_image(r.pointer.as_ptr())
}
+ pub fn set_image_edit_path(&self, r: &RequestHandle, path: &Path, bounds: [i32; 4]) -> Result<(), String> {
+ self.check(r)?;
+ self.inner.set_image_edit_path(r.pointer.as_ptr(), path, bounds)
+ }
+ pub fn set_image_edit_rgb(&self, r: &RequestHandle, pixels: &[u8], width: i32, height: i32, bounds: [i32; 4]) -> Result<(), String> {
+ self.check(r)?;
+ self.inner.set_image_edit_rgb(r.pointer.as_ptr(), pixels, width, height, bounds)
+ }
pub fn set_save_latents(&self, r: &RequestHandle, path: &Path) -> Result<(), String> {
self.check(r)?;
self.inner.set_save_latents(r.pointer.as_ptr(), path)
diff --git a/src-tauri/src/slopfab/ffi/raw/mod.rs b/src-tauri/src/slopfab/ffi/raw/mod.rs
index f508799..707eae3 100644
--- a/src-tauri/src/slopfab/ffi/raw/mod.rs
+++ b/src-tauri/src/slopfab/ffi/raw/mod.rs
@@ -60,6 +60,8 @@ pub struct Output {
pub type ProgressFn = Option<unsafe extern "C" fn(*const Progress, *mut c_void)>;
type SetMotionCache =
unsafe extern "C" fn(*mut Request, i32, f32, f32, i32, i32, f32, f32, i32, i32) -> i32;
+type SetImageEditPath = unsafe extern "C" fn(*mut Request, *const c_char, i32, i32, i32, i32, f32, i32) -> i32;
+type SetImageEditRgb = unsafe extern "C" fn(*mut Request, *const u8, usize, i32, i32, usize, i32, i32, i32, i32, f32, i32) -> i32;
pub struct Api {
_library: Library,
@@ -77,6 +79,8 @@ pub struct Api {
set_resolution: unsafe extern "C" fn(*mut Request, i32, i32) -> i32,
set_frames: unsafe extern "C" fn(*mut Request, i32) -> i32,
set_still_image: Option<unsafe extern "C" fn(*mut Request, i32) -> i32>,
+ set_image_edit_path: Option<SetImageEditPath>,
+ set_image_edit_rgb: Option<SetImageEditRgb>,
set_save_latents: Option<unsafe extern "C" fn(*mut Request, *const c_char) -> i32>,
set_continuation_file: Option<unsafe extern "C" fn(*mut Request, *const c_char, i32) -> i32>,
set_video_transition: Option<unsafe extern "C" fn(*mut Request, i32) -> i32>,
@@ -188,6 +192,8 @@ impl Api {
)
.ok()
.map(|symbol| *symbol),
+ set_image_edit_path: library.get::<SetImageEditPath>(b"slopfab_request_set_image_edit_path\0").ok().map(|symbol| *symbol),
+ set_image_edit_rgb: library.get::<SetImageEditRgb>(b"slopfab_request_set_image_edit_rgb24\0").ok().map(|symbol| *symbol),
set_steps: symbol!(
"slopfab_request_set_steps",
unsafe extern "C" fn(*mut Request, i32) -> i32
@@ -374,6 +380,18 @@ impl Api {
let set = self.set_still_image.ok_or("This slopfab.dll does not support still-image generation. Update the runtime to generate images and reference icons.")?;
self.error(unsafe { set(r, 1) })
}
+ pub fn set_image_edit_path(&self, r: *mut Request, path: &Path, bounds: [i32; 4]) -> Result<(), String> {
+ let set = self.set_image_edit_path.ok_or("This slopfab.dll does not support bounding-box image editing. Update the runtime.")?;
+ let path = path_cstring(path)?;
+ self.error(unsafe { set(r, path.as_ptr(), bounds[0], bounds[1], bounds[2], bounds[3], 1.0, 0) })
+ }
+ pub fn set_image_edit_rgb(&self, r: *mut Request, pixels: &[u8], width: i32, height: i32, bounds: [i32; 4]) -> Result<(), String> {
+ let set = self.set_image_edit_rgb.ok_or("This slopfab.dll does not support sequential image editing. Update the runtime.")?;
+ if width <= 0 || height <= 0 || pixels.len() != width as usize * height as usize * 3 {
+ return Err("Invalid image edit RGB buffer.".into());
+ }
+ self.error(unsafe { set(r, pixels.as_ptr(), pixels.len(), width, height, width as usize * 3, bounds[0], bounds[1], bounds[2], bounds[3], 1.0, 0) })
+ }
pub fn set_save_latents(&self, r: *mut Request, path: &Path) -> Result<(), String> {
let set = self.set_save_latents.ok_or(
"Saving generation latents requires slopfab.dll API 1.9 or later. Update the runtime.",
diff --git a/src-tauri/src/slopfab/ffi/tests.rs b/src-tauri/src/slopfab/ffi/tests.rs
index 425ec9d..ccda874 100644
--- a/src-tauri/src/slopfab/ffi/tests.rs
+++ b/src-tauri/src/slopfab/ffi/tests.rs
@@ -1,5 +1,25 @@
use super::*;
+#[test]
+fn image_edit_setters_use_original_pixel_boxes_and_pad_the_plan() {
+ let api = Api::load(&crate::slopfab::default_dll_path()).unwrap();
+ let request = RequestHandle::new(&api).unwrap();
+ let pixels = vec![100; 65 * 41 * 3];
+ api.set_image_edit_rgb(&request, &pixels, 65, 41, [60, 35, 5, 6]).unwrap();
+ let plan = api.resolve(&request).unwrap();
+ assert_eq!((plan.canvas_width, plan.canvas_height, plan.aligned_frames), (96, 64, 1));
+ assert!(api.set_image_edit_rgb(&request, &pixels, 65, 41, [60, 35, 6, 6]).is_err());
+ assert!(api.set_image_edit_rgb(&request, &pixels[..3], 65, 41, [0, 0, 5, 6]).is_err());
+ assert_eq!(api.resolve(&request).unwrap().canvas_width, 96);
+ let folder = tempfile::tempdir().unwrap();
+ let path = folder.path().join("source.png");
+ image::RgbImage::from_raw(65, 41, pixels).unwrap().save(&path).unwrap();
+ let from_path = RequestHandle::new(&api).unwrap();
+ api.set_image_edit_path(&from_path, &path, [0, 0, 65, 41]).unwrap();
+ std::fs::remove_file(path).unwrap(); // The setter snapshots pixels synchronously.
+ assert_eq!(api.resolve(&from_path).unwrap().aligned_frames, 1);
+}
+
#[test]
fn older_runtimes_report_missing_lora_preparation_without_breaking_planning() {
let mut api = Api::load(&crate::slopfab::default_dll_path()).unwrap();
diff --git a/src-tauri/src/slopfab/generation.rs b/src-tauri/src/slopfab/generation.rs
index 16e95a5..97391be 100644
--- a/src-tauri/src/slopfab/generation.rs
+++ b/src-tauri/src/slopfab/generation.rs
@@ -10,6 +10,10 @@ use super::{
use crate::{diagnostics, rendered};
use std::sync::atomic::Ordering;
+#[cfg(test)]
+#[path = "generation_tests.rs"]
+mod tests;
+
struct FrameSource {
generation: ffi::FinishedGeneration,
frame_offset: u32,
@@ -25,6 +29,68 @@ impl rendered::FrameSource for FrameSource {
pub(super) fn run_generation(
item: &QueueItem,
+) -> Result<(OutputMetadata, rendered::RenderedVideo), String> {
+ run_edit_sequence(item, run_single_generation)
+}
+
+fn run_edit_sequence(
+ item: &QueueItem,
+ mut generate: impl FnMut(&QueueItem) -> Result<(OutputMetadata, rendered::RenderedVideo), String>,
+) -> Result<(OutputMetadata, rendered::RenderedVideo), String> {
+ let Some(edit) = &item.request.image_edit else { return generate(item); };
+ let count = edit.edits.len();
+ let mut pixels = None;
+ let mut elapsed = 0.0;
+ let mut computed = 0;
+ let mut skipped = 0;
+ for (index, step) in edit.edits.iter().enumerate() {
+ if item.cancel.load(Ordering::Acquire) { return Err("Image editing cancelled.".into()); }
+ let mut request = item.request.clone();
+ request.prompt = step.prompt.clone();
+ request.image_edit.as_mut().unwrap().edits = vec![step.clone()];
+ request.image_edit_pixels = pixels.take();
+ let sink = item.events.clone();
+ let offset = elapsed;
+ let stage = QueueItem {
+ request, configuration: item.configuration.clone(), references: item.references.clone(), cancel: item.cancel.clone(),
+ events: std::sync::Arc::new(move |event| {
+ let event = match event {
+ super::events::NativeEvent::Progress(mut progress) => {
+ let per_edit = progress.planned_steps.max(1);
+ progress.step = index as i32 * per_edit + progress.step.clamp(0, per_edit);
+ progress.total_steps = per_edit * count as i32;
+ progress.planned_steps = progress.total_steps;
+ progress.elapsed_seconds += offset;
+ progress.timing_profile.push_str(&format!("|imageEdits={count}"));
+ super::events::NativeEvent::Progress(progress)
+ }
+ event => event,
+ };
+ sink(event);
+ }),
+ };
+ let (mut metadata, result) = generate(&stage)?;
+ if item.cancel.load(Ordering::Acquire) { return Err("Image editing cancelled.".into()); }
+ elapsed += metadata.seconds_total;
+ computed += metadata.steps_computed;
+ skipped += metadata.steps_skipped;
+ if index + 1 == count {
+ metadata.seconds_total = elapsed;
+ metadata.steps_computed = computed;
+ metadata.steps_skipped = skipped;
+ metadata.timing_profile.push_str(&format!("|imageEdits={count}"));
+ return Ok((metadata, result));
+ }
+ let rgba = result.frame(0)?.ok_or("Image edit returned no pixels.")?;
+ // Copy only the completed still, then release its generation before
+ // starting the next edit. No intermediate file or lossy encode.
+ pixels = Some(std::sync::Arc::new(rgba.chunks_exact(4).flat_map(|pixel| pixel[..3].iter().copied()).collect()));
+ }
+ Err("Add an image edit before generating.".into())
+}
+
+fn run_single_generation(
+ item: &QueueItem,
) -> Result<(OutputMetadata, rendered::RenderedVideo), String> {
let _models = super::MODEL_ACCESS
.read()
diff --git a/src-tauri/src/slopfab/generation_tests.rs b/src-tauri/src/slopfab/generation_tests.rs
new file mode 100644
index 0000000..867b927
--- /dev/null
+++ b/src-tauri/src/slopfab/generation_tests.rs
@@ -0,0 +1,70 @@
+use super::*;
+use crate::slopfab::{config::Configuration, events::NativeEvent, references::ReferenceVideos, types::{GenerationRequest, ProgressEvent}};
+use std::sync::{Arc, Mutex, atomic::AtomicBool};
+
+struct Pixels(u8);
+impl rendered::FrameSource for Pixels {
+ fn frame_rgba(&self, _: u32, _: u32, _: u32) -> Result<Vec<u8>, String> { Ok(vec![self.0, 20, 30, 255]) }
+}
+fn result(value: u8) -> (OutputMetadata, rendered::RenderedVideo) {
+ let metadata = OutputMetadata { frames: 1, width: 1, height: 1, fps: 1.0, audio_channels: 0, audio_sample_rate: 0, audio_samples: 0,
+ reference_count: 0, timing_profile: "test".into(), seconds_conditioning: 0.0, seconds_denoise: 1.0, seconds_video_decode: 0.0,
+ seconds_audio_decode: 0.0, seconds_total: 1.0, steps_computed: 2, steps_skipped: 0, duration_seconds: 1.0, boundary: "test" };
+ (metadata, rendered::from_source("edit", Box::new(Pixels(value)), vec![], 1, 1, 1, 3, 3, 1.0, 0, 0).unwrap())
+}
+fn item() -> QueueItem {
+ let request: GenerationRequest = serde_json::from_value(serde_json::json!({
+ "jobId":"edit", "prompt":"first", "frames":1, "stillImage":true, "steps":2, "seed":1, "canvasWidth":1, "canvasHeight":1,
+ "imageEdit":{"sourceRelativePath":"media/source.png","edits":[
+ {"prompt":"first","x":0,"y":0,"width":1,"height":1},
+ {"prompt":"second","x":0,"y":0,"width":1,"height":1}
+ ]}
+ })).unwrap();
+ QueueItem { request, configuration: Configuration::from_settings(&Default::default()), references: ReferenceVideos::default(),
+ events: Arc::new(|_| {}), cancel: Arc::new(AtomicBool::new(false)) }
+}
+
+#[test]
+fn sequential_image_edits_feed_previous_pixels_and_report_combined_progress() {
+ let mut item = item();
+ let progress = Arc::new(Mutex::new(Vec::new()));
+ let sink = progress.clone();
+ item.events = Arc::new(move |event| { if let NativeEvent::Progress(event) = event { sink.lock().unwrap().push(event); } });
+ let mut calls = 0;
+ let (metadata, output) = run_edit_sequence(&item, |stage| {
+ calls += 1;
+ assert_eq!(stage.request.image_edit.as_ref().unwrap().edits.len(), 1);
+ if calls == 1 {
+ assert_eq!(stage.request.prompt, "first");
+ assert!(stage.request.image_edit_pixels.is_none());
+ } else {
+ assert_eq!(stage.request.prompt, "second");
+ assert_eq!(stage.request.image_edit_pixels.as_ref().unwrap().as_slice(), &[1, 20, 30]);
+ }
+ (stage.events)(NativeEvent::Progress(ProgressEvent { job_id: "edit".into(), stage: "denoising", step: 2, total_steps: 2,
+ planned_steps: 2, elapsed_seconds: 1.0, frames: 1, canvas_width: 1, canvas_height: 1, reference_count: 0, timing_profile: "test".into() }));
+ Ok(result(calls))
+ }).unwrap();
+ assert_eq!(calls, 2);
+ assert_eq!(output.frame(0).unwrap().unwrap(), [2, 20, 30, 255]);
+ assert_eq!(metadata.seconds_total, 2.0);
+ assert_eq!(metadata.steps_computed, 4);
+ let progress = progress.lock().unwrap();
+ assert_eq!((progress[0].step, progress[1].step, progress[1].total_steps), (2, 4, 4));
+ assert_eq!(progress[1].elapsed_seconds, 2.0);
+}
+
+#[test]
+fn cancellation_and_failures_do_not_publish_partial_image_edits() {
+ for cancel in [false, true] {
+ let item = item();
+ let mut calls = 0;
+ let result = run_edit_sequence(&item, |_| {
+ calls += 1;
+ if cancel { item.cancel.store(true, Ordering::Release); Ok(result(1)) }
+ else { Err("inpainting failed".into()) }
+ });
+ assert!(result.is_err());
+ assert_eq!(calls, 1);
+ }
+}
diff --git a/src-tauri/src/slopfab/h3.rs b/src-tauri/src/slopfab/h3.rs
index c0b5396..83dbfcd 100644
--- a/src-tauri/src/slopfab/h3.rs
+++ b/src-tauri/src/slopfab/h3.rs
@@ -54,10 +54,21 @@ pub(super) fn configure_request(
}
api.set_steps(handle, configuration.generation_steps(request.steps)?)?;
api.set_seed(handle, generation_seed(request.seed)?)?;
- api.set_resolution(handle, request.canvas_width, request.canvas_height)?;
+ if let Some(edit) = &request.image_edit {
+ let step = edit.edits.first().ok_or("Add an image edit before generating.")?;
+ api.set_prompt(handle, &step.prompt)?;
+ let bounds = [step.x, step.y, step.width, step.height];
+ if let Some(pixels) = &request.image_edit_pixels {
+ api.set_image_edit_rgb(handle, pixels, request.canvas_width, request.canvas_height, bounds)?;
+ } else {
+ api.set_image_edit_path(handle, request.image_edit_path.as_deref().ok_or("Image edit source has not been resolved.")?, bounds)?;
+ }
+ } else {
+ api.set_resolution(handle, request.canvas_width, request.canvas_height)?;
+ }
api.set_inference_backend(handle, platform.backend())?;
api.set_attention(handle, configuration.attention)?;
- api.set_motion_cache(handle, configuration.motion_cache)?;
+ api.set_motion_cache(handle, configuration.motion_cache && request.image_edit.is_none())?;
api.set_verbose(handle, false)?;
if let Some(mode) = request.video_transition.as_deref() {
api.set_video_transition(handle, if mode == "bridge" { 2 } else { 1 })?;
diff --git a/src-tauri/src/slopfab/planning.rs b/src-tauri/src/slopfab/planning.rs
index 6dd0725..d4a015f 100644
--- a/src-tauri/src/slopfab/planning.rs
+++ b/src-tauri/src/slopfab/planning.rs
@@ -46,7 +46,7 @@ pub fn resolve_plan(
canvas_height: plan.canvas_height,
aligned_frames: frames,
duration_seconds: plan.duration_seconds * frames as f64 / plan.aligned_frames as f64,
- model_evaluations: plan.num_model_evaluations,
+ model_evaluations: plan.num_model_evaluations * request.image_edit.as_ref().map_or(1, |edit| edit.edits.len() as i32),
sequence_rows_without_text: plan.sequence_rows_without_text,
latent_frames: plan.latent_frames,
latent_width: plan.latent_width,
@@ -57,6 +57,15 @@ pub fn resolve_plan(
}
pub(super) fn validate_generation_controls(request: &GenerationRequest) -> Result<(), String> {
+ if let Some(edit) = &request.image_edit {
+ if !request.still_image || request.frames != 1 || request.continuation_relative_path.is_some() || request.continuation_path.is_some() || request.video_transition.is_some()
+ || edit.edits.is_empty() || edit.edits.len() > 499 || request.canvas_width > 8192 || request.canvas_height > 8192
+ || edit.edits.iter().any(|step| step.prompt.trim().is_empty() || step.x < 0 || step.y < 0 || step.width <= 0 || step.height <= 0
+ || i64::from(step.x) + i64::from(step.width) > i64::from(request.canvas_width)
+ || i64::from(step.y) + i64::from(step.height) > i64::from(request.canvas_height)) {
+ return Err("Image edits require one source image and nonempty boxes inside its dimensions.".into());
+ }
+ }
let limits = crate::generation::models::h3::MODEL.capabilities;
if request.reference_paths.len() > limits.max_image_references
|| request.reference_video_ids.len() > limits.max_video_references
diff --git a/src-tauri/src/slopfab/tests.rs b/src-tauri/src/slopfab/tests.rs
index 8a37b57..36b7ee0 100644
--- a/src-tauri/src/slopfab/tests.rs
+++ b/src-tauri/src/slopfab/tests.rs
@@ -5,6 +5,35 @@ use super::{
use crate::project::{ProviderOption, ProviderSetting};
use std::collections::BTreeMap;
+#[test]
+fn image_edit_recipe_disables_motion_cache_and_keeps_source_geometry() {
+ let references = ReferenceVideos::default();
+ let mut configuration = Configuration::from_settings(&BTreeMap::new());
+ configuration.motion_cache = true;
+ let api = ffi::Api::load(&configuration.dll_path).unwrap();
+ let request = GenerationRequest {
+ prompt: "A red vase".into(), still_image: true, frames: 1, steps: 4, seed: 1,
+ canvas_width: 65, canvas_height: 41,
+ image_edit_pixels: Some(std::sync::Arc::new(vec![100; 65 * 41 * 3])),
+ image_edit: Some(types::ImageEditRequest { source_relative_path: "media/source.png".into(), edits: vec![
+ types::ImageEditStep { prompt: "A red vase".into(), x: 30, y: 10, width: 35, height: 31 }
+ ] }), ..Default::default()
+ };
+ validate_generation_controls(&request).unwrap();
+ for platform in [ComputePlatform::Cuda13, ComputePlatform::Vulkan] {
+ for purpose in [RequestPurpose::Plan, RequestPurpose::Generate] {
+ let handle = RequestHandle::new(&api).unwrap();
+ configure_request(&api, &handle, &request, &configuration, platform, purpose, &references).unwrap();
+ // Resolving fails if MotionCache is still enabled for inpainting.
+ let plan = api.resolve(&handle).unwrap();
+ assert_eq!((plan.canvas_width, plan.canvas_height, plan.aligned_frames), (96, 64, 1));
+ }
+ }
+ let mut invalid = request;
+ invalid.image_edit.as_mut().unwrap().edits[0].width = 36;
+ assert!(validate_generation_controls(&invalid).is_err());
+}
+
#[test]
fn video_transitions_use_encoded_boundaries_with_the_bundled_runtime() {
let references = ReferenceVideos::default();
diff --git a/src-tauri/src/slopfab/types.rs b/src-tauri/src/slopfab/types.rs
index d9f8ce3..2fc526e 100644
--- a/src-tauri/src/slopfab/types.rs
+++ b/src-tauri/src/slopfab/types.rs
@@ -45,6 +45,12 @@ pub struct GenerationRequest {
pub frames: i32,
#[serde(default)]
pub still_image: bool,
+ #[serde(default)]
+ pub image_edit: Option<ImageEditRequest>,
+ #[serde(skip)]
+ pub image_edit_path: Option<PathBuf>,
+ #[serde(skip)]
+ pub image_edit_pixels: Option<std::sync::Arc<Vec<u8>>>,
pub steps: i32,
/// -1 asks the host to draw a fresh seed for this generation.
pub seed: i64,
@@ -65,6 +71,21 @@ pub struct GenerationRequest {
pub save_latents_path: Option<PathBuf>,
}
+#[derive(Debug, Clone, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct ImageEditRequest {
+ pub source_relative_path: String,
+ pub edits: Vec<ImageEditStep>,
+}
+#[derive(Debug, Clone, Deserialize)]
+pub struct ImageEditStep {
+ pub prompt: String,
+ pub x: i32,
+ pub y: i32,
+ pub width: i32,
+ pub height: i32,
+}
+
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct RefmodInput {
pub path: String,
diff --git a/src-tauri/src/tests/images.rs b/src-tauri/src/tests/images.rs
index 2428ccd..59b3ce4 100644
--- a/src-tauri/src/tests/images.rs
+++ b/src-tauri/src/tests/images.rs
@@ -4,6 +4,40 @@ fn image_fixture() -> ProjectConfig {
serde_json::from_str(include_str!("../../../fixtures/project-v1-image.json")).unwrap()
}
+#[test]
+fn edited_png_preserves_exact_pixels_and_exports_to_jpeg() {
+ let folder = project_folder();
+ let rgba: Vec<u8> = (0..65 * 41).flat_map(|index| [(index % 256) as u8, 80, 160, 255]).collect();
+ let saved = crate::commands::artifacts::write_generated_png_frame(&folder.path().to_string_lossy(), "edit-test", 65, 41, &rgba).unwrap();
+ let source = folder.path().join(saved.relative_path);
+ assert_eq!(::image::open(&source).unwrap().to_rgba8().as_raw(), &rgba);
+ let copy = folder.path().join("copy.png");
+ export_image_file(&source, &copy).unwrap();
+ assert_eq!(fs::read(&source).unwrap(), fs::read(copy).unwrap());
+ let jpeg = folder.path().join("export.jpg");
+ export_image_file(&source, &jpeg).unwrap();
+ assert_eq!(::image::image_dimensions(jpeg).unwrap(), (65, 41));
+}
+
+#[test]
+fn image_root_survives_save_and_agent_hierarchy_replacement() {
+ use crate::project::commands::{execute_commands_at, ProjectCommand};
+ let mut config = image_fixture();
+ let scene = config.image_scene.as_mut().unwrap();
+ scene.root_type = Some("image".into());
+ scene.source_image = Some(crate::project::image::ImageSource { relative_path: "media/imported/source.png".into(), name: "Source".into(), width: 65, height: 41 });
+ let root = tempfile::tempdir().unwrap();
+ let created = create_project_in(root.path(), &config).unwrap();
+ let reopened = read_project(Path::new(&created.folder_path)).unwrap();
+ assert_eq!(reopened.config.image_scene, config.image_scene);
+ let mut command = serde_json::to_value(config.image_scene.as_ref().unwrap()).unwrap();
+ for field in ["rootType", "sourceImage", "outputAssetId"] { command.as_object_mut().unwrap().remove(field); }
+ command["op"] = serde_json::json!("image.set");
+ let parsed: ProjectCommand = serde_json::from_value(command).unwrap();
+ let changed = execute_commands_at(&config, &[parsed], &config.updated_at).unwrap();
+ assert_eq!(changed.image_scene, config.image_scene);
+}
+
#[test]
fn generated_jpeg_preserves_resolution_and_stays_inside_the_project() {
let folder = project_folder();
diff --git a/src/components/workspace/ImageEditor.test.tsx b/src/components/workspace/ImageEditor.test.tsx
index d4f4eb9..2375fbb 100644
--- a/src/components/workspace/ImageEditor.test.tsx
+++ b/src/components/workspace/ImageEditor.test.tsx
@@ -23,6 +23,23 @@ function setup(initial = parseProjectConfig(fixture)) {
return () => latest;
}
+it("starts an image root from the thumbnail Edit menu and hides root text controls", () => {
+ const initial = parseProjectConfig(fixture);
+ initial.assets = [{ id: "saved", name: "Saved", kind: "image", relativePath: "media/generated/saved.jpg", mimeType: "image/jpeg", width: 101, height: 77, createdAt: initial.createdAt }];
+ const current = setup(initial);
+ fireEvent.contextMenu(screen.getByRole("button", { name: "View Saved" }));
+ fireEvent.click(screen.getByRole("menuitem", { name: "Edit" }));
+ expect(current().imageScene).toMatchObject({ rootType: "image", sourceImage: { relativePath: "media/generated/saved.jpg", width: 101, height: 77 }, outputAssetId: null });
+ expect(current().imageScene!.nodes).toHaveLength(1);
+ expect(screen.queryByLabelText("Prompt (high-level description)")).not.toBeInTheDocument();
+ expect(screen.getByRole("button", { name: "Open image" })).toBeInTheDocument();
+ expect(screen.getByLabelText("Image placement canvas").parentElement!.style.getPropertyValue("--image-ratio")).toBe(String(101 / 77));
+ fireEvent.change(screen.getByLabelText("Root type"), { target: { value: "prompt" } });
+ expect(screen.getByLabelText("Prompt (high-level description)")).toBeInTheDocument();
+ fireEvent.change(screen.getByLabelText("Root type"), { target: { value: "image" } });
+ expect(current().imageScene!.sourceImage).toBeNull();
+});
+
it("restores the hierarchy, prompts, settings and generator when selecting a saved result", () => {
const initial = parseProjectConfig(fixture);
const original = imageGenerationSnapshot(initial, "Original prompt", defaultGeneratorTemplate().id);
diff --git a/src/components/workspace/ImageEditor.tsx b/src/components/workspace/ImageEditor.tsx
index 56cc2af..18a1c65 100644
--- a/src/components/workspace/ImageEditor.tsx
+++ b/src/components/workspace/ImageEditor.tsx
@@ -1,6 +1,8 @@
import { useEffect, useMemo, useRef, useState, useSyncExternalStore, type CSSProperties, type DragEvent, type PointerEvent } from "react";
-import { Box, ChevronDown, ChevronRight, ClipboardPaste, Copy, FolderPlus, Image, Pencil, Plus, Redo2, Sparkles, Square, Trash2, Type, Undo2 } from "lucide-react";
-import { addImageNode, createImageScene, duplicateImageNode, imageDescendants, imageScenePrompt, removeImageNode, resizeImageNode, type ImageBox, type ImageNode, type ImageScene } from "../../lib/imageScene";
+import { Box, ChevronDown, ChevronRight, ClipboardPaste, Copy, FolderOpen, FolderPlus, Image, Pencil, Plus, Redo2, Sparkles, Square, Trash2, Type, Undo2 } from "lucide-react";
+import { invoke } from "@tauri-apps/api/core";
+import { addImageNode, createImageEditScene, createImageScene, duplicateImageNode, imageDescendants, imageScenePrompt, removeImageNode, resizeImageNode, type ImageBox, type ImageNode, type ImageScene, type ImageSource } from "../../lib/imageScene";
+import { compileImageEdits, editGeneratedImage, imageEditDebugPrompt } from "../../lib/imageEditing";
import { outputDimensions } from "../../lib/export";
import { compileImagePrompt } from "../../lib/imagePrompt";
import { restoreGeneratedImage } from "../../lib/imageHistory";
@@ -27,6 +29,12 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
}) {
const scene = useMemo(() => config.imageScene ?? createImageScene(config.brief.prompt), [config.imageScene, config.brief.prompt]);
const root = scene.nodes.find((node) => node.kind === "root")!;
+ const imageRoot = scene.rootType === "image";
+ const editPlan = useMemo(() => {
+ if (!imageRoot) return null;
+ try { return { ...compileImageEdits(config), error: null }; }
+ catch (reason) { return { edits: [], error: String(reason instanceof Error ? reason.message : reason) }; }
+ }, [config, imageRoot]);
const [selection, setSelection] = useState(root.id);
const selected = scene.nodes.find((node) => node.id === selection) ?? root;
const [collapsed, setCollapsed] = useState(new Set<string>());
@@ -58,6 +66,12 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
const redo = useRef<ImageScene[]>([]);
const [, historyChanged] = useState(0);
const pointer = useRef<{ x: number; y: number; scene: ImageScene; id?: string; resize: boolean } | null>(null);
+ useEffect(() => {
+ // Applied edits belong to the old source and must not be resurrected by Undo.
+ undo.current = []; redo.current = []; historyChanged((value) => value + 1);
+ pointer.current = null; setDraftBox(null); setDrawKind(null); setRenaming(null);
+ setSelection(root.id); setCollapsed(new Set());
+ }, [scene.sourceImage?.relativePath, root.id]);
useEffect(() => {
const element = viewport.current;
if (!element) return;
@@ -108,7 +122,7 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
}, [images.length]);
// Generated assets retain the dimensions returned by the native encoder.
// Project settings size only the empty canvas and future generation requests.
- const { width, height } = output?.width && output?.height
+ const { width, height } = imageRoot && scene.sourceImage ? scene.sourceImage : output?.width && output?.height
? { width: output.width, height: output.height }
: outputDimensions(config.settings.resolution, config.settings.aspectRatio);
useEffect(() => subscribeGeneratorTemplates(() => setTemplates(loadGeneratorTemplateSettings())), []);
@@ -158,6 +172,12 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
resetImageEditing(id);
onChange((current) => restoreGeneratedImage(current, id));
};
+ const replaceScene = (next: ImageScene) => {
+ undo.current = []; redo.current = []; historyChanged((value) => value + 1);
+ pointer.current = null; setDraftBox(null); setDrawKind(null); setCollapsed(new Set());
+ setSelection(next.nodes.find((node) => node.kind === "root")!.id);
+ onChange((current) => ({ ...current, imageScene: next }));
+ };
const removeImage = (id: string) => {
onChange((current) => {
const removed = current.assets.find((asset) => asset.id === id && asset.kind === "image");
@@ -326,11 +346,12 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
{ label: "Delete", icon: <Trash2 size={15} />, shortcut: "Delete", separator: true, danger: true, disabled: selected.kind === "root", action: removeSelected },
]} />}
{imageMenu && <HierarchyContextMenu {...imageMenu} label="Generated image actions" onClose={closeImageMenu} items={[
+ { label: "Edit", icon: <Pencil size={15} />, action: () => attempt(() => replaceScene(editGeneratedImage(config, imageMenu.id).imageScene!)) },
{ label: "Remove", icon: <Trash2 size={15} />, danger: true, action: () => removeImage(imageMenu.id) },
]} />}
<section className="image-center" aria-label="Image panel">
<div className="image-canvas-tools">
- <button className="primary-button image-generate-button" disabled={active ? work.cancelling || work.status === "encoding" : !isTauri() || !template || templateNeedsDownload(template) || !imageScenePrompt(scene)} onClick={() => attempt(() => active ? onCancel(work.id) : onGenerate(template!))}><Sparkles size={16} />{active ? work.cancelling ? "Cancelling…" : "Cancel" : "Generate"}</button>
+ <button className="primary-button image-generate-button" title={editPlan?.error ?? undefined} disabled={active ? work.cancelling || work.status === "encoding" : !isTauri() || !template || templateNeedsDownload(template) || (imageRoot ? Boolean(editPlan?.error) : !imageScenePrompt(scene))} onClick={() => attempt(() => active ? onCancel(work.id) : onGenerate(template!))}><Sparkles size={16} />{active ? work.cancelling ? "Cancelling…" : "Cancel" : "Generate"}</button>
<label className="image-generator">Generator<select aria-label="Generator" value={template?.id ?? ""} disabled={Boolean(active)} onChange={(event) => { setTemplateId(event.target.value); localStorage.setItem("slopus.image-generator-template.v1", event.target.value); const next = imageTemplates.find((candidate) => candidate.id === event.target.value); if (next) commit({ ...scene, steps: next.defaultSteps }); }}>
{!imageTemplates.length && <option value="">No MiniMax H3 templates</option>}{imageTemplates.map((item) => <option key={item.id} value={item.id} disabled={templateNeedsDownload(item)}>{item.name}{templateNeedsDownload(item) ? " (download in Settings)" : ""}</option>)}
</select></label>
@@ -357,7 +378,7 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
onPointerUpCapture={endPan} onPointerCancelCapture={endPan} onLostPointerCapture={endPan}
onAuxClick={(event) => { if (event.button === 1) event.preventDefault(); }}>
<div className="image-frame" style={{ "--image-ratio": width / height, "--image-zoom": view.zoom, "--image-pan-x": `${view.x}px`, "--image-pan-y": `${view.y}px` } as CSSProperties}>
- {output ? <ReferenceImage folderPath={folderPath} relativePath={output.relativePath} sourcePath={output.sourcePath} alt={output.name} /> : <div className="image-empty"><Image size={42} /><strong>Compose your image</strong><span>Add objects, text, and groups, then describe them in the inspector.</span></div>}
+ {imageRoot && scene.sourceImage ? <ReferenceImage folderPath={folderPath} relativePath={scene.sourceImage.relativePath} alt={scene.sourceImage.name} /> : output && !imageRoot ? <ReferenceImage folderPath={folderPath} relativePath={output.relativePath} sourcePath={output.sourcePath} alt={output.name} /> : <div className="image-empty"><Image size={42} /><strong>{imageRoot ? "Open an image to edit" : "Compose your image"}</strong><span>{imageRoot ? "Choose Open image in the inspector, then add Object nodes for edits." : "Add objects, text, and groups, then describe them in the inspector."}</span></div>}
<svg className={`image-overlay ${drawKind ? "drawing" : ""}`} viewBox="0 0 1000 1000" preserveAspectRatio="none" aria-label="Image placement canvas" onPointerDown={pointerDown} onPointerMove={pointerMove} onPointerUp={pointerUp} onPointerCancel={() => { pointer.current = null; setDraftBox(null); }}>
{boxes && scene.nodes.filter((node) => node.box).map((node) => {
const box = pointer.current?.id === node.id && draftBox ? draftBox : node.box!;
@@ -367,7 +388,7 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
</svg>
{boxes && scene.nodes.filter((node) => node.box).map((node) => { const box = pointer.current?.id === node.id && draftBox ? draftBox : node.box!; return <span key={node.id} className="image-box-label" style={{ left: `${box.x / 10}%`, top: `${box.y / 10}%` }}>{node.name}</span>; })}
</div></div>
- <div className="image-status" role="status">{active && <progress value={work.progress} max="1" />}<span>{active ? work.detail : !isTauri() ? "Image generation is available in the desktop app." : `${width} × ${height} · Placement boxes guide the prompt.`}</span></div>
+ <div className="image-status" role="status">{active && <progress value={work.progress} max="1" />}<span>{active ? work.detail : imageRoot ? editPlan?.error ?? `${width} × ${height} · ${editPlan?.edits.length} edits in hierarchy order` : !isTauri() ? "Image generation is available in the desktop app." : `${width} × ${height} · Placement boxes guide the prompt.`}</span></div>
{(error || work?.error) && <p className="image-error" role="alert">{error ?? work?.error}</p>}
{images.length > 0 && <div ref={imageResults} className="image-results" aria-label="Generated images">{images.map((asset) => <button key={asset.id} data-image-asset={asset.id} title={asset.name} aria-label={`View ${asset.name}`} aria-pressed={asset.id === scene.outputAssetId}
onContextMenu={(event) => { event.preventDefault(); setContextMenu(null); setImageMenu({ id: asset.id, x: event.clientX, y: event.clientY }); }}
@@ -381,10 +402,19 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
onClick={() => selectImage(asset.id)}><ReferenceImage folderPath={folderPath} relativePath={asset.relativePath} sourcePath={asset.sourcePath} alt={asset.name} /></button>)}</div>}
</section>
<aside className="image-inspector" aria-label="Image node inspector"><header><strong>Inspector</strong><span>{selected.kind}</span></header><div className="image-inspector__fields">
- <label>Name<input value={selected.name} maxLength={120} onChange={(event) => { if (event.target.value.trim()) patchNode({ name: event.target.value }); }} /></label>
- <label>{selected.kind === "root" ? "Prompt (high-level description)" : "Description"}<textarea rows={4} value={selected.description} onChange={(event) => patchNode({ description: event.target.value })} /></label>
+ {selected.kind === "root" && <label>Root type<select value={imageRoot ? "image" : "prompt"} onChange={(event) => replaceScene(event.target.value === "image" ? createImageEditScene(null, scene) : { ...createImageScene(), steps: scene.steps, seed: scene.seed })}><option value="prompt">Prompt root</option><option value="image">Image root</option></select></label>}
+ {selected.kind === "root" && imageRoot ? <>
+ <button className="secondary-button" disabled={!isTauri()} onClick={() => attempt(async () => {
+ const source = await invoke<ImageSource | null>("open_image_source", { folderPath });
+ if (source) replaceScene(createImageEditScene(source, scene));
+ })}><FolderOpen size={16} />Open image</button>
+ {scene.sourceImage && <p>{scene.sourceImage.name} · {scene.sourceImage.width} × {scene.sourceImage.height}</p>}
+ </> : <>
+ <label>Name<input value={selected.name} maxLength={120} onChange={(event) => { if (event.target.value.trim()) patchNode({ name: event.target.value }); }} /></label>
+ <label>{selected.kind === "root" ? "Prompt (high-level description)" : "Description"}<textarea rows={4} value={selected.description} onChange={(event) => patchNode({ description: event.target.value })} /></label>
+ </>}
{selected.kind === "text" && <label>Text to render<input value={selected.text} onChange={(event) => patchNode({ text: event.target.value })} /></label>}
- {selected.kind === "root" && <>
+ {selected.kind === "root" && !imageRoot && <>
<label>Background (environment)<textarea rows={3} value={scene.background} onChange={(event) => commit({ ...scene, background: event.target.value })} /></label>
<h3>Image generation</h3><label>Steps<input type="number" min="2" max="1000" value={scene.steps} onChange={(event) => { const value = Number(event.target.value); if (Number.isInteger(value) && value >= 2 && value <= 1000) commit({ ...scene, steps: value }); }} /></label>
<label>Seed (−1 = random)<input type="number" min="-1" max={Number.MAX_SAFE_INTEGER} value={scene.seed} onChange={(event) => { const value = Number(event.target.value); if (Number.isSafeInteger(value) && value >= -1) commit({ ...scene, seed: value }); }} /></label>
@@ -398,9 +428,9 @@ export function ImageEditor({ config, folderPath, onChange, onGenerate, onCancel
<h3>References</h3>{config.references.filter((reference) => reference.kind === "image" || reference.kind === "text").map((reference) => <label className="image-check" key={reference.id}><input type="checkbox" checked={scene.referenceIds.includes(reference.id)} onChange={(event) => commit({ ...scene, referenceIds: event.target.checked ? [...scene.referenceIds, reference.id] : scene.referenceIds.filter((id) => id !== reference.id) })} />{reference.name}</label>)}{!config.references.length && <p>Add image or text references in the References tab.</p>}
</>}
{selected.kind !== "root" && <><h3>Placement (0–1000)</h3><label className="image-check"><input type="checkbox" checked={Boolean(selected.box)} onChange={(event) => patchNode({ box: event.target.checked ? { x: 250, y: 250, width: 500, height: 500 } : null })} />Explicit placement</label>{selected.box && <div className="image-box-fields">{(["x", "y", "width", "height"] as const).map((field) => <label key={field}>{field}<input type="number" min={field === "x" || field === "y" ? 0 : 1} max="1000" value={selected.box![field]} onChange={(event) => { const value = Number(event.target.value); const box = { ...selected.box!, [field]: value }; if (Number.isFinite(value) && box.x >= 0 && box.y >= 0 && box.width > 0 && box.height > 0 && box.x + box.width <= 1000 && box.y + box.height <= 1000) commit(resizeImageNode(scene, selected.id, box)); }} /></label>)}</div>}</>}
- <h3>Color palette</h3><div className="image-palette">{selected.colors.map((color, index) => <div key={index}><input aria-label={`Palette color ${index + 1}`} type="color" value={color} onChange={(event) => patchNode({ colors: selected.colors.map((old, i) => i === index ? event.target.value : old) })} /><button className="icon-button" aria-label={`Remove color ${index + 1}`} onClick={() => patchNode({ colors: selected.colors.filter((_, i) => i !== index) })}><Trash2 size={13} /></button></div>)}<button className="secondary-button" disabled={selected.colors.length >= (selected.kind === "root" ? 16 : 5)} onClick={() => patchNode({ colors: [...selected.colors, "#808080"] })}><Plus size={14} />Color</button></div>
+ {!(selected.kind === "root" && imageRoot) && <><h3>Color palette</h3><div className="image-palette">{selected.colors.map((color, index) => <div key={index}><input aria-label={`Palette color ${index + 1}`} type="color" value={color} onChange={(event) => patchNode({ colors: selected.colors.map((old, i) => i === index ? event.target.value : old) })} /><button className="icon-button" aria-label={`Remove color ${index + 1}`} onClick={() => patchNode({ colors: selected.colors.filter((_, i) => i !== index) })}><Trash2 size={13} /></button></div>)}<button className="secondary-button" disabled={selected.colors.length >= (selected.kind === "root" ? 16 : 5)} onClick={() => patchNode({ colors: [...selected.colors, "#808080"] })}><Plus size={14} />Color</button></div></>}
</div>
- {debugEnabled && <div className="debug-prompt"><button type="button" className="secondary-button debug-prompt__toggle" aria-haspopup="dialog" onClick={() => attempt(() => setDebugPrompt(compileImagePrompt(config).prompt))}>Debug Prompt</button></div>}
+ {debugEnabled && <div className="debug-prompt"><button type="button" className="secondary-button debug-prompt__toggle" aria-haspopup="dialog" onClick={() => attempt(() => setDebugPrompt(imageRoot ? imageEditDebugPrompt(compileImageEdits(config).edits) : compileImagePrompt(config).prompt))}>Debug Prompt</button></div>}
</aside>
{debugEnabled && debugPrompt !== null && <DebugPromptDialog sceneTitle={config.name} segments={[{ kind: "brief", value: debugPrompt }]} onClose={() => setDebugPrompt(null)} />}
</div>;
diff --git a/src/lib/imageCommands.ts b/src/lib/imageCommands.ts
index dc284c7..7c37889 100644
--- a/src/lib/imageCommands.ts
+++ b/src/lib/imageCommands.ts
@@ -4,7 +4,7 @@ import { imageDescendants, imageNodeSchema, imageSceneSchema, removeImageNode, r
const id = z.string().min(1);
const before = id.nullish();
const editableNode = imageNodeSchema.pick({ name: true, description: true, text: true, box: true, colors: true }).partial();
-const authoredScene = imageSceneSchema.innerType().omit({ outputAssetId: true });
+const authoredScene = imageSceneSchema.innerType().omit({ outputAssetId: true, rootType: true, sourceImage: true });
const configuration = authoredScene.omit({ nodes: true, style: true }).partial().extend({ prompt: z.string().optional(), style: authoredScene.shape.style.partial().strict().optional() });
export const imageCommandSchema = z.discriminatedUnion("op", [
authoredScene.extend({ op: z.literal("image.set") }).strict(),
@@ -41,7 +41,7 @@ export function applyImageCommand(current: ImageScene, command: ImageCommand): I
switch (command.op) {
case "image.set": {
const { op: _op, ...authored } = command;
- scene = { ...authored, outputAssetId: current.outputAssetId };
+ scene = { ...current, ...authored, outputAssetId: current.outputAssetId };
break;
}
case "image.configure": {
diff --git a/src/lib/imageEditing.test.ts b/src/lib/imageEditing.test.ts
new file mode 100644
index 0000000..cf2f827
--- /dev/null
+++ b/src/lib/imageEditing.test.ts
@@ -0,0 +1,42 @@
+import { expect, it } from "vitest";
+import { parseProjectConfig } from "./project";
+import { addImageNode, createImageEditScene } from "./imageScene";
+import { compileImageEdits, editGeneratedImage } from "./imageEditing";
+import fixture from "../../fixtures/project-v1-image.json";
+
+it("uses hierarchy order, skips empty groups, and converts boxes to original pixels", () => {
+ const config = parseProjectConfig(fixture);
+ let scene = createImageEditScene({ relativePath: "media/imported/source.png", name: "Source", width: 101, height: 77 });
+ scene = addImageNode(scene, "image-root", "group");
+ const group = scene.nodes[1].id;
+ scene = addImageNode(scene, "image-root", "object");
+ scene.nodes[2].description = "A blue boat";
+ scene = addImageNode(scene, group, "object", { x: 900, y: 900, width: 100, height: 100 });
+ scene.nodes[3].description = "A red balloon";
+ config.imageScene = scene;
+ config.settings.defaultLook = "watercolor";
+ const { edits } = compileImageEdits(config);
+ expect(edits).toHaveLength(2);
+ expect(edits[0]).toMatchObject({ x: 90, y: 69, width: 11, height: 8 });
+ expect(edits[0].prompt).toContain("A red balloon");
+ expect(edits[1]).toMatchObject({ x: 25, y: 19, width: 51, height: 39 });
+ expect(edits[1].prompt).toContain("A blue boat");
+ expect(edits[0].prompt).not.toContain("watercolor");
+ expect(edits[0].prompt).toContain("[Shot 1]");
+ scene.nodes[3].box = null;
+ expect(() => compileImageEdits(config)).toThrow("placement box");
+});
+
+it("starts with a clean image root and requires described edits before submission", () => {
+ const config = parseProjectConfig(fixture);
+ config.assets.push({ id: "source", name: "Source", relativePath: "media/generated/source.jpg", mimeType: "image/jpeg", width: 101, height: 77, kind: "image", createdAt: config.createdAt });
+ const edited = editGeneratedImage(config, "source");
+ expect(edited.imageScene).toMatchObject({ rootType: "image", sourceImage: { width: 101, height: 77 }, outputAssetId: null });
+ expect(edited.imageScene!.nodes).toHaveLength(1);
+ expect(edited.imageScene!.nodes[0].description).toBe("");
+ expect(() => compileImageEdits(edited)).toThrow("Add an Object");
+ edited.imageScene = addImageNode(edited.imageScene!, "image-root", "object");
+ expect(() => compileImageEdits(edited)).toThrow("Describe the edit");
+ expect(config.imageScene!.nodes.length).toBeGreaterThan(1);
+ expect(parseProjectConfig(JSON.parse(JSON.stringify(edited))).imageScene).toEqual(edited.imageScene);
+});
diff --git a/src/lib/imageEditing.ts b/src/lib/imageEditing.ts
new file mode 100644
index 0000000..ec94648
--- /dev/null
+++ b/src/lib/imageEditing.ts
@@ -0,0 +1,38 @@
+import { createImageEditScene, createImageScene, imageSourceSchema, type ImageNode, type ImageSource } from "./imageScene";
+import { compileImagePrompt } from "./imagePrompt";
+import type { ProjectConfig } from "./project";
+
+export interface ImageEditStep { prompt: string; x: number; y: number; width: number; height: number }
+export const imageEditDebugPrompt = (edits: ImageEditStep[]) => edits.map((edit, index) => `Edit ${index + 1} (${edit.x}, ${edit.y}, ${edit.width}, ${edit.height})\n${edit.prompt}`).join("\n\n");
+export function compileImageEdits(config: ProjectConfig): { source: ImageSource; edits: ImageEditStep[] } {
+ const scene = config.imageScene!;
+ const source = scene.sourceImage;
+ if (!source) throw new Error("Open an image before adding edits.");
+ const ordered: ImageNode[] = [];
+ const visit = (parent: string) => { for (const node of scene.nodes.filter((item) => item.parentId === parent)) { ordered.push(node); visit(node.id); } };
+ visit(scene.nodes.find((node) => node.kind === "root")!.id);
+ const edits = ordered.filter((node) => node.kind !== "group" || node.description.trim() || node.text || node.colors.length).map((node) => {
+ if (!node.box) throw new Error(`Set a placement box for '${node.name}'.`);
+ if (!node.description.trim() && !node.text && !node.colors.length) throw new Error(`Describe the edit for '${node.name}'.`);
+ const x = Math.min(source.width - 1, Math.floor(node.box.x * source.width / 1000));
+ const y = Math.min(source.height - 1, Math.floor(node.box.y * source.height / 1000));
+ const right = Math.min(source.width, Math.ceil((node.box.x + node.box.width) * source.width / 1000));
+ const bottom = Math.min(source.height, Math.ceil((node.box.y + node.box.height) * source.height / 1000));
+ const description = ["Edit the masked rectangle of the supplied image. Preserve the surrounding composition, medium and lighting.", node.description,
+ node.text ? `Render the exact text "${node.text}".` : "", node.colors.length ? `Use these colors: ${node.colors.join(", ")}.` : ""].filter(Boolean).join(" ");
+ const editScene = { ...createImageScene(description), referenceIds: scene.referenceIds };
+ const compiled = compileImagePrompt({ ...config, settings: { ...config.settings, defaultLook: null }, imageScene: editScene });
+ // Keep the existing image's medium rather than imposing Photo/Art defaults.
+ const prompt = compiled.prompt.replace("A still photograph.", "A still image retaining the supplied image's visual style.");
+ return { prompt, x, y, width: Math.max(1, right - x), height: Math.max(1, bottom - y) };
+ });
+ if (!edits.length) throw new Error("Add an Object node and describe an edit before generating.");
+ return { source, edits };
+}
+
+export function editGeneratedImage(config: ProjectConfig, id: string): ProjectConfig {
+ const asset = config.assets.find((item) => item.id === id && item.kind === "image");
+ if (!asset?.relativePath || !asset.width || !asset.height) throw new Error("The image must have a saved file and dimensions before editing.");
+ const source = imageSourceSchema.parse({ relativePath: asset.relativePath, name: asset.name, width: asset.width, height: asset.height });
+ return { ...config, imageScene: createImageEditScene(source, config.imageScene ?? undefined) };
+}
diff --git a/src/lib/imageHistory.test.ts b/src/lib/imageHistory.test.ts
index dbf9b1e..225d2c8 100644
--- a/src/lib/imageHistory.test.ts
+++ b/src/lib/imageHistory.test.ts
@@ -3,6 +3,7 @@ import { imageGenerationSnapshot, restoreGeneratedImage } from "./imageHistory";
import { imageGenerationSnapshotSchema, parseProjectConfig } from "./project";
import fixture from "../../fixtures/project-v1-image.json";
import snapshotFixture from "../../fixtures/image-generation-snapshot.json";
+import { createImageEditScene } from "./imageScene";
describe("generated image history", () => {
it("restores saved inputs and references after reopening without changing other results", () => {
@@ -49,4 +50,14 @@ describe("generated image history", () => {
expect(imageGenerationSnapshotSchema.safeParse({ ...snapshotFixture, references: [] }).success).toBe(false);
expect(imageGenerationSnapshotSchema.safeParse({ ...snapshotFixture, references: [...snapshotFixture.references, ...snapshotFixture.references] }).success).toBe(false);
});
+
+ it("leaves image-root mode when selecting an older result without a snapshot", () => {
+ const config = parseProjectConfig(fixture);
+ config.imageScene = createImageEditScene({ relativePath: "media/imported/source.png", name: "Source", width: 65, height: 41 });
+ config.assets = [{ id: "legacy", kind: "image", name: "Legacy", relativePath: "media/generated/legacy.jpg", mimeType: "image/jpeg", createdAt: config.createdAt }];
+ const restored = restoreGeneratedImage(config, "legacy");
+ expect(restored.imageScene!.rootType).not.toBe("image");
+ expect(restored.imageScene!.outputAssetId).toBe("legacy");
+ expect(restored.imageScene!.sourceImage).toBeUndefined();
+ });
});
diff --git a/src/lib/imageHistory.ts b/src/lib/imageHistory.ts
index 7a789e1..abc6636 100644
--- a/src/lib/imageHistory.ts
+++ b/src/lib/imageHistory.ts
@@ -1,4 +1,4 @@
-import { createImageScene } from "./imageScene";
+import { createImageEditScene, createImageScene } from "./imageScene";
import type { ImageGenerationSnapshot, ProjectConfig } from "./project";
export function imageGenerationSnapshot(config: ProjectConfig, prompt: string, generatorTemplateId: string): ImageGenerationSnapshot {
@@ -21,11 +21,14 @@ export function restoreGeneratedImage(config: ProjectConfig, id: string): Projec
if (!asset) return config;
const snapshot = asset.imageGeneration;
if (!snapshot) return { ...config, thumbnail: asset.relativePath ?? null,
- imageScene: { ...(config.imageScene ?? createImageScene(config.brief.prompt)), outputAssetId: id } };
+ imageScene: { ...(config.imageScene?.rootType === "image" ? createImageScene() : config.imageScene ?? createImageScene(config.brief.prompt)), outputAssetId: id } };
const restored = structuredClone(snapshot);
+ const scene = restored.scene.rootType === "image" && asset.relativePath && asset.width && asset.height
+ ? createImageEditScene({ relativePath: asset.relativePath, name: asset.name, width: asset.width, height: asset.height }, restored.scene)
+ : restored.scene;
const references = new Map(restored.references.map((reference) => [reference.id, reference]));
return { ...config, thumbnail: asset.relativePath ?? null,
- imageScene: { ...restored.scene, outputAssetId: id },
+ imageScene: { ...scene, outputAssetId: id },
settings: { ...config.settings, resolution: restored.resolution, aspectRatio: restored.aspectRatio, defaultLook: restored.defaultLook },
brief: { ...config.brief, prompt: restored.briefPrompt, resolution: restored.resolution, aspectRatio: restored.aspectRatio },
references: [...config.references.map((reference) => references.get(reference.id) ?? reference),
diff --git a/src/lib/imageScene.ts b/src/lib/imageScene.ts
index 1263168..794d3c0 100644
--- a/src/lib/imageScene.ts
+++ b/src/lib/imageScene.ts
@@ -8,7 +8,14 @@ export const imageNodeSchema = z.object({
name: z.string().min(1).max(120), description: z.string(), text: z.string(),
box: boxSchema.nullable(), colors: z.array(z.string().regex(/^#[0-9a-fA-F]{6}$/)).max(16),
});
+export const imageSourceSchema = z.object({
+ relativePath: z.string().min(1).refine((path) => !path.startsWith("/") && !path.includes("\\") && !path.includes(":") && !path.split("/").includes(".."), "Image source must be a project-relative path."),
+ name: z.string().min(1), width: z.number().int().min(1).max(8192), height: z.number().int().min(1).max(8192),
+});
+export type ImageSource = z.infer<typeof imageSourceSchema>;
export const imageSceneSchema = z.object({
+ rootType: z.enum(["prompt", "image"]).optional(),
+ sourceImage: imageSourceSchema.nullish(),
nodes: z.array(imageNodeSchema).min(1).max(500),
background: z.string(),
style: z.object({ mode: z.enum(["photo", "art"]), aesthetics: z.string(), lighting: z.string(), medium: z.string(), detail: z.string() }),
@@ -38,6 +45,10 @@ export function createImageScene(prompt = ""): ImageScene {
return { nodes: [{ id: "image-root", parentId: null, kind: "root", name: "Image", description: prompt, text: "", box: null, colors: [] }],
background: "", style: { mode: "photo", aesthetics: "", lighting: "", medium: "", detail: "" }, steps: 20, seed: -1, referenceIds: [], outputAssetId: null };
}
+export function createImageEditScene(sourceImage: ImageSource | null, previous?: ImageScene): ImageScene {
+ return { ...createImageScene(), rootType: "image", sourceImage,
+ steps: previous?.steps ?? 20, seed: previous?.seed ?? -1, referenceIds: previous?.referenceIds ?? [] };
+}
export function imageDescendants(scene: ImageScene, id: string): Set<string> {
const ids = new Set([id]);
for (let changed = true; changed;) {
diff --git a/src/lib/runtime.ts b/src/lib/runtime.ts
index 014bd3d..2f7a0a7 100644
--- a/src/lib/runtime.ts
+++ b/src/lib/runtime.ts
@@ -88,6 +88,7 @@ export interface SlopfabGenerationRequest {
prompt: string;
frames: number;
stillImage?: boolean;
+ imageEdit?: { sourceRelativePath: string; edits: import("./imageEditing").ImageEditStep[] };
steps: number;
seed: number;
canvasWidth: number;
diff --git a/src/lib/workQueue.test.ts b/src/lib/workQueue.test.ts
index 60ab775..950103e 100644
--- a/src/lib/workQueue.test.ts
+++ b/src/lib/workQueue.test.ts
@@ -13,6 +13,8 @@ import { saveSceneLastFrame } from "./sceneLastFrame";
import { prepareReferenceVideos, releaseReferenceVideos } from "./referenceVideo";
import { downloadTemplateWeights, getWeightDownloadState } from "./weightDownloads";
import { compileImagePrompt } from "./imagePrompt";
+import { addImageNode, createImageEditScene } from "./imageScene";
+import { restoreGeneratedImage } from "./imageHistory";
vi.mock("@tauri-apps/api/core", () => ({ invoke: vi.fn() }));
vi.mock("@tauri-apps/api/event", () => ({ listen: vi.fn() }));
@@ -56,6 +58,33 @@ const finish = async (queue: WorkQueue, id: string) => {
describe("image generation work", () => {
const template: GeneratorTemplate = { id: "image-test", name: "MiniMax H3", defaultSteps: 20, attention: "sage2", paths: { ...EMPTY_ENGINE_SETTINGS, transformer: "C:/h3.safetensors" } };
const imageProject = (): ProjectRecord => ({ folderPath: "C:/Image", config: createProjectConfig({ name: "Poster", prompt: "An ocean poster", generationType: "image", aspectRatio: "1:1", resolution: "768p", targetDurationSeconds: 60 }) });
+ it("submits ordered inpainting as one job, saves PNG, and leaves a clean edit root", async () => {
+ const { queue } = setup();
+ const record = imageProject();
+ let scene = createImageEditScene({ name: "Source", relativePath: "media/generated/source.jpg", width: 101, height: 77 });
+ for (const description of ["A red balloon", "A blue boat"]) {
+ scene = addImageNode(scene, "image-root", "object");
+ scene.nodes.at(-1)!.description = description;
+ }
+ record.config.imageScene = scene;
+ const session = queue.project(record);
+ vi.mocked(invoke).mockResolvedValue({ relativePath: "media/generated/edited.png", width: 101, height: 77 });
+ queue.enqueueImage(session, template);
+ await waitFor(() => expect(enqueueSlopfabGeneration).toHaveBeenCalledOnce());
+ const request = vi.mocked(enqueueSlopfabGeneration).mock.calls[0][0];
+ expect(request).toMatchObject({ stillImage: true, frames: 1, canvasWidth: 101, canvasHeight: 77, imageEdit: { sourceRelativePath: "media/generated/source.jpg", edits: [
+ { prompt: expect.stringContaining("A red balloon"), x: 25, y: 19, width: 51, height: 39 },
+ { prompt: expect.stringContaining("A blue boat"), x: 25, y: 19, width: 51, height: 39 },
+ ] } });
+ await finish(queue, request.jobId);
+ expect(invoke).toHaveBeenCalledWith("save_generated_image", { folderPath: record.folderPath, jobId: request.jobId, format: "png" });
+ const result = session.getSnapshot().config;
+ expect(result.imageScene!.nodes).toHaveLength(1);
+ expect(result.imageScene).toMatchObject({ rootType: "image", sourceImage: { relativePath: "media/generated/edited.png", width: 101, height: 77 } });
+ expect(result.assets[0]).toMatchObject({ mimeType: "image/png", imageGeneration: { scene, prompt: expect.stringContaining("Edit 2 (25, 19, 51, 39)") } });
+ expect(restoreGeneratedImage(result, request.jobId).imageScene!.nodes).toHaveLength(1);
+ expect(releaseRendered).toHaveBeenCalledWith(request.jobId);
+ });
it("forwards selected refmods to still-image planning and generation with their strength and copies", async () => {
const { queue } = setup();
const project = imageProject();
diff --git a/src/lib/workQueue.ts b/src/lib/workQueue.ts
index e588f52..6ac1c24 100644
--- a/src/lib/workQueue.ts
+++ b/src/lib/workQueue.ts
@@ -1,6 +1,7 @@
import { listen } from "@tauri-apps/api/event";
import { invoke } from "@tauri-apps/api/core";
-import { createImageScene, imageScenePrompt } from "./imageScene";
+import { createImageEditScene, createImageScene, imageScenePrompt } from "./imageScene";
+import { compileImageEdits, imageEditDebugPrompt } from "./imageEditing";
import { compileImagePrompt } from "./imagePrompt";
import { imageGenerationSnapshot } from "./imageHistory";
import type { ImageGenerationSnapshot } from "./project";
@@ -203,22 +204,24 @@ export class WorkQueue {
const current = session.getSnapshot().config;
if (current.generationType !== "image") throw new Error("Open an image project to generate images.");
const scene = current.imageScene ?? createImageScene(current.brief.prompt);
+ const edit = scene.rootType === "image" ? compileImageEdits(current) : null;
if (!imageScenePrompt(scene)) throw new Error("Describe the image or add an object before generating.");
const projectKey = projectQueueKey(session.record);
if (this.items.some((item) => item.projectKey === projectKey && item.kind === "image" && isWorkActive(item))) return;
const config = structuredClone({ ...current, providerSettings: { ...current.providerSettings,
slopfab: engineProviderSetting(template.paths, current.providerSettings.slopfab, template.attention, template.loras, "prompt", template.additionalSafetensors, false) } });
const { prompt, references } = compileImagePrompt(current);
- const { width, height } = outputDimensions(current.settings.resolution, current.settings.aspectRatio);
+ const { width, height } = edit?.source ?? outputDimensions(current.settings.resolution, current.settings.aspectRatio);
const id = `image-${crypto.randomUUID()}`;
let finish!: () => void;
const done = new Promise<void>((resolve) => { finish = resolve; });
- const request: SlopfabGenerationRequest = { jobId: id, stillImage: true, frames: 1, prompt,
+ const request: SlopfabGenerationRequest = { jobId: id, stillImage: true, frames: 1, prompt: edit ? edit.edits[0].prompt : prompt,
+ ...(edit ? { imageEdit: { sourceRelativePath: edit.source.relativePath, edits: edit.edits } } : {}),
canvasWidth: width, canvasHeight: height, steps: generationStepsWithLoras(scene.steps, config), seed: scene.seed,
referencePaths: references.flatMap((reference) => referenceImages(reference).map((image) => projectItemPath(session.record.folderPath, image)!)),
refmods: referenceRefmodInputs(session.record.folderPath, references),
};
- this.work.set(id, { id, image: true, imageGeneration: imageGenerationSnapshot(config, prompt, template.id), session, sceneId: scene.nodes.find((node) => node.kind === "root")!.id, config, snapshot: JSON.stringify(scene), request, submitted: false, cancelled: false, done, finish });
+ this.work.set(id, { id, image: true, imageGeneration: imageGenerationSnapshot(config, edit ? imageEditDebugPrompt(edit.edits) : prompt, template.id), session, sceneId: scene.nodes.find((node) => node.kind === "root")!.id, config, snapshot: JSON.stringify(scene), request, submitted: false, cancelled: false, done, finish });
this.items = [...this.items, { id, kind: "image", projectKey, folderPath: session.record.folderPath, projectName: config.name, sceneId: scene.nodes.find((node) => node.kind === "root")!.id,
title: "Image · " + config.name, submittedAt: new Date().toISOString(), status: "queued", progress: 0, detail: "Waiting to generate image", error: null, completionAt: null, cancelling: false, needsSave: false,
settings: { frames: 1, steps: request.steps, seed: request.seed, canvasWidth: width, canvasHeight: height } }];
@@ -334,7 +337,7 @@ export class WorkQueue {
const item = this.items.find((candidate) => candidate.id === event.jobId);
if (!work || !item || !["preparing", "generating"].includes(item.status)) return;
const progress = Math.max(item.progress, event.totalSteps > 0 ? Math.min(0.88, 0.12 + Math.max(0, event.step) / event.totalSteps * 0.76) : event.stage === "delivering" ? 0.88 : 0.08);
- this.patch(work.id, { status: "generating", progress, completionAt: this.timing.update(event), detail: item.cancelling ? "Cancelling generation" : event.stage === "starting" || event.stage === "transformerLoad" ? "Loading MiniMax H3" : work.image ? "Generating image" : "Generating video" });
+ this.patch(work.id, { status: "generating", progress, completionAt: this.timing.update(event), detail: item.cancelling ? "Cancelling generation" : work.request.imageEdit ? `Applying image edits (${Math.min(work.request.imageEdit.edits.length, Math.floor(event.step / Math.max(1, event.totalSteps) * work.request.imageEdit.edits.length) + 1)} of ${work.request.imageEdit.edits.length})` : event.stage === "starting" || event.stage === "transformerLoad" ? "Loading MiniMax H3" : work.image ? "Generating image" : "Generating video" });
this.updateScene(work, { status: "generating", stage: event.stage === "starting" || event.stage === "transformerLoad" ? "preparing" : "generating", progress }, false);
}
private event(event: JobEvent) {
@@ -398,10 +401,10 @@ export class WorkQueue {
private async saveImage(work: PendingWork) {
this.patch(work.id, { status: "encoding", progress: 0.9, detail: "Saving image", completionAt: null });
try {
- const saved = await invoke<{ relativePath: string; width: number; height: number }>("save_generated_image", { folderPath: work.session.record.folderPath, jobId: work.id });
+ const saved = await invoke<{ relativePath: string; width: number; height: number }>("save_generated_image", { folderPath: work.session.record.folderPath, jobId: work.id, ...(work.request.imageEdit ? { format: "png" } : {}) });
work.session.update((current) => ({ ...current, thumbnail: saved.relativePath,
- imageScene: { ...(current.imageScene ?? createImageScene()), outputAssetId: work.id },
- assets: [...current.assets, { id: work.id, kind: "image", name: `Image ${current.assets.filter((asset) => asset.kind === "image").length + 1}`, ...saved, imageGeneration: work.imageGeneration, mimeType: "image/jpeg", createdAt: new Date().toISOString() }],
+ imageScene: { ...(work.request.imageEdit && JSON.stringify(current.imageScene) === work.snapshot ? createImageEditScene({ ...saved, name: "Edited image" }, current.imageScene ?? undefined) : current.imageScene ?? createImageScene()), outputAssetId: work.id },
+ assets: [...current.assets, { id: work.id, kind: "image", name: `Image ${current.assets.filter((asset) => asset.kind === "image").length + 1}`, ...saved, imageGeneration: work.imageGeneration, mimeType: work.request.imageEdit ? "image/png" : "image/jpeg", createdAt: new Date().toISOString() }],
}));
try { await work.session.save(); this.patch(work.id, { status: "completed", progress: 1, detail: "Image saved" }); }
catch (reason) { this.patch(work.id, { status: "failed", progress: 1, needsSave: true, detail: "Image created; project save failed", error: describeDiagnosticError(reason) }); }