Skip to content

AI media harness for planning, generating, and editing locally

TypeScript

Reduce Zoom extension seams

Bittia7ee96f70d15
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index cde774c..c40dda1 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -30,7 +30,7 @@ This alpha update adds SeedVR2 and Real-ESRGAN export upscaling, Long Shot and B
- Fixed CUDA transformer-loading memory failures caused by pinning entire large checkpoints in system memory. Uploads now use bounded buffers, account for weight-conversion memory and retry allocation failures with more CPU offloading while preserving precision and output settings.
- Agent chat appends replies and retains earlier messages, corrections and errors throughout the conversation. Raw JSON and command payloads stay hidden; readable replies and summaries are shown.
- Backdrop prompts describe a flat, constant color field in every shot, limit reference guidance to subject appearance, and use neutral studio styling when no look is specified.
-- Zoom uses SlopFab's native edge blend directly, avoiding a second feather or color correction when saving. Source transparency is retained.
+- Zoom reduces visible extension seams by keeping grey canvas padding out of source VAE encoding and widening the native edge blend. The full source remains the composition reference, small source areas retain an unblended interior, and source transparency is retained.
- Generate all includes the final scene in an even-length Long Shot sequence, continuing from the preceding scene and preserving earlier joined video and audio. Four scenes generate in 1 → 3 → 2 → 4 order; adding another scene turns the final continuation into a bridge on its next generation.
- Long Shot timeline playback keeps the same video player running across adjacent scenes in a joined generation, avoiding a playback restart at bridge boundaries.
- Reduced memory retained between generations by releasing consumed SlopFab samples and handles, completed queue snapshots, encoder buffers on errors, and abandoned LAN generation results.
diff --git a/docs/slopfab-compatibility.md b/docs/slopfab-compatibility.md
index ae244d7..e88a9c3 100644
--- a/docs/slopfab-compatibility.md
+++ b/docs/slopfab-compatibility.md
@@ -1,5 +1,33 @@
# Slopfab compatibility
+## Zoom boundary correction (1.28.2)
+
+The bundled Windows and Linux runtimes use SlopFab commit `8fe7da0` (1.28.2),
+built with `SLOPFAB_WITH_FFMPEG=OFF`. Outpaint VAE preparation extends source
+edge pixels into the unknown canvas before encoding. Previously, grey padding
+entered the VAE receptive field and contaminated locked source-edge latents
+and positioned anchors with a rectangular boundary. Only the encoding input
+changes: the semantic source reference, source rectangle, latent mask and
+final composite still use their original geometry. Both CUDA and Vulkan use
+this shared preparation path. Ordinary image edits are unchanged.
+
+Zoom requests a 33-pixel Gaussian kernel, blending up to 32 pixels inside
+source edges facing generated space (previously 8). For small intersections,
+the kernel shrinks to retain an unblended interior at least 32 pixels wide.
+Canvas edges without generated space do not fade. The save path retains the
+native RGB result and restores source alpha. Crop regeneration is unchanged.
+
+This is a runtime fix with no new C API or worker protocol. Restart Slopus
+and update LAN workers to load the replacement library. Regression coverage
+checks that unknown canvas pixels cannot affect outpaint encoding, source
+pixels remain intact, and small intersections retain an unblended interior.
+A real CUDA comparison at 416 x 416, 30 steps and seed 17 was visually checked;
+both runs preserved their source interiors exactly and used 174 evaluations.
+Slopus's GPU generation/save smoke test also passed with the bundled runtime;
+the saved PNG retained the native blend exactly. Inpaint regression tests and
+C API/tokenizer tests passed on Windows and Linux, along with the frontend
+build, focused Zoom tests and 61 Slopus runtime integration tests.
+
## CUDA transformer loading memory recovery
Large transformer checkpoints now upload through a bounded staging ring
diff --git a/lib/slopfab/libslopfab.so b/lib/slopfab/libslopfab.so
index 83acdb7..fe1203b 100644
--- a/lib/slopfab/libslopfab.so
+++ b/lib/slopfab/libslopfab.so
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
-oid sha256:c27c47ef8da6ecf6f61f15aa5c4ba2aa16656ad2995e17866b7340887d42879d
+oid sha256:4fb9fdf31a2901ff808d9d64345134522dc47bf7e934d121b804adfa0c2bc922
size 20375072
diff --git a/lib/slopfab/slopfab.dll b/lib/slopfab/slopfab.dll
index 54b41a3..38c2b3d 100644
--- a/lib/slopfab/slopfab.dll
+++ b/lib/slopfab/slopfab.dll
@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
-oid sha256:75a2a0f7047e620fffdce493ee1a636673097284c8b5e6093b33c6a01816439d
-size 17656832
+oid sha256:554ebc8b8948832af537ce9426f368fd6cbf2ceec37ecd72a393e205aa9cd074
+size 17657344
diff --git a/src-tauri/src/commands/extend_image.rs b/src-tauri/src/commands/extend_image.rs
index 6bfa7ed..e8e4483 100644
--- a/src-tauri/src/commands/extend_image.rs
+++ b/src-tauri/src/commands/extend_image.rs
@@ -283,7 +283,6 @@ pub(crate) fn discard_extend_image(folder_path: String, job_id: String) -> Resul
mod tests {
use super::*;
- /// Reuse a GPU smoke capture to compare compositing without another inference.
/// Opt-in GPU smoke test. Supply an existing source image, model directory
/// and artifact directory; ordinary test runs do not load any models.
#[test]
@@ -402,7 +401,7 @@ mod tests {
let step = if regenerate {
serde_json::json!({"x":0,"y":0,"width":size,"height":size,"strength":0.65,"feather":0,"prompt":prompt})
} else {
- serde_json::json!({"x":69,"y":69,"width":277,"height":277,"invertMask":true,"feather":0,"outpaintBlendOverlap":9,"outpaintLangevinSteps":5,"prompt":prompt})
+ serde_json::json!({"x":69,"y":69,"width":277,"height":277,"invertMask":true,"feather":0,"outpaintBlendOverlap":33,"outpaintLangevinSteps":5,"prompt":prompt})
};
let mut request: GenerationRequest = serde_json::from_value(serde_json::json!({
"jobId":job, "stillImage":true, "frames":1, "steps":20, "seed":17, "canvasWidth":size, "canvasHeight":size, "prompt":prompt,
@@ -452,8 +451,8 @@ mod tests {
assert_ne!(result, snapshot, "Generation must change the resized crop");
} else {
assert_eq!(result, raw, "Saving must retain the native Gaussian blend");
- for y in 8..snapshot.height() - 8 {
- for x in 8..snapshot.width() - 8 {
+ for y in 32..snapshot.height() - 32 {
+ for x in 32..snapshot.width() - 32 {
assert_eq!(result.get_pixel(x + 69, y + 69), snapshot.get_pixel(x, y));
}
}
diff --git a/src/lib/extendImage.test.ts b/src/lib/extendImage.test.ts
index 3266f0d..a3e5b3c 100644
--- a/src/lib/extendImage.test.ts
+++ b/src/lib/extendImage.test.ts
@@ -33,7 +33,7 @@ it("generates the entire surround in one pass while preserving the original in a
for (const x of [-64, 0, 32]) for (const y of [-32, 0, 32]) {
const { layout, edits } = compileExtendImage(config, source, "cache/canvas.png", { bounds: { x, y, width: 192, height: 128 }, prompt: "Forest", steps: 20, seed: 0 });
expect(edits).toHaveLength(1);
- expect(edits[0]).toMatchObject({ ...extendContext(layout.preserved), invertMask: true, feather: 0, outpaintBlendOverlap: 9, outpaintLangevinSteps: 5 });
+ expect(edits[0]).toMatchObject({ ...extendContext(layout.preserved), invertMask: true, feather: 0, outpaintBlendOverlap: 33, outpaintLangevinSteps: 5 });
const contains = (r: { x: number; y: number; width: number; height: number }, px: number, py: number) => px >= r.x && py >= r.y && px < r.x + r.width && py < r.y + r.height;
let allNewPixelsGenerated = true;
for (let py = 0; py < layout.output.height; py++) for (let px = 0; px < layout.output.width; px++) {
@@ -133,6 +133,21 @@ it("passes the full source rectangle to native blending and retains a positioned
expect(() => extendContext({ x: 16, y: 16, width: 32, height: 32 })).toThrow("larger area");
});
+it("limits the blend for small source intersections without erasing their interior", () => {
+ const config = createProjectConfig({ name: "Zoom", prompt: "", aspectRatio: "1:1", resolution: "416p", targetDurationSeconds: 15 });
+ for (const width of [32, 64, 96, 128, 256]) {
+ const source = { id: "source", name: "Source", kind: "image" as const, mimeType: "image/png", width, height: width, createdAt: config.createdAt };
+ const { edits: [edit] } = compileExtendImage(config, source, "cache/canvas.png",
+ { bounds: { x: -32, y: -32, width: 416, height: 416 }, prompt: "", steps: 30, seed: 0 });
+ if (!("outpaintBlendOverlap" in edit)) throw new Error("Expected an outpaint edit");
+ expect(edit.outpaintBlendOverlap).toBeGreaterThanOrEqual(1);
+ expect(edit.outpaintBlendOverlap).toBeLessThanOrEqual(33);
+ expect(edit.outpaintBlendOverlap % 2).toBe(1);
+ expect(edit.width - 2 * (edit.outpaintBlendOverlap - 1)).toBeGreaterThanOrEqual(32);
+ expect(edit.height - 2 * (edit.outpaintBlendOverlap - 1)).toBeGreaterThanOrEqual(32);
+ }
+});
+
it("aligns resize dimensions while preserving the opposite corner and caps every sizing path", () => {
expect(alignExtendBounds({ x: -10, y: -7, width: 610, height: 455 }, "nw")).toEqual({ x: -8, y: 0, width: 608, height: 448 });
for (const mode of ["draw", "nw", "e", "s"]) {
diff --git a/src/lib/extendImage.ts b/src/lib/extendImage.ts
index 095bcc7..c643b39 100644
--- a/src/lib/extendImage.ts
+++ b/src/lib/extendImage.ts
@@ -5,7 +5,7 @@ import { outputDimensions } from "./export";
import type { ProjectAsset, ProjectConfig } from "./project";
export const EXTEND_GRID = 32;
-export const ZOOM_BLEND_OVERLAP = 9;
+export const ZOOM_BLEND_OVERLAP = 33;
export const ZOOM_LANGEVIN_STEPS = 5;
// Largest current generation preset (2048p landscape), also usable in portrait.
const MAX_GENERATION_SIZE = outputDimensions("2048p", "16:9");
@@ -100,6 +100,18 @@ export function extendContext(context: ExtendBounds) {
return context;
}
+function extendBlendOverlap(context: ExtendBounds, output: { width: number; height: number }) {
+ const room = (start: number, size: number, extent: number) => {
+ const edges = Number(start > 0) + Number(start + size < extent);
+ return edges ? (size - EXTEND_GRID) / edges : Infinity;
+ };
+ // Blend across two latent cells at exposed edges, but retain an unblended
+ // interior when only a small intersection of the source is on the canvas.
+ const band = Math.max(0, Math.min(ZOOM_BLEND_OVERLAP - 1,
+ room(context.x, context.width, output.width), room(context.y, context.height, output.height)));
+ return Math.floor(band / 2) * 2 + 1;
+}
+
export function compileExtendImage(config: ProjectConfig, source: ProjectAsset, sourceRelativePath: string, options: ExtendOptions) {
if (!source.width || !source.height) throw new Error("Wait for the source image dimensions to load.");
const layout = extendLayout(config, { width: source.width, height: source.height }, options.bounds);
@@ -117,6 +129,6 @@ export function compileExtendImage(config: ProjectConfig, source: ProjectAsset,
// Outpainting locks the interior and generates all borders together. Crops
// refine the whole frame with partial denoising to retain the composition.
const edits = [{ ...context, prompt: compiled.prompt, feather: 0, invertMask: !regenerate,
- ...(regenerate ? { strength: 0.65 } : { outpaintBlendOverlap: ZOOM_BLEND_OVERLAP, outpaintLangevinSteps: ZOOM_LANGEVIN_STEPS }) }];
+ ...(regenerate ? { strength: 0.65 } : { outpaintBlendOverlap: extendBlendOverlap(context, output), outpaintLangevinSteps: ZOOM_LANGEVIN_STEPS }) }];
return { ...compiled, scene, edits, layout };
}
diff --git a/src/lib/extendWork.test.ts b/src/lib/extendWork.test.ts
index 0d2ad34..2be6b53 100644
--- a/src/lib/extendWork.test.ts
+++ b/src/lib/extendWork.test.ts
@@ -65,7 +65,7 @@ it.each([false, true])("extends a frozen source and saves a child with a durable
const { jobId } = request;
expect(request).toMatchObject({ stillImage: true, frames: 1, seed: 0, steps: 25, canvasWidth: 960, canvasHeight: 576,
imageEdit: { sourceRelativePath: `cache/extend-images/${jobId}/canvas.png` }, referencePaths: [] });
- expect(request.imageEdit!.edits).toEqual([{ x: 250, y: 115, width: 461, height: 346, invertMask: true, feather: 0, outpaintBlendOverlap: 9, outpaintLangevinSteps: 5, prompt: request.prompt }]);
+ expect(request.imageEdit!.edits).toEqual([{ x: 250, y: 115, width: 461, height: 346, invertMask: true, feather: 0, outpaintBlendOverlap: 33, outpaintLangevinSteps: 5, prompt: request.prompt }]);
expect(request.prompt).toContain("Source scene's setting");
expect(request.prompt).not.toContain("<Picture");
expect(request.prompt).not.toContain("composition anchor");