Skip to content

Free open-source image editor for the browser

Reserve a Remove field for each stroke version when it is edited, so undo and drafts keep the fill shown (#82)

Rodrigo Belfiore9f77669ac516
diff --git a/API.md b/API.md
index 238fdf2..37d6a69 100644
--- a/API.md
+++ b/API.md
@@ -66,7 +66,7 @@ Create an empty Heal effect with `addLayer("heal")`.
`addHealStroke(id, patchId, stroke)` appends a painted or erased `BrushStroke` to an existing patch as one undoable edit. It preserves the mode, donor, feather, and opacity. Feather is a patch-wide blend based on the first stroke's diameter, shared by later additions. All patches expose an ordered `strokes` array; each path stays separate, coverage is combined on the GPU, and moving the patch moves paint and erase together. In the editor, Shift adds and Alt/Option subtracts on the selected patch, committing on release. Alt-click chooses a donor for Heal/Clone when no patch is selected. Older saved scenes with a single `stroke` load as one-element arrays.
-`addRemovePatch(id, stroke)` appends a Remove patch and returns its ID. It synthesizes the covered region from unmasked surrounding context using multiscale GPU PatchMatch; no offset or manual source is needed. `setHealSource` rejects Remove patches. The same feather, opacity, destination, duplicate, and delete edits apply. In the editor, the first full-resolution render synthesizes where each pixel copies from, and `getState()` then shows it as the patch's `field`, naming its stored texels: changes to the layers below, undo, export, and scene files keep it, while the copied colors follow those layers. `exportImage` and `exportScene` first read back any field the editor holds but the scene lacks, and reject when that readback fails. Adding a stroke with `addHealStroke` keeps what the field already filled and synthesizes only the rest; moving the patch synthesizes it anew. The solver and compositor run through `createEditorRenderer` without React; [the solver documentation](src/features/heal/inpaint/algorithm.md) describes limits and reproducible benchmarks.
+`addRemovePatch(id, stroke)` appends a Remove patch and returns its ID. It synthesizes the covered region from unmasked surrounding context using multiscale GPU PatchMatch; no offset or manual source is needed. `setHealSource` rejects Remove patches. The same feather, opacity, destination, duplicate, and delete edits apply. The patch's `field` names, by ID, where each pixel copies from; a new ID is reserved whenever its strokes change. In the editor, the first full-resolution render synthesizes that field and keeps it under its ID: changes to the layers below, undo, export, and scene files keep it, while the copied colors follow those layers. `exportImage` and `exportScene` first read back any field the editor shows that the document still waits for, and reject when that readback fails. Adding a stroke with `addHealStroke` keeps what the field already filled and synthesizes only the rest; moving the patch synthesizes it anew. The solver and compositor run through `createEditorRenderer` without React; [the solver documentation](src/features/heal/inpaint/algorithm.md) describes limits and reproducible benchmarks.
Create an empty Paint layer with `addLayer("paint")`. `addPaintStroke(id, stroke)` appends a `BrushStroke` with a `color`, `#rrggbb` sRGB: a painting stroke lays that color over the layer's earlier paint, and an erasing one removes paint whatever its color. `setPaintBlend(id, blend)` blends the layer's paint over the image below with the modes `setFill` takes, on encoded values as Photoshop does. A paint stroke builds up as a mask's does. In the editor, a layer's strokes settle into pixels once it has 100 and no gesture is open: `getState()` then shows a `raster` ID and fewer strokes, undo still reaches every stroke, and scene files keep the pixels. A photo holds up to 4 Paint layers and 10 brush masks; adding more throws.
@@ -169,18 +169,19 @@ The image renders at the document dimensions and downsamples to `longEdge` with
## Scene files
-`exportScene()` returns `Promise<File>`: the document as an `.openlight` file named after its source image, which **Save scene** in the Export panel downloads. It is a ZIP archive holding the source file's original bytes at `sources/<id>` and a deflated `scene.json`:
+`exportScene()` returns `Promise<File>`: the document as an `.openlight` file named after its source image, which **Save scene** in the Export panel downloads. It is a ZIP archive holding the source file's original bytes at `sources/<id>`, the deflated texels of each Remove field at `fields/<id>`, and a deflated `scene.json`:
```json
{
"format": "openlight",
- "version": 1,
+ "version": 2,
"sources": { "<id>": { "name": "photo.jpg", "type": "image/jpeg" } },
+ "fields": { "<id>": { "origin": [320, 180], "scale": 1, "size": [96, 64] } },
"scene": { "frame": {}, "layers": [] }
}
```
-`scene` is the `getState()` scene; each image layer's `source` names an entry in `sources`. Opening the file with `loadScene`, `openFile`, a drop, or the file picker decodes the stored source again and restores the frame and every layer as a new document with empty history. Preview settings and history are not saved.
+`scene` is the `getState()` scene; each image layer's `source` names an entry in `sources`, and each Remove patch's `field` an entry in `fields`: where its texels sit, or, for a field not yet synthesized, `{ "base": "<id>" }` naming the field it extends. Version 1 files, which kept each Remove field inside its patch, open as version 2. Opening the file with `loadScene`, `openFile`, a drop, or the file picker decodes the stored source again and restores the frame and every layer as a new document with empty history. Preview settings and history are not saved.
Opening validates every value as the matching command does, and a file that fails leaves the workspace in its error state with a message naming the first invalid field. Fields OpenLight does not know are dropped. A parameter missing from `adjustments`, `details`, `vignette`, `grain`, `fill`, or `colorMixer` takes its default, so older files still open when a group gains a parameter; a RAW image without a white balance uses its As Shot value. `version` increases only when older files can no longer open as written; a newer version is rejected.
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 5fb0536..b1fc424 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -86,7 +86,7 @@ Dependencies point downward. Features do not import each other; `app/` connects
- Strokes are scene content; the renderer caches their rasterized coverage, or a Paint layer's premultiplied 8-bit color, at half the source's resolution each way, a quarter of its pixels, which the passes that read them sample back to full size, and stamps only appended dabs. A Paint layer's or brush mask's raster comes with its first paint and stays, cleared if need be, until the layer goes, so painting and undoing allocate nothing. A photo holds up to 4 Paint layers and 10 brush masks.
- A mask's or Paint layer's latest stroke builds up in a half-float stroke buffer at the same resolution, one stroke at a time; its raster takes it in one pass when the next stroke starts, so it rounds to 8 bits once, however many moves drew it. Until then a Paint layer composes the buffer over its raster, and a mask's readers get its raster with the stroke laid over, in a view redrawn where the stroke grows. A Healing or Clone patch records its mode, reuses the same coverage, and keeps editable feather and opacity. Clone copies donor color directly, skipping the correction passes. A Healing patch records ordered hard paint/erase strokes, stamped through the shared GPU brush engine into one raster over only the 256 px tiles its paint reaches. Appending a gesture stamps only new dabs; undo or moved geometry replays the sequence. Shift adds and Alt/Option subtracts on the selected patch, preserving its donor and blend; separate strokes stay separate. The local SVG preview composes the same ordered operations and shows union boundaries and erased holes. Scene loading migrates earlier single-stroke patches once at the boundary.
- Remove patches store hard paint/erase strokes without a donor offset. `features/heal/inpaint` synthesizes a field of each hole texel's offset to its donor with a local, multiscale PatchMatch graph, entirely on the GPU: exclusion mask, preserved texture descriptors, nearest valid donor initialization, parallel propagation/random search, and overlap voting. Matching is bounded to 512 texels along the long side; the output samples donors at full source resolution. The Heal compositor applies the patch's coverage, feather, and opacity.
-- A Remove field is synthesized once per set of strokes, from the image below the patch at full resolution, and kept: the renderer holds it, and once no gesture is open `features/heal` reads it back and the patch names it as a document resource, as settled paint is, without a history step; a stroke added during the readback takes it as its base. The layers below, a proxy, hiding, undo, export, and a reopened scene all use a saved field and never solve again. Before export and scene files take their snapshot, the document has `features/heal` save any field the scene lacks, so one whose readback failed is read again and a second failure fails the snapshot; a draft saves again once a field changes the scene. Composition samples donor color from the current image below, so the patch follows changes under it, such as exposure. A stroke added to or erased from the patch keeps the texels the earlier field filled, on the same lattice while it fits, as long as they stay in the hole and their donors stay clean, and synthesizes only the rest; moving the patch synthesizes it anew. Synthesis needs no React and no readback. See [the solver](src/features/heal/inpaint/algorithm.md) for its constraints and standalone benchmarks.
+- A Remove field is synthesized once per stroke version and kept. Each edit that changes a Remove patch's strokes reserves a new field in the document's resources, naming the field it extends, and the patch names it, so every scene with those strokes, in history, drafts, scene files, and exports, shows one fill however late it arrives; filling it never changes the scene or history. The renderer keeps fields by ID: it synthesizes a waiting one once, at full resolution, from the image below, extending the nearest field it names that it holds or the document saved, and reads it back for the document at once, rendering nothing meanwhile. Patches that share a field, such as duplicates, share its synthesis. The layers below, a proxy, hiding, undo, export, and a reopened scene all use that field and never solve again; a version never shown synthesizes when it first shows. Export, scene files, and drafts complete their snapshot with the fields the editor shows that the document still waits for, and fail when that readback fails. Composition samples donor color from the current image below, so the patch follows changes under it, such as exposure. A stroke added to or erased from the patch keeps the texels the earlier field filled, on the same lattice while it fits, as long as they stay in the hole and their donors stay clean, and synthesizes only the rest; moving the patch synthesizes it anew. Synthesis needs no React. See [the solver](src/features/heal/inpaint/algorithm.md) for its constraints and standalone benchmarks.
- A brush mask paints coverage as a Paint layer paints color: both are a painting, the latest strokes over settled pixels, in one raster module (r8 for coverage, rgba8 for color), which also owns the open stroke coverage view. At 100 strokes, once no gesture is open, the renderer reads a painting's raster band by band through one mapped buffer, and the pixels, deflated, become a document resource the layer names in place of its strokes, without a history step. Rendering waits until both the scene and cached raster accept the settled pixels. History scenes name the pixels they drew over, so undo reaches past a settle: the renderer loads those pixels, inflating them band by band through a staging texture, and draws the strokes again. Resources go once no retained scene names them.
- A mask that only gradients shape needs no texture: the mix pass computes it, children included. A mask with a painted brush or a range combines in the render graph: each child's coverage, a brush's cached raster, a gradient drawn by a node, or a range read from the image below the mask, folds into the group's in stored order through one combine pass, and the mix pass shares its add, subtract, and intersect. The group takes the image's resolution when a range takes part, since a range follows the photo's edges, and the brushes' otherwise. The renderer gives it out, with each range's own coverage, for the overlay and thumbnails. While a color is picked, renders also keep the image below the mask group, which the picker reads through one GPU pass and 16 bytes of readback per pick.
- Open history groups render a reduced proxy. Every render image carries `scale`, its source pixels per texel; shaders that take document coordinates or radii apply it.
diff --git a/src/app/draft/store.ts b/src/app/draft/store.ts
index 0b01239..7857f63 100644
--- a/src/app/draft/store.ts
+++ b/src/app/draft/store.ts
@@ -1,6 +1,6 @@
import { type DBSchema, type IDBPDatabase, openDB } from "idb";
import { z } from "zod/mini";
-import { openScene, snapshotScene } from "@/app/scene-file";
+import { completeScene, openScene, snapshotScene } from "@/app/scene-file";
import type { EditorDocument } from "@/core/document";
import type { ImageSource } from "@/core/image";
import { parse } from "@/lib/parse";
@@ -20,9 +20,18 @@ const recordSchema = z.object(
export type DraftRecord = z.output<typeof recordSchema>;
export type Draft = { record: DraftRecord; files: ReadonlyMap<string, Blob> };
-/** A draft of the document shown as `name`, captured synchronously so the document may change or close meanwhile. */
-export function snapshotDraft(document: EditorDocument, name: string) {
- const { json, sources, paint, fields } = snapshotScene(document);
+/**
+ * A draft of the document shown as `name`, taken synchronously so the document may change or close
+ * meanwhile, once the Remove fields it shows are read back.
+ */
+export async function snapshotDraft(
+ document: EditorDocument,
+ name: string,
+): Promise<Draft> {
+ const { json, sources, paint, fields } = await completeScene(
+ document,
+ snapshotScene(document),
+ );
return {
record: { version, name, scene: json },
files: new Map<string, Blob>([...sources, ...paint, ...fields]),
@@ -86,9 +95,16 @@ export function createDraftStore(name = "openlight") {
}
return {
- /** Keeps files already stored under their ID and deletes the ones the draft no longer uses. */
- save({ record, files }: Draft) {
+ /**
+ * Keeps files already stored under their ID and deletes the ones the draft no longer uses. A draft
+ * still completing saves in call order, so an older one never replaces a newer one.
+ */
+ save(draft: Draft | Promise<Draft>) {
+ const ready = Promise.resolve(draft);
+ // A draft that fails before its turn rejects this save, not the page.
+ ready.catch(() => {});
return run(async (database) => {
+ const { record, files } = await ready;
const transaction = database.transaction(
["draft", "sources"],
"readwrite",
diff --git a/src/app/editor/export/export-image.ts b/src/app/editor/export/export-image.ts
index 5a36eba..ddf5d62 100644
--- a/src/app/editor/export/export-image.ts
+++ b/src/app/editor/export/export-image.ts
@@ -1,8 +1,9 @@
import type { Gpu, Target } from "vgpu";
import { createEditorRenderer } from "@/app/editor/renderer";
import {
+ completeFields,
type EditorDocument,
- fieldTexels,
+ fieldRecords,
settledPixels,
} from "@/core/document";
import type { Point } from "@/core/image/frame";
@@ -78,14 +79,14 @@ export async function encodeImage(
return blob;
}
-/** The scene with its source and the stored pixels and texels its renderer loads, taken together. */
+/** The scene with its source and the stored pixels and Remove fields its renderer loads, taken together. */
function snapshotRender(document: EditorDocument) {
const scene = document.scene.getState();
return {
scene,
source: document.resources.get(scene.layers[0].source),
pixels: settledPixels(document, scene),
- fields: fieldTexels(document, scene),
+ fields: fieldRecords(document, scene),
};
}
@@ -102,13 +103,7 @@ async function renderSnapshot(
}
return blob;
},
- fieldTexels: (id) => {
- const blob = fields.get(id);
- if (!blob) {
- throw Error("Remove field is unavailable.");
- }
- return blob;
- },
+ field: (id) => fields.get(id),
});
try {
await renderer.update(scene);
@@ -128,14 +123,12 @@ export async function exportImage(
document: EditorDocument,
options: ExportOptions = {},
) {
- // Taken now, since another photo may close the document meanwhile, and again once the Remove fields
- // the scene lacks are read back, unless it did.
- const requested = snapshotRender(document);
- const release = requested.source.retain();
+ // Taken now, since another photo may close the document meanwhile; its Remove fields complete after.
+ const snapshot = snapshotRender(document);
+ const release = snapshot.source.retain();
try {
- await document.prepareSnapshot();
- const snapshot = document.closed ? requested : snapshotRender(document);
- return await renderSnapshot(gpu, snapshot, options);
+ const fields = await completeFields(document, snapshot.fields);
+ return await renderSnapshot(gpu, { ...snapshot, fields }, options);
} finally {
release();
}
diff --git a/src/app/editor/index.tsx b/src/app/editor/index.tsx
index 5c87cb4..d78ad72 100644
--- a/src/app/editor/index.tsx
+++ b/src/app/editor/index.tsx
@@ -21,7 +21,6 @@ import { EditorViewport, ViewportStage } from "@/components/editor/viewport";
import type { Mask } from "@/core/document";
import { locateLayer } from "@/core/document";
import { HealingProvider, useHealBrush } from "@/features/heal/mode";
-import { RemoveFieldSaving } from "@/features/heal/save-fields";
import { addLayer } from "@/features/layers/edits";
import { MaskToolProvider, type Nesting } from "@/features/layers/mask-tool";
import { CanvasToolbar } from "@/features/layers/toolbar";
@@ -221,7 +220,6 @@ function DocumentEditor({
onDismiss={onDismissFailure}
/>
<PaintSettling />
- <RemoveFieldSaving />
<BrushKeys />
<ToolShortcuts />
</HealingTools>
diff --git a/src/app/editor/renderer.ts b/src/app/editor/renderer.ts
index bb8e831..8e427d3 100644
--- a/src/app/editor/renderer.ts
+++ b/src/app/editor/renderer.ts
@@ -119,7 +119,7 @@ function composeLayer(
);
break;
case "heal": {
- const result = heal(below, layer.id, layer.patches, name, composition);
+ const result = heal(below, layer.patches, name, composition);
edited = result.image;
input = result.input;
break;
diff --git a/src/app/loaders/scene.ts b/src/app/loaders/scene.ts
index f4f739c..40c5c4e 100644
--- a/src/app/loaders/scene.ts
+++ b/src/app/loaders/scene.ts
@@ -1,5 +1,5 @@
import type { Gpu } from "vgpu";
-import { openScene, snapshotScene } from "@/app/scene-file";
+import { completeScene, openScene, snapshotScene } from "@/app/scene-file";
import type { Workspace } from "@/app/workspace";
import type { EditorDocument } from "@/core/document";
import type { ImageSource } from "@/core/image";
@@ -20,13 +20,11 @@ function isSceneFile(file: File) {
* already deflated pixels paint settled into at `paint/<id>`, and Remove fields at `fields/<id>`.
*/
export async function writeSceneFile(document: EditorDocument) {
- // Taken now, since another photo may close the document meanwhile, and again once the Remove fields
- // the scene lacks are read back, unless it did.
- const requested = snapshotScene(document);
- await document.prepareSnapshot();
- const { json, sources, paint, fields } = document.closed
- ? requested
- : snapshotScene(document);
+ // Taken now, since another photo may close the document meanwhile; its Remove fields complete after.
+ const { json, sources, paint, fields } = await completeScene(
+ document,
+ snapshotScene(document),
+ );
const { writeZip } = await import("@/lib/zip");
const archive = await writeZip([
{
diff --git a/src/app/scene-file.ts b/src/app/scene-file.ts
index 4b762f8..20ca55a 100644
--- a/src/app/scene-file.ts
+++ b/src/app/scene-file.ts
@@ -1,9 +1,11 @@
import { z } from "zod/mini";
import {
+ completeFields,
createDocument,
createResources,
type EditorDocument,
- fieldTexels,
+ type FieldRecord,
+ fieldRecords,
type ProcessingLayer,
paintingOf,
removePatches,
@@ -22,7 +24,7 @@ import { defaultMixer, mixerSchema } from "@/features/color-mixer/model";
import { defaultDetails, detailsSchema } from "@/features/details/model";
import { defaultFill, fillSchema } from "@/features/fill/model";
import { defaultGrain, grainSchema } from "@/features/grain/model";
-import { healPatchSchema } from "@/features/heal/model";
+import { fieldLatticeSchema, healPatchSchema } from "@/features/heal/model";
import { validateBrushLayers } from "@/features/layers/edits";
import {
layerSettings,
@@ -37,7 +39,7 @@ import { whiteBalanceSchema } from "@/features/white-balance/edits";
import { parse, withDefaults } from "@/lib/parse";
/** Raised only when older files can no longer load as written; a parameter added later takes its default. */
-const version = 1;
+const version = 2;
const savedHealPatch = z.pipe(
z.transform((input) => {
@@ -55,33 +57,78 @@ const savedHealPatch = z.pipe(
healPatchSchema,
);
-/** A saved scene: the scene as edited and the name and type of each source file, stored beside it by ID. */
+/** Where a saved Remove field's texels sit, or, while it waits to be synthesized, the field it extends. */
+type SavedField =
+ | {
+ readonly origin: readonly [number, number];
+ readonly scale: number;
+ readonly size: readonly [number, number];
+ }
+ | { readonly base?: string };
+
+/**
+ * A saved scene: the scene as edited, the name and type of each source file, and each Remove field it
+ * names, whose texels and files are stored beside it by ID.
+ */
export type SceneJson = {
format: "openlight";
version: number;
sources: Record<string, { name: string; type: string }>;
+ fields: Record<string, SavedField>;
scene: Scene;
};
/**
- * The document's scene, the source file it references, the pixels its paint settled into, and its
- * Remove fields, read without rendering.
+ * The document's scene and what it names, taken now without rendering: the source file, the pixels its
+ * paint settled into, and its Remove fields, some of which may still be waiting to be synthesized.
*/
export function snapshotScene(document: EditorDocument) {
const scene = document.scene.getState();
const { source } = scene.layers[0];
- const { file } = document.resources.get(source);
+ return {
+ scene,
+ source: { id: source, file: document.resources.get(source).file },
+ paint: settledPixels(document, scene),
+ fields: fieldRecords(document, scene),
+ };
+}
+
+export type SceneSnapshot = ReturnType<typeof snapshotScene>;
+
+/**
+ * A snapshot as a saved scene's JSON and the files it names, once the editor reads back the Remove
+ * fields it shows that the snapshot still waits for; rejects when a readback fails.
+ */
+export async function completeScene(
+ document: EditorDocument,
+ { scene, source, paint, fields: records }: SceneSnapshot,
+) {
+ const completed = await completeFields(document, records);
+ const fields: Record<string, SavedField> = {};
+ const texels = new Map<string, Blob>();
+ for (const [id, record] of completed) {
+ if (record.field) {
+ const { texels: data, ...lattice } = record.field;
+ fields[id] = lattice;
+ texels.set(id, data);
+ } else {
+ fields[id] = record.base ? { base: record.base } : {};
+ }
+ }
const json: SceneJson = {
format: "openlight",
version,
- sources: { [source]: { name: file.name, type: file.type } },
+ sources: {
+ [source.id]: { name: source.file.name, type: source.file.type },
+ },
+ fields,
scene,
};
return {
json,
- sources: new Map([[source, file]]),
- paint: settledPixels(document, scene),
- fields: fieldTexels(document, scene),
+ sources: new Map([[source.id, source.file]]),
+ paint,
+ fields: texels,
};
}
@@ -198,14 +245,89 @@ const header = z.object(
},
notScene,
);
+const savedField = z.union([
+ fieldLatticeSchema,
+ z.strictObject({ base: z.optional(id) }),
+]) satisfies z.ZodMiniType<SavedField>;
const savedSchema = z.extend(header, {
sources: z.record(
z.string(),
z.object({ name: z.string(), type: z.string() }),
),
+ fields: z.record(id, savedField),
scene: sceneSchema,
});
+function isRecord(value: unknown): value is Record<string, unknown> {
+ return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+
+/** Every patch object among saved layers and their children, as written. */
+function* savedPatches(layers: unknown): Generator<Record<string, unknown>> {
+ if (!Array.isArray(layers)) return;
+ for (const layer of layers) {
+ if (!isRecord(layer)) continue;
+ if (Array.isArray(layer.patches)) {
+ yield* layer.patches.filter(isRecord);
+ }
+ yield* savedPatches(layer.children);
+ }
+}
+
+/**
+ * A version 1 scene as version 2 writes it: each Remove patch named its field inline, with how many
+ * strokes it covered, and now names it by ID beside the scene. A field for fewer strokes becomes the
+ * base of a new one, and a patch without one reserves one.
+ */
+function migrateFields(saved: unknown) {
+ if (!isRecord(saved) || saved.version !== 1 || !isRecord(saved.scene)) {
+ return saved;
+ }
+ const migrated = structuredClone(saved);
+ const fields: Record<string, unknown> = {};
+ for (const patch of savedPatches(
+ Reflect.get(Object(migrated.scene), "layers"),
+ )) {
+ if (patch.mode !== "remove") continue;
+ const strokes = Array.isArray(patch.strokes) ? patch.strokes.length : 0;
+ const {
+ texels,
+ strokes: covered,
+ ...lattice
+ } = isRecord(patch.field) ? patch.field : {};
+ if (typeof texels !== "string") {
+ const reserved = crypto.randomUUID();
+ patch.field = reserved;
+ fields[reserved] = {};
+ continue;
+ }
+ fields[texels] = lattice;
+ if (typeof covered === "number" && covered < strokes) {
+ const extending = crypto.randomUUID();
+ patch.field = extending;
+ fields[extending] = { base: texels };
+ } else {
+ patch.field = texels;
+ }
+ }
+ return { ...migrated, fields };
+}
+
+/** Whether every field a patch names, and every field one extends, is saved, with no loop of bases. */
+function fieldsComplete(scene: Scene, fields: SceneJson["fields"]) {
+ for (const { patch } of removePatches(scene.layers)) {
+ const seen = new Set<string>();
+ let id: string | undefined = patch.field;
+ while (id) {
+ const field: SavedField | undefined = fields[id];
+ if (!field || seen.has(id)) return false;
+ seen.add(id);
+ id = "base" in field ? field.base : undefined;
+ }
+ }
+ return true;
+}
+
/**
* Opens a saved scene as a new document, validating every value as the edit that made it.
* Scene files and drafts both open through here; `files` holds each source's bytes, the pixels each
@@ -219,8 +341,27 @@ export async function openScene(
if (parse(header, saved, "Invalid scene").version > version) {
throw Error("This scene needs a newer version of OpenLight.");
}
- const { sources, scene } = parse(savedSchema, saved, "Invalid scene");
+ const { sources, fields, scene } = parse(
+ savedSchema,
+ migrateFields(saved),
+ "Invalid scene",
+ );
validateBrushLayers(scene.layers);
+ if (!fieldsComplete(scene, fields)) {
+ throw Error("The scene's Remove fields are missing.");
+ }
+ const records = new Map<string, FieldRecord>();
+ for (const [id, field] of Object.entries(fields)) {
+ if (!("origin" in field)) {
+ records.set(id, field);
+ continue;
+ }
+ const texels = files.get(id);
+ if (!texels) {
+ throw Error("The scene's Remove fields are missing.");
+ }
+ records.set(id, { field: { texels, ...field } });
+ }
const [image, ...layers] = scene.layers;
const source = sources[image.source];
const data = files.get(image.source);
@@ -252,14 +393,8 @@ export async function openScene(
resources.addPaint(pixels, raster);
}
}
- for (const { patch } of removePatches(layers)) {
- if (patch.field) {
- const texels = files.get(patch.field.texels);
- if (!texels) {
- throw Error("The scene's Remove fields are missing.");
- }
- resources.addField(texels, patch.field.texels);
- }
+ for (const [id, record] of records) {
+ resources.addField(id, record);
}
return createDocument(
{
diff --git a/src/components/editor/pipeline.tsx b/src/components/editor/pipeline.tsx
index f3a4728..4e6b355 100644
--- a/src/components/editor/pipeline.tsx
+++ b/src/components/editor/pipeline.tsx
@@ -55,12 +55,19 @@ export function RendererProvider({
() =>
createRenderer(gpu, source, {
paintPixels: (id) => document.resources.paint(id),
- fieldTexels: (id) => document.resources.field(id),
+ field: (id) => document.resources.field(id),
+ saveField: (id, field) => document.resources.fillField(id, field),
}),
[gpu, document, source, createRenderer],
);
const [error, setError] = useState<string>();
+ // A snapshot reads back the Remove fields this renderer shows that the document still waits for.
+ useEffect(
+ () => document.onCaptureFields((ids) => renderer.captureFields(ids)),
+ [renderer, document],
+ );
+
useEffect(() => {
let active = true;
let requestedScene: ReturnType<typeof document.scene.getState> | undefined;
diff --git a/src/core/document/index.ts b/src/core/document/index.ts
index c63e431..b9bb4d6 100644
--- a/src/core/document/index.ts
+++ b/src/core/document/index.ts
@@ -1,10 +1,15 @@
import { createStore } from "zustand/vanilla";
import { validateFrame } from "@/core/image/frame";
import { createHistory } from "./history";
-import { createResources } from "./resources";
+import {
+ createResources,
+ type FieldRecord,
+ type RemoveField,
+} from "./resources";
import type { Mask, MaskModifier, Scene } from "./scene";
import { findLayer, paintingOf, removePatches, walkLayers } from "./tree";
+export type { FieldRecord, RemoveField } from "./resources";
export type {
Adjustments,
Blend,
@@ -34,7 +39,6 @@ export type {
ProcessingLayer,
RadialGradient,
RangeMask,
- RemoveField,
RemovePatch,
Scene,
StrokePoint,
@@ -98,11 +102,24 @@ function equal(a: unknown, b: unknown): boolean {
);
}
+type Resources = ReturnType<typeof createResources>;
+
+/** A Remove field and, while it waits to be synthesized, the fields it extends, up to a synthesized one. */
+function fieldChain(resources: Resources, id: string) {
+ const ids = [id];
+ let record = resources.field(id);
+ while (record && !record.field && record.base) {
+ ids.push(record.base);
+ record = resources.field(record.base);
+ }
+ return ids;
+}
+
/**
* The resources a scene names: its image source, the pixels its paintings settled into, and its
- * Remove fields.
+ * Remove fields with the fields they extend.
*/
-function resourceIds(scene: Scene) {
+function resourceIds(scene: Scene, resources: Resources) {
const ids = [scene.layers[0].source];
for (const { layer } of walkLayers(scene.layers)) {
const raster = paintingOf(layer)?.raster;
@@ -111,9 +128,7 @@ function resourceIds(scene: Scene) {
}
}
for (const { patch } of removePatches(scene.layers)) {
- if (patch.field) {
- ids.push(patch.field.texels);
- }
+ ids.push(...fieldChain(resources, patch.field));
}
return ids;
}
@@ -133,23 +148,45 @@ export function settledPixels(
return pixels;
}
-/** The Remove fields `scene` names, taken now, like `settledPixels`. */
-export function fieldTexels(
+/** The Remove fields `scene` names, with the fields those still waiting extend, taken now like `settledPixels`. */
+export function fieldRecords(
document: EditorDocument,
scene = document.scene.getState(),
) {
- const texels = new Map<string, Blob>();
+ const records = new Map<string, FieldRecord>();
for (const { patch } of removePatches(scene.layers)) {
- if (patch.field) {
- texels.set(
- patch.field.texels,
- document.resources.field(patch.field.texels),
- );
+ for (const id of fieldChain(document.resources, patch.field)) {
+ records.set(id, document.resources.field(id) ?? {});
}
}
- return texels;
+ return records;
+}
+
+/**
+ * `records` taken for a snapshot, with the fields the editor shows but they still waited for read back
+ * since; rejects when a readback fails.
+ */
+export async function completeFields(
+ document: EditorDocument,
+ records: ReadonlyMap<string, FieldRecord>,
+) {
+ const waiting = [...records]
+ .filter(([, record]) => !record.field)
+ .map(([id]) => id);
+ const captured = await document.captureFields(waiting);
+ return new Map(
+ [...records].map(([id, record]): [string, FieldRecord] => {
+ const field = record.field ?? captured.get(id);
+ return [id, field ? { field } : record];
+ }),
+ );
}
+/** Reads back the Remove fields among `ids` the editor holds but resources wait for. */
+type FieldCapture = (
+ ids: readonly string[],
+) => Promise<ReadonlyMap<string, RemoveField>>;
+
/** One independent editing session. No React, decoders, or file workflows. */
export function createDocument(initial: Scene, resources = createResources()) {
const scene = createStore(() => initial);
@@ -159,7 +196,9 @@ export function createDocument(initial: Scene, resources = createResources()) {
equal,
100,
(retained) => {
- resources.retain(new Set(retained.flatMap(resourceIds)));
+ resources.retain(
+ new Set(retained.flatMap((scene) => resourceIds(scene, resources))),
+ );
},
);
const unsubscribe = scene.subscribe((state) => {
@@ -168,7 +207,7 @@ export function createDocument(initial: Scene, resources = createResources()) {
selection.setState({ layerId: state.layers[0].id });
}
});
- const preparations = new Set<() => Promise<void>>();
+ let capture: FieldCapture | undefined;
let closed = false;
return {
id: crypto.randomUUID(),
@@ -213,19 +252,23 @@ export function createDocument(initial: Scene, resources = createResources()) {
}
replace(next);
},
- /**
- * Adds `prepare`, which a snapshot runs first to replace the scene in place with what the editor
- * shows but the scene lacks, such as Remove fields read back. Returns its removal.
- */
- onSnapshot(prepare: () => Promise<void>) {
- preparations.add(prepare);
+ /** Sets how the editor reads back the Remove fields it shows, for `captureFields`. Returns its removal. */
+ onCaptureFields(next: FieldCapture) {
+ capture = next;
return () => {
- preparations.delete(prepare);
+ if (capture === next) {
+ capture = undefined;
+ }
};
},
- /** Runs every preparation, so a snapshot holds what the editor shows; rejects when one fails. */
- async prepareSnapshot() {
- await Promise.all([...preparations].map((prepare) => prepare()));
+ /**
+ * The fields among `ids` that the editor shows but resources still wait for, read back once the
+ * syntheses in flight finish, so a snapshot holds what the editor shows; rejects when a readback fails.
+ */
+ async captureFields(
+ ids: readonly string[],
+ ): Promise<ReadonlyMap<string, RemoveField>> {
+ return (await capture?.(ids)) ?? new Map();
},
/** Whether the document was disposed, so work that outlived it can drop its result. */
get closed() {
diff --git a/src/core/document/resources.ts b/src/core/document/resources.ts
index 058f0c5..fd6efbe 100644
--- a/src/core/document/resources.ts
+++ b/src/core/document/resources.ts
@@ -1,13 +1,35 @@
import type { ImageSource } from "@/core/image";
+import type { Point } from "@/core/image/frame";
+
+/**
+ * Where each pixel of a Remove patch copies from, synthesized once and kept, so the layers below and
+ * the renderer can't change what the patch shows; the copied colors still follow the layers below.
+ */
+export type RemoveField = {
+ /** Each texel's offset to its donor, in texels, deflated; zero keeps the pixel. */
+ readonly texels: Blob;
+ /** Where its first texel's corner sits, in source pixels. */
+ readonly origin: Point;
+ /** Source pixels per texel. */
+ readonly scale: number;
+ /** Texels across and down. */
+ readonly size: Point;
+};
+
+/** A Remove field resource: its field once synthesized, or until then the field its synthesis extends. */
+export type FieldRecord = {
+ readonly field?: RemoveField;
+ readonly base?: string;
+};
/**
* Owns the document's image files and GPU targets, the deflated pixels paint layers settled into, and
- * the deflated texels of Remove fields, outside scene history.
+ * Remove fields, outside scene history.
*/
export function createResources() {
const images = new Map<string, { file: File } & ImageSource>();
const paints = new Map<string, Blob>();
- const fields = new Map<string, Blob>();
+ const fields = new Map<string, FieldRecord>();
let disposed = false;
function open() {
if (disposed) {
@@ -44,18 +66,26 @@ export function createResources() {
}
return pixels;
},
- /** A saved document restores Remove fields under their original IDs too. */
- addField(texels: Blob, id: string = crypto.randomUUID()) {
+ /** Reserves a Remove field for a new set of strokes, which extends `base` once synthesized. */
+ reserveField(base?: string) {
open();
- fields.set(id, texels);
+ const id = crypto.randomUUID();
+ fields.set(id, base ? { base } : {});
return id;
},
- field(id: string) {
- const texels = fields.get(id);
- if (!texels) {
- throw new Error("Remove field is unavailable.");
+ /** A saved document restores Remove fields under their original IDs. */
+ addField(id: string, record: FieldRecord) {
+ open();
+ fields.set(id, record);
+ },
+ /** Keeps the field synthesized for a reserved ID; the first one stays, and a closed document drops it. */
+ fillField(id: string, field: RemoveField) {
+ if (!disposed && fields.has(id) && !fields.get(id)?.field) {
+ fields.set(id, { field });
}
- return texels;
+ },
+ field(id: string) {
+ return fields.get(id);
},
/** Frees what no retained scene names. */
retain(ids: ReadonlySet<string>) {
diff --git a/src/core/document/scene.ts b/src/core/document/scene.ts
index 43a32b5..f3eec98 100644
--- a/src/core/document/scene.ts
+++ b/src/core/document/scene.ts
@@ -115,22 +115,6 @@ export type Painting<S extends BrushStroke = BrushStroke> = {
/** Coverage painted with strokes; the renderer rasterizes them into a cached texture. */
export type BrushMask = { readonly kind: "brush" } & Painting;
export type HealMode = "heal" | "clone" | "remove";
-/**
- * Where each pixel of a Remove patch copies from, synthesized once and kept, so the layers below and
- * the renderer can't change what the patch shows; the copied colors still follow the layers below.
- */
-export type RemoveField = {
- /** The document resource of each texel's offset to its donor, in texels; zero keeps the pixel. */
- readonly texels: string;
- /** How many of the patch's strokes, from the first, it was synthesized for; later strokes extend it. */
- readonly strokes: number;
- /** Where its first texel's corner sits, in source pixels. */
- readonly origin: Point;
- /** Source pixels per texel. */
- readonly scale: number;
- /** Texels across and down. */
- readonly size: Point;
-};
/** A painted repair: Heal/Clone use a donor offset; Remove synthesizes surrounding texture. */
export type HealPatch = {
readonly id: string;
@@ -141,7 +125,14 @@ export type HealPatch = {
readonly opacity: number;
} & (
| { readonly mode: "heal" | "clone"; readonly offset: Point }
- | { readonly mode: "remove"; readonly field?: RemoveField }
+ | {
+ readonly mode: "remove";
+ /**
+ * The document resource of where each pixel copies from, reserved for exactly these strokes
+ * when they change and filled once synthesized, so every scene with them shows one fill.
+ */
+ readonly field: string;
+ }
);
export type RemovePatch = Extract<HealPatch, { mode: "remove" }>;
/**
diff --git a/src/core/renderer/fields.ts b/src/core/renderer/fields.ts
index 493c133..529af17 100644
--- a/src/core/renderer/fields.ts
+++ b/src/core/renderer/fields.ts
@@ -1,5 +1,5 @@
import type { Gpu, Target } from "vgpu";
-import type { BrushStroke, RemoveField } from "@/core/document";
+import type { RemoveField } from "@/core/document";
import { input, type RenderInput } from "./node";
import { createRasterCache } from "./raster-cache";
import { copyRaster, readRaster, writeRaster } from "./transfer";
@@ -7,128 +7,54 @@ import { copyRaster, readRaster, writeRaster } from "./transfer";
/** Where a field's texels sit in the photo. */
export type FieldLattice = Pick<RemoveField, "origin" | "scale" | "size">;
-/** A Remove field a renderer holds and the strokes it was synthesized for. */
-export type HeldField = {
- texels: RenderInput;
- lattice: FieldLattice;
- strokes: readonly BrushStroke[];
-};
+/** A Remove field a renderer holds. */
+export type HeldField = { texels: RenderInput; lattice: FieldLattice };
-type Entry = {
- target: Target;
- lattice: FieldLattice;
- strokes: readonly BrushStroke[];
-};
+/** Its lattice once its texels are written, so a load that fails midway holds nothing. */
+type Entry = { target: Target; lattice?: FieldLattice };
-/** Whether `strokes` start with every stroke of `first`, the same objects. */
-function startsWith(
- strokes: readonly BrushStroke[],
- first: readonly BrushStroke[],
-) {
- return (
- first.length <= strokes.length &&
- first.every((stroke, i) => stroke === strokes[i])
- );
-}
-
-/**
- * Holds one Remove field per patch, synthesized by a render or loaded from the document, with the
- * strokes it was synthesized for. A field for the patch's first strokes is the base that later
- * strokes extend.
- */
+/** Holds Remove fields by ID, synthesized by a render or loaded from the document. */
export function createFieldStore(gpu: Gpu) {
const fields = createRasterCache<Entry>(gpu, "rg16float", (target) => ({
target,
- lattice: { origin: [0, 0], scale: 1, size: target.size },
- strokes: [],
}));
- function usable(
- key: string,
- strokes: readonly BrushStroke[],
- saved?: RemoveField,
- ) {
- const entry = fields.get(key);
- if (!entry?.strokes.length || !startsWith(strokes, entry.strokes)) {
- return undefined;
- }
- // A saved field for every stroke is what the patch shows, not an extension of fewer.
- if (
- saved?.strokes === strokes.length &&
- entry.strokes.length < strokes.length
- ) {
- return undefined;
- }
- return entry;
- }
return {
- /** Whether a patch's saved field must load first, since nothing held serves its strokes. */
- needs(
- key: string,
- strokes: readonly BrushStroke[],
- saved?: RemoveField,
- ): saved is RemoveField {
- return saved !== undefined && !usable(key, strokes, saved);
+ get(id: string): HeldField | undefined {
+ const entry = fields.get(id);
+ return (
+ entry?.lattice && {
+ texels: input(entry.target),
+ lattice: entry.lattice,
+ }
+ );
},
- async load(
- key: string,
- strokes: readonly BrushStroke[],
- saved: RemoveField,
- texels: Blob,
- ) {
- const entry = fields.reserve(key, saved.size);
- entry.strokes = [];
+ async load(id: string, { texels, origin, scale, size }: RemoveField) {
+ const entry = fields.reserve(id, size);
+ entry.lattice = undefined;
await writeRaster(gpu, entry.target, texels);
- const { origin, scale, size } = saved;
entry.lattice = { origin, scale, size };
- entry.strokes = strokes.slice(0, saved.strokes);
},
- /** The field held for a patch's strokes, or for strokes they extend. */
- get(
- key: string,
- strokes: readonly BrushStroke[],
- saved?: RemoveField,
- ): HeldField | undefined {
- const entry = usable(key, strokes, saved);
- if (!entry) {
+ /** Holds a field a render synthesized. */
+ keep(id: string, lattice: FieldLattice, texels: Target) {
+ const entry = fields.reserve(id, lattice.size);
+ copyRaster(gpu, texels, entry.target);
+ entry.lattice = lattice;
+ },
+ /** Reads a held field back, deflated, to save it. */
+ async read(id: string): Promise<RemoveField | undefined> {
+ const entry = fields.get(id);
+ if (!entry?.lattice) {
return undefined;
}
- return {
- texels: input(entry.target),
- lattice: entry.lattice,
- strokes: entry.strokes,
- };
+ const { lattice } = entry;
+ return { texels: await readRaster(gpu, entry.target), ...lattice };
},
- /** Keeps a patch's field past the next sweep. */
- retain(key: string) {
- const entry = fields.get(key);
+ /** Keeps a field past the next sweep. */
+ retain(id: string) {
+ const entry = fields.get(id);
if (entry) {
- fields.reserve(key, entry.target.size);
- }
- },
- /** Holds a field a render synthesized for a patch's strokes, in place of what it held. */
- keep(
- key: string,
- strokes: readonly BrushStroke[],
- lattice: FieldLattice,
- texels: Target,
- ) {
- const entry = fields.reserve(key, lattice.size);
- copyRaster(gpu, texels, entry.target);
- Object.assign(entry, { lattice, strokes });
- },
- /** Reads the field held for exactly a patch's strokes, deflated, to save it. */
- async read(key: string, strokes: readonly BrushStroke[]) {
- const entry = fields.get(key);
- if (
- entry?.strokes.length !== strokes.length ||
- !startsWith(strokes, entry.strokes)
- ) {
- return undefined;
+ fields.reserve(id, entry.target.size);
}
- return {
- texels: await readRaster(gpu, entry.target),
- lattice: entry.lattice,
- };
},
sweep: fields.sweep,
inspect: () => fields.inspect("/field"),
diff --git a/src/core/renderer/index.ts b/src/core/renderer/index.ts
index 0f9d286..2815ef7 100644
--- a/src/core/renderer/index.ts
+++ b/src/core/renderer/index.ts
@@ -1,10 +1,12 @@
import type { Gpu, Target, Timer } from "vgpu";
import {
type BrushStroke,
+ type FieldRecord,
hasPaint,
type MaskLayer,
type PaintLayer,
paintingOf,
+ type RemoveField,
type RemovePatch,
removePatches,
type Scene,
@@ -12,7 +14,7 @@ import {
} from "@/core/document";
import type { ImageSource, WhiteBalance } from "@/core/image";
import type { Point } from "@/core/image/frame";
-import { createFieldStore, type FieldLattice, type HeldField } from "./fields";
+import { createFieldStore, type FieldLattice } from "./fields";
import { createRenderGraph } from "./graph";
import { createMaskCoverage } from "./mask";
import { createPatchRaster, type PatchInput } from "./mask/patches";
@@ -35,7 +37,7 @@ export {
renderCoverage,
type View,
} from "./display";
-export type { FieldLattice, HeldField } from "./fields";
+export type { FieldLattice } from "./fields";
export type { PatchInput } from "./mask/patches";
export {
input,
@@ -70,17 +72,23 @@ export type Composition = {
paint: (layer: PaintLayer) => PaintInput | undefined;
/** Rasterized coverage of an effect's own stroke, such as a Healing patch. */
patch: (id: string, strokes: readonly BrushStroke[]) => PatchInput;
- /** The Remove field the renderer holds for a patch's strokes, or for strokes they extend. */
- field: (layerId: string, patch: RemovePatch) => HeldField | undefined;
- /** Keeps a Remove field this render synthesizes for a patch's strokes, for the renders after it. */
+ /** The Remove field a patch shows, or else the nearest one the renderer holds that its synthesis extends. */
+ field: (patch: RemovePatch) => PatchField | undefined;
+ /** Keeps a Remove field this render synthesizes for a patch, for the renders after it. */
keepField: (
- layerId: string,
patch: RemovePatch,
lattice: FieldLattice,
texels: RenderImage,
) => void;
};
+/** A Remove field a render shows, `complete` when it is the patch's own rather than one it extends. */
+export type PatchField = {
+ texels: RenderImage;
+ lattice: FieldLattice;
+ complete: boolean;
+};
+
/** App composition describes requested outputs; the engine owns their storage. */
export type SceneProcessing = (
source: RenderImage,
@@ -106,18 +114,15 @@ function sameBalance(a: WhiteBalance | undefined, b: WhiteBalance | undefined) {
return a?.temperature === b?.temperature && a?.tint === b?.tint;
}
-/** A Remove patch's field, held by Healing layer and patch, since duplicated layers repeat patch IDs. */
-function fieldKey(layerId: string, patch: RemovePatch) {
- return `${layerId}/${patch.id}`;
-}
-
/** A renderer's timer, and where it reads what the document stores outside the scene, by the IDs the scene names. */
export type RendererOptions = {
timer?: Timer;
/** The settled pixels a paint layer's or brush mask's `raster` names. */
paintPixels?: (id: string) => Blob;
- /** The texels a Remove patch's field names. */
- fieldTexels?: (id: string) => Blob;
+ /** A Remove field resource: its field once synthesized, or else the field its synthesis extends. */
+ field?: (id: string) => FieldRecord | undefined;
+ /** Receives each Remove field the renderer synthesizes, read back; without it nothing is read back. */
+ saveField?: (id: string, field: RemoveField) => void;
};
/**
@@ -133,9 +138,8 @@ export function createRenderer(
paintPixels = () => {
throw Error("This renderer has no settled paint.");
},
- fieldTexels = () => {
- throw Error("This renderer has no Remove fields.");
- },
+ field: fieldRecord = () => undefined,
+ saveField,
}: RendererOptions = {},
) {
const source = resource.image;
@@ -167,6 +171,8 @@ export function createRenderer(
let pending: Promise<void> | undefined;
/** A raster being read back, a painting to settle or a Remove field to save; nothing renders meanwhile. */
let settling: Promise<unknown> | undefined;
+ /** Readbacks of synthesized fields in flight, by ID, which finish even once the renderer closes. */
+ const capturing = new Map<string, Promise<RemoveField | undefined>>();
let disposed = false;
let instances = new Set<string>();
function render(request: RenderRequest) {
@@ -191,9 +197,12 @@ export function createRenderer(
paints.draw(layer.id, layer, developed.size);
}
}
- // Remove fields stay while their patches do, shown or not.
- for (const { layer, patch } of removePatches(scene.layers)) {
- fields.retain(fieldKey(layer.id, patch));
+ // Remove fields stay while their patches do, shown or not, and so do the fields they extend.
+ for (const { patch } of removePatches(scene.layers)) {
+ const held = heldField(patch.field);
+ if (held) {
+ fields.retain(held);
+ }
}
const image =
factor > 1 ? proxy.render(developed, factor, version) : input(developed);
@@ -210,8 +219,7 @@ export function createRenderer(
}
}
const syntheses: {
- key: string;
- patch: RemovePatch;
+ id: string;
lattice: FieldLattice;
texels: RenderImage;
}[] = [];
@@ -226,15 +234,19 @@ export function createRenderer(
}),
paint: (layer) => paints.input(layer),
patch: (id, strokes) => patches.patch(id, strokes, developed.size),
- field: (layerId, patch) =>
- fields.get(fieldKey(layerId, patch), patch.strokes, patch.field),
- keepField: (layerId, patch, lattice, texels) =>
- syntheses.push({
- key: fieldKey(layerId, patch),
- patch,
- lattice,
- texels,
- }),
+ field: (patch) => {
+ // Patches that share a field, such as duplicates, show the one this render synthesizes.
+ const synthesized = syntheses.find(({ id }) => id === patch.field);
+ if (synthesized) {
+ const { texels, lattice } = synthesized;
+ return { texels, lattice, complete: true };
+ }
+ const id = heldField(patch.field);
+ const held = id && fields.get(id);
+ return held ? { ...held, complete: id === patch.field } : undefined;
+ },
+ keepField: (patch, lattice, texels) =>
+ syntheses.push({ id: patch.field, lattice, texels }),
});
for (const id of instances) {
if (!active.has(id)) graph.release(`${id}/`);
@@ -265,10 +277,13 @@ export function createRenderer(
3 + inputs.length + sources.length + covered.size,
);
const synthesized = targets.slice(targets.length - syntheses.length);
- for (const [i, { key, patch, lattice }] of syntheses.entries()) {
- fields.keep(key, patch.strokes, lattice, synthesized[i]);
+ for (const [i, { id, lattice }] of syntheses.entries()) {
+ fields.keep(id, lattice, synthesized[i]);
}
fields.sweep();
+ if (saveField && syntheses.length) {
+ capture(syntheses.map(({ id }) => id));
+ }
inspected =
inputId && images.input ? { id: inputId, image: inputTarget } : undefined;
kept =
@@ -286,16 +301,65 @@ export function createRenderer(
function paintRaster(id: string) {
return paints.get(id) ? paints : brushes;
}
- /** Remove fields to load before rendering: saved ones the renderer holds nothing for. */
+ /** A Remove field's ID, then, while each waits to be synthesized, the IDs of the fields it extends. */
+ function* lineage(id: string) {
+ let next: string | undefined = id;
+ while (next) {
+ yield next;
+ const record = fieldRecord(next);
+ next = record?.field ? undefined : record?.base;
+ }
+ }
+ /** The ID of the field held for a patch's own field: that field, or the nearest one it extends. */
+ function heldField(id: string) {
+ for (const ancestor of lineage(id)) {
+ if (fields.get(ancestor)) {
+ return ancestor;
+ }
+ }
+ return undefined;
+ }
+ /** Remove fields to load before rendering: for a patch with none held, the saved one nearest its own. */
function staleFields(scene: Scene) {
- const stale = [];
- for (const { layer, patch } of removePatches(scene.layers)) {
- const key = fieldKey(layer.id, patch);
- if (fields.needs(key, patch.strokes, patch.field)) {
- stale.push({ key, strokes: patch.strokes, field: patch.field });
+ const stale = new Map<string, RemoveField>();
+ for (const { patch } of removePatches(scene.layers)) {
+ if (heldField(patch.field)) {
+ continue;
+ }
+ const nearest = [...lineage(patch.field)].at(-1) ?? patch.field;
+ const saved = fieldRecord(nearest)?.field;
+ if (saved) {
+ stale.set(nearest, saved);
}
}
- return stale;
+ return [...stale];
+ }
+ /**
+ * Reads the fields a render synthesized back for the document, rendering nothing meanwhile, so no
+ * render replaces one first. A field whose readback fails stays held, and `captureFields` reads it
+ * again for a snapshot, which reports the failure.
+ */
+ function capture(ids: readonly string[]) {
+ let reading: Promise<unknown> = Promise.resolve();
+ for (const id of ids) {
+ const read = reading.then(() => fields.read(id));
+ capturing.set(id, read);
+ reading = read
+ .then((field) => field && saveField?.(id, field))
+ .catch(() => {})
+ .finally(() => {
+ if (capturing.get(id) === read) {
+ capturing.delete(id);
+ }
+ });
+ }
+ const done = reading;
+ settling = done;
+ void done.then(() => {
+ if (settling === done) {
+ settling = undefined;
+ }
+ });
}
/** Paintings whose rasters must load settled pixels before they draw. */
function stalePaint(scene: Scene) {
@@ -338,8 +402,8 @@ export function createRenderer(
return;
}
}
- for (const { key, strokes, field } of staleFields(scene)) {
- await fields.load(key, strokes, field, fieldTexels(field.texels));
+ for (const [id, field] of staleFields(scene)) {
+ await fields.load(id, field);
if (disposed) {
return;
}
@@ -474,11 +538,51 @@ export function createRenderer(
);
},
/**
- * Reads the Remove field held for exactly a patch's strokes, deflated, once renders in flight
- * finish, holding back the next ones meanwhile; undefined when none is held.
+ * The fields among `ids` that the document saved or this renderer holds, once renders and readbacks
+ * in flight finish, saving those the document still waits for; rejects when a readback fails. Once
+ * the renderer closes, it gives what its readbacks in flight read, and rejects for a field it held
+ * that none was reading.
*/
- readField(layerId: string, patch: RemovePatch) {
- return hold(() => fields.read(fieldKey(layerId, patch), patch.strokes));
+ async captureFields(ids: readonly string[]) {
+ const inFlight = ids.flatMap((id) => {
+ const read = capturing.get(id);
+ return read ? [[id, read] as const] : [];
+ });
+ const unread = ids.some(
+ (id) => !capturing.has(id) && fields.get(id) && !fieldRecord(id)?.field,
+ );
+ const found = await hold(async () => {
+ const found = new Map<string, RemoveField>();
+ for (const id of ids) {
+ const saved = fieldRecord(id)?.field;
+ if (saved) {
+ found.set(id, saved);
+ continue;
+ }
+ const read = await fields.read(id);
+ if (read) {
+ found.set(id, read);
+ saveField?.(id, read);
+ }
+ }
+ return found;
+ });
+ if (found) {
+ return found;
+ }
+ if (unread) {
+ throw Error(
+ "The photo closed before its Remove fields were read back.",
+ );
+ }
+ const read = new Map<string, RemoveField>();
+ for (const [id, reading] of inFlight) {
+ const field = await reading;
+ if (field) {
+ read.set(id, field);
+ }
+ }
+ return read;
},
dispose() {
if (disposed) {
diff --git a/src/features/heal/edits.ts b/src/features/heal/edits.ts
index efd453e..2677cda 100644
--- a/src/features/heal/edits.ts
+++ b/src/features/heal/edits.ts
@@ -52,6 +52,7 @@ export function addRemovePatch(
feather: painted.feather,
strokes: [{ ...painted, feather: 0, flow: 1 }],
opacity: 1,
+ field: document.resources.reserveField(),
};
editLayer(document, id, (layer) => ({
...layer,
@@ -60,6 +61,19 @@ export function addRemovePatch(
return patch.id;
}
+/** A patch with new strokes; a Remove patch takes a new field, which extends its earlier one. */
+function withStrokes(
+ document: EditorDocument,
+ patch: HealPatch,
+ strokes: readonly BrushStroke[],
+): HealPatch {
+ if (patch.mode !== "remove") {
+ return { ...patch, strokes };
+ }
+ const field = document.resources.reserveField(patch.field);
+ return { ...patch, strokes, field };
+}
+
/** Rewrites the patch list around one patch, found by id. */
function editPatches(
document: EditorDocument,
@@ -76,8 +90,8 @@ function editPatches(
}
/**
- * Adds or subtracts a separate stroke without changing the patch's donor or blend. A Remove patch keeps
- * its field, which the next synthesis extends.
+ * Adds or subtracts a separate stroke without changing the patch's donor or blend. A Remove patch takes
+ * a new field, which extends its earlier one.
*/
export function addHealStroke(
document: EditorDocument,
@@ -87,10 +101,13 @@ export function addHealStroke(
) {
const painted = parse(strokeSchema, stroke, "Invalid heal stroke");
editPatches(document, id, patchId, (patches, index) =>
- patches.with(index, {
- ...patches[index],
- strokes: [...patches[index].strokes, { ...painted, feather: 0, flow: 1 }],
- }),
+ patches.with(
+ index,
+ withStrokes(document, patches[index], [
+ ...patches[index].strokes,
+ { ...painted, feather: 0, flow: 1 },
+ ]),
+ ),
);
}
@@ -152,15 +169,10 @@ export function extendHealPatch(
...last,
points: [...last.points, ...added],
});
- // A Remove field counts whole strokes, so a stroke that grows takes a new one.
- if (patch.mode === "remove") {
- const { field, ...unfilled } = patch;
- return {
- ...layer,
- patches: layer.patches.with(-1, { ...unfilled, strokes }),
- };
- }
- return { ...layer, patches: layer.patches.with(-1, { ...patch, strokes }) };
+ return {
+ ...layer,
+ patches: layer.patches.with(-1, withStrokes(document, patch, strokes)),
+ };
});
}
@@ -197,8 +209,9 @@ export function setHealDestination(
),
}));
if (patch.mode === "remove") {
- const { field, ...unfilled } = patch;
- return patches.with(index, { ...unfilled, strokes });
+ // A patch in a new place keeps nothing it filled.
+ const field = document.resources.reserveField();
+ return patches.with(index, { ...patch, strokes, field });
}
return patches.with(index, {
...patch,
diff --git a/src/features/heal/inpaint/pass.ts b/src/features/heal/inpaint/pass.ts
index cfd6bba..902f299 100644
--- a/src/features/heal/inpaint/pass.ts
+++ b/src/features/heal/inpaint/pass.ts
@@ -20,21 +20,20 @@ import shader from "./blend.wgsl";
type FieldComposition = Pick<Composition, "field" | "keepField">;
/**
- * The field a patch shows: the one held for its strokes, or else one synthesized from the image below
- * at full resolution, which keeps what the field held for its first strokes filled. A proxy shows the
- * held field meanwhile, since its reduced image would synthesize another.
+ * The field a patch shows: its own, or else one synthesized from the image below at full resolution,
+ * which keeps what the field it extends filled. A proxy shows that field meanwhile, since its reduced
+ * image would synthesize another.
*/
function patchField(
source: RenderImage,
coverage: PatchInput,
region: InpaintRegion,
- layerId: string,
patch: RemovePatch,
name: string,
composition: FieldComposition,
): { texels: RenderImage; lattice: FieldLattice } | undefined {
- const held = composition.field(layerId, patch);
- if (held?.strokes.length === patch.strokes.length) return held;
+ const held = composition.field(patch);
+ if (held?.complete) return held;
if (source.scale.some((scale) => scale > 1)) return held;
const lattice = fieldLattice(region, held?.lattice);
const texels = inpaintField(
@@ -44,14 +43,13 @@ function patchField(
`${name}/inpaint`,
held,
);
- composition.keepField(layerId, patch, lattice, texels);
+ composition.keepField(patch, lattice, texels);
return { texels, lattice };
}
export function removePatch(
source: RenderImage,
coverage: PatchInput,
- layerId: string,
patch: RemovePatch,
name: string,
composition: FieldComposition,
@@ -62,15 +60,7 @@ export function removePatch(
return Math.max(margin, stroke.size);
}, 32);
const region = patchBounds(patch.strokes, dimensions, margin);
- const field = patchField(
- source,
- coverage,
- region,
- layerId,
- patch,
- name,
- composition,
- );
+ const field = patchField(source, coverage, region, patch, name, composition);
if (!field) return source;
const filled = sampleInpaint(
source,
diff --git a/src/features/heal/model.ts b/src/features/heal/model.ts
index 74d3757..0cfdc0e 100644
--- a/src/features/heal/model.ts
+++ b/src/features/heal/model.ts
@@ -133,9 +133,8 @@ const patchShape = {
const fieldSide = z.int().check(z.minimum(1), z.maximum(fieldLongSide));
-const removeFieldSchema = z.object({
- texels: z.string().check(z.minLength(1)),
- strokes: z.int().check(z.minimum(1)),
+/** Where a Remove field's texels sit in the photo, as a scene file stores it beside the texels. */
+export const fieldLatticeSchema = z.object({
origin: point,
scale: z.number().check(z.minimum(1)),
size: z.tuple([fieldSide, fieldSide]),
@@ -147,16 +146,9 @@ export const healPatchSchema = z.union([
mode: z._default(donorModeSchema, "heal"),
offset: point,
}),
- z
- .object({
- ...patchShape,
- mode: z.literal("remove"),
- field: z.optional(removeFieldSchema),
- })
- .check(
- z.refine(
- (patch) => (patch.field?.strokes ?? 0) <= patch.strokes.length,
- "A Remove field covers at most its patch's strokes",
- ),
- ),
+ z.object({
+ ...patchShape,
+ mode: z.literal("remove"),
+ field: z.string().check(z.minLength(1)),
+ }),
]) satisfies z.ZodMiniType<HealPatch>;
diff --git a/src/features/heal/pass.ts b/src/features/heal/pass.ts
index 3f97f23..9d52343 100644
--- a/src/features/heal/pass.ts
+++ b/src/features/heal/pass.ts
@@ -106,7 +106,6 @@ export function retainHealPatches(
export function heal(
source: RenderImage,
- layerId: string,
patches: readonly HealPatch[],
name: string,
composition: HealComposition,
@@ -120,7 +119,7 @@ export function heal(
if (!repairsPixels(patch)) continue;
const coverage = composition.patch(id, patch.strokes);
if (patch.mode === "remove") {
- image = removePatch(image, coverage, layerId, patch, id, composition);
+ image = removePatch(image, coverage, patch, id, composition);
} else {
image = healPatch(image, coverage, patch, id);
}
diff --git a/src/features/heal/save-fields.tsx b/src/features/heal/save-fields.tsx
deleted file mode 100644
index ae8c7f3..0000000
--- a/src/features/heal/save-fields.tsx
+++ /dev/null
@@ -1,153 +0,0 @@
-import { useEffect } from "react";
-import { useRenderer } from "@/components/editor/pipeline";
-import { useDocument } from "@/components/editor/session";
-import {
- type BrushStroke,
- type EditorDocument,
- findLayer,
- type RemovePatch,
- removePatches,
- updateLayer,
-} from "@/core/document";
-
-type Renderer = ReturnType<typeof useRenderer>;
-
-/** Whether a patch's saved field covers every stroke it has. */
-function saved(patch: RemovePatch) {
- return patch.field?.strokes === patch.strokes.length;
-}
-
-/** Whether `strokes` start with every stroke of `first`, the same objects. */
-function startsWith(
- strokes: readonly BrushStroke[],
- first: readonly BrushStroke[],
-) {
- return (
- first.length <= strokes.length &&
- first.every((stroke, i) => stroke === strokes[i])
- );
-}
-
-/**
- * Saves the field the renderer synthesized for a Remove patch's strokes into the document. The scene
- * changes in place: the patch shows the same, and from then on undo, scene files, and other renderers
- * show it so too. A stroke added meanwhile keeps the field as the base its synthesis extends; any
- * other change to the patch, or a gesture opened meanwhile, drops it.
- */
-async function saveRemoveField(
- document: EditorDocument,
- renderer: Pick<Renderer, "readField">,
- layerId: string,
- patch: RemovePatch,
-) {
- const read = await renderer.readField(layerId, patch);
- if (!read || document.closed || document.history.status.getState().editing) {
- return;
- }
- const scene = document.scene.getState();
- const healing = findLayer(scene.layers, layerId);
- const current =
- healing?.kind === "heal"
- ? healing.patches.find(({ id }) => id === patch.id)
- : undefined;
- if (
- current?.mode !== "remove" ||
- !startsWith(current.strokes, patch.strokes) ||
- (current.field?.strokes ?? 0) >= patch.strokes.length
- ) {
- return;
- }
- const field = {
- texels: document.resources.addField(read.texels),
- strokes: patch.strokes.length,
- ...read.lattice,
- };
- document.replace(
- updateLayer(scene, layerId, (layer) =>
- layer.kind === "heal"
- ? {
- ...layer,
- patches: layer.patches.map((item) =>
- item === current ? { ...current, field } : item,
- ),
- }
- : layer,
- ),
- );
-}
-
-/**
- * Saves each Remove field once the renderer synthesizes it and no gesture is open, checking after
- * every render and when a gesture ends. A render during a save checks again after it; a patch the
- * renderer holds no field for, such as one on a hidden layer, waits for the next render. A snapshot
- * first saves what the scene lacks, so a field whose save failed tries again. Returns the stop.
- */
-export function watchRemoveFields(
- document: EditorDocument,
- renderer: Pick<Renderer, "readField" | "subscribe">,
-) {
- let active = true;
- let pass: Promise<void> | undefined;
- let rendered = false;
- async function save() {
- const patches = [...removePatches(document.scene.getState().layers)];
- for (const { layer, patch } of patches) {
- // A gesture stops the pass, since a readback holds renders; its end checks again.
- if (!active || document.history.status.getState().editing) {
- return;
- }
- if (!saved(patch)) {
- await saveRemoveField(document, renderer, layer.id, patch);
- }
- }
- }
- function start() {
- rendered = false;
- pass = save().finally(() => {
- pass = undefined;
- if (rendered) {
- check();
- }
- });
- return pass;
- }
- function check() {
- if (!active || document.history.status.getState().editing) {
- return;
- }
- if (pass) {
- rendered = true;
- return;
- }
- const scene = document.scene.getState();
- if ([...removePatches(scene.layers)].every(({ patch }) => saved(patch))) {
- return;
- }
- void start().catch((error) =>
- console.error("A Remove field couldn't be saved.", error),
- );
- }
- /** Saves what the scene lacks after the passes in flight, and fails with its readback. */
- async function prepare() {
- while (pass) {
- await pass.catch(() => {});
- }
- await start();
- }
- const unsubscribe = renderer.subscribe(check);
- const detach = document.history.status.subscribe(check);
- const removePreparation = document.onSnapshot(prepare);
- return () => {
- active = false;
- unsubscribe();
- detach();
- removePreparation();
- };
-}
-
-export function RemoveFieldSaving() {
- const document = useDocument();
- const renderer = useRenderer();
- useEffect(() => watchRemoveFields(document, renderer), [document, renderer]);
- return null;
-}
diff --git a/tests/draft.test.ts b/tests/draft.test.ts
index 0c28cef..b8ae01d 100644
--- a/tests/draft.test.ts
+++ b/tests/draft.test.ts
@@ -33,9 +33,11 @@ test("a draft survives storage's structured clone, reopens under its source ID,
setVignette(document, { intensity: 30, softness: 70 }, vignette);
const edited = document.scene.getState();
- // IndexedDB stores values with the structured clone algorithm, which keeps File objects.
- const stored = structuredClone(snapshotDraft(document, "photo.png"));
+ // Taken before the document closes, a draft still completes; IndexedDB stores it with the structured
+ // clone algorithm, which keeps File objects.
+ const draft = snapshotDraft(document, "photo.png");
document.dispose();
+ const stored = structuredClone(await draft);
expect(stored.record).toMatchObject({ version: 1, name: "photo.png" });
expect(await stored.files.get(source)?.bytes()).toEqual(bytes);
const recovered = await openDraft(stored, decode);
diff --git a/tests/heal-edit.e2e.ts b/tests/heal-edit.e2e.ts
index 4e88886..f105f45 100644
--- a/tests/heal-edit.e2e.ts
+++ b/tests/heal-edit.e2e.ts
@@ -64,18 +64,6 @@ for (const mode of ["Remove", "Heal"]) {
.scene?.layers.find((layer) => layer.kind === "heal");
return layer?.kind === "heal" ? layer.patches : [];
});
- // Remove saves the field it synthesized into its patch once the render reads it back.
- const saved = () =>
- expect
- .poll(async () =>
- (await patches()).every(
- (patch) =>
- patch.mode !== "remove" ||
- patch.field?.strokes === patch.strokes.length,
- ),
- )
- .toBe(true);
- await saved();
const original = (await patches())[0];
const removed = await readImage(page, undefined, samples);
const layerId = await page.evaluate(
@@ -103,7 +91,6 @@ for (const mode of ["Remove", "Heal"]) {
await page.mouse.up();
await page.keyboard.up("Shift");
await expect(preview).toHaveCount(0);
- await saved();
const added = (await patches())[0];
expect(await patches()).toHaveLength(1);
expect(added).toMatchObject({
diff --git a/tests/heal.test.ts b/tests/heal.test.ts
index cfca230..a17c353 100644
--- a/tests/heal.test.ts
+++ b/tests/heal.test.ts
@@ -4,14 +4,16 @@ import { createImageLayer, createLayer, createMask } from "@/app/editor/layers";
import { createEditorRenderer } from "@/app/editor/renderer";
import {
type BrushStroke,
+ completeFields,
createDocument,
createResources,
+ fieldRecords,
findLayer,
removePatches,
} from "@/core/document";
import { createImageSource } from "@/core/image";
import { imageFrame } from "@/core/image/frame";
-import { createRenderGraph, type FieldLattice, input } from "@/core/renderer";
+import { createRenderGraph, input } from "@/core/renderer";
import { setAdjustments } from "@/features/adjustments/edits";
import {
addHealPatch,
@@ -25,7 +27,6 @@ import {
setHealSource,
} from "@/features/heal/edits";
import { inpaintField } from "@/features/heal/inpaint";
-import { watchRemoveFields } from "@/features/heal/save-fields";
import { addLayer, deleteLayer, setLayer } from "@/features/layers/edits";
function healFixture(size: readonly [number, number] = [64, 64]) {
@@ -185,13 +186,16 @@ function patchesOf(document: ReturnType<typeof createDocument>, layer: string) {
return healing.patches;
}
-test("a nested Remove synthesizes once for its strokes, whatever changes below or hides it, and extends on new strokes", async () => {
+test("a nested Remove synthesizes once for its strokes, whatever changes below or hides it, extends on new strokes, and undoes without solving again", async () => {
const gpu = await init();
const source = createImageSource(
target(gpu, { size: [128, 96], format: "rgba16float" }),
);
const { document } = healFixture([128, 96]);
- const renderer = createEditorRenderer(gpu, source);
+ const renderer = createEditorRenderer(gpu, source, {
+ field: (id) => document.resources.field(id),
+ saveField: (id, field) => document.resources.fillField(id, field),
+ });
const mask = addLayer(
document,
createMask({ kind: "linear", start: [0, 0], end: [0, 96] }),
@@ -207,10 +211,22 @@ test("a nested Remove synthesizes once for its strokes, whatever changes below o
renderer.inspect().passes.some((name) => name.includes("/inpaint/"));
const fields = () =>
renderer.inspect().rasters.filter(({ id }) => id.endsWith("/field"));
+ const own = () => {
+ const healing = findLayer(document.scene.getState().layers, layer);
+ const found = healing?.kind === "heal" ? healing.patches[0] : undefined;
+ if (found?.mode !== "remove") throw Error("The Remove patch is gone.");
+ return found.field;
+ };
+ /** Whether the document saved the field the renderer synthesized, once its readback finishes. */
+ const saved = async () => {
+ await renderer.captureFields([]);
+ return document.resources.field(own())?.field !== undefined;
+ };
try {
await render();
expect(solves()).toBe(true);
- expect(fields()).toMatchObject([{ id: `${layer}/${patch}/field` }]);
+ expect(fields()).toMatchObject([{ id: `${own()}/field` }]);
+ expect(await saved()).toBe(true);
setHealPatch(document, layer, patch, { opacity: 0.5, feather: 0.4 });
await render();
expect(solves()).toBe(false);
@@ -232,23 +248,36 @@ test("a nested Remove synthesizes once for its strokes, whatever changes below o
await render(true);
expect(solves()).toBe(false);
for (const mode of ["paint", "erase"] as const) {
+ const before = own();
addHealStroke(document, layer, patch, {
...dab,
mode,
size: 2,
points: [[64, 48, 1]],
});
+ // A new stroke takes a new field, which extends the earlier one.
+ expect(document.resources.field(own())).toEqual({ base: before });
// A proxy keeps the field it holds; only full resolution synthesizes the new stroke.
await render(true);
expect(solves()).toBe(false);
await render();
expect(solves()).toBe(true);
+ expect(await saved()).toBe(true);
setHealPatch(document, layer, patch, {
opacity: mode === "paint" ? 0.6 : 0.5,
});
await render();
expect(solves()).toBe(false);
}
+ // Undo and redo reach every stroke version's saved field, and show it without solving again, even
+ // when the field it extends is the one held.
+ for (const travel of ["undo", "redo"] as const) {
+ for (let step = 0; step < 4; step++) {
+ document.history[travel]();
+ await render();
+ expect(solves()).toBe(false);
+ }
+ }
setHealDestination(document, layer, patch, [40, 40]);
await render();
expect(solves()).toBe(true);
@@ -263,88 +292,68 @@ test("a nested Remove synthesizes once for its strokes, whatever changes below o
}
});
-test("Remove fields save once no gesture is open, a stroke added during a readback takes its field as a base, and a snapshot saves what the scene lacks or fails with its readback", async () => {
- const { document, layer } = healFixture();
- // A renderer whose renders the test runs, and whose readbacks finish or fail, oldest first, when it
- // says.
- let render = () => {};
- const reads: ((error?: Error) => void)[] = [];
- const renderer = {
- subscribe(listener: () => void) {
- render = listener;
- return () => {};
- },
- readField: () =>
- new Promise<{ texels: Blob; lattice: FieldLattice }>((resolve, reject) =>
- reads.push((error) =>
- error
- ? reject(error)
- : resolve({
- texels: new Blob(["field"]),
- lattice: { origin: [0, 0], scale: 1, size: [8, 8] },
- }),
- ),
- ),
- };
- async function finish(error?: Error) {
- reads.shift()?.(error);
- await new Promise((resolve) => setTimeout(resolve, 0));
- }
- const patches = () =>
- [...removePatches(document.scene.getState().layers)].map(
- ({ patch }) => patch,
- );
- const first = addRemovePatch(document, layer, dab);
- addRemovePatch(document, layer, dab);
- const { undoCount } = document.history.status.getState();
- const stop = watchRemoveFields(document, renderer);
+test("a snapshot taken while a Remove field is read back keeps it once its photo closes", async () => {
+ const gpu = await init();
+ const source = createImageSource(
+ target(gpu, { size: [128, 96], format: "rgba16float" }),
+ );
+ const { document, layer } = healFixture([128, 96]);
+ const renderer = createEditorRenderer(gpu, source, {
+ field: (id) => document.resources.field(id),
+ saveField: (id, field) => document.resources.fillField(id, field),
+ });
+ document.onCaptureFields((ids) => renderer.captureFields(ids));
try {
- // A gesture drops the readback that finishes inside it and reads nothing more until it ends.
- render();
- document.history.begin();
- await finish();
- expect(reads).toHaveLength(0);
- expect(patches().map(({ field }) => field)).toEqual([undefined, undefined]);
- // Its end saves both fields, with no undo step.
- document.history.commit();
- await finish();
- await finish();
- expect(patches().map(({ field }) => field?.strokes)).toEqual([1, 1]);
- expect(document.history.status.getState().undoCount).toBe(undoCount);
-
- addHealStroke(document, layer, first, dab);
- addHealStroke(document, layer, first, dab);
- await finish();
- expect(patches()[0].strokes).toHaveLength(3);
- expect(patches()[0].field?.strokes).toBe(2);
- // The field for every stroke is being read back, and a snapshot waits for it.
- const snapshot = document
- .prepareSnapshot()
- .then(() => patches()[0].field?.strokes);
- await finish();
- expect(await snapshot).toBe(3);
+ addRemovePatch(document, layer, { ...dab, points: [[64, 48, 1]] });
+ const [{ patch }] = removePatches(document.scene.getState().layers);
+ // The render synthesizes the field and starts reading it back; another photo then closes this one.
+ void renderer.update(document.scene.getState());
+ const snapshot = completeFields(document, fieldRecords(document));
+ renderer.dispose();
+ document.dispose();
+ expect((await snapshot).get(patch.field)?.field).toBeDefined();
+ } finally {
+ source.dispose();
+ gpu.dispose();
+ }
+});
- // A snapshot after a failed readback reads the field again, and fails when that fails too.
- const errors = spyOn(console, "error").mockImplementation(() => {});
- addHealStroke(document, layer, first, dab);
- await finish(Error("Device lost"));
- expect(errors).toHaveBeenCalledTimes(1);
- const failed = document.prepareSnapshot().then(
- () => undefined,
- (error: Error) => error.message,
+test("a snapshot reads back a Remove field whose save failed, and fails when that fails too", async () => {
+ const gpu = await init();
+ const source = createImageSource(
+ target(gpu, { size: [128, 96], format: "rgba16float" }),
+ );
+ const { document, layer } = healFixture([128, 96]);
+ // A save that fails while `failing` holds, as a readback the device refuses would.
+ let failing = true;
+ const renderer = createEditorRenderer(gpu, source, {
+ field: (id) => document.resources.field(id),
+ saveField: (id, field) => {
+ if (failing) throw Error("Device lost");
+ document.resources.fillField(id, field);
+ },
+ });
+ const stop = document.onCaptureFields((ids) => renderer.captureFields(ids));
+ try {
+ addRemovePatch(document, layer, { ...dab, points: [[64, 48, 1]] });
+ const [{ patch }] = removePatches(document.scene.getState().layers);
+ await renderer.update(document.scene.getState());
+ await renderer.captureFields([]);
+ expect(document.resources.field(patch.field)).toEqual({});
+ const records = fieldRecords(document);
+ await expect(completeFields(document, records)).rejects.toThrow(
+ "Device lost",
);
- expect(reads).toHaveLength(1);
- await finish(Error("Device lost"));
- expect(await failed).toBe("Device lost");
- const retried = document
- .prepareSnapshot()
- .then(() => patches()[0].field?.strokes);
- await finish();
- expect(await retried).toBe(4);
- errors.mockRestore();
+ failing = false;
+ const completed = await completeFields(document, records);
+ expect(completed.get(patch.field)?.field).toBeDefined();
+ expect(document.resources.field(patch.field)?.field).toBeDefined();
} finally {
stop();
+ renderer.dispose();
document.dispose();
+ source.dispose();
+ gpu.dispose();
}
});
diff --git a/tests/inpaint-benchmark.ts b/tests/inpaint-benchmark.ts
index c035cd6..208dc5c 100644
--- a/tests/inpaint-benchmark.ts
+++ b/tests/inpaint-benchmark.ts
@@ -68,6 +68,8 @@ export async function benchmarkInpaint(
{
id: "spot",
mode: "remove",
+ // Each render's stroke is a new version, which synthesizes its own field.
+ field: `spot/${index}`,
feather: 0.4,
opacity: 1,
strokes: [strokeAt(index)],
diff --git a/tests/inpaint-gpu.ts b/tests/inpaint-gpu.ts
index 8a9b4cb..e147f83 100644
--- a/tests/inpaint-gpu.ts
+++ b/tests/inpaint-gpu.ts
@@ -2,7 +2,7 @@ import { effect, frame, init, target } from "vgpu";
import { encodeImage } from "@/app/editor/export/export-image";
import { createImageLayer, createLayer } from "@/app/editor/layers";
import { createEditorRenderer } from "@/app/editor/renderer";
-import type { HealPatch, Scene } from "@/core/document";
+import type { FieldRecord, HealPatch, Scene } from "@/core/document";
import { createImageSource } from "@/core/image";
import { imageFrame } from "@/core/image/frame";
@@ -53,6 +53,7 @@ export async function renderInpaintReference(
const patch: HealPatch = {
id: "object",
mode: "remove",
+ field: "object",
feather: 0,
opacity: 1,
strokes: [
@@ -257,7 +258,16 @@ export async function renderInpaintExtension() {
const size: [number, number] = [192, 144];
const image = target(gpu, { size, format: "rgba16float" });
const source = createImageSource(image);
- const renderer = createEditorRenderer(gpu, source);
+ // Each stroke count is a version with its own field, extending the one before, as edits reserve them.
+ const fields = ["spots/1", "spots/2", "spots/3"];
+ const records = new Map<string, FieldRecord>([
+ [fields[0], {}],
+ [fields[1], { base: fields[0] }],
+ [fields[2], { base: fields[1] }],
+ ]);
+ const renderer = createEditorRenderer(gpu, source, {
+ field: (id) => records.get(id),
+ });
const first = [56, 72] as const;
const second = [140, 72] as const;
const erased = [48, 72] as const;
@@ -289,6 +299,7 @@ export async function renderInpaintExtension() {
{
id: "spots",
mode: "remove",
+ field: fields[count - 1],
feather: 0,
opacity: 1,
strokes: strokes.slice(0, count),
diff --git a/tests/inpaint-photo.ts b/tests/inpaint-photo.ts
index a0ec578..a0a0344 100644
--- a/tests/inpaint-photo.ts
+++ b/tests/inpaint-photo.ts
@@ -59,6 +59,7 @@ export async function removeFromPhoto(stroke: BrushStroke, spot = false) {
{
id: "object",
mode: "remove",
+ field: "object",
feather: stroke.feather,
opacity: 1,
strokes: [{ ...stroke, feather: 0 }],
diff --git a/tests/rendering-benchmark.ts b/tests/rendering-benchmark.ts
index 3dadb81..eb39215 100644
--- a/tests/rendering-benchmark.ts
+++ b/tests/rendering-benchmark.ts
@@ -302,7 +302,7 @@ export async function benchmarkRendering(
const donorMode = workload === "clone" ? "clone" : "heal";
const patch: HealPatch =
workload === "remove"
- ? { ...shape, mode: "remove" }
+ ? { ...shape, mode: "remove", field: "benchmark-heal" }
: { ...shape, mode: donorMode, offset: [320, 0] };
effects.push({
...common,
diff --git a/tests/scene-file.e2e.ts b/tests/scene-file.e2e.ts
index cba1a87..afde268 100644
--- a/tests/scene-file.e2e.ts
+++ b/tests/scene-file.e2e.ts
@@ -1,4 +1,5 @@
import { readFile } from "node:fs/promises";
+import { readZip } from "@/lib/zip";
import { expect, openPhoto, test } from "./fixtures";
import { readImage } from "./images";
@@ -34,8 +35,8 @@ test("a saved scene reopens the photo with its edits and keeps editing", async (
flow: 1,
points: [[445, 295, 1]],
});
- // Saved in the same task as the stroke, before its field is read back, the scene still keeps the
- // field, so the reopened patch fills the same way.
+ // Saved in the same task as the stroke, before Remove synthesizes its field, the scene file still
+ // holds the field, so the reopened patch fills the same way.
const save = [...document.querySelectorAll("button")].find(
(button) => button.textContent === "Save scene",
);
@@ -44,6 +45,7 @@ test("a saved scene reopens the photo with its edits and keeps editing", async (
});
const download = await pending;
expect(download.suggestedFilename()).toBe("photo.openlight");
+ const bytes = await readFile(await download.path());
const field = () =>
page.evaluate(() => {
const layer = window.openlight
@@ -52,8 +54,11 @@ test("a saved scene reopens the photo with its edits and keeps editing", async (
const patch = layer?.kind === "heal" ? layer.patches[0] : undefined;
return patch?.mode === "remove" ? patch.field : undefined;
});
- await expect.poll(field).toMatchObject({ strokes: 1 });
const saved = await field();
+ const entries = await readZip(new Blob([bytes]));
+ const json = JSON.parse((await entries.get("scene.json")?.text()) ?? "");
+ expect(json.fields[saved ?? ""]).toHaveProperty("origin");
+ expect(entries.has(`fields/${saved}`)).toBe(true);
const edited = await readImage(page, undefined, samples);
await page.reload();
@@ -62,7 +67,7 @@ test("a saved scene reopens the photo with its edits and keeps editing", async (
await picker.setInputFiles({
name: "photo.openlight",
mimeType: "",
- buffer: await readFile(await download.path()),
+ buffer: bytes,
});
const exposure = page.getByRole("textbox", { name: "Exposure", exact: true });
await expect(exposure).toHaveValue("-1.00");
diff --git a/tests/scene-file.test.ts b/tests/scene-file.test.ts
index edd13df..03d6dab 100644
--- a/tests/scene-file.test.ts
+++ b/tests/scene-file.test.ts
@@ -142,10 +142,16 @@ test("scene files reopen the photo with every layer for further editing", async
expect(file.name).toBe("photo.openlight");
const entries = await readZip(file);
const json = JSON.parse((await entries.get("scene.json")?.text()) ?? "");
+ // The Remove patch was never rendered, so its field waits, with nothing it extends.
+ const removal = edited.layers
+ .flatMap((layer) => (layer.kind === "heal" ? layer.patches : []))
+ .find((patch) => patch.mode === "remove");
+ if (removal?.mode !== "remove") throw Error("Missing Remove patch.");
expect(json).toMatchObject({
format: "openlight",
- version: 1,
+ version: 2,
sources: { [source]: { name: "photo.nef", type: "image/x-nikon-nef" } },
+ fields: { [removal.field]: {} },
scene: edited,
});
const opened = await openSceneFile(file, decode);
@@ -173,7 +179,27 @@ test("scene files reopen the photo with every layer for further editing", async
const invalid: [Blob, string][] = [
[new Blob(["not a zip"]), "Not a ZIP archive."],
[await archive({ ...json, format: "other" }), "an OpenLight scene"],
- [await archive({ ...json, version: 2 }), "needs a newer version"],
+ [await archive({ ...json, version: 3 }), "needs a newer version"],
+ [await archive({ ...json, fields: {} }), "Remove fields are missing"],
+ [
+ await archive({
+ ...json,
+ fields: {
+ [removal.field]: { base: "loop" },
+ loop: { base: removal.field },
+ },
+ }),
+ "Remove fields are missing",
+ ],
+ [
+ await archive({
+ ...json,
+ fields: {
+ [removal.field]: { origin: [40, 12], scale: 1, size: [16, 16] },
+ },
+ }),
+ "Remove fields are missing",
+ ],
[await archive(json, new Map()), "The scene's image is missing."],
[await adding(saved[1]), "IDs must be unique"],
[await adding({ id: "new", kind: "text" }), "Unknown layer kind: text."],
@@ -212,16 +238,29 @@ test("scene files reopen the photo with every layer for further editing", async
patch.stroke = patch.strokes[0];
delete patch.strokes;
}
- const older = await openSceneFile(await archive(json), decode);
+ // Version 1 wrote a Remove field inside its patch, with how many strokes it covered.
+ const lattice = { origin: [40, 12], scale: 1, size: [16, 16] } as const;
+ healing.patches[2].field = { texels: "inline", strokes: 1, ...lattice };
+ const texels = new Blob(["texels"]);
+ const older = await openSceneFile(
+ await archive(
+ { ...json, version: 1, fields: undefined },
+ new Map([...entries, ["fields/inline", texels]]),
+ ),
+ decode,
+ );
const loaded = older.scene.getState();
expect(loaded.layers[0].whiteBalance).toEqual(asShot);
expect(loaded.layers.find((layer) => layer.kind === "heal")).toMatchObject({
patches: [
{ mode: "heal", strokes: [{ mode: "paint" }] },
{ mode: "clone", strokes: [{ mode: "paint" }] },
- { mode: "remove", strokes: [{ mode: "paint" }] },
+ { mode: "remove", strokes: [{ mode: "paint" }], field: "inline" },
],
});
+ expect(older.resources.field("inline")).toEqual({
+ field: { texels: expect.any(Blob), ...lattice },
+ });
expect(
loaded.layers.find((layer) => layer.kind === "vignette"),
@@ -231,7 +270,7 @@ test("scene files reopen the photo with every layer for further editing", async
gpu.dispose();
});
-test("a scene file waits for its preparations and still saves once another photo closes its document", async () => {
+test("a scene file waits for the fields the editor reads back and still saves once another photo closes its document", async () => {
const gpu = await init();
const resources = createResources();
const png = new File([bytes], "photo.png", { type: "image/png" });
@@ -246,16 +285,16 @@ test("a scene file waits for its preparations and still saves once another photo
},
resources,
);
- let prepared = () => {};
- document.onSnapshot(
+ let captured = () => {};
+ document.onCaptureFields(
() =>
- new Promise<void>((resolve) => {
- prepared = resolve;
+ new Promise((resolve) => {
+ captured = () => resolve(new Map());
}),
);
const saving = writeSceneFile(document);
document.dispose();
- prepared();
+ captured();
const entries = await readZip(await saving);
expect(await entries.get(`sources/${source}`)?.bytes()).toEqual(bytes);
gpu.dispose();