Skip to content

Free open-source image editor for the browser

Show each patch's opacity in its row, synthesize a changed Remove patch whole, and select patches from their first point (#85)

Rodrigo Belfioreb3991f77e82d
diff --git a/API.md b/API.md
index b5c95a5..c98cb5c 100644
--- a/API.md
+++ b/API.md
@@ -66,7 +66,7 @@ Create an empty Heal effect with `addLayer("heal")`.
`addHealStroke(id, patchId, stroke)` appends a painted or erased `BrushStroke` to an existing patch as one undoable edit. It preserves the mode, donor, feather, and opacity. Feather is a patch-wide blend based on the first stroke's diameter, shared by later additions. All patches expose an ordered `strokes` array; each path stays separate, coverage is combined on the GPU, and moving the patch moves paint and erase together. In the editor, Shift adds and Alt/Option subtracts on the selected patch, committing on release. Alt-click chooses a donor for Heal/Clone when no patch is selected. Older saved scenes with a single `stroke` load as one-element arrays.
-`addRemovePatch(id, stroke)` appends a Remove patch and returns its ID. It synthesizes the covered region from unmasked surrounding context using multiscale GPU PatchMatch; no offset or manual source is needed. `setHealSource` rejects Remove patches. The same feather, opacity, destination, duplicate, and delete edits apply. The patch's `field` names, by ID, where each pixel copies from; a new ID is reserved whenever its strokes change. In the editor, the first full-resolution render synthesizes that field and keeps it under its ID: changes to the layers below, undo, export, and scene files keep it, while the copied colors follow those layers. `exportImage` and `exportScene` first read back any field the editor shows that the document still waits for, and reject when that readback fails. Adding a stroke with `addHealStroke` keeps what the field already filled and synthesizes only the rest; moving the patch synthesizes it anew. The solver and compositor run through `createEditorRenderer` without React; [the solver documentation](src/features/heal/inpaint/algorithm.md) describes limits and reproducible benchmarks.
+`addRemovePatch(id, stroke)` appends a Remove patch and returns its ID. It synthesizes the covered region from unmasked surrounding context using multiscale GPU PatchMatch; no offset or manual source is needed. `setHealSource` rejects Remove patches. The same feather, opacity, destination, duplicate, and delete edits apply. The patch's `field` names, by ID, where each pixel copies from; a new ID is reserved whenever its strokes change. In the editor, the first full-resolution render synthesizes that field and keeps it under its ID: changes to the layers below, undo, export, and scene files keep it, while the copied colors follow those layers. `exportImage` and `exportScene` first read back any field the editor shows that the document still waits for, and reject when that readback fails. A stroke added or erased with `addHealStroke`, or a move, synthesizes the patch's whole new shape. The solver and compositor run through `createEditorRenderer` without React; [the solver documentation](src/features/heal/inpaint/algorithm.md) describes limits and reproducible benchmarks.
Create an empty Paint layer with `addLayer("paint")`. `addPaintStroke(id, stroke)` appends a `BrushStroke` with a `color`, `#rrggbb` sRGB: a painting stroke lays that color over the layer's earlier paint, and an erasing one removes paint whatever its color. `setPaintBlend(id, blend)` blends the layer's paint over the image below with the modes `setFill` takes, on encoded values as Photoshop does. A paint stroke builds up as a mask's does. In the editor, a layer's strokes settle into pixels once it has 100 and no gesture is open: `getState()` then shows a `raster` ID and fewer strokes, undo still reaches every stroke, and scene files keep the pixels. A photo holds up to 4 Paint layers and 10 brush masks; adding more throws.
@@ -183,7 +183,7 @@ The image renders at the document dimensions and downsamples to `longEdge` with
}
```
-`scene` is the `getState()` scene; each image layer's `source` names an entry in `sources`, and each Remove patch's `field` an entry in `fields`: where its texels sit, or, for a field not yet synthesized, `{ "base": "<id>" }` naming the field it extends. Version 1 files, which kept each Remove field inside its patch, open as version 2. Opening the file with `loadScene`, `openFile`, a drop, or the file picker decodes the stored source again and restores the frame and every layer as a new document with empty history. Preview settings and history are not saved.
+`scene` is the `getState()` scene; each image layer's `source` names an entry in `sources`, and each Remove patch's `field` an entry in `fields`: where its texels sit, or `{}` for a field not yet synthesized. Version 1 files, which kept each Remove field inside its patch, open as version 2. Opening the file with `loadScene`, `openFile`, a drop, or the file picker decodes the stored source again and restores the frame and every layer as a new document with empty history. Preview settings and history are not saved.
Opening validates every value as the matching command does, and a file that fails leaves the workspace in its error state with a message naming the first invalid field. Fields OpenLight does not know are dropped. A parameter missing from `adjustments`, `details`, `vignette`, `grain`, `fill`, or `colorMixer` takes its default, so older files still open when a group gains a parameter; a RAW image without a white balance uses its As Shot value, and a frame without `perspective` has none. `version` increases only when older files can no longer open as written, or an older OpenLight would drop what a file holds: a scene that corrects perspective is version 3, which OpenLight before perspective correction rejects, and every other scene stays version 2. A newer version is rejected.
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 7359479..7a247b9 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -87,7 +87,7 @@ Dependencies point downward. Features do not import each other; `app/` connects
- Strokes are scene content; the renderer caches their rasterized coverage, or a Paint layer's premultiplied 8-bit color, at half the source's resolution each way, a quarter of its pixels, which the passes that read them sample back to full size, and stamps only appended dabs. A Paint layer's or brush mask's raster comes with its first paint and stays, cleared if need be, until the layer goes, so painting and undoing allocate nothing. A photo holds up to 4 Paint layers and 10 brush masks.
- A mask's or Paint layer's latest stroke builds up in a half-float stroke buffer at the same resolution, one stroke at a time; its raster takes it in one pass when the next stroke starts, so it rounds to 8 bits once, however many moves drew it. Until then a Paint layer composes the buffer over its raster, and a mask's readers get its raster with the stroke laid over, in a view redrawn where the stroke grows. A Healing or Clone patch records its mode, reuses the same coverage, and keeps editable feather and opacity. Clone copies donor color directly, skipping the correction passes. A Healing patch records ordered hard paint/erase strokes, stamped through the shared GPU brush engine into one raster over only the 256 px tiles its paint reaches. Appending a gesture stamps only new dabs; undo or moved geometry replays the sequence. Shift adds and Alt/Option subtracts on the selected patch, preserving its donor and blend; separate strokes stay separate. The local SVG preview composes the same ordered operations and shows union boundaries and erased holes. Scene loading migrates earlier single-stroke patches once at the boundary.
- Remove patches store hard paint/erase strokes without a donor offset. `features/heal/inpaint` synthesizes a field of each hole texel's offset to its donor with a local, multiscale PatchMatch graph, entirely on the GPU: exclusion mask, preserved texture descriptors, nearest valid donor initialization, parallel propagation/random search, and overlap voting. Matching is bounded to 512 texels along the long side; the output samples donors at full source resolution. The Heal compositor applies the patch's coverage, feather, and opacity.
-- A Remove field is synthesized once per stroke version and kept. Each edit that changes a Remove patch's strokes reserves a new field in the document's resources, naming the field it extends, and the patch names it, so every scene with those strokes, in history, drafts, scene files, and exports, shows one fill however late it arrives; filling it never changes the scene or history. The renderer keeps fields by ID: it synthesizes a waiting one once, at full resolution, from the image below, extending the nearest field it names that it holds or the document saved, and reads it back for the document at once, rendering nothing meanwhile. Patches that share a field, such as duplicates, share its synthesis. The layers below, a proxy, hiding, undo, export, and a reopened scene all use that field and never solve again; a version never shown synthesizes when it first shows. Export, scene files, and drafts complete their snapshot with the fields the editor shows that the document still waits for, and fail when that readback fails. Composition samples donor color from the current image below, so the patch follows changes under it, such as exposure. A stroke added to or erased from the patch keeps the texels the earlier field filled, on the same lattice while it fits, as long as they stay in the hole and their donors stay clean, and synthesizes only the rest; moving the patch synthesizes it anew. Synthesis needs no React. See [the solver](src/features/heal/inpaint/algorithm.md) for its constraints and standalone benchmarks.
+- A Remove field is synthesized once per stroke version and kept. Each edit that changes a Remove patch's strokes, adding, erasing, or moving them, reserves a new field in the document's resources, and the patch names it, so every scene with those strokes, in history, drafts, scene files, and exports, shows one fill however late it arrives; filling it never changes the scene or history. The renderer keeps fields by ID: it synthesizes a waiting one once, at full resolution, for the patch's whole shape, from the image below, and reads it back for the document at once, rendering nothing meanwhile. Patches that share a field, such as duplicates, share its synthesis. The layers below, a proxy, hiding, undo, export, and a reopened scene all use that field and never solve again; a version never shown synthesizes when it first shows. Export, scene files, and drafts complete their snapshot with the fields the editor shows that the document still waits for, and fail when that readback fails. Composition samples donor color from the current image below, so the patch follows changes under it, such as exposure. A stroke added over part of an object the first stroke missed therefore leaves none of it, since the whole shape synthesizes from context without it. Synthesis needs no React. See [the solver](src/features/heal/inpaint/algorithm.md) for its constraints and standalone benchmarks.
- A brush mask paints coverage as a Paint layer paints color: both are a painting, the latest strokes over settled pixels, in one raster module (r8 for coverage, rgba8 for color), which also owns the open stroke coverage view. At 100 strokes, once no gesture is open, the renderer reads a painting's raster band by band through one mapped buffer, and the pixels, deflated, become a document resource the layer names in place of its strokes, without a history step. Rendering waits until both the scene and cached raster accept the settled pixels. History scenes name the pixels they drew over, so undo reaches past a settle: the renderer loads those pixels, inflating them band by band through a staging texture, and draws the strokes again. Resources go once no retained scene names them.
- A mask that only gradients shape needs no texture: the mix pass computes it, children included. A mask with a painted brush or a range combines in the render graph: each child's coverage, a brush's cached raster, a gradient drawn by a node, or a range read from the image below the mask, folds into the group's in stored order through one combine pass, and the mix pass shares its add, subtract, and intersect. The group takes the image's resolution when a range takes part, since a range follows the photo's edges, and the brushes' otherwise. The renderer gives it out, with each range's own coverage, for the overlay and thumbnails. While a color is picked, renders also keep the image below the mask group, which the picker reads through one GPU pass and 16 bytes of readback per pick.
- Open history groups render a reduced proxy. Every render image carries `scale`, its source pixels per texel; shaders that take document coordinates or radii apply it.
diff --git a/CONTEXT.md b/CONTEXT.md
index 3716d8a..9822c02 100644
--- a/CONTEXT.md
+++ b/CONTEXT.md
@@ -9,7 +9,7 @@ OpenLight edits documents containing image content and adjustments.
| **Layer** | A source, effect, or mask in the composition. Image layers own basic adjustments; effects process the image below; masks scope their basic adjustments and child effects. Linear, radial, brush, and range child masks add, subtract, or intersect coverage. A Paint layer is an effect that lays its strokes' colors over the image with a blend mode. Layers compose render nodes without owning textures. |
| **Stroke** | One brush drag: paint or erase, size, feather, flow, and points with pressure in source pixels; a Paint layer's strokes also carry a color. Strokes are content; the renderer rasterizes them into a cached texture at half the source's resolution, coverage for a mask and premultiplied color for a Paint layer, and the latest stroke into a shared stroke buffer until the next one starts. A Paint layer's or brush mask's strokes settle into pixels every 100 strokes, which it then paints over; undo still reaches every stroke. A Healing patch stores ordered hard paint/erase strokes, rasterized over only the tiles its paint reaches, and feathers its accumulated shape separately. |
| **Range** | A mask that selects pixels of the image below its mask group by lightness, a luminance range, or by color, a color range, rather than by position. A new color range has no coverage until it takes a color from a click on the photo. |
-| **Healing patch** | One painted, non-destructive repair in a Healing layer. Its Heal mode copies a nearby donor, found automatically or set by hand, with a bounded boundary color correction. Clone mode copies the donor without color correction. Remove mode synthesizes many local donor patches from surrounding texture and has no donor offset; it keeps where each pixel copies from as its field, so the layers below change only the copied colors, and a stroke added or erased keeps what it already filled. Patches replay in order and each sees the result of the preceding patches. Each gesture keeps its recorded size and separate path; Shift adds and Alt/Option subtracts on the selected patch. Feather and opacity remain editable on its accumulated shape. |
+| **Healing patch** | One painted, non-destructive repair in a Healing layer. Its Heal mode copies a nearby donor, found automatically or set by hand, with a bounded boundary color correction. Clone mode copies the donor without color correction. Remove mode synthesizes many local donor patches from surrounding texture and has no donor offset; it keeps where each pixel copies from as its field, so the layers below change only the copied colors, and a stroke added or erased synthesizes its whole new shape. Patches replay in order and each sees the result of the preceding patches. Each gesture keeps its recorded size and separate path; Shift adds and Alt/Option subtracts on the selected patch. Feather and opacity remain editable on its accumulated shape. |
| **Perspective correction** | Part of the frame: Vertical and Horizontal keystone that makes lines converging in the photo, such as a building's edges, parallel, before the crop, rotation, and flips. Layers keep their source pixels and follow it; tools draw and place them as it maps them. |
| **Image source** | An image used as input to the document's composition. It is content within a document. |
| **LUT** | A 3D color lookup table, such as a creative grade, read from a `.cube` file. A LUT layer holds one and applies it to the image below, at the layer's opacity. |
diff --git a/src/app/editor/export/export-image.ts b/src/app/editor/export/export-image.ts
index ddf5d62..2a0f5f3 100644
--- a/src/app/editor/export/export-image.ts
+++ b/src/app/editor/export/export-image.ts
@@ -3,7 +3,7 @@ import { createEditorRenderer } from "@/app/editor/renderer";
import {
completeFields,
type EditorDocument,
- fieldRecords,
+ sceneFields,
settledPixels,
} from "@/core/document";
import type { Point } from "@/core/image/frame";
@@ -86,7 +86,7 @@ function snapshotRender(document: EditorDocument) {
scene,
source: document.resources.get(scene.layers[0].source),
pixels: settledPixels(document, scene),
- fields: fieldRecords(document, scene),
+ fields: sceneFields(document, scene),
};
}
diff --git a/src/app/scene-file.ts b/src/app/scene-file.ts
index 9c89f98..9fa1727 100644
--- a/src/app/scene-file.ts
+++ b/src/app/scene-file.ts
@@ -4,12 +4,12 @@ import {
createDocument,
createResources,
type EditorDocument,
- type FieldRecord,
- fieldRecords,
type ProcessingLayer,
paintingOf,
+ type RemoveField,
removePatches,
type Scene,
+ sceneFields,
settledPixels,
walkLayers,
} from "@/core/document";
@@ -65,7 +65,10 @@ const savedHealPatch = z.pipe(
healPatchSchema,
);
-/** Where a saved Remove field's texels sit, or, while it waits to be synthesized, the field it extends. */
+/**
+ * Where a saved Remove field's texels sit, or nothing while it waits to be synthesized. Files from when
+ * a field extended an earlier one may name that `base`, which opening ignores.
+ */
type SavedField =
| {
readonly origin: readonly [number, number];
@@ -97,7 +100,7 @@ export function snapshotScene(document: EditorDocument) {
scene,
source: { id: source, file: document.resources.get(source).file },
paint: settledPixels(document, scene),
- fields: fieldRecords(document, scene),
+ fields: sceneFields(document, scene),
};
}
@@ -109,18 +112,17 @@ export type SceneSnapshot = ReturnType<typeof snapshotScene>;
*/
export async function completeScene(
document: EditorDocument,
- { scene, source, paint, fields: records }: SceneSnapshot,
+ { scene, source, paint, fields: taken }: SceneSnapshot,
) {
- const completed = await completeFields(document, records);
const fields: Record<string, SavedField> = {};
const texels = new Map<string, Blob>();
- for (const [id, record] of completed) {
- if (record.field) {
- const { texels: data, ...lattice } = record.field;
+ for (const [id, field] of await completeFields(document, taken)) {
+ if (field) {
+ const { texels: data, ...lattice } = field;
fields[id] = lattice;
texels.set(id, data);
} else {
- fields[id] = record.base ? { base: record.base } : {};
+ fields[id] = {};
}
}
const json: SceneJson = {
@@ -284,8 +286,8 @@ function* savedPatches(layers: unknown): Generator<Record<string, unknown>> {
/**
* A version 1 scene as version 2 writes it: each Remove patch named its field inline, with how many
- * strokes it covered, and now names it by ID beside the scene. A field for fewer strokes becomes the
- * base of a new one, and a patch without one reserves one.
+ * strokes it covered, and now names it by ID beside the scene. A patch whose field covered fewer of
+ * its strokes, or that had none, reserves one to synthesize its whole shape.
*/
function migrateFields(saved: unknown) {
if (!isRecord(saved) || saved.version !== 1 || !isRecord(saved.scene)) {
@@ -297,43 +299,30 @@ function migrateFields(saved: unknown) {
Reflect.get(Object(migrated.scene), "layers"),
)) {
if (patch.mode !== "remove") continue;
- const strokes = Array.isArray(patch.strokes) ? patch.strokes.length : 0;
+ // A patch from before patches held strokes held one.
+ const strokes = Array.isArray(patch.strokes) ? patch.strokes.length : 1;
const {
texels,
strokes: covered,
...lattice
} = isRecord(patch.field) ? patch.field : {};
- if (typeof texels !== "string") {
+ if (typeof texels === "string" && covered === strokes) {
+ fields[texels] = lattice;
+ patch.field = texels;
+ } else {
const reserved = crypto.randomUUID();
- patch.field = reserved;
fields[reserved] = {};
- continue;
- }
- fields[texels] = lattice;
- if (typeof covered === "number" && covered < strokes) {
- const extending = crypto.randomUUID();
- patch.field = extending;
- fields[extending] = { base: texels };
- } else {
- patch.field = texels;
+ patch.field = reserved;
}
}
return { ...migrated, fields };
}
-/** Whether every field a patch names, and every field one extends, is saved, with no loop of bases. */
+/** Whether every field a patch names is saved. */
function fieldsComplete(scene: Scene, fields: SceneJson["fields"]) {
- for (const { patch } of removePatches(scene.layers)) {
- const seen = new Set<string>();
- let id: string | undefined = patch.field;
- while (id) {
- const field: SavedField | undefined = fields[id];
- if (!field || seen.has(id)) return false;
- seen.add(id);
- id = "base" in field ? field.base : undefined;
- }
- }
- return true;
+ return [...removePatches(scene.layers)].every(({ patch }) =>
+ Object.hasOwn(fields, patch.field),
+ );
}
/**
@@ -358,17 +347,17 @@ export async function openScene(
if (!fieldsComplete(scene, fields)) {
throw Error("The scene's Remove fields are missing.");
}
- const records = new Map<string, FieldRecord>();
+ const stored = new Map<string, RemoveField | undefined>();
for (const [id, field] of Object.entries(fields)) {
if (!("origin" in field)) {
- records.set(id, field);
+ stored.set(id, undefined);
continue;
}
const texels = files.get(id);
if (!texels) {
throw Error("The scene's Remove fields are missing.");
}
- records.set(id, { field: { texels, ...field } });
+ stored.set(id, { texels, ...field });
}
const [image, ...layers] = scene.layers;
const source = sources[image.source];
@@ -401,8 +390,8 @@ export async function openScene(
resources.addPaint(pixels, raster);
}
}
- for (const [id, record] of records) {
- resources.addField(id, record);
+ for (const [id, field] of stored) {
+ resources.addField(id, field);
}
return createDocument(
{
diff --git a/src/components/editor/opacity-input.tsx b/src/components/editor/opacity-input.tsx
new file mode 100644
index 0000000..1afe45b
--- /dev/null
+++ b/src/components/editor/opacity-input.tsx
@@ -0,0 +1,31 @@
+import { ScrubInput } from "@roprgm/ui/scrub-input";
+import { useEditGesture } from "@/hooks/use-edit-gesture";
+import { useDocument } from "./session";
+
+/** An opacity from 0 to 1 as a percentage that drags sideways as one edit and types on click. */
+export function OpacityInput({
+ label,
+ value,
+ onChange,
+}: {
+ label: string;
+ value: number;
+ onChange: (opacity: number) => void;
+}) {
+ const document = useDocument();
+ const gesture = useEditGesture(document.history);
+ return (
+ <span {...gesture} className="flex">
+ <ScrubInput
+ aria-label={label}
+ value={value * 100}
+ min={0}
+ max={100}
+ defaultValue={100}
+ format={(percent) => `${percent}%`}
+ minChars={3}
+ onChange={(percent) => onChange(percent / 100)}
+ />
+ </span>
+ );
+}
diff --git a/src/core/document/index.ts b/src/core/document/index.ts
index b9bb4d6..04eb0b3 100644
--- a/src/core/document/index.ts
+++ b/src/core/document/index.ts
@@ -1,15 +1,11 @@
import { createStore } from "zustand/vanilla";
import { validateFrame } from "@/core/image/frame";
import { createHistory } from "./history";
-import {
- createResources,
- type FieldRecord,
- type RemoveField,
-} from "./resources";
+import { createResources, type RemoveField } from "./resources";
import type { Mask, MaskModifier, Scene } from "./scene";
import { findLayer, paintingOf, removePatches, walkLayers } from "./tree";
-export type { FieldRecord, RemoveField } from "./resources";
+export type { RemoveField } from "./resources";
export type {
Adjustments,
Blend,
@@ -102,24 +98,11 @@ function equal(a: unknown, b: unknown): boolean {
);
}
-type Resources = ReturnType<typeof createResources>;
-
-/** A Remove field and, while it waits to be synthesized, the fields it extends, up to a synthesized one. */
-function fieldChain(resources: Resources, id: string) {
- const ids = [id];
- let record = resources.field(id);
- while (record && !record.field && record.base) {
- ids.push(record.base);
- record = resources.field(record.base);
- }
- return ids;
-}
-
/**
* The resources a scene names: its image source, the pixels its paintings settled into, and its
- * Remove fields with the fields they extend.
+ * Remove fields.
*/
-function resourceIds(scene: Scene, resources: Resources) {
+function resourceIds(scene: Scene) {
const ids = [scene.layers[0].source];
for (const { layer } of walkLayers(scene.layers)) {
const raster = paintingOf(layer)?.raster;
@@ -128,7 +111,7 @@ function resourceIds(scene: Scene, resources: Resources) {
}
}
for (const { patch } of removePatches(scene.layers)) {
- ids.push(...fieldChain(resources, patch.field));
+ ids.push(patch.field);
}
return ids;
}
@@ -148,37 +131,30 @@ export function settledPixels(
return pixels;
}
-/** The Remove fields `scene` names, with the fields those still waiting extend, taken now like `settledPixels`. */
-export function fieldRecords(
+/** The Remove fields `scene` names, undefined while one waits to be synthesized, taken now like `settledPixels`. */
+export function sceneFields(
document: EditorDocument,
scene = document.scene.getState(),
) {
- const records = new Map<string, FieldRecord>();
+ const fields = new Map<string, RemoveField | undefined>();
for (const { patch } of removePatches(scene.layers)) {
- for (const id of fieldChain(document.resources, patch.field)) {
- records.set(id, document.resources.field(id) ?? {});
- }
+ fields.set(patch.field, document.resources.field(patch.field));
}
- return records;
+ return fields;
}
/**
- * `records` taken for a snapshot, with the fields the editor shows but they still waited for read back
- * since; rejects when a readback fails.
+ * `fields` taken for a snapshot, with those that waited and that the editor shows read back since;
+ * rejects when a readback fails.
*/
export async function completeFields(
document: EditorDocument,
- records: ReadonlyMap<string, FieldRecord>,
+ fields: ReadonlyMap<string, RemoveField | undefined>,
) {
- const waiting = [...records]
- .filter(([, record]) => !record.field)
- .map(([id]) => id);
+ const waiting = [...fields].filter(([, field]) => !field).map(([id]) => id);
const captured = await document.captureFields(waiting);
return new Map(
- [...records].map(([id, record]): [string, FieldRecord] => {
- const field = record.field ?? captured.get(id);
- return [id, field ? { field } : record];
- }),
+ [...fields].map(([id, field]) => [id, field ?? captured.get(id)] as const),
);
}
@@ -196,9 +172,7 @@ export function createDocument(initial: Scene, resources = createResources()) {
equal,
100,
(retained) => {
- resources.retain(
- new Set(retained.flatMap((scene) => resourceIds(scene, resources))),
- );
+ resources.retain(new Set(retained.flatMap(resourceIds)));
},
);
const unsubscribe = scene.subscribe((state) => {
diff --git a/src/core/document/resources.ts b/src/core/document/resources.ts
index fd6efbe..2a2a9d1 100644
--- a/src/core/document/resources.ts
+++ b/src/core/document/resources.ts
@@ -16,12 +16,6 @@ export type RemoveField = {
readonly size: Point;
};
-/** A Remove field resource: its field once synthesized, or until then the field its synthesis extends. */
-export type FieldRecord = {
- readonly field?: RemoveField;
- readonly base?: string;
-};
-
/**
* Owns the document's image files and GPU targets, the deflated pixels paint layers settled into, and
* Remove fields, outside scene history.
@@ -29,7 +23,8 @@ export type FieldRecord = {
export function createResources() {
const images = new Map<string, { file: File } & ImageSource>();
const paints = new Map<string, Blob>();
- const fields = new Map<string, FieldRecord>();
+ /** Each Remove field by ID, undefined while it waits to be synthesized. */
+ const fields = new Map<string, RemoveField | undefined>();
let disposed = false;
function open() {
if (disposed) {
@@ -66,22 +61,22 @@ export function createResources() {
}
return pixels;
},
- /** Reserves a Remove field for a new set of strokes, which extends `base` once synthesized. */
- reserveField(base?: string) {
+ /** Reserves a Remove field for a new set of strokes, to be synthesized. */
+ reserveField() {
open();
const id = crypto.randomUUID();
- fields.set(id, base ? { base } : {});
+ fields.set(id, undefined);
return id;
},
- /** A saved document restores Remove fields under their original IDs. */
- addField(id: string, record: FieldRecord) {
+ /** A saved document restores Remove fields under their original IDs, synthesized or not. */
+ addField(id: string, field?: RemoveField) {
open();
- fields.set(id, record);
+ fields.set(id, field);
},
/** Keeps the field synthesized for a reserved ID; the first one stays, and a closed document drops it. */
fillField(id: string, field: RemoveField) {
- if (!disposed && fields.has(id) && !fields.get(id)?.field) {
- fields.set(id, { field });
+ if (!disposed && fields.has(id) && !fields.get(id)) {
+ fields.set(id, field);
}
},
field(id: string) {
diff --git a/src/core/renderer/index.ts b/src/core/renderer/index.ts
index 2815ef7..701b875 100644
--- a/src/core/renderer/index.ts
+++ b/src/core/renderer/index.ts
@@ -1,7 +1,6 @@
import type { Gpu, Target, Timer } from "vgpu";
import {
type BrushStroke,
- type FieldRecord,
hasPaint,
type MaskLayer,
type PaintLayer,
@@ -72,7 +71,7 @@ export type Composition = {
paint: (layer: PaintLayer) => PaintInput | undefined;
/** Rasterized coverage of an effect's own stroke, such as a Healing patch. */
patch: (id: string, strokes: readonly BrushStroke[]) => PatchInput;
- /** The Remove field a patch shows, or else the nearest one the renderer holds that its synthesis extends. */
+ /** The Remove field a patch shows, when the renderer holds it. */
field: (patch: RemovePatch) => PatchField | undefined;
/** Keeps a Remove field this render synthesizes for a patch, for the renders after it. */
keepField: (
@@ -82,12 +81,8 @@ export type Composition = {
) => void;
};
-/** A Remove field a render shows, `complete` when it is the patch's own rather than one it extends. */
-export type PatchField = {
- texels: RenderImage;
- lattice: FieldLattice;
- complete: boolean;
-};
+/** A Remove field a render shows. */
+export type PatchField = { texels: RenderImage; lattice: FieldLattice };
/** App composition describes requested outputs; the engine owns their storage. */
export type SceneProcessing = (
@@ -119,8 +114,8 @@ export type RendererOptions = {
timer?: Timer;
/** The settled pixels a paint layer's or brush mask's `raster` names. */
paintPixels?: (id: string) => Blob;
- /** A Remove field resource: its field once synthesized, or else the field its synthesis extends. */
- field?: (id: string) => FieldRecord | undefined;
+ /** The Remove field the document saved under an ID, once synthesized. */
+ field?: (id: string) => RemoveField | undefined;
/** Receives each Remove field the renderer synthesizes, read back; without it nothing is read back. */
saveField?: (id: string, field: RemoveField) => void;
};
@@ -138,7 +133,7 @@ export function createRenderer(
paintPixels = () => {
throw Error("This renderer has no settled paint.");
},
- field: fieldRecord = () => undefined,
+ field: savedField = () => undefined,
saveField,
}: RendererOptions = {},
) {
@@ -197,12 +192,9 @@ export function createRenderer(
paints.draw(layer.id, layer, developed.size);
}
}
- // Remove fields stay while their patches do, shown or not, and so do the fields they extend.
+ // Remove fields stay while their patches do, shown or not.
for (const { patch } of removePatches(scene.layers)) {
- const held = heldField(patch.field);
- if (held) {
- fields.retain(held);
- }
+ fields.retain(patch.field);
}
const image =
factor > 1 ? proxy.render(developed, factor, version) : input(developed);
@@ -234,17 +226,10 @@ export function createRenderer(
}),
paint: (layer) => paints.input(layer),
patch: (id, strokes) => patches.patch(id, strokes, developed.size),
- field: (patch) => {
- // Patches that share a field, such as duplicates, show the one this render synthesizes.
- const synthesized = syntheses.find(({ id }) => id === patch.field);
- if (synthesized) {
- const { texels, lattice } = synthesized;
- return { texels, lattice, complete: true };
- }
- const id = heldField(patch.field);
- const held = id && fields.get(id);
- return held ? { ...held, complete: id === patch.field } : undefined;
- },
+ // Patches that share a field, such as duplicates, show the one this render synthesizes.
+ field: (patch) =>
+ syntheses.find(({ id }) => id === patch.field) ??
+ fields.get(patch.field),
keepField: (patch, lattice, texels) =>
syntheses.push({ id: patch.field, lattice, texels }),
});
@@ -301,35 +286,13 @@ export function createRenderer(
function paintRaster(id: string) {
return paints.get(id) ? paints : brushes;
}
- /** A Remove field's ID, then, while each waits to be synthesized, the IDs of the fields it extends. */
- function* lineage(id: string) {
- let next: string | undefined = id;
- while (next) {
- yield next;
- const record = fieldRecord(next);
- next = record?.field ? undefined : record?.base;
- }
- }
- /** The ID of the field held for a patch's own field: that field, or the nearest one it extends. */
- function heldField(id: string) {
- for (const ancestor of lineage(id)) {
- if (fields.get(ancestor)) {
- return ancestor;
- }
- }
- return undefined;
- }
- /** Remove fields to load before rendering: for a patch with none held, the saved one nearest its own. */
+ /** Remove fields to load before rendering: saved ones the renderer holds nothing for. */
function staleFields(scene: Scene) {
const stale = new Map<string, RemoveField>();
for (const { patch } of removePatches(scene.layers)) {
- if (heldField(patch.field)) {
- continue;
- }
- const nearest = [...lineage(patch.field)].at(-1) ?? patch.field;
- const saved = fieldRecord(nearest)?.field;
- if (saved) {
- stale.set(nearest, saved);
+ const saved = savedField(patch.field);
+ if (saved && !fields.get(patch.field)) {
+ stale.set(patch.field, saved);
}
}
return [...stale];
@@ -549,12 +512,12 @@ export function createRenderer(
return read ? [[id, read] as const] : [];
});
const unread = ids.some(
- (id) => !capturing.has(id) && fields.get(id) && !fieldRecord(id)?.field,
+ (id) => !capturing.has(id) && fields.get(id) && !savedField(id),
);
const found = await hold(async () => {
const found = new Map<string, RemoveField>();
for (const id of ids) {
- const saved = fieldRecord(id)?.field;
+ const saved = savedField(id);
if (saved) {
found.set(id, saved);
continue;
diff --git a/src/features/heal/anchor.tsx b/src/features/heal/anchor.tsx
index 43c2c8b..c70442d 100644
--- a/src/features/heal/anchor.tsx
+++ b/src/features/heal/anchor.tsx
@@ -25,6 +25,9 @@ export type AnchorDrag = {
const marker = { r: 7, fill: "#3b82f6", stroke: "white", strokeWidth: 2 };
+/** Viewport pixels from an anchor's center that grab or select it. */
+export const anchorReach = 14;
+
/** The first-point anchor of a destination or source contour, draggable while its patch is selected. */
export function HealAnchor({
kind,
@@ -163,7 +166,7 @@ function DraggedAnchor({
data-hide-brush-cursor="true"
cx={center[0]}
cy={center[1]}
- r={14}
+ r={anchorReach}
fill="transparent"
className="pointer-events-auto cursor-grab active:cursor-grabbing"
onPointerDown={start}
diff --git a/src/features/heal/controls.tsx b/src/features/heal/controls.tsx
index 4fe5772..1902528 100644
--- a/src/features/heal/controls.tsx
+++ b/src/features/heal/controls.tsx
@@ -2,10 +2,11 @@ import { IconButton } from "@roprgm/ui/icon-button";
import { ListItem, ListItemAction } from "@roprgm/ui/list-item";
import { Menu, MenuContent, MenuItem, MenuTrigger } from "@roprgm/ui/menu";
import { CoverageThumbnail } from "@/components/editor/coverage-thumbnail";
+import { OpacityInput } from "@/components/editor/opacity-input";
import { useDocument } from "@/components/editor/session";
import { Icon } from "@/components/icons/icon";
import type { HealPatch } from "@/core/document";
-import { deleteHealPatch, duplicateHealPatch } from "./edits";
+import { deleteHealPatch, duplicateHealPatch, setHealPatch } from "./edits";
import { useHealing } from "./mode";
import { PatchModeIcon } from "./mode-buttons";
import { patchThumbnailRegion } from "./model";
@@ -74,14 +75,6 @@ function PatchActions({
);
}
-/** The brush size, and the opacity once lowered; the donor is edited on the canvas. */
-function patchSummary(patch: HealPatch) {
- const size = `${Math.round(patch.strokes[0].size)} px`;
- return patch.opacity < 1
- ? `${size} · ${Math.round(patch.opacity * 100)}%`
- : size;
-}
-
/** Shows the ordered repair stack; selection is shared with the canvas and floating bar. */
export function HealControls({
id,
@@ -134,11 +127,15 @@ export function HealControls({
mode={patch.mode}
className="size-3.5 shrink-0 text-secondary"
/>
- <span className="shrink-0 text-secondary tabular-nums">
- {patchSummary(patch)}
- </span>
</button>
<ListItemAction>
+ <OpacityInput
+ label={`Patch ${index + 1} opacity`}
+ value={patch.opacity}
+ onChange={(opacity) =>
+ setHealPatch(document, id, patch.id, { opacity })
+ }
+ />
<PatchActions
layer={id}
patch={patch}
diff --git a/src/features/heal/edits.ts b/src/features/heal/edits.ts
index 2677cda..47275d6 100644
--- a/src/features/heal/edits.ts
+++ b/src/features/heal/edits.ts
@@ -61,7 +61,7 @@ export function addRemovePatch(
return patch.id;
}
-/** A patch with new strokes; a Remove patch takes a new field, which extends its earlier one. */
+/** A patch with new strokes; a Remove patch takes a new field, synthesized for its whole new shape. */
function withStrokes(
document: EditorDocument,
patch: HealPatch,
@@ -70,8 +70,7 @@ function withStrokes(
if (patch.mode !== "remove") {
return { ...patch, strokes };
}
- const field = document.resources.reserveField(patch.field);
- return { ...patch, strokes, field };
+ return { ...patch, strokes, field: document.resources.reserveField() };
}
/** Rewrites the patch list around one patch, found by id. */
@@ -90,8 +89,8 @@ function editPatches(
}
/**
- * Adds or subtracts a separate stroke without changing the patch's donor or blend. A Remove patch takes
- * a new field, which extends its earlier one.
+ * Adds or subtracts a separate stroke without changing the patch's donor or blend. A Remove patch
+ * synthesizes its whole new shape.
*/
export function addHealStroke(
document: EditorDocument,
@@ -209,9 +208,7 @@ export function setHealDestination(
),
}));
if (patch.mode === "remove") {
- // A patch in a new place keeps nothing it filled.
- const field = document.resources.reserveField();
- return patches.with(index, { ...patch, strokes, field });
+ return patches.with(index, withStrokes(document, patch, strokes));
}
return patches.with(index, {
...patch,
diff --git a/src/features/heal/inpaint/algorithm.md b/src/features/heal/inpaint/algorithm.md
index e8c0008..9935906 100644
--- a/src/features/heal/inpaint/algorithm.md
+++ b/src/features/heal/inpaint/algorithm.md
@@ -10,7 +10,7 @@ The algorithm follows non-local, multiscale texture synthesis with PatchMatch, a
4. Alternate parallel PatchMatch propagation at jumps 8, 4, 2, 1 and eight shrinking-radius random proposals with weighted overlapping reconstruction. Use six iterations at the coarsest level, four at finer levels. Compare nine samples across each 7×7 footprint plus texture descriptors; increase reconstructed-region confidence as refinement proceeds.
5. Keep each hole texel's offset. Sample donors at whole source texels from the image below for the final image rather than exporting the averaged matching pyramid, so the patch follows later changes to that image without solving again. Different pixels can use different offsets. Apply the existing inward feather, patch opacity, and original alpha in a separate blend.
-A stroke added to or erased from a patch extends its field: the new field keeps the earlier one's texels while they still fit 512 along the long side, and at every level pins each texel the earlier field filled to its donor, as long as the texel stays in the hole and the donor stays out of it. Pinned texels count as known context, so the new texels match them, and only the rest are searched. Past that size the lattice coarsens, and the earlier offsets land on the nearest texels.
+A stroke added to or erased from a patch synthesizes its whole new shape, so what an earlier stroke missed of an object no longer shapes the fill.
Known, wholly clean patches skip search. Candidate comparison stops once its nonnegative partial error cannot beat the current score. Each search pass owns its uniforms; vgpu shares shader modules and pipelines, and the graph reuses temporary textures across iterations. Reconstruction passes share an effect because they have no uniforms. Random proposals are deterministic for the same inputs; since the inputs include the image below, a change there would find other donors, which is why the editor keeps the field instead of solving again.
@@ -22,7 +22,7 @@ The user should paint over the entire object, including its halo or shadow. Pixe
## Verification and measurement
-`tests/inpaint-gpu.ts` creates known flat, gradient, periodic, edge, and brick backgrounds, allowing error measurement against a clean reference. Browser tests check actual removal, HDR/alpha, untouched pixels, proxy rendering, a kept field that follows the exposure below without solving again, extensions that keep what was filled, release, unavailable donors, and the H/paint/export/undo workflow. Scene files round-trip Remove patches with their fields and without offsets. `tests/inpaint-photo.ts` runs decoding, removal, and export on the demo photograph without mounting React.
+`tests/inpaint-gpu.ts` creates known flat, gradient, periodic, edge, and brick backgrounds, allowing error measurement against a clean reference. Browser tests check actual removal, HDR/alpha, untouched pixels, proxy rendering, a kept field that follows the exposure below without solving again, a stroke added over a missed tip that fills as painting the whole shape at once does, release, unavailable donors, and the H/paint/export/undo workflow. Scene files round-trip Remove patches with their fields and without offsets. `tests/inpaint-photo.ts` runs decoding, removal, and export on the demo photograph without mounting React.
See [PERFORMANCE.md](../../../../PERFORMANCE.md#object-removal) for standalone and integrated benchmarks, including an unchanged-patch cache workload. Report the actual adapter; software-backend timings do not predict physical-GPU latency.
diff --git a/src/features/heal/inpaint/index.ts b/src/features/heal/inpaint/index.ts
index 12c9a87..b145e35 100644
--- a/src/features/heal/inpaint/index.ts
+++ b/src/features/heal/inpaint/index.ts
@@ -13,14 +13,11 @@ import finishShader from "./finish.wgsl";
import matchShader from "./match.wgsl";
import nearestShader from "./nearest.wgsl";
import offsetsShader from "./offsets.wgsl";
-import pinShader from "./pin.wgsl";
import prepareShader from "./prepare.wgsl";
import pyramidShader from "./pyramid.wgsl";
import reconstructShader from "./reconstruct.wgsl";
export type InpaintRegion = { origin: Point; extent: Point };
-/** An earlier field whose filled texels a new synthesis keeps. */
-export type InpaintBase = { texels: RenderImage; lattice: FieldLattice };
/** Texels along a field's longer side at most; a larger region takes larger texels. */
export const fieldLongSide = 512;
const samplers = {
@@ -65,38 +62,11 @@ function reconstruct(
);
}
-/** A base field to pin in one level, whose texels span `scale` source pixels from `origin`. */
-type Pinning = { base: InpaintBase; origin: Point; scale: number };
-
-/** Pins the texels the base filled to their donors. */
-function pin(
- source: RenderImage,
- field: RenderImage,
- { base, origin, scale }: Pinning,
- name: string,
-) {
- return merge(
- { source, field, base: base.texels },
- node(name, pinShader, {
- format: "rgba32float",
- set: {
- params: {
- origin,
- scale,
- baseOrigin: base.lattice.origin,
- baseScale: base.lattice.scale,
- },
- },
- }),
- );
-}
-
function solve(
source: RenderImage,
features: RenderImage,
coarse: RenderImage | undefined,
name: string,
- pinning?: Pinning,
) {
const closest = nearest(features, name);
const params = {
@@ -121,7 +91,6 @@ function solve(
set: { params },
}),
);
- if (pinning) field = pin(source, field, pinning, `${name}/pin`);
let current = reconstruct(
source,
field,
@@ -184,32 +153,8 @@ function solve(
return field;
}
-/**
- * The texels a field over `bounds` takes. An extension keeps its base's texels while they fit, so the
- * texels the base filled keep their donors exactly.
- */
-export function fieldLattice(
- bounds: InpaintRegion,
- base?: FieldLattice,
-): FieldLattice {
- const end = [
- bounds.origin[0] + bounds.extent[0],
- bounds.origin[1] + bounds.extent[1],
- ] as const;
- if (base) {
- const { scale } = base;
- const origin: Point = [
- base.origin[0] +
- Math.floor((bounds.origin[0] - base.origin[0]) / scale) * scale,
- base.origin[1] +
- Math.floor((bounds.origin[1] - base.origin[1]) / scale) * scale,
- ];
- const size: Point = [
- Math.ceil((end[0] - origin[0]) / scale),
- Math.ceil((end[1] - origin[1]) / scale),
- ];
- if (Math.max(...size) <= fieldLongSide) return { origin, scale, size };
- }
+/** The texels a field over `bounds` takes. */
+export function fieldLattice(bounds: InpaintRegion): FieldLattice {
const scale = Math.max(1, Math.max(...bounds.extent) / fieldLongSide);
return {
origin: bounds.origin,
@@ -223,15 +168,13 @@ export function fieldLattice(
/**
* Multiscale, non-local synthesis of each hole texel's offset to its donor, in texels, at full
- * resolution; texels a `base` filled keep their donors. Every pixel operation stays on the GPU; no
- * React or readback.
+ * resolution. Every pixel operation stays on the GPU; no React or readback.
*/
export function inpaintField(
source: RenderImage,
patch: PatchInput,
lattice: FieldLattice,
name: string,
- base?: InpaintBase,
): RenderNode {
const params = {
origin: lattice.origin,
@@ -275,14 +218,11 @@ export function inpaintField(
}
let field: RenderImage | undefined;
for (let level = pyramid.length - 1; level >= 0; level--) {
- // Each level halves the one below, so its texels span twice the source pixels.
- const scale = lattice.scale * 2 ** level;
field = solve(
pyramid[level],
descriptors[level],
field,
`${name}/level/${level}`,
- base && { base, origin: lattice.origin, scale },
);
}
if (!field) throw Error("An inpainting pyramid is required.");
diff --git a/src/features/heal/inpaint/match.wgsl b/src/features/heal/inpaint/match.wgsl
index 230441f..ee8ddc3 100644
--- a/src/features/heal/inpaint/match.wgsl
+++ b/src/features/heal/inpaint/match.wgsl
@@ -36,9 +36,7 @@ fn distance(p: vec2i, q: vec2i, limit: f32) -> f32 {
let a = clamp(p + vec2i(x, y), vec2i(0), end);
let b = q + vec2i(x, y);
let masked = textureLoad(source, a, 0).a;
- let correspondence = textureLoad(field, a, 0);
- let pinned = params.initialize == 0u && correspondence.z < 0.0;
- let reconstructed = select(0.0, select(params.confidence, 1.0, pinned), correspondence.w > 0.5);
+ let reconstructed = select(0.0, params.confidence, textureLoad(field, a, 0).w > 0.5);
let confidence = select(1.0, reconstructed, masked > 0.5);
let destination = textureLoad(current, a, 0).rgb;
let donor = textureLoad(source, b, 0).rgb;
@@ -62,10 +60,6 @@ fn consider(p: vec2i, offset: vec2i, best: vec4f) -> vec4f {
let p = vec2i(position.xy);
let size = vec2i(textureDimensions(source));
if (textureLoad(features, p, 0).a > 0.5) { return vec4f(0.0, 0.0, 0.0, 1.0); }
- if (params.initialize == 0u) {
- let pinned = textureLoad(field, p, 0);
- if (pinned.z < 0.0) { return pinned; }
- }
if (textureLoad(source, p, 0).a > 0.5 && sqrt(textureLoad(nearest, p, 0).z) > params.radius) { return vec4f(0.0, 0.0, 1e10, 0.0); }
var best = vec4f(0.0, 0.0, 1e10, 0.0);
if (params.initialize != 0u) {
diff --git a/src/features/heal/inpaint/pass.ts b/src/features/heal/inpaint/pass.ts
index 902f299..f163952 100644
--- a/src/features/heal/inpaint/pass.ts
+++ b/src/features/heal/inpaint/pass.ts
@@ -20,9 +20,8 @@ import shader from "./blend.wgsl";
type FieldComposition = Pick<Composition, "field" | "keepField">;
/**
- * The field a patch shows: its own, or else one synthesized from the image below at full resolution,
- * which keeps what the field it extends filled. A proxy shows that field meanwhile, since its reduced
- * image would synthesize another.
+ * The field a patch shows: its own, or else one synthesized for its whole shape from the image below at
+ * full resolution. A proxy shows none until then, since its reduced image would synthesize another.
*/
function patchField(
source: RenderImage,
@@ -33,16 +32,9 @@ function patchField(
composition: FieldComposition,
): { texels: RenderImage; lattice: FieldLattice } | undefined {
const held = composition.field(patch);
- if (held?.complete) return held;
- if (source.scale.some((scale) => scale > 1)) return held;
- const lattice = fieldLattice(region, held?.lattice);
- const texels = inpaintField(
- source,
- coverage,
- lattice,
- `${name}/inpaint`,
- held,
- );
+ if (held || source.scale.some((scale) => scale > 1)) return held;
+ const lattice = fieldLattice(region);
+ const texels = inpaintField(source, coverage, lattice, `${name}/inpaint`);
composition.keepField(patch, lattice, texels);
return { texels, lattice };
}
diff --git a/src/features/heal/inpaint/pin.wgsl b/src/features/heal/inpaint/pin.wgsl
deleted file mode 100644
index 9a11f05..0000000
--- a/src/features/heal/inpaint/pin.wgsl
+++ /dev/null
@@ -1,20 +0,0 @@
-// A texel an earlier field filled keeps its donor while it stays in the hole and its donor stays out of it.
-struct Params { origin: vec2f, baseOrigin: vec2f, scale: f32, baseScale: f32 }
-@group(0) @binding(0) var source: texture_2d<f32>;
-@group(0) @binding(1) var field: texture_2d<f32>;
-@group(0) @binding(2) var base: texture_2d<f32>;
-@group(0) @binding(3) var<uniform> params: Params;
-@fragment fn fs_main(@builtin(position) position: vec4f) -> @location(0) vec4f {
- let p = vec2i(position.xy);
- let current = textureLoad(field, p, 0);
- if (textureLoad(source, p, 0).a < 0.5) { return current; }
- let cell = vec2i(floor((params.origin + position.xy * params.scale - params.baseOrigin) / params.baseScale));
- if (any(cell < vec2i(0)) || any(cell >= vec2i(textureDimensions(base)))) { return current; }
- let offset = textureLoad(base, cell, 0).xy;
- if (all(offset == vec2f(0.0))) { return current; }
- let donor = p + vec2i(round(offset * params.baseScale / params.scale));
- let end = vec2i(textureDimensions(source)) - 1;
- if (any(donor < vec2i(0)) || any(donor > end) || textureLoad(source, donor, 0).a > 0.5) { return current; }
- // A negative cost pins the texel for the passes after this one.
- return vec4f(vec2f(donor - p), -1.0, 1.0);
-}
diff --git a/src/features/heal/inpaint/reconstruct.wgsl b/src/features/heal/inpaint/reconstruct.wgsl
index 68b19fb..7486edf 100644
--- a/src/features/heal/inpaint/reconstruct.wgsl
+++ b/src/features/heal/inpaint/reconstruct.wgsl
@@ -15,8 +15,7 @@
let donor = clamp(p + vec2i(correspondence.xy), vec2i(0), end);
let sample = textureLoad(source, donor, 0);
if (sample.a > 0.5) { continue; }
- // A pinned texel's negative cost counts as a perfect match.
- let confidence = exp(-clamp(correspondence.z * 80.0, 0.0, 20.0));
+ let confidence = exp(-min(correspondence.z * 80.0, 20.0));
color += sample.rgb * confidence;
weight += confidence;
}
diff --git a/src/features/heal/model.ts b/src/features/heal/model.ts
index 0cfdc0e..9d9e773 100644
--- a/src/features/heal/model.ts
+++ b/src/features/heal/model.ts
@@ -30,36 +30,6 @@ export function dabTouchesImage(
);
}
-/** Tests ordered coverage, so an erased hole is available for a new patch. */
-export function patchContains(
- strokes: readonly BrushStroke[],
- at: Point,
- minimumRadius = 0,
-) {
- let covered = false;
- for (const stroke of strokes) {
- const radius =
- stroke.mode === "paint"
- ? Math.max(minimumRadius, stroke.size / 2)
- : stroke.size / 2;
- const touches = stroke.points.some(([x, y], index) => {
- const [ax, ay] = stroke.points[Math.max(0, index - 1)];
- const dx = x - ax;
- const dy = y - ay;
- const length = dx * dx + dy * dy;
- const t = length
- ? Math.max(
- 0,
- Math.min(1, ((at[0] - ax) * dx + (at[1] - ay) * dy) / length),
- )
- : 0;
- return Math.hypot(at[0] - ax - t * dx, at[1] - ay - t * dy) <= radius;
- });
- if (touches) covered = stroke.mode === "paint";
- }
- return covered;
-}
-
/** Bounds include the whole soft brush edge, with room for a known boundary. */
export function patchBounds(
strokes: readonly BrushStroke[],
diff --git a/src/features/heal/overlay.tsx b/src/features/heal/overlay.tsx
index 3e801e5..83b0799 100644
--- a/src/features/heal/overlay.tsx
+++ b/src/features/heal/overlay.tsx
@@ -6,10 +6,16 @@ import { useRenderer } from "@/components/editor/pipeline";
import { useDocument, useSelectedLayer } from "@/components/editor/session";
import { useToolLayer } from "@/components/editor/tool-layer";
import { useViewport } from "@/components/editor/viewport";
-import { type BrushStroke, findLayer, type Layer } from "@/core/document";
+import {
+ type BrushStroke,
+ findLayer,
+ type HealPatch,
+ type Layer,
+} from "@/core/document";
import type { Point } from "@/core/image/frame";
import { useDisposable } from "@/hooks/use-disposable";
import { containsTarget } from "@/lib/dom";
+import { anchorReach, HealAnchor } from "./anchor";
import {
addHealPatch,
addHealStroke,
@@ -18,7 +24,7 @@ import {
setHealSource,
} from "./edits";
import { useHealing } from "./mode";
-import { dabTouchesImage, findHealPatch, patchContains } from "./model";
+import { dabTouchesImage, findHealPatch } from "./model";
import { HealPatchOutline, HealStrokePreview } from "./outline";
import { createHealSearch } from "./source";
@@ -67,6 +73,11 @@ export function HealOverlay({
? hoveredPatch
: undefined;
const visiblePatch = drawingPatch ?? hovered ?? selectedPatch;
+ /** The patch whose handles show: the selected one, while its outline shows and no stroke draws on it. */
+ const handled =
+ selected && visiblePatch === selected.id && drawingPatch !== selected.id
+ ? selected
+ : undefined;
const pending = useRef<
| { layer: string; stroke: BrushStroke; patch?: string }
| {
@@ -112,9 +123,16 @@ export function HealOverlay({
}
}
const marker = source && mapping.toScreen(source);
+ /** Where a patch's first point shows, moved by `offset` for its source; none behind the horizon. */
+ function firstShown(patch: HealPatch, offset: Point = [0, 0]) {
+ const [x, y] = patch.strokes[0].points[0];
+ return mapping.toScreen([x + offset[0], y + offset[1]]);
+ }
/**
- * A press within 8 viewport pixels of another patch selects the topmost one instead of painting; a
- * second finger belongs to the canvas, which turns a stroke into a pinch.
+ * A press on the first point of a patch that isn't selected selects the topmost one instead of
+ * painting, unless the selected patch's handles show there to take it; elsewhere it paints, so a new
+ * patch can start over another. A second finger belongs to the canvas, which turns a stroke into a
+ * pinch.
*/
function selectPatchAt(event: PointerEvent<HTMLDivElement>) {
if (
@@ -128,16 +146,23 @@ export function HealOverlay({
return;
}
const box = camera.ref.current?.getBoundingClientRect();
- const at =
- box &&
- mapping.toDocument([event.clientX - box.left, event.clientY - box.top]);
- const hit =
- at &&
- patches.findLast(
- (patch) =>
- patch.id !== selectedPatch &&
- patchContains(patch.strokes, at, 8 / mapping.scale(at)),
- );
+ if (!box) return;
+ const at: Point = [event.clientX - box.left, event.clientY - box.top];
+ const reaches = (anchor?: Point) =>
+ anchor !== undefined &&
+ Math.hypot(anchor[0] - at[0], anchor[1] - at[1]) <= anchorReach;
+ if (
+ handled &&
+ (reaches(firstShown(handled)) ||
+ (handled.mode !== "remove" &&
+ handled.id !== resolvingSource &&
+ reaches(firstShown(handled, handled.offset))))
+ ) {
+ return;
+ }
+ const hit = patches.findLast(
+ (patch) => patch.id !== selectedPatch && reaches(firstShown(patch)),
+ );
if (!hit) return;
event.preventDefault();
event.stopPropagation();
@@ -258,6 +283,20 @@ export function HealOverlay({
className="absolute inset-0 size-full overflow-visible"
style={{ pointerEvents: "none" }}
>
+ {/* Every other patch shows its first point, which selects it, under the outline shown. */}
+ {patches.map((patch) => {
+ if (patch.id === visiblePatch) return null;
+ const center = firstShown(patch);
+ return (
+ center && (
+ <HealAnchor
+ key={`anchor-${patch.id}`}
+ kind="destination"
+ center={center}
+ />
+ )
+ );
+ })}
{patches.map(
(patch) =>
visiblePatch === patch.id &&
@@ -270,9 +309,7 @@ export function HealOverlay({
patch.id !== drawingPatch &&
patch.id !== resolvingSource
}
- interactive={
- patch.id === selectedPatch && patch.id !== drawingPatch
- }
+ interactive={patch.id === handled?.id}
/>
</g>
),
diff --git a/src/features/layers/controls.tsx b/src/features/layers/controls.tsx
index c55cf2e..dedce1e 100644
--- a/src/features/layers/controls.tsx
+++ b/src/features/layers/controls.tsx
@@ -21,6 +21,7 @@ import {
useState,
} from "react";
import { useStore } from "zustand";
+import { OpacityInput } from "@/components/editor/opacity-input";
import { PanelHeader } from "@/components/editor/panel";
import { useDocument, useScene } from "@/components/editor/session";
import { Icon, type IconProps } from "@/components/icons/icon";
@@ -39,7 +40,6 @@ import { useShortcuts } from "@/hooks/use-shortcuts";
import { layerDrop } from "./drop";
import { deleteLayer, moveLayer, setLayer } from "./edits";
import { LayerName } from "./layer-name";
-import { LayerOpacity } from "./layer-opacity";
import { useMaskTool } from "./mask-tool";
import { LayerActions } from "./menu";
import { ImageThumbnail, MaskThumbnail } from "./thumbnails";
@@ -228,7 +228,13 @@ const LayerRow = memo(function LayerRow({
)}
{layer.kind !== "image" && (
<>
- <LayerOpacity layer={layer} />
+ <OpacityInput
+ label={`${layer.name} opacity`}
+ value={layer.opacity}
+ onChange={(opacity) =>
+ setLayer(document, layer.id, { opacity })
+ }
+ />
<LayerActions layer={layer} onSelect={onSelect} />
</>
)}
diff --git a/src/features/layers/layer-opacity.tsx b/src/features/layers/layer-opacity.tsx
deleted file mode 100644
index db7bc72..0000000
--- a/src/features/layers/layer-opacity.tsx
+++ /dev/null
@@ -1,27 +0,0 @@
-import { ScrubInput } from "@roprgm/ui/scrub-input";
-import { useDocument } from "@/components/editor/session";
-import type { ProcessingLayer } from "@/core/document";
-import { useEditGesture } from "@/hooks/use-edit-gesture";
-import { setLayer } from "./edits";
-
-/** The row's opacity, which drags sideways as one edit and types on click. */
-export function LayerOpacity({ layer }: { layer: ProcessingLayer }) {
- const document = useDocument();
- const gesture = useEditGesture(document.history);
- return (
- <span {...gesture} className="flex">
- <ScrubInput
- aria-label={`${layer.name} opacity`}
- value={layer.opacity * 100}
- min={0}
- max={100}
- defaultValue={100}
- format={(value) => `${value}%`}
- minChars={3}
- onChange={(value) =>
- setLayer(document, layer.id, { opacity: value / 100 })
- }
- />
- </span>
- );
-}
diff --git a/tests/heal-edit.e2e.ts b/tests/heal-edit.e2e.ts
index f105f45..3a6b016 100644
--- a/tests/heal-edit.e2e.ts
+++ b/tests/heal-edit.e2e.ts
@@ -65,7 +65,6 @@ for (const mode of ["Remove", "Heal"]) {
return layer?.kind === "heal" ? layer.patches : [];
});
const original = (await patches())[0];
- const removed = await readImage(page, undefined, samples);
const layerId = await page.evaluate(
() => window.openlight.getState().selectedLayerId,
);
@@ -110,9 +109,6 @@ for (const mode of ["Remove", "Heal"]) {
}
// Separate strokes never draw a connecting segment through the uncovered gap.
expect(repaired.samples?.[2]).toEqual(before.samples?.[2]);
- // Remove keeps what it filled and synthesizes only the new stroke.
- if (original.mode === "remove")
- expect(repaired.samples?.[0]).toEqual(removed.samples?.[0]);
await size.fill("20");
await size.press("Enter");
await page.keyboard.down("Alt");
diff --git a/tests/heal.e2e.ts b/tests/heal.e2e.ts
index e9a3e6c..42c6c37 100644
--- a/tests/heal.e2e.ts
+++ b/tests/heal.e2e.ts
@@ -1,6 +1,6 @@
import { expect, test } from "./fixtures";
import { readImage } from "./images";
-import { box } from "./pointer";
+import { box, drag } from "./pointer";
test("healing preserves an edge, alpha, and HDR texture at full and proxy resolution", async ({
page,
@@ -257,19 +257,26 @@ test("Healing paints patches, edits them, and undoes", async ({ page }) => {
await expect(canvas.locator(`[data-heal-patch="${manual.id}"]`)).toHaveCount(
1,
);
- await page.mouse.click(
- bounds.x + bounds.width / 2 - 250 * scale,
- bounds.y + bounds.height / 2 - 200 * scale,
- );
- await expect(firstPatch).toHaveAttribute("aria-pressed", "true");
- expect(
- await page.evaluate(() => {
+ const patchCount = () =>
+ page.evaluate(() => {
const healing = window.openlight
.getState()
.scene?.layers.find((item) => item.kind === "heal");
return healing?.kind === "heal" ? healing.patches.length : 0;
- }),
- ).toBe(2);
+ });
+ // Painting over a patch that isn't selected starts a new one; only its first point selects it.
+ const firstPoint = [
+ bounds.x + bounds.width / 2 - 250 * scale,
+ bounds.y + bounds.height / 2 - 200 * scale,
+ ] as const;
+ await page.mouse.click(firstPoint[0] + 60, firstPoint[1]);
+ await expect.poll(patchCount).toBe(3);
+ await expect(firstPatch).toHaveAttribute("aria-pressed", "false");
+ await page.keyboard.press("ControlOrMeta+z");
+ await expect.poll(patchCount).toBe(2);
+ await page.mouse.click(...firstPoint);
+ await expect(firstPatch).toHaveAttribute("aria-pressed", "true");
+ expect(await patchCount()).toBe(2);
const sourceX = () =>
page.evaluate((patchId) => {
const healing = window.openlight
@@ -360,6 +367,28 @@ test("Healing paints patches, edits them, and undoes", async ({ page }) => {
});
await opacity.fill("50");
await opacity.press("Enter");
+ // The patch's row drags its opacity to zero as one edit, as a layer's row does.
+ const opacityOf = () =>
+ page.evaluate((patchId) => {
+ const healing = window.openlight
+ .getState()
+ .scene?.layers.find((item) => item.kind === "heal");
+ return healing?.kind === "heal"
+ ? healing.patches.find((item) => item.id === patchId)?.opacity
+ : undefined;
+ }, manual.id);
+ const rowOpacity = await box(
+ patchList.getByRole("textbox", { name: "Patch 1 opacity", exact: true }),
+ );
+ const rowY = rowOpacity.y + rowOpacity.height / 2;
+ await drag(
+ page,
+ [rowOpacity.x + rowOpacity.width / 2, rowY],
+ [rowOpacity.x - 300, rowY],
+ );
+ expect(await opacityOf()).toBe(0);
+ await page.keyboard.press("ControlOrMeta+z");
+ expect(await opacityOf()).toBe(0.5);
const thumbnail = patchList.getByLabel("Patch shape").first();
const thumbnailBytes = await thumbnail.screenshot();
const thumbnailMask = await page.evaluate(
diff --git a/tests/heal.test.ts b/tests/heal.test.ts
index a17c353..1a3869b 100644
--- a/tests/heal.test.ts
+++ b/tests/heal.test.ts
@@ -7,9 +7,9 @@ import {
completeFields,
createDocument,
createResources,
- fieldRecords,
findLayer,
removePatches,
+ sceneFields,
} from "@/core/document";
import { createImageSource } from "@/core/image";
import { imageFrame } from "@/core/image/frame";
@@ -54,7 +54,6 @@ test("Remove passes never overwrite distinct uniforms before their frame submits
const gpu = await init();
const source = target(gpu, { size: [64, 64], format: "rgba16float" });
const coverage = target(gpu, { size: [64, 64], format: "r8unorm" });
- const base = target(gpu, { size: [64, 64], format: "rg16float" });
const graph = createRenderGraph(gpu);
const uploads = spyOn(gpu.gpu.queue, "writeBuffer");
const lattice = { origin: [0, 0], scale: 1, size: source.size } as const;
@@ -65,7 +64,6 @@ test("Remove passes never overwrite distinct uniforms before their frame submits
{ coverage: input(coverage), origin: [0, 0] },
lattice,
"remove",
- { texels: input(base), lattice },
),
]);
const uniforms = new Map<GPUBuffer, Uint8Array>();
@@ -81,7 +79,6 @@ test("Remove passes never overwrite distinct uniforms before their frame submits
} finally {
uploads.mockRestore();
graph.dispose();
- base.color.dispose();
coverage.color.dispose();
source.color.dispose();
gpu.dispose();
@@ -186,7 +183,7 @@ function patchesOf(document: ReturnType<typeof createDocument>, layer: string) {
return healing.patches;
}
-test("a nested Remove synthesizes once for its strokes, whatever changes below or hides it, extends on new strokes, and undoes without solving again", async () => {
+test("a nested Remove synthesizes once for its strokes, whatever changes below or hides it, synthesizes its whole shape again on new strokes, and undoes without solving again", async () => {
const gpu = await init();
const source = createImageSource(
target(gpu, { size: [128, 96], format: "rgba16float" }),
@@ -220,7 +217,7 @@ test("a nested Remove synthesizes once for its strokes, whatever changes below o
/** Whether the document saved the field the renderer synthesized, once its readback finishes. */
const saved = async () => {
await renderer.captureFields([]);
- return document.resources.field(own())?.field !== undefined;
+ return document.resources.field(own()) !== undefined;
};
try {
await render();
@@ -255,8 +252,9 @@ test("a nested Remove synthesizes once for its strokes, whatever changes below o
size: 2,
points: [[64, 48, 1]],
});
- // A new stroke takes a new field, which extends the earlier one.
- expect(document.resources.field(own())).toEqual({ base: before });
+ // A new stroke takes a new field, to synthesize for the whole new shape.
+ expect(own()).not.toBe(before);
+ expect(document.resources.field(own())).toBeUndefined();
// A proxy keeps the field it holds; only full resolution synthesizes the new stroke.
await render(true);
expect(solves()).toBe(false);
@@ -269,8 +267,7 @@ test("a nested Remove synthesizes once for its strokes, whatever changes below o
await render();
expect(solves()).toBe(false);
}
- // Undo and redo reach every stroke version's saved field, and show it without solving again, even
- // when the field it extends is the one held.
+ // Undo and redo reach every stroke version's saved field, and show it without solving again.
for (const travel of ["undo", "redo"] as const) {
for (let step = 0; step < 4; step++) {
document.history[travel]();
@@ -308,10 +305,10 @@ test("a snapshot taken while a Remove field is read back keeps it once its photo
const [{ patch }] = removePatches(document.scene.getState().layers);
// The render synthesizes the field and starts reading it back; another photo then closes this one.
void renderer.update(document.scene.getState());
- const snapshot = completeFields(document, fieldRecords(document));
+ const snapshot = completeFields(document, sceneFields(document));
renderer.dispose();
document.dispose();
- expect((await snapshot).get(patch.field)?.field).toBeDefined();
+ expect((await snapshot).get(patch.field)).toBeDefined();
} finally {
source.dispose();
gpu.dispose();
@@ -339,15 +336,15 @@ test("a snapshot reads back a Remove field whose save failed, and fails when tha
const [{ patch }] = removePatches(document.scene.getState().layers);
await renderer.update(document.scene.getState());
await renderer.captureFields([]);
- expect(document.resources.field(patch.field)).toEqual({});
- const records = fieldRecords(document);
+ expect(document.resources.field(patch.field)).toBeUndefined();
+ const records = sceneFields(document);
await expect(completeFields(document, records)).rejects.toThrow(
"Device lost",
);
failing = false;
const completed = await completeFields(document, records);
- expect(completed.get(patch.field)?.field).toBeDefined();
- expect(document.resources.field(patch.field)?.field).toBeDefined();
+ expect(completed.get(patch.field)).toBeDefined();
+ expect(document.resources.field(patch.field)).toBeDefined();
} finally {
stop();
renderer.dispose();
diff --git a/tests/inpaint-gpu.ts b/tests/inpaint-gpu.ts
index e147f83..7e5c594 100644
--- a/tests/inpaint-gpu.ts
+++ b/tests/inpaint-gpu.ts
@@ -2,9 +2,16 @@ import { effect, frame, init, target } from "vgpu";
import { encodeImage } from "@/app/editor/export/export-image";
import { createImageLayer, createLayer } from "@/app/editor/layers";
import { createEditorRenderer } from "@/app/editor/renderer";
-import type { FieldRecord, HealPatch, Scene } from "@/core/document";
+import {
+ createDocument,
+ createResources,
+ type HealPatch,
+ type Scene,
+} from "@/core/document";
import { createImageSource } from "@/core/image";
import { imageFrame } from "@/core/image/frame";
+import { addHealStroke, addRemovePatch } from "@/features/heal/edits";
+import { addLayer } from "@/features/layers/edits";
export type InpaintFixture =
| "flat"
@@ -248,32 +255,37 @@ export async function renderInpaintReference(
}
/**
- * A Remove patch over two spots on noise, where any other donor shows: a stroke added to it, or one
- * erasing part of it, leaves what it filled as it was and synthesizes only the rest.
+ * A Remove patch over a dark bar on noise whose first stroke stops short of the bar's tip: a stroke
+ * added over the tip, and one erasing part of the patch, each synthesize the whole new shape, as
+ * painting it at once does, so nothing of the bar stays.
*/
-export async function renderInpaintExtension() {
+export async function renderInpaintReshape() {
const gpu = await init();
const errors: string[] = [];
gpu.onError((error) => errors.push(error.message));
const size: [number, number] = [192, 144];
const image = target(gpu, { size, format: "rgba16float" });
const source = createImageSource(image);
- // Each stroke count is a version with its own field, extending the one before, as edits reserve them.
- const fields = ["spots/1", "spots/2", "spots/3"];
- const records = new Map<string, FieldRecord>([
- [fields[0], {}],
- [fields[1], { base: fields[0] }],
- [fields[2], { base: fields[1] }],
- ]);
+ const resources = createResources();
+ const document = createDocument(
+ {
+ frame: imageFrame(size),
+ layers: [
+ createImageLayer(resources.add(new File([], "bar.png"), source), "Bar"),
+ ],
+ },
+ resources,
+ );
+ // The editor's renderer keeps each version's field in the document; another paints shapes at once.
const renderer = createEditorRenderer(gpu, source, {
- field: (id) => records.get(id),
+ field: (id) => document.resources.field(id),
+ saveField: (id, field) => document.resources.fillField(id, field),
});
- const first = [56, 72] as const;
- const second = [140, 72] as const;
- const erased = [48, 72] as const;
+ const fresh = createEditorRenderer(gpu, source);
const stroke = (
mode: "paint" | "erase",
- [x, y]: readonly [number, number],
+ from: readonly [number, number],
+ to: readonly [number, number],
diameter: number,
) =>
({
@@ -281,33 +293,17 @@ export async function renderInpaintExtension() {
size: diameter,
feather: 0,
flow: 1,
- points: [[x, y, 1]],
+ points: [
+ [from[0], from[1], 1],
+ [to[0], to[1], 1],
+ ],
}) as const;
+ const erased = [70, 60] as const;
const strokes = [
- stroke("paint", first, 32),
- stroke("paint", second, 32),
- stroke("erase", erased, 12),
+ stroke("paint", [40, 72], [118, 72], 28),
+ stroke("paint", [118, 72], [156, 72], 28),
+ stroke("erase", erased, erased, 8),
];
- const healing = createLayer("heal");
- const scene = (count: number): Scene => ({
- frame: imageFrame(size),
- layers: [
- createImageLayer("source", "Spots"),
- {
- ...healing,
- patches: [
- {
- id: "spots",
- mode: "remove",
- field: fields[count - 1],
- feather: 0,
- opacity: 1,
- strokes: strokes.slice(0, count),
- },
- ],
- },
- ],
- });
try {
frame(gpu, (f) =>
f.pass(
@@ -317,70 +313,86 @@ export async function renderInpaintExtension() {
`
@fragment fn fs_main(@builtin(position) p: vec4f) -> @location(0) vec4f {
let noise = fract(sin(dot(floor(p.xy), vec2f(12.9898, 78.233))) * 43758.5453);
- let spot = distance(p.xy, vec2f(${first.join(", ")})) < 12.0 || distance(p.xy, vec2f(${second.join(", ")})) < 12.0;
- return vec4f(select(vec3f(0.3 + 0.2 * noise, 0.4, 0.5), vec3f(0.015), spot), 1.0);
+ let bar = p.x >= 40.0 && p.x <= 150.0 && abs(p.y - 72.0) < 6.0;
+ return vec4f(select(vec3f(0.3 + 0.2 * noise, 0.4, 0.5), vec3f(0.015), bar), 1.0);
}
`,
),
),
);
- const renders = [];
- for (const count of [1, 2, 3]) {
- await renderer.update(scene(count));
- renders.push({
- pixels: await renderer.fullImage().readFloats(),
- solved: renderer
- .inspect()
- .passes.some((name) => name.includes("/inpaint/")),
+ const original = await image.readFloats();
+ // Each stroke edits the patch as the editor does, and every version renders as it is reached.
+ const layer = addLayer(document, createLayer("heal"));
+ const patch = addRemovePatch(document, layer, strokes[0]);
+ const reshaped = [];
+ for (const next of [undefined, strokes[1], strokes[2]]) {
+ if (next) addHealStroke(document, layer, patch, next);
+ await renderer.update(document.scene.getState());
+ reshaped.push(await renderer.fullImage().readFloats());
+ }
+ // The same shapes painted at once: the scene of each version, rendered with no field held.
+ const atOnce = [];
+ for (const count of [2, 3]) {
+ const scene = document.scene.getState();
+ const [, healing] = scene.layers;
+ if (healing.kind !== "heal") throw Error("The Healing layer is gone.");
+ const [painted] = healing.patches;
+ if (painted?.mode !== "remove") throw Error("The Remove patch is gone.");
+ await fresh.update({
+ ...scene,
+ layers: [
+ scene.layers[0],
+ {
+ ...healing,
+ patches: [
+ {
+ ...painted,
+ field: `at-once/${count}`,
+ strokes: painted.strokes.slice(0, count),
+ },
+ ],
+ },
+ ],
});
+ atOnce.push(await fresh.fullImage().readFloats());
}
- const original = await image.readFloats();
- // The largest change in red between two renders over the pixels `within` selects.
- function change(
- a: Float32Array,
- b: Float32Array,
+ /** The red of each pixel `within` selects. */
+ function reds(
+ pixels: Float32Array,
within: (x: number, y: number) => boolean,
) {
- let largest = 0;
- for (let index = 0; index < a.length; index += 4) {
+ const selected = [];
+ for (let index = 0; index < pixels.length; index += 4) {
const x = ((index / 4) % size[0]) + 0.5;
const y = Math.floor(index / 4 / size[0]) + 0.5;
- if (within(x, y))
- largest = Math.max(largest, Math.abs(a[index] - b[index]));
+ if (within(x, y)) selected.push(pixels[index]);
}
- return largest;
+ return selected;
+ }
+ function change(a: Float32Array, b: Float32Array) {
+ return Math.max(...a.map((value, index) => Math.abs(value - b[index])));
}
- const near =
- ([cx, cy]: readonly [number, number], radius: number) =>
- (x: number, y: number) =>
- Math.hypot(x - cx, y - cy) < radius;
- const [filled, extended, subtracted] = renders.map(({ pixels }) => pixels);
+ const bar = (x: number, y: number) =>
+ x >= 40 && x <= 150 && Math.abs(y - 72) < 6;
+ const tip = (x: number, y: number) => bar(x, y) && x > 136;
+ const near = (x: number, y: number) =>
+ Math.hypot(x - erased[0], y - erased[1]) < 3;
+ const [first, added, subtracted] = reshaped;
return {
errors,
- solved: renders.map(({ solved }) => solved),
- // The first spot, as the first stroke filled it.
- keptByAdding: change(filled, extended, near(first, 14)),
- keptByErasing: change(
- filled,
- subtracted,
- (x, y) => near(first, 14)(x, y) && !near(erased, 8)(x, y),
+ missedTip: Math.min(...reds(first, tip)),
+ barLeft: Math.min(...reds(added, bar)),
+ added: change(added, atOnce[0]),
+ erased: change(subtracted, atOnce[1]),
+ erasedRestored: change(
+ new Float32Array(reds(original, near)),
+ new Float32Array(reds(subtracted, near)),
),
- // The second spot was dark before the stroke over it, and the erased part is again.
- secondFilled: Math.min(
- ...[...extended].filter(
- (_, index) =>
- index % 4 === 0 &&
- near(second, 10)(
- ((index / 4) % size[0]) + 0.5,
- Math.floor(index / 4 / size[0]) + 0.5,
- ),
- ),
- ),
- erasedRestored: change(original, subtracted, near(erased, 4)),
};
} finally {
renderer.dispose();
- source.dispose();
+ fresh.dispose();
+ document.dispose();
gpu.dispose();
}
}
diff --git a/tests/inpaint.e2e.ts b/tests/inpaint.e2e.ts
index 13dea4a..40256bf 100644
--- a/tests/inpaint.e2e.ts
+++ b/tests/inpaint.e2e.ts
@@ -260,22 +260,25 @@ test("Remove synthesizes a spot once, without donor selection, and follows the l
expect(result.releasedFields).toBe(0);
});
-test("Remove keeps what it filled when a stroke adds to or erases part of the patch", async ({
+test("Remove synthesizes the whole patch again when a stroke adds to or erases part of it", async ({
page,
}) => {
await page.goto("/tests/gpu.html");
const result = await page.evaluate(async () => {
const path = "/tests/inpaint-gpu.ts";
- const { renderInpaintExtension } = (await import(
+ const { renderInpaintReshape } = (await import(
path
)) as typeof import("./inpaint-gpu");
- return renderInpaintExtension();
+ return renderInpaintReshape();
});
expect(result.errors).toEqual([]);
- expect(result.solved).toEqual([true, true, true]);
- expect(result.keptByAdding).toBeLessThan(0.002);
- expect(result.keptByErasing).toBeLessThan(0.002);
- expect(result.secondFilled).toBeGreaterThan(0.25);
+ // The first stroke stops short of the bar's tip, which stays dark.
+ expect(result.missedTip).toBeLessThan(0.05);
+ // A stroke over the tip fills as painting both at once does, and nothing of the bar stays.
+ expect(result.added).toBeLessThan(0.002);
+ expect(result.barLeft).toBeGreaterThan(0.25);
+ // Erasing part of the patch synthesizes the rest again too, and restores what it erases.
+ expect(result.erased).toBeLessThan(0.002);
expect(result.erasedRestored).toBe(0);
});
diff --git a/tests/scene-file.test.ts b/tests/scene-file.test.ts
index 3806a6e..1a07167 100644
--- a/tests/scene-file.test.ts
+++ b/tests/scene-file.test.ts
@@ -187,16 +187,6 @@ test("scene files reopen the photo with every layer for further editing", async
[await archive({ ...json, format: "other" }), "an OpenLight scene"],
[await archive({ ...json, version: 4 }), "needs a newer version"],
[await archive({ ...json, fields: {} }), "Remove fields are missing"],
- [
- await archive({
- ...json,
- fields: {
- [removal.field]: { base: "loop" },
- loop: { base: removal.field },
- },
- }),
- "Remove fields are missing",
- ],
[
await archive({
...json,
@@ -277,7 +267,8 @@ test("scene files reopen the photo with every layer for further editing", async
],
});
expect(older.resources.field("inline")).toEqual({
- field: { texels: expect.any(Blob), ...lattice },
+ texels: expect.any(Blob),
+ ...lattice,
});
expect(