Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(look): adjustment layers, filters, every blend mode, feather; repaint removed

yhh7400a2f80dbe
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 424fd299a..f8cb5e7d8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,7 +22,7 @@ published separately; it ships inside the editor.
- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.
- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.
-- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do; one `sweep` instruction lays a mass of hair gathered into locks, one `field` covers a region in strokes along a flow, with named brushes from ink and pencil to oil and watercolor, and one `along` repeats strokes along a line. A layer can work across everything under it, to fuse separate parts into one painting, and `image_read` shows lightness alone and reads colours. Text in any script, windows in rows on a face, buildings in true perspective, reflections on wet ground and bloom over what is bright are one instruction each, and words and strokes can be laid on a building's face in its own coordinates. `repaint` paints the finished picture again in tens of thousands of strokes, and an edit draws only the layers from the change up. A `figure` layer is a posed skeleton in plain blocks from any angle to draw over, and an `image` layer brings in the team's own art: both stay out of the `.png`, and the figure reports where each joint landed and lets broad light follow the form. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.
+- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do; one `sweep` instruction lays a mass of hair gathered into locks, one `field` covers a region in strokes along a flow, with named brushes from ink and pencil to oil and watercolor, and one `along` repeats strokes along a line. A layer can work across everything under it, to fuse separate parts into one painting, and `image_read` shows lightness alone and reads colours. Text in any script, windows in rows on a face, buildings in true perspective, reflections on wet ground and bloom over what is bright are one instruction each, and words and strokes can be laid on a building's face in its own coordinates. An image editor's finish is there to turn by hand: adjustment layers from levels and curves to gradient maps, twenty filters from lens blur to fog and light rays, the full set of blend modes, feathered masks and bevels. An edit draws only the layers from the change up. A `figure` layer is a posed skeleton in plain blocks from any angle to draw over, and an `image` layer brings in the team's own art: both stay out of the `.png`, and the figure reports where each joint landed and lets broad light follow the form. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.
- **An AI can build a person, and not a picture of one.** The `humanoid` step makes a whole body from a few numbers, with fingers, a face and ears, seen right from every side, and `rig: { humanoid: true }` gives it bones that fit. `garment` makes clothes from the body's own surface: loose or tight, a bell sleeve that hangs, a clean hem, a thickness; what a body wears now takes its weights from the skin under it, so an arm stays in its sleeve. `pose` holds a body in a pose by angles, or with `reach` by where a hand should be. `organic` gains `limb`, cross-sections along a path, and `fine`, regions split finer on one skin. `displace` gains `folds`, running the way cloth hangs. `scene_views` no longer takes on a shot's lens blur and grade.
diff --git a/desktop b/desktop
index e0c778259..ee4766dd8 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit e0c778259c5311355c7ed5ef9dc244304ef38bb6
+Subproject commit ee4766dd8c6fadfea6c22c9e581983e061a5cfb8
diff --git a/pipeline/src/image/imageAdjust.ts b/pipeline/src/image/imageAdjust.ts
new file mode 100644
index 000000000..273315319
--- /dev/null
+++ b/pipeline/src/image/imageAdjust.ts
@@ -0,0 +1,518 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file imageAdjust.ts
+ * @brief What a photo editor offers over a picture so far: adjustments of its colour, and filters over its pixels.
+ *
+ * Each is one operation with numbers to turn, and the layer it sits on says where it applies and how much. Which to use, and how far, is the model's.
+ * Pixels here are stored as the canvas stores them, colour already times alpha.
+ */
+import { blurMask, fractalNoise, parseColor } from './imageRaster';
+import type { ImageAdjust, ImageFilterName } from './imageSource';
+
+const clamp01 = (v: number): number => Math.max(0, Math.min(1, v));
+const smooth = (t: number): number => t * t * (3 - 2 * t);
+export const lightnessOf = (r: number, g: number, b: number): number => r * 0.2126 + g * 0.7152 + b * 0.0722;
+
+// ---- Colour ----------------------------------------------------------------------
+
+function rgbToHsl(r: number, g: number, b: number): [number, number, number] {
+ const [hi, lo] = [Math.max(r, g, b), Math.min(r, g, b)];
+ const l = (hi + lo) / 2;
+ if (hi === lo) return [0, 0, l];
+ const d = hi - lo;
+ const s = l > 0.5 ? d / (2 - hi - lo) : d / (hi + lo);
+ const h = hi === r ? (g - b) / d + (g < b ? 6 : 0) : hi === g ? (b - r) / d + 2 : (r - g) / d + 4;
+ return [h / 6, s, l];
+}
+
+function hslToRgb(h: number, s: number, l: number): [number, number, number] {
+ if (s === 0) return [l, l, l];
+ const q = l < 0.5 ? l * (1 + s) : l + s - l * s;
+ const p = 2 * l - q;
+ const channel = (t0: number): number => {
+ const t = t0 < 0 ? t0 + 1 : t0 > 1 ? t0 - 1 : t0;
+ if (t < 1 / 6) return p + (q - p) * 6 * t;
+ if (t < 1 / 2) return q;
+ if (t < 2 / 3) return p + (q - p) * (2 / 3 - t) * 6;
+ return p;
+ };
+ return [channel(h + 1 / 3), channel(h), channel(h - 1 / 3)];
+}
+
+/** Through its points without overshooting them, as a curves dialog draws it: a table for 0..1. */
+function curveTable(points: readonly (readonly number[])[]): Float32Array {
+ const sorted = [...points].sort((a, b) => a[0]! - b[0]!);
+ const xs = sorted.map((p) => p[0]!), ys = sorted.map((p) => p[1]!);
+ const n = xs.length;
+ const slopes = xs.slice(0, -1).map((x, i) => (ys[i + 1]! - ys[i]!) / Math.max(1e-6, xs[i + 1]! - x));
+ const tangents = xs.map((_, i) => {
+ if (i === 0) return slopes[0] ?? 0;
+ if (i === n - 1) return slopes[n - 2] ?? 0;
+ return slopes[i - 1]! * slopes[i]! <= 0 ? 0 : (slopes[i - 1]! + slopes[i]!) / 2;
+ });
+ const table = new Float32Array(1024);
+ for (let k = 0; k < table.length; k++) {
+ const x = k / (table.length - 1);
+ if (x <= xs[0]!) { table[k] = ys[0]!; continue; }
+ if (x >= xs[n - 1]!) { table[k] = ys[n - 1]!; continue; }
+ let i = 0;
+ while (x > xs[i + 1]!) i++;
+ const h = xs[i + 1]! - xs[i]!;
+ const t = (x - xs[i]!) / h;
+ const [t2, t3] = [t * t, t * t * t];
+ table[k] = clamp01((2 * t3 - 3 * t2 + 1) * ys[i]! + (t3 - 2 * t2 + t) * h * tangents[i]! + (3 * t2 - 2 * t3) * ys[i + 1]! + (t3 - t2) * h * tangents[i + 1]!);
+ }
+ return table;
+}
+const look = (table: Float32Array, v: number): number => table[Math.round(clamp01(v) * (table.length - 1))]!;
+
+/** Colour along stops, for a gradient map. */
+function stopsTable(stops: readonly (readonly [number, string])[]): Float32Array {
+ const sorted = [...stops].sort((a, b) => a[0] - b[0]).map(([t, c]) => [t, parseColor(c)] as const);
+ const out = new Float32Array(256 * 3);
+ for (let k = 0; k < 256; k++) {
+ const v = k / 255;
+ let i = 0;
+ while (i + 2 < sorted.length && v > sorted[i + 1]![0]) i++;
+ const [a, b] = [sorted[i]!, sorted[Math.min(sorted.length - 1, i + 1)]!];
+ const t = clamp01((v - a[0]) / Math.max(1e-6, b[0] - a[0]));
+ for (let n = 0; n < 3; n++) out[k * 3 + n] = a[1][n]! + (b[1][n]! - a[1][n]!) * t;
+ }
+ return out;
+}
+
+/** The picture with its colour changed, in the order a stack of adjustment layers would most often take. */
+export function applyAdjust(px: Float32Array, w: number, h: number, a: ImageAdjust): Float32Array {
+ const out = new Float32Array(px.length);
+ const levels = a.levels;
+ const [inLow, inHigh] = levels?.input ?? [0, 1];
+ const [outLow, outHigh] = levels?.output ?? [0, 1];
+ const gamma = levels?.gamma ?? 1;
+ const curves = a.curves ? (Array.isArray(a.curves) ? { all: curveTable(a.curves as readonly (readonly number[])[]) } : Object.fromEntries(Object.entries(a.curves).map(([k, v]) => [k, curveTable(v as readonly (readonly number[])[])]))) as Record<string, Float32Array> : null;
+ const exposure = a.exposure !== undefined ? 2 ** a.exposure : 1;
+ const [brightness, contrast] = [a.brightness ?? 0, a.contrast ?? 0];
+ const contrastGain = contrast >= 0 ? 1 / Math.max(0.01, 1 - contrast * 0.98) : 1 + contrast;
+ const [warm, tint] = [a.temperature ?? 0, a.tint ?? 0];
+ const [hue, saturation, lightness] = [(a.hue ?? 0) / 360, a.saturation ?? 0, a.lightness ?? 0];
+ const vibrance = a.vibrance ?? 0;
+ const map = a.gradientMap ? stopsTable(a.gradientMap) : null;
+ const levelsStep = a.posterize !== undefined ? Math.max(2, Math.round(a.posterize)) : 0;
+ const balance = a.balance;
+ for (let i = 0; i < w * h; i++) {
+ const at = i * 4, alpha = px[at + 3]!;
+ if (alpha <= 0) continue;
+ let [r, g, b] = [px[at]! / alpha, px[at + 1]! / alpha, px[at + 2]! / alpha];
+ if (exposure !== 1) { r *= exposure; g *= exposure; b *= exposure; }
+ if (levels) {
+ const f = (v: number): number => outLow + (outHigh - outLow) * clamp01((v - inLow) / Math.max(1e-6, inHigh - inLow)) ** (1 / gamma);
+ r = f(r); g = f(g); b = f(b);
+ }
+ if (curves) {
+ if (curves.all) { r = look(curves.all, r); g = look(curves.all, g); b = look(curves.all, b); }
+ if (curves.r) r = look(curves.r, r);
+ if (curves.g) g = look(curves.g, g);
+ if (curves.b) b = look(curves.b, b);
+ }
+ if (brightness || contrast) {
+ const f = (v: number): number => (v + brightness - 0.5) * contrastGain + 0.5;
+ r = f(r); g = f(g); b = f(b);
+ }
+ if (warm || tint) { r += warm * 0.2 + tint * 0.1; g -= tint * 0.2; b += -warm * 0.2 + tint * 0.1; }
+ if (balance) {
+ const l = clamp01(lightnessOf(r, g, b));
+ const [dark, mid, bright] = [smooth(clamp01(1 - l * 2.5)), 1 - Math.abs(l - 0.5) * 2, smooth(clamp01((l - 0.6) * 2.5))];
+ for (const [weight, shift] of [[dark, balance.shadows], [mid, balance.midtones], [bright, balance.highlights]] as const) {
+ if (!shift) continue;
+ r += shift[0]! * weight * 0.5; g += shift[1]! * weight * 0.5; b += shift[2]! * weight * 0.5;
+ }
+ }
+ if (hue || saturation || lightness) {
+ const [hh, ss, ll] = rgbToHsl(clamp01(r), clamp01(g), clamp01(b));
+ const s2 = saturation >= 0 ? ss + (1 - ss) * saturation * (ss > 0 ? 1 : 0) : ss * (1 + saturation);
+ const l2 = lightness >= 0 ? ll + (1 - ll) * lightness : ll * (1 + lightness);
+ [r, g, b] = hslToRgb((hh + hue + 1) % 1, clamp01(s2), clamp01(l2));
+ }
+ if (vibrance) {
+ // More for what is dull than for what is already vivid, so skin and neon are not pushed over.
+ const [hi, lo] = [Math.max(r, g, b), Math.min(r, g, b)];
+ const amount = vibrance * (1 - clamp01(hi - lo));
+ const l = lightnessOf(r, g, b);
+ r = l + (r - l) * (1 + amount); g = l + (g - l) * (1 + amount); b = l + (b - l) * (1 + amount);
+ }
+ if (a.blackWhite) {
+ const [cr, cg, cb] = Array.isArray(a.blackWhite) ? a.blackWhite as readonly number[] : [0.2126, 0.7152, 0.0722];
+ r = g = b = r * cr! + g * cg! + b * cb!;
+ }
+ if (map) {
+ const k = Math.round(clamp01(lightnessOf(r, g, b)) * 255) * 3;
+ [r, g, b] = [map[k]!, map[k + 1]!, map[k + 2]!];
+ }
+ if (a.invert) { r = 1 - r; g = 1 - g; b = 1 - b; }
+ if (levelsStep) {
+ const f = (v: number): number => Math.round(clamp01(v) * (levelsStep - 1)) / (levelsStep - 1);
+ r = f(r); g = f(g); b = f(b);
+ }
+ if (a.threshold !== undefined) r = g = b = lightnessOf(r, g, b) >= a.threshold ? 1 : 0;
+ out[at] = clamp01(r) * alpha; out[at + 1] = clamp01(g) * alpha; out[at + 2] = clamp01(b) * alpha; out[at + 3] = alpha;
+ }
+ return out;
+}
+
+// ---- Filters ---------------------------------------------------------------------
+
+const bilinear = (px: Float32Array, w: number, h: number, x: number, y: number, out: Float32Array, to: number): void => {
+ const [fx, fy] = [Math.max(0, Math.min(w - 1.001, x - 0.5)), Math.max(0, Math.min(h - 1.001, y - 0.5))];
+ const [ix, iy] = [Math.floor(fx), Math.floor(fy)];
+ const [tx, ty] = [fx - ix, fy - iy];
+ const [a, b, c, d] = [(iy * w + ix) * 4, (iy * w + ix + 1) * 4, ((iy + 1) * w + ix) * 4, ((iy + 1) * w + ix + 1) * 4];
+ for (let n = 0; n < 4; n++) {
+ out[to + n] = (px[a + n]! * (1 - tx) + px[b + n]! * tx) * (1 - ty) + (px[c + n]! * (1 - tx) + px[d + n]! * tx) * ty;
+ }
+};
+
+const blurAll = (px: Float32Array, w: number, h: number, sigma: number): Float32Array => {
+ const out = new Float32Array(px.length);
+ for (let n = 0; n < 4; n++) {
+ const channel = new Float32Array(w * h);
+ for (let i = 0; i < channel.length; i++) channel[i] = px[i * 4 + n]!;
+ const soft = blurMask(channel, w, h, sigma);
+ for (let i = 0; i < channel.length; i++) out[i * 4 + n] = soft[i]!;
+ }
+ return out;
+};
+
+/** Every pixel from somewhere else in the picture. */
+function warped(px: Float32Array, w: number, h: number, from: (x: number, y: number) => readonly [number, number]): Float32Array {
+ const out = new Float32Array(px.length);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const [sx, sy] = from(x + 0.5, y + 0.5);
+ bilinear(px, w, h, sx, sy, out, (y * w + x) * 4);
+ }
+ }
+ return out;
+}
+
+/** Every pixel the mean of several places, each given by how far along 0..1 it is. */
+function averaged(px: Float32Array, w: number, h: number, samples: number, from: (x: number, y: number, t: number) => readonly [number, number]): Float32Array {
+ const out = new Float32Array(px.length);
+ const one = new Float32Array(4);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const to = (y * w + x) * 4;
+ for (let s = 0; s < samples; s++) {
+ const [sx, sy] = from(x + 0.5, y + 0.5, samples === 1 ? 0 : s / (samples - 1));
+ bilinear(px, w, h, sx, sy, one, 0);
+ for (let n = 0; n < 4; n++) out[to + n]! += one[n]!;
+ }
+ for (let n = 0; n < 4; n++) out[to + n]! /= samples;
+ }
+ }
+ return out;
+}
+
+/** Light over a pixel without covering it: `add` 0..1 in each colour on top of what is there. */
+function screened(px: Float32Array, w: number, h: number, light: (x: number, y: number, into: Float32Array) => void): Float32Array {
+ const out = new Float32Array(px);
+ const add = new Float32Array(3);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ add.fill(0);
+ light(x + 0.5, y + 0.5, add);
+ const at = (y * w + x) * 4;
+ let most = out[at + 3]!;
+ for (let n = 0; n < 3; n++) {
+ out[at + n] = out[at + n]! + (1 - out[at + n]!) * clamp01(add[n]!);
+ most = Math.max(most, out[at + n]!);
+ }
+ out[at + 3] = most;
+ }
+ }
+ return out;
+}
+
+type P = Readonly<Record<string, unknown>>;
+const num = (p: P, key: string, fallback: number): number => (p[key] as number | undefined) ?? fallback;
+const pointOf = (p: P, key: string, fallback: readonly [number, number]): readonly [number, number] => (p[key] as readonly [number, number] | undefined) ?? fallback;
+
+/** The picture with a filter over it: what it would be everywhere, which the layer then lays down where and as much as it says. @p k is the size of a pixel in the source's terms. */
+export function applyFilter(name: ImageFilterName, p: P, px: Float32Array, w: number, h: number, k: number): Float32Array {
+ const middle = [w / (2 * k), h / (2 * k)] as const;
+ const centre = (key = 'at'): readonly [number, number] => { const c = pointOf(p, key, middle); return [c[0] * k, c[1] * k]; };
+ switch (name) {
+ case 'gaussian':
+ return blurAll(px, w, h, num(p, 'radius', 4) * k);
+ case 'motion': {
+ const a = (num(p, 'angle', 0) * Math.PI) / 180, d = num(p, 'distance', 16) * k;
+ const [dx, dy] = [Math.cos(a) * d, Math.sin(a) * d];
+ return averaged(px, w, h, Math.max(2, Math.min(64, Math.ceil(d))), (x, y, t) => [x + dx * (t - 0.5), y + dy * (t - 0.5)]);
+ }
+ case 'spin': {
+ const [cx, cy] = centre();
+ const amount = (num(p, 'angle', 10) * Math.PI) / 180;
+ return averaged(px, w, h, 24, (x, y, t) => {
+ const r = amount * (t - 0.5);
+ const [dx, dy] = [x - cx, y - cy];
+ return [cx + dx * Math.cos(r) - dy * Math.sin(r), cy + dx * Math.sin(r) + dy * Math.cos(r)];
+ });
+ }
+ case 'zoom': {
+ const [cx, cy] = centre();
+ const amount = num(p, 'amount', 0.1);
+ return averaged(px, w, h, 24, (x, y, t) => { const s = 1 + amount * (t - 0.5); return [cx + (x - cx) * s, cy + (y - cy) * s]; });
+ }
+ case 'lens': {
+ // A disc and not a bell, so highlights open into the round lights of a real lens.
+ const radius = num(p, 'radius', 6) * k;
+ const bright = num(p, 'highlights', 1);
+ const rings = Math.max(2, Math.min(6, Math.ceil(radius / 2)));
+ const offsets: [number, number][] = [[0, 0]];
+ for (let ring = 1; ring <= rings; ring++) {
+ const count = ring * 7;
+ for (let s = 0; s < count; s++) {
+ const a = (s / count) * Math.PI * 2 + ring;
+ offsets.push([Math.cos(a) * (radius * ring) / rings, Math.sin(a) * (radius * ring) / rings]);
+ }
+ }
+ const out = new Float32Array(px.length);
+ const one = new Float32Array(4);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const to = (y * w + x) * 4;
+ let weight = 0;
+ for (const [ox, oy] of offsets) {
+ bilinear(px, w, h, x + 0.5 + ox, y + 0.5 + oy, one, 0);
+ const l = lightnessOf(one[0]!, one[1]!, one[2]!);
+ const q = 1 + bright * 2 * l * l * l * l;
+ for (let n = 0; n < 4; n++) out[to + n]! += one[n]! * q;
+ weight += q;
+ }
+ for (let n = 0; n < 4; n++) out[to + n]! /= weight;
+ }
+ }
+ return out;
+ }
+ case 'sharpen': {
+ const soft = blurAll(px, w, h, num(p, 'radius', 1.5) * k);
+ const [amount, threshold] = [num(p, 'amount', 0.8), num(p, 'threshold', 0)];
+ const out = new Float32Array(px);
+ for (let i = 0; i < w * h; i++) {
+ const at = i * 4, a = px[at + 3]!;
+ for (let n = 0; n < 3; n++) {
+ const diff = px[at + n]! - soft[at + n]!;
+ if (Math.abs(diff) >= threshold) out[at + n] = Math.max(0, Math.min(a, px[at + n]! + diff * amount));
+ }
+ }
+ return out;
+ }
+ case 'highPass': {
+ const soft = blurAll(px, w, h, num(p, 'radius', 6) * k);
+ const out = new Float32Array(px.length);
+ for (let i = 0; i < w * h; i++) {
+ const at = i * 4, a = px[at + 3]!;
+ out[at + 3] = a;
+ for (let n = 0; n < 3; n++) out[at + n] = clamp01(0.5 + px[at + n]! - soft[at + n]!) * a;
+ }
+ return out;
+ }
+ case 'noise': {
+ const [amount, scale, seed, mono] = [num(p, 'amount', 0.08), num(p, 'scale', 1) * k, num(p, 'seed', 1), p.mono !== false];
+ const out = new Float32Array(px);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const at = (y * w + x) * 4, a = px[at + 3]!;
+ for (let n = 0; n < 3; n++) {
+ const v = fractalNoise(x / scale, y / scale, 1, mono ? seed : seed + n * 13) - 0.5;
+ out[at + n] = Math.max(0, Math.min(a, px[at + n]! + v * amount * 2 * a));
+ }
+ }
+ }
+ return out;
+ }
+ case 'median': {
+ const r = Math.max(1, Math.min(6, Math.round(num(p, 'radius', 1) * k)));
+ const out = new Float32Array(px.length);
+ const lights: [number, number][] = [];
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ lights.length = 0;
+ for (let dy = -r; dy <= r; dy++) {
+ for (let dx = -r; dx <= r; dx++) {
+ const at = (Math.max(0, Math.min(h - 1, y + dy)) * w + Math.max(0, Math.min(w - 1, x + dx))) * 4;
+ lights.push([lightnessOf(px[at]!, px[at + 1]!, px[at + 2]!), at]);
+ }
+ }
+ lights.sort((a, b) => a[0] - b[0]);
+ const from = lights[lights.length >> 1]![1], to = (y * w + x) * 4;
+ for (let n = 0; n < 4; n++) out[to + n] = px[from + n]!;
+ }
+ }
+ return out;
+ }
+ case 'emboss': {
+ const a = (num(p, 'angle', 135) * Math.PI) / 180, d = num(p, 'height', 2) * k, amount = num(p, 'amount', 2);
+ const [dx, dy] = [Math.round(Math.cos(a) * d), Math.round(Math.sin(a) * d)];
+ const out = new Float32Array(px.length);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const at = (y * w + x) * 4;
+ const there = (Math.max(0, Math.min(h - 1, y + dy)) * w + Math.max(0, Math.min(w - 1, x + dx))) * 4;
+ const diff = lightnessOf(px[at]!, px[at + 1]!, px[at + 2]!) - lightnessOf(px[there]!, px[there + 1]!, px[there + 2]!);
+ const v = clamp01(0.5 + diff * amount);
+ const alpha = px[at + 3]!;
+ out[at] = out[at + 1] = out[at + 2] = v * alpha;
+ out[at + 3] = alpha;
+ }
+ }
+ return out;
+ }
+ case 'ripple': {
+ const [amount, size, seed] = [num(p, 'amount', 4) * k, num(p, 'size', 24) * k, num(p, 'seed', 1)];
+ return warped(px, w, h, (x, y) => [x + Math.sin((y / size) * Math.PI * 2 + seed) * amount, y + Math.sin((x / size) * Math.PI * 2 + seed * 1.7) * amount]);
+ }
+ case 'displace': {
+ const [amount, size, seed] = [num(p, 'amount', 8) * k, num(p, 'size', 40) * k, num(p, 'seed', 1)];
+ return warped(px, w, h, (x, y) => [x + (fractalNoise(x / size, y / size, 3, seed) - 0.5) * amount * 2, y + (fractalNoise(x / size, y / size, 3, seed + 7) - 0.5) * amount * 2]);
+ }
+ case 'twirl': {
+ const [cx, cy] = centre();
+ const [radius, angle] = [num(p, 'radius', Math.min(w, h) / (2 * k)) * k, (num(p, 'angle', 45) * Math.PI) / 180];
+ return warped(px, w, h, (x, y) => {
+ const [dx, dy] = [x - cx, y - cy];
+ const d = Math.hypot(dx, dy);
+ if (d >= radius) return [x, y];
+ const a = angle * (1 - d / radius) ** 2;
+ return [cx + dx * Math.cos(a) - dy * Math.sin(a), cy + dx * Math.sin(a) + dy * Math.cos(a)];
+ });
+ }
+ case 'pixelate': {
+ const cell = Math.max(2, Math.round(num(p, 'cell', 8) * k));
+ const out = new Float32Array(px.length);
+ for (let gy = 0; gy < h; gy += cell) {
+ for (let gx = 0; gx < w; gx += cell) {
+ const sum = [0, 0, 0, 0];
+ let n = 0;
+ for (let y = gy; y < Math.min(h, gy + cell); y++) for (let x = gx; x < Math.min(w, gx + cell); x++) { const at = (y * w + x) * 4; for (let c = 0; c < 4; c++) sum[c]! += px[at + c]!; n++; }
+ for (let y = gy; y < Math.min(h, gy + cell); y++) for (let x = gx; x < Math.min(w, gx + cell); x++) { const at = (y * w + x) * 4; for (let c = 0; c < 4; c++) out[at + c] = sum[c]! / n; }
+ }
+ }
+ return out;
+ }
+ case 'chromatic': {
+ const [cx, cy] = centre();
+ const amount = num(p, 'amount', 2) * k / Math.max(w, h);
+ const out = new Float32Array(px);
+ const one = new Float32Array(4);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const at = (y * w + x) * 4;
+ const [dx, dy] = [x + 0.5 - cx, y + 0.5 - cy];
+ bilinear(px, w, h, cx + dx * (1 + amount), cy + dy * (1 + amount), one, 0);
+ out[at] = one[0]!;
+ bilinear(px, w, h, cx + dx * (1 - amount), cy + dy * (1 - amount), one, 0);
+ out[at + 2] = one[2]!;
+ }
+ }
+ return out;
+ }
+ case 'vignette': {
+ const [cx, cy] = centre();
+ const [amount, inner, outer] = [num(p, 'amount', 0.6), num(p, 'radius', 0.55), num(p, 'outer', 1.05)];
+ const col = parseColor((p.color as string | undefined) ?? '#000000');
+ const reach = Math.hypot(w, h) / 2;
+ const out = new Float32Array(px);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const at = (y * w + x) * 4;
+ const t = amount * smooth(clamp01((Math.hypot(x + 0.5 - cx, y + 0.5 - cy) / reach - inner) / Math.max(0.01, outer - inner)));
+ for (let n = 0; n < 3; n++) out[at + n] = px[at + n]! + (col[n]! * px[at + 3]! - px[at + n]!) * t;
+ }
+ }
+ return out;
+ }
+ case 'fog': {
+ // Thick at `to`, clear at `from`, in drifting patches: air between things, over and not behind them.
+ const col = parseColor((p.color as string | undefined) ?? '#c8d0e0');
+ const [fx, fy] = [pointOf(p, 'from', [0, 0])[0] * k, pointOf(p, 'from', [0, 0])[1] * k];
+ const [tx, ty] = [pointOf(p, 'to', [0, h / k])[0] * k, pointOf(p, 'to', [0, h / k])[1] * k];
+ const [dx, dy] = [tx - fx, ty - fy];
+ const len = dx * dx + dy * dy || 1;
+ const [density, patch, size, seed] = [num(p, 'density', 0.6), num(p, 'patchiness', 0.5), num(p, 'size', 120) * k, num(p, 'seed', 1)];
+ const out = new Float32Array(px);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const at = (y * w + x) * 4;
+ const along = clamp01(((x + 0.5 - fx) * dx + (y + 0.5 - fy) * dy) / len);
+ const cloud = patch > 0 ? 1 - patch + patch * 2 * fractalNoise(x / (size * 2), y / size, 4, seed) : 1;
+ const t = clamp01(smooth(along) * density * cloud) * col[3];
+ for (let n = 0; n < 3; n++) out[at + n] = px[at + n]! * (1 - t) + col[n]! * t;
+ out[at + 3] = px[at + 3]! * (1 - t) + t;
+ }
+ }
+ return out;
+ }
+ case 'rays': {
+ // What is bright, drawn out from a point: light through air, in the colour of what shines.
+ const [cx, cy] = centre();
+ const [threshold, length, strength, decay] = [num(p, 'threshold', 0.6), num(p, 'length', 0.6), num(p, 'strength', 0.8), num(p, 'decay', 0.96)];
+ const bright = new Float32Array(px.length);
+ for (let i = 0; i < w * h; i++) {
+ const at = i * 4;
+ const over = clamp01((Math.max(px[at]!, px[at + 1]!, px[at + 2]!) - threshold) / Math.max(0.01, 1 - threshold));
+ for (let n = 0; n < 3; n++) bright[at + n] = px[at + n]! * over;
+ bright[at + 3] = 1;
+ }
+ const STEPS = 48;
+ const one = new Float32Array(4);
+ return screened(px, w, h, (x, y, into) => {
+ let weight = 1;
+ for (let s = 0; s < STEPS; s++) {
+ const t = 1 - (s / STEPS) * length;
+ bilinear(bright, w, h, cx + (x - cx) * t, cy + (y - cy) * t, one, 0);
+ for (let n = 0; n < 3; n++) into[n]! += one[n]! * weight * strength / STEPS;
+ weight *= decay;
+ }
+ });
+ }
+ case 'flare': {
+ const [cx, cy] = centre();
+ const col = parseColor((p.color as string | undefined) ?? '#ffe0b0');
+ const [size, strength] = [num(p, 'size', 60) * k, num(p, 'strength', 1)];
+ const [mx, my] = [w / 2, h / 2];
+ const ghosts = [[-0.4, 0.22, 0.14], [-0.8, 0.12, 0.2], [-1.3, 0.34, 0.09], [-1.7, 0.17, 0.13]] as const;
+ return screened(px, w, h, (x, y, into) => {
+ const d = Math.hypot(x - cx, y - cy);
+ let v = 0.9 * Math.exp(-((d / size) ** 2)) + 0.25 * Math.exp(-d / (size * 2.5));
+ // A streak along the sensor, and ghosts in a line through the middle of the frame.
+ v += 0.5 * Math.exp(-Math.abs(y - cy) / (size * 0.05)) * Math.exp(-Math.abs(x - cx) / (size * 3.5));
+ for (const [t, r, a] of ghosts) {
+ const [gx, gy] = [cx + (mx - cx) * (1 - t), cy + (my - cy) * (1 - t)];
+ const gd = Math.hypot(x - gx, y - gy) / (size * r * 2.2);
+ if (gd < 1) v += a * (1 - gd * gd * gd * gd);
+ }
+ for (let n = 0; n < 3; n++) into[n]! += col[n]! * v * strength;
+ });
+ }
+ case 'texture': {
+ // A relief lit from one side and not a stain: canvas, paper, plaster.
+ const [scale, amount, seed] = [num(p, 'scale', 3) * k, num(p, 'amount', 0.3), num(p, 'seed', 1)];
+ const a = (num(p, 'angle', 135) * Math.PI) / 180;
+ const [lx, ly] = [Math.cos(a), Math.sin(a)];
+ const out = new Float32Array(px);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const at = (y * w + x) * 4;
+ const v = fractalNoise(x / scale, y / scale, 3, seed);
+ const slope = v - fractalNoise((x + lx) / scale, (y + ly) / scale, 3, seed);
+ const shade = 1 + slope * amount * 3;
+ const alpha = px[at + 3]!;
+ for (let n = 0; n < 3; n++) out[at + n] = Math.max(0, Math.min(alpha, px[at + n]! * shade));
+ }
+ }
+ return out;
+ }
+ default:
+ throw new Error(`no filter "${String(name)}"`);
+ }
+}
diff --git a/pipeline/src/image/imageRaster.ts b/pipeline/src/image/imageRaster.ts
index 232ac9ece..b9e7514c2 100644
--- a/pipeline/src/image/imageRaster.ts
+++ b/pipeline/src/image/imageRaster.ts
@@ -7,6 +7,7 @@
* effects, and this owns everything that has to be exact: antialiased edges, gradients, soft shadows.
* On the CPU rather than through a canvas, so the same source makes the same bytes on any machine and a test can read them.
*/
+import { applyAdjust, applyFilter, lightnessOf } from './imageAdjust';
import { figureLight, flattenFigure, type FlatFigure } from './imageFigure';
import { IMAGE_BRUSHES, isGuideLayer, MAX_MARK_STROKES } from './imageSource';
import type { ImageFill, ImageLayer, ImageSource, ImageAlong, ImageBrushSettings, ImageEffect, ImageField, ImageMark, ImageStroke, ImageSweep, ImageTransform } from './imageSource';
@@ -372,7 +373,7 @@ function strokeMask(runs: readonly { points: readonly Pt[]; closed: boolean }[],
}
/** Three box blurs, which is a gaussian to the eye at a cost that does not grow with the radius. */
-function blurMask(src: Float32Array, w: number, h: number, sigma: number): Float32Array {
+export function blurMask(src: Float32Array, w: number, h: number, sigma: number): Float32Array {
if (sigma <= 0) return src;
const r = Math.max(1, Math.round(sigma * 1.2));
let cur = src;
@@ -448,7 +449,7 @@ function gradientNoise(x: number, y: number, seed: number): number {
/** Spread out to 0..1, so that the stops at the ends are reached. */
const NOISE_CONTRAST = [1.15, 1.3, 1.4, 1.45, 1.45, 1.45, 1.45, 1.45];
-function fractalNoise(x: number, y: number, octaves: number, seed: number): number {
+export function fractalNoise(x: number, y: number, octaves: number, seed: number): number {
let [sum, weight, amplitude, frequency] = [0, 0, 1, 1];
for (let i = 0; i < octaves; i++) {
// Off the lattice, where this noise is always nothing.
@@ -621,16 +622,65 @@ class Canvas {
}
type Blend = NonNullable<ImageLayer['blend']>;
+const softLight = (b: number, s: number): number => {
+ if (s <= 0.5) return b - (1 - 2 * s) * b * (1 - b);
+ const d = b <= 0.25 ? ((16 * b - 12) * b + 4) * b : Math.sqrt(b);
+ return b + (2 * s - 1) * (d - b);
+};
+const dodge = (b: number, s: number): number => (b <= 0 ? 0 : s >= 1 ? 1 : Math.min(1, b / (1 - s)));
+const burn = (b: number, s: number): number => (b >= 1 ? 1 : s <= 0 ? 0 : 1 - Math.min(1, (1 - b) / s));
+const hard = (b: number, s: number): number => (s < 0.5 ? 2 * b * s : 1 - 2 * (1 - b) * (1 - s));
const blendChannel = (mode: Blend, b: number, s: number): number => {
switch (mode) {
case 'multiply': return b * s;
case 'screen': return b + s - b * s;
- case 'overlay': return b < 0.5 ? 2 * b * s : 1 - 2 * (1 - b) * (1 - s);
+ case 'overlay': return hard(s, b);
case 'add': return Math.min(1, b + s);
+ case 'softLight': return softLight(b, s);
+ case 'hardLight': return hard(b, s);
+ case 'vividLight': return s < 0.5 ? burn(b, 2 * s) : dodge(b, 2 * s - 1);
+ case 'linearLight': return Math.max(0, Math.min(1, b + 2 * s - 1));
+ case 'pinLight': return s < 0.5 ? Math.min(b, 2 * s) : Math.max(b, 2 * s - 1);
+ case 'colorDodge': return dodge(b, s);
+ case 'colorBurn': return burn(b, s);
+ case 'linearBurn': return Math.max(0, b + s - 1);
+ case 'darken': return Math.min(b, s);
+ case 'lighten': return Math.max(b, s);
+ case 'difference': return Math.abs(b - s);
+ case 'exclusion': return b + s - 2 * b * s;
+ case 'subtract': return Math.max(0, b - s);
+ case 'divide': return s <= 0 ? 1 : Math.min(1, b / s);
default: return s;
}
};
+/** Keeping a colour's hue and saturation in range when its lightness is set, as the colour blend modes need. */
+function withLight(c: readonly [number, number, number], light: number): [number, number, number] {
+ const d = light - lightnessOf(c[0], c[1], c[2]);
+ const out: [number, number, number] = [c[0] + d, c[1] + d, c[2] + d];
+ const l = lightnessOf(out[0], out[1], out[2]);
+ const [lo, hi] = [Math.min(...out), Math.max(...out)];
+ if (lo < 0) for (let i = 0; i < 3; i++) out[i] = l + ((out[i]! - l) * l) / (l - lo);
+ if (hi > 1) for (let i = 0; i < 3; i++) out[i] = l + ((out[i]! - l) * (1 - l)) / (hi - l);
+ return out;
+}
+const colourfulness = (c: readonly [number, number, number]): number => Math.max(...c) - Math.min(...c);
+function withColourfulness(c: readonly [number, number, number], sat: number): [number, number, number] {
+ const [lo, hi] = [Math.min(...c), Math.max(...c)];
+ if (hi === lo) return [0, 0, 0];
+ return [((c[0] - lo) * sat) / (hi - lo), ((c[1] - lo) * sat) / (hi - lo), ((c[2] - lo) * sat) / (hi - lo)];
+}
+/** The modes that work on a colour whole: this hue on that lightness, and so on. */
+function blendWhole(mode: Blend, d: readonly [number, number, number], s: readonly [number, number, number]): [number, number, number] {
+ switch (mode) {
+ case 'hue': return withLight(withColourfulness(s, colourfulness(d)), lightnessOf(d[0], d[1], d[2]));
+ case 'saturation': return withLight(withColourfulness(d, colourfulness(s)), lightnessOf(d[0], d[1], d[2]));
+ case 'color': return withLight(s, lightnessOf(d[0], d[1], d[2]));
+ default: return withLight(d, lightnessOf(s[0], s[1], s[2]));
+ }
+}
+const WHOLE_MODES: ReadonlySet<Blend> = new Set<Blend>(['hue', 'saturation', 'color', 'luminosity']);
+
/** @p color straight, at coverage @p m, over what is there. */
function put(c: Canvas, i: number, r: number, g: number, b: number, a: number, mode: Blend): void {
if (a <= 0) return;
@@ -640,17 +690,59 @@ function put(c: Canvas, i: number, r: number, g: number, b: number, a: number, m
p[at] = p[at]! * (1 - a) + r * a; p[at + 1] = p[at + 1]! * (1 - a) + g * a; p[at + 2] = p[at + 2]! * (1 - a) + b * a;
} else {
const [dr, dg, db] = [p[at]! / da, p[at + 1]! / da, p[at + 2]! / da];
+ if (WHOLE_MODES.has(mode)) {
+ const whole = blendWhole(mode, [dr, dg, db], [r, g, b]);
+ const [mr, mg, mb] = [(1 - da) * r + da * whole[0], (1 - da) * g + da * whole[1], (1 - da) * b + da * whole[2]];
+ p[at] = p[at]! * (1 - a) + mr * a; p[at + 1] = p[at + 1]! * (1 - a) + mg * a; p[at + 2] = p[at + 2]! * (1 - a) + mb * a;
+ p[at + 3] = da + a - da * a;
+ return;
+ }
const mix = (d: number, s: number): number => (1 - da) * s + da * blendChannel(mode, d, s);
p[at] = p[at]! * (1 - a) + mix(dr, r) * a; p[at + 1] = p[at + 1]! * (1 - a) + mix(dg, g) * a; p[at + 2] = p[at + 2]! * (1 - a) + mix(db, b) * a;
}
p[at + 3] = da + a - da * a;
}
+/** Raised edges from how the shape's edge slopes: lit on the side toward the light, and shaded away from it. */
+function bevelEffect(e: ImageEffect, c: Canvas, mask: Float32Array): void {
+ const { w, h } = c;
+ const soft = blurMask(mask, w, h, Math.max(0.5, (e.size ?? 4) / 2));
+ const a = ((e.angle ?? 225) * Math.PI) / 180;
+ const [lx, ly] = [Math.cos(a), Math.sin(a)];
+ const [hr, hg, hb, ha] = parseColor(e.color ?? '#ffffff');
+ const [sr, sg, sb, sa] = parseColor(e.shade ?? '#000000');
+ const gain = (e.depth ?? 1) * (e.size ?? 4) * 2;
+ const op = e.opacity ?? 0.8;
+ for (let y = 1; y < h - 1; y++) {
+ for (let x = 1; x < w - 1; x++) {
+ const i = y * w + x;
+ if (mask[i]! <= 0) continue;
+ // The edge rises inward, so it faces against its slope.
+ const facing = -((soft[i + 1]! - soft[i - 1]!) * lx + (soft[i + w]! - soft[i - w]!) * ly) * gain;
+ if (facing > 0) put(c, i, hr, hg, hb, Math.min(1, facing) * op * ha * mask[i]!, 'screen');
+ else if (facing < 0) put(c, i, sr, sg, sb, Math.min(1, -facing) * op * sa * mask[i]!, 'multiply');
+ }
+ }
+}
+
function effectLayers(effects: readonly ImageEffect[] | undefined, under: boolean, c: Canvas, mask: Float32Array): void {
const { w, h } = c;
for (const e of effects ?? []) {
const isUnder = e.type === 'shadow' || e.type === 'glow';
if (isUnder !== under) continue;
+ if (e.type === 'bevel') { bevelEffect(e, c, mask); continue; }
+ if (e.type === 'overlay') {
+ const paint = paintOf(e.fill!, IDENTITY);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ const i = y * w + x;
+ if (mask[i]! <= 0) continue;
+ const col = paint(x + 0.5, y + 0.5);
+ put(c, i, col[0], col[1], col[2], col[3] * mask[i]! * (e.opacity ?? 1), e.blend ?? 'normal');
+ }
+ }
+ continue;
+ }
const [r, g, b, ca] = parseColor(e.color ?? (e.type === 'glow' || e.type === 'innerGlow' ? '#ffffff' : '#000000'));
const op = (e.opacity ?? 1) * ca;
const sigma = e.blur ?? (e.type === 'glow' ? 8 : 4);
@@ -728,6 +820,13 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):
const blend: Blend = layer.blend ?? 'normal';
const named = (id: string): Float32Array | undefined => drawn.masks.get(id.replace(/^@/, ''));
let clip = layer.clip ? named(layer.clip) : undefined;
+ const softened = (mask: Float32Array): Float32Array => {
+ let out = mask;
+ if (layer.clipGrow) out = grownMask(out, layer.clipGrow * drawnScale(parent), w, h);
+ if (layer.clipFeather) out = blurMask(out, w, h, layer.clipFeather * drawnScale(parent));
+ return layer.clipInvert ? out.map((v) => 1 - v) : out;
+ };
+ if (clip) clip = softened(clip);
// Nothing, and not everything: a clip that failed would otherwise cover the picture in what it was meant to hold in.
if (layer.clip && !clip) {
drawn.notes.push(`layer ${layer.id ?? layer.shape}: clip "${layer.clip}" names no layer drawn before it, so it drew nothing`);
@@ -739,8 +838,58 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):
else clip = out.map((v, i) => (1 - v) * (clip ? clip[i]! : 1));
}
+ if (layer.blendIf) {
+ // Reads the picture as it stands now, so a highlight is chosen by being one.
+ const [low, high, soft = 0.1] = layer.blendIf as [number, number, number?];
+ const chosen = new Float32Array(w * h);
+ for (let i = 0; i < chosen.length; i++) {
+ const a = c.px[i * 4 + 3]!;
+ const l = a > 0 ? lightnessOf(c.px[i * 4]! / a, c.px[i * 4 + 1]! / a, c.px[i * 4 + 2]! / a) : 0;
+ const v = Math.min(soft > 0 ? (l - low) / soft + 1 : l >= low ? 1 : 0, soft > 0 ? (high - l) / soft + 1 : l <= high ? 1 : 0);
+ chosen[i] = Math.max(0, Math.min(1, v)) * (clip ? clip[i]! : 1);
+ }
+ clip = chosen;
+ }
+
+ if (layer.feather) {
+ const { feather, opacity: _o, blend: _b, clip: _c, except: _e, blendIf: _f, clipFeather: _g, clipGrow: _h, clipInvert: _i, ...sharp } = layer;
+ const inner = new Canvas(w, h);
+ drawLayer(inner, sharp, parent, drawn);
+ const alpha = new Float32Array(w * h);
+ for (let i = 0; i < alpha.length; i++) alpha[i] = inner.px[i * 4 + 3]!;
+ const soft = blurMask(alpha, w, h, feather * drawnScale(parent));
+ const mask = new Float32Array(w * h);
+ for (let i = 0; i < alpha.length; i++) {
+ const a = alpha[i]!;
+ if (a <= 0) continue;
+ // Fading in from the edge: half way down its blur is the edge, and all of the colour there is the layer's own.
+ const kept = a * smoothed(Math.max(0, Math.min(1, (soft[i]! / a - 0.5) * 2.5)));
+ mask[i] = kept * (clip ? clip[i]! : 1);
+ put(c, i, inner.px[i * 4]! / a, inner.px[i * 4 + 1]! / a, inner.px[i * 4 + 2]! / a, mask[i]! * opacity, blend);
+ }
+ if (layer.id) drawn.masks.set(layer.id, mask);
+ return;
+ }
+
+ if (layer.shape === 'adjust' || layer.shape === 'filter') {
+ const k = drawnScale(parent);
+ const next = layer.shape === 'adjust' ? applyAdjust(c.px, w, h, layer.adjust!) : applyFilter(layer.filter!, layer.params ?? {}, c.px, w, h, k);
+ for (let i = 0; i < w * h; i++) {
+ const keep = opacity * (clip ? clip[i]! : 1);
+ if (keep <= 0) continue;
+ const at = i * 4;
+ if (blend !== 'normal') {
+ const a = next[at + 3]!;
+ if (a > 0) put(c, i, next[at]! / a, next[at + 1]! / a, next[at + 2]! / a, a * keep, blend);
+ continue;
+ }
+ for (let n = 0; n < 4; n++) c.px[at + n] = c.px[at + n]! + (next[at + n]! - c.px[at + n]!) * keep;
+ }
+ return;
+ }
+
if (layer.blur) {
- const { blur, opacity: _o, blend: _b, clip: _c, except: _e, ...sharp } = layer;
+ const { blur, opacity: _o, blend: _b, clip: _c, except: _e, blendIf: _f, clipFeather: _g, clipGrow: _h, clipInvert: _i, ...sharp } = layer;
const inner = new Canvas(w, h);
drawLayer(inner, sharp, parent, drawn);
// Blurred as stored, colour already times alpha: blurring straight colour would bring the black of empty pixels in round the edge.
@@ -822,11 +971,6 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):
return;
}
- if (layer.shape === 'repaint') {
- drawRepaint(c, layer, drawn, opacity, clip);
- return;
- }
-
if (layer.face && layer.shape === 'rect') {
const corners = faceCorners(layer, drawn);
if (!corners) return;
@@ -916,6 +1060,17 @@ const smoothed = (t: number): number => t * t * (3 - 2 * t);
interface PathPoint { x: number; y: number; pressure: number; along: number }
+/** The tooth of the paper, worked out once: every stroke with grain asks it at every pixel it touches. */
+let paper: { key: string; values: Float32Array } | null = null;
+function paperAt(w: number, h: number, scale: number): Float32Array {
+ const key = `${w}x${h}:${scale}`;
+ if (paper?.key === key) return paper.values;
+ const values = new Float32Array(w * h);
+ for (let y = 0; y < h; y++) for (let x = 0; x < w; x++) values[y * w + x] = fractalNoise((x + 0.5) / scale, (y + 0.5) / scale, 3, 77);
+ paper = { key, values };
+ return values;
+}
+
/** A stroke's points as a smooth line about a pixel apart, Catmull-Rom so it passes through each. */
function brushPath(points: readonly (readonly number[])[], m: Matrix): PathPoint[] {
const placed = points.map((p) => {
@@ -1004,7 +1159,8 @@ function paintStrand(c: Canvas, s: ImageStroke, path: readonly PathPoint[], stra
if (grain <= 0 && dry <= 0) return 1;
let kept = 1;
// A wide band and not a threshold: paper takes more or less pigment at each place, and specks of all or nothing read as static.
- if (grain > 0) kept *= Math.max(0, Math.min(1, (fractalNoise(x / grainScale, y / grainScale, 3, 77) - (0.12 + grain * 0.62)) / 0.34 + 0.5));
+ // A wide band and not a threshold: paper takes more or less pigment at each place, and specks of all or nothing read as static.
+ if (grain > 0) kept *= Math.max(0, Math.min(1, (paperAt(w, h, grainScale)[Math.floor(y) * w + Math.floor(x)]! - (0.12 + grain * 0.62)) / 0.34 + 0.5));
if (dry > 0) {
// Hairs of the brush: fine across its width and long along its travel.
const across = (x - cx) * nx + (y - cy) * ny + away;
@@ -1481,100 +1637,6 @@ function textOnQuad(layer: ImageLayer, m: Matrix, w: number, h: number, draw: Im
return out;
}
-/** [brush size at 960 pixels across, how wrong a cell must be, 0..255, before it gets a stroke]. */
-const REPAINT_LEVELS: readonly (readonly (readonly [number, number])[])[] = [
- [[26, 0], [13, 24], [6, 30]],
- [[22, 0], [11, 20], [6, 22], [3.5, 20]],
- [[20, 0], [10, 18], [5, 18], [3, 16], [2, 14]],
-];
-const REPAINT_BOOST: readonly (readonly [number, number])[] = [[3, 14], [1.8, 10]];
-
-/**
- * The picture so far painted again, coarse to fine, each stroke taking its colour from under where it starts and running along the edges there.
- * Shapes become brushwork without a stroke being written: what makes a built picture read as a painted one. It costs seconds, so it comes last.
- */
-function drawRepaint(c: Canvas, layer: ImageLayer, drawn: Drawn, opacity: number, clip: Float32Array | undefined): void {
- const { w, h } = c;
- const k = Math.max(w, h) / 960;
- const brush = layer.brush ?? 'oil';
- const seed = layer.seed ?? 1;
- const inner = new Canvas(w, h);
- const buffer: StrokeBuffer = { mask: new Float32Array(w * h), rgb: new Float32Array(w * h * 3), touched: [] };
- const boosted = (layer.boost ?? []).map((id) => {
- const mask = drawn.masks.get(id.replace(/^@/, ''));
- if (!mask) drawn.notes.push(`layer ${layer.id ?? 'repaint'}: boost "${id}" names no layer drawn before it`);
- return mask;
- }).filter((m): m is Float32Array => !!m);
- const passes = [
- ...REPAINT_LEVELS[layer.detail ?? 2]!.map(([size, tolerance]) => ({ size, tolerance, only: null as Float32Array[] | null })),
- ...(boosted.length ? REPAINT_BOOST.map(([size, tolerance]) => ({ size, tolerance, only: boosted })) : []),
- ];
- let count = 0;
- for (const pass of passes) {
- const size = pass.size * k;
- const sigma = Math.max(0.6, size / 3.2);
- const target = [0, 1, 2, 3].map((n) => {
- const channel = new Float32Array(w * h);
- for (let i = 0; i < channel.length; i++) channel[i] = c.px[i * 4 + n]!;
- return blurMask(channel, w, h, sigma);
- }) as [Float32Array, Float32Array, Float32Array, Float32Array];
- const light = new Float32Array(w * h);
- for (let i = 0; i < light.length; i++) light[i] = target[0][i]! * 0.3 + target[1][i]! * 0.59 + target[2][i]! * 0.11;
- const cell = Math.max(2, Math.round(size * 0.8));
- const [columns, rows] = [Math.ceil(w / cell), Math.ceil(h / cell)];
- // In no order: swept in rows, each stroke would lie the same way over the last.
- const order = Array.from({ length: columns * rows }, (_, i) => i).sort((a, b) => hash01(seed * 61 + a, count & 255) - hash01(seed * 61 + b, count & 255));
- const starts: { x: number; y: number }[] = [];
- for (const index of order) {
- const [gx, gy] = [(index % columns) * cell, Math.floor(index / columns) * cell];
- let [worst, bx, by, total, n] = [-1, gx, gy, 0, 0];
- for (let y = gy; y < Math.min(h, gy + cell); y++) {
- for (let x = gx; x < Math.min(w, gx + cell); x++) {
- const i = y * w + x;
- const e = Math.hypot(inner.px[i * 4]! - target[0][i]!, inner.px[i * 4 + 1]! - target[1][i]!, inner.px[i * 4 + 2]! - target[2][i]!) * 255;
- total += e; n++;
- if (e > worst) { worst = e; bx = x; by = y; }
- }
- }
- if (n === 0 || total / n < pass.tolerance) continue;
- if (pass.only && !pass.only.some((m) => m[by * w + bx]! >= 0.5)) continue;
- starts.push({ x: bx, y: by });
- }
- for (const { x: bx, y: by } of starts) {
- const at = by * w + bx;
- const alpha = target[3][at]!;
- if (alpha < 0.02) continue;
- const color: Rgba = [target[0][at]! / alpha, target[1][at]! / alpha, target[2][at]! / alpha, alpha];
- const points: number[][] = [[bx, by]];
- let [x, y, dx, dy] = [bx, by, 0, 0];
- for (let step = 0; step < 5; step++) {
- const [ix, iy] = [Math.max(1, Math.min(w - 2, Math.round(x))), Math.max(1, Math.min(h - 2, Math.round(y)))];
- const [gx, gy] = [light[iy * w + ix + 1]! - light[iy * w + ix - 1]!, light[(iy + 1) * w + ix]! - light[(iy - 1) * w + ix]!];
- const g = Math.hypot(gx, gy);
- // Along an edge where there is one, and a way of its own where there is not.
- const free = hash01(seed * 67 + bx, by) * Math.PI * 2;
- let [nx, ny] = g < 0.006 ? [Math.cos(free), Math.sin(free)] : [-gy / g, gx / g];
- if (nx * dx + ny * dy < 0) [nx, ny] = [-nx, -ny];
- [dx, dy] = dx === 0 && dy === 0 ? [nx, ny] : [dx * 0.5 + nx * 0.5, dy * 0.5 + ny * 0.5];
- const len = Math.hypot(dx, dy) || 1;
- x += (dx / len) * size * 0.9; y += (dy / len) * size * 0.9;
- if (x < 0 || y < 0 || x >= w || y >= h) break;
- const here = Math.round(y) * w + Math.round(x);
- if (Math.hypot(target[0][here]! - target[0][at]!, target[1][here]! - target[1][at]!, target[2][here]! - target[2][at]!) * 255 > 45) break;
- points.push([x, y]);
- }
- paintStroke(inner, { brush, points, color: toHex(color), size, seed: seed + count }, IDENTITY, 1, count, buffer);
- count++;
- }
- }
- if (layer.id) drawn.painted.set(layer.id, { entries: passes.length, strokes: count });
- for (let i = 0; i < w * h; i++) {
- const a = inner.px[i * 4 + 3]!;
- if (a <= 0) continue;
- put(c, i, inner.px[i * 4]! / a, inner.px[i * 4 + 1]! / a, inner.px[i * 4 + 2]! / a, a * opacity * (clip ? clip[i]! : 1), layer.blend ?? 'normal');
- }
-}
-
/** What is bright in the picture so far, spilling over what is round it: a tight glow and a wide one, as a lens and the air both give. */
function drawBloom(c: Canvas, layer: ImageLayer, parent: Matrix, opacity: number, clip: Float32Array | undefined): void {
const { w, h } = c;
@@ -1649,6 +1711,30 @@ function wornText(mask: Float32Array, layer: ImageLayer, w: number): Float32Arra
});
}
+/** A mask made wider by @p pixels, or narrower if they are less than none: the highest or lowest within reach, across and then down. */
+function grownMask(mask: Float32Array, pixels: number, w: number, h: number): Float32Array {
+ const r = Math.round(Math.abs(pixels));
+ if (r === 0) return mask;
+ const pick = pixels > 0 ? Math.max : Math.min;
+ const across = new Float32Array(w * h);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ let v = mask[y * w + x]!;
+ for (let d = -r; d <= r; d++) { const at = x + d; v = pick(v, at < 0 || at >= w ? (pixels > 0 ? 0 : v) : mask[y * w + at]!); }
+ across[y * w + x] = v;
+ }
+ }
+ const out = new Float32Array(w * h);
+ for (let x = 0; x < w; x++) {
+ for (let y = 0; y < h; y++) {
+ let v = across[y * w + x]!;
+ for (let d = -r; d <= r; d++) { const at = y + d; v = pick(v, at < 0 || at >= h ? (pixels > 0 ? 0 : v) : across[at * w + x]!); }
+ out[y * w + x] = v;
+ }
+ }
+ return out;
+}
+
function drawText(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {
const { w, h } = c;
if (!drawn.text) throw new Error(`layer ${layer.id ?? 'text'}: text needs the editor's fonts, and this render has none`);
diff --git a/pipeline/src/image/imageSource.ts b/pipeline/src/image/imageSource.ts
index 61eae0eaf..97f9b3418 100644
--- a/pipeline/src/image/imageSource.ts
+++ b/pipeline/src/image/imageSource.ts
@@ -5,6 +5,101 @@
*/
import { parsePath } from './imageRaster';
+export const IMAGE_BLENDS = [
+ 'normal', 'multiply', 'screen', 'overlay', 'add', 'softLight', 'hardLight', 'vividLight', 'linearLight', 'pinLight', 'colorDodge', 'colorBurn', 'linearBurn',
+ 'darken', 'lighten', 'difference', 'exclusion', 'subtract', 'divide', 'hue', 'saturation', 'color', 'luminosity',
+] as const;
+export type ImageBlend = (typeof IMAGE_BLENDS)[number];
+
+/** Adjustments of the picture so far. Any of them together, applied in this order. */
+export interface ImageAdjust {
+ /** Stops, as a photo's: 1 is twice as bright. */
+ exposure?: number;
+ levels?: { input?: readonly [number, number]; gamma?: number; output?: readonly [number, number] };
+ /** [[in, out]...] 0..1 for all channels, or { r, g, b } for each. */
+ curves?: readonly (readonly number[])[] | { r?: readonly (readonly number[])[]; g?: readonly (readonly number[])[]; b?: readonly (readonly number[])[] };
+ brightness?: number;
+ contrast?: number;
+ temperature?: number;
+ tint?: number;
+ /** Each [r, g, b], -1..1, pushing that part of the tonal range. */
+ balance?: { shadows?: readonly number[]; midtones?: readonly number[]; highlights?: readonly number[] };
+ /** Degrees round the wheel, and -1..1 each. */
+ hue?: number;
+ saturation?: number;
+ lightness?: number;
+ vibrance?: number;
+ /** true, or how much of [r, g, b] each makes the grey. */
+ blackWhite?: boolean | readonly number[];
+ gradientMap?: readonly (readonly [number, string])[];
+ invert?: boolean;
+ posterize?: number;
+ threshold?: number;
+}
+
+const ADJUST_RANGES: Readonly<Record<string, readonly [number, number]>> = { exposure: [-8, 8], brightness: [-1, 1], contrast: [-1, 1], temperature: [-1, 1], tint: [-1, 1], hue: [-360, 360], saturation: [-1, 1], lightness: [-1, 1], vibrance: [-1, 1], posterize: [2, 256], threshold: [0, 1] };
+
+function adjustProblem(a: unknown): string | null {
+ if (typeof a !== 'object' || a === null) return 'must be an object of what to change';
+ const o = a as Record<string, unknown>;
+ const known = [...Object.keys(ADJUST_RANGES), 'levels', 'curves', 'balance', 'blackWhite', 'gradientMap', 'invert'];
+ for (const key of Object.keys(o)) if (!known.includes(key)) return `has unknown "${key}"; there are ${known.join(', ')}`;
+ if (Object.keys(o).length === 0) return 'changes nothing';
+ for (const [key, [lo, hi]] of Object.entries(ADJUST_RANGES)) if (o[key] !== undefined && !(isNum(o[key]) && (o[key] as number) >= lo && (o[key] as number) <= hi)) return `${key} must be ${lo}..${hi}`;
+ const unitPair = (v: unknown): boolean => isPair(v) && (v as number[]).every((n) => n >= 0 && n <= 1);
+ const curve = (v: unknown): boolean => Array.isArray(v) && v.length >= 2 && v.every(unitPair);
+ if (o.levels !== undefined) {
+ const l = o.levels as Record<string, unknown>;
+ if (typeof l !== 'object' || l === null) return 'levels must be { input?, gamma?, output? }';
+ for (const key of Object.keys(l)) if (!['input', 'gamma', 'output'].includes(key)) return `levels has unknown "${key}"; there are input, gamma, output`;
+ for (const key of ['input', 'output']) if (l[key] !== undefined && !unitPair(l[key])) return `levels ${key} must be [low, high], each 0..1`;
+ if (l.gamma !== undefined && !(isNum(l.gamma) && (l.gamma as number) > 0)) return 'levels gamma must be positive';
+ }
+ if (o.curves !== undefined) {
+ if (Array.isArray(o.curves)) { if (!curve(o.curves)) return 'curves must be two or more [in, out], each 0..1'; }
+ else {
+ const c = o.curves as Record<string, unknown>;
+ if (typeof c !== 'object' || c === null) return 'curves must be [[in, out]...] or { r, g, b }';
+ for (const [key, v] of Object.entries(c)) { if (!['r', 'g', 'b'].includes(key)) return `curves has unknown "${key}"; there are r, g, b`; if (!curve(v)) return `curves ${key} must be two or more [in, out], each 0..1`; }
+ }
+ }
+ if (o.balance !== undefined) {
+ for (const [key, v] of Object.entries(o.balance as Record<string, unknown>)) {
+ if (!['shadows', 'midtones', 'highlights'].includes(key)) return `balance has unknown "${key}"; there are shadows, midtones, highlights`;
+ if (!(Array.isArray(v) && v.length === 3 && v.every(isNum))) return `balance ${key} must be [r, g, b]`;
+ }
+ }
+ if (o.blackWhite !== undefined && typeof o.blackWhite !== 'boolean' && !(Array.isArray(o.blackWhite) && o.blackWhite.length === 3 && o.blackWhite.every(isNum))) return 'blackWhite must be true or [r, g, b]';
+ if (o.invert !== undefined && typeof o.invert !== 'boolean') return 'invert must be true or false';
+ if (o.gradientMap !== undefined && !(Array.isArray(o.gradientMap) && o.gradientMap.length >= 2 && o.gradientMap.every((s) => Array.isArray(s) && s.length === 2 && isNum(s[0]) && isColor(s[1])))) return 'gradientMap must be two or more [0..1, colour]';
+ return null;
+}
+
+/** Every filter and what it takes. A number but for: `at`, `from`, `to` a place; `color` a colour; `mono` true or false. */
+export const IMAGE_FILTERS = {
+ gaussian: ['radius'], motion: ['angle', 'distance'], spin: ['at', 'angle'], zoom: ['at', 'amount'], lens: ['radius', 'highlights'],
+ sharpen: ['amount', 'radius', 'threshold'], highPass: ['radius'], noise: ['amount', 'scale', 'mono', 'seed'], median: ['radius'], emboss: ['angle', 'height', 'amount'],
+ ripple: ['amount', 'size', 'seed'], displace: ['amount', 'size', 'seed'], twirl: ['at', 'radius', 'angle'], pixelate: ['cell'], chromatic: ['at', 'amount'],
+ vignette: ['at', 'amount', 'radius', 'outer', 'color'], fog: ['color', 'density', 'from', 'to', 'patchiness', 'size', 'seed'], rays: ['at', 'threshold', 'length', 'strength', 'decay'],
+ flare: ['at', 'size', 'color', 'strength'], texture: ['scale', 'amount', 'angle', 'seed'],
+} as const;
+export type ImageFilterName = keyof typeof IMAGE_FILTERS;
+
+function filterProblem(name: unknown, params: unknown): string | null {
+ if (typeof name !== 'string' || !Object.hasOwn(IMAGE_FILTERS, name)) return `filter ${JSON.stringify(name)} must be one of ${Object.keys(IMAGE_FILTERS).join(', ')}`;
+ if (params === undefined) return null;
+ if (typeof params !== 'object' || params === null) return 'params must be an object';
+ const known = IMAGE_FILTERS[name as ImageFilterName] as readonly string[];
+ for (const [key, v] of Object.entries(params)) {
+ if (!known.includes(key)) return `filter ${name} has unknown "${key}"; it takes ${known.join(', ')}`;
+ if (key === 'color') { if (!isColor(v)) return `filter ${name}: color must be #rgb..#rrggbbaa`; }
+ else if (key === 'at' || key === 'from' || key === 'to') { if (!isPair(v)) return `filter ${name}: ${key} must be [x, y]`; }
+ else if (key === 'mono') { if (typeof v !== 'boolean') return `filter ${name}: mono must be true or false`; }
+ else if (!isNum(v)) return `filter ${name}: ${key} must be a number`;
+ }
+ return null;
+}
+
export const IMAGE_SOURCE_EXT = '.esimage';
export const IMAGE_SOURCE_VERSION = 1;
export const MAX_IMAGE_SIZE = 2048;
@@ -38,7 +133,15 @@ export type ImageFill = string
};
export interface ImageEffect {
- type: 'shadow' | 'glow' | 'innerShadow' | 'innerGlow';
+ type: 'shadow' | 'glow' | 'innerShadow' | 'innerGlow' | 'bevel' | 'overlay';
+ /** A bevel: how wide, the light's direction in degrees, how strong, and the colour on its dark side. */
+ size?: number;
+ angle?: number;
+ depth?: number;
+ shade?: string;
+ /** An overlay: what covers the shape. */
+ fill?: ImageFill;
+ blend?: ImageBlend;
color?: string;
x?: number;
y?: number;
@@ -55,7 +158,7 @@ export interface ImageTransform {
origin?: readonly [number, number];
}
-export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'figure' | 'image' | 'text' | 'block' | 'perspective' | 'reflect' | 'bloom' | 'repaint' | 'group';
+export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'figure' | 'image' | 'text' | 'block' | 'perspective' | 'reflect' | 'bloom' | 'adjust' | 'filter' | 'group';
/** How a brush lays colour. Any of these on a stroke, a sweep or a field, over what its `brush` starts from. */
export interface ImageBrushSettings {
@@ -133,7 +236,7 @@ export interface ImageStroke extends ImageBrushSettings {
/** 0..1: drags what is already on this layer along, instead of laying colour. */
smudge?: number;
erase?: boolean;
- blend?: 'normal' | 'multiply' | 'screen' | 'overlay' | 'add';
+ blend?: ImageBlend;
}
/** Many hairs between guide lines, gathered into locks. A few words of intent, and hundreds of strokes when drawn. */
@@ -264,12 +367,17 @@ export interface ImageLayer {
within?: readonly number[];
/** Text: four corners in the picture it is laid on, in order round from the top left, so it lies on a wall in perspective. */
quad?: readonly (readonly number[])[];
- /** A repaint: the brush the whole picture so far is painted again with. */
- brush?: string;
- /** A repaint: how far down its strokes go, 0 broad to 2 fine, and what gets finer strokes still. */
- detail?: number;
- boost?: readonly string[];
- seed?: number;
+ /** Pixels of soft edge on the layer, fading inward, keeping its colour where a blur would smear it. */
+ feather?: number;
+ /** What it clips to, softened, grown or shrunk, or turned inside out. */
+ clipFeather?: number;
+ clipGrow?: number;
+ clipInvert?: boolean;
+ /** Only where the picture so far is this light, [low, high, softness?] each 0..1: a highlight or shadow selected by what it is. */
+ blendIf?: readonly number[];
+ adjust?: ImageAdjust;
+ filter?: ImageFilterName;
+ params?: Readonly<Record<string, unknown>>;
/** A bloom: how bright a pixel must be to shine, 0..1, and how much it does. */
threshold?: number;
strength?: number;
@@ -295,7 +403,7 @@ export interface ImageLayer {
fill?: ImageFill;
stroke?: { color: ImageFill; width: number; align?: 'center' | 'inside' | 'outside'; join?: 'round' | 'miter' | 'bevel'; cap?: 'round' | 'butt' | 'square' };
opacity?: number;
- blend?: 'normal' | 'multiply' | 'screen' | 'overlay' | 'add';
+ blend?: ImageBlend;
transform?: ImageTransform;
mirror?: 'x' | 'y';
effects?: readonly ImageEffect[];
@@ -355,11 +463,11 @@ function checkFill(v: unknown): string | null {
return null;
}
-const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes', 'from', 'guide', 'height', 'yaw', 'pitch', 'build', 'pose', 'light', 'src', 'except', 'text', 'fontSize', 'font', 'weight', 'align', 'vertical', 'tracking', 'vanish', 'depth', 'faces', 'roof', 'y', 'edge', 'reach', 'ripple', 'smear', 'quad', 'threshold', 'strength', 'thin', 'grain', 'grainScale', 'dry', 'face', 'within', 'brush', 'detail', 'boost', 'seed'];
+const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes', 'from', 'guide', 'height', 'yaw', 'pitch', 'build', 'pose', 'light', 'src', 'except', 'text', 'fontSize', 'font', 'weight', 'align', 'vertical', 'tracking', 'vanish', 'depth', 'faces', 'roof', 'y', 'edge', 'reach', 'ripple', 'smear', 'quad', 'threshold', 'strength', 'thin', 'grain', 'grainScale', 'dry', 'face', 'within', 'feather', 'clipFeather', 'clipGrow', 'clipInvert', 'blendIf', 'adjust', 'filter', 'params'];
/** Layers that are aids to drawing unless told otherwise: seen when looking, and not in the .png. */
export const isGuideLayer = (l: ImageLayer): boolean => l.guide ?? (l.shape === 'figure' || l.shape === 'image' || l.shape === 'perspective');
-const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'bloom', 'repaint', 'group'];
+const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'bloom', 'adjust', 'filter', 'group'];
const STROKE_FIELDS = ['points', 'color', 'to', 'size', 'hardness', 'opacity', 'taper', 'bristles', 'strand', 'spread', 'vary', 'wobble', 'seed', 'smudge', 'erase', 'blend', ...BRUSH_FIELDS];
/** What a brush's own settings must be, on whatever carries them. */
@@ -458,7 +566,7 @@ function strokeProblem(s: Record<string, unknown>): string | null {
if (s.bristles !== undefined && !(Number.isInteger(s.bristles) && (s.bristles as number) >= 1 && (s.bristles as number) <= 200)) return 'bristles must be a whole number from 1 to 200';
if (s.seed !== undefined && !Number.isInteger(s.seed)) return 'seed must be a whole number';
if (s.taper !== undefined && !(Array.isArray(s.taper) && s.taper.length === 2 && s.taper.every((t) => isNum(t) && t >= 0 && t <= 1))) return 'taper must be [start, end], each 0..1';
- if (s.blend !== undefined && !['normal', 'multiply', 'screen', 'overlay', 'add'].includes(s.blend as string)) return 'blend must be normal, multiply, screen, overlay or add';
+ if (s.blend !== undefined && !IMAGE_BLENDS.includes(s.blend as ImageBlend)) return `blend must be one of ${IMAGE_BLENDS.join(', ')}`;
return null;
}
@@ -534,12 +642,16 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
if (l.grainScale !== undefined && !(isNum(l.grainScale) && (l.grainScale as number) > 0)) fail('grainScale must be positive');
if (l.fill === undefined && l.stroke === undefined) fail('it needs a fill or a stroke, or it draws nothing');
break;
- case 'repaint':
- if (l.brush !== undefined && !(typeof l.brush === 'string' && Object.hasOwn(IMAGE_BRUSHES, l.brush))) fail(`brush ${JSON.stringify(l.brush)} must be one of ${Object.keys(IMAGE_BRUSHES).join(', ')}`);
- if (l.detail !== undefined && ![0, 1, 2].includes(l.detail as number)) fail('detail must be 0, 1 or 2');
- if (l.boost !== undefined && !(Array.isArray(l.boost) && l.boost.every((b) => typeof b === 'string'))) fail('boost must be ids of layers drawn before it');
- if (l.seed !== undefined && !Number.isInteger(l.seed)) fail('seed must be a whole number');
+ case 'adjust': {
+ const bad = adjustProblem(l.adjust);
+ if (bad) fail(`adjust ${bad}`);
break;
+ }
+ case 'filter': {
+ const bad = filterProblem(l.filter, l.params);
+ if (bad) fail(bad);
+ break;
+ }
case 'bloom':
for (const key of ['threshold', 'strength']) if (l[key] !== undefined && !(isNum(l[key]) && (l[key] as number) >= 0)) fail(`${key} must be 0 or more`);
if (l.threshold !== undefined && (l.threshold as number) > 1) fail('threshold must be 0..1');
@@ -612,13 +724,17 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
if (s.join !== undefined && !['round', 'miter', 'bevel'].includes(s.join as string)) fail('stroke join must be round, miter or bevel');
if (s.cap !== undefined && !['round', 'butt', 'square'].includes(s.cap as string)) fail('stroke cap must be round, butt or square');
}
- if (!['group', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'bloom', 'repaint', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');
+ if (!['group', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'bloom', 'adjust', 'filter', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');
if (l.from !== undefined && l.shape !== 'paint') fail('from is for a paint layer');
if (l.guide !== undefined && typeof l.guide !== 'boolean') fail('guide must be true or false');
if (l.blur !== undefined && !(isNum(l.blur) && (l.blur as number) >= 0)) fail('blur must be 0 or more, in pixels');
if (l.blur && l.erase) fail('blur does not go with erase; put what cuts in a group with what it cuts, and blur the group');
if (l.opacity !== undefined && !(isNum(l.opacity) && l.opacity >= 0 && l.opacity <= 1)) fail('opacity must be 0..1');
- if (l.blend !== undefined && !['normal', 'multiply', 'screen', 'overlay', 'add'].includes(l.blend as string)) fail('blend must be normal, multiply, screen, overlay or add');
+ if (l.blend !== undefined && !IMAGE_BLENDS.includes(l.blend as ImageBlend)) fail(`blend must be one of ${IMAGE_BLENDS.join(', ')}`);
+ for (const key of ['feather', 'clipFeather']) if (l[key] !== undefined && !(isNum(l[key]) && (l[key] as number) >= 0)) fail(`${key} must be 0 or more pixels`);
+ if (l.clipGrow !== undefined && !isNum(l.clipGrow)) fail('clipGrow must be pixels, more to grow what it clips to and less to shrink it');
+ if (l.clipInvert !== undefined && typeof l.clipInvert !== 'boolean') fail('clipInvert must be true or false');
+ if (l.blendIf !== undefined && !(Array.isArray(l.blendIf) && (l.blendIf.length === 2 || l.blendIf.length === 3) && l.blendIf.every((v) => isNum(v) && v >= 0 && v <= 1))) fail('blendIf must be [low, high] or [low, high, softness], each 0..1');
if (l.mirror !== undefined && l.mirror !== 'x' && l.mirror !== 'y') fail('mirror must be x or y');
if (l.clip !== undefined && typeof l.clip !== 'string') fail('clip must be a layer id');
if (l.transform !== undefined) {
@@ -632,10 +748,13 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
if (l.effects !== undefined) {
if (!Array.isArray(l.effects)) fail('effects must be an array');
for (const e of l.effects as Record<string, unknown>[]) {
- if (!['shadow', 'glow', 'innerShadow', 'innerGlow'].includes(e?.type as string)) fail(`an effect has type ${JSON.stringify(e?.type)}; there are shadow, glow, innerShadow, innerGlow`);
- for (const key of Object.keys(e)) if (!['type', 'color', 'x', 'y', 'blur', 'opacity'].includes(key)) fail(`effect ${String(e.type)} has an unknown field "${key}"`);
+ if (!['shadow', 'glow', 'innerShadow', 'innerGlow', 'bevel', 'overlay'].includes(e?.type as string)) fail(`an effect has type ${JSON.stringify(e?.type)}; there are shadow, glow, innerShadow, innerGlow, bevel, overlay`);
+ if (e.type === 'overlay') { const bad = checkFill(e.fill); if (bad) fail(`effect overlay: fill ${bad}`); }
+ if (e.blend !== undefined && !IMAGE_BLENDS.includes(e.blend as ImageBlend)) fail(`effect ${String(e.type)}: blend must be one of ${IMAGE_BLENDS.join(', ')}`);
+ if (e.shade !== undefined && !isColor(e.shade)) fail(`effect ${String(e.type)}: shade must be #rgb..#rrggbbaa`);
+ for (const key of Object.keys(e)) if (!['type', 'color', 'x', 'y', 'blur', 'opacity', 'size', 'angle', 'depth', 'shade', 'fill', 'blend'].includes(key)) fail(`effect ${String(e.type)} has an unknown field "${key}"`);
if (e.color !== undefined && !isColor(e.color)) fail(`effect ${String(e.type)}: color must be #rgb..#rrggbbaa`);
- for (const k of ['x', 'y', 'blur', 'opacity']) if (e[k] !== undefined && !isNum(e[k])) fail(`effect ${String(e.type)}: ${k} must be a number`);
+ for (const k of ['x', 'y', 'blur', 'opacity', 'size', 'angle', 'depth']) if (e[k] !== undefined && !isNum(e[k])) fail(`effect ${String(e.type)}: ${k} must be a number`);
}
}
}
diff --git a/pipeline/tests/image-raster.test.ts b/pipeline/tests/image-raster.test.ts
index ec8e81aae..1b9c81797 100644
--- a/pipeline/tests/image-raster.test.ts
+++ b/pipeline/tests/image-raster.test.ts
@@ -3,7 +3,7 @@
/** A picture from layers: each draws what it says and where, edges are antialiased, and what is wrong with a source is said before anything is drawn. */
import { describe, expect, it } from 'vitest';
import { imageCacheStats, imageSheet, parseColor, parsePath, pickColors, renderImage, setSheet, valuesImage, zoomImage } from '../src/image/imageRaster';
-import { applyImageEdits, IMAGE_BRUSHES, strokeWidths, validateImageSource, type ImageLayer } from '../src/image/imageSource';
+import { applyImageEdits, IMAGE_BLENDS, IMAGE_BRUSHES, IMAGE_FILTERS, strokeWidths, validateImageSource, type ImageLayer } from '../src/image/imageSource';
const draw = (layers: unknown[], size: number | [number, number] = 64, background?: string): ReturnType<typeof renderImage> => renderImage(validateImageSource({ size, layers, ...(background ? { background } : {}) }));
const px = (img: { width: number; rgba: Uint8Array }, x: number, y: number): number[] => Array.from(img.rgba.subarray((y * img.width + x) * 4, (y * img.width + x) * 4 + 4));
@@ -1265,60 +1265,284 @@ describe('drawing only what changed', () => {
});
});
-describe('repainting the picture so far', () => {
- const halves: ImageLayer[] = [
- { id: 'l', shape: 'rect', at: [0, 0], size: [64, 128], fill: RED },
- { id: 'r', shape: 'rect', at: [64, 0], size: [64, 128], fill: BLUE },
- { id: 'mark', shape: 'rect', at: [20, 20], size: [24, 24], fill: '#ffffff' },
+describe('blend modes', () => {
+ const over = (blend: string, under = '#607090', top = '#c04020'): number[] => {
+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: under }, { shape: 'rect', at: [0, 0], size: [64, 64], fill: top, blend } as never]);
+ return px(img, 32, 32).slice(0, 3);
+ };
+
+ it('takes every one of them, and each differs from plain painting', () => {
+ for (const mode of IMAGE_BLENDS) {
+ if (mode === 'normal') continue;
+ expect(over(mode), mode).not.toEqual(over('normal'));
+ }
+ });
+
+ it('does what the arithmetic says', () => {
+ expect(over('darken', '#808080', '#c04020')).toEqual([128, 64, 32]);
+ expect(over('lighten', '#808080', '#c04020')).toEqual([192, 128, 128]);
+ expect(over('difference', '#808080', '#c04020')).toEqual([64, 64, 96]);
+ expect(over('subtract', '#c0c0c0', '#404040')).toEqual([128, 128, 128]);
+ expect(over('linearBurn', '#c0c0c0', '#c0c0c0')[0]).toBeGreaterThan(120);
+ expect(over('colorDodge', '#404040', '#ffffff')).toEqual([255, 255, 255]);
+ });
+
+ it('color takes the hue and keeps the light; luminosity the other way', () => {
+ const light = (c: number[]): number => c[0]! * 0.2126 + c[1]! * 0.7152 + c[2]! * 0.0722;
+ const tinted = over('color', '#808080', '#ff0000');
+ expect(Math.abs(light(tinted) - 128)).toBeLessThan(3);
+ expect(tinted[0]).toBeGreaterThan(tinted[1]! + 40);
+ const relit = over('luminosity', '#ff0000', '#ffffff');
+ expect(light(relit)).toBeGreaterThan(200);
+ });
+
+ it('says what is not one', () => {
+ expect(() => validateImageSource({ size: 8, layers: [{ shape: 'rect', at: [0, 0], size: [8, 8], fill: RED, blend: 'glow' }] })).toThrow(/blend must be one of normal, multiply/);
+ });
+});
+
+describe('feather and masks', () => {
+ const block: ImageLayer = { shape: 'rect', at: [16, 16], size: [32, 32], fill: RED };
+
+ it('feathers an edge inward, keeping the colour and not spreading', () => {
+ const img = draw([{ ...block, feather: 6 }]);
+ expect(px(img, 32, 32)).toEqual([255, 0, 0, 255]);
+ expect(px(img, 14, 32)[3]).toBe(0);
+ const edge = px(img, 17, 32);
+ expect(edge[3]).toBeLessThan(120);
+ if (edge[3]! > 2) expect(edge[0]).toBeGreaterThan(245);
+ });
+
+ it('softens, grows, shrinks and inverts what it clips to', () => {
+ const mask: ImageLayer = { id: 'm', shape: 'rect', at: [16, 16], size: [32, 32], fill: '#00000001' };
+ const fill: ImageLayer = { shape: 'rect', at: [0, 0], size: [64, 64], fill: RED, clip: 'm' };
+ expect(px(draw([mask, fill]), 14, 32)[3]).toBe(0);
+ expect(px(draw([mask, { ...fill, clipFeather: 4 }]), 14, 32)[3]).toBeGreaterThan(10);
+ expect(px(draw([mask, { ...fill, clipGrow: 4 }]), 14, 32)[3]).toBe(255);
+ expect(px(draw([mask, { ...fill, clipGrow: -4 }]), 18, 32)[3]).toBeLessThanOrEqual(1);
+ const inverted = draw([mask, { ...fill, clipInvert: true }]);
+ expect(px(inverted, 4, 4)[0]).toBe(255);
+ expect(px(inverted, 32, 32)[0]).toBe(0);
+ });
+
+ it('applies only where the picture is that light', () => {
+ const scene: ImageLayer[] = [{ shape: 'rect', at: [0, 0], size: [32, 64], fill: '#101010' }, { shape: 'rect', at: [32, 0], size: [32, 64], fill: '#f0f0f0' }];
+ const img = draw([...scene, { shape: 'rect', at: [0, 0], size: [64, 64], fill: RED, blendIf: [0.7, 1, 0] }]);
+ expect(px(img, 10, 32)[0]).toBe(0x10);
+ expect(px(img, 50, 32)).toEqual([255, 0, 0, 255]);
+ });
+
+ it('says what is wrong', () => {
+ const bad = (l: object): unknown => validateImageSource({ size: 8, layers: [{ ...block, ...l }] });
+ expect(() => bad({ feather: -1 })).toThrow(/feather must be/);
+ expect(() => bad({ blendIf: [0, 2] })).toThrow(/blendIf must be/);
+ expect(() => bad({ clipInvert: 1 })).toThrow(/clipInvert must be/);
+ });
+});
+
+describe('adjustment layers', () => {
+ const grey = (hex: string): ImageLayer => ({ shape: 'rect', at: [0, 0], size: [64, 64], fill: hex });
+ const adjusted = (adjust: object, under = '#808080', extra: object = {}): number[] => px(draw([grey(under), { shape: 'adjust', adjust, ...extra } as never]), 32, 32).slice(0, 3);
+
+ it('moves levels: input, gamma and output', () => {
+ expect(adjusted({ levels: { input: [0, 0.5] } })[0]).toBe(255);
+ expect(adjusted({ levels: { gamma: 2 } })[0]).toBeGreaterThan(160);
+ expect(adjusted({ levels: { output: [0.5, 0.5] } })[0]).toBe(128);
+ });
+
+ it('follows a curve, whole or by channel, without overshoot', () => {
+ expect(adjusted({ curves: [[0, 0], [0.5, 0.8], [1, 1]] })[0]).toBeGreaterThan(195);
+ const red = adjusted({ curves: { r: [[0, 0], [0.5, 1], [1, 1]] } });
+ expect(red[0]).toBe(255);
+ expect(red[1]).toBe(128);
+ });
+
+ it('takes exposure in stops, and brightness and contrast', () => {
+ expect(adjusted({ exposure: 1 }, '#404040')[0]).toBeGreaterThan(120);
+ expect(adjusted({ brightness: 0.2 })[0]).toBeGreaterThan(165);
+ expect(adjusted({ contrast: 0.5 }, '#a0a0a0')[0]).toBeGreaterThan(0xa0 + 20);
+ expect(adjusted({ contrast: 0.5 }, '#606060')[0]).toBeLessThan(0x60 - 20);
+ });
+
+ it('turns hue, saturation and lightness', () => {
+ const blue = adjusted({ hue: 240 }, '#ff0000');
+ expect(blue[2]).toBeGreaterThan(blue[0]!);
+ const dull = adjusted({ saturation: -1 }, '#ff0000');
+ expect(dull[0]).toBe(dull[1]);
+ expect(adjusted({ lightness: 0.5 }, '#404040')[0]).toBeGreaterThan(0x40 + 40);
+ });
+
+ it('lifts dull colour more than vivid with vibrance', () => {
+ const gain = (under: string): number => { const [a, b] = [adjusted({ vibrance: 1 }, under), parseColor(under).slice(0, 3).map((v) => v * 255)]; return Math.max(...a) - Math.min(...a) - (Math.max(...b) - Math.min(...b)); };
+ expect(gain('#a09088')).toBeGreaterThan(0);
+ expect(gain('#ff0010')).toBeLessThan(gain('#c06050') + 1);
+ });
+
+ it('warms and cools, and shifts shadows and highlights apart', () => {
+ const warm = adjusted({ temperature: 0.5 });
+ expect(warm[0]).toBeGreaterThan(warm[2]!);
+ const split = { balance: { shadows: [0, 0, 0.5], highlights: [0.5, 0, 0] } };
+ const dark = adjusted(split, '#101010'), bright = adjusted(split, '#f0f0f0');
+ expect(dark[2]).toBeGreaterThan(dark[0]!);
+ expect(bright[0]).toBeGreaterThanOrEqual(bright[2]!);
+ });
+
+ it('goes to grey, to a gradient map, inverted, posterized or thresholded', () => {
+ const bw = adjusted({ blackWhite: true }, '#ff0000');
+ expect(bw[0]).toBe(bw[1]);
+ expect(adjusted({ gradientMap: [[0, '#000000'], [1, '#ff0000']] }, '#ffffff')).toEqual([255, 0, 0]);
+ expect(adjusted({ invert: true }, '#ff0000')).toEqual([0, 255, 255]);
+ expect(adjusted({ posterize: 2 }, '#a0a0a0')[0]).toBe(255);
+ expect(adjusted({ threshold: 0.7 }, '#a0a0a0')[0]).toBe(0);
+ });
+
+ it('keeps to its clip, its range and its opacity, and leaves clear pixels clear', () => {
+ const img = draw([{ id: 'c', shape: 'rect', at: [0, 0], size: [32, 64], fill: '#808080' }, { shape: 'adjust', adjust: { invert: true }, opacity: 0.5 } as never]);
+ expect(px(img, 50, 32)[3]).toBe(0);
+ expect(Math.abs(px(img, 10, 32)[0]! - 128)).toBeLessThan(3);
+ const half = adjusted({ invert: true }, '#ff0000', { opacity: 0.5 });
+ expect(Math.abs(half[1]! - 128)).toBeLessThan(2);
+ });
+
+ it('says what is wrong', () => {
+ const bad = (adjust: unknown): unknown => validateImageSource({ size: 8, layers: [{ shape: 'adjust', adjust }] });
+ expect(() => bad(undefined)).toThrow(/adjust must be an object/);
+ expect(() => bad({})).toThrow(/changes nothing/);
+ expect(() => bad({ glow: 1 })).toThrow(/unknown "glow"/);
+ expect(() => bad({ hue: 999 })).toThrow(/hue must be/);
+ expect(() => bad({ curves: [[0, 0]] })).toThrow(/two or more/);
+ expect(() => bad({ levels: { input: [0, 3] } })).toThrow(/levels input/);
+ expect(() => bad({ balance: { sky: [0, 0, 0] } })).toThrow(/unknown "sky"/);
+ });
+});
+
+describe('filters', () => {
+ const scene: ImageLayer[] = [
+ { shape: 'rect', at: [0, 0], size: [64, 64], fill: '#202028' },
+ { shape: 'rect', at: [24, 24], size: [16, 16], fill: '#ffffff' },
];
- const paintOver = (fields: object = {}): ReturnType<typeof draw> => draw([...halves, { id: 'over', shape: 'repaint', ...fields } as never], [128, 128]);
+ const filtered = (filter: string, params: object = {}, extra: object = {}): ReturnType<typeof draw> => draw([...scene, { shape: 'filter', filter, params, ...extra } as never]);
+ const plain = draw(scene);
+
+ it('every filter changes the picture, and is the same every time', () => {
+ const busy = [...scene, { shape: 'circle', at: [11, 13], radius: 3, fill: '#ff8040' }] as ImageLayer[];
+ const before = draw(busy).rgba;
+ const run = (name: string): Uint8Array => draw([...busy, { shape: 'filter', filter: name } as never]).rgba;
+ for (const name of Object.keys(IMAGE_FILTERS)) {
+ expect(run(name), name).not.toEqual(before);
+ expect(run(name), name).toEqual(run(name));
+ }
+ });
- it('keeps the picture, its colours where they were, and is no longer the same pixels', () => {
- const before = draw(halves, [128, 128]);
- const after = paintOver();
- expect(px(after, 20, 90)[0]).toBeGreaterThan(200);
- expect(px(after, 110, 90)[2]).toBeGreaterThan(200);
- expect(px(after, 32, 32)[1]).toBeGreaterThan(160);
- expect(after.rgba).not.toEqual(before.rgba);
- expect(after.painted.over!.strokes).toBeGreaterThan(40);
+ it('blurs by radius, along a line, and round or out from a point', () => {
+ expect(px(filtered('gaussian', { radius: 4 }), 22, 32)[0]).toBeGreaterThan(0x30);
+ const along = filtered('motion', { angle: 0, distance: 20 });
+ expect(px(along, 20, 32)[0]).toBeGreaterThan(0x40);
+ expect(px(along, 32, 20)[0]).toBe(0x20);
+ expect(px(filtered('zoom', { at: [32, 32], amount: 1 }), 44, 32)[0]).toBeGreaterThan(0x20);
});
- it('breaks a hard edge into strokes', () => {
- const edgeOf = (img: ReturnType<typeof draw>): Set<number> => { const s = new Set<number>(); for (let y = 8; y < 120; y += 2) { let x = 0; while (x < 127 && px(img, x, y)[0]! > 128) x++; s.add(x); } return s; };
- expect(edgeOf(draw(halves.slice(0, 2), [128, 128])).size).toBe(1);
- expect(edgeOf(draw([...halves.slice(0, 2), { shape: 'repaint' } as never], [128, 128])).size).toBeGreaterThan(1);
+ it('opens lights into discs with a lens blur, unlike a plain blur', () => {
+ const dotted: ImageLayer[] = [scene[0]!, { shape: 'circle', at: [32, 32], radius: 2, fill: '#ffffff' }];
+ const at = (x: number, type: string, params: object): number => px(draw([...dotted, { shape: 'filter', filter: type, params } as never]), x, 32)[0]!;
+ const [middle, edge] = [at(32, 'lens', { radius: 8 }), at(38, 'lens', { radius: 8 })];
+ expect(edge).toBeGreaterThan(0x28);
+ expect(Math.abs(middle - edge)).toBeLessThan(90);
});
- it('takes more strokes as it goes finer, and more again where it is boosted', () => {
- const strokes = (fields: object): number => paintOver(fields).painted.over!.strokes;
- const scene: ImageLayer[] = [...halves, { id: 'noisy', shape: 'circle', at: [90, 90], radius: 24, fill: { type: 'noise', scale: 4, stops: [[0, '#000000'], [1, '#ffffff']] } }];
- const at = (fields: object): number => draw([...scene, { id: 'over', shape: 'repaint', ...fields } as never], [128, 128]).painted.over!.strokes;
- expect(at({ detail: 2 })).toBeGreaterThan(at({ detail: 0 }));
- expect(at({ detail: 0, boost: ['noisy'] })).toBeGreaterThan(at({ detail: 0 }));
- expect(strokes({})).toBeGreaterThan(0);
+ it('sharpens an edge, and leaves a flat area alone', () => {
+ const soft = draw([...scene, { shape: 'filter', filter: 'gaussian', params: { radius: 1.5 } } as never]);
+ const crisp = draw([...scene, { shape: 'filter', filter: 'gaussian', params: { radius: 1.5 } } as never, { shape: 'filter', filter: 'sharpen', params: { amount: 1, radius: 2 } } as never]);
+ expect(px(crisp, 25, 32)[0]).toBeGreaterThan(px(soft, 25, 32)[0]!);
+ expect(px(crisp, 8, 8)[0]).toBe(px(soft, 8, 8)[0]);
});
- it('leaves what it is told to, exactly', () => {
- const kept = paintOver({ except: 'mark' });
- expect(px(kept, 22, 22)).toEqual([255, 255, 255, 255]);
- expect(px(kept, 42, 42)).toEqual([255, 255, 255, 255]);
+ it('high pass is mid grey where nothing changes', () => {
+ const img = filtered('highPass', { radius: 4 });
+ expect(Math.abs(px(img, 4, 4)[0]! - 128)).toBeLessThan(2);
+ expect(px(img, 25, 32)[0]).toBeGreaterThan(150);
});
- it('takes the brush it is told, and is the same every time', () => {
- expect(paintOver({ brush: 'charcoal' }).rgba).not.toEqual(paintOver({ brush: 'oil' }).rgba);
- imageCacheStats(true);
- const a = paintOver({ seed: 4 }).rgba;
- imageCacheStats(true);
- expect(paintOver({ seed: 4 }).rgba).toEqual(a);
- expect(paintOver({ seed: 5 }).rgba).not.toEqual(a);
+ it('moves pixels with twirl, ripple and displace, keeping them inside', () => {
+ for (const name of ['twirl', 'ripple', 'displace']) {
+ const moved = filtered(name, name === 'twirl' ? { at: [32, 32], radius: 30, angle: 90 } : { amount: 6, size: 12 });
+ expect(moved.rgba).not.toEqual(plain.rgba);
+ expect(px(moved, 2, 2)[3]).toBe(255);
+ }
});
- it('says what is wrong with one', () => {
- const bad = (l: object): unknown => validateImageSource({ size: 8, layers: [{ shape: 'repaint', ...l }] });
- expect(() => bad({ brush: 'crayon' })).toThrow(/brush "crayon" must be one of/);
- expect(() => bad({ detail: 3 })).toThrow(/detail must be 0, 1 or 2/);
- expect(() => bad({ boost: 'x' })).toThrow(/boost must be/);
- expect(paintOver({ boost: ['none'] }).notes.join()).toMatch(/boost "none" names no layer/);
+ it('pixelates into blocks of one colour', () => {
+ const img = filtered('pixelate', { cell: 8 });
+ expect(px(img, 24, 24)).toEqual(px(img, 31, 31));
+ });
+
+ it('fog is clear at its start and thick at its end, in its own colour', () => {
+ const img = filtered('fog', { color: '#ff0000', density: 1, from: [0, 0], to: [0, 64], patchiness: 0 });
+ expect(px(img, 4, 2)[0]).toBeLessThan(0x30);
+ expect(px(img, 4, 62)[0]).toBeGreaterThan(220);
+ });
+
+ it('rays draw bright things out from a point', () => {
+ const img = filtered('rays', { at: [32, 4], threshold: 0.5, length: 0.8, strength: 1 });
+ expect(px(img, 32, 52)[0]).toBeGreaterThan(0x20 + 12);
+ expect(px(img, 4, 4)[0]).toBeLessThan(0x20 + 6);
+ });
+
+ it('a vignette darkens edges and not the middle; a flare lights round its point', () => {
+ const dim = filtered('vignette', { amount: 1, radius: 0.3, outer: 1 });
+ expect(px(dim, 1, 1)[0]).toBeLessThan(0x20);
+ expect(px(dim, 32, 32)[0]).toBe(255);
+ const lit = filtered('flare', { at: [10, 10], size: 12, strength: 1 });
+ expect(px(lit, 10, 10)[0]).toBeGreaterThan(px(lit, 54, 10)[0]!);
+ });
+
+ it('splits colour at the edge of the frame with chromatic, and not at its middle', () => {
+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: '#000000' }, { shape: 'rect', at: [50, 20], size: [6, 24], fill: '#ffffff' }, { shape: 'filter', filter: 'chromatic', params: { amount: 8 } } as never]);
+ let split = 0;
+ for (let x = 40; x < 64; x++) if (px(img, x, 32)[0] !== px(img, x, 32)[2]) split++;
+ expect(split).toBeGreaterThan(0);
+ expect(px(img, 53, 32)[1]).toBe(255);
+ });
+
+ it('keeps to its clip, softly if asked, and takes a blend', () => {
+ const held = draw([...scene, { id: 'm', shape: 'rect', at: [0, 0], size: [20, 64], fill: '#00000001' }, { shape: 'filter', filter: 'motion', params: { angle: 0, distance: 30 }, clip: 'm' } as never]);
+ expect(px(held, 32, 32)[0]).toBe(255);
+ const over = filtered('highPass', { radius: 4 }, { blend: 'overlay' });
+ expect(px(over, 8, 8)[0]).toBeLessThan(0x26);
+ });
+
+ it('says what is wrong', () => {
+ const bad = (filter: unknown, params?: unknown): unknown => validateImageSource({ size: 8, layers: [{ shape: 'filter', filter, ...(params ? { params } : {}) }] });
+ expect(() => bad('sparkle')).toThrow(/filter "sparkle" must be one of gaussian/);
+ expect(() => bad('motion', { size: 2 })).toThrow(/filter motion has unknown "size"; it takes angle, distance/);
+ expect(() => bad('fog', { color: 'red' })).toThrow(/color must be/);
+ expect(() => bad('rays', { at: 3 })).toThrow(/at must be \[x, y\]/);
+ expect(() => bad('gaussian', { radius: 'big' })).toThrow(/radius must be a number/);
+ });
+});
+
+describe('layer styles', () => {
+ const block = (effects: object[]): ImageLayer => ({ shape: 'rect', at: [16, 16], size: [32, 32], fill: '#606060', effects } as never);
+
+ it('a bevel lights the edge toward its light and shades the far one', () => {
+ const img = draw([block([{ type: 'bevel', size: 6, angle: 225 }])]);
+ expect(px(img, 18, 18)[0]).toBeGreaterThan(0x60 + 10);
+ expect(px(img, 45, 45)[0]).toBeLessThan(0x60 - 10);
+ expect(px(img, 32, 32)[0]).toBe(0x60);
+ });
+
+ it('an overlay covers the shape in a colour or a gradient, and stops at its edge', () => {
+ const flat = draw([block([{ type: 'overlay', fill: RED }])]);
+ expect(px(flat, 32, 32)).toEqual([255, 0, 0, 255]);
+ expect(px(flat, 4, 4)[3]).toBe(0);
+ const graded = draw([block([{ type: 'overlay', fill: { type: 'linear', from: [16, 0], to: [48, 0], stops: [[0, RED], [1, BLUE]] } }])]);
+ expect(px(graded, 18, 32)[0]).toBeGreaterThan(px(graded, 46, 32)[0]!);
+ });
+
+ it('says what is wrong', () => {
+ const bad = (e: object): unknown => validateImageSource({ size: 8, layers: [block([e])] });
+ expect(() => bad({ type: 'overlay' })).toThrow(/overlay: fill/);
+ expect(() => bad({ type: 'bevel', blend: 'x' })).toThrow(/blend must be one of/);
});
});
@@ -1777,7 +2001,7 @@ describe('the source', () => {
expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: RED, wobble: 1 }]))).toThrow(/unknown field "wobble"/);
expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: { type: 'linear', from: [0, 0], to: [1, 1], stops: [[0, RED]] } }]))).toThrow(/two stops/);
expect(() => validateImageSource(ok([{ shape: 'ring', at: [0, 0], radius: 1, inner: 2, fill: RED }]))).toThrow(/inner must be less/);
- expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: RED, effects: [{ type: 'bevel' }] }]))).toThrow(/bevel/);
+ expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: RED, effects: [{ type: 'sparkle' }] }]))).toThrow(/sparkle/);
expect(() => validateImageSource(ok([{ id: 'a', shape: 'circle', at: [0, 0], radius: 1, fill: RED }, { id: 'a', shape: 'circle', at: [0, 0], radius: 1, fill: RED }]))).toThrow(/already used/);
expect(() => validateImageSource(ok([{ shape: 'group', layers: [{ shape: 'blob' }] }]))).toThrow(/layers\[0\]/);
});
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 850386d21..8c47c4485 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -3892,6 +3892,10 @@ export const NOTED = {
{"note":"An AI can draw 2D pictures."},
"feat(mcp): image_write draws text in the editor's fonts, and documents the built world":
{"note":"An AI can draw 2D pictures."},
+ "feat(look): adjustment layers, filters, every blend mode, feather; repaint removed":
+ {"note":"An AI can draw 2D pictures."},
+ "feat(mcp): image_write teaches the finish of a picture, in place of repaint":
+ {"note":"An AI can draw 2D pictures."},
"feat(look): repaint the picture so far in strokes, and an edit draws only what changed":
{"note":"An AI can draw 2D pictures."},
"feat(mcp): image_write teaches repaint":