A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(look): layers and pixels drawn to a picture, on the CPU
yhh6bcb63af2c71
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7595a45c7..d31c91a6b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,6 +22,8 @@ published separately; it ships inside the editor.- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.+- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on dark and light grounds and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer without restating the rest. Drawn on the CPU, so a source makes the same picture on any machine.+- **An AI can build a person, and not a picture of one.** The `humanoid` step makes a whole body from a few numbers, with fingers, a face and ears, seen right from every side, and `rig: { humanoid: true }` gives it bones that fit. `garment` makes clothes from the body's own surface: loose or tight, a bell sleeve that hangs, a clean hem, a thickness; what a body wears now takes its weights from the skin under it, so an arm stays in its sleeve. `pose` holds a body in a pose by angles, or with `reach` by where a hand should be. `organic` gains `limb`, cross-sections along a path, and `fine`, regions split finer on one skin. `displace` gains `folds`, running the way cloth hangs. `scene_views` no longer takes on a shot's lens blur and grade.- **An AI can read shapes and colours off a picture as numbers.** `reference_trace` grows a part of a reference from places inside it and answers its outline as points, ready for an `outline`, or its strongest edges as lines, ready for a `sculpt` press. `reference_probe` answers the colour at the same places in the reference and the game. `reference_compare` with `overlay: "parts"` draws only the outline of the entities named on the reference. Each shows what it found on the picture, and judges nothing.diff --git a/desktop b/desktopindex 80067ee04..b6e7cbd93 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit 80067ee04f06f6bd2c2306026041c69e571ba736+Subproject commit b6e7cbd9367beeb57577794b4b440c37f85dba39diff --git a/pipeline/src/image/imageRaster.ts b/pipeline/src/image/imageRaster.tsnew file mode 100644index 000000000..60f850176--- /dev/null+++ b/pipeline/src/image/imageRaster.ts@@ -0,0 +1,645 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file imageRaster.ts — a picture from a list of layers, on the CPU.+ *+ * A model describes structure well and cannot write a hundred thousand colours. So it states shapes, fills and+ * effects, and this owns everything that has to be exact: antialiased edges, gradients, soft shadows.+ * On the CPU rather than through a canvas, so the same source makes the same bytes on any machine and a test can read them.+ */+import type { ImageFill, ImageLayer, ImageSource, ImageEffect, ImageTransform } from './imageSource';++export interface RgbaBitmap { width: number; height: number; rgba: Uint8Array }++type Rgba = readonly [number, number, number, number];+type Pt = readonly [number, number];+type Matrix = readonly [number, number, number, number, number, number];++const IDENTITY: Matrix = [1, 0, 0, 1, 0, 0];+const SUBSCANS = 8;+const CURVE_STEPS = 20;++export function parseColor(text: string): Rgba {+ const h = /^#([0-9a-fA-F]+)$/.exec(text)?.[1];+ if (!h || ![3, 4, 6, 8].includes(h.length)) throw new Error(`color ${JSON.stringify(text)} must be #rgb, #rgba, #rrggbb or #rrggbbaa`);+ const full = h.length <= 4 ? [...h].map((c) => c + c).join('') : h;+ const at = (i: number): number => parseInt(full.slice(i, i + 2), 16) / 255;+ return [at(0), at(2), at(4), full.length === 8 ? at(6) : 1];+}++const times = (a: Matrix, b: Matrix): Matrix => [+ a[0] * b[0] + a[2] * b[1], a[1] * b[0] + a[3] * b[1],+ a[0] * b[2] + a[2] * b[3], a[1] * b[2] + a[3] * b[3],+ a[0] * b[4] + a[2] * b[5] + a[4], a[1] * b[4] + a[3] * b[5] + a[5],+];+const apply = (m: Matrix, [x, y]: Pt): Pt => [m[0] * x + m[2] * y + m[4], m[1] * x + m[3] * y + m[5]];+function invert(m: Matrix): Matrix {+ const det = m[0] * m[3] - m[1] * m[2] || 1e-12;+ return [m[3] / det, -m[1] / det, -m[2] / det, m[0] / det, (m[2] * m[5] - m[3] * m[4]) / det, (m[1] * m[4] - m[0] * m[5]) / det];+}++function transformOf(t: ImageTransform | undefined, origin: Pt): Matrix {+ if (!t) return IDENTITY;+ const [ox, oy] = t.origin ?? origin;+ const r = ((t.rotate ?? 0) * Math.PI) / 180;+ const [sx, sy] = typeof t.scale === 'number' ? [t.scale, t.scale] : t.scale ?? [1, 1];+ const [tx, ty] = t.translate ?? [0, 0];+ let m: Matrix = [1, 0, 0, 1, tx + ox, ty + oy];+ m = times(m, [Math.cos(r), Math.sin(r), -Math.sin(r), Math.cos(r), 0, 0]);+ m = times(m, [sx, 0, 0, sy, 0, 0]);+ return times(m, [1, 0, 0, 1, -ox, -oy]);+}++// ---- Outlines -------------------------------------------------------------------++function ellipsePoints(cx: number, cy: number, rx: number, ry: number): Pt[] {+ const n = Math.max(48, Math.ceil(Math.max(rx, ry) * 2));+ return Array.from({ length: n }, (_, i): Pt => [cx + rx * Math.cos((i / n) * Math.PI * 2), cy + ry * Math.sin((i / n) * Math.PI * 2)]);+}++function roundedRect(x: number, y: number, w: number, h: number, radius: number | readonly number[]): Pt[] {+ const rs = typeof radius === 'number' ? [radius, radius, radius, radius] : radius;+ const limit = Math.min(w, h) / 2;+ const [tl, tr, br, bl] = [0, 1, 2, 3].map((i) => Math.max(0, Math.min(limit, rs[i] ?? 0))) as [number, number, number, number];+ const out: Pt[] = [];+ const corner = (cx: number, cy: number, r: number, from: number): void => {+ if (r <= 0) { out.push([cx, cy]); return; }+ const n = Math.max(4, Math.ceil(r));+ for (let i = 0; i <= n; i++) {+ const a = from + (i / n) * (Math.PI / 2);+ out.push([cx + r * Math.cos(a), cy + r * Math.sin(a)]);+ }+ };+ corner(x + tl, y + tl, tl, Math.PI);+ corner(x + w - tr, y + tr, tr, -Math.PI / 2);+ corner(x + w - br, y + h - br, br, 0);+ corner(x + bl, y + h - bl, bl, Math.PI / 2);+ return out;+}++/** An SVG path's closed and open runs: M L H V C S Q T Z, either case. */+export function parsePath(d: string): { points: Pt[]; closed: boolean }[] {+ const tokens = d.match(/[a-zA-Z]|-?(?:\d+\.?\d*|\.\d+)(?:[eE][-+]?\d+)?/g) ?? [];+ const runs: { points: Pt[]; closed: boolean }[] = [];+ let run: Pt[] | null = null;+ let [x, y, sx, sy, lastCx, lastCy] = [0, 0, 0, 0, 0, 0];+ let cmd = '', prev = '', i = 0;+ const num = (): number => {+ const t = tokens[i++];+ if (t === undefined || /[a-zA-Z]/.test(t)) throw new Error(`path ${JSON.stringify(d.slice(0, 60))}: command ${cmd} is short of numbers`);+ return Number(t);+ };+ const start = (): void => {+ if (!run) { run = [[x, y]]; runs.push({ points: run, closed: false }); }+ };+ const curve = (f: (t: number) => Pt): void => {+ start();+ for (let s = 1; s <= CURVE_STEPS; s++) run!.push(f(s / CURVE_STEPS));+ };+ while (i < tokens.length) {+ if (/[a-zA-Z]/.test(tokens[i]!)) cmd = tokens[i++]!;+ else if (!cmd) throw new Error(`path ${JSON.stringify(d.slice(0, 60))} must start with a command`);+ const rel = cmd === cmd.toLowerCase();+ const [ox, oy] = rel ? [x, y] : [0, 0];+ switch (cmd.toUpperCase()) {+ case 'M':+ x = ox + num(); y = oy + num(); sx = x; sy = y;+ run = [[x, y]];+ runs.push({ points: run, closed: false });+ cmd = rel ? 'l' : 'L';+ break;+ case 'L': x = ox + num(); y = oy + num(); start(); run!.push([x, y]); break;+ case 'H': x = ox + num(); start(); run!.push([x, y]); break;+ case 'V': y = oy + num(); start(); run!.push([x, y]); break;+ case 'C': case 'S': {+ const smooth = cmd.toUpperCase() === 'S';+ const [c1x, c1y] = smooth ? (prev === 'C' || prev === 'S' ? [2 * x - lastCx, 2 * y - lastCy] : [x, y]) : [ox + num(), oy + num()];+ const [c2x, c2y, ex, ey] = [ox + num(), oy + num(), ox + num(), oy + num()];+ const [x0, y0] = [x, y];+ curve((t) => {+ const u = 1 - t;+ return [u * u * u * x0 + 3 * u * u * t * c1x + 3 * u * t * t * c2x + t * t * t * ex, u * u * u * y0 + 3 * u * u * t * c1y + 3 * u * t * t * c2y + t * t * t * ey];+ });+ [x, y, lastCx, lastCy] = [ex, ey, c2x, c2y];+ break;+ }+ case 'Q': case 'T': {+ const smooth = cmd.toUpperCase() === 'T';+ const [cx, cy] = smooth ? (prev === 'Q' || prev === 'T' ? [2 * x - lastCx, 2 * y - lastCy] : [x, y]) : [ox + num(), oy + num()];+ const [ex, ey] = [ox + num(), oy + num()];+ const [x0, y0] = [x, y];+ curve((t) => {+ const u = 1 - t;+ return [u * u * x0 + 2 * u * t * cx + t * t * ex, u * u * y0 + 2 * u * t * cy + t * t * ey];+ });+ [x, y, lastCx, lastCy] = [ex, ey, cx, cy];+ break;+ }+ case 'Z':+ if (runs.length) runs[runs.length - 1]!.closed = true;+ x = sx; y = sy; run = null;+ cmd = '';+ break;+ default:+ throw new Error(`path command ${cmd} is not one this draws; there are M L H V C S Q T Z`);+ }+ prev = cmd.toUpperCase() === 'L' && prev === 'M' ? 'L' : cmd.toUpperCase();+ }+ return runs;+}++/** A layer's outlines in its own coordinates. */+function outlinesOf(layer: ImageLayer): { runs: { points: Pt[]; closed: boolean }[]; middle: Pt } {+ const closed = (points: Pt[]): { points: Pt[]; closed: boolean }[] => [{ points, closed: true }];+ switch (layer.shape) {+ case 'rect': {+ const [x, y] = layer.at!, [w, h] = layer.size!;+ return { runs: closed(roundedRect(x, y, w, h, layer.radius ?? 0)), middle: [x + w / 2, y + h / 2] };+ }+ case 'circle': {+ const [cx, cy] = layer.at!;+ return { runs: closed(ellipsePoints(cx, cy, layer.radius as number, layer.radius as number)), middle: [cx, cy] };+ }+ case 'ellipse': {+ const [cx, cy] = layer.at!, [rx, ry] = layer.radii!;+ return { runs: closed(ellipsePoints(cx, cy, rx, ry)), middle: [cx, cy] };+ }+ case 'ring': {+ const [cx, cy] = layer.at!;+ const inner = ellipsePoints(cx, cy, layer.inner!, layer.inner!).reverse();+ return { runs: [...closed(ellipsePoints(cx, cy, layer.radius as number, layer.radius as number)), ...closed(inner)], middle: [cx, cy] };+ }+ case 'polygon': case 'line': {+ const pts = layer.points as Pt[];+ const c = pts.reduce<[number, number]>((s, p) => [s[0] + p[0] / pts.length, s[1] + p[1] / pts.length], [0, 0]);+ return { runs: [{ points: pts, closed: layer.shape === 'polygon' }], middle: c };+ }+ case 'star': case 'gear': {+ const [cx, cy] = layer.at!;+ const n = layer.count!, outer = layer.radius as number, inner = layer.inner!;+ const out: Pt[] = [];+ const start = ((layer.turn ?? -90) * Math.PI) / 180;+ if (layer.shape === 'star') {+ for (let i = 0; i < n * 2; i++) {+ const r = i % 2 === 0 ? outer : inner;+ const a = start + (i / (n * 2)) * Math.PI * 2;+ out.push([cx + r * Math.cos(a), cy + r * Math.sin(a)]);+ }+ } else {+ // A tooth is flat on top, narrower than its root, with the same gap between.+ const step = (Math.PI * 2) / n, tip = (layer.tooth ?? 0.3) * step, root = tip * 1.5;+ for (let i = 0; i < n; i++) {+ const a = start + i * step;+ for (const [r, d] of [[inner, -root], [outer, -tip], [outer, tip], [inner, root]] as const) out.push([cx + r * Math.cos(a + d), cy + r * Math.sin(a + d)]);+ }+ }+ return { runs: closed(out), middle: [cx, cy] };+ }+ case 'path': {+ const runs = parsePath(layer.d!);+ const all = runs.flatMap((r) => r.points);+ if (all.length === 0) throw new Error('path draws nothing');+ const c = all.reduce<[number, number]>((s, p) => [s[0] + p[0] / all.length, s[1] + p[1] / all.length], [0, 0]);+ return { runs, middle: c };+ }+ default:+ return { runs: [], middle: [0, 0] };+ }+}++// ---- Masks ----------------------------------------------------------------------++/** What each pixel has of a filled outline, non-zero: a span's ends count for what they cover, so an edge has no stairs. */+function fillMask(runs: readonly (readonly Pt[])[], w: number, h: number): Float32Array {+ const mask = new Float32Array(w * h);+ const edges: { x0: number; y0: number; x1: number; y1: number; dir: number }[] = [];+ let top = Infinity, bottom = -Infinity;+ for (const run of runs) {+ for (let i = 0; i < run.length; i++) {+ const [a, b] = [run[i]!, run[(i + 1) % run.length]!];+ if (a[1] === b[1]) continue;+ const down = a[1] < b[1];+ edges.push(down ? { x0: a[0], y0: a[1], x1: b[0], y1: b[1], dir: 1 } : { x0: b[0], y0: b[1], x1: a[0], y1: a[1], dir: -1 });+ top = Math.min(top, a[1], b[1]); bottom = Math.max(bottom, a[1], b[1]);+ }+ }+ if (edges.length === 0) return mask;+ const share = 1 / SUBSCANS;+ const from = Math.max(0, Math.floor(top)), to = Math.min(h - 1, Math.ceil(bottom));+ const hits: { x: number; dir: number }[] = [];+ for (let y = from; y <= to; y++) {+ for (let s = 0; s < SUBSCANS; s++) {+ const sy = y + (s + 0.5) * share;+ hits.length = 0;+ for (const e of edges) {+ if (sy < e.y0 || sy >= e.y1) continue;+ hits.push({ x: e.x0 + ((sy - e.y0) / (e.y1 - e.y0)) * (e.x1 - e.x0), dir: e.dir });+ }+ if (hits.length < 2) continue;+ hits.sort((a, b) => a.x - b.x);+ let winding = 0;+ for (let i = 0; i + 1 < hits.length; i++) {+ winding += hits[i]!.dir;+ if (winding === 0) continue;+ const [l, r] = [Math.max(0, hits[i]!.x), Math.min(w, hits[i + 1]!.x)];+ if (r <= l) continue;+ const [il, ir] = [Math.floor(l), Math.min(w - 1, Math.floor(r))];+ const row = y * w;+ if (il === ir) { mask[row + il]! += (r - l) * share; continue; }+ mask[row + il]! += (il + 1 - l) * share;+ for (let x = il + 1; x < ir; x++) mask[row + x]! += share;+ mask[row + ir]! += (r - ir) * share;+ }+ }+ }+ for (let i = 0; i < mask.length; i++) if (mask[i]! > 1) mask[i] = 1;+ return mask;+}++/** What each pixel has of a stroke: by its distance from the line, so joins and ends come out round. */+function strokeMask(runs: readonly { points: readonly Pt[]; closed: boolean }[], width: number, w: number, h: number): Float32Array {+ const mask = new Float32Array(w * h);+ const half = width / 2;+ for (const run of runs) {+ const count = run.closed ? run.points.length : run.points.length - 1;+ for (let i = 0; i < count; i++) {+ const [a, b] = [run.points[i]!, run.points[(i + 1) % run.points.length]!];+ const [x0, x1] = [Math.max(0, Math.floor(Math.min(a[0], b[0]) - half - 1)), Math.min(w - 1, Math.ceil(Math.max(a[0], b[0]) + half + 1))];+ const [y0, y1] = [Math.max(0, Math.floor(Math.min(a[1], b[1]) - half - 1)), Math.min(h - 1, Math.ceil(Math.max(a[1], b[1]) + half + 1))];+ const [ex, ey] = [b[0] - a[0], b[1] - a[1]];+ const len = ex * ex + ey * ey;+ for (let y = y0; y <= y1; y++) {+ for (let x = x0; x <= x1; x++) {+ const [px, py] = [x + 0.5 - a[0], y + 0.5 - a[1]];+ const t = len > 0 ? Math.max(0, Math.min(1, (px * ex + py * ey) / len)) : 0;+ const v = Math.max(0, Math.min(1, half - Math.hypot(px - ex * t, py - ey * t) + 0.5));+ if (v > mask[y * w + x]!) mask[y * w + x] = v;+ }+ }+ }+ }+ return mask;+}++/** Three box blurs, which is a gaussian to the eye at a cost that does not grow with the radius. */+function blurMask(src: Float32Array, w: number, h: number, sigma: number): Float32Array {+ if (sigma <= 0) return src;+ const r = Math.max(1, Math.round(sigma * 1.2));+ let cur = src;+ for (let pass = 0; pass < 3; pass++) {+ const a = new Float32Array(w * h);+ for (let y = 0; y < h; y++) {+ let sum = 0;+ for (let x = -r; x <= r; x++) sum += cur[y * w + Math.max(0, Math.min(w - 1, x))]!;+ for (let x = 0; x < w; x++) {+ a[y * w + x] = sum / (2 * r + 1);+ sum += cur[y * w + Math.min(w - 1, x + r + 1)]! - cur[y * w + Math.max(0, x - r)]!;+ }+ }+ const b = new Float32Array(w * h);+ for (let x = 0; x < w; x++) {+ let sum = 0;+ for (let y = -r; y <= r; y++) sum += a[Math.max(0, Math.min(h - 1, y)) * w + x]!;+ for (let y = 0; y < h; y++) {+ b[y * w + x] = sum / (2 * r + 1);+ sum += a[Math.min(h - 1, y + r + 1) * w + x]! - a[Math.max(0, y - r) * w + x]!;+ }+ }+ cur = b;+ }+ return cur;+}++function shifted(src: Float32Array, w: number, h: number, dx: number, dy: number): Float32Array {+ const [ix, iy] = [Math.round(dx), Math.round(dy)];+ if (ix === 0 && iy === 0) return src;+ const out = new Float32Array(w * h);+ for (let y = 0; y < h; y++) {+ const fy = y - iy;+ if (fy < 0 || fy >= h) continue;+ for (let x = 0; x < w; x++) {+ const fx = x - ix;+ if (fx >= 0 && fx < w) out[y * w + x] = src[fy * w + fx]!;+ }+ }+ return out;+}++// ---- Paint ----------------------------------------------------------------------++/** A fill as a function of a pixel. Its coordinates go with its layer's, turned and all. */+function paintOf(fill: ImageFill, m: Matrix): (x: number, y: number) => Rgba {+ if (typeof fill === 'string') {+ const c = parseColor(fill);+ return () => c;+ }+ const stops = [...fill.stops].map(([t, c]) => [t, parseColor(c)] as const).sort((a, b) => a[0] - b[0]);+ const inverse = invert(m);+ const at = (t: number): Rgba => {+ if (t <= stops[0]![0]) return stops[0]![1];+ for (let i = 1; i < stops.length; i++) {+ if (t <= stops[i]![0]) {+ const [t0, a] = stops[i - 1]!, [t1, b] = stops[i]!;+ const s = (t - t0) / Math.max(1e-6, t1 - t0);+ return [a[0] + (b[0] - a[0]) * s, a[1] + (b[1] - a[1]) * s, a[2] + (b[2] - a[2]) * s, a[3] + (b[3] - a[3]) * s];+ }+ }+ return stops[stops.length - 1]![1];+ };+ if (fill.type === 'linear') {+ const [a, b] = [fill.from, fill.to];+ const [ex, ey] = [b[0] - a[0], b[1] - a[1]];+ const len = ex * ex + ey * ey || 1;+ return (x, y) => {+ const p = apply(inverse, [x, y]);+ return at(((p[0] - a[0]) * ex + (p[1] - a[1]) * ey) / len);+ };+ }+ const [cx, cy] = fill.at;+ const r = fill.radius;+ return (x, y) => {+ const p = apply(inverse, [x, y]);+ return at(Math.hypot(p[0] - cx, p[1] - cy) / r);+ };+}++/** Premultiplied: what blending wants, and what a blur of an edge wants. */+class Canvas {+ readonly px: Float32Array;+ constructor(readonly w: number, readonly h: number) { this.px = new Float32Array(w * h * 4); }+}++type Blend = NonNullable<ImageLayer['blend']>;+const blendChannel = (mode: Blend, b: number, s: number): number => {+ switch (mode) {+ case 'multiply': return b * s;+ case 'screen': return b + s - b * s;+ case 'overlay': return b < 0.5 ? 2 * b * s : 1 - 2 * (1 - b) * (1 - s);+ case 'add': return Math.min(1, b + s);+ default: return s;+ }+};++/** @p color straight, at coverage @p m, over what is there. */+function put(c: Canvas, i: number, r: number, g: number, b: number, a: number, mode: Blend): void {+ if (a <= 0) return;+ const p = c.px, at = i * 4;+ const da = p[at + 3]!;+ if (mode === 'normal' || da <= 0) {+ p[at] = p[at]! * (1 - a) + r * a; p[at + 1] = p[at + 1]! * (1 - a) + g * a; p[at + 2] = p[at + 2]! * (1 - a) + b * a;+ } else {+ const [dr, dg, db] = [p[at]! / da, p[at + 1]! / da, p[at + 2]! / da];+ const mix = (d: number, s: number): number => (1 - da) * s + da * blendChannel(mode, d, s);+ p[at] = p[at]! * (1 - a) + mix(dr, r) * a; p[at + 1] = p[at + 1]! * (1 - a) + mix(dg, g) * a; p[at + 2] = p[at + 2]! * (1 - a) + mix(db, b) * a;+ }+ p[at + 3] = da + a - da * a;+}++function effectLayers(effects: readonly ImageEffect[] | undefined, under: boolean, c: Canvas, mask: Float32Array): void {+ const { w, h } = c;+ for (const e of effects ?? []) {+ const isUnder = e.type === 'shadow' || e.type === 'glow';+ if (isUnder !== under) continue;+ const [r, g, b, ca] = parseColor(e.color ?? (e.type === 'glow' || e.type === 'innerGlow' ? '#ffffff' : '#000000'));+ const op = (e.opacity ?? 1) * ca;+ const sigma = e.blur ?? (e.type === 'glow' ? 8 : 4);+ if (isUnder) {+ let src = shifted(mask, w, h, e.x ?? 0, e.y ?? 0);+ // Spread first, so a glow is bright at the edge and not merely soft.+ if (e.type === 'glow') src = src.map((v) => Math.min(1, v * 1.6));+ const soft = blurMask(src, w, h, sigma);+ for (let i = 0; i < soft.length; i++) put(c, i, r, g, b, Math.min(1, soft[i]! * op), e.type === 'glow' ? 'screen' : 'normal');+ } else {+ // Inside: what the shape leaves out, softened and moved, and kept within the shape.+ const inv = mask.map((v) => 1 - v);+ const soft = blurMask(shifted(inv, w, h, e.x ?? 0, e.y ?? 0), w, h, sigma);+ const mode: Blend = e.type === 'innerGlow' ? 'screen' : 'normal';+ for (let i = 0; i < soft.length; i++) put(c, i, r, g, b, Math.min(1, soft[i]! * mask[i]! * op), mode);+ }+ }+}++interface Drawn { masks: Map<string, Float32Array>; notes: string[] }++function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn): void {+ if (layer.visible === false) return;+ const { w, h } = c;+ if (layer.mirror) {+ const { mirror, ...rest } = layer;+ const axis = mirror === 'x' ? { x: -1, y: 1 } : { x: 1, y: -1 };+ // Across the middle of the picture.+ const flip: Matrix = axis.x < 0 ? [-1, 0, 0, 1, w / drawnScale(parent), 0] : [1, 0, 0, -1, 0, h / drawnScale(parent)];+ drawLayer(c, rest, parent, drawn);+ drawLayer(c, { ...rest, ...(layer.id ? { id: `${layer.id}.mirrored` } : {}) }, times(parent, flip), drawn);+ return;+ }+ const opacity = layer.opacity ?? 1;+ const blend: Blend = layer.blend ?? 'normal';+ const clip = layer.clip ? drawn.masks.get(layer.clip) : undefined;+ if (layer.clip && !clip) drawn.notes.push(`layer ${layer.id ?? layer.shape}: clip "${layer.clip}" names no layer drawn before it`);++ if (layer.shape === 'group') {+ const inner = new Canvas(w, h);+ const m = times(parent, transformOf(layer.transform, [w / 2, h / 2]));+ for (const child of layer.layers ?? []) drawLayer(inner, child, m, drawn);+ const alpha = new Float32Array(w * h);+ for (let i = 0; i < alpha.length; i++) alpha[i] = inner.px[i * 4 + 3]!;+ if (layer.id) drawn.masks.set(layer.id, alpha);+ effectLayers(layer.effects, true, c, alpha);+ for (let i = 0; i < alpha.length; i++) {+ const a = alpha[i]! * opacity * (clip ? clip[i]! : 1);+ if (a <= 0) continue;+ const s = inner.px[i * 4 + 3]! || 1;+ put(c, i, inner.px[i * 4]! / s, inner.px[i * 4 + 1]! / s, inner.px[i * 4 + 2]! / s, a, blend);+ }+ effectLayers(layer.effects, false, c, alpha);+ return;+ }++ if (layer.shape === 'pixels') {+ drawPixels(c, layer, parent, drawn, opacity, blend, clip);+ return;+ }++ const { runs, middle } = outlinesOf(layer);+ const m = times(parent, transformOf(layer.transform, middle));+ const placed = runs.map((run) => ({ closed: run.closed, points: run.points.map((p) => apply(m, p)) }));+ const scale = Math.sqrt(Math.abs(m[0] * m[3] - m[1] * m[2]));+ const hasFill = layer.fill !== undefined && layer.shape !== 'line';+ const body = layer.shape === 'line' ? new Float32Array(w * h) : fillMask(placed.map((r) => r.points), w, h);+ const line = layer.stroke ? strokeMask(placed, layer.stroke.width * scale, w, h) : null;+ if (line && layer.stroke!.align && layer.stroke!.align !== 'center') {+ const inside = layer.stroke!.align === 'inside';+ for (let i = 0; i < line.length; i++) line[i]! *= inside ? body[i]! : 1 - body[i]!;+ }+ const whole = new Float32Array(w * h);+ let any = 0;+ for (let i = 0; i < whole.length; i++) {+ const v = Math.max(hasFill ? body[i]! : 0, line ? line[i]! : 0) * (clip ? clip[i]! : 1);+ whole[i] = v;+ any += v;+ }+ if (layer.id) drawn.masks.set(layer.id, whole);+ if (any === 0) drawn.notes.push(`layer ${layer.id ?? layer.shape} draws nothing inside the picture${clip ? ', or nothing inside its clip' : ''}`);+ if (layer.erase) {+ for (let i = 0; i < whole.length; i++) {+ const keep = 1 - whole[i]! * opacity;+ for (let k = 0; k < 4; k++) c.px[i * 4 + k]! *= keep;+ }+ return;+ }+ effectLayers(layer.effects, true, c, whole);+ if (hasFill) {+ const paint = paintOf(layer.fill!, m);+ for (let y = 0; y < h; y++) {+ for (let x = 0; x < w; x++) {+ const i = y * w + x;+ const v = body[i]! * (clip ? clip[i]! : 1);+ if (v <= 0) continue;+ const col = paint(x + 0.5, y + 0.5);+ put(c, i, col[0], col[1], col[2], v * col[3] * opacity, blend);+ }+ }+ }+ effectLayers(layer.effects, false, c, whole);+ if (line) {+ const paint = paintOf(layer.stroke!.color, m);+ for (let y = 0; y < h; y++) {+ for (let x = 0; x < w; x++) {+ const i = y * w + x;+ const v = line[i]! * (clip ? clip[i]! : 1);+ if (v <= 0) continue;+ const col = paint(x + 0.5, y + 0.5);+ put(c, i, col[0], col[1], col[2], v * col[3] * opacity, blend);+ }+ }+ }+}++const drawnScale = (m: Matrix): number => Math.sqrt(Math.abs(m[0] * m[3] - m[1] * m[2])) || 1;++/** Rows of characters, each a palette's colour, drawn as hard squares. */+function drawPixels(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {+ const { w, h } = c;+ const rows = layer.rows!;+ const columns = Math.max(...rows.map((r) => r.length));+ const scale = (layer.scale ?? Math.max(1, Math.floor(Math.min(w / columns, h / rows.length)))) * drawnScale(parent);+ const [ax, ay] = layer.at ? apply(parent, layer.at) : [(w - columns * scale) / 2, (h - rows.length * scale) / 2];+ const palette = new Map(Object.entries(layer.palette ?? {}).map(([k, v]) => [k, parseColor(v)]));+ const whole = new Float32Array(w * h);+ const colors = new Float32Array(w * h * 3);+ rows.forEach((row, ry) => {+ [...row].forEach((ch, rx) => {+ const col = palette.get(ch);+ if (!col) return;+ for (let y = Math.max(0, Math.round(ay + ry * scale)); y < Math.min(h, Math.round(ay + (ry + 1) * scale)); y++) {+ for (let x = Math.max(0, Math.round(ax + rx * scale)); x < Math.min(w, Math.round(ax + (rx + 1) * scale)); x++) {+ whole[y * w + x] = col[3] * (clip ? clip[y * w + x]! : 1);+ colors[(y * w + x) * 3] = col[0]; colors[(y * w + x) * 3 + 1] = col[1]; colors[(y * w + x) * 3 + 2] = col[2];+ }+ }+ });+ });+ if (layer.id) drawn.masks.set(layer.id, whole);+ effectLayers(layer.effects, true, c, whole);+ for (let i = 0; i < whole.length; i++) put(c, i, colors[i * 3]!, colors[i * 3 + 1]!, colors[i * 3 + 2]!, whole[i]! * opacity, blend);+ effectLayers(layer.effects, false, c, whole);+}++export interface RenderedImage extends RgbaBitmap { notes: string[] }++/** Draws a source that has been validated. */+export function renderImage(source: ImageSource): RenderedImage {+ const [w, h] = typeof source.size === 'number' ? [source.size, source.size] : source.size;+ const canvas = new Canvas(w, h);+ if (source.background) {+ const [r, g, b, a] = parseColor(source.background);+ for (let i = 0; i < w * h; i++) put(canvas, i, r, g, b, a, 'normal');+ }+ const drawn: Drawn = { masks: new Map(), notes: [] };+ for (const layer of source.layers) drawLayer(canvas, layer, IDENTITY, drawn);+ const rgba = new Uint8Array(w * h * 4);+ for (let i = 0; i < w * h; i++) {+ const a = canvas.px[i * 4 + 3]!;+ const d = a > 1e-5 ? 1 / a : 0;+ rgba[i * 4] = Math.round(Math.min(1, canvas.px[i * 4]! * d) * 255);+ rgba[i * 4 + 1] = Math.round(Math.min(1, canvas.px[i * 4 + 1]! * d) * 255);+ rgba[i * 4 + 2] = Math.round(Math.min(1, canvas.px[i * 4 + 2]! * d) * 255);+ rgba[i * 4 + 3] = Math.round(Math.min(1, a) * 255);+ }+ return { width: w, height: h, rgba, notes: drawn.notes };+}++function shrink(src: RgbaBitmap, width: number, height: number): RgbaBitmap {+ // Averaged by alpha: a transparent pixel's colour must not bleed into the edge beside it.+ const rgba = new Uint8Array(width * height * 4);+ for (let y = 0; y < height; y++) {+ const [y0, y1] = [Math.floor((y * src.height) / height), Math.max(Math.floor((y * src.height) / height) + 1, Math.floor(((y + 1) * src.height) / height))];+ for (let x = 0; x < width; x++) {+ const [x0, x1] = [Math.floor((x * src.width) / width), Math.max(Math.floor((x * src.width) / width) + 1, Math.floor(((x + 1) * src.width) / width))];+ let [r, g, b, a, n] = [0, 0, 0, 0, 0];+ for (let sy = y0; sy < y1; sy++) {+ for (let sx = x0; sx < x1; sx++) {+ const at = (sy * src.width + sx) * 4;+ const al = src.rgba[at + 3]!;+ r += src.rgba[at]! * al; g += src.rgba[at + 1]! * al; b += src.rgba[at + 2]! * al; a += al; n++;+ }+ }+ const to = (y * width + x) * 4;+ if (a > 0) { rgba[to] = r / a; rgba[to + 1] = g / a; rgba[to + 2] = b / a; }+ rgba[to + 3] = a / n;+ }+ }+ return { width, height, rgba };+}++/** Something to look at: the picture on dark and on a light check, as a transparent image needs both, and at the sizes an icon is really seen. */+export function imageSheet(image: RgbaBitmap, sizes: readonly number[] = [96, 64, 48, 32]): RgbaBitmap {+ const big = Math.max(image.width, image.height);+ const k = big < 384 ? Math.max(1, Math.floor(384 / big)) : Math.min(1, 512 / big);+ const [pw, ph] = [Math.round(image.width * k), Math.round(image.height * k)];+ const gap = 8;+ const smallest = sizes.length ? Math.max(...sizes) : 0;+ const W = pw * 2 + gap * 3 + (smallest ? smallest + gap : 0);+ const H = Math.max(ph + gap * 2, sizes.reduce((s, v) => s + v + gap, gap) * 1);+ const rgba = new Uint8Array(W * H * 4);+ for (let i = 0; i < W * H; i++) rgba.set([64, 64, 68, 255], i * 4);+ const scaled = k >= 1 ? nearestScale(image, pw, ph) : shrink(image, pw, ph);+ const lay = (pic: RgbaBitmap, ox: number, oy: number, ground: (x: number, y: number) => readonly [number, number, number]): void => {+ for (let y = 0; y < pic.height; y++) {+ for (let x = 0; x < pic.width; x++) {+ const from = (y * pic.width + x) * 4;+ const a = pic.rgba[from + 3]! / 255;+ const g = ground(x, y);+ const to = ((oy + y) * W + ox + x) * 4;+ rgba[to] = pic.rgba[from]! * a + g[0] * (1 - a);+ rgba[to + 1] = pic.rgba[from + 1]! * a + g[1] * (1 - a);+ rgba[to + 2] = pic.rgba[from + 2]! * a + g[2] * (1 - a);+ rgba[to + 3] = 255;+ }+ }+ };+ const dark = (): readonly [number, number, number] => [24, 26, 32];+ const check = (x: number, y: number): readonly [number, number, number] => ((Math.floor(x / 12) + Math.floor(y / 12)) % 2 ? [226, 226, 230] : [250, 250, 252]);+ lay(scaled, gap, gap, dark);+ lay(scaled, gap * 2 + pw, gap, check);+ let oy = gap;+ for (const s of sizes) {+ const small = shrink(image, s, Math.max(1, Math.round((s * image.height) / image.width)));+ lay(small, gap * 3 + pw * 2, oy, dark);+ oy += small.height + gap;+ }+ return { width: W, height: H, rgba };+}++function nearestScale(src: RgbaBitmap, width: number, height: number): RgbaBitmap {+ const rgba = new Uint8Array(width * height * 4);+ for (let y = 0; y < height; y++) {+ const sy = Math.floor((y * src.height) / height);+ for (let x = 0; x < width; x++) {+ const from = (sy * src.width + Math.floor((x * src.width) / width)) * 4;+ rgba.set(src.rgba.subarray(from, from + 4), (y * width + x) * 4);+ }+ }+ return { width, height, rgba };+}diff --git a/pipeline/src/image/imageSource.ts b/pipeline/src/image/imageSource.tsnew file mode 100644index 000000000..cac4b318d--- /dev/null+++ b/pipeline/src/image/imageSource.ts@@ -0,0 +1,255 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file imageSource.ts — `.esimage`: a picture as a list of layers, what a model writes and edits, and what is wrong with one.+ */+export const IMAGE_SOURCE_EXT = '.esimage';+export const IMAGE_SOURCE_VERSION = 1;+export const MAX_IMAGE_SIZE = 2048;++export type ImageFill = string+ | { type: 'linear'; from: readonly [number, number]; to: readonly [number, number]; stops: readonly (readonly [number, string])[] }+ | { type: 'radial'; at: readonly [number, number]; radius: number; stops: readonly (readonly [number, string])[] };++export interface ImageEffect {+ type: 'shadow' | 'glow' | 'innerShadow' | 'innerGlow';+ color?: string;+ x?: number;+ y?: number;+ blur?: number;+ opacity?: number;+}++export interface ImageTransform {+ translate?: readonly [number, number];+ /** Degrees, clockwise on screen, about `origin`. */+ rotate?: number;+ scale?: number | readonly [number, number];+ /** Default the layer's middle. */+ origin?: readonly [number, number];+}++export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'group';++export interface ImageLayer {+ id?: string;+ shape: ImageShape;+ at?: readonly [number, number];+ size?: readonly [number, number];+ radius?: number | readonly number[];+ radii?: readonly [number, number];+ inner?: number;+ count?: number;+ tooth?: number;+ turn?: number;+ points?: readonly (readonly [number, number])[];+ d?: string;+ rows?: readonly string[];+ palette?: Readonly<Record<string, string>>;+ scale?: number;+ layers?: readonly ImageLayer[];+ fill?: ImageFill;+ stroke?: { color: ImageFill; width: number; align?: 'center' | 'inside' | 'outside' };+ opacity?: number;+ blend?: 'normal' | 'multiply' | 'screen' | 'overlay' | 'add';+ transform?: ImageTransform;+ mirror?: 'x' | 'y';+ effects?: readonly ImageEffect[];+ clip?: string;+ erase?: boolean;+ visible?: boolean;+}++export interface ImageSource {+ version?: number;+ size: number | readonly [number, number];+ background?: string;+ layers: readonly ImageLayer[];+}++const isNum = (v: unknown): v is number => typeof v === 'number' && Number.isFinite(v);+const isPair = (v: unknown): boolean => Array.isArray(v) && v.length === 2 && v.every(isNum);+const isColor = (v: unknown): boolean => typeof v === 'string' && /^#(?:[0-9a-fA-F]{3,4}|[0-9a-fA-F]{6}|[0-9a-fA-F]{8})$/.test(v);++function checkFill(v: unknown): string | null {+ if (typeof v === 'string') return isColor(v) ? null : `${JSON.stringify(v)} must be #rgb, #rgba, #rrggbb or #rrggbbaa`;+ const f = v as Record<string, unknown> | null;+ if (typeof f !== 'object' || f === null) return 'must be a colour or a gradient';+ if (f.type !== 'linear' && f.type !== 'radial') return 'a gradient has type linear or radial';+ if (f.type === 'linear' && !(isPair(f.from) && isPair(f.to))) return 'a linear gradient needs from and to, each [x, y]';+ if (f.type === 'radial' && !(isPair(f.at) && isNum(f.radius) && (f.radius as number) > 0)) return 'a radial gradient needs at: [x, y] and a positive radius';+ if (!Array.isArray(f.stops) || f.stops.length < 2) return 'a gradient needs two stops or more, each [0..1, colour]';+ for (const s of f.stops as unknown[]) {+ if (!Array.isArray(s) || s.length !== 2 || !isNum(s[0]) || !isColor(s[1])) return `a stop ${JSON.stringify(s)} must be [0..1, colour]`;+ }+ return null;+}++const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible'];+const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'group'];++function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>): void {+ const fail = (what: string): never => { throw new Error(`${where}: ${what}`); };+ if (typeof l !== 'object' || l === null) fail('must be an object');+ for (const key of Object.keys(l)) if (!FIELDS.includes(key)) fail(`unknown field "${key}"; a layer takes ${FIELDS.join(', ')}`);+ if (!SHAPES.includes(l.shape as ImageShape)) fail(`shape ${JSON.stringify(l.shape)} must be one of ${SHAPES.join(', ')}`);+ if (l.id !== undefined) {+ if (typeof l.id !== 'string' || !l.id) fail('id must be a string');+ if (ids.has(l.id as string)) fail(`the id "${String(l.id)}" is already used`);+ ids.add(l.id as string);+ }+ const num = (k: string, ok: (n: number) => boolean = () => true): void => { if (!isNum(l[k]) || !ok(l[k] as number)) fail(`${k} must be a number${ok === positive ? ' above 0' : ''}`); };+ const pair = (k: string): void => { if (!isPair(l[k])) fail(`${k} must be [a, b]`); };+ const positive = (n: number): boolean => n > 0;+ switch (l.shape) {+ case 'rect': pair('at'); pair('size'); if (l.radius !== undefined && !isNum(l.radius) && !(Array.isArray(l.radius) && l.radius.length === 4 && l.radius.every(isNum))) fail('radius must be a number or four corners'); break;+ case 'circle': pair('at'); num('radius', positive); break;+ case 'ellipse': pair('at'); pair('radii'); break;+ case 'ring': pair('at'); num('radius', positive); num('inner', positive); if ((l.inner as number) >= (l.radius as number)) fail('inner must be less than radius'); break;+ case 'polygon': case 'line':+ if (!Array.isArray(l.points) || l.points.length < (l.shape === 'line' ? 2 : 3) || !l.points.every(isPair)) fail(`points must be ${l.shape === 'line' ? 'two' : 'three'} or more [x, y]`);+ if (l.shape === 'line' && !l.stroke) fail('a line needs a stroke: { color, width }');+ break;+ case 'star': case 'gear': pair('at'); num('radius', positive); num('inner', positive); if (!Number.isInteger(l.count) || (l.count as number) < 3) fail('count must be 3 or more'); break;+ case 'path': if (typeof l.d !== 'string' || !l.d.trim()) fail('d must be an SVG path: M L H V C S Q T Z'); break;+ case 'pixels': {+ if (!Array.isArray(l.rows) || l.rows.length === 0 || !l.rows.every((r) => typeof r === 'string')) fail('rows must be strings, one character a pixel');+ const pal = l.palette as Record<string, unknown> | undefined;+ if (typeof pal !== 'object' || pal === null) fail('palette must be { character: colour }');+ for (const [k, v] of Object.entries(pal!)) {+ if ([...k].length !== 1) fail(`palette key ${JSON.stringify(k)} must be one character`);+ if (!isColor(v)) fail(`palette "${k}" ${JSON.stringify(v)} must be a colour`);+ }+ const known = new Set(Object.keys(pal!));+ const width = (l.rows as string[])[0]!.length;+ (l.rows as string[]).forEach((row, i) => {+ if ([...row].length !== width) fail(`row ${i} is ${[...row].length} wide and row 0 is ${width}: a slip here shifts everything after it`);+ for (const ch of row) if (ch !== '.' && ch !== ' ' && !known.has(ch)) fail(`row ${i} has "${ch}", which is not in the palette; "." and " " are nothing`);+ });+ if (l.scale !== undefined) num('scale', positive);+ if (l.at !== undefined) pair('at');+ break;+ }+ case 'group':+ if (!Array.isArray(l.layers)) fail('a group needs layers');+ (l.layers as Record<string, unknown>[]).forEach((c, i) => checkLayer(c, `${where}.layers[${i}]`, ids));+ break;+ default:+ }+ if (l.fill !== undefined) { const bad = checkFill(l.fill); if (bad) fail(`fill ${bad}`); }+ if (l.stroke !== undefined) {+ const s = l.stroke as Record<string, unknown>;+ if (typeof s !== 'object' || s === null || !isNum(s.width) || (s.width as number) <= 0) fail('stroke must be { color, width, align? }');+ const bad = checkFill(s.color);+ if (bad) fail(`stroke color ${bad}`);+ if (s.align !== undefined && !['center', 'inside', 'outside'].includes(s.align as string)) fail('stroke align must be center, inside or outside');+ }+ if (!['group', 'pixels', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');+ if (l.opacity !== undefined && !(isNum(l.opacity) && l.opacity >= 0 && l.opacity <= 1)) fail('opacity must be 0..1');+ if (l.blend !== undefined && !['normal', 'multiply', 'screen', 'overlay', 'add'].includes(l.blend as string)) fail('blend must be normal, multiply, screen, overlay or add');+ if (l.mirror !== undefined && l.mirror !== 'x' && l.mirror !== 'y') fail('mirror must be x or y');+ if (l.clip !== undefined && typeof l.clip !== 'string') fail('clip must be a layer id');+ if (l.transform !== undefined) {+ const t = l.transform as Record<string, unknown>;+ for (const key of Object.keys(t)) if (!['translate', 'rotate', 'scale', 'origin'].includes(key)) fail(`transform has an unknown field "${key}"`);+ if (t.translate !== undefined && !isPair(t.translate)) fail('transform.translate must be [x, y]');+ if (t.origin !== undefined && !isPair(t.origin)) fail('transform.origin must be [x, y]');+ if (t.rotate !== undefined && !isNum(t.rotate)) fail('transform.rotate must be degrees');+ if (t.scale !== undefined && !isNum(t.scale) && !isPair(t.scale)) fail('transform.scale must be a number or [x, y]');+ }+ if (l.effects !== undefined) {+ if (!Array.isArray(l.effects)) fail('effects must be an array');+ for (const e of l.effects as Record<string, unknown>[]) {+ if (!['shadow', 'glow', 'innerShadow', 'innerGlow'].includes(e?.type as string)) fail(`an effect has type ${JSON.stringify(e?.type)}; there are shadow, glow, innerShadow, innerGlow`);+ for (const key of Object.keys(e)) if (!['type', 'color', 'x', 'y', 'blur', 'opacity'].includes(key)) fail(`effect ${String(e.type)} has an unknown field "${key}"`);+ if (e.color !== undefined && !isColor(e.color)) fail(`effect ${String(e.type)}: color must be #rgb..#rrggbbaa`);+ for (const k of ['x', 'y', 'blur', 'opacity']) if (e[k] !== undefined && !isNum(e[k])) fail(`effect ${String(e.type)}: ${k} must be a number`);+ }+ }+}++/** The source if it is well formed; otherwise throws, naming the layer and the field. */+export function validateImageSource(raw: unknown): ImageSource {+ if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) throw new Error('an image source must be { size, layers }');+ const s = raw as Record<string, unknown>;+ for (const key of Object.keys(s)) if (!['version', 'size', 'background', 'layers'].includes(key)) throw new Error(`unknown field "${key}"; an image has version, size, background, layers`);+ if (s.version !== undefined && s.version !== IMAGE_SOURCE_VERSION) throw new Error(`version ${String(s.version)} is not one this editor reads (${IMAGE_SOURCE_VERSION})`);+ const [w, h] = typeof s.size === 'number' ? [s.size, s.size] : isPair(s.size) ? (s.size as number[]) : [NaN, NaN];+ if (!Number.isInteger(w) || !Number.isInteger(h) || w! < 1 || h! < 1 || w! > MAX_IMAGE_SIZE || h! > MAX_IMAGE_SIZE) throw new Error(`size must be a whole number or [width, height], 1..${MAX_IMAGE_SIZE}`);+ if (s.background !== undefined && !isColor(s.background)) throw new Error('background must be a colour; leave it out for a transparent picture');+ if (!Array.isArray(s.layers)) throw new Error('layers must be an array');+ const ids = new Set<string>();+ (s.layers as Record<string, unknown>[]).forEach((l, i) => checkLayer(l, `layer ${i}${l?.id ? ` "${String(l.id)}"` : ''}`, ids));+ return { version: IMAGE_SOURCE_VERSION, ...(s as unknown as ImageSource) };+}++export type ImageEdit =+ | { op: 'set'; layer: string; fields: Record<string, unknown> }+ | { op: 'insert'; layer: ImageLayer; before?: string; after?: string; into?: string }+ | { op: 'remove'; layer: string }+ | { op: 'settings'; fields: { size?: ImageSource['size']; background?: string | null } };++/** A source with @p edits applied in order, where layers are named by id at any depth. Throws, naming what it could not find. */+export function applyImageEdits(source: ImageSource, edits: readonly ImageEdit[]): ImageSource {+ const clone = JSON.parse(JSON.stringify(source)) as { size: ImageSource['size']; background?: string; version?: number; layers: (ImageLayer & Record<string, unknown>)[] };+ type Row = ImageLayer & Record<string, unknown>;+ const allIds: string[] = [];+ const find = (id: string, list: Row[] = clone.layers): { list: Row[]; index: number } | null => {+ for (let i = 0; i < list.length; i++) {+ if (list[i]!.id === id) return { list, index: i };+ if (list[i]!.layers) {+ const hit = find(id, list[i]!.layers as Row[]);+ if (hit) return hit;+ }+ }+ return null;+ };+ const need = (id: string): { list: Row[]; index: number } => {+ const hit = find(id);+ if (hit) return hit;+ allIds.length = 0;+ const walk = (list: Row[]): void => { for (const l of list) { if (l.id) allIds.push(l.id); if (l.layers) walk(l.layers as Row[]); } };+ walk(clone.layers);+ throw new Error(`no layer "${id}"; ids: ${allIds.join(', ') || '(none: give layers an id to edit them)'}`);+ };+ for (const edit of edits) {+ switch (edit?.op) {+ case 'set': {+ const { list, index } = need(edit.layer);+ for (const [k, v] of Object.entries(edit.fields ?? {})) {+ if (v === null) delete list[index]![k];+ else list[index]![k] = v;+ }+ break;+ }+ case 'insert': {+ if (edit.into !== undefined) {+ const { list, index } = need(edit.into);+ const group = list[index]!;+ if (group.shape !== 'group') throw new Error(`layer "${edit.into}" is not a group`);+ (group.layers as Row[]).push(edit.layer as Row);+ } else if (edit.before !== undefined) {+ const { list, index } = need(edit.before);+ list.splice(index, 0, edit.layer as Row);+ } else if (edit.after !== undefined) {+ const { list, index } = need(edit.after);+ list.splice(index + 1, 0, edit.layer as Row);+ } else clone.layers.push(edit.layer as Row);+ break;+ }+ case 'remove': {+ const { list, index } = need(edit.layer);+ list.splice(index, 1);+ break;+ }+ case 'settings':+ if (edit.fields?.size !== undefined) clone.size = edit.fields.size;+ if (edit.fields?.background === null) delete clone.background;+ else if (edit.fields?.background !== undefined) clone.background = edit.fields.background;+ break;+ default:+ throw new Error(`an edit has op ${JSON.stringify((edit as { op?: unknown })?.op)}; there are set, insert, remove, settings`);+ }+ }+ return validateImageSource(clone);+}diff --git a/pipeline/tests/image-raster.test.ts b/pipeline/tests/image-raster.test.tsnew file mode 100644index 000000000..a4aeb606f--- /dev/null+++ b/pipeline/tests/image-raster.test.ts@@ -0,0 +1,383 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/** A picture from layers: each draws what it says and where, edges are antialiased, and what is wrong with a source is said before anything is drawn. */+import { describe, expect, it } from 'vitest';+import { imageSheet, parseColor, parsePath, renderImage } from '../src/image/imageRaster';+import { applyImageEdits, validateImageSource, type ImageLayer } from '../src/image/imageSource';++const draw = (layers: unknown[], size: number | [number, number] = 64, background?: string): ReturnType<typeof renderImage> => renderImage(validateImageSource({ size, layers, ...(background ? { background } : {}) }));+const px = (img: { width: number; rgba: Uint8Array }, x: number, y: number): number[] => Array.from(img.rgba.subarray((y * img.width + x) * 4, (y * img.width + x) * 4 + 4));+const RED = '#ff0000', BLUE = '#0000ff';++describe('colours', () => {+ it('reads every length', () => {+ expect(parseColor('#f00')).toEqual([1, 0, 0, 1]);+ expect(parseColor('#ff000080')[3]).toBeCloseTo(0.5, 2);+ expect(parseColor('#0f08')[3]).toBeCloseTo(0.533, 2);+ expect(() => parseColor('red')).toThrow(/must be/);+ });+});++describe('shapes', () => {+ it('a rect fills what it says, and nothing else is touched', () => {+ const img = draw([{ shape: 'rect', at: [10, 20], size: [30, 10], fill: RED }]);+ expect(px(img, 20, 25)).toEqual([255, 0, 0, 255]);+ expect(px(img, 5, 25)[3]).toBe(0);+ expect(px(img, 45, 25)[3]).toBe(0);+ expect(px(img, 20, 35)[3]).toBe(0);+ });++ it('a round corner takes the corner off', () => {+ const img = draw([{ shape: 'rect', at: [10, 10], size: [40, 40], radius: 12, fill: RED }]);+ expect(px(img, 10, 10)[3]).toBe(0);+ expect(px(img, 30, 11)[3]).toBe(255);+ const corners = draw([{ shape: 'rect', at: [10, 10], size: [40, 40], radius: [12, 0, 0, 0], fill: RED }]);+ expect(px(corners, 10, 10)[3]).toBe(0);+ expect(px(corners, 49, 10)[3]).toBe(255);+ });++ it('edges are antialiased: a circle has pixels partly covered, in about the right amount', () => {+ const img = draw([{ shape: 'circle', at: [32, 32], radius: 20, fill: RED }]);+ let partial = 0, area = 0;+ for (let i = 0; i < 64 * 64; i++) {+ const a = img.rgba[i * 4 + 3]!;+ if (a > 0 && a < 255) partial++;+ area += a / 255;+ }+ expect(partial).toBeGreaterThan(40);+ expect(area).toBeCloseTo(Math.PI * 400, -1);+ });++ it('a half-covered pixel is half: a vertical edge through the middle of a column', () => {+ const img = draw([{ shape: 'rect', at: [10.5, 0], size: [20, 64], fill: RED }]);+ expect(px(img, 10, 30)[3]).toBeGreaterThan(120);+ expect(px(img, 10, 30)[3]).toBeLessThan(135);+ });++ it('a ring has a hole, a star its points, a gear its teeth', () => {+ const ring = draw([{ shape: 'ring', at: [32, 32], radius: 20, inner: 10, fill: RED }]);+ expect(px(ring, 32, 32)[3]).toBe(0);+ expect(px(ring, 32, 17)[3]).toBe(255);+ const star = draw([{ shape: 'star', at: [32, 32], radius: 28, inner: 10, count: 5, fill: RED }]);+ expect(px(star, 32, 8)[3]).toBe(255);+ expect(px(star, 50, 12)[3]).toBe(0);+ const gear = draw([{ shape: 'gear', at: [32, 32], radius: 28, inner: 20, count: 8, fill: RED }], 64);+ expect(px(gear, 32, 6)[3]).toBe(255);+ // Between two teeth, at the height of their tips.+ const a = (-90 + 22.5) * (Math.PI / 180);+ expect(px(gear, Math.round(32 + 26 * Math.cos(a)), Math.round(32 + 26 * Math.sin(a)))[3]).toBe(0);+ });++ it('a polygon is what its points say, and a line needs a stroke', () => {+ const tri = draw([{ shape: 'polygon', points: [[10, 10], [50, 10], [10, 50]], fill: RED }]);+ expect(px(tri, 15, 15)[3]).toBe(255);+ expect(px(tri, 45, 45)[3]).toBe(0);+ const line = draw([{ shape: 'line', points: [[10, 32], [50, 32]], stroke: { color: RED, width: 4 } }]);+ expect(px(line, 30, 32)[3]).toBe(255);+ expect(px(line, 30, 40)[3]).toBe(0);+ });+});++describe('paths', () => {+ it('reads its commands, absolute and relative', () => {+ expect(parsePath('M0 0 L10 0 L10 10 Z')[0]).toEqual({ points: [[0, 0], [10, 0], [10, 10]], closed: true });+ expect(parsePath('M1 1 l2 0 h3 v4')[0]!.points).toEqual([[1, 1], [3, 1], [6, 1], [6, 5]]);+ });++ it('a curve goes through its ends and bows toward its handles', () => {+ const pts = parsePath('M0 0 Q5 10 10 0')[0]!.points;+ expect(pts[0]).toEqual([0, 0]);+ expect(pts[pts.length - 1]).toEqual([10, 0]);+ expect(Math.max(...pts.map((p) => p[1]))).toBeCloseTo(5, 1);+ const cubic = parsePath('M0 0 C0 10 10 10 10 0 S20 -10 20 0')[0]!.points;+ expect(Math.min(...cubic.map((p) => p[1]))).toBeLessThan(-5);+ expect(cubic[cubic.length - 1]).toEqual([20, 0]);+ });++ it('has more than one run, and a hole winds the other way', () => {+ expect(parsePath('M0 0 L1 0 L1 1 Z M5 5 L6 5 L6 6 Z')).toHaveLength(2);+ const hole = draw([{ shape: 'path', d: 'M10 10 H54 V54 H10 Z M20 20 V44 H44 V20 Z', fill: RED }]);+ expect(px(hole, 32, 32)[3]).toBe(0);+ expect(px(hole, 15, 32)[3]).toBe(255);+ });++ it('says what it does not draw', () => {+ expect(() => parsePath('M0 0 A5 5 0 0 1 10 10')).toThrow(/M L H V C S Q T Z/);+ expect(() => parsePath('M0')).toThrow(/short of numbers/);+ });+});++describe('fills and strokes', () => {+ it('a linear gradient runs from one end to the other', () => {+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: { type: 'linear', from: [0, 0], to: [64, 0], stops: [[0, '#000000'], [1, '#ffffff']] } }]);+ expect(px(img, 1, 30)[0]).toBeLessThan(15);+ expect(px(img, 62, 30)[0]).toBeGreaterThan(240);+ expect(px(img, 32, 30)[0]).toBeGreaterThan(115);+ expect(px(img, 32, 30)[0]).toBeLessThan(140);+ });++ it('a radial gradient runs from its middle out, and a gradient turns with its layer', () => {+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: { type: 'radial', at: [32, 32], radius: 30, stops: [[0, '#ffffff'], [1, '#000000']] } }]);+ expect(px(img, 32, 32)[0]).toBeGreaterThan(245);+ expect(px(img, 2, 32)[0]).toBeLessThan(10);+ const turned = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], transform: { rotate: 90, origin: [32, 32] }, fill: { type: 'linear', from: [0, 0], to: [64, 0], stops: [[0, '#000000'], [1, '#ffffff']] } }]);+ expect(px(turned, 32, 2)[0]).toBeLessThan(15);+ expect(px(turned, 32, 61)[0]).toBeGreaterThan(240);+ });++ it('a stroke goes round the edge, centred, inside or outside', () => {+ const at = { shape: 'rect', at: [20, 20], size: [24, 24], fill: RED };+ const center = draw([{ ...at, stroke: { color: BLUE, width: 4 } }]);+ expect(px(center, 19, 32)[2]).toBe(255);+ expect(px(center, 20, 32)[2]).toBe(255);+ const inside = draw([{ ...at, stroke: { color: BLUE, width: 4, align: 'inside' } }]);+ expect(px(inside, 18, 32)[3]).toBe(0);+ expect(px(inside, 21, 32)[2]).toBeGreaterThan(200);+ const outside = draw([{ ...at, stroke: { color: BLUE, width: 4, align: 'outside' } }]);+ expect(px(outside, 18, 32)[2]).toBeGreaterThan(200);+ expect(px(outside, 21, 32)).toEqual([255, 0, 0, 255]);+ });+});++describe('layers', () => {+ it('later ones go over earlier ones', () => {+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: RED }, { shape: 'rect', at: [0, 0], size: [32, 64], fill: BLUE }]);+ expect(px(img, 10, 10)).toEqual([0, 0, 255, 255]);+ expect(px(img, 50, 10)).toEqual([255, 0, 0, 255]);+ });++ it('opacity lets what is behind through, and a colour\'s own alpha counts too', () => {+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: '#ffffff' }, { shape: 'rect', at: [0, 0], size: [64, 64], fill: '#000000', opacity: 0.5 }]);+ expect(px(img, 10, 10)[0]).toBeGreaterThan(120);+ expect(px(img, 10, 10)[0]).toBeLessThan(135);+ expect(px(img, 10, 10)[3]).toBe(255);+ const own = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: '#00000080' }]);+ expect(px(own, 10, 10)[3]).toBeCloseTo(128, -1);+ });++ it('blends: multiply darkens, screen and add lighten', () => {+ const base = { shape: 'rect', at: [0, 0], size: [64, 64], fill: '#808080' };+ const over = (blend: string): number => px(draw([base, { shape: 'rect', at: [0, 0], size: [64, 64], fill: '#808080', blend }]), 10, 10)[0]!;+ expect(over('multiply')).toBeLessThan(80);+ expect(over('screen')).toBeGreaterThan(180);+ expect(over('add')).toBe(255);+ });++ it('transforms: moved, turned about its middle, scaled', () => {+ const moved = draw([{ shape: 'rect', at: [0, 0], size: [10, 10], fill: RED, transform: { translate: [30, 30] } }]);+ expect(px(moved, 35, 35)[3]).toBe(255);+ expect(px(moved, 5, 5)[3]).toBe(0);+ const turned = draw([{ shape: 'rect', at: [12, 30], size: [40, 4], fill: RED, transform: { rotate: 90 } }]);+ expect(px(turned, 32, 15)[3]).toBe(255);+ expect(px(turned, 15, 32)[3]).toBe(0);+ const big = draw([{ shape: 'rect', at: [27, 27], size: [10, 10], fill: RED, transform: { scale: 2 } }]);+ expect(px(big, 24, 24)[3]).toBe(255);+ });++ it('mirror draws it again across the picture', () => {+ const img = draw([{ shape: 'rect', at: [4, 10], size: [8, 8], fill: RED, mirror: 'x' }]);+ expect(px(img, 8, 14)[3]).toBe(255);+ expect(px(img, 55, 14)[3]).toBe(255);+ expect(px(img, 32, 14)[3]).toBe(0);+ const down = draw([{ shape: 'rect', at: [4, 4], size: [8, 8], fill: RED, mirror: 'y' }]);+ expect(px(down, 8, 55)[3]).toBe(255);+ });++ it('clip keeps it within an earlier layer', () => {+ const img = draw([{ id: 'c', shape: 'circle', at: [32, 32], radius: 10, fill: BLUE }, { shape: 'rect', at: [0, 32], size: [64, 32], fill: RED, clip: 'c' }]);+ expect(px(img, 32, 38)).toEqual([255, 0, 0, 255]);+ expect(px(img, 32, 26)).toEqual([0, 0, 255, 255]);+ expect(px(img, 5, 50)[3]).toBe(0);+ });++ it('says when a clip names nothing, and when a layer draws nothing', () => {+ expect(draw([{ shape: 'rect', at: [0, 0], size: [10, 10], fill: RED, clip: 'gone' }]).notes.join()).toMatch(/clip "gone"/);+ expect(draw([{ shape: 'rect', at: [500, 500], size: [10, 10], fill: RED, id: 'away' }]).notes.join()).toMatch(/away draws nothing/);+ });++ it('erase cuts through what is there', () => {+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: RED }, { shape: 'circle', at: [32, 32], radius: 10, fill: '#000000', erase: true }]);+ expect(px(img, 32, 32)[3]).toBe(0);+ expect(px(img, 5, 5)[3]).toBe(255);+ });++ it('a group is one thing: its effect goes round the whole, its erase stays within it, and its transform carries its children', () => {+ const img = draw([+ { shape: 'rect', at: [0, 0], size: [64, 64], fill: BLUE },+ { shape: 'group', layers: [{ shape: 'rect', at: [10, 10], size: [40, 40], fill: RED }, { shape: 'circle', at: [30, 30], radius: 6, fill: '#000', erase: true }] },+ ]);+ expect(px(img, 30, 30)).toEqual([0, 0, 255, 255]);+ const half = draw([{ shape: 'group', opacity: 0.5, layers: [{ shape: 'rect', at: [0, 0], size: [64, 64], fill: RED }, { shape: 'rect', at: [0, 0], size: [64, 64], fill: BLUE }] }]);+ expect(px(half, 30, 30)[0]).toBe(0);+ expect(px(half, 30, 30)[3]).toBeCloseTo(128, -1);+ const moved = draw([{ shape: 'group', transform: { translate: [30, 0] }, layers: [{ shape: 'rect', at: [0, 0], size: [10, 10], fill: RED }] }]);+ expect(px(moved, 35, 5)[3]).toBe(255);+ expect(px(moved, 5, 5)[3]).toBe(0);+ });+});++describe('effects', () => {+ const square = { shape: 'rect', at: [24, 24], size: [16, 16], fill: '#ffffff' };++ it('a shadow falls where it is put, softly, and under the shape', () => {+ const img = draw([{ ...square, effects: [{ type: 'shadow', x: 6, y: 6, blur: 2, color: '#000000' }] }]);+ expect(px(img, 32, 32)).toEqual([255, 255, 255, 255]);+ expect(px(img, 42, 42)[3]).toBeGreaterThan(100);+ expect(px(img, 21, 21)[3]).toBeLessThan(30);+ // Soft: there is a fall from solid to nothing.+ const a = [41, 44, 47, 50].map((x) => px(img, x, 34)[3]!);+ expect(a[0]!).toBeGreaterThan(a[3]!);+ expect(a.some((v) => v > 0 && v < 255)).toBe(true);+ });++ it('a glow goes out all round', () => {+ const img = draw([{ ...square, effects: [{ type: 'glow', blur: 4, color: '#ff0000' }] }]);+ for (const [x, y] of [[20, 32], [44, 32], [32, 20], [32, 44]]) {+ expect(px(img, x!, y!)[3]).toBeGreaterThan(20);+ expect(px(img, x!, y!)[0]).toBe(255);+ }+ });++ it('an inner shadow stays inside, on the side it is put', () => {+ const img = draw([{ shape: 'rect', at: [16, 16], size: [32, 32], fill: '#ffffff', effects: [{ type: 'innerShadow', y: 6, blur: 2, color: '#000000' }] }]);+ expect(px(img, 32, 18)[0]).toBeLessThan(120);+ expect(px(img, 32, 45)[0]).toBeGreaterThan(220);+ expect(px(img, 32, 10)[3]).toBe(0);+ });++ it('an inner glow lightens the edge', () => {+ const img = draw([{ shape: 'rect', at: [16, 16], size: [32, 32], fill: '#404040', effects: [{ type: 'innerGlow', blur: 3, color: '#ffffff' }] }]);+ expect(px(img, 17, 32)[0]).toBeGreaterThan(px(img, 32, 32)[0]! + 40);+ });+});++describe('pixels', () => {+ const palette = { r: RED, b: BLUE };++ it('draws hard squares, as big as fit, in the middle', () => {+ const img = draw([{ shape: 'pixels', palette, rows: ['r.', '.b'] }], 8);+ expect(px(img, 0, 0)).toEqual([255, 0, 0, 255]);+ expect(px(img, 3, 3)).toEqual([255, 0, 0, 255]);+ expect(px(img, 4, 0)[3]).toBe(0);+ expect(px(img, 4, 4)).toEqual([0, 0, 255, 255]);+ // No pixel in it is partly anything.+ for (let i = 0; i < 64; i++) expect([0, 255]).toContain(img.rgba[i * 4 + 3]);+ });++ it('takes a scale and a place', () => {+ const img = draw([{ shape: 'pixels', palette, rows: ['r'], scale: 3, at: [10, 10] }], 32);+ expect(px(img, 12, 12)[3]).toBe(255);+ expect(px(img, 13, 12)[3]).toBe(0);+ });++ it('takes effects like any layer', () => {+ const img = draw([{ shape: 'pixels', palette, rows: ['r'], scale: 8, at: [12, 12], effects: [{ type: 'shadow', x: 4, y: 4, blur: 0, color: '#000000' }] }], 32);+ expect(px(img, 22, 22)).toEqual([0, 0, 0, 255]);+ });++ it('says what is wrong with its rows, which a slip would otherwise shift', () => {+ expect(() => draw([{ shape: 'pixels', palette, rows: ['rr', 'r'] }])).toThrow(/row 1 is 1 wide and row 0 is 2/);+ expect(() => draw([{ shape: 'pixels', palette, rows: ['x'] }])).toThrow(/"x", which is not in the palette/);+ expect(() => draw([{ shape: 'pixels', palette: { ab: RED }, rows: ['a'] }])).toThrow(/one character/);+ });+});++describe('a picture', () => {+ it('is the same every time', () => {+ const layers = [{ shape: 'star', at: [32, 32], radius: 28, inner: 12, count: 7, fill: { type: 'radial', at: [32, 32], radius: 28, stops: [[0, '#fff'], [1, '#f00']] }, stroke: { color: '#000', width: 2 }, effects: [{ type: 'glow', color: '#0ff' }, { type: 'innerShadow', y: 3 }] }];+ expect(draw(layers).rgba).toEqual(draw(layers).rgba);+ });++ it('has a background, or is transparent, and a shape of its own', () => {+ expect(px(draw([], 8, '#123456'), 2, 2)).toEqual([0x12, 0x34, 0x56, 255]);+ expect(px(draw([], 8), 2, 2)[3]).toBe(0);+ const wide = draw([], [16, 4]);+ expect([wide.width, wide.height]).toEqual([16, 4]);+ });++ it('a colour does not go dark at a soft edge', () => {+ const img = draw([{ shape: 'circle', at: [32, 32], radius: 10.3, fill: '#ff8040' }]);+ let seen = 0;+ for (let i = 0; i < 64 * 64; i++) {+ const a = img.rgba[i * 4 + 3]!;+ if (a > 8 && a < 247) { seen++; expect(img.rgba[i * 4]).toBe(255); expect(Math.abs(img.rgba[i * 4 + 1]! - 0x80)).toBeLessThan(5); }+ }+ expect(seen).toBeGreaterThan(10);+ });+});++describe('the sheet to look at', () => {+ const img = draw([{ shape: 'rect', at: [0, 0], size: [64, 64], fill: '#ff000080' }]);++ it('is opaque, with the picture over two grounds and at small sizes', () => {+ const sheet = imageSheet(img, [32, 16]);+ for (let i = 3; i < sheet.rgba.length; i += 4) expect(sheet.rgba[i]).toBe(255);+ expect(sheet.width).toBeGreaterThan(img.width * 2);+ const k = Math.floor(384 / 64);+ const onDark = Array.from(sheet.rgba.subarray((20 * sheet.width + 20) * 4, (20 * sheet.width + 20) * 4 + 3));+ const onLight = Array.from(sheet.rgba.subarray((20 * sheet.width + 8 * 2 + 64 * k + 20) * 4, (20 * sheet.width + 8 * 2 + 64 * k + 20) * 4 + 3));+ expect(onLight[1]!).toBeGreaterThan(onDark[1]! + 60);+ });++ it('enlarges a small picture without blurring it', () => {+ const pixel = draw([{ shape: 'pixels', palette: { r: RED }, rows: ['r.', '.r'] }], 2);+ const sheet = imageSheet(pixel, []);+ const edge = Array.from(sheet.rgba.subarray((10 * sheet.width + 8 + 100) * 4, (10 * sheet.width + 8 + 100) * 4 + 3));+ expect(edge).toEqual([255, 0, 0]);+ });+});++describe('the source', () => {+ const ok = (layers: unknown[]): unknown => ({ size: 16, layers });++ it('names the layer and the field', () => {+ expect(() => validateImageSource({ layers: [] })).toThrow(/size/);+ expect(() => validateImageSource({ size: 0, layers: [] })).toThrow(/size/);+ expect(() => validateImageSource({ size: 9999, layers: [] })).toThrow(/size/);+ expect(() => validateImageSource({ size: 16 })).toThrow(/layers/);+ expect(() => validateImageSource({ size: 16, layers: [], extra: 1 })).toThrow(/unknown field "extra"/);+ expect(() => validateImageSource(ok([{ shape: 'blob' }]))).toThrow(/layer 0.*shape "blob"/);+ expect(() => validateImageSource(ok([{ shape: 'rect', at: [0, 0], size: [1, 1] }]))).toThrow(/needs a fill or a stroke/);+ expect(() => validateImageSource(ok([{ shape: 'rect', at: [0, 0], fill: RED }]))).toThrow(/size/);+ expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: 'red' }]))).toThrow(/fill "red"/);+ expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: RED, wobble: 1 }]))).toThrow(/unknown field "wobble"/);+ expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: { type: 'linear', from: [0, 0], to: [1, 1], stops: [[0, RED]] } }]))).toThrow(/two stops/);+ expect(() => validateImageSource(ok([{ shape: 'ring', at: [0, 0], radius: 1, inner: 2, fill: RED }]))).toThrow(/inner must be less/);+ expect(() => validateImageSource(ok([{ shape: 'circle', at: [0, 0], radius: 1, fill: RED, effects: [{ type: 'bevel' }] }]))).toThrow(/bevel/);+ expect(() => validateImageSource(ok([{ id: 'a', shape: 'circle', at: [0, 0], radius: 1, fill: RED }, { id: 'a', shape: 'circle', at: [0, 0], radius: 1, fill: RED }]))).toThrow(/already used/);+ expect(() => validateImageSource(ok([{ shape: 'group', layers: [{ shape: 'blob' }] }]))).toThrow(/layers\[0\]/);+ });+});++describe('edits', () => {+ const base = validateImageSource({ size: 16, layers: [{ id: 'a', shape: 'circle', at: [8, 8], radius: 4, fill: RED }, { id: 'g', shape: 'group', layers: [{ id: 'b', shape: 'circle', at: [8, 8], radius: 2, fill: BLUE }] }] });+ const ids = (layers: readonly ImageLayer[]): string[] => layers.map((l) => l.id!);++ it('sets a field, at any depth, and a null takes it away', () => {+ const next = applyImageEdits(base, [{ op: 'set', layer: 'b', fields: { radius: 3 } }, { op: 'set', layer: 'a', fields: { opacity: 0.5 } }, { op: 'set', layer: 'a', fields: { opacity: null } }]);+ expect(next.layers[1]!.layers![0]!.radius).toBe(3);+ expect(next.layers[0]!.opacity).toBeUndefined();+ expect(base.layers[1]!.layers![0]!.radius).toBe(2);+ });++ it('inserts where it is told', () => {+ const c = { id: 'c', shape: 'circle' as const, at: [1, 1] as const, radius: 1, fill: RED };+ expect(ids(applyImageEdits(base, [{ op: 'insert', layer: c }]).layers)).toEqual(['a', 'g', 'c']);+ expect(ids(applyImageEdits(base, [{ op: 'insert', layer: c, before: 'a' }]).layers)).toEqual(['c', 'a', 'g']);+ expect(ids(applyImageEdits(base, [{ op: 'insert', layer: c, after: 'a' }]).layers)).toEqual(['a', 'c', 'g']);+ expect(ids(applyImageEdits(base, [{ op: 'insert', layer: c, into: 'g' }]).layers[1]!.layers!)).toEqual(['b', 'c']);+ });++ it('removes, and changes the settings', () => {+ expect(ids(applyImageEdits(base, [{ op: 'remove', layer: 'a' }]).layers)).toEqual(['g']);+ expect(applyImageEdits(base, [{ op: 'settings', fields: { size: 32, background: '#000' } }]).size).toBe(32);+ expect(applyImageEdits(base, [{ op: 'settings', fields: { background: '#000' } }, { op: 'settings', fields: { background: null } }]).background).toBeUndefined();+ });++ it('says what it could not find, and refuses what would not be valid', () => {+ expect(() => applyImageEdits(base, [{ op: 'remove', layer: 'zz' }])).toThrow(/no layer "zz"; ids: a, g, b/);+ expect(() => applyImageEdits(base, [{ op: 'insert', layer: {} as never, into: 'a' }])).toThrow(/not a group/);+ expect(() => applyImageEdits(base, [{ op: 'set', layer: 'a', fields: { radius: -1 } }])).toThrow(/radius/);+ expect(() => applyImageEdits(base, [{ op: 'spin' } as never])).toThrow(/set, insert, remove, settings/);+ });+});diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex 7bb5ffa1d..307125c16 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -3884,6 +3884,10 @@ export const NOTED = {{"note":"A material's normal and metal-rough maps are read as data in the editor's Play."},"fix(minigame): no mini-game module needs BigInt64Array, so WeChat DevTools boots the game":{"note":"A WeChat mini-game opens in WeChat DevTools."},+ "feat(look): layers and pixels drawn to a picture, on the CPU":+ {"note":"An AI can draw 2D pictures."},+ "feat(mcp): image_write, image_edit and image_read":+ {"note":"An AI can draw 2D pictures."},"feat(modeling): a whole person from a few numbers, clothes from the body's own surface, and a pose by where a hand should be":{"note":"An AI can build a person, and not a picture of one."},"feat(mcp): model_write describes a person, and scene_views looks round a subject without a shot's lens":