A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(look): a paint layer, brush strokes on a canvas replayed in order
yhh2e638b375c02
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 784a4ae71..039e1524f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,7 +22,7 @@ published separately; it ships inside the editor.- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.-- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.+- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.- **An AI can build a person, and not a picture of one.** The `humanoid` step makes a whole body from a few numbers, with fingers, a face and ears, seen right from every side, and `rig: { humanoid: true }` gives it bones that fit. `garment` makes clothes from the body's own surface: loose or tight, a bell sleeve that hangs, a clean hem, a thickness; what a body wears now takes its weights from the skin under it, so an arm stays in its sleeve. `pose` holds a body in a pose by angles, or with `reach` by where a hand should be. `organic` gains `limb`, cross-sections along a path, and `fine`, regions split finer on one skin. `displace` gains `folds`, running the way cloth hangs. `scene_views` no longer takes on a shot's lens blur and grade.diff --git a/desktop b/desktopindex 6bee848c8..963622c6d 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit 6bee848c8cb599abe7b2c84f39debf0e4012ac92+Subproject commit 963622c6dfa8aa61f566b878751de2572da34017diff --git a/pipeline/src/image/imageRaster.ts b/pipeline/src/image/imageRaster.tsindex dd8f70066..88ae725ae 100644--- a/pipeline/src/image/imageRaster.ts+++ b/pipeline/src/image/imageRaster.ts@@ -7,7 +7,7 @@* effects, and this owns everything that has to be exact: antialiased edges, gradients, soft shadows.* On the CPU rather than through a canvas, so the same source makes the same bytes on any machine and a test can read them.*/-import type { ImageFill, ImageLayer, ImageSource, ImageEffect, ImageTransform } from './imageSource';+import type { ImageFill, ImageLayer, ImageSource, ImageEffect, ImageStroke, ImageTransform } from './imageSource';export interface RgbaBitmap { width: number; height: number; rgba: Uint8Array }@@ -642,6 +642,11 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):return;}+ if (layer.shape === 'paint') {+ drawPaint(c, layer, parent, drawn, opacity, blend, clip);+ return;+ }+const { runs, middle } = outlinesOf(layer);const m = times(parent, transformOf(layer.transform, middle));const placed = runs.map((run) => ({ closed: run.closed, points: run.points.map((p) => apply(m, p)) }));@@ -709,6 +714,186 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):const drawnScale = (m: Matrix): number => Math.sqrt(Math.abs(m[0] * m[3] - m[1] * m[2])) || 1;/** Rows of characters, each a palette's colour, drawn as hard squares. */+// ---- Brushes ---------------------------------------------------------------------++const hash01 = (a: number, b: number): number => {+ let h = Math.imul(a, 374761393) + Math.imul(b, 668265263);+ h = Math.imul(h ^ (h >>> 13), 1274126177);+ h = Math.imul(h ^ (h >>> 16), 2246822519);+ return ((h ^ (h >>> 16)) >>> 0) / 4294967295;+};++const smoothed = (t: number): number => t * t * (3 - 2 * t);++interface PathPoint { x: number; y: number; pressure: number; along: number }++/** A stroke's points as a smooth line about a pixel apart, Catmull-Rom so it passes through each. */+function brushPath(points: readonly (readonly number[])[], m: Matrix): PathPoint[] {+ const placed = points.map((p) => {+ const q = apply(m, [p[0]!, p[1]!]);+ return [q[0], q[1], p[2] ?? 1] as const;+ });+ const out: PathPoint[] = [{ x: placed[0]![0], y: placed[0]![1], pressure: placed[0]![2], along: 0 }];+ for (let i = 0; i + 1 < placed.length; i++) {+ const [p0, p1, p2, p3] = [placed[Math.max(0, i - 1)]!, placed[i]!, placed[i + 1]!, placed[Math.min(placed.length - 1, i + 2)]!];+ const steps = Math.max(1, Math.ceil(Math.hypot(p2[0] - p1[0], p2[1] - p1[1])));+ for (let s = 1; s <= steps; s++) {+ const t = s / steps, t2 = t * t, t3 = t2 * t;+ const curve = (k: 0 | 1): number => 0.5 * (2 * p1[k] + (p2[k] - p0[k]) * t + (2 * p0[k] - 5 * p1[k] + 4 * p2[k] - p3[k]) * t2 + (3 * p1[k] - p0[k] - 3 * p2[k] + p3[k]) * t3);+ const [x, y] = [curve(0), curve(1)];+ const last = out[out.length - 1]!;+ out.push({ x, y, pressure: p1[2] + (p2[2] - p1[2]) * t, along: last.along + Math.hypot(x - last.x, y - last.y) });+ }+ }+ return out;+}++/** One stroke's coverage before it meets the canvas, so a stroke overlapping itself does not build up into beads. */+interface StrokeBuffer { mask: Float32Array; rgb: Float32Array; touched: number[] }++function paintStrand(c: Canvas, s: ImageStroke, path: readonly PathPoint[], strand: { offset: number; radius: number | null; from: number; to: number; shade: number; strength: number; seed: number }, scale: number, buffer: StrokeBuffer): void {+ const { w, h } = c;+ const total = path[path.length - 1]!.along;+ const [startColor, endColor] = [parseColor(s.color ?? '#000000'), parseColor(s.to ?? s.color ?? '#000000')];+ const [taperIn, taperOut] = s.taper ?? [0, 0];+ const hardness = s.hardness ?? 0.5;+ const size = s.size * scale;+ const smudge = s.smudge ?? 0;+ const wobble = (s.wobble ?? 0) * scale;++ const sample = (x: number, y: number): [number, number, number, number] => {+ const [px, py] = [Math.max(0, Math.min(w - 1, Math.floor(x))), Math.max(0, Math.min(h - 1, Math.floor(y)))];+ const at = (py * w + px) * 4;+ return [c.px[at]!, c.px[at + 1]!, c.px[at + 2]!, c.px[at + 3]!];+ };+ let carried: [number, number, number, number] | null = null;++ let index = 0;+ let distance = total * strand.from;+ const end = total * strand.to;+ for (;;) {+ while (index + 1 < path.length - 1 && path[index + 1]!.along < distance) index++;+ const [a, b] = [path[index]!, path[Math.min(path.length - 1, index + 1)]!];+ const span = b.along - a.along;+ const t = span > 0 ? Math.max(0, Math.min(1, (distance - a.along) / span)) : 0;+ const position = total > 0 ? distance / total : 0;+ let narrowing = 1;+ if (taperIn > 0 && position < taperIn) narrowing *= smoothed(position / taperIn);+ if (taperOut > 0 && position > 1 - taperOut) narrowing *= smoothed((1 - position) / taperOut);+ const width = size * (a.pressure + (b.pressure - a.pressure) * t) * narrowing;+ const radius = strand.radius === null ? width / 2 : Math.min(strand.radius, Math.max(width, strand.radius * 0.4));+ const [nx, ny] = span > 0 ? [-(b.y - a.y) / span, (b.x - a.x) / span] : [0, 0];+ const drift = wobble > 0 ? gradientNoise(distance / (size * 3 + 8), strand.seed * 0.37 + 0.5, strand.seed) * 2 * wobble : 0;+ const away = strand.offset * width + drift;+ const [cx, cy] = [a.x + (b.x - a.x) * t + nx * away, a.y + (b.y - a.y) * t + ny * away];++ if (radius >= 0.12) {+ const soft = Math.max(1, radius * (1 - hardness));+ const thin = Math.min(1, radius * 2);+ // Flat at the top: stamps of a pointed falloff, overlapped, ripple along the stroke.+ const cover = (d: number): number => {+ const v = Math.max(0, Math.min(1, (radius + 0.5 - d) / soft));+ return soft > 1 ? smoothed(v) : v;+ };+ const [x0, x1] = [Math.max(0, Math.floor(cx - radius - 1)), Math.min(w - 1, Math.ceil(cx + radius + 1))];+ const [y0, y1] = [Math.max(0, Math.floor(cy - radius - 1)), Math.min(h - 1, Math.ceil(cy + radius + 1))];+ if (smudge > 0) {+ carried ??= sample(cx, cy);+ for (let y = y0; y <= y1; y++) {+ for (let x = x0; x <= x1; x++) {+ const v = cover(Math.hypot(x + 0.5 - cx, y + 0.5 - cy)) * thin * smudge * (s.opacity ?? 1);+ if (v <= 0) continue;+ const at = (y * w + x) * 4;+ for (let k = 0; k < 4; k++) c.px[at + k] = c.px[at + k]! + (carried[k]! - c.px[at + k]!) * v * 0.5;+ }+ }+ const here = sample(cx, cy);+ for (let k = 0; k < 4; k++) carried[k] = carried[k]! + (here[k]! - carried[k]!) * (1 - smudge) * 0.5;+ } else {+ const along = total > 0 ? distance / total : 0;+ const mix = (k: number): number => Math.max(0, Math.min(1, (startColor[k]! + (endColor[k]! - startColor[k]!) * along) * strand.shade));+ const [r, g, bl] = [mix(0), mix(1), mix(2)];+ const alpha = (startColor[3] + (endColor[3] - startColor[3]) * along) * (s.opacity ?? 1) * strand.strength;+ for (let y = y0; y <= y1; y++) {+ for (let x = x0; x <= x1; x++) {+ const v = cover(Math.hypot(x + 0.5 - cx, y + 0.5 - cy)) * thin * alpha;+ const i = y * w + x;+ if (v <= buffer.mask[i]!) continue;+ if (buffer.mask[i] === 0) buffer.touched.push(i);+ buffer.mask[i] = v;+ buffer.rgb[i * 3] = r; buffer.rgb[i * 3 + 1] = g; buffer.rgb[i * 3 + 2] = bl;+ }+ }+ }+ }+ if (distance >= end || total === 0) break;+ distance = Math.min(end, distance + Math.max(0.4, radius * 0.1));+ }+ // Laid down all at once, so opacity is the stroke's and not the sum of its stamps.+ const mode: Blend = s.blend ?? 'normal';+ for (const i of buffer.touched) {+ const v = buffer.mask[i]!;+ if (s.erase) for (let k = 0; k < 4; k++) c.px[i * 4 + k]! *= 1 - v;+ else put(c, i, buffer.rgb[i * 3]!, buffer.rgb[i * 3 + 1]!, buffer.rgb[i * 3 + 2]!, v, mode);+ buffer.mask[i] = 0;+ }+ buffer.touched.length = 0;+}++function paintStroke(c: Canvas, s: ImageStroke, m: Matrix, scale: number, n: number, buffer: StrokeBuffer): void {+ const path = brushPath(s.points, m);+ const seed = s.seed ?? n + 1;+ const count = s.bristles ?? 1;+ if (count === 1) {+ paintStrand(c, s, path, { offset: 0, radius: null, from: 0, to: 1, shade: 1, strength: 1, seed }, scale, buffer);+ return;+ }+ const spread = s.spread ?? 1;+ const vary = s.vary ?? 0.15;+ const radius = ((s.strand ?? Math.max(0.8, (s.size / count) * 2.2)) * scale) / 2;+ for (let j = 0; j < count; j++) {+ const pick = (k: number): number => hash01(seed * 131 + j, k);+ paintStrand(c, s, path, {+ // Even across the width and then shaken, so bristles neither clump nor stand in rows.+ offset: (((j + pick(1)) / count) - 0.5) * spread,+ radius,+ from: pick(2) * 0.06,+ to: 1 - pick(3) * 0.1,+ shade: 1 + (pick(4) - 0.5) * 2 * vary,+ strength: 1 - pick(5) * vary * 1.5,+ seed: seed * 31 + j,+ }, scale, buffer);+ }+}++function drawPaint(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {+ const { w, h } = c;+ const m = times(parent, transformOf(layer.transform, [w / 2, h / 2]));+ const inner = new Canvas(w, h);+ if (layer.fill !== undefined) {+ const paint = paintOf(layer.fill, m);+ for (let y = 0; y < h; y++) {+ for (let x = 0; x < w; x++) {+ const col = paint(x + 0.5, y + 0.5);+ put(inner, y * w + x, col[0], col[1], col[2], col[3], 'normal');+ }+ }+ }+ const buffer: StrokeBuffer = { mask: new Float32Array(w * h), rgb: new Float32Array(w * h * 3), touched: [] };+ (layer.strokes ?? []).forEach((s, n) => paintStroke(inner, s, m, drawnScale(m), n, buffer));+ const alpha = new Float32Array(w * h);+ for (let i = 0; i < alpha.length; i++) alpha[i] = inner.px[i * 4 + 3]!;+ if (layer.id) drawn.masks.set(layer.id, alpha);+ noteBox(drawn, layer.id, alpha, w, h);+ effectLayers(layer.effects, true, c, alpha);+ for (let i = 0; i < alpha.length; i++) {+ const a = alpha[i]!;+ if (a <= 0) continue;+ put(c, i, inner.px[i * 4]! / a, inner.px[i * 4 + 1]! / a, inner.px[i * 4 + 2]! / a, a * opacity * (clip ? clip[i]! : 1), blend);+ }+ effectLayers(layer.effects, false, c, alpha);+}+function drawPixels(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {const { w, h } = c;const rows = layer.rows!;diff --git a/pipeline/src/image/imageSource.ts b/pipeline/src/image/imageSource.tsindex 980ff9326..1d131c70c 100644--- a/pipeline/src/image/imageSource.ts+++ b/pipeline/src/image/imageSource.ts@@ -33,7 +33,36 @@ export interface ImageTransform {origin?: readonly [number, number];}-export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'group';+export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'group';++/** One movement of a brush. Replayed in order on a `paint` layer, so the same list is the same picture. */+export interface ImageStroke {+ /** Through these, smoothly; a third number is pressure, 1 by default. One point is a dab. */+ points: readonly (readonly number[])[];+ color?: string;+ /** The colour it has become by the end. */+ to?: string;+ size: number;+ /** 1 is a hard edge, 0 fades from the middle. */+ hardness?: number;+ opacity?: number;+ /** How much of the length at each end narrows to a point, each 0..1. */+ taper?: readonly [number, number];+ /** Separate hairs across its width, which meet where it narrows. */+ bristles?: number;+ /** The width of a bristle; by default what spreads them across the stroke. */+ strand?: number;+ spread?: number;+ /** How far each bristle's lightness and strength differ, 0..1. */+ vary?: number;+ /** How far a bristle wanders from its line, in pixels. */+ wobble?: number;+ seed?: number;+ /** 0..1: drags what is already on this layer along, instead of laying colour. */+ smudge?: number;+ erase?: boolean;+ blend?: 'normal' | 'multiply' | 'screen' | 'overlay' | 'add';+}export interface ImageLayer {id?: string;@@ -52,6 +81,7 @@ export interface ImageLayer {palette?: Readonly<Record<string, string>>;scale?: number;layers?: readonly ImageLayer[];+ strokes?: readonly ImageStroke[];fill?: ImageFill;stroke?: { color: ImageFill; width: number; align?: 'center' | 'inside' | 'outside'; join?: 'round' | 'miter' | 'bevel'; cap?: 'round' | 'butt' | 'square' };opacity?: number;@@ -96,8 +126,26 @@ function checkFill(v: unknown): string | null {return null;}-const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur'];-const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'group'];+const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes'];+const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'group'];+const STROKE_FIELDS = ['points', 'color', 'to', 'size', 'hardness', 'opacity', 'taper', 'bristles', 'strand', 'spread', 'vary', 'wobble', 'seed', 'smudge', 'erase', 'blend'];++function strokeProblem(s: Record<string, unknown>): string | null {+ if (typeof s !== 'object' || s === null) return 'must be an object';+ for (const key of Object.keys(s)) if (!STROKE_FIELDS.includes(key)) return `has unknown field "${key}"; a stroke takes ${STROKE_FIELDS.join(', ')}`;+ if (!Array.isArray(s.points) || s.points.length === 0 || !s.points.every((p) => Array.isArray(p) && (p.length === 2 || p.length === 3) && p.every(isNum))) return 'needs points, each [x, y] or [x, y, pressure]';+ if (!(isNum(s.size) && (s.size as number) > 0)) return 'needs a positive size';+ const paints = !s.erase && !s.smudge;+ if (paints && s.color === undefined) return 'needs a color, unless it erases or smudges';+ for (const key of ['color', 'to']) if (s[key] !== undefined && !isColor(s[key])) return `${key} must be #rgb, #rgba, #rrggbb or #rrggbbaa`;+ for (const key of ['hardness', 'opacity', 'vary', 'smudge']) if (s[key] !== undefined && !(isNum(s[key]) && (s[key] as number) >= 0 && (s[key] as number) <= 1)) return `${key} must be 0..1`;+ for (const key of ['strand', 'spread', 'wobble']) if (s[key] !== undefined && !(isNum(s[key]) && (s[key] as number) >= 0)) return `${key} must be 0 or more`;+ if (s.bristles !== undefined && !(Number.isInteger(s.bristles) && (s.bristles as number) >= 1 && (s.bristles as number) <= 200)) return 'bristles must be a whole number from 1 to 200';+ if (s.seed !== undefined && !Number.isInteger(s.seed)) return 'seed must be a whole number';+ if (s.taper !== undefined && !(Array.isArray(s.taper) && s.taper.length === 2 && s.taper.every((t) => isNum(t) && t >= 0 && t <= 1))) return 'taper must be [start, end], each 0..1';+ if (s.blend !== undefined && !['normal', 'multiply', 'screen', 'overlay', 'add'].includes(s.blend as string)) return 'blend must be normal, multiply, screen, overlay or add';+ return null;+}function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>): void {const fail = (what: string): never => { throw new Error(`${where}: ${what}`); };@@ -131,6 +179,14 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)fail((e as Error).message);}break;+ case 'paint': {+ if (!Array.isArray(l.strokes)) fail('strokes must be an array, which may start empty');+ (l.strokes as Record<string, unknown>[]).forEach((s, n) => {+ const bad = strokeProblem(s);+ if (bad) fail(`stroke ${n} ${bad}`);+ });+ break;+ }case 'pixels': {if (!Array.isArray(l.rows) || l.rows.length === 0 || !l.rows.every((r) => typeof r === 'string')) fail('rows must be strings, one character a pixel');const pal = l.palette as Record<string, unknown> | undefined;@@ -165,7 +221,7 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)if (s.join !== undefined && !['round', 'miter', 'bevel'].includes(s.join as string)) fail('stroke join must be round, miter or bevel');if (s.cap !== undefined && !['round', 'butt', 'square'].includes(s.cap as string)) fail('stroke cap must be round, butt or square');}- if (!['group', 'pixels', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');+ if (!['group', 'pixels', 'paint', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');if (l.blur !== undefined && !(isNum(l.blur) && (l.blur as number) >= 0)) fail('blur must be 0 or more, in pixels');if (l.blur && l.erase) fail('blur does not go with erase; put what cuts in a group with what it cuts, and blur the group');if (l.opacity !== undefined && !(isNum(l.opacity) && l.opacity >= 0 && l.opacity <= 1)) fail('opacity must be 0..1');@@ -211,6 +267,7 @@ export type ImageEdit =| { op: 'insert'; layer: ImageLayer; before?: string; after?: string; into?: string }| { op: 'remove'; layer: string }| { op: 'move'; layer: string; before?: string; after?: string; into?: string }+ | { op: 'paint'; layer: string; strokes?: readonly ImageStroke[]; drop?: number }| { op: 'wrap'; layers: readonly string[]; group: Partial<ImageLayer> }| { op: 'unwrap'; layer: string }| {@@ -312,6 +369,16 @@ export function applyImageEdits(source: ImageSource, edits: readonly ImageEdit[]} else clone.layers.push(row!);break;}+ case 'paint': {+ const { list, index } = need(edit.layer);+ const layer = list[index]!;+ if (layer.shape !== 'paint') throw new Error(`layer "${edit.layer}" is not paint`);+ const had = layer.strokes as readonly ImageStroke[];+ const drop = edit.drop ?? 0;+ if (!Number.isInteger(drop) || drop < 0 || drop > had.length) throw new Error(`layer "${edit.layer}" has ${had.length} strokes: cannot drop ${String(edit.drop)}`);+ layer.strokes = [...had.slice(0, had.length - drop), ...(edit.strokes ?? [])];+ break;+ }case 'wrap': {if (!Array.isArray(edit.layers) || edit.layers.length === 0) throw new Error('wrap needs layers, the ids to put in the group');const found = edit.layers.map((id) => ({ id, ...need(id) }));@@ -376,7 +443,7 @@ export function applyImageEdits(source: ImageSource, edits: readonly ImageEdit[]else if (edit.fields?.background !== undefined) clone.background = edit.fields.background;break;default:- throw new Error(`an edit has op ${JSON.stringify((edit as { op?: unknown })?.op)}; there are set, insert, remove, move, wrap, unwrap, pixels, settings`);+ throw new Error(`an edit has op ${JSON.stringify((edit as { op?: unknown })?.op)}; there are set, insert, remove, move, paint, wrap, unwrap, pixels, settings`);}}return validateImageSource(clone);diff --git a/pipeline/tests/image-raster.test.ts b/pipeline/tests/image-raster.test.tsindex d625678b0..28890626a 100644--- a/pipeline/tests/image-raster.test.ts+++ b/pipeline/tests/image-raster.test.ts@@ -612,6 +612,137 @@ describe('noise', () => {});});+describe('brushes', () => {+ const paint = (strokes: object[], extra: object = {}): ImageLayer => ({ id: 'p', shape: 'paint', strokes, ...extra } as never);+ const across = (fields: object = {}): object => ({ points: [[8, 32], [56, 32]], color: RED, size: 8, hardness: 1, ...fields });++ it('lays a line of its size between its points, and a dab at one', () => {+ const img = draw([paint([across()])]);+ expect(px(img, 32, 32)).toEqual([255, 0, 0, 255]);+ expect(px(img, 32, 20)[3]).toBe(0);+ const dab = draw([paint([{ points: [[32, 32]], color: BLUE, size: 10, hardness: 1 }])]);+ expect(px(dab, 32, 32)).toEqual([0, 0, 255, 255]);+ expect(px(dab, 32, 45)[3]).toBe(0);+ });++ it('passes through each point of a curve', () => {+ const img = draw([paint([{ points: [[8, 50], [32, 12], [56, 50]], color: RED, size: 6, hardness: 1 }])]);+ expect(px(img, 32, 12)[3]).toBe(255);+ expect(px(img, 32, 40)[3]).toBe(0);+ });++ it('does not build into beads: opacity is the stroke\'s, however it overlaps itself', () => {+ const img = draw([paint([across({ opacity: 0.5, points: [[8, 32], [56, 32], [8, 32]] })])]);+ const values = new Set<number>();+ for (let x = 14; x < 50; x++) values.add(px(img, x, 32)[3]!);+ expect(values.size).toBeLessThanOrEqual(2);+ expect(px(img, 32, 32)[3]).toBeGreaterThan(120);+ expect(px(img, 32, 32)[3]).toBeLessThan(135);+ });++ it('leaves no ripple of its stamps along a large soft stroke', () => {+ const img = draw([paint([{ points: [[-40, 90], [300, 90]], color: '#ffffff', size: 120, hardness: 0 }])], [256, 256]);+ for (const y of [90, 110, 130]) {+ const values: number[] = [];+ for (let x = 40; x < 216; x++) values.push(px(img, x, y)[3]!);+ expect(Math.max(...values) - Math.min(...values)).toBeLessThanOrEqual(3);+ }+ });++ it('is softer the lower its hardness', () => {+ const edge = (hardness: number): number => px(draw([paint([across({ size: 24, hardness })])]), 32, 24)[3]!;+ expect(edge(1)).toBe(255);+ expect(edge(0)).toBeLessThan(200);+ });++ it('narrows to a point where it tapers, and with pressure', () => {+ const img = draw([paint([across({ size: 16, taper: [0, 0.6] })])]);+ const width = (x: number): number => { let n = 0; for (let y = 16; y < 48; y++) if (px(img, x, y)[3]! > 128) n++; return n; };+ expect(width(16)).toBeGreaterThan(width(50));+ expect(width(50)).toBeLessThan(6);+ const pressed = draw([paint([{ points: [[8, 32, 1], [56, 32, 0.1]], color: RED, size: 16, hardness: 1 }])]);+ expect(px(pressed, 12, 26)[3]).toBe(255);+ expect(px(pressed, 52, 26)[3]).toBe(0);+ });++ it('turns from one colour to the next', () => {+ const img = draw([paint([across({ to: BLUE })])]);+ expect(px(img, 12, 32)[0]).toBeGreaterThan(px(img, 12, 32)[2]!);+ expect(px(img, 52, 32)[2]).toBeGreaterThan(px(img, 52, 32)[0]!);+ });++ it('with bristles leaves separate hairs that meet where it narrows', () => {+ const hairs = draw([paint([across({ size: 40, bristles: 8, strand: 1, vary: 0, taper: [0, 0.3], seed: 2 })])], [64, 64]);+ let runs = 0, inside = false;+ for (let y = 8; y < 56; y++) {+ const on = px(hairs, 16, y)[3]! > 100;+ if (on && !inside) runs++;+ inside = on;+ }+ expect(runs).toBeGreaterThan(3);+ let far = 0;+ for (let y = 0; y < 64; y++) if (px(hairs, 50, y)[3]! > 60 && Math.abs(y - 32) > 12) far++;+ expect(far).toBe(0);+ });++ it('is the same from the same list, and bristles differ by seed', () => {+ const s = across({ size: 30, bristles: 12, vary: 0.3, wobble: 2, seed: 4 });+ expect(draw([paint([s])]).rgba).toEqual(draw([paint([s])]).rgba);+ expect(draw([paint([s])]).rgba).not.toEqual(draw([paint([{ ...s, seed: 5 }])]).rgba);+ });++ it('smudges what is on the layer along, and lays no colour of its own', () => {+ const base = { points: [[20, 10], [20, 54]], color: RED, size: 20, hardness: 1 };+ const before = draw([paint([base])]);+ const after = draw([paint([base, { points: [[20, 32], [56, 32]], size: 10, smudge: 0.9 }])]);+ expect(px(before, 40, 32)[3]).toBe(0);+ expect(px(after, 40, 32)[3]).toBeGreaterThan(20);+ expect(px(after, 40, 32)[0]).toBeGreaterThan(200);+ });++ it('rubs out, and paints by blend', () => {+ const rubbed = draw([paint([across({ size: 20 }), { points: [[32, 10], [32, 54]], size: 8, hardness: 1, erase: true }])]);+ expect(px(rubbed, 32, 32)[3]).toBe(0);+ expect(px(rubbed, 16, 32)[3]).toBe(255);+ const shaded = draw([paint([across({ size: 20 }), across({ color: '#808080', size: 6, blend: 'multiply' })])]);+ expect(px(shaded, 32, 32)[0]).toBeGreaterThan(110);+ expect(px(shaded, 32, 32)[0]).toBeLessThan(140);+ });++ it('starts from a fill, follows the transform, the clip and the layer\'s opacity, and is measured', () => {+ const filled = draw([paint([], { fill: BLUE })]);+ expect(px(filled, 3, 3)).toEqual([0, 0, 255, 255]);+ const moved = draw([paint([across()], { transform: { translate: [0, 20], origin: [0, 0] } })]);+ expect(px(moved, 32, 52)[3]).toBe(255);+ expect(moved.layerBounds.p![1]).toBeGreaterThanOrEqual(46);+ const clipped = draw([{ id: 'c', shape: 'rect', at: [0, 0], size: [32, 64], fill: BLUE }, paint([across()], { clip: 'c', opacity: 0.5 })]);+ expect(px(clipped, 40, 32)[0]).toBe(0);+ expect(px(clipped, 20, 32)[0]).toBeGreaterThan(110);+ });++ it('takes more strokes without restating the rest, and drops the last few', () => {+ const start = validateImageSource({ size: 64, layers: [paint([across()])] });+ const more = applyImageEdits(start, [{ op: 'paint', layer: 'p', strokes: [across({ color: BLUE }) as never] }]);+ expect(more.layers[0]!.strokes!.length).toBe(2);+ expect(applyImageEdits(more, [{ op: 'paint', layer: 'p', drop: 1 }]).layers[0]!.strokes!.length).toBe(1);+ expect(() => applyImageEdits(more, [{ op: 'paint', layer: 'p', drop: 3 }])).toThrow(/has 2 strokes/);+ expect(() => applyImageEdits(more, [{ op: 'paint', layer: 'p', strokes: [{ size: 3 } as never] }])).toThrow(/stroke 2 needs points/);+ const notPaint = validateImageSource({ size: 8, layers: [{ id: 'r', shape: 'rect', at: [0, 0], size: [2, 2], fill: RED }] });+ expect(() => applyImageEdits(notPaint, [{ op: 'paint', layer: 'r', strokes: [] }])).toThrow(/not paint/);+ });++ it('says which stroke is wrong, and how', () => {+ const bad = (s: object): unknown => validateImageSource({ size: 8, layers: [{ id: 'p', shape: 'paint', strokes: [{ points: [[1, 1]], size: 2, color: RED, ...s }] }] });+ expect(() => bad({ points: [] })).toThrow(/layer 0 "p".*stroke 0 needs points/);+ expect(() => bad({ size: 0 })).toThrow(/positive size/);+ expect(() => bad({ color: undefined })).toThrow(/needs a color/);+ expect(() => bad({ color: undefined, smudge: 0.5 })).not.toThrow();+ expect(() => bad({ bristles: 300 })).toThrow(/bristles must be/);+ expect(() => bad({ taper: [0, 2] })).toThrow(/taper must be/);+ expect(() => bad({ blur: 2 })).toThrow(/unknown field "blur"/);+ });+});+describe('what it measures', () => {it('finds where the picture is solid and how much of it that is', () => {const img = draw([{ shape: 'rect', at: [10, 20], size: [16, 8], fill: RED }]);@@ -807,6 +938,6 @@ describe('edits', () => {expect(() => applyImageEdits(base, [{ op: 'remove', layer: 'zz' }])).toThrow(/no layer "zz"; ids: a, g, b/);expect(() => applyImageEdits(base, [{ op: 'insert', layer: {} as never, into: 'a' }])).toThrow(/not a group/);expect(() => applyImageEdits(base, [{ op: 'set', layer: 'a', fields: { radius: -1 } }])).toThrow(/radius/);- expect(() => applyImageEdits(base, [{ op: 'spin' } as never])).toThrow(/set, insert, remove, move, wrap, unwrap, pixels, settings/);+ expect(() => applyImageEdits(base, [{ op: 'spin' } as never])).toThrow(/set, insert, remove, move, paint, wrap, unwrap, pixels, settings/);});});diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex 82a9d2912..29efd60e0 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -3888,6 +3888,10 @@ export const NOTED = {{"note":"An AI can draw 2D pictures."},"feat(look): layers moved and wrapped, one pixel changed, a set and a close look drawn":{"note":"An AI can draw 2D pictures."},+ "feat(look): a paint layer, brush strokes on a canvas replayed in order":+ {"note":"An AI can draw 2D pictures."},+ "feat(mcp): image_write and image_edit paint with brush strokes":+ {"note":"An AI can draw 2D pictures."},"feat(look): a picture's layers blurred, and noise as a fill":{"note":"An AI can draw 2D pictures."},"feat(mcp): image_write documents blur and noise":