A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(look): a posed body to draw over, its joints in the picture and light by its form
yhh2351f87a2da4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 79b52c3ca..c8cc5c39f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,7 +22,7 @@ published separately; it ships inside the editor.- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.-- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do; one `sweep` instruction lays a mass of hair gathered into locks, one `field` covers a region in strokes along a flow, and one `along` repeats strokes along a line. A layer can work across everything under it, to fuse separate parts into one painting, and `image_read` shows lightness alone and reads colours. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.+- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do; one `sweep` instruction lays a mass of hair gathered into locks, one `field` covers a region in strokes along a flow, and one `along` repeats strokes along a line. A layer can work across everything under it, to fuse separate parts into one painting, and `image_read` shows lightness alone and reads colours. A `figure` layer is a real posed body from any angle to draw over: left out of the `.png`, it reports where each joint landed, offers each body part as a region, and lets light follow the form. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.- **An AI can build a person, and not a picture of one.** The `humanoid` step makes a whole body from a few numbers, with fingers, a face and ears, seen right from every side, and `rig: { humanoid: true }` gives it bones that fit. `garment` makes clothes from the body's own surface: loose or tight, a bell sleeve that hangs, a clean hem, a thickness; what a body wears now takes its weights from the skin under it, so an arm stays in its sleeve. `pose` holds a body in a pose by angles, or with `reach` by where a hand should be. `organic` gains `limb`, cross-sections along a path, and `fine`, regions split finer on one skin. `displace` gains `folds`, running the way cloth hangs. `scene_views` no longer takes on a shot's lens blur and grade.diff --git a/desktop b/desktopindex b0652f84b..33505a82d 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit b0652f84b13824d1a5155b486f61719f07262726+Subproject commit 33505a82d2ce560078c993d74a8bf3e0f92b3e65diff --git a/pipeline/src/image/imageFigure.ts b/pipeline/src/image/imageFigure.tsnew file mode 100644index 000000000..90e71f6d9--- /dev/null+++ b/pipeline/src/image/imageFigure.ts@@ -0,0 +1,199 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file imageFigure.ts+ * @brief A real human body, posed and seen from an angle, flattened to what a 2D picture can draw over.+ *+ * Coordinates typed blind drift: a body comes out wide or stiff and nothing says so. This is the 3D body the modeling tools build,+ * so its proportions and its foreshortening are right from any side, and what is reported is where each joint landed in the picture.+ */+import { humanoid, type HumanoidBuild } from '../modeling/humanoid';+import { poseJoints, poseStep, type PoseParams } from '../modeling/pose';+import type { PolyMesh, Vec3 } from '../modeling/polyMesh';+import type { ImageLayer } from './imageSource';++/** The table's own height: figures are built at it, and sized in the picture. */+const BODY_HEIGHT = 1.6;+/** The lowest the body builds at. Finer costs seconds and shows nothing at picture scale. */+const BODY_RESOLUTION = 160;+const SUPERSAMPLE = 2;+const KEPT = 6;++export const FIGURE_TAGS = ['torso', 'arm', 'hand', 'leg', 'foot', 'neck', 'face', 'scalp', 'ear', 'lips', 'lid', 'eye', 'iris'] as const;+export const FIGURE_JOINTS = [+ 'crown', 'head', 'neck', 'chest', 'spine', 'hips',+ 'upperArm.L', 'forearm.L', 'hand.L', 'middle4.L', 'upperArm.R', 'forearm.R', 'hand.R', 'middle4.R',+ 'thigh.L', 'shin.L', 'foot.L', 'toeTip.L', 'thigh.R', 'shin.R', 'foot.R', 'toeTip.R',+] as const;++/** Most recently used last, as building takes seconds and the same body is drawn again on every edit. */+const cache = new Map<string, unknown>();+function remembered<T>(key: string, make: () => T): T {+ if (cache.has(key)) {+ const value = cache.get(key) as T;+ cache.delete(key);+ cache.set(key, value);+ return value;+ }+ const value = make();+ cache.set(key, value);+ if (cache.size > KEPT) cache.delete(cache.keys().next().value as string);+ return value;+}++interface PosedBody { positions: readonly Vec3[]; normals: Float32Array; triangles: Uint32Array; tags: Uint8Array }++function bodyOf(build: HumanoidBuild, eyes: 'open' | 'closed' | undefined, pose: PoseParams): PosedBody {+ const buildKey = JSON.stringify({ build, eyes });+ return remembered(`posed:${buildKey}:${JSON.stringify(pose)}`, () => {+ const rest = remembered(`rest:${buildKey}`, () => humanoid({ positions: [], faces: [] } as PolyMesh, { ...build, height: BODY_HEIGHT, resolution: BODY_RESOLUTION, ...(eyes ? { eyes } : {}) }));+ const mesh = poseStep(rest, pose);+ let count = 0;+ for (const f of mesh.faces) count += f.v.length - 2;+ const triangles = new Uint32Array(count * 3);+ const tags = new Uint8Array(count);+ const normals = new Float32Array(mesh.positions.length * 3);+ let t = 0;+ for (const f of mesh.faces) {+ const tag = Math.max(0, FIGURE_TAGS.indexOf(f.tag as (typeof FIGURE_TAGS)[number]));+ for (let i = 1; i + 1 < f.v.length; i++, t++) {+ const [a, b, c] = [f.v[0]!, f.v[i]!, f.v[i + 1]!];+ triangles[t * 3] = a; triangles[t * 3 + 1] = b; triangles[t * 3 + 2] = c;+ tags[t] = tag;+ const [pa, pb, pc] = [mesh.positions[a]!, mesh.positions[b]!, mesh.positions[c]!];+ const [ux, uy, uz, vx, vy, vz] = [pb[0] - pa[0], pb[1] - pa[1], pb[2] - pa[2], pc[0] - pa[0], pc[1] - pa[1], pc[2] - pa[2]];+ const [nx, ny, nz] = [uy * vz - uz * vy, uz * vx - ux * vz, ux * vy - uy * vx];+ for (const v of [a, b, c]) { normals[v * 3]! += nx; normals[v * 3 + 1]! += ny; normals[v * 3 + 2]! += nz; }+ }+ }+ for (let v = 0; v < mesh.positions.length; v++) {+ const len = Math.hypot(normals[v * 3]!, normals[v * 3 + 1]!, normals[v * 3 + 2]!) || 1;+ normals[v * 3]! /= len; normals[v * 3 + 1]! /= len; normals[v * 3 + 2]! /= len;+ }+ return { positions: mesh.positions, normals, triangles, tags };+ });+}++export interface FlatFigure {+ /** Coverage of the whole body in picture pixels, then of each kind of part as it shows. */+ mask: Float32Array;+ parts: Record<string, Float32Array>;+ /** Which way the surface faces at each pixel, x right, y up, z toward the viewer; zero off the body. */+ normals: Float32Array;+ /** Dark where one surface passes in front of another, for seeing the form through a guide. */+ contour: Float32Array;+ joints: Record<string, readonly [number, number]>;+ light: readonly [number, number, number];+}++const normalized = (v: readonly number[]): [number, number, number] => {+ const len = Math.hypot(v[0]!, v[1]!, v[2]!) || 1;+ return [v[0]! / len, v[1]! / len, v[2]! / len];+};++/** Flattens a `figure` layer. @p project takes a point in the picture before any group's transform, and @p scale is that transform's size. */+export function flattenFigure(layer: ImageLayer, w: number, h: number, project: (p: readonly [number, number]) => readonly [number, number], scale: number): FlatFigure {+ const [ax, ay] = project(layer.at!);+ const s = ((layer.height as number) / BODY_HEIGHT) * scale;+ const yaw = ((layer.yaw ?? 0) * Math.PI) / 180, pitch = ((layer.pitch ?? 0) * Math.PI) / 180;+ const [cy, sy, cp, sp] = [Math.cos(yaw), Math.sin(yaw), Math.cos(pitch), Math.sin(pitch)];+ /** Yaw about the body, then pitch toward the viewer: what the body's axes are in the view. */+ const toView = (p: readonly number[]): [number, number, number] => {+ const [x, y, z] = [p[0]! * cy + p[2]! * sy, p[1]!, -p[0]! * sy + p[2]! * cy];+ return [x, y * cp - z * sp, y * sp + z * cp];+ };+ const fromView = (v: readonly number[]): [number, number, number] => {+ const [y, z] = [v[1]! * cp + v[2]! * sp, -v[1]! * sp + v[2]! * cp];+ return [v[0]! * cy - z * sy, y, v[0]! * sy + z * cy];+ };+ const build = (layer.build ?? {}) as HumanoidBuild;+ const pose = layer.pose ?? {};+ const held = { humanoid: build, ...(pose.rotate ? { rotate: pose.rotate } : {}), ...(pose.move ? { move: pose.move } : {}) } as PoseParams;+ // A reach given by two numbers is a place in the picture: where along the line of sight is where the joint already is.+ const reach = Object.entries(pose.reach ?? {}).map(([name, r]) => [name, r] as const);+ let resolved: PoseParams = held;+ if (reach.length) {+ const before = poseJoints(held);+ const entries = reach.map(([name, r]) => {+ if (!before[name]) throw new Error(`no joint "${name}"; there are ${Object.keys(before).join(', ')}`);+ const at = r.at.length === 3 ? r.at : fromView([(r.at[0]! - ax) / s, (ay - r.at[1]!) / s, toView(before[name]!)[2]]);+ return [name, { at, ...(r.pole ? { pole: r.pole } : {}) }] as const;+ });+ resolved = { ...held, reach: Object.fromEntries(entries) };+ }+ const body = bodyOf(build, layer.eyes, resolved);+ const placed = poseJoints(resolved);+ const joints: Record<string, readonly [number, number]> = {};+ for (const name of FIGURE_JOINTS) {+ const v = toView(placed[name]!);+ joints[name] = [Math.round(ax + v[0] * s), Math.round(ay - v[1] * s)];+ }++ const [W, H] = [w * SUPERSAMPLE, h * SUPERSAMPLE];+ const depth = new Float32Array(W * H).fill(-Infinity);+ const tagAt = new Int8Array(W * H).fill(-1);+ const nAt = new Float32Array(W * H * 3);+ const count = body.positions.length;+ const [sxs, sys, szs] = [new Float32Array(count), new Float32Array(count), new Float32Array(count)];+ for (let i = 0; i < count; i++) {+ const v = toView(body.positions[i]!);+ sxs[i] = (ax + v[0] * s) * SUPERSAMPLE; sys[i] = (ay - v[1] * s) * SUPERSAMPLE; szs[i] = v[2];+ }+ const viewNormals = new Float32Array(count * 3);+ for (let i = 0; i < count; i++) {+ const v = toView([body.normals[i * 3]!, body.normals[i * 3 + 1]!, body.normals[i * 3 + 2]!]);+ viewNormals[i * 3] = v[0]; viewNormals[i * 3 + 1] = v[1]; viewNormals[i * 3 + 2] = v[2];+ }+ for (let t = 0; t < body.tags.length; t++) {+ const [a, b, c] = [body.triangles[t * 3]!, body.triangles[t * 3 + 1]!, body.triangles[t * 3 + 2]!];+ const [x0, y0, x1, y1, x2, y2] = [sxs[a]!, sys[a]!, sxs[b]!, sys[b]!, sxs[c]!, sys[c]!];+ const area = (x1 - x0) * (y2 - y0) - (x2 - x0) * (y1 - y0);+ if (area === 0) continue;+ const [left, right] = [Math.max(0, Math.floor(Math.min(x0, x1, x2))), Math.min(W - 1, Math.ceil(Math.max(x0, x1, x2)))];+ const [top, bottom] = [Math.max(0, Math.floor(Math.min(y0, y1, y2))), Math.min(H - 1, Math.ceil(Math.max(y0, y1, y2)))];+ for (let y = top; y <= bottom; y++) {+ for (let x = left; x <= right; x++) {+ const [px, py] = [x + 0.5, y + 0.5];+ const w1 = ((px - x0) * (y2 - y0) - (x2 - x0) * (py - y0)) / area;+ const w2 = ((x1 - x0) * (py - y0) - (px - x0) * (y1 - y0)) / area;+ const w0 = 1 - w1 - w2;+ if (w0 < 0 || w1 < 0 || w2 < 0) continue;+ const z = w0 * szs[a]! + w1 * szs[b]! + w2 * szs[c]!;+ const i = y * W + x;+ if (z <= depth[i]!) continue;+ depth[i] = z;+ tagAt[i] = body.tags[t]!;+ for (let k = 0; k < 3; k++) nAt[i * 3 + k] = w0 * viewNormals[a * 3 + k]! + w1 * viewNormals[b * 3 + k]! + w2 * viewNormals[c * 3 + k]!;+ }+ }+ }+ const mask = new Float32Array(w * h);+ const normals = new Float32Array(w * h * 3);+ const contour = new Float32Array(w * h);+ const parts: Record<string, Float32Array> = Object.fromEntries(FIGURE_TAGS.map((tag) => [tag, new Float32Array(w * h)]));+ const share = 1 / (SUPERSAMPLE * SUPERSAMPLE);+ // How far one surface must stand in front of the next, in body units, to count as an edge of the form.+ const step = 0.012;+ for (let y = 0; y < H; y++) {+ for (let x = 0; x < W; x++) {+ const i = y * W + x, o = (y >> 1) * w + (x >> 1);+ const tag = tagAt[i]!;+ const right = x + 1 < W ? depth[i + 1]! : -Infinity, below = y + 1 < H ? depth[i + W]! : -Infinity;+ if (tag >= 0 || right > -Infinity || below > -Infinity) {+ const here = depth[i]!;+ if (Math.abs(here - right) > step || Math.abs(here - below) > step) contour[o] = Math.min(1, contour[o]! + share * 2);+ }+ if (tag < 0) continue;+ mask[o]! += share;+ parts[FIGURE_TAGS[tag]!]![o]! += share;+ for (let k = 0; k < 3; k++) normals[o * 3 + k]! += nAt[i * 3 + k]! * share;+ }+ }+ for (let o = 0; o < mask.length; o++) {+ const len = Math.hypot(normals[o * 3]!, normals[o * 3 + 1]!, normals[o * 3 + 2]!);+ if (len > 0) for (let k = 0; k < 3; k++) normals[o * 3 + k]! /= len;+ }+ return { mask, parts, normals, contour, joints, light: normalized(layer.light ?? [-0.5, 0.6, 0.6]) };+}++export const figureLight = normalized;diff --git a/pipeline/src/image/imageRaster.ts b/pipeline/src/image/imageRaster.tsindex 1c3de7daf..ddf2b2635 100644--- a/pipeline/src/image/imageRaster.ts+++ b/pipeline/src/image/imageRaster.ts@@ -7,6 +7,7 @@* effects, and this owns everything that has to be exact: antialiased edges, gradients, soft shadows.* On the CPU rather than through a canvas, so the same source makes the same bytes on any machine and a test can read them.*/+import { figureLight, flattenFigure, type FlatFigure } from './imageFigure';import { MAX_MARK_STROKES } from './imageSource';import type { ImageFill, ImageLayer, ImageSource, ImageAlong, ImageEffect, ImageField, ImageMark, ImageStroke, ImageSweep, ImageTransform } from './imageSource';@@ -459,7 +460,7 @@ function fractalNoise(x: number, y: number, octaves: number, seed: number): numbreturn Math.max(0, Math.min(1, 0.5 + (sum / weight) * NOISE_CONTRAST[octaves - 1]!));}-function paintOf(fill: ImageFill, m: Matrix): (x: number, y: number) => Rgba {+function paintOf(fill: ImageFill, m: Matrix, drawn?: Drawn, w = 0): (x: number, y: number) => Rgba {if (typeof fill === 'string') {const c = parseColor(fill);return () => c;@@ -477,6 +478,19 @@ function paintOf(fill: ImageFill, m: Matrix): (x: number, y: number) => Rgba {}return stops[stops.length - 1]![1];};+ if (fill.type === 'form') {+ const figure = drawn?.figures.get(fill.figure);+ if (!figure) {+ drawn?.notes.push(`a form names figure "${fill.figure}", which was not drawn before it`);+ return () => at(0);+ }+ const light = fill.light ? figureLight(fill.light) : figure.light;+ return (x, y) => {+ const i = Math.floor(y) * w + Math.floor(x);+ const [nx, ny, nz] = [figure.normals[i * 3] ?? 0, figure.normals[i * 3 + 1] ?? 0, figure.normals[i * 3 + 2] ?? 0];+ return at(0.5 + 0.5 * (nx * light[0] + ny * light[1] + nz * light[2]));+ };+ }if (fill.type === 'noise') {const [scale, octaves, seed] = [fill.scale ?? 32, fill.octaves ?? 4, fill.seed ?? 0];return (x, y) => {@@ -557,7 +571,7 @@ function effectLayers(effects: readonly ImageEffect[] | undefined, under: boolea}}-interface Drawn { masks: Map<string, Float32Array>; notes: string[]; bounds: Map<string, ImageBox>; painted: Map<string, { entries: number; strokes: number }> }+interface Drawn { masks: Map<string, Float32Array>; notes: string[]; bounds: Map<string, ImageBox>; painted: Map<string, { entries: number; strokes: number }>; figures: Map<string, FlatFigure>; figureJoints: Map<string, Record<string, readonly [number, number]>>; showGuides: boolean }/** x, y, width and height in the picture's pixels. */export type ImageBox = readonly [number, number, number, number];@@ -586,6 +600,11 @@ function noteBox(drawn: Drawn, id: string | undefined, mask: Float32Array, w: nufunction drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn): void {if (layer.visible === false) return;const { w, h } = c;+ // What a guide masks is still wanted when it is not seen, so it draws somewhere that is thrown away.+ if (!drawn.showGuides && (layer.guide ?? layer.shape === 'figure')) {+ drawLayer(new Canvas(w, h), { ...layer, guide: false }, parent, drawn);+ return;+ }if (layer.mirror) {const { mirror, ...rest } = layer;const axis = mirror === 'x' ? { x: -1, y: 1 } : { x: 1, y: -1 };@@ -648,6 +667,11 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):return;}+ if (layer.shape === 'figure') {+ drawFigure(c, layer, parent, drawn, opacity, blend, clip);+ return;+ }+const { runs, middle } = outlinesOf(layer);const m = times(parent, transformOf(layer.transform, middle));const placed = runs.map((run) => ({ closed: run.closed, points: run.points.map((p) => apply(m, p)) }));@@ -684,7 +708,7 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):}effectLayers(layer.effects, true, c, whole);const paintLine = (): void => {- const paint = paintOf(layer.stroke!.color, m);+ const paint = paintOf(layer.stroke!.color, m, drawn, w);for (let y = 0; y < h; y++) {for (let x = 0; x < w; x++) {const i = y * w + x;@@ -697,7 +721,7 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):};if (line && align === 'outside') paintLine();if (hasFill) {- const paint = paintOf(layer.fill!, m);+ const paint = paintOf(layer.fill!, m, drawn, w);for (let y = 0; y < h; y++) {for (let x = 0; x < w; x++) {const i = y * w + x;@@ -1000,7 +1024,8 @@ function fieldStrokes(s: ImageField, w: number, h: number, m: Matrix, drawn: Draconst len = Math.hypot(dx, dy) || 1;return [dx / len, dy / len];};- const paint = s.tone !== undefined ? paintOf(s.tone, IDENTITY) : null;+ const toned = s.tone !== undefined && typeof s.tone !== 'string' && s.tone.type === 'form' ? paintOf(s.tone, IDENTITY, drawn, w) : null;+ const paint = toned ? (x: number, y: number): Rgba => { const p = apply(m, [x, y]); return toned(p[0], p[1]); } : s.tone !== undefined ? paintOf(s.tone, IDENTITY) : null;const length = s.length ?? s.size * 3;const count = Math.min(MAX_MARK_STROKES, Math.ceil(((s.coverage ?? 4) * Math.max(1, (x1 - x0) * (y1 - y0))) / (s.size * length)));const spread = ((s.spread ?? 12) * Math.PI) / 180;@@ -1095,7 +1120,7 @@ function drawPaint(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, oconst inner = new Canvas(w, h);if (layer.from === 'below') inner.px.set(c.px);if (layer.fill !== undefined) {- const paint = paintOf(layer.fill, m);+ const paint = paintOf(layer.fill, m, drawn, w);for (let y = 0; y < h; y++) {for (let x = 0; x < w; x++) {const col = paint(x + 0.5, y + 0.5);@@ -1130,6 +1155,38 @@ function drawPaint(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, oeffectLayers(layer.effects, false, c, alpha);}+/** The body in clay, with the edges of its form and a mark on each joint: enough to draw over, and nothing to be mistaken for the drawing. */+function drawFigure(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {+ const { w, h } = c;+ const figure = flattenFigure(layer, w, h, (p) => apply(parent, p), drawnScale(parent));+ if (layer.id) {+ drawn.figures.set(layer.id, figure);+ drawn.masks.set(layer.id, figure.mask);+ noteBox(drawn, layer.id, figure.mask, w, h);+ for (const [tag, mask] of Object.entries(figure.parts)) {+ drawn.masks.set(`${layer.id}.${tag}`, mask);+ noteBox(drawn, `${layer.id}.${tag}`, mask, w, h);+ }+ }+ const [lx, ly, lz] = figure.light;+ for (let i = 0; i < figure.mask.length; i++) {+ const a = figure.mask[i]!;+ if (a <= 0) continue;+ const lit = 0.25 + 0.7 * Math.max(0, 0.5 + 0.5 * (figure.normals[i * 3]! * lx + figure.normals[i * 3 + 1]! * ly + figure.normals[i * 3 + 2]! * lz));+ const line = 1 - figure.contour[i]! * 0.85;+ put(c, i, lit * line, lit * line * 0.96, lit * line * 0.92, a * (layer.opacity ?? opacity) * (clip ? clip[i]! : 1), blend);+ }+ for (const [x, y] of Object.values(figure.joints)) {+ for (let dy = -2; dy <= 2; dy++) {+ for (let dx = -2; dx <= 2; dx++) {+ if (x + dx < 0 || y + dy < 0 || x + dx >= w || y + dy >= h || dx * dx + dy * dy > 5) continue;+ put(c, (y + dy) * w + x + dx, 0.95, 0.2, 0.25, 0.85, 'normal');+ }+ }+ }+ if (layer.id) drawn.figureJoints.set(layer.id, figure.joints);+}+function drawPixels(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {const { w, h } = c;const rows = layer.rows!;@@ -1167,17 +1224,19 @@ export interface RenderedImage extends RgbaBitmap {layerBounds: Record<string, ImageBox>;/** For each paint layer with an id: its entries, which `drop` counts, and the strokes they came to. */painted: Record<string, { entries: number; strokes: number }>;+ /** For each figure with an id: where its joints landed in the picture. */+ figures: Record<string, Record<string, readonly [number, number]>>;}/** Draws a source that has been validated. */-export function renderImage(source: ImageSource): RenderedImage {+export function renderImage(source: ImageSource, options: { guides?: boolean } = {}): RenderedImage {const [w, h] = typeof source.size === 'number' ? [source.size, source.size] : source.size;const canvas = new Canvas(w, h);if (source.background) {const [r, g, b, a] = parseColor(source.background);for (let i = 0; i < w * h; i++) put(canvas, i, r, g, b, a, 'normal');}- const drawn: Drawn = { masks: new Map(), notes: [], bounds: new Map(), painted: new Map() };+ const drawn: Drawn = { masks: new Map(), notes: [], bounds: new Map(), painted: new Map(), figures: new Map(), figureJoints: new Map(), showGuides: options.guides === true };for (const layer of source.layers) drawLayer(canvas, layer, IDENTITY, drawn);const rgba = new Uint8Array(w * h * 4);for (let i = 0; i < w * h; i++) {@@ -1196,6 +1255,7 @@ export function renderImage(source: ImageSource): RenderedImage {covers: Math.round((solid / (w * h)) * 1000) / 1000,layerBounds: Object.fromEntries(drawn.bounds),painted: Object.fromEntries(drawn.painted),+ figures: Object.fromEntries(drawn.figureJoints),};}diff --git a/pipeline/src/image/imageSource.ts b/pipeline/src/image/imageSource.tsindex a0ed8d941..bdf6e3f7d 100644--- a/pipeline/src/image/imageSource.ts+++ b/pipeline/src/image/imageSource.ts@@ -13,7 +13,9 @@ export type ImageFill = string| { type: 'linear'; from: readonly [number, number]; to: readonly [number, number]; stops: readonly (readonly [number, string])[] }| { type: 'radial'; at: readonly [number, number]; radius: number; stops: readonly (readonly [number, string])[] }/** The same picture on every machine: it comes from `seed` and never from a random number. */- | { type: 'noise'; scale?: number; octaves?: number; seed?: number; stops: readonly (readonly [number, string])[] };+ | { type: 'noise'; scale?: number; octaves?: number; seed?: number; stops: readonly (readonly [number, string])[] }+ /** By which way a figure's surface faces: 1 toward the light, 0.5 side on, 0 away. So light turns the form and does not sit on its edge. */+ | { type: 'form'; figure: string; light?: readonly number[]; stops: readonly (readonly [number, string])[] };export interface ImageEffect {type: 'shadow' | 'glow' | 'innerShadow' | 'innerGlow';@@ -33,7 +35,7 @@ export interface ImageTransform {origin?: readonly [number, number];}-export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'group';+export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'figure' | 'group';/** One movement of a brush. Replayed in order on a `paint` layer, so the same list is the same picture. */export interface ImageStroke {@@ -152,6 +154,24 @@ export interface ImageLayer {strokes?: readonly ImageMark[];/** "below": a paint layer's canvas starts as the picture so far, so its strokes and smudges work across everything under it. */from?: 'below';+ /** Seen when looking and left out of the .png, while what it masks still works. A figure is one unless told otherwise. */+ guide?: boolean;+ /** A figure: pixels from floor to crown, standing straight. */+ height?: number;+ /** A figure: degrees turned about the body, 0 facing the viewer, 90 facing picture right, 180 away. */+ yaw?: number;+ /** A figure: degrees seen from above. */+ pitch?: number;+ build?: { head?: number; shoulders?: number; chest?: number; waist?: number; hips?: number; limbs?: number };+ pose?: {+ rotate?: Readonly<Record<string, readonly number[]>>;+ move?: Readonly<Record<string, readonly number[]>>;+ /** A joint and where it must be: [x, y] in the picture, or [x, y, z] on the body's own axes. */+ reach?: Readonly<Record<string, { at: readonly number[]; pole?: readonly number[] }>>;+ };+ eyes?: 'open' | 'closed';+ /** A figure: where the light is, x right, y up, z toward the viewer. */+ light?: readonly number[];fill?: ImageFill;stroke?: { color: ImageFill; width: number; align?: 'center' | 'inside' | 'outside'; join?: 'round' | 'miter' | 'bevel'; cap?: 'round' | 'butt' | 'square' };opacity?: number;@@ -181,7 +201,11 @@ function checkFill(v: unknown): string | null {if (typeof v === 'string') return isColor(v) ? null : `${JSON.stringify(v)} must be #rgb, #rgba, #rrggbb or #rrggbbaa`;const f = v as Record<string, unknown> | null;if (typeof f !== 'object' || f === null) return 'must be a colour or a gradient';- if (f.type !== 'linear' && f.type !== 'radial' && f.type !== 'noise') return 'a gradient has type linear, radial or noise';+ if (f.type !== 'linear' && f.type !== 'radial' && f.type !== 'noise' && f.type !== 'form') return 'a gradient has type linear, radial, noise or form';+ if (f.type === 'form') {+ if (typeof f.figure !== 'string' || !f.figure) return 'a form needs figure, the id of a figure drawn before it';+ if (f.light !== undefined && !(Array.isArray(f.light) && f.light.length === 3 && f.light.every(isNum))) return 'a form\'s light must be [x, y, z], x right, y up, z toward the viewer';+ }if (f.type === 'noise') {if (f.scale !== undefined && !(isNum(f.scale) && (f.scale as number) > 0)) return 'noise scale must be a positive number of pixels';if (f.octaves !== undefined && !(Number.isInteger(f.octaves) && (f.octaves as number) >= 1 && (f.octaves as number) <= 8)) return 'noise octaves must be a whole number from 1 to 8';@@ -197,8 +221,8 @@ function checkFill(v: unknown): string | null {return null;}-const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes', 'from'];-const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'group'];+const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes', 'from', 'guide', 'height', 'yaw', 'pitch', 'build', 'pose', 'eyes', 'light'];+const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'figure', 'group'];const STROKE_FIELDS = ['points', 'color', 'to', 'size', 'hardness', 'opacity', 'taper', 'bristles', 'strand', 'spread', 'vary', 'wobble', 'seed', 'smudge', 'erase', 'blend'];const SWEEP_FIELDS = ['type', 'guides', 'colors', 'count', 'tip', 'size', 'clumps', 'gather', 'flyaway', 'vary', 'wobble', 'opacity', 'taper', 'seed'];@@ -311,6 +335,28 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)fail((e as Error).message);}break;+ case 'figure': {+ if (!isPair(l.at)) fail('at must be [x, y]: the place on the floor under her');+ if (!(isNum(l.height) && (l.height as number) > 0)) fail('height must be pixels from floor to crown');+ for (const key of ['yaw', 'pitch']) if (l[key] !== undefined && !isNum(l[key])) fail(`${key} must be degrees`);+ if (l.eyes !== undefined && l.eyes !== 'open' && l.eyes !== 'closed') fail('eyes must be open or closed');+ if (l.light !== undefined && !(Array.isArray(l.light) && l.light.length === 3 && l.light.every(isNum))) fail('light must be [x, y, z], x right, y up, z toward the viewer');+ const build = l.build as Record<string, unknown> | undefined;+ for (const [key, value] of Object.entries(build ?? {})) {+ if (!['head', 'shoulders', 'chest', 'waist', 'hips', 'limbs'].includes(key)) fail(`build has unknown "${key}"; it takes head, shoulders, chest, waist, hips, limbs`);+ if (!(isNum(value) && (value as number) > 0)) fail(`build ${key} must be positive`);+ }+ const pose = l.pose as Record<string, Record<string, unknown>> | undefined;+ for (const key of Object.keys(pose ?? {})) if (!['rotate', 'move', 'reach'].includes(key)) fail(`pose has unknown "${key}"; it takes rotate, move, reach`);+ for (const [key, list] of Object.entries(pose ?? {})) {+ for (const [joint, value] of Object.entries(list)) {+ const numbers = key === 'reach' ? (value as { at?: unknown }).at : value;+ const lengths = key === 'reach' ? [2, 3] : [3];+ if (!(Array.isArray(numbers) && lengths.includes(numbers.length) && numbers.every(isNum))) fail(`pose ${key} "${joint}" must be ${key === 'reach' ? '{ at: [x, y] or [x, y, z] }' : '[x, y, z]'}`);+ }+ }+ break;+ }case 'paint': {if (!Array.isArray(l.strokes)) fail('strokes must be an array, which may start empty');if (l.from !== undefined && l.from !== 'below') fail('from must be "below"');@@ -355,8 +401,9 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)if (s.join !== undefined && !['round', 'miter', 'bevel'].includes(s.join as string)) fail('stroke join must be round, miter or bevel');if (s.cap !== undefined && !['round', 'butt', 'square'].includes(s.cap as string)) fail('stroke cap must be round, butt or square');}- if (!['group', 'pixels', 'paint', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');+ if (!['group', 'pixels', 'paint', 'figure', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');if (l.from !== undefined && l.shape !== 'paint') fail('from is for a paint layer');+ if (l.guide !== undefined && typeof l.guide !== 'boolean') fail('guide must be true or false');if (l.blur !== undefined && !(isNum(l.blur) && (l.blur as number) >= 0)) fail('blur must be 0 or more, in pixels');if (l.blur && l.erase) fail('blur does not go with erase; put what cuts in a group with what it cuts, and blur the group');if (l.opacity !== undefined && !(isNum(l.opacity) && l.opacity >= 0 && l.opacity <= 1)) fail('opacity must be 0..1');diff --git a/pipeline/src/modeling/pose.ts b/pipeline/src/modeling/pose.tsindex b16de1b2c..cb77c7fb0 100644--- a/pipeline/src/modeling/pose.ts+++ b/pipeline/src/modeling/pose.ts@@ -8,8 +8,8 @@* rather than to a drawing.*/import { HUMANOID_TAGS, humanoidJoints, type HumanoidBuild } from './humanoid';-import { compact, type PolyMesh } from './polyMesh';-import { computeWornWeights, posedPositions, resolveJoints, type AnimationTrack, type RigJoint, type SkinWeights } from './rig';+import { compact, type PolyMesh, type Vec3 } from './polyMesh';+import { computeWornWeights, jointPositions, posedPositions, resolveJoints, type AnimationSource, type AnimationTrack, type RigJoint, type SkinWeights } from './rig';export interface PoseParams {/** The bones of a humanoid of this build; true for the default one. */@@ -24,7 +24,8 @@ export interface PoseParams {rigid?: string;}-export function poseStep(mesh: PolyMesh, p: PoseParams): PolyMesh {+/** A pose as the clip that holds it, on the bones it is for. */+function poseClip(p: PoseParams): { joints: RigJoint[]; animation: AnimationSource } {if (p.humanoid === undefined) throw new Error('`humanoid` says whose bones: true, or the build it was given');const joints: RigJoint[] = resolveJoints({ joints: humanoidJoints(p.humanoid === true ? {} : p.humanoid) });const names = new Set(joints.map((j) => j.name));@@ -47,12 +48,22 @@ export function poseStep(mesh: PolyMesh, p: PoseParams): PolyMesh {if (r.at?.length !== 3) throw new Error(`reach of "${name}" needs at: [x, y, z]`);tracks.set(name, { ...tracks.get(name), reach: [[0, r.at[0]!, r.at[1]!, r.at[2]!]], ...(r.pole ? { pole: r.pole } : {}) });}- const animation = { name: 'pose', duration: 1, loop: false, tracks: [...tracks].map(([joint, t]) => ({ joint, ...t })) as AnimationTrack[] };+ return { joints, animation: { name: 'pose', duration: 1, loop: false, tracks: [...tracks].map(([joint, t]) => ({ joint, ...t })) as AnimationTrack[] } };+}++/** Where each joint is in a pose, without the seconds a surface takes. */+export function poseJoints(p: PoseParams): Record<string, Vec3> {+ const { joints, animation } = poseClip(p);+ return jointPositions(joints, animation, 0);+}++export function poseStep(mesh: PolyMesh, p: PoseParams): PolyMesh {+ const { joints, animation } = poseClip(p);if (mesh.positions.length === 0) throw new Error('there is nothing yet to pose');let skin: SkinWeights;if (p.rigid !== undefined) {- known(p.rigid);const j = joints.findIndex((q) => q.name === p.rigid);+ if (j < 0) throw new Error(`no joint "${p.rigid}"; there are ${joints.map((q) => q.name).join(', ')}`);const count = mesh.positions.length;const jointsOut = new Uint16Array(count * 4), weights = new Float32Array(count * 4);for (let v = 0; v < count; v++) { jointsOut[v * 4] = j; weights[v * 4] = 1; }diff --git a/pipeline/tests/image-raster.test.ts b/pipeline/tests/image-raster.test.tsindex ef6fef056..f333e3c51 100644--- a/pipeline/tests/image-raster.test.ts+++ b/pipeline/tests/image-raster.test.ts@@ -608,7 +608,7 @@ describe('noise', () => {expect(() => bad({ scale: 0 })).toThrow(/scale must be a positive/);expect(() => bad({ octaves: 9 })).toThrow(/octaves must be/);expect(() => bad({ seed: 1.5 })).toThrow(/seed must be/);- expect(() => validateImageSource({ size: 8, layers: [{ shape: 'rect', at: [0, 0], size: [8, 8], fill: { type: 'wavy', stops: [] } }] })).toThrow(/linear, radial or noise/);+ expect(() => validateImageSource({ size: 8, layers: [{ shape: 'rect', at: [0, 0], size: [8, 8], fill: { type: 'wavy', stops: [] } }] })).toThrow(/linear, radial, noise or form/);});});@@ -966,6 +966,85 @@ describe('working across the picture', () => {});});+describe('a figure to draw over', () => {+ const LONG = 120000;+ const person = (fields: object = {}): ImageLayer => ({ id: 'f', shape: 'figure', at: [128, 240], height: 220, ...fields } as never);+ const look = (layers: ImageLayer[]): ReturnType<typeof renderImage> => renderImage(validateImageSource({ size: 256, layers }), { guides: true });+ const plain = (layers: ImageLayer[]): ReturnType<typeof renderImage> => renderImage(validateImageSource({ size: 256, layers }));++ it('is as tall as it is told, and stands on its place', () => {+ const box = plain([person()]).layerBounds.f!;+ expect(box[1] + box[3]).toBeGreaterThan(232);+ expect(box[1] + box[3]).toBeLessThanOrEqual(241);+ expect(box[3]).toBeGreaterThan(210);+ expect(box[3]).toBeLessThan(226);+ }, LONG);++ it('is seen when looking and is not in the picture', () => {+ const seen = look([person()]);+ expect(seen.rgba[(100 * 256 + 128) * 4 + 3]).toBeGreaterThan(200);+ const out = plain([person()]);+ for (let i = 3; i < out.rgba.length; i += 4) expect(out.rgba[i]).toBe(0);+ }, LONG);++ it('turns by yaw: toes to the picture right at 90, and left at -90', () => {+ const toe = (yaw: number): number => plain([person({ yaw })]).figures.f!['toeTip.R']![0];+ const ankle = (yaw: number): number => plain([person({ yaw })]).figures.f!['foot.R']![0];+ expect(toe(90)).toBeGreaterThan(ankle(90) + 8);+ expect(toe(-90)).toBeLessThan(ankle(-90) - 8);+ const front = plain([person()]).figures.f!;+ expect(front['hand.L']![0]).toBeGreaterThan(front['hand.R']![0]);+ expect(plain([person({ yaw: 90 })]).layerBounds.f![2]).toBeLessThan(plain([person()]).layerBounds.f![2]);+ }, LONG);++ it('reports joints that land on the body', () => {+ const out = plain([person()]);+ const m = out.figures.f!;+ expect(m.crown![1]).toBeLessThan(m.neck![1]!);+ expect(m.neck![1]).toBeLessThan(m.hips![1]!);+ expect(m['foot.L']![1]).toBeGreaterThan(m['shin.L']![1]!);+ }, LONG);++ it('puts a hand where it is asked in the picture, and the arm follows', () => {+ const m = plain([person({ pose: { reach: { 'hand.R': { at: [60, 60] } } } })]).figures.f!;+ expect(Math.hypot(m['hand.R']![0] - 60, m['hand.R']![1] - 60)).toBeLessThan(40);+ const near = plain([person({ pose: { reach: { 'hand.R': { at: [90, 110] } } } })]).figures.f!;+ expect(Math.hypot(near['hand.R']![0] - 90, near['hand.R']![1] - 110)).toBeLessThan(6);+ }, LONG);++ it('bows its head with a rotation about x', () => {+ const crownX = (x: number): number => plain([person({ yaw: 90, pose: { rotate: { neck: [x, 0, 0] } } })]).figures.f!.crown![0];+ expect(crownX(30)).toBeGreaterThan(crownX(0) + 5);+ }, LONG);++ it('makes each part a mask, while the guide itself stays out', () => {+ const painted = plain([person(), { shape: 'rect', at: [0, 0], size: [256, 256], fill: RED, clip: 'f.torso' }]);+ const box = painted.layerBounds['f.torso']!;+ expect(px(painted, box[0] + Math.floor(box[2] / 2), box[1] + Math.floor(box[3] / 2))[0]).toBe(255);+ expect(px(painted, 128, painted.figures.f!.crown![1] + 4)[3]).toBe(0);+ const field = plain([person(), { shape: 'paint', strokes: [{ type: 'field', region: '@f.leg', tone: BLUE, size: 3, length: 6, angle: 90, seed: 1, hardness: 1, opacity: 1, vary: 0 }] } as never]);+ const leg = field.layerBounds['f.leg']!;+ expect(px(field, 5, 5)[3]).toBe(0);+ expect(leg[3]).toBeGreaterThan(60);+ }, LONG);++ it('a form follows the surface: lighter toward the light, across one limb', () => {+ const lit = plain([person(), { shape: 'rect', at: [0, 0], size: [256, 256], clip: 'f', fill: { type: 'form', figure: 'f', light: [-1, 0, 0.2], stops: [[0, '#000000'], [1, '#ffffff']] } } as never]);+ const chest = lit.figures.f!.chest!;+ const [left, right] = [px(lit, chest[0] - 12, chest[1])[0]!, px(lit, chest[0] + 12, chest[1])[0]!];+ expect(left).toBeGreaterThan(right + 60);+ }, LONG);++ it('says what is wrong with one', () => {+ const bad = (l: object): unknown => validateImageSource({ size: 64, layers: [{ shape: 'figure', at: [4, 4], height: 40, ...l }] });+ expect(() => bad({ height: undefined })).toThrow(/height must be pixels/);+ expect(() => bad({ build: { nose: 2 } })).toThrow(/build has unknown "nose"/);+ expect(() => bad({ pose: { twist: {} } })).toThrow(/pose has unknown "twist"/);+ expect(() => bad({ pose: { rotate: { neck: [1] } } })).toThrow(/rotate "neck" must be/);+ expect(() => renderImage(validateImageSource({ size: 64, layers: [{ shape: 'figure', at: [4, 4], height: 40, pose: { rotate: { tail: [1, 0, 0] } } }] }))).toThrow(/no joint "tail"/);+ }, LONG);+});+describe('what it measures', () => {it('finds where the picture is solid and how much of it that is', () => {const img = draw([{ shape: 'rect', at: [10, 20], size: [16, 8], fill: RED }]);diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex 737eb6953..42e5e2199 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -3888,6 +3888,10 @@ export const NOTED = {{"note":"An AI can draw 2D pictures."},"feat(look): layers moved and wrapped, one pixel changed, a set and a close look drawn":{"note":"An AI can draw 2D pictures."},+ "feat(look): a posed body to draw over, its joints in the picture and light by its form":+ {"note":"An AI can draw 2D pictures."},+ "feat(mcp): image_write takes a figure as a guide":+ {"note":"An AI can draw 2D pictures."},"feat(look): painting across the picture, strokes along a line, values and colours read":{"note":"An AI can draw 2D pictures."},"feat(mcp): image_read shows values and picks colours; the painting text in full":