Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(look): text and strokes on a face, a text's core and wear, bloom

yhh4e9b148ddf02
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 93188739e..95cb01dfd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,7 +22,7 @@ published separately; it ships inside the editor.
- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.
- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.
-- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do; one `sweep` instruction lays a mass of hair gathered into locks, one `field` covers a region in strokes along a flow, with named brushes from ink and pencil to oil and watercolor, and one `along` repeats strokes along a line. A layer can work across everything under it, to fuse separate parts into one painting, and `image_read` shows lightness alone and reads colours. Text in any script, windows in rows on a face, buildings in true perspective and reflections on wet ground are one instruction each. A `figure` layer is a posed skeleton in plain blocks from any angle to draw over, and an `image` layer brings in the team's own art: both stay out of the `.png`, and the figure reports where each joint landed and lets broad light follow the form. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.
+- **An AI can draw 2D pictures.** `image_write` turns layers into a `.png` beside an `.esimage`: shapes and SVG paths, gradients, strokes, shadows and glows, clipping and cutting, or pixel art as rows of characters. It answers with the picture on a mid ground and a light check and at 96, 64, 48 and 32 pixels, to look at and correct; `image_edit` changes one layer, a row or a block of pixels without restating the rest, and moves layers or wraps them in a group. Strokes can keep sharp corners, and the reply measures where each layer ended up. A layer or group can be blurred, for depth of field, and a fill can be noise for cloud, texture and film grain, the same picture from the same seed. A `paint` layer is a canvas of brush strokes with pressure, softness, tapering, bristles and smudging, added to round by round, for what shapes cannot do; one `sweep` instruction lays a mass of hair gathered into locks, one `field` covers a region in strokes along a flow, with named brushes from ink and pencil to oil and watercolor, and one `along` repeats strokes along a line. A layer can work across everything under it, to fuse separate parts into one painting, and `image_read` shows lightness alone and reads colours. Text in any script, windows in rows on a face, buildings in true perspective, reflections on wet ground and bloom over what is bright are one instruction each, and words and strokes can be laid on a building's face in its own coordinates. A `figure` layer is a posed skeleton in plain blocks from any angle to draw over, and an `image` layer brings in the team's own art: both stay out of the `.png`, and the figure reports where each joint landed and lets broad light follow the form. `image_read` enlarges a part, and `image_set` shows several together with the stroke widths and size of each. Drawn on the CPU, so a source makes the same picture on any machine.
- **An AI can build a person, and not a picture of one.** The `humanoid` step makes a whole body from a few numbers, with fingers, a face and ears, seen right from every side, and `rig: { humanoid: true }` gives it bones that fit. `garment` makes clothes from the body's own surface: loose or tight, a bell sleeve that hangs, a clean hem, a thickness; what a body wears now takes its weights from the skin under it, so an arm stays in its sleeve. `pose` holds a body in a pose by angles, or with `reach` by where a hand should be. `organic` gains `limb`, cross-sections along a path, and `fine`, regions split finer on one skin. `displace` gains `folds`, running the way cloth hangs. `scene_views` no longer takes on a shot's lens blur and grade.
diff --git a/desktop b/desktop
index e550ab4e8..6d80bbc69 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit e550ab4e88f36aaa1e19cbbfec81f1643d8ea12a
+Subproject commit 6d80bbc69aadfae39847e3cbdfbc8257c4df7178
diff --git a/pipeline/src/image/imageRaster.ts b/pipeline/src/image/imageRaster.ts
index 69fb4fc94..0e1777248 100644
--- a/pipeline/src/image/imageRaster.ts
+++ b/pipeline/src/image/imageRaster.ts
@@ -477,6 +477,33 @@ function quadToUnit(q: readonly (readonly number[])[]): ((x: number, y: number)
};
}
+/** The other way: from 0..1 each way on four corners to the place in the picture. */
+function unitToQuad(q: readonly (readonly number[])[]): (u: number, v: number) => Pt {
+ const [[x0, y0], [x1, y1], [x2, y2], [x3, y3]] = q as unknown as [[number, number], [number, number], [number, number], [number, number]];
+ const [dx1, dx2, sx, dy1, dy2, sy] = [x1 - x2, x3 - x2, x0 - x1 + x2 - x3, y1 - y2, y3 - y2, y0 - y1 + y2 - y3];
+ const det = dx1 * dy2 - dx2 * dy1 || 1e-9;
+ const g = (sx * dy2 - dx2 * sy) / det, h = (dx1 * sy - sx * dy1) / det;
+ const [a, b, c, d, e, f] = [x1 - x0 + g * x1, x3 - x0 + h * x3, x0, y1 - y0 + g * y1, y3 - y0 + h * y3, y0];
+ return (u, v) => {
+ const z = g * u + h * v + 1;
+ return [(a * u + b * v + c) / z, (d * u + e * v + f) / z];
+ };
+}
+
+/** A block's face, named as "id.a", and the part of it asked for. A name that is not there says so. */
+function faceCorners(layer: ImageLayer, drawn: Drawn): readonly (readonly number[])[] | null {
+ const [id, key] = (layer.face as string).replace(/^@/, '').split('.') as [string, string | undefined];
+ const quad = key ? drawn.blocks.get(id)?.[key] : undefined;
+ if (!quad) {
+ drawn.notes.push(`layer ${layer.id ?? layer.shape}: face "${layer.face}" names no face of a block drawn before it, so it drew nothing`);
+ return null;
+ }
+ if (!layer.within) return quad;
+ const to = unitToQuad(quad);
+ const [l, t, r, b] = layer.within as [number, number, number, number];
+ return [to(l, t), to(r, t), to(r, b), to(l, b)];
+}
+
/** Marks in rows and columns, each with its own colour and whether it is on, sampled within the pixel so distant rows blend and do not shimmer. */
function gridPaint(fill: Extract<ImageFill, { type: 'grid' }>, m: Matrix): (x: number, y: number) => Rgba {
const colors = fill.colors.map(parseColor);
@@ -790,6 +817,20 @@ function drawLayer(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn):
return;
}
+ if (layer.shape === 'bloom') {
+ drawBloom(c, layer, parent, opacity, clip);
+ return;
+ }
+
+ if (layer.face && layer.shape === 'rect') {
+ const corners = faceCorners(layer, drawn);
+ if (!corners) return;
+ const { face: _f, within: _w, at: _a, size: _s, radius: _r, ...rest } = layer;
+ const fill = rest.fill && typeof rest.fill !== 'string' && rest.fill.type === 'grid' && !rest.fill.quad ? { ...rest.fill, quad: corners } : rest.fill;
+ drawLayer(c, { ...rest, ...(fill ? { fill } : {}), shape: 'polygon', points: corners as [number, number][] }, parent, drawn);
+ return;
+ }
+
const { runs, middle } = outlinesOf(layer);
const m = times(parent, transformOf(layer.transform, middle));
const placed = runs.map((run) => ({ closed: run.closed, points: run.points.map((p) => apply(m, p)) }));
@@ -1311,7 +1352,18 @@ function drawPaint(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, o
}
const buffer: StrokeBuffer = { mask: new Float32Array(w * h), rgb: new Float32Array(w * h * 3), touched: [] };
let drawnStrokes = 0;
- (layer.strokes ?? []).forEach((mark) => {
+ const onFace = layer.face ? faceCorners(layer, drawn) : null;
+ if (layer.face && !onFace) return;
+ const place = onFace ? unitToQuad(onFace) : null;
+ const lay = (points: readonly (readonly number[])[]): number[][] => points.map((p) => [...place!(p[0]!, p[1]!), ...p.slice(2)]);
+ const placed = (mark: ImageMark): ImageMark => {
+ if (!place) return mark;
+ if (!('type' in mark)) return { ...mark, points: lay(mark.points) };
+ if (mark.type === 'sweep') return { ...mark, guides: mark.guides.map(lay) };
+ if (mark.type === 'along') return { ...mark, line: lay(mark.line) };
+ return mark;
+ };
+ (layer.strokes ?? []).map(placed).forEach((mark) => {
for (const s of expandMark(mark, w, h, m, drawn, `layer ${layer.id ?? 'paint'}`)) paintStroke(inner, s, m, drawnScale(m), drawnStrokes++, buffer);
});
if (layer.id) drawn.painted.set(layer.id, { entries: (layer.strokes ?? []).length, strokes: drawnStrokes });
@@ -1383,11 +1435,130 @@ function drawFigure(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn,
if (layer.id) drawn.figureJoints.set(layer.id, figure.joints);
}
+/** Pixels a text is drawn at before it is laid on its corners: a font hinted small would not survive being stretched. */
+const QUAD_TEXT_SIZE = 96;
+
+/** The words drawn large and flat, cut to where they are, and looked up from each pixel of the corners they are laid on. */
+function textOnQuad(layer: ImageLayer, m: Matrix, w: number, h: number, draw: ImageTextDrawer): { fill: Float32Array; line: Float32Array | null } {
+ const count = [...(layer.text as string)].length;
+ const k = QUAD_TEXT_SIZE / (layer.fontSize ?? QUAD_TEXT_SIZE);
+ const long = Math.ceil(QUAD_TEXT_SIZE * 1.4 * (count + 1) + Math.abs(layer.tracking ?? 0) * k * count);
+ const across = QUAD_TEXT_SIZE * 2;
+ const [W, H] = layer.vertical ? [across, long] : [long, across];
+ const flat = draw({
+ ...layer, fontSize: QUAD_TEXT_SIZE, align: 'left', at: layer.vertical ? [across / 2, QUAD_TEXT_SIZE * 0.2] : [QUAD_TEXT_SIZE * 0.2, QUAD_TEXT_SIZE * 1.3],
+ ...(layer.tracking ? { tracking: layer.tracking * k } : {}), ...(layer.stroke ? { stroke: { ...layer.stroke, width: layer.stroke.width * k } } : {}),
+ }, IDENTITY, W, H);
+ const box = boxOf(W, H, (i) => flat.fill[i]! >= 0.5 || (flat.line ? flat.line[i]! >= 0.5 : false));
+ const out = { fill: new Float32Array(w * h), line: flat.line ? new Float32Array(w * h) : null };
+ const corners = layer.quad!.map((p) => apply(m, p as unknown as Pt));
+ const toUnit = quadToUnit(corners);
+ if (!box || !toUnit) return out;
+ const [x0, x1] = [Math.max(0, Math.floor(Math.min(...corners.map((p) => p[0])))), Math.min(w - 1, Math.ceil(Math.max(...corners.map((p) => p[0]))))];
+ const [y0, y1] = [Math.max(0, Math.floor(Math.min(...corners.map((p) => p[1])))), Math.min(h - 1, Math.ceil(Math.max(...corners.map((p) => p[1]))))];
+ const SAMPLES = 3;
+ for (let y = y0; y <= y1; y++) {
+ for (let x = x0; x <= x1; x++) {
+ let [f, l] = [0, 0];
+ for (let sy = 0; sy < SAMPLES; sy++) {
+ for (let sx = 0; sx < SAMPLES; sx++) {
+ const [u, v] = toUnit(x + (sx + 0.5) / SAMPLES, y + (sy + 0.5) / SAMPLES);
+ if (u < 0 || v < 0 || u >= 1 || v >= 1) continue;
+ const at = Math.floor(box[1] + v * box[3]) * W + Math.floor(box[0] + u * box[2]);
+ f += flat.fill[at]!;
+ if (flat.line) l += flat.line[at]!;
+ }
+ }
+ out.fill[y * w + x] = f / (SAMPLES * SAMPLES);
+ if (out.line) out.line[y * w + x] = l / (SAMPLES * SAMPLES);
+ }
+ }
+ return out;
+}
+
+/** What is bright in the picture so far, spilling over what is round it: a tight glow and a wide one, as a lens and the air both give. */
+function drawBloom(c: Canvas, layer: ImageLayer, parent: Matrix, opacity: number, clip: Float32Array | undefined): void {
+ const { w, h } = c;
+ const threshold = layer.threshold ?? 0.6;
+ const strength = (layer.strength ?? 1) * opacity;
+ const radius = ((layer.radius as number | undefined) ?? 10) * drawnScale(parent);
+ const bright = [new Float32Array(w * h), new Float32Array(w * h), new Float32Array(w * h)] as const;
+ const span = Math.max(0.001, 1 - threshold);
+ for (let i = 0; i < w * h; i++) {
+ const [r, g, b] = [c.px[i * 4]!, c.px[i * 4 + 1]!, c.px[i * 4 + 2]!];
+ const peak = Math.max(r, g, b);
+ if (peak <= threshold) continue;
+ // Its own colour, in proportion to how far over it is: only the brightest of a soft edge shines.
+ const over = smoothed(Math.min(1, (peak - threshold) / span));
+ bright[0][i] = r * over; bright[1][i] = g * over; bright[2][i] = b * over;
+ }
+ for (let k = 0; k < 3; k++) {
+ const [tight, wide] = [blurMask(bright[k]!, w, h, radius * 0.5), blurMask(bright[k]!, w, h, radius * 2.5)];
+ for (let i = 0; i < w * h; i++) {
+ const shine = Math.min(1, (tight[i]! * 0.7 + wide[i]! * 0.9) * strength) * (clip ? clip[i]! : 1);
+ if (shine <= 0) continue;
+ const at = i * 4;
+ c.px[at + k] = c.px[at + k]! + (1 - c.px[at + k]!) * shine;
+ if (shine > c.px[at + 3]!) c.px[at + 3] = shine;
+ }
+ }
+}
+
+/** Coverage with this many pixels taken off every edge: the lowest within reach, across and then down. */
+function thinned(mask: Float32Array, pixels: number, w: number, h: number): Float32Array {
+ const r = Math.round(pixels);
+ if (r <= 0) return mask;
+ const across = new Float32Array(w * h);
+ for (let y = 0; y < h; y++) {
+ for (let x = 0; x < w; x++) {
+ let low = 1;
+ for (let d = -r; d <= r; d++) {
+ const at = x + d;
+ const v = at < 0 || at >= w ? 0 : mask[y * w + at]!;
+ if (v < low) low = v;
+ }
+ across[y * w + x] = low;
+ }
+ }
+ const out = new Float32Array(w * h);
+ for (let x = 0; x < w; x++) {
+ for (let y = 0; y < h; y++) {
+ let low = 1;
+ for (let d = -r; d <= r; d++) {
+ const at = y + d;
+ const v = at < 0 || at >= h ? 0 : across[at * w + x]!;
+ if (v < low) low = v;
+ }
+ out[y * w + x] = low;
+ }
+ }
+ return out;
+}
+
+/** Letters worn as a brush wears a stroke, by the same paper and in streaks across them. */
+function wornText(mask: Float32Array, layer: ImageLayer, w: number): Float32Array {
+ const [grain, dry] = [layer.grain ?? 0, layer.dry ?? 0];
+ if (grain <= 0 && dry <= 0) return mask;
+ const scale = layer.grainScale ?? 2;
+ return mask.map((v, i) => {
+ if (v <= 0) return 0;
+ const [x, y] = [(i % w) + 0.5, Math.floor(i / w) + 0.5];
+ let kept = v;
+ if (grain > 0) kept *= Math.max(0, Math.min(1, (fractalNoise(x / scale, y / scale, 3, 77) - (0.12 + grain * 0.62)) / 0.34 + 0.5));
+ if (dry > 0) kept *= Math.max(0, Math.min(1, (0.5 + 1.3 * gradientNoise(x / 40, y / 1.8, 9) - (0.12 + dry * 0.68)) / 0.22 + 0.5));
+ return kept;
+ });
+}
+
function drawText(c: Canvas, layer: ImageLayer, parent: Matrix, drawn: Drawn, opacity: number, blend: Blend, clip: Float32Array | undefined): void {
const { w, h } = c;
if (!drawn.text) throw new Error(`layer ${layer.id ?? 'text'}: text needs the editor's fonts, and this render has none`);
- const m = times(parent, transformOf(layer.transform, layer.at as Pt));
- const { fill, line } = drawn.text(layer, m, w, h);
+ const m = times(parent, transformOf(layer.transform, (layer.at ?? [0, 0]) as Pt));
+ const corners = layer.face ? faceCorners(layer, drawn) : layer.quad;
+ if (layer.face && !corners) return;
+ const drawnText = corners ? textOnQuad({ ...layer, quad: corners }, m, w, h, drawn.text) : drawn.text(layer, m, w, h);
+ const line = drawnText.line;
+ const fill = wornText(thinned(drawnText.fill, (layer.thin ?? 0) * drawnScale(m), w, h), layer, w);
const whole = new Float32Array(w * h);
for (let i = 0; i < whole.length; i++) whole[i] = Math.max(layer.fill !== undefined ? fill[i]! : 0, line ? line[i]! : 0) * (clip ? clip[i]! : 1);
if (layer.id) drawn.masks.set(layer.id, whole);
diff --git a/pipeline/src/image/imageSource.ts b/pipeline/src/image/imageSource.ts
index d49f9f342..9ea21c395 100644
--- a/pipeline/src/image/imageSource.ts
+++ b/pipeline/src/image/imageSource.ts
@@ -55,7 +55,7 @@ export interface ImageTransform {
origin?: readonly [number, number];
}
-export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'figure' | 'image' | 'text' | 'block' | 'perspective' | 'reflect' | 'group';
+export type ImageShape = 'rect' | 'circle' | 'ellipse' | 'ring' | 'polygon' | 'line' | 'star' | 'gear' | 'path' | 'pixels' | 'paint' | 'figure' | 'image' | 'text' | 'block' | 'perspective' | 'reflect' | 'bloom' | 'group';
/** How a brush lays colour. Any of these on a stroke, a sweep or a field, over what its `brush` starts from. */
export interface ImageBrushSettings {
@@ -252,6 +252,21 @@ export interface ImageLayer {
vertical?: boolean;
/** Extra between characters, in pixels. */
tracking?: number;
+ /** Text: pixels taken off each stroke of the letters, for the white-hot core of a neon tube. */
+ thin?: number;
+ /** Text: 0..1 how much the surface shows through, as worn paint, and how much it is rubbed away in streaks. */
+ grain?: number;
+ grainScale?: number;
+ dry?: number;
+ /** Text or paint: a face of a block already drawn, "id.a", to lie on: text within `within`, and strokes with their points counted 0..1 across it and 0..1 down. */
+ face?: string;
+ /** On a face, the part it takes: left, top, right, bottom, each 0..1. */
+ within?: readonly number[];
+ /** Text: four corners in the picture it is laid on, in order round from the top left, so it lies on a wall in perspective. */
+ quad?: readonly (readonly number[])[];
+ /** A bloom: how bright a pixel must be to shine, 0..1, and how much it does. */
+ threshold?: number;
+ strength?: number;
/** A block or a perspective: the points lines recede to. */
vanish?: readonly (readonly number[])[];
/** A block: how far each face runs toward its vanishing point, 0..1. */
@@ -334,11 +349,11 @@ function checkFill(v: unknown): string | null {
return null;
}
-const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes', 'from', 'guide', 'height', 'yaw', 'pitch', 'build', 'pose', 'light', 'src', 'except', 'text', 'fontSize', 'font', 'weight', 'align', 'vertical', 'tracking', 'vanish', 'depth', 'faces', 'roof', 'y', 'edge', 'reach', 'ripple', 'smear'];
+const FIELDS = ['id', 'shape', 'at', 'size', 'radius', 'radii', 'inner', 'count', 'tooth', 'turn', 'points', 'd', 'rows', 'palette', 'scale', 'layers', 'fill', 'stroke', 'opacity', 'blend', 'transform', 'mirror', 'effects', 'clip', 'erase', 'visible', 'blur', 'strokes', 'from', 'guide', 'height', 'yaw', 'pitch', 'build', 'pose', 'light', 'src', 'except', 'text', 'fontSize', 'font', 'weight', 'align', 'vertical', 'tracking', 'vanish', 'depth', 'faces', 'roof', 'y', 'edge', 'reach', 'ripple', 'smear', 'quad', 'threshold', 'strength', 'thin', 'grain', 'grainScale', 'dry', 'face', 'within'];
/** Layers that are aids to drawing unless told otherwise: seen when looking, and not in the .png. */
export const isGuideLayer = (l: ImageLayer): boolean => l.guide ?? (l.shape === 'figure' || l.shape === 'image' || l.shape === 'perspective');
-const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'group'];
+const SHAPES: readonly ImageShape[] = ['rect', 'circle', 'ellipse', 'ring', 'polygon', 'line', 'star', 'gear', 'path', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'bloom', 'group'];
const STROKE_FIELDS = ['points', 'color', 'to', 'size', 'hardness', 'opacity', 'taper', 'bristles', 'strand', 'spread', 'vary', 'wobble', 'seed', 'smudge', 'erase', 'blend', ...BRUSH_FIELDS];
/** What a brush's own settings must be, on whatever carries them. */
@@ -455,7 +470,7 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
const pair = (k: string): void => { if (!isPair(l[k])) fail(`${k} must be [a, b]`); };
const positive = (n: number): boolean => n > 0;
switch (l.shape) {
- case 'rect': pair('at'); pair('size'); if (l.radius !== undefined && !isNum(l.radius) && !(Array.isArray(l.radius) && l.radius.length === 4 && l.radius.every(isNum))) fail('radius must be a number or four corners'); break;
+ case 'rect': if (l.face === undefined) { pair('at'); pair('size'); } if (l.radius !== undefined && !isNum(l.radius) && !(Array.isArray(l.radius) && l.radius.length === 4 && l.radius.every(isNum))) fail('radius must be a number or four corners'); break;
case 'circle': pair('at'); num('radius', positive); break;
case 'ellipse': pair('at'); pair('radii'); break;
case 'ring': pair('at'); num('radius', positive); num('inner', positive); if ((l.inner as number) >= (l.radius as number)) fail('inner must be less than radius'); break;
@@ -496,13 +511,28 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
}
case 'text':
if (typeof l.text !== 'string' || !l.text) fail('text must say something');
- pair('at');
- if (!(isNum(l.fontSize) && (l.fontSize as number) > 0)) fail('fontSize must be positive, in pixels');
+ if (l.within !== undefined && !(Array.isArray(l.within) && l.within.length === 4 && l.within.every((v) => isNum(v) && v >= 0 && v <= 1))) fail('within must be [left, top, right, bottom], each 0..1');
+ if (l.face !== undefined) {
+ if (typeof l.face !== 'string') fail('face must name a face of a block, as "id.a"');
+ } else if (l.quad !== undefined) {
+ if (!(Array.isArray(l.quad) && l.quad.length === 4 && l.quad.every(isPair))) fail('quad must be four [x, y], in order round from the top left');
+ } else {
+ pair('at');
+ if (!(isNum(l.fontSize) && (l.fontSize as number) > 0)) fail('fontSize must be positive, in pixels');
+ }
if (l.font !== undefined && typeof l.font !== 'string') fail('font must be a family name or a font file in the project');
if (l.align !== undefined && !['left', 'center', 'right'].includes(l.align as string)) fail('align must be left, center or right');
if (l.tracking !== undefined && !isNum(l.tracking)) fail('tracking must be pixels');
+ if (l.thin !== undefined && !(isNum(l.thin) && (l.thin as number) >= 0)) fail('thin must be 0 or more pixels');
+ for (const key of ['grain', 'dry']) if (l[key] !== undefined && !(isNum(l[key]) && (l[key] as number) >= 0 && (l[key] as number) <= 1)) fail(`${key} must be 0..1`);
+ if (l.grainScale !== undefined && !(isNum(l.grainScale) && (l.grainScale as number) > 0)) fail('grainScale must be positive');
if (l.fill === undefined && l.stroke === undefined) fail('it needs a fill or a stroke, or it draws nothing');
break;
+ case 'bloom':
+ for (const key of ['threshold', 'strength']) if (l[key] !== undefined && !(isNum(l[key]) && (l[key] as number) >= 0)) fail(`${key} must be 0 or more`);
+ if (l.threshold !== undefined && (l.threshold as number) > 1) fail('threshold must be 0..1');
+ if (l.radius !== undefined && !(isNum(l.radius) && (l.radius as number) > 0)) fail('radius must be positive, in pixels');
+ break;
case 'perspective':
if (!Array.isArray(l.vanish) || l.vanish.length === 0 || !l.vanish.every(isPair)) fail('vanish must be one or more [x, y]');
break;
@@ -527,6 +557,7 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
break;
case 'paint': {
if (!Array.isArray(l.strokes)) fail('strokes must be an array, which may start empty');
+ if (l.face !== undefined && typeof l.face !== 'string') fail('face must name a face of a block, as "id.a"');
if (l.from !== undefined && l.from !== 'below') fail('from must be "below"');
if (l.from !== undefined && (l.fill !== undefined || l.effects !== undefined || l.blend !== undefined)) fail('from "below" works on the picture itself, and takes no fill, effects or blend');
(l.strokes as Record<string, unknown>[]).forEach((s, n) => {
@@ -569,7 +600,7 @@ function checkLayer(l: Record<string, unknown>, where: string, ids: Set<string>)
if (s.join !== undefined && !['round', 'miter', 'bevel'].includes(s.join as string)) fail('stroke join must be round, miter or bevel');
if (s.cap !== undefined && !['round', 'butt', 'square'].includes(s.cap as string)) fail('stroke cap must be round, butt or square');
}
- if (!['group', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');
+ if (!['group', 'pixels', 'paint', 'figure', 'image', 'text', 'block', 'perspective', 'reflect', 'bloom', 'line'].includes(l.shape as string) && l.fill === undefined && l.stroke === undefined && !l.erase) fail('it needs a fill or a stroke, or it draws nothing');
if (l.from !== undefined && l.shape !== 'paint') fail('from is for a paint layer');
if (l.guide !== undefined && typeof l.guide !== 'boolean') fail('guide must be true or false');
if (l.blur !== undefined && !(isNum(l.blur) && (l.blur as number) >= 0)) fail('blur must be 0 or more, in pixels');
diff --git a/pipeline/tests/image-raster.test.ts b/pipeline/tests/image-raster.test.ts
index abca59021..4a6a579aa 100644
--- a/pipeline/tests/image-raster.test.ts
+++ b/pipeline/tests/image-raster.test.ts
@@ -1117,6 +1117,137 @@ describe('a picture to draw over', () => {
});
});
+describe('words on a wall, and light spilling', () => {
+ // Fonts are not here: a solid bar stands in for the words, drawn where it is told.
+ const bar = (layer: ImageLayer, m: readonly number[], w: number, h: number): { fill: Float32Array; line: Float32Array | null } => {
+ const fill = new Float32Array(w * h);
+ const size = layer.fontSize as number;
+ const [x, y] = [Math.round(layer.at![0] + m[4]!), Math.round(layer.at![1] + m[5]!)];
+ for (let j = 0; j < size; j++) for (let i = 0; i < size * 3; i++) { const at = (y - j) * w + x + i; if (at >= 0 && at < fill.length) fill[at] = 1; }
+ return { fill, line: null };
+ };
+ const onWall = { id: 't', shape: 'text', text: 'SIGN', quad: [[8, 8], [56, 20], [56, 44], [8, 56]], fill: RED } as ImageLayer;
+
+ it('lays text on four corners: it fills them, and narrows to the far side', () => {
+ const img = renderImage(validateImageSource({ size: 64, layers: [onWall] }), { text: bar });
+ const tall = (x: number): number => { let n = 0; for (let y = 0; y < 64; y++) if (px(img as never, x, y)[3]! > 128) n++; return n; };
+ expect(tall(12)).toBeGreaterThan(tall(52) + 8);
+ expect(px(img as never, 32, 32)[0]).toBe(255);
+ expect(px(img as never, 4, 32)[3]).toBe(0);
+ expect(px(img as never, 60, 32)[3]).toBe(0);
+ });
+
+ it('needs no at or fontSize on corners, and still says what is wrong', () => {
+ const bad = (l: object): unknown => validateImageSource({ size: 8, layers: [{ shape: 'text', text: 'a', fill: RED, ...l }] });
+ expect(() => bad({ quad: [[0, 0], [1, 0], [1, 1], [0, 1]] })).not.toThrow();
+ expect(() => bad({ quad: [[0, 0]] })).toThrow(/four \[x, y\]/);
+ expect(() => bad({})).toThrow(/at must/);
+ });
+
+ const label = { id: 't', shape: 'text', text: 'A', at: [8, 40], fontSize: 16, fill: RED } as ImageLayer;
+ const count = (img: ReturnType<typeof renderImage>): number => { let n = 0; for (let i = 3; i < img.rgba.length; i += 4) if (img.rgba[i]! > 128) n++; return n; };
+ const render = (l: object): ReturnType<typeof renderImage> => renderImage(validateImageSource({ size: 64, layers: [{ ...label, ...l }] }), { text: bar });
+
+ it('thins its letters by the pixels asked, for a core inside a tube', () => {
+ const [whole, core] = [render({}), render({ thin: 2 })];
+ expect(count(whole)).toBe(48 * 16);
+ expect(count(core)).toBe(44 * 12);
+ expect(px(core as never, 8, 32)[3]).toBe(0);
+ expect(px(core as never, 20, 32)[3]).toBe(255);
+ });
+
+ it('wears its letters as told, the same every time', () => {
+ expect(count(render({ grain: 0.8 }))).toBeLessThan(count(render({})) * 0.7);
+ expect(count(render({ dry: 0.7 }))).toBeLessThan(count(render({})) * 0.9);
+ expect(render({ grain: 0.8 }).rgba).toEqual(render({ grain: 0.8 }).rgba);
+ expect(() => render({ grain: 2 })).toThrow(/grain must be 0..1/);
+ expect(() => render({ thin: -1 })).toThrow(/thin must be/);
+ });
+
+ it('stacks: glow under, tube, then a thin core, all from the same words', () => {
+ const stack = renderImage(validateImageSource({ size: 64, background: '#101018', layers: [
+ { ...label, id: 'halo', fill: '#ff2080', blend: 'add', blur: 6, opacity: 0.6 },
+ { ...label, id: 'tube', fill: '#ff2080' },
+ { ...label, id: 'core', fill: '#fff0f6', thin: 2 },
+ ] }), { text: bar });
+ expect(px(stack as never, 20, 32)[1]).toBeGreaterThan(200);
+ expect(px(stack as never, 9, 32)[1]).toBeLessThan(80);
+ expect(px(stack as never, 4, 32)[0]).toBeGreaterThan(0x10 + 10);
+ });
+
+ const dot: ImageLayer = { shape: 'rect', at: [28, 28], size: [8, 8], fill: '#ffffff' };
+ const ground: ImageLayer = { shape: 'rect', at: [0, 0], size: [64, 64], fill: '#202020' };
+
+ it('spills what is bright over what is round it, falling away', () => {
+ const [plain, lit] = [draw([ground, dot]), draw([ground, dot, { shape: 'bloom', radius: 6 } as never])];
+ expect(px(lit, 24, 32)[0]).toBeGreaterThan(px(plain, 24, 32)[0]! + 15);
+ expect(px(lit, 24, 32)[0]).toBeGreaterThan(px(lit, 8, 32)[0]!);
+ });
+
+ it('leaves what is below the threshold alone, and takes the colour of what shines', () => {
+ const dim = draw([ground, { shape: 'bloom', threshold: 0.6 } as never]);
+ expect(px(dim, 32, 32)[0]).toBe(0x20);
+ const red = draw([ground, { ...dot, fill: RED }, { shape: 'bloom', radius: 6 } as never]);
+ expect(px(red, 24, 32)[0]).toBeGreaterThan(px(red, 24, 32)[1]! + 15);
+ });
+
+ it('is stronger when told, and keeps to its clip', () => {
+ const at = (strength: number): number => px(draw([ground, dot, { shape: 'bloom', radius: 6, strength } as never]), 24, 32)[0]!;
+ expect(at(2)).toBeGreaterThan(at(0.5));
+ const clipped = draw([ground, { id: 'c', shape: 'rect', at: [0, 0], size: [20, 64], fill: '#00000001' }, dot, { shape: 'bloom', radius: 8, clip: 'c' } as never]);
+ expect(px(clipped, 24, 32)[0]).toBe(0x20);
+ });
+
+ it('says what is wrong with one', () => {
+ const bad = (l: object): unknown => validateImageSource({ size: 8, layers: [{ shape: 'bloom', ...l }] });
+ expect(() => bad({ threshold: 2 })).toThrow(/threshold must be 0..1/);
+ expect(() => bad({ radius: 0 })).toThrow(/radius must be positive/);
+ });
+});
+
+describe('working on a face', () => {
+ const bar = (layer: ImageLayer, m: readonly number[], w: number, h: number): { fill: Float32Array; line: Float32Array | null } => {
+ const fill = new Float32Array(w * h);
+ const size = layer.fontSize as number;
+ const [x, y] = [Math.round(layer.at![0] + m[4]!), Math.round(layer.at![1] + m[5]!)];
+ for (let j = 0; j < size; j++) for (let i = 0; i < size * 3; i++) { const at = (y - j) * w + x + i; if (at >= 0 && at < fill.length) fill[at] = 1; }
+ return { fill, line: null };
+ };
+ const wall = { id: 'b', shape: 'block', at: [10, 110], height: 100, vanish: [[200, 40]], depth: [0.5], faces: ['#202030'] } as ImageLayer;
+ const render = (l: object[]): ReturnType<typeof renderImage> => renderImage(validateImageSource({ size: 128, layers: [wall, ...l] as never }), { text: bar });
+
+ it('paints across a face in its own terms, the line toward the vanishing point by itself', () => {
+ const img = render([{ shape: 'paint', face: 'b.a', strokes: [{ points: [[0, 0.5], [1, 0.5]], color: RED, size: 3, hardness: 1 }] }]);
+ const q = img.blocks.b!.a!;
+ const mid = (a: readonly number[], c: readonly number[]): number[] => [(a[0]! + c[0]!) / 2, (a[1]! + c[1]!) / 2];
+ const [nearMid, farMid] = [mid(q[0]!, q[3]!), mid(q[1]!, q[2]!)];
+ expect(px(img as never, Math.round(nearMid[0]!) + 2, Math.round(nearMid[1]!))[0]).toBe(255);
+ expect(px(img as never, Math.round(farMid[0]!) - 2, Math.round(farMid[1]!))[0]).toBe(255);
+ });
+
+ it('lays words on part of a face, narrowing as it goes', () => {
+ const img = render([{ id: 't', shape: 'text', text: 'AD', face: 'b.a', within: [0.1, 0.1, 0.9, 0.5], fill: RED }]);
+ const box = img.layerBounds.t!;
+ const tall = (x: number): number => { let n = 0; for (let y = 0; y < 128; y++) if (px(img as never, x, y)[0]! > 128) n++; return n; };
+ expect(tall(box[0] + 2)).toBeGreaterThan(tall(box[0] + box[2] - 3));
+ expect(box[2]).toBeLessThan(img.layerBounds['b.a']![2]);
+ });
+
+ it('puts a rect on part of a face, cornered as the face is', () => {
+ const img = render([{ id: 'r', shape: 'rect', face: 'b.a', within: [0.2, 0.2, 0.8, 0.6], fill: RED }]);
+ const box = img.layerBounds.r!;
+ const tall = (x: number): number => { let n = 0; for (let y = 0; y < 128; y++) if (px(img as never, x, y)[0]! > 128) n++; return n; };
+ expect(tall(box[0] + 1)).toBeGreaterThan(tall(box[0] + box[2] - 2) + 3);
+ expect(() => validateImageSource({ size: 8, layers: [{ shape: 'rect', fill: RED }] })).toThrow(/at must/);
+ });
+
+ it('says when there is no such face', () => {
+ const img = render([{ shape: 'paint', face: 'b.z', strokes: [] }, { shape: 'text', text: 'A', face: 'nope.a', fill: RED }]);
+ expect(img.notes.join()).toMatch(/face "b.z" names no face/);
+ expect(img.notes.join()).toMatch(/face "nope.a" names no face/);
+ });
+});
+
describe('a built world', () => {
const windows = (fields: object): object => ({ type: 'grid', colors: [RED], ground: BLUE, ...fields });
const rect = (fill: object): ImageLayer => ({ shape: 'rect', at: [0, 0], size: [64, 64], fill } as never);
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index faea36be9..8edf2c3e3 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -3892,6 +3892,10 @@ export const NOTED = {
{"note":"An AI can draw 2D pictures."},
"feat(mcp): image_write draws text in the editor's fonts, and documents the built world":
{"note":"An AI can draw 2D pictures."},
+ "feat(look): text and strokes on a face, a text's core and wear, bloom":
+ {"note":"An AI can draw 2D pictures."},
+ "feat(mcp): image_write teaches the order of a picture, neon and paint on a wall":
+ {"note":"An AI can draw 2D pictures."},
"fix(look): what clips to a clipped layer stays inside what that layer holds":
{"internal":"the paint, figure and image layers it fixes have not been released"},
"fix(look): a hidden guide still works, opacity 0 hides its bones, a faint layer has a real box":