Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

feat(modeling): a stain step colours a surface by what is there, and a bake keeps it as a colour map

yhh48c93d25f8a9
diff --git a/CHANGELOG.md b/CHANGELOG.md
index da72a3d57..2bfa26632 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,6 +22,8 @@ published separately; it ships inside the editor.
- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.
- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.
+- **An AI building a model can colour it finely, and keep that on a thin mesh.** A `stain` step lays a colour over a surface by a pattern, in what is sunken or on what stands out, by which way it faces or around a spot: moss on top, dirt in the cracks, a pupil. `normalBake` with `color: true` writes that as a base colour map, in linear and gamma projects alike.
+
- **An AI building a model can carve fine detail into it.** A `displace` step pushes bumps, wrinkles, cracks, scales, pores, ribs, weave or scratches out of a surface along its normal. A pattern needs no UV and has no seams, can be limited to some faces or to around a point, stacks with others, and splits the surface as far as it needs. Followed by `decimate` and `normalBake`, half a million triangles of detail draw on 1,500.
- **An AI building a model can make it light and keep its detail.** A `decimate` step thins a model to a share or a number of triangles, keeping paint and tags on the side they were on, and rigs and shapes are found on what it leaves. With `from` it thins the form as it stood before the fine detail, so edges and features keep their triangles. `normalBake` writes the surface as it stood before that as a normal map on the thinned one, rigged or not: a 77,000-triangle rock at 800 triangles lights close to the original, and the map bakes in about a second.
diff --git a/desktop b/desktop
index 965e71635..6962ff1c4 160000
--- a/desktop
+++ b/desktop
@@ -1 +1 @@
-Subproject commit 965e7163507fc62dacb0310be9b1253e6d7c48e5
+Subproject commit 6962ff1c4b645d414f5780b4a2d46f0d36a3c094
diff --git a/pipeline/src/modeling/displace.ts b/pipeline/src/modeling/displace.ts
index a12c56b70..6d59c5687 100644
--- a/pipeline/src/modeling/displace.ts
+++ b/pipeline/src/modeling/displace.ts
@@ -190,7 +190,7 @@ function patternAt(p: DisplaceParams, x: number, y: number, z: number, n: Vec3):
}
/** The longest edge that still draws the pattern, as a share of one feature: a rise and fall needs several points, and a line needs some across its width. */
-function finestShare(p: DisplaceParams): number {
+export function finestShare(p: DisplaceParams): number {
const layers = 2 ** ((p.octaves ?? 3) - 1);
switch (p.pattern) {
case 'bumps': return 1 / (4 * layers);
@@ -209,7 +209,7 @@ function finestShare(p: DisplaceParams): number {
* like any other, and the new vertex joins that face's outline: it keeps its shape and no gap opens. Left whole, such an edge
* could never get short, and the triangles against it would go on splitting for as long as there was budget.
*/
-function refine(mesh: PolyMesh, chosen: ReadonlySet<number>, longest: number, budget: number): { mesh: PolyMesh; short: boolean } {
+export function refine(mesh: PolyMesh, chosen: ReadonlySet<number>, longest: number, budget: number): { mesh: PolyMesh; short: boolean } {
const positions: Vec3[] = [...mesh.positions];
// Room for every vertex the budget could make, in a number that stays exact.
const keyOf = (a: number, b: number): number => (a < b ? a * 67108864 + b : b * 67108864 + a);
@@ -303,6 +303,52 @@ function refine(mesh: PolyMesh, chosen: ReadonlySet<number>, longest: number, bu
return { mesh: { positions, faces }, short };
}
+/** A point on the side of the mirror patterns and spots are worked out on. */
+export function mirroredPoint(mirror: DisplaceParams['mirror'], at: Vec3): [number, number, number] {
+ return [mirror === 'x' ? Math.abs(at[0]) : at[0], mirror === 'y' ? Math.abs(at[1]) : at[1], mirror === 'z' ? Math.abs(at[2]) : at[2]];
+}
+
+/** How much of a spot reaches a mirrored point: 1 at its middle, 0 from its edge. One spot named on one side is that spot on both. */
+export function nearWeight(near: NonNullable<DisplaceParams['near']>, mirror: DisplaceParams['mirror'], x: number, y: number, z: number): number {
+ const c = mirroredPoint(mirror, near.at);
+ const t = 1 - Math.hypot(x - c[0], y - c[1], z - c[2]) / near.radius;
+ return t <= 0 ? 0 : t * t * (3 - 2 * t);
+}
+
+/** Those of @p hit a spot can touch, by the box round each: one large face can cross it with every corner far outside. */
+export function reachedFaces(mesh: PolyMesh, hit: readonly number[], near: NonNullable<DisplaceParams['near']>, mirror: DisplaceParams['mirror']): number[] {
+ const { at, radius } = near;
+ const across = mirror ? { x: 0, y: 1, z: 2 }[mirror] : -1;
+ const twin: Vec3 = across === 0 ? [-at[0], at[1], at[2]] : across === 1 ? [at[0], -at[1], at[2]] : across === 2 ? [at[0], at[1], -at[2]] : at;
+ return hit.filter((f) => {
+ const { min, max } = boundsOf(mesh, mesh.faces[f]!.v);
+ return [at, twin].some((c) => {
+ const gap = [0, 1, 2].map((k) => Math.max(min[k]! - c[k]!, 0, c[k]! - max[k]!));
+ return Math.hypot(gap[0]!, gap[1]!, gap[2]!) < radius;
+ });
+ });
+}
+
+/** The pattern of @p p as a function of a mirrored point and its normal. */
+export function patternField(p: DisplaceParams): (x: number, y: number, z: number, n: Vec3) => number {
+ const stretch = p.stretch ?? [1, 1, 1];
+ const [sx, sy, sz] = [p.scale * stretch[0], p.scale * stretch[1], p.scale * stretch[2]];
+ const seed = p.seed ?? 1;
+ const warp = p.warp ?? 0;
+ return (px, py, pz, n) => {
+ let [x, y, z] = [px / sx, py / sy, pz / sz];
+ if (warp > 0) {
+ const [wx, wy, wz] = [
+ gradientNoise(x * 0.7 + 3.1, y * 0.7, z * 0.7, seed + 40),
+ gradientNoise(x * 0.7, y * 0.7 + 5.7, z * 0.7, seed + 41),
+ gradientNoise(x * 0.7, y * 0.7, z * 0.7 + 9.3, seed + 42),
+ ];
+ x += wx * warp; y += wy * warp; z += wz * warp;
+ }
+ return patternAt(p, x, y, z, n);
+ };
+}
+
export function displace(mesh: PolyMesh, p: DisplaceParams, budget = MAX_DISPLACE_TRIANGLES): PolyMesh {
if (!DISPLACE_PATTERNS.includes(p.pattern)) throw new Error(`pattern ${JSON.stringify(p.pattern)} must be one of ${DISPLACE_PATTERNS.join(', ')}`);
if (!(p.scale > 0)) throw new Error(`scale ${p.scale} must be positive`);
@@ -316,18 +362,8 @@ export function displace(mesh: PolyMesh, p: DisplaceParams, budget = MAX_DISPLAC
let hit = selectFaces(mesh, selector);
if (hit.length === 0) throw new Error(`\`faces\` ${JSON.stringify(selector)} matched no face`);
if (p.near) {
- const { at, radius } = p.near;
- // By the box round a face: one large face can cross the ball with every corner far outside it.
- const across = p.mirror ? { x: 0, y: 1, z: 2 }[p.mirror] : -1;
- const twin: Vec3 = across === 0 ? [-at[0], at[1], at[2]] : across === 1 ? [at[0], -at[1], at[2]] : across === 2 ? [at[0], at[1], -at[2]] : at;
- hit = hit.filter((f) => {
- const { min, max } = boundsOf(mesh, mesh.faces[f]!.v);
- return [at, twin].some((c) => {
- const gap = [0, 1, 2].map((k) => Math.max(min[k]! - c[k]!, 0, c[k]! - max[k]!));
- return Math.hypot(gap[0]!, gap[1]!, gap[2]!) < radius;
- });
- });
- if (hit.length === 0) throw new Error(`no surface of \`faces\` within ${radius} of ${JSON.stringify(at)}`);
+ hit = reachedFaces(mesh, hit, p.near, p.mirror);
+ if (hit.length === 0) throw new Error(`no surface of \`faces\` within ${p.near.radius} of ${JSON.stringify(p.near.at)}`);
}
const smallest = Math.min(...(p.stretch ?? [1, 1, 1])) * p.scale;
@@ -356,36 +392,14 @@ export function displace(mesh: PolyMesh, p: DisplaceParams, budget = MAX_DISPLAC
normals[v * 3 + 2]! += n[2] * area;
}
}
- const stretch = p.stretch ?? [1, 1, 1];
- const [sx, sy, sz] = [p.scale * stretch[0], p.scale * stretch[1], p.scale * stretch[2]];
- const seed = p.seed ?? 1;
- const warp = p.warp ?? 0;
+ const field = patternField(p);
const positions: Vec3[] = fine.positions.map((at, v) => {
if (!inside[v]) return at;
- // Mirrored first, so `near` is mirrored with the pattern: one spot asked for on one side is that spot on both.
- let [x, y, z] = [at[0], at[1], at[2]];
- if (p.mirror === 'x') x = Math.abs(x);
- if (p.mirror === 'y') y = Math.abs(y);
- if (p.mirror === 'z') z = Math.abs(z);
- let weight = 1;
- if (p.near) {
- const c = p.near.at;
- const toward = (axis: 'x' | 'y' | 'z', k: 0 | 1 | 2, v: number): number => (p.mirror === axis ? Math.abs(c[k]) : v);
- const t = 1 - Math.hypot(x - toward('x', 0, c[0]), y - toward('y', 1, c[1]), z - toward('z', 2, c[2])) / p.near.radius;
- if (t <= 0) return at;
- weight = t * t * (3 - 2 * t);
- }
+ const [x, y, z] = mirroredPoint(p.mirror, at);
+ const weight = p.near ? nearWeight(p.near, p.mirror, x, y, z) : 1;
+ if (weight <= 0) return at;
const n = normalize([normals[v * 3]!, normals[v * 3 + 1]!, normals[v * 3 + 2]!]);
- x /= sx; y /= sy; z /= sz;
- if (warp > 0) {
- const [wx, wy, wz] = [
- gradientNoise(x * 0.7 + 3.1, y * 0.7, z * 0.7, seed + 40),
- gradientNoise(x * 0.7, y * 0.7 + 5.7, z * 0.7, seed + 41),
- gradientNoise(x * 0.7, y * 0.7, z * 0.7 + 9.3, seed + 42),
- ];
- x += wx * warp; y += wy * warp; z += wz * warp;
- }
- const push = patternAt(p, x, y, z, n) * p.amplitude * weight;
+ const push = field(x, y, z, n) * p.amplitude * weight;
return [at[0] + n[0] * push, at[1] + n[1] * push, at[2] + n[2] * push];
});
diff --git a/pipeline/src/modeling/modelBake.ts b/pipeline/src/modeling/modelBake.ts
index 7f5a092da..9b0834098 100644
--- a/pipeline/src/modeling/modelBake.ts
+++ b/pipeline/src/modeling/modelBake.ts
@@ -64,7 +64,7 @@ export interface BakedModel {
/** One triangle per entry: its three positions' indices, and the face it came from. */
triangles: { v: [number, number, number]; face: number }[];
/** Present where a source has a normalBake: what bakeNormalMap carries, onto a mesh already unwrapped and given tangents for it. */
- detail?: { high: PolyMesh; size: number; distance?: number; smoothAngle: number };
+ detail?: { high: PolyMesh; size: number; distance?: number; smoothAngle: number; lowColors?: Uint8Array };
}
// ---- Report -----------------------------------------------------------------------
@@ -233,6 +233,21 @@ function copyLightmapUvToBase(mesh: MeshData): void {
}
}
+/**
+ * Whitens a mesh's vertex colours in place, for a colour map that holds them instead: the two multiply, and painted twice
+ * a colour would come out squared. Answers what they were, 4 bytes a vertex: where a map finds no surface, it falls back on them.
+ */
+function whitenVertexColors(mesh: MeshData): Uint8Array {
+ const { offset } = mesh.channels.find((c) => c.semantic === MeshChannel.Color)!;
+ const out = new Uint8Array(mesh.vertexCount * 4);
+ for (let v = 0; v < mesh.vertexCount; v++) {
+ const at = v * mesh.vertexStride + offset;
+ out.set(mesh.vertices.subarray(at, at + 4), v * 4);
+ mesh.vertices.fill(255, at, at + 4);
+ }
+ return out;
+}
+
/** The mesh with the frame a normal map is read in, from the UVs and normals as stored. */
function withTangents(mesh: MeshData): MeshData {
const view = new DataView(mesh.vertices.buffer, mesh.vertices.byteOffset, mesh.vertices.byteLength);
@@ -296,6 +311,7 @@ export function bakeModel(raw: unknown): BakedModel {
}
}
if (bake) unwrapped = withTangents(unwrapped);
+ const lowColors = bake?.color ? whitenVertexColors(unwrapped) : undefined;
const edges = [...collectEdges(mesh).values()];
const openEdges = edges.filter((e) => e.faces.length === 1).length;
@@ -353,7 +369,7 @@ export function bakeModel(raw: unknown): BakedModel {
data: unwrapped,
...(rig ? { rig } : {}),
...(shapes ? { shapes } : {}),
- ...(bake && high ? { detail: { high, size, ...(bake.distance !== undefined ? { distance: bake.distance } : {}), smoothAngle: source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE } } : {}),
+ ...(bake && high ? { detail: { high, size, ...(bake.distance !== undefined ? { distance: bake.distance } : {}), smoothAngle: source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE, ...(lowColors ? { lowColors } : {}) } } : {}),
triangles: triangles.flatMap((tris, face) => tris.map((v) => ({ v, face }))),
report: {
vertices: unwrapped.vertexCount,
diff --git a/pipeline/src/modeling/modelSource.ts b/pipeline/src/modeling/modelSource.ts
index 7d0508d89..748fd8203 100644
--- a/pipeline/src/modeling/modelSource.ts
+++ b/pipeline/src/modeling/modelSource.ts
@@ -20,6 +20,7 @@ import { lathe, loft } from './surfaces';
import { subdivide } from './subdivide';
import { decimate, triangleCount } from './decimate';
import { displace, DISPLACE_PATTERNS } from './displace';
+import { stain } from './stain';
import { checkAnimations, checkController, checkRig, checkShapes, type AnimationSource, type ControllerSource, type RigSource, type ShapeSource } from './rig';
import { compact, type Face, type FaceSelector, type PolyMesh, type Vec3 } from './polyMesh';
@@ -61,10 +62,12 @@ export interface NormalBakeSource {
size?: number;
/** How far from the final surface to look for the dense one, in model units. Default a twentieth of the diagonal. */
distance?: number;
+ /** Also writes the dense surface's paint as a base colour map, leaving the mesh's own vertex colours white. */
+ color?: boolean;
}
type Kind = 'number' | 'integer' | 'boolean' | 'string' | 'vec3' | 'number|vec3' | 'number|vec2' | 'number|numbers'
- | 'color' | 'points' | 'faces' | 'edges' | 'axis' | 'pivot' | 'parts' | 'noise' | 'sections' | 'section' | 'vec3s' | 'numbers' | 'operation' | 'uvmode' | 'number|vec2only' | 'vec2' | 'pattern' | 'near';
+ | 'color' | 'points' | 'faces' | 'edges' | 'axis' | 'pivot' | 'parts' | 'noise' | 'sections' | 'section' | 'vec3s' | 'numbers' | 'operation' | 'uvmode' | 'number|vec2only' | 'vec2' | 'pattern' | 'near' | 'where';
interface OpSpec {
params: Record<string, Kind>;
@@ -97,6 +100,7 @@ export const MODEL_OPS: Record<string, OpSpec> = {
boolean: { params: { operation: 'operation', with: 'faces', from: 'faces', keep: 'boolean', tag: 'string', color: 'color' }, required: ['operation', 'with'], run: booleanOp as unknown as OpSpec['run'], says: 'cuts or joins closed solids: subtract `with` from `from` makes a real hole' },
subdivide: { params: { faces: 'faces', levels: 'integer', crease: 'number' }, required: [], run: subdivide as unknown as OpSpec['run'], says: 'rounds whole shells into smooth surfaces, keeping edges sharper than `crease` degrees' },
displace: { params: { faces: 'faces', near: 'near', pattern: 'pattern', amplitude: 'number', scale: 'number', octaves: 'integer', seed: 'integer', stretch: 'vec3', warp: 'number', width: 'number', axis: 'axis', mirror: 'axis', detail: 'number' }, required: ['pattern', 'amplitude', 'scale'], run: ((mesh: PolyMesh, params: never) => displace(mesh, params)) as OpSpec['run'], says: 'pushes fine detail out of the surface along its normal, splitting faces as far as the pattern needs: bumps, wrinkles, cracks, scales, pores, ribs, weave or scratches' },
+ stain: { params: { faces: 'faces', near: 'near', hard: 'boolean', color: 'color', amount: 'number', pattern: 'pattern', scale: 'number', octaves: 'integer', seed: 'integer', stretch: 'vec3', warp: 'number', width: 'number', axis: 'axis', mirror: 'axis', threshold: 'number', softness: 'number', where: 'where', reach: 'number', facing: 'vec3', detail: 'number' }, required: ['color'], run: ((mesh: PolyMesh, params: never) => stain(mesh, params)) as OpSpec['run'], says: 'lays colour over the surface by a pattern, by shape (`where`: cavity for dirt in what is sunken, edges for wear on what stands out), by which way it faces, or around a spot: pupils, moss, grime' },
decimate: { params: { from: 'string', faces: 'faces', ratio: 'number', triangles: 'integer', error: 'number' }, required: [], run: decimate as unknown as OpSpec['run'], says: 'leaves fewer triangles over the same surface: `ratio` to keep or `triangles` to end with, stopping sooner if the surface would move more than `error` (default 0.02) of its size; `from` thins the surface as it stood after that step, before its fine detail' },
uv: { params: { faces: 'faces', mode: 'uvmode', axis: 'string', at: 'vec3', scale: 'number|vec2only', repeat: 'number|vec2only', offset: 'vec2', rotate: 'number' }, required: ['mode'], run: uvStep as unknown as OpSpec['run'], says: 'sets how faces take a texture: box, planar, cylinder or sphere' },
delete: { params: { faces: 'faces' }, required: ['faces'], run: ops.deleteFaces as OpSpec['run'], says: 'removes faces, leaving a hole' },
@@ -255,6 +259,7 @@ const KIND_CHECK: Record<Kind, (v: unknown) => string | null> = {
'number|vec2only': (v) => (isNum(v) || isNums(v, 2) ? null : 'must be a number or [u, v]'),
vec2: (v) => (isNums(v, 2) ? null : 'must be [u, v]'),
pattern: (v) => (DISPLACE_PATTERNS.includes(v as never) ? null : `must be one of ${DISPLACE_PATTERNS.join(', ')}`),
+ where: (v) => (v === 'cavity' || v === 'edges' ? null : 'must be cavity or edges'),
near: (v) => {
const n = v as { at?: unknown; radius?: unknown } | null;
return typeof n === 'object' && n !== null && isNums(n.at, 3) && isNum(n.radius) && Object.keys(n).every((k) => k === 'at' || k === 'radius') ? null : 'must be { at: [x, y, z], radius }';
@@ -271,12 +276,13 @@ export class ModelBuildError extends Error {
}
function checkNormalBake(v: unknown, steps: readonly Partial<ModelStep>[]): string | null {
- if (typeof v !== 'object' || v === null || Array.isArray(v)) return 'must be { from?, size?, distance? }';
+ if (typeof v !== 'object' || v === null || Array.isArray(v)) return 'must be { from?, size?, distance?, color? }';
const bake = v as Record<string, unknown>;
- for (const key of Object.keys(bake)) if (!['from', 'size', 'distance'].includes(key)) return `has an unknown field "${key}"; it takes from, size, distance`;
+ for (const key of Object.keys(bake)) if (!['from', 'size', 'distance', 'color'].includes(key)) return `has an unknown field "${key}"; it takes from, size, distance, color`;
const { size, distance, from } = bake;
if (size !== undefined && !(Number.isInteger(size) && (size as number) >= 64 && (size as number) <= 4096 && ((size as number) & ((size as number) - 1)) === 0)) return 'size must be a power of two, 64..4096';
if (distance !== undefined && !(isNum(distance) && distance > 0)) return 'distance must be a positive number';
+ if (bake.color !== undefined && typeof bake.color !== 'boolean') return 'color must be true or false';
if (from !== undefined) {
if (typeof from !== 'string') return 'from must be a step id';
if (!steps.some((s) => s?.id === from)) return `from "${from}" is no step's id`;
diff --git a/pipeline/src/modeling/normalBake.ts b/pipeline/src/modeling/normalBake.ts
index 26fecd866..0ad7e0592 100644
--- a/pipeline/src/modeling/normalBake.ts
+++ b/pipeline/src/modeling/normalBake.ts
@@ -49,6 +49,12 @@ export interface NormalMap {
size: number;
/** RGBA, row by row from the top: what a PNG holds. Data, not colour: it must be read without sRGB decoding. */
rgba: Uint8Array;
+ /**
+ * The dense surface's paint, with `normalBake.color`, laid out the same. Linear, as a vertex colour is, and so also
+ * read without sRGB decoding: a vertex colour reaches the shader unconverted in a gamma project and a linear one
+ * alike, so this reaches it as the same number in both.
+ */
+ colorRgba?: Uint8Array;
report: NormalBakeReport;
warnings: string[];
}
@@ -188,18 +194,26 @@ function readFloats(mesh: MeshData, semantic: number, components: number): Float
}
/** The dense surface as flat triangles, and the shading normal at each of their corners. */
-function denseSurface(high: PolyMesh, smoothAngle: number): { corners: Float64Array; normals: Float64Array } {
+function denseSurface(high: PolyMesh, smoothAngle: number): { corners: Float64Array; normals: Float64Array; colors: Uint8Array } {
const normalsOf = cornerNormals(high, smoothAngle);
- const positions: number[] = [], normals: number[] = [];
+ let count = 0;
+ for (const face of high.faces) count += Math.max(0, face.v.length - 2);
+ const corners = new Float64Array(count * 9), normals = new Float64Array(count * 9);
+ const colors = new Uint8Array(count * 4);
+ let t = 0;
high.faces.forEach((face, f) => {
+ const color = face.color ?? [1, 1, 1, 1];
+ const bytes = [0, 1, 2, 3].map((k) => Math.round(Math.max(0, Math.min(1, color[k]!)) * 255));
for (const tri of triangulateFace(high, face)) {
- for (const v of tri) {
- positions.push(...high.positions[v]!);
- normals.push(...normalsOf[f]![face.v.indexOf(v)]!);
- }
+ tri.forEach((v, c) => {
+ const [p, n] = [high.positions[v]!, normalsOf[f]![face.v.indexOf(v)]!];
+ for (let k = 0; k < 3; k++) { corners[t * 9 + c * 3 + k] = p[k]!; normals[t * 9 + c * 3 + k] = n[k]!; }
+ });
+ colors.set(bytes, t * 4);
+ t++;
}
});
- return { corners: Float64Array.from(positions), normals: Float64Array.from(normals) };
+ return { corners: t === count ? corners : corners.subarray(0, t * 9), normals, colors };
}
/** Bakes @p model's `normalBake`. Throws if it has none. */
@@ -209,7 +223,7 @@ export function bakeNormalMap(model: BakedModel): NormalMap {
const { size, high, smoothAngle } = detail;
const { min, max } = boundsOf(high);
const distance = detail.distance ?? Math.hypot(max[0] - min[0], max[1] - min[1], max[2] - min[2]) * DEFAULT_DISTANCE;
- const { corners, normals } = denseSurface(high, smoothAngle);
+ const { corners, normals, colors } = denseSurface(high, smoothAngle);
const tree = new TriangleTree(corners);
const mesh = model.data;
@@ -220,6 +234,8 @@ export function bakeNormalMap(model: BakedModel): NormalMap {
const rgba = new Uint8Array(size * size * 4);
for (let i = 0; i < size * size; i++) { rgba[i * 4] = 128; rgba[i * 4 + 1] = 128; rgba[i * 4 + 2] = 255; rgba[i * 4 + 3] = 255; }
+ const lowColors = detail.lowColors;
+ const colorRgba = lowColors ? new Uint8Array(size * size * 4) : undefined;
const covered = new Uint8Array(size * size);
const place = new Float64Array(2);
let hits = 0, missed = 0, bend = 0;
@@ -256,8 +272,14 @@ export function bakeNormalMap(model: BakedModel): NormalMap {
w0 * P[a * 3 + 2]! + w1 * P[b * 3 + 2]! + w2 * P[c * 3 + 2]!,
nx, ny, nz, distance, place,
);
- if (hit < 0) { missed++; continue; }
+ if (hit < 0) {
+ missed++;
+ // No surface to read: what the thin mesh was painted itself.
+ if (colorRgba) for (let k = 0; k < 4; k++) colorRgba[texel * 4 + k] = Math.round(w0 * lowColors![a * 4 + k]! + w1 * lowColors![b * 4 + k]! + w2 * lowColors![c * 4 + k]!);
+ continue;
+ }
hits++;
+ if (colorRgba) colorRgba.set(colors.subarray(hit * 4, hit * 4 + 4), texel * 4);
const [hv, hw] = [place[0]!, place[1]!];
const hu = 1 - hv - hw;
@@ -290,7 +312,7 @@ export function bakeNormalMap(model: BakedModel): NormalMap {
}
}
const inside = hits + missed;
- dilate(rgba, covered, size);
+ dilate(colorRgba ? [rgba, colorRgba] : [rgba], covered, size);
const from = corners.length / 9, to = mesh.indices.length / 3;
const warnings: string[] = [];
@@ -300,7 +322,7 @@ export function bakeNormalMap(model: BakedModel): NormalMap {
if (inside / (size * size) < SPARSE_COVERAGE) warnings.push(`the surface broke into so many pieces that they fill only ${Math.round((inside / (size * size)) * 100)}% of the map, so most of its resolution is lost: lower the amplitude before the decimate, or keep more triangles`);
if (from <= to) warnings.push(`the surface baked from has ${from} triangles and this one ${to}: there is no detail to carry. normalBake.from should name a step before the decimate`);
return {
- size, rgba, warnings,
+ size, rgba, warnings, ...(colorRgba ? { colorRgba } : {}),
report: {
size, from, to,
coverage: Math.round((inside / (size * size)) * 1000) / 1000,
@@ -311,8 +333,8 @@ export function bakeNormalMap(model: BakedModel): NormalMap {
};
}
-/** Spreads each chart's edge outward, nearest first. */
-function dilate(rgba: Uint8Array, covered: Uint8Array, size: number): void {
+/** Spreads each chart's edge outward through every image, nearest first. */
+function dilate(images: readonly Uint8Array[], covered: Uint8Array, size: number): void {
let edge: number[] = [];
for (let i = 0; i < covered.length; i++) if (covered[i]) edge.push(i);
for (let pass = 0; pass < GUTTER && edge.length > 0; pass++) {
@@ -326,9 +348,7 @@ function dilate(rgba: Uint8Array, covered: Uint8Array, size: number): void {
const q = qy * size + qx;
if (covered[q]) continue;
covered[q] = 1;
- rgba[q * 4] = rgba[i * 4]!;
- rgba[q * 4 + 1] = rgba[i * 4 + 1]!;
- rgba[q * 4 + 2] = rgba[i * 4 + 2]!;
+ for (const image of images) image.copyWithin(q * 4, i * 4, i * 4 + 4);
next.push(q);
}
}
diff --git a/pipeline/src/modeling/rig.ts b/pipeline/src/modeling/rig.ts
index f6fdfbf05..aefa08189 100644
--- a/pipeline/src/modeling/rig.ts
+++ b/pipeline/src/modeling/rig.ts
@@ -930,7 +930,7 @@ export function timelineOf(animation: AnimationSource, joints: readonly RigJoint
* The prefab that draws and moves it: the mesh on the root, the joints below it as
* entities named so a clip's paths find them, and a player for the first clip.
*/
-export function prefabOf(name: string, meshPath: string, joints: readonly RigJoint[], cullBackfaces: boolean, firstClip: string | null, controller: string | null = null, extra: { goals?: readonly RigGoal[]; shapes?: number; normalMap?: string } = {}): Record<string, unknown> {
+export function prefabOf(name: string, meshPath: string, joints: readonly RigJoint[], cullBackfaces: boolean, firstClip: string | null, controller: string | null = null, extra: { goals?: readonly RigGoal[]; shapes?: number; normalMap?: string; texture?: string } = {}): Record<string, unknown> {
const idOf = (i: number): string => `j${i}`;
const goals = extra.goals ?? [];
const entities: Record<string, unknown>[] = [{
@@ -939,7 +939,7 @@ export function prefabOf(name: string, meshPath: string, joints: readonly RigJoi
visible: true,
components: [
{ type: 'Transform', data: {} },
- { type: 'MeshRenderer', data: { mesh: meshPath, ...(extra.normalMap ? { normalMap: extra.normalMap } : {}), lit: true, opaque: true, cullBackfaces, enabled: true } },
+ { type: 'MeshRenderer', data: { mesh: meshPath, ...(extra.normalMap ? { normalMap: extra.normalMap } : {}), ...(extra.texture ? { texture: extra.texture } : {}), lit: true, opaque: true, cullBackfaces, enabled: true } },
...(joints.length ? [{ type: 'MeshSkin', data: { joints: joints.map((_, i) => idOf(i)) } }] : []),
...(extra.shapes ? [{ type: 'MeshMorph', data: { weights: Array.from({ length: extra.shapes }, () => 0) } }] : []),
// A controller decides what plays; a player beside it would fight it for the same joints.
diff --git a/pipeline/src/modeling/stain.ts b/pipeline/src/modeling/stain.ts
new file mode 100644
index 000000000..c17b0b691
--- /dev/null
+++ b/pipeline/src/modeling/stain.ts
@@ -0,0 +1,187 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file stain.ts — the `stain` step: colour laid over a surface by what is there, in place of one flat paint.
+ *
+ * Where it lands comes from up to four things multiplied together: a pattern, how the surface
+ * is shaped there, which way it faces, and a spot. It works on faces, as all paint here does, so
+ * how fine it can be is how fine the faces are: it splits them as far as its pattern needs, and a
+ * normalBake with `color` carries the result onto a thin mesh.
+ */
+import { DISPLACE_PATTERNS, finestShare, mirroredPoint, nearWeight, patternField, reachedFaces, refine, MAX_DISPLACE_TRIANGLES, type DisplaceParams, type DisplacePattern } from './displace';
+import { compact, faceArea, faceNormal, normalize, selectFaces, type Face, type FaceSelector, type PolyMesh, type Vec3 } from './polyMesh';
+
+export interface StainParams {
+ faces?: FaceSelector;
+ /** Only around here. Fades to its edge, or with `hard` stops there: a pupil. */
+ near?: { at: Vec3; radius: number };
+ hard?: boolean;
+ /** Linear [r, g, b], over what is there. */
+ color: readonly number[];
+ /** How strongly, 0..1, where the rest say all the way. Default 1. */
+ amount?: number;
+ pattern?: DisplacePattern;
+ scale?: number;
+ octaves?: number;
+ seed?: number;
+ stretch?: Vec3;
+ warp?: number;
+ width?: number;
+ axis?: 'x' | 'y' | 'z';
+ mirror?: 'x' | 'y' | 'z';
+ /** How much of the pattern it takes, 0..1, higher being less of it. Default 0.5. */
+ threshold?: number;
+ /** How gradually it starts, around the threshold. Default 0.15. */
+ softness?: number;
+ /** Dirt in what is sunken, or wear on what stands out. */
+ where?: 'cavity' | 'edges';
+ /** How wide a dip or a ridge counts, in model units. Default about eight edges. */
+ reach?: number;
+ /** Only where the surface faces this way: moss on top, damp below. */
+ facing?: Vec3;
+ /** The longest edge to leave. */
+ detail?: number;
+}
+
+const smooth = (lo: number, hi: number, x: number): number => {
+ const t = Math.max(0, Math.min(1, (x - lo) / (hi - lo)));
+ return t * t * (3 - 2 * t);
+};
+
+const MAX_SMOOTHING = 40;
+/** Shape is judged against the steepest few percent: a model's own depth decides what counts as deep, whatever its units. */
+const SHAPE_PERCENTILE = 0.95;
+/** Patterns that are dents (0 and below) rather than a rise and fall about 0. */
+const DENTS: ReadonlySet<DisplacePattern> = new Set(['wrinkles', 'cracks', 'pores', 'scratches']);
+
+/**
+ * How far each vertex sits below (positive) or above its surroundings, in shares of the model's deepest: the surface
+ * against a smoothed copy of itself, along the normal.
+ */
+function relief(mesh: PolyMesh, normals: Float64Array, reach: number | undefined): Float32Array {
+ const count = mesh.positions.length;
+ const smoothed = new Float64Array(count * 3);
+ mesh.positions.forEach((p, v) => { smoothed[v * 3] = p[0]; smoothed[v * 3 + 1] = p[1]; smoothed[v * 3 + 2] = p[2]; });
+ let edgeCount = 0;
+ for (const face of mesh.faces) edgeCount += face.v.length;
+ const from = new Int32Array(edgeCount), to = new Int32Array(edgeCount);
+ let n = 0, total = 0;
+ for (const face of mesh.faces) {
+ for (let i = 0; i < face.v.length; i++) {
+ const [a, b] = [face.v[i]!, face.v[(i + 1) % face.v.length]!];
+ from[n] = a; to[n] = b; n++;
+ const [p, q] = [mesh.positions[a]!, mesh.positions[b]!];
+ total += Math.hypot(p[0] - q[0], p[1] - q[1], p[2] - q[2]);
+ }
+ }
+ const edge = total / Math.max(1, n);
+ // Averaging spreads about one edge per pass squared, so the passes go with the square of how far it should reach.
+ const passes = Math.max(1, Math.min(MAX_SMOOTHING, Math.round(((reach ?? edge * 8) / edge) ** 2)));
+ const sum = new Float64Array(count * 3);
+ const links = new Int32Array(count);
+ for (let e = 0; e < n; e++) { links[from[e]!]!++; links[to[e]!]!++; }
+ for (let pass = 0; pass < passes; pass++) {
+ sum.fill(0);
+ for (let e = 0; e < n; e++) {
+ const [a, b] = [from[e]! * 3, to[e]! * 3];
+ for (let k = 0; k < 3; k++) { sum[a + k]! += smoothed[b + k]!; sum[b + k]! += smoothed[a + k]!; }
+ }
+ for (let v = 0; v < count; v++) {
+ if (links[v] === 0) continue;
+ for (let k = 0; k < 3; k++) smoothed[v * 3 + k] = sum[v * 3 + k]! / links[v]!;
+ }
+ }
+ const out = new Float32Array(count);
+ mesh.positions.forEach((p, v) => {
+ out[v] = (smoothed[v * 3]! - p[0]) * normals[v * 3]! + (smoothed[v * 3 + 1]! - p[1]) * normals[v * 3 + 1]! + (smoothed[v * 3 + 2]! - p[2]) * normals[v * 3 + 2]!;
+ });
+ const sorted = Float32Array.from(out, Math.abs).sort();
+ const deep = sorted[Math.floor((sorted.length - 1) * SHAPE_PERCENTILE)]! || 1;
+ for (let v = 0; v < count; v++) out[v]! /= deep;
+ return out;
+}
+
+export function stain(mesh: PolyMesh, p: StainParams, budget = MAX_DISPLACE_TRIANGLES): PolyMesh {
+ if (p.color.length < 3) throw new Error('color must be [r, g, b]');
+ if (p.pattern !== undefined && !DISPLACE_PATTERNS.includes(p.pattern)) throw new Error(`pattern ${JSON.stringify(p.pattern)} must be one of ${DISPLACE_PATTERNS.join(', ')}`);
+ if (p.pattern !== undefined && !(p.scale! > 0)) throw new Error('a pattern needs a positive `scale`');
+ if (p.amount !== undefined && !(p.amount >= 0 && p.amount <= 1)) throw new Error(`amount ${p.amount} must be 0..1`);
+ if (p.softness !== undefined && !(p.softness > 0)) throw new Error(`softness ${p.softness} must be positive`);
+ if (p.near && !(p.near.radius > 0)) throw new Error(`near.radius ${p.near.radius} must be positive`);
+ if (p.pattern === undefined && p.where === undefined && p.facing === undefined && p.near === undefined) throw new Error('say where: give at least one of pattern, where, facing, near; `paint` is for all of a set of faces');
+
+ const selector = p.faces ?? 'all';
+ let hit = selectFaces(mesh, selector);
+ if (hit.length === 0) throw new Error(`\`faces\` ${JSON.stringify(selector)} matched no face`);
+ if (p.near) {
+ hit = reachedFaces(mesh, hit, p.near, p.mirror);
+ if (hit.length === 0) throw new Error(`no surface of \`faces\` within ${p.near.radius} of ${JSON.stringify(p.near.at)}`);
+ }
+ const asPattern = { ...p, pattern: p.pattern ?? 'bumps', amplitude: 1, scale: p.scale ?? 1 } as DisplaceParams;
+ const longest = p.detail
+ ?? (p.pattern ? Math.min(...(p.stretch ?? [1, 1, 1])) * p.scale! * finestShare(asPattern)
+ // A hard edge is as clean as the faces it runs along.
+ : p.near && p.hard ? p.near.radius / 8 : null);
+ const { mesh: fine, short } = longest === null ? { mesh, short: false } : refine(mesh, new Set(hit), longest, budget);
+
+ const chosen = new Set(hit);
+ const isChosen = (f: number): boolean => (fine === mesh ? chosen.has(f) : f >= mesh.faces.length - hit.length);
+ const normals = new Float64Array(fine.positions.length * 3);
+ for (const face of fine.faces) {
+ const n = faceNormal(fine, face);
+ const area = faceArea(fine, face);
+ for (const v of face.v) { normals[v * 3]! += n[0] * area; normals[v * 3 + 1]! += n[1] * area; normals[v * 3 + 2]! += n[2] * area; }
+ }
+ for (let v = 0; v < fine.positions.length; v++) {
+ const l = Math.hypot(normals[v * 3]!, normals[v * 3 + 1]!, normals[v * 3 + 2]!) || 1;
+ normals[v * 3]! /= l; normals[v * 3 + 1]! /= l; normals[v * 3 + 2]! /= l;
+ }
+ const shape = p.where ? relief(fine, normals, p.reach) : null;
+ const facing = p.facing ? normalize(p.facing) : null;
+ const field = p.pattern ? patternField(asPattern) : null;
+ const threshold = p.threshold ?? 0.5, softness = p.softness ?? 0.15;
+ const amount = p.amount ?? 1;
+ const color = [p.color[0]!, p.color[1]!, p.color[2]!] as const;
+
+ const maskAt = (v: number): number => {
+ let m = 1;
+ const at = fine.positions[v]!;
+ const [x, y, z] = mirroredPoint(p.mirror, at);
+ if (p.near) {
+ const w = nearWeight(p.near, p.mirror, x, y, z);
+ m *= p.hard ? (w > 0 ? 1 : 0) : w;
+ if (m === 0) return 0;
+ }
+ const n: Vec3 = [normals[v * 3]!, normals[v * 3 + 1]!, normals[v * 3 + 2]!];
+ if (facing) {
+ m *= smooth(0.2, 0.75, n[0] * facing[0] + n[1] * facing[1] + n[2] * facing[2]);
+ if (m === 0) return 0;
+ }
+ if (shape) {
+ const sunk = p.where === 'cavity' ? shape[v]! : -shape[v]!;
+ m *= smooth(threshold * 0.4 - softness, threshold * 0.4 + softness, sunk);
+ if (m === 0) return 0;
+ }
+ if (field) {
+ const value = field(x, y, z, n);
+ const share = DENTS.has(p.pattern!) ? Math.max(0, -value) : value * 0.5 + 0.5;
+ m *= smooth(threshold - softness, threshold + softness, share);
+ }
+ return m;
+ };
+ const masks = new Float32Array(fine.positions.length).fill(-1);
+ const faces: Face[] = fine.faces.map((face, f) => {
+ if (!isChosen(f)) return face;
+ let sum = 0;
+ for (const v of face.v) {
+ if (masks[v]! < 0) masks[v] = maskAt(v);
+ sum += masks[v]!;
+ }
+ const m = (sum / face.v.length) * amount;
+ if (m <= 0) return face;
+ const was = face.color ?? [1, 1, 1, 1];
+ return { ...face, color: [was[0] + (color[0] - was[0]) * m, was[1] + (color[1] - was[1]) * m, was[2] + (color[2] - was[2]) * m, was[3]] as const };
+ });
+ const out = compact({ positions: fine.positions, faces });
+ return short ? { ...out, notes: [`this stain wants edges of ${longest!.toPrecision(3)} and stopped at ${budget} triangles, so it is coarser than asked: narrow it with \`faces\` or \`near\`, or give a larger \`detail\``] } : out;
+}
diff --git a/pipeline/tests/modeling-decimate-bake.test.ts b/pipeline/tests/modeling-decimate-bake.test.ts
index 016b86707..107cfc386 100644
--- a/pipeline/tests/modeling-decimate-bake.test.ts
+++ b/pipeline/tests/modeling-decimate-bake.test.ts
@@ -130,7 +130,7 @@ describe('decimate', () => {
it('`from` thins the surface before its fine detail, so the triangles go to the form and not to the noise', () => {
const box: ModelStep = { id: 'form', op: 'organic', resolution: 64, blend: 0.02, parts: [{ shape: 'box', size: [2, 2, 2], rounding: 0.05 }] };
- const noise: ModelStep = { op: 'displace', pattern: 'bumps', amplitude: 0.04, scale: 0.1, octaves: 2 };
+ const noise: ModelStep = { op: 'displace', pattern: 'bumps', amplitude: 0.04, scale: 0.1, octaves: 2, detail: 0.025 };
const after = model(box, noise, { op: 'decimate', triangles: 300, error: 0.5 });
const before = model(box, noise, { op: 'decimate', from: 'form', triangles: 300, error: 0.5 });
// Its corners are where the form put them: the noise cannot have moved or spent them.
@@ -141,7 +141,7 @@ describe('decimate', () => {
}, 60000);
it('`from` with a bake carries what the step before it made onto what it left', () => {
- const m = bakeModel({ version: 1, normalBake: { size: 128 }, steps: [{ ...ball, id: 'form' }, { op: 'displace', pattern: 'bumps', amplitude: 0.04, scale: 0.2, octaves: 2 }, { op: 'decimate', from: 'form', triangles: 300, error: 0.5 }] });
+ const m = bakeModel({ version: 1, normalBake: { size: 128 }, steps: [{ ...ball, id: 'form' }, { op: 'displace', pattern: 'bumps', amplitude: 0.04, scale: 0.2, octaves: 2, detail: 0.03 }, { op: 'decimate', from: 'form', triangles: 300, error: 0.5 }] });
const map = bakeNormalMap(m);
expect(map.report.from).toBeGreaterThan(map.report.to * 10);
expect(map.report.meanBend).toBeGreaterThan(3);
diff --git a/pipeline/tests/modeling-displace.test.ts b/pipeline/tests/modeling-displace.test.ts
index 48d471b23..e5a499638 100644
--- a/pipeline/tests/modeling-displace.test.ts
+++ b/pipeline/tests/modeling-displace.test.ts
@@ -24,7 +24,7 @@ function heights(m: BakedModel): { min: number; max: number; mean: number } {
describe('displace', () => {
it.each(DISPLACE_PATTERNS)('%s moves a closed ball within its amplitude, and leaves it closed', (pattern) => {
- const m = model(ball, { op: 'displace', pattern, amplitude: 0.05, scale: 0.8 });
+ const m = model(ball, { op: 'displace', pattern, amplitude: 0.05, scale: 0.8, detail: 0.04 });
expect(m.report.openEdges).toBe(0);
expect(m.report.nonManifoldEdges).toBe(0);
expect(m.report.insideOutShells).toBe(0);
@@ -35,9 +35,9 @@ describe('displace', () => {
});
it('is the same every time, and a seed makes another', () => {
- const a = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.4 });
- const b = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.4 });
- const c = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.4, seed: 9 });
+ const a = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.4, detail: 0.05 });
+ const b = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.4, detail: 0.05 });
+ const c = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.4, seed: 9, detail: 0.05 });
expect(a.poly.positions).toEqual(b.poly.positions);
expect(a.poly.positions).not.toEqual(c.poly.positions);
});
@@ -55,8 +55,8 @@ describe('displace', () => {
it('cracks and wrinkles are dents at a positive amplitude and ridges at a negative one', () => {
for (const pattern of ['cracks', 'wrinkles', 'pores', 'scratches'] as const) {
- const down = heights(model(sheet, { op: 'displace', pattern, amplitude: 0.1, scale: 0.5 }));
- const up = heights(model(sheet, { op: 'displace', pattern, amplitude: -0.1, scale: 0.5 }));
+ const down = heights(model(sheet, { op: 'displace', pattern, amplitude: 0.1, scale: 0.5, detail: 0.03 }));
+ const up = heights(model(sheet, { op: 'displace', pattern, amplitude: -0.1, scale: 0.5, detail: 0.03 }));
expect(down.max).toBeLessThan(0.001);
expect(down.min).toBeLessThan(-0.01);
expect(up.min).toBeGreaterThan(-0.001);
@@ -65,7 +65,7 @@ describe('displace', () => {
});
it('`near` moves only inside its radius', () => {
- const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.2, near: { at: [0.5, 0, 0], radius: 0.3 } });
+ const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.2, near: { at: [0.5, 0, 0], radius: 0.3 }, detail: 0.03 });
let moved = 0;
for (const p of m.poly.positions) {
const inside = Math.hypot(p[0] - 0.5, p[2]) < 0.3 + 0.03;
@@ -93,18 +93,18 @@ describe('displace', () => {
});
it('splits only what its pattern needs: an edge shared with a face outside does not make it run away', () => {
- const m = model({ op: 'box', size: 2 }, { op: 'tag', faces: '+y', tag: 'top' }, { op: 'displace', faces: 'top', pattern: 'bumps', amplitude: 0.1, scale: 0.4 });
+ const m = model({ op: 'box', size: 2 }, { op: 'tag', faces: '+y', tag: 'top' }, { op: 'displace', faces: 'top', pattern: 'bumps', amplitude: 0.1, scale: 0.4, detail: 0.05 });
expect(m.report.triangles).toBeLessThan(100000);
expect(m.report.warnings.join()).not.toMatch(/stopped at/);
});
it('`near` reaches a large face with every corner outside it', () => {
- const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.1, near: { at: [0, 0, 0], radius: 0.2 } });
+ const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.1, near: { at: [0, 0, 0], radius: 0.2 }, detail: 0.02 });
expect(m.poly.positions.some((p) => Math.abs(p[1]) > 0.005)).toBe(true);
});
it('keeps paint and tags on what it splits', () => {
- const m = model({ ...sheet, color: [0.2, 0.4, 0.6] }, { op: 'displace', pattern: 'bumps', amplitude: 0.05, scale: 0.3 });
+ const m = model({ ...sheet, color: [0.2, 0.4, 0.6] }, { op: 'displace', pattern: 'bumps', amplitude: 0.05, scale: 0.3, detail: 0.05 });
for (const f of m.poly.faces) { expect(f.color).toEqual([0.2, 0.4, 0.6, 1]); expect(f.tag).toBe('sheet'); }
});
@@ -119,7 +119,7 @@ describe('displace', () => {
});
it('`mirror` gives both sides the same', () => {
- const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.3, mirror: 'x' });
+ const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.3, mirror: 'x', detail: 0.03 });
const at = new Map(m.poly.positions.map((p) => [`${p[0].toFixed(4)},${p[2].toFixed(4)}`, p[1]]));
let pairs = 0;
for (const p of m.poly.positions) {
@@ -130,7 +130,7 @@ describe('displace', () => {
});
it('`mirror` takes `near` with it: a spot named on one side is on both', () => {
- const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.15, near: { at: [0.5, 0, 0], radius: 0.3 }, mirror: 'x' });
+ const m = model(sheet, { op: 'displace', pattern: 'bumps', amplitude: 0.1, scale: 0.15, near: { at: [0.5, 0, 0], radius: 0.3 }, mirror: 'x', detail: 0.02 });
const movedAt = (side: number): number => m.poly.positions.filter((p) => Math.sign(p[0]) === side && Math.abs(p[1]) > 1e-4).length;
expect(movedAt(1)).toBeGreaterThan(20);
expect(movedAt(-1)).toBe(movedAt(1));
@@ -169,7 +169,7 @@ describe('displace', () => {
}, 60000);
it('scratches are lines, not dots: where one is, it runs on', () => {
- const m = model(sheet, { op: 'displace', pattern: 'scratches', amplitude: 0.1, scale: 0.5, width: 0.1, detail: 0.008 });
+ const m = model(sheet, { op: 'displace', pattern: 'scratches', amplitude: 0.1, scale: 0.5, width: 0.1, detail: 0.012 });
// Bin the cut into a grid. Dots would leave cells apart; lines run through the grid, so most cut cells have a cut neighbour and the longest run is long.
const N = 125;
const cut = new Uint8Array(N * N);
@@ -200,7 +200,7 @@ describe('displace', () => {
it('what it adds survives a decimate as a normal map', () => {
const without = bakeModel({ version: 1, normalBake: { size: 128 }, steps: [ball, { op: 'decimate', triangles: 400, error: 0.3 }] });
- const withIt = bakeModel({ version: 1, normalBake: { size: 128 }, steps: [ball, { op: 'displace', pattern: 'wrinkles', amplitude: 0.03, scale: 0.3 }, { op: 'decimate', triangles: 400, error: 0.3 }] });
+ const withIt = bakeModel({ version: 1, normalBake: { size: 128 }, steps: [ball, { op: 'displace', pattern: 'wrinkles', amplitude: 0.03, scale: 0.3, detail: 0.02 }, { op: 'decimate', triangles: 400, error: 0.3 }] });
expect(bakeNormalMap(withIt).report.meanBend).toBeGreaterThan(bakeNormalMap(without).report.meanBend + 2);
}, 60000);
diff --git a/pipeline/tests/modeling-stain.test.ts b/pipeline/tests/modeling-stain.test.ts
new file mode 100644
index 000000000..d99df4f8a
--- /dev/null
+++ b/pipeline/tests/modeling-stain.test.ts
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/** Colour laid over a surface by what is there, and that colour carried onto a thin mesh by the bake. */
+import { beforeAll, describe, expect, it } from 'vitest';
+import { MeshChannel } from 'esengine';
+import { applyModelEdits, bakeModel, bakeNormalMap, prepareModeling, validateModelSource, type BakedModel, type ModelStep } from '../src/modeling';
+
+beforeAll(() => prepareModeling());
+
+const model = (...steps: ModelStep[]): BakedModel => bakeModel({ version: 1, steps });
+const ball: ModelStep = { id: 'form', op: 'sphere', radius: 1, rings: 32, sides: 64, color: [1, 1, 1], tag: 'ball' };
+const sheet: ModelStep = { op: 'plane', size: 2, color: [1, 1, 1], tag: 'sheet' };
+const RED = [1, 0, 0];
+
+const centre = (m: BakedModel, f: number): number[] => {
+ const v = m.poly.faces[f]!.v;
+ return [0, 1, 2].map((k) => v.reduce((sum, i) => sum + m.poly.positions[i]![k]!, 0) / v.length);
+};
+const redness = (m: BakedModel, f: number): number => 1 - (m.poly.faces[f]!.color?.[1] ?? 1);
+
+describe('stain', () => {
+ it('`near` with `hard` is a clean disc: all colour inside, none out', () => {
+ const m = model(sheet, { op: 'stain', near: { at: [0, 0, 0], radius: 0.3 }, hard: true, color: RED });
+ let inside = 0, outside = 0;
+ m.poly.faces.forEach((_, f) => {
+ const [x, , z] = centre(m, f);
+ const d = Math.hypot(x!, z!);
+ if (d < 0.25) { expect(redness(m, f)).toBe(1); inside++; }
+ if (d > 0.45) { expect(redness(m, f)).toBe(0); outside++; }
+ });
+ expect(inside).toBeGreaterThan(10);
+ expect(outside).toBeGreaterThan(0);
+ });
+
+ it('`near` without `hard` fades toward its edge', () => {
+ const m = model(sheet, { op: 'stain', near: { at: [0, 0, 0], radius: 0.8 }, color: RED, detail: 0.05 });
+ const byRing = (lo: number, hi: number): number => {
+ const hits = m.poly.faces.map((_, f) => f).filter((f) => { const [x, , z] = centre(m, f); const d = Math.hypot(x!, z!); return d >= lo && d < hi; });
+ return hits.reduce((sum, f) => sum + redness(m, f), 0) / hits.length;
+ };
+ expect(byRing(0, 0.1)).toBeGreaterThan(0.9);
+ expect(byRing(0.35, 0.45)).toBeGreaterThan(0.2);
+ expect(byRing(0.35, 0.45)).toBeLessThan(0.8);
+ expect(byRing(0.9, 1.5)).toBe(0);
+ });
+
+ it('`facing` takes what looks that way', () => {
+ const m = model(ball, { op: 'stain', facing: [0, 1, 0], color: RED });
+ m.poly.faces.forEach((_, f) => {
+ const y = centre(m, f)[1]!;
+ if (y > 0.85) expect(redness(m, f)).toBeGreaterThan(0.9);
+ if (y < 0) expect(redness(m, f)).toBe(0);
+ });
+ });
+
+ it('`where: cavity` finds the groove and `where: edges` the ridge', () => {
+ // One straight groove across a sheet, and a ridge further along.
+ const shaped: ModelStep[] = [
+ { op: 'plane', size: 4, color: [1, 1, 1] }, { op: 'slice', axis: 'x', at: Array.from({ length: 79 }, (_, i) => -1.95 + i * 0.05) },
+ { op: 'slice', axis: 'z', at: Array.from({ length: 79 }, (_, i) => -1.95 + i * 0.05) }, { op: 'weld' },
+ { op: 'transform', faces: { within: { min: [-1.2, null, null], max: [-0.8, null, null] } }, translate: [0, -0.2, 0] },
+ { op: 'transform', faces: { within: { min: [0.8, null, null], max: [1.2, null, null] } }, translate: [0, 0.2, 0] },
+ ];
+ const dirt = model(...shaped, { op: 'stain', where: 'cavity', reach: 0.4, color: RED });
+ const wear = model(...shaped, { op: 'stain', where: 'edges', reach: 0.4, color: RED });
+ const mean = (m: BakedModel, lo: number, hi: number): number => {
+ const hits = m.poly.faces.map((_, f) => f).filter((f) => { const c = centre(m, f); return c[0]! > lo && c[0]! < hi && Math.abs(c[2]!) < 1; });
+ return hits.reduce((sum, f) => sum + redness(m, f), 0) / hits.length;
+ };
+ expect(mean(dirt, -1.1, -0.9)).toBeGreaterThan(0.5);
+ expect(mean(dirt, 0.9, 1.1)).toBeLessThan(0.1);
+ expect(mean(wear, 0.9, 1.1)).toBeGreaterThan(0.5);
+ expect(mean(wear, -1.1, -0.9)).toBeLessThan(0.1);
+ }, 60000);
+
+ it('a pattern takes the share `threshold` says', () => {
+ const share = (threshold: number): number => {
+ const m = model(sheet, { op: 'stain', pattern: 'bumps', scale: 0.3, threshold, softness: 0.02, color: RED, detail: 0.03 });
+ return m.poly.faces.reduce((sum, _, f) => sum + (redness(m, f) > 0.5 ? 1 : 0), 0) / m.poly.faces.length;
+ };
+ expect(share(0.3)).toBeGreaterThan(share(0.5));
+ expect(share(0.5)).toBeGreaterThan(share(0.7));
+ });
+
+ it('`amount` stops part of the way, and goes over what is there', () => {
+ const m = model({ ...sheet, color: [0, 0, 1] }, { op: 'stain', near: { at: [0, 0, 0], radius: 10 }, hard: true, amount: 0.5, color: RED, detail: 2 });
+ for (const f of m.poly.faces) expect(f.color).toEqual([0.5, 0, 0.5, 1]);
+ });
+
+ it('`faces` keeps it off the others, and `mirror` puts a spot on both sides', () => {
+ const off = model(ball, { op: 'sphere', radius: 0.2, at: [3, 0, 0], tag: 'eye', color: [0, 1, 0] }, { op: 'stain', faces: 'eye', near: { at: [3, 0, 0], radius: 1 }, hard: true, color: RED });
+ for (const f of off.poly.faces) if (f.tag === 'ball') expect(f.color).toEqual([1, 1, 1, 1]);
+ const both = model(sheet, { op: 'stain', near: { at: [0.5, 0, 0], radius: 0.2 }, hard: true, mirror: 'x', color: RED });
+ const sides = [1, -1].map((side) => both.poly.faces.filter((_, f) => redness(both, f) > 0.5 && Math.sign(centre(both, f)[0]!) === side).length);
+ expect(sides[0]).toBeGreaterThan(5);
+ expect(sides[1]).toBe(sides[0]);
+ });
+
+ it('moves nothing: it is colour, and the shape stays as it was', () => {
+ const a = model(ball);
+ const b = model(ball, { op: 'stain', facing: [0, 1, 0], color: RED });
+ expect(b.poly.positions).toEqual(a.poly.positions);
+ });
+
+ it('refuses what cannot be meant', () => {
+ expect(() => model(ball, { op: 'stain', color: RED })).toThrow(/say where/);
+ expect(() => model(ball, { op: 'stain', color: RED, pattern: 'bumps' })).toThrow(/positive `scale`/);
+ expect(() => model(ball, { op: 'stain', color: RED, facing: [0, 1, 0], amount: 2 })).toThrow(/amount/);
+ expect(() => validateModelSource({ steps: [ball, { op: 'stain', color: RED, where: 'moss' }] })).toThrow(/must be cavity or edges/);
+ expect(() => model(ball, { op: 'stain', color: RED, near: { at: [9, 9, 9], radius: 1 } })).toThrow(/no surface/);
+ });
+});
+
+describe('normalBake color', () => {
+ const SIZE = 128;
+ /** The colour map at a point of a triangle of the mesh. */
+ function read(m: BakedModel, rgba: Uint8Array, triangle: number, w: readonly number[]): number[] {
+ const { data } = m;
+ const { offset } = data.channels.find((c) => c.semantic === MeshChannel.TexCoord0)!;
+ const view = new DataView(data.vertices.buffer, data.vertices.byteOffset, data.vertices.byteLength);
+ let [u, v] = [0, 0];
+ for (let k = 0; k < 3; k++) {
+ const at = data.indices[triangle * 3 + k]! * data.vertexStride + offset;
+ u += view.getFloat32(at, true) * w[k]!;
+ v += view.getFloat32(at + 4, true) * w[k]!;
+ }
+ const x = Math.min(SIZE - 1, Math.floor(u * SIZE)), y = Math.min(SIZE - 1, Math.floor((1 - v) * SIZE));
+ return [...rgba.subarray((y * SIZE + x) * 4, (y * SIZE + x) * 4 + 4)];
+ }
+ const positionsOf = (m: BakedModel, triangle: number, w: readonly number[]): number[] => {
+ const { data } = m;
+ const { offset } = data.channels.find((c) => c.semantic === MeshChannel.Position)!;
+ const view = new DataView(data.vertices.buffer, data.vertices.byteOffset, data.vertices.byteLength);
+ return [0, 1, 2].map((a) => [0, 1, 2].reduce((sum, k) => sum + view.getFloat32(data.indices[triangle * 3 + k]! * data.vertexStride + offset + a * 4, true) * w[k]!, 0));
+ };
+ const allWhite = (m: BakedModel): boolean => {
+ const { data } = m;
+ const { offset } = data.channels.find((c) => c.semantic === MeshChannel.Color)!;
+ for (let v = 0; v < data.vertexCount; v++) for (let k = 0; k < 4; k++) if (data.vertices[v * data.vertexStride + offset + k] !== 255) return false;
+ return true;
+ };
+
+ it('finer paint than the mesh has triangles lands where it was: upper half red, lower half blue, on 200 triangles', () => {
+ const m = bakeModel({ version: 1, normalBake: { size: SIZE, color: true }, steps: [
+ { ...ball, color: [0, 0, 1], rings: 64, sides: 128 }, { op: 'stain', facing: [0, 1, 0], color: RED, faces: { within: { min: [null, 0, null] } }, threshold: 0.1 },
+ { op: 'decimate', from: 'form', triangles: 200, error: 0.5 },
+ ] });
+ const map = bakeNormalMap(m);
+ expect(map.colorRgba).toBeDefined();
+ const seen = { red: 0, blue: 0 };
+ for (let t = 0; t < m.data.indices.length / 3; t++) {
+ const middle = [1 / 3, 1 / 3, 1 / 3];
+ const y = positionsOf(m, t, middle)[1]!;
+ if (y > 0.75) { expect(read(m, map.colorRgba!, t, middle)[0]).toBeGreaterThan(200); expect(read(m, map.colorRgba!, t, middle)[2]).toBeLessThan(60); seen.red++; }
+ if (y < -0.2) { expect(read(m, map.colorRgba!, t, middle)[2]).toBeGreaterThan(200); expect(read(m, map.colorRgba!, t, middle)[0]).toBeLessThan(10); seen.blue++; }
+ }
+ expect(seen.red).toBeGreaterThan(3);
+ expect(seen.blue).toBeGreaterThan(3);
+ }, 60000);
+
+ it('holds a colour as the same number a vertex colour would be: linear, not encoded', () => {
+ const m = bakeModel({ version: 1, normalBake: { size: 64, color: true, from: 'high' }, steps: [{ id: 'high', op: 'box', size: 2, color: [0.25, 0.5, 0.75] }] });
+ const map = bakeNormalMap(m);
+ expect(read2(m, map.colorRgba!, 64)).toEqual([Math.round(0.25 * 255), Math.round(0.5 * 255), Math.round(0.75 * 255), 255]);
+ });
+ function read2(m: BakedModel, rgba: Uint8Array, size: number): number[] {
+ const { offset } = m.data.channels.find((c) => c.semantic === MeshChannel.TexCoord0)!;
+ const view = new DataView(m.data.vertices.buffer, m.data.vertices.byteOffset, m.data.vertices.byteLength);
+ const [a, b, c] = [m.data.indices[0]!, m.data.indices[1]!, m.data.indices[2]!];
+ const uv = [a, b, c].map((i) => [view.getFloat32(i * m.data.vertexStride + offset, true), view.getFloat32(i * m.data.vertexStride + offset + 4, true)]);
+ const [u, v] = [(uv[0]![0]! + uv[1]![0]! + uv[2]![0]!) / 3, (uv[0]![1]! + uv[1]![1]! + uv[2]![1]!) / 3];
+ const at = (Math.floor((1 - v) * size) * size + Math.floor(u * size)) * 4;
+ return [...rgba.subarray(at, at + 4)];
+ }
+
+ it('turns the mesh\'s own colours white, or they would multiply in twice; without `color` they stay', () => {
+ const steps: ModelStep[] = [{ ...ball, color: [0.2, 0.4, 0.6] }, { op: 'decimate', ratio: 0.5, error: 0.2 }];
+ const colored = bakeModel({ version: 1, normalBake: { size: 64, color: true }, steps });
+ expect(allWhite(colored)).toBe(true);
+ const plain = bakeModel({ version: 1, normalBake: { size: 64 }, steps });
+ expect(allWhite(plain)).toBe(false);
+ expect(bakeNormalMap(plain).colorRgba).toBeUndefined();
+ });
+
+ it('where it finds no surface it falls back on what the mesh was painted, not on nothing', () => {
+ const m = bakeModel({ version: 1, normalBake: { size: 64, color: true, from: 'away', distance: 0.1 }, steps: [
+ { op: 'plane', size: 4, at: [0, 5, 0], color: [1, 0, 0] }, { id: 'away', op: 'transform', scale: 1 }, { op: 'delete', faces: 'all' }, { op: 'plane', size: 1, color: [0, 1, 0] },
+ ] });
+ const map = bakeNormalMap(m);
+ expect(map.report.missed).toBe(1);
+ expect(read2(m, map.colorRgba!, 64)).toEqual([0, 255, 0, 255]);
+ });
+
+ it('works on a rigged mesh', () => {
+ const m = bakeModel({ version: 1, normalBake: { size: 64, color: true }, rig: { joints: [{ name: 'root', at: [0, -1, 0] }, { name: 'top', at: [0, 1, 0], parent: 'root' }] }, steps: [{ ...ball, color: [0.5, 0.2, 0.1] }, { op: 'decimate', ratio: 0.3, error: 0.2 }] });
+ expect(allWhite(m)).toBe(true);
+ expect(read2(m, bakeNormalMap(m).colorRgba!, 64).slice(0, 3)).toEqual([Math.round(0.5 * 255), Math.round(0.2 * 255), Math.round(0.1 * 255)]);
+ });
+
+ it('`color` must be true or false, and an edit can take it away', () => {
+ expect(() => validateModelSource({ steps: [ball, { op: 'decimate', ratio: 0.5 }], normalBake: { color: 'yes' } })).toThrow(/color must be true or false/);
+ const source = validateModelSource({ steps: [ball, { op: 'decimate', ratio: 0.5 }], normalBake: { color: true } });
+ expect(applyModelEdits(source, [{ op: 'settings', fields: { normalBake: { size: 64 } } }]).normalBake?.color).toBeUndefined();
+ });
+});
diff --git a/sdk/src/lightmap/unwrap.ts b/sdk/src/lightmap/unwrap.ts
index 756fb4c32..7939eb093 100644
--- a/sdk/src/lightmap/unwrap.ts
+++ b/sdk/src/lightmap/unwrap.ts
@@ -40,8 +40,8 @@ const DEFAULTS = { maxChartAngleDegrees: 60, paddingTexels: 2, atlasSize: 1024 }
/** Cells along a side when laying out by outline. Past this a chart's outline is finer than anything packing gains from. */
const MAX_LAYOUT_GRID = 512;
-/** Beyond these, charts are packed as rectangles: outlines cost a pass over every triangle per size tried. */
-const MAX_OUTLINE_CHARTS = 1500;
+/** Beyond these, charts are packed as rectangles: outlines cost seconds, and many small charts gain little from them. */
+const MAX_OUTLINE_CHARTS = 200;
const MAX_OUTLINE_TRIANGLES = 200000;
const SHRINK = 0.8;
/** Halvings of the gap between a size that fits and one that does not: from a gap of 20%, to about a third of a percent. */
diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjs
index 1bba714ef..c8a007633 100644
--- a/tools/releaseNotes.mjs
+++ b/tools/releaseNotes.mjs
@@ -3878,6 +3878,10 @@ export const NOTED = {
{"note":"An AI building a model can make it light and keep its detail."},
"feat(mcp): model_write teaches thinning the form rather than its detail":
{"note":"An AI building a model can make it light and keep its detail."},
+ "feat(modeling): a stain step colours a surface by what is there, and a bake keeps it as a colour map":
+ {"note":"An AI building a model can colour it finely, and keep that on a thin mesh."},
+ "feat(mcp): model_write stains and bakes colour, checked in both colour spaces":
+ {"note":"An AI building a model can colour it finely, and keep that on a thin mesh."},
"feat(modeling): a displace step carves wrinkles, cracks, scales and weave into a surface":
{"note":"An AI building a model can carve fine detail into it."},
"feat(mcp): model_write carves fine detail, and a check takes every pattern through the bake":