A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
feat(modeling): a bake works out how much sky a dense surface sees, and shades its creases by it
yhh80f2cabe3c1a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 8b3f7e3e4..64432ff1d 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,6 +22,8 @@ published separately; it ships inside the editor.- **Glyphs can be put on the atlas before they are shown.** The first page of a novel is hundreds of new characters, all landing in one frame. `preloadGlyphs(app, label, text, { budgetMs })` makes them ahead of time, a slice a frame, drawn as that `Text` will draw them, and returns `progress`, `done` and `release()`. On a Snapdragon 8 Elite a book's 2992 characters at 28 px took 145 ms over 25 frames with the worst frame at 9.5 ms, after which the first page's frame went from 49 to 11 ms and no page made a new glyph. Preloaded glyphs hold their atlas pages (4 MB each, a few hundred characters at that size) until `release()`.- **Voxel GI lights what the camera does not see, and bounces light more than once.** The cache now reads the sun's shadow map back into the world, so a surface the sun lights behind or beside the camera still sends light into the view. Each surface also gives off again the bounce light it received, so a corner or a floor in shade gets light that took two or more bounces to reach it. A ray now brings what it meets at full strength however far it went (it used to fade with distance, 40% at 4 m with 12 cm voxels). Each voxel is weighed by how much of it a surface covers, so a bright patch no longer lights like the whole voxel around it. Together these keep the result close to a path tracer's whatever the voxel size. In a sunlit test room the whole frame carries 0.95 of Cycles' light at the default size and 0.74 at 12 cm voxels; before, it was 0.54 at the default and darker the finer the grid. With the lit patch behind the camera, the walls went from no bounce light to 25–57%. A closed room the sun only lights from outside stays dark. Two inspector controls tune it: **Off-screen Sun Albedo** (the colour the shadow map cannot give, assumed) and **Multi-bounce**. A fine grid no longer drops every voxel: a count truncated by the cache's ageing had kept them below the two samples a ray asks for. The WebGL2 fallback does the same in its own passes: each voxel asks the sun's map whether a sunlit surface stands in it, and traces two rays through last frame's grid for the light it is given. In the test room it carries 1.00 of Cycles' light at the default size (it was 0.56) and 0.80 at 12 cm; with the lit patch behind the camera 0.20 (it was 0). Its voxel pass costs about 1.3 ms more at 720p on a desktop.+- **An AI building a model can shade its creases.** `normalBake` with `occlusion: true` works out how much sky each place on the dense surface sees and writes that as an occlusion map, so cracks, folds and the contact between parts read as depth under ambient light.+- **An AI building a model can say what it is made of.** `material` sets roughness and metal for a whole model, `paint` and `stain` for parts of it, and `normalBake` with `surface: true` keeps the differences as a map: wet moss on dry stone, bare steel where paint wore off.- **An AI building a model can colour it finely, and keep that on a thin mesh.** A `stain` step lays a colour over a surface by a pattern, in what is sunken or on what stands out, by which way it faces or around a spot: moss on top, dirt in the cracks, a pupil. `normalBake` with `color: true` writes that as a base colour map, in linear and gamma projects alike.diff --git a/desktop b/desktopindex b1832769d..4e7ae93ca 160000--- a/desktop+++ b/desktop@@ -1 +1 @@-Subproject commit b1832769d87d44224e62b4e74e07b460f274c0a0+Subproject commit 4e7ae93cad1d54db56c4840d43e93190d78dacbddiff --git a/pipeline/src/modeling/modelBake.ts b/pipeline/src/modeling/modelBake.tsindex 23a92e480..c60300693 100644--- a/pipeline/src/modeling/modelBake.ts+++ b/pipeline/src/modeling/modelBake.ts@@ -64,7 +64,7 @@ export interface BakedModel {/** One triangle per entry: its three positions' indices, and the face it came from. */triangles: { v: [number, number, number]; face: number }[];/** Present where a source has a normalBake: what bakeNormalMap carries, onto a mesh already unwrapped and given tangents for it. */- detail?: { high: PolyMesh; size: number; distance?: number; smoothAngle: number; lowColors?: Uint8Array; surface?: { roughness: number; metallic: number } };+ detail?: { high: PolyMesh; size: number; distance?: number; smoothAngle: number; lowColors?: Uint8Array; surface?: { roughness: number; metallic: number }; occlusion?: { size?: number; reach?: number; samples?: number } };}// ---- Report -----------------------------------------------------------------------@@ -272,9 +272,9 @@ function withTangents(mesh: MeshData): MeshData {}/** What a material has to say about a model, or null where a plain MeshRenderer says all of it. */-export function modelMaterialDoc(source: ModelSource, cullBackfaces: boolean, maps: { normal?: string | null; surface?: string | null }): Record<string, unknown> | null {+export function modelMaterialDoc(source: ModelSource, cullBackfaces: boolean, maps: { normal?: string | null; surface?: string | null; occlusion?: string | null }): Record<string, unknown> | null {const { material } = source;- if (!material && !maps.surface) return null;+ if (!material && !maps.surface && !maps.occlusion) return null;return {version: MATERIAL_FORMAT_VERSION,type: 'material',@@ -285,6 +285,10 @@ export function modelMaterialDoc(source: ModelSource, cullBackfaces: boolean, macull: cullBackfaces ? CullMode.Back : CullMode.None,properties: {...(maps.normal ? { u_normalMap: maps.normal } : {}),+ ...(maps.occlusion ? {+ u_occlusionMap: maps.occlusion,+ u_occlusionStrength: typeof source.normalBake?.occlusion === 'object' ? source.normalBake.occlusion.strength ?? 1 : 1,+ } : {}),// A map holds the values and the factors multiply it, so they stand aside....(maps.surface? { u_metallicRoughnessMap: maps.surface, u_roughness: 1, u_metallic: 1 }@@ -395,7 +399,7 @@ export function bakeModel(raw: unknown): BakedModel {data: unwrapped,...(rig ? { rig } : {}),...(shapes ? { shapes } : {}),- ...(bake && high ? { detail: { high, size, ...(bake.distance !== undefined ? { distance: bake.distance } : {}), smoothAngle: source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE, ...(lowColors ? { lowColors } : {}), ...(bake.surface ? { surface: { roughness: source.material?.roughness ?? 1, metallic: source.material?.metallic ?? 0 } } : {}) } } : {}),+ ...(bake && high ? { detail: { high, size, ...(bake.distance !== undefined ? { distance: bake.distance } : {}), smoothAngle: source.smoothAngle ?? DEFAULT_SMOOTH_ANGLE, ...(lowColors ? { lowColors } : {}), ...(bake.occlusion ? { occlusion: typeof bake.occlusion === 'object' ? bake.occlusion : {} } : {}), ...(bake.surface ? { surface: { roughness: source.material?.roughness ?? 1, metallic: source.material?.metallic ?? 0 } } : {}) } } : {}),triangles: triangles.flatMap((tris, face) => tris.map((v) => ({ v, face }))),report: {vertices: unwrapped.vertexCount,diff --git a/pipeline/src/modeling/modelSource.ts b/pipeline/src/modeling/modelSource.tsindex 0fa3dc735..d8a8d4436 100644--- a/pipeline/src/modeling/modelSource.ts+++ b/pipeline/src/modeling/modelSource.ts@@ -64,6 +64,17 @@ export interface ModelMaterial {metallic?: number;}+export interface NormalBakeOcclusion {+ /** How far a ray looks, in model units. Default about 6% of the model's diagonal. */+ reach?: number;+ /** Rays a texel, more for less noise. Default 25. */+ samples?: number;+ /** Pixels along a side of the map. Default a quarter of `size`, 128 to 512. */+ size?: number;+ /** How much it darkens, 0..1. Default 1. */+ strength?: number;+}+export interface NormalBakeSource {/** The step whose result holds the detail. Default: what the last `decimate` started from. */from?: string;@@ -75,6 +86,8 @@ export interface NormalBakeSource {color?: boolean;/** Also writes the dense surface's roughness and metal as a map. */surface?: boolean;+ /** Also works out how much of the sky each place on the dense surface sees: dark in creases, where ambient light does not reach. */+ occlusion?: boolean | NormalBakeOcclusion;}type Kind = 'number' | 'integer' | 'boolean' | 'string' | 'vec3' | 'number|vec3' | 'number|vec2' | 'number|numbers'@@ -287,14 +300,23 @@ export class ModelBuildError extends Error {}function checkNormalBake(v: unknown, steps: readonly Partial<ModelStep>[]): string | null {- if (typeof v !== 'object' || v === null || Array.isArray(v)) return 'must be { from?, size?, distance?, color?, surface? }';+ if (typeof v !== 'object' || v === null || Array.isArray(v)) return 'must be { from?, size?, distance?, color?, surface?, occlusion? }';const bake = v as Record<string, unknown>;- for (const key of Object.keys(bake)) if (!['from', 'size', 'distance', 'color', 'surface'].includes(key)) return `has an unknown field "${key}"; it takes from, size, distance, color, surface`;+ for (const key of Object.keys(bake)) if (!['from', 'size', 'distance', 'color', 'surface', 'occlusion'].includes(key)) return `has an unknown field "${key}"; it takes from, size, distance, color, surface, occlusion`;const { size, distance, from } = bake;if (size !== undefined && !(Number.isInteger(size) && (size as number) >= 64 && (size as number) <= 4096 && ((size as number) & ((size as number) - 1)) === 0)) return 'size must be a power of two, 64..4096';if (distance !== undefined && !(isNum(distance) && distance > 0)) return 'distance must be a positive number';if (bake.color !== undefined && typeof bake.color !== 'boolean') return 'color must be true or false';if (bake.surface !== undefined && typeof bake.surface !== 'boolean') return 'surface must be true or false';+ if (bake.occlusion !== undefined && typeof bake.occlusion !== 'boolean') {+ const o = bake.occlusion as Record<string, unknown> | null;+ if (typeof o !== 'object' || o === null || Array.isArray(o)) return 'occlusion must be true, false or { reach?, samples?, size?, strength? }';+ for (const key of Object.keys(o)) if (!['reach', 'samples', 'size', 'strength'].includes(key)) return `occlusion has an unknown field "${key}"; it takes reach, samples, size, strength`;+ if (o.reach !== undefined && !(isNum(o.reach) && o.reach > 0)) return 'occlusion.reach must be a positive number';+ if (o.samples !== undefined && !(Number.isInteger(o.samples) && (o.samples as number) >= 1 && (o.samples as number) <= 256)) return 'occlusion.samples must be 1..256';+ if (o.size !== undefined && !(Number.isInteger(o.size) && (o.size as number) >= 16 && (o.size as number) <= 2048 && ((o.size as number) & ((o.size as number) - 1)) === 0)) return 'occlusion.size must be a power of two, 16..2048';+ if (o.strength !== undefined && !(isNum(o.strength) && o.strength >= 0 && o.strength <= 1)) return 'occlusion.strength must be 0..1';+ }if (from !== undefined) {if (typeof from !== 'string') return 'from must be a step id';if (!steps.some((s) => s?.id === from)) return `from "${from}" is no step's id`;diff --git a/pipeline/src/modeling/normalBake.ts b/pipeline/src/modeling/normalBake.tsindex 2824f1f98..aba0d31cf 100644--- a/pipeline/src/modeling/normalBake.ts+++ b/pipeline/src/modeling/normalBake.ts@@ -27,6 +27,11 @@ const LEAF = 4;const MISSED_WARNING = 0.05;/** Degrees of mean bend past which a map is mostly slopes too steep to shade. */const STEEP_BEND = 25;+const MIN_OCCLUSION_SIZE = 128;+const MAX_OCCLUSION_SIZE = 512;+/** How far a ray looks, as a share of the model's diagonal. */+const DEFAULT_OCCLUSION_REACH = 0.06;+const DEFAULT_OCCLUSION_SAMPLES = 25;/** Share of the image below which the surface broke into so many pieces that the map's resolution is mostly gone. */const SPARSE_COVERAGE = 0.1;@@ -57,6 +62,8 @@ export interface NormalMap {colorRgba?: Uint8Array;/** The dense surface's roughness in green and metal in blue, with `normalBake.surface`: glTF's packing, which u_metallicRoughnessMap reads. */surfaceRgba?: Uint8Array;+ /** How much sky each texel sees, with `normalBake.occlusion`, grey. Smaller than the others: what it holds has no fine detail, and each texel costs dozens of rays. */+ occlusion?: { size: number; rgba: Uint8Array };report: NormalBakeReport;warnings: string[];}@@ -181,6 +188,84 @@ class TriangleTree {}return found;}++ /** Whether anything lies on the ray from @p o along @p d within @p reach, whichever way it faces. */+ blocked(ox: number, oy: number, oz: number, dx: number, dy: number, dz: number, reach: number): boolean {+ const [boxes, links, order, c, stack] = [this.boxes, this.links, this.order, this.corners, this.stack];+ let top = 0;+ stack[top++] = 0;+ while (top > 0) {+ const node = stack[--top]!;+ const at = node * 6;+ let near = 0, far = reach;+ let inside = true;+ for (let k = 0; k < 3 && inside; k++) {+ const d = k === 0 ? dx : k === 1 ? dy : dz, o = k === 0 ? ox : k === 1 ? oy : oz;+ const lo = boxes[at + k]!, hi = boxes[at + 3 + k]!;+ if (d === 0) { inside = o >= lo && o <= hi; continue; }+ let t0 = (lo - o) / d, t1 = (hi - o) / d;+ if (t0 > t1) { const s = t0; t0 = t1; t1 = s; }+ if (t0 > near) near = t0;+ if (t1 < far) far = t1;+ inside = near <= far;+ }+ if (!inside) continue;+ const count = links[node * 2 + 1]!;+ if (count === 0) {+ stack[top++] = links[node * 2]!;+ stack[top++] = node + 1;+ continue;+ }+ const first = links[node * 2]!;+ for (let i = first; i < first + count; i++) {+ const p = order[i]! * 9;+ const ax = c[p]!, ay = c[p + 1]!, az = c[p + 2]!;+ const e1x = c[p + 3]! - ax, e1y = c[p + 4]! - ay, e1z = c[p + 5]! - az;+ const e2x = c[p + 6]! - ax, e2y = c[p + 7]! - ay, e2z = c[p + 8]! - az;+ const px = dy * e2z - dz * e2y, py = dz * e2x - dx * e2z, pz = dx * e2y - dy * e2x;+ const det = e1x * px + e1y * py + e1z * pz;+ if (det > -1e-14 && det < 1e-14) continue;+ const inv = 1 / det;+ const sx = ox - ax, sy = oy - ay, sz = oz - az;+ const u = (sx * px + sy * py + sz * pz) * inv;+ if (u < 0 || u > 1) continue;+ const qx = sy * e1z - sz * e1y, qy = sz * e1x - sx * e1z, qz = sx * e1y - sy * e1x;+ const v = (dx * qx + dy * qy + dz * qz) * inv;+ if (v < 0 || u + v > 1) continue;+ const along = (e2x * qx + e2y * qy + e2z * qz) * inv;+ if (along > 0 && along < reach) return true;+ }+ }+ return false;+ }+}++/** Every texel centre inside a triangle of the mesh, with the triangle's corners and how much of each it is. */+function rasterTexels(+ indices: ArrayLike<number>, UV: Float32Array, size: number,+ visit: (texel: number, a: number, b: number, c: number, w0: number, w1: number, w2: number) => void,+): void {+ for (let t = 0; t + 2 < indices.length; t += 3) {+ const [a, b, c] = [indices[t]!, indices[t + 1]!, indices[t + 2]!];+ // A texture's rows run down from the top and the engine's v runs up from the bottom.+ const ax = UV[a * 2]! * size, ay = (1 - UV[a * 2 + 1]!) * size;+ const bx = UV[b * 2]! * size, by = (1 - UV[b * 2 + 1]!) * size;+ const cx = UV[c * 2]! * size, cy = (1 - UV[c * 2 + 1]!) * size;+ const area = (bx - ax) * (cy - ay) - (cx - ax) * (by - ay);+ if (Math.abs(area) < 1e-12) continue;+ const x0 = Math.max(0, Math.floor(Math.min(ax, bx, cx))), x1 = Math.min(size - 1, Math.ceil(Math.max(ax, bx, cx)));+ const y0 = Math.max(0, Math.floor(Math.min(ay, by, cy))), y1 = Math.min(size - 1, Math.ceil(Math.max(ay, by, cy)));+ for (let y = y0; y <= y1; y++) {+ for (let x = x0; x <= x1; x++) {+ const px = x + 0.5, py = y + 0.5;+ const w0 = ((bx - px) * (cy - py) - (cx - px) * (by - py)) / area;+ const w1 = ((cx - px) * (ay - py) - (ax - px) * (cy - py)) / area;+ const w2 = 1 - w0 - w1;+ if (w0 < 0 || w1 < 0 || w2 < 0) continue;+ visit(y * size + x, a, b, c, w0, w1, w2);+ }+ }+ }}/** Reads the channels the bake needs off the baked mesh, wherever the layout put them. */@@ -256,78 +341,60 @@ export function bakeNormalMap(model: BakedModel): NormalMap {const place = new Float64Array(2);let hits = 0, missed = 0, bend = 0;- for (let t = 0; t + 2 < mesh.indices.length; t += 3) {- const [a, b, c] = [mesh.indices[t]!, mesh.indices[t + 1]!, mesh.indices[t + 2]!];- // A texture's rows run down from the top and the engine's v runs up from the bottom.- const ax = UV[a * 2]! * size, ay = (1 - UV[a * 2 + 1]!) * size;- const bx = UV[b * 2]! * size, by = (1 - UV[b * 2 + 1]!) * size;- const cx = UV[c * 2]! * size, cy = (1 - UV[c * 2 + 1]!) * size;- const area = (bx - ax) * (cy - ay) - (cx - ax) * (by - ay);- if (Math.abs(area) < 1e-12) continue;- const x0 = Math.max(0, Math.floor(Math.min(ax, bx, cx))), x1 = Math.min(size - 1, Math.ceil(Math.max(ax, bx, cx)));- const y0 = Math.max(0, Math.floor(Math.min(ay, by, cy))), y1 = Math.min(size - 1, Math.ceil(Math.max(ay, by, cy)));- for (let y = y0; y <= y1; y++) {- for (let x = x0; x <= x1; x++) {- const texel = y * size + x;- if (covered[texel]) continue;- const px = x + 0.5, py = y + 0.5;- const w0 = ((bx - px) * (cy - py) - (cx - px) * (by - py)) / area;- const w1 = ((cx - px) * (ay - py) - (ax - px) * (cy - py)) / area;- const w2 = 1 - w0 - w1;- if (w0 < 0 || w1 < 0 || w2 < 0) continue;- covered[texel] = 1;+ rasterTexels(mesh.indices, UV, size, (texel, a, b, c, w0, w1, w2) => {+ if (covered[texel]) return;+ covered[texel] = 1;- let nx = w0 * N[a * 3]! + w1 * N[b * 3]! + w2 * N[c * 3]!;- let ny = w0 * N[a * 3 + 1]! + w1 * N[b * 3 + 1]! + w2 * N[c * 3 + 1]!;- let nz = w0 * N[a * 3 + 2]! + w1 * N[b * 3 + 2]! + w2 * N[c * 3 + 2]!;- const nl = Math.hypot(nx, ny, nz) || 1;- nx /= nl; ny /= nl; nz /= nl;- const hit = tree.nearest(- w0 * P[a * 3]! + w1 * P[b * 3]! + w2 * P[c * 3]!,- w0 * P[a * 3 + 1]! + w1 * P[b * 3 + 1]! + w2 * P[c * 3 + 1]!,- w0 * P[a * 3 + 2]! + w1 * P[b * 3 + 2]! + w2 * P[c * 3 + 2]!,- nx, ny, nz, distance, place,- );- if (hit < 0) {- missed++;- // No surface to read: what the thin mesh was painted itself.- if (colorRgba) for (let k = 0; k < 4; k++) colorRgba[texel * 4 + k] = Math.round(w0 * lowColors![a * 4 + k]! + w1 * lowColors![b * 4 + k]! + w2 * lowColors![c * 4 + k]!);- continue;- }- hits++;- if (colorRgba) colorRgba.set(colors.subarray(hit * 4, hit * 4 + 4), texel * 4);- if (surfaceRgba) { surfaceRgba[texel * 4 + 1] = surfaces[hit * 2]!; surfaceRgba[texel * 4 + 2] = surfaces[hit * 2 + 1]!; }+ let nx = w0 * N[a * 3]! + w1 * N[b * 3]! + w2 * N[c * 3]!;+ let ny = w0 * N[a * 3 + 1]! + w1 * N[b * 3 + 1]! + w2 * N[c * 3 + 1]!;+ let nz = w0 * N[a * 3 + 2]! + w1 * N[b * 3 + 2]! + w2 * N[c * 3 + 2]!;+ const nl = Math.hypot(nx, ny, nz) || 1;+ nx /= nl; ny /= nl; nz /= nl;+ const hit = tree.nearest(+ w0 * P[a * 3]! + w1 * P[b * 3]! + w2 * P[c * 3]!,+ w0 * P[a * 3 + 1]! + w1 * P[b * 3 + 1]! + w2 * P[c * 3 + 1]!,+ w0 * P[a * 3 + 2]! + w1 * P[b * 3 + 2]! + w2 * P[c * 3 + 2]!,+ nx, ny, nz, distance, place,+ );+ if (hit < 0) {+ missed++;+ // No surface to read: what the thin mesh was painted itself.+ if (colorRgba) for (let k = 0; k < 4; k++) colorRgba[texel * 4 + k] = Math.round(w0 * lowColors![a * 4 + k]! + w1 * lowColors![b * 4 + k]! + w2 * lowColors![c * 4 + k]!);+ return;+ }+ hits++;+ if (colorRgba) colorRgba.set(colors.subarray(hit * 4, hit * 4 + 4), texel * 4);+ if (surfaceRgba) { surfaceRgba[texel * 4 + 1] = surfaces[hit * 2]!; surfaceRgba[texel * 4 + 2] = surfaces[hit * 2 + 1]!; }- const [hv, hw] = [place[0]!, place[1]!];- const hu = 1 - hv - hw;- const at = hit * 9;- let sx = hu * normals[at]! + hv * normals[at + 3]! + hw * normals[at + 6]!;- let sy = hu * normals[at + 1]! + hv * normals[at + 4]! + hw * normals[at + 7]!;- let sz = hu * normals[at + 2]! + hv * normals[at + 5]! + hw * normals[at + 8]!;- const sl = Math.hypot(sx, sy, sz) || 1;- sx /= sl; sy /= sl; sz /= sl;+ const [hv, hw] = [place[0]!, place[1]!];+ const hu = 1 - hv - hw;+ const at = hit * 9;+ let sx = hu * normals[at]! + hv * normals[at + 3]! + hw * normals[at + 6]!;+ let sy = hu * normals[at + 1]! + hv * normals[at + 4]! + hw * normals[at + 7]!;+ let sz = hu * normals[at + 2]! + hv * normals[at + 5]! + hw * normals[at + 8]!;+ const sl = Math.hypot(sx, sy, sz) || 1;+ sx /= sl; sy /= sl; sz /= sl;- // The frame the shader builds, in the order it builds it.- const tx = w0 * T[a * 4]! + w1 * T[b * 4]! + w2 * T[c * 4]!;- const ty = w0 * T[a * 4 + 1]! + w1 * T[b * 4 + 1]! + w2 * T[c * 4 + 1]!;- const tz = w0 * T[a * 4 + 2]! + w1 * T[b * 4 + 2]! + w2 * T[c * 4 + 2]!;- const handed = w0 * T[a * 4 + 3]! + w1 * T[b * 4 + 3]! + w2 * T[c * 4 + 3]!;- const lean = tx * nx + ty * ny + tz * nz;- let gx = tx - nx * lean, gy = ty - ny * lean, gz = tz - nz * lean;- const gl = Math.hypot(gx, gy, gz) || 1;- gx /= gl; gy /= gl; gz /= gl;- const ex = (ny * gz - nz * gy) * handed, ey = (nz * gx - nx * gz) * handed, ez = (nx * gy - ny * gx) * handed;+ // The frame the shader builds, in the order it builds it.+ const tx = w0 * T[a * 4]! + w1 * T[b * 4]! + w2 * T[c * 4]!;+ const ty = w0 * T[a * 4 + 1]! + w1 * T[b * 4 + 1]! + w2 * T[c * 4 + 1]!;+ const tz = w0 * T[a * 4 + 2]! + w1 * T[b * 4 + 2]! + w2 * T[c * 4 + 2]!;+ const handed = w0 * T[a * 4 + 3]! + w1 * T[b * 4 + 3]! + w2 * T[c * 4 + 3]!;+ const lean = tx * nx + ty * ny + tz * nz;+ let gx = tx - nx * lean, gy = ty - ny * lean, gz = tz - nz * lean;+ const gl = Math.hypot(gx, gy, gz) || 1;+ gx /= gl; gy /= gl; gz /= gl;+ const ex = (ny * gz - nz * gy) * handed, ey = (nz * gx - nx * gz) * handed, ez = (nx * gy - ny * gx) * handed;- const u = sx * gx + sy * gy + sz * gz;- const v = sx * ex + sy * ey + sz * ez;- const w = sx * nx + sy * ny + sz * nz;- bend += Math.acos(Math.max(-1, Math.min(1, w)));- rgba[texel * 4] = Math.round((u * 0.5 + 0.5) * 255);- rgba[texel * 4 + 1] = Math.round((v * 0.5 + 0.5) * 255);- rgba[texel * 4 + 2] = Math.round((Math.max(w, 0) * 0.5 + 0.5) * 255);- }- }- }+ const u = sx * gx + sy * gy + sz * gz;+ const v = sx * ex + sy * ey + sz * ez;+ const w = sx * nx + sy * ny + sz * nz;+ bend += Math.acos(Math.max(-1, Math.min(1, w)));+ rgba[texel * 4] = Math.round((u * 0.5 + 0.5) * 255);+ rgba[texel * 4 + 1] = Math.round((v * 0.5 + 0.5) * 255);+ rgba[texel * 4 + 2] = Math.round((Math.max(w, 0) * 0.5 + 0.5) * 255);+ });+ const occlusion = detail.occlusion ? bakeOcclusion(tree, corners, normals, mesh.indices, P, N, UV, size, distance, detail.occlusion, Math.hypot(max[0] - min[0], max[1] - min[1], max[2] - min[2])) : undefined;const inside = hits + missed;dilate([rgba, ...(colorRgba ? [colorRgba] : []), ...(surfaceRgba ? [surfaceRgba] : [])], covered, size);@@ -339,7 +406,7 @@ export function bakeNormalMap(model: BakedModel): NormalMap {if (inside / (size * size) < SPARSE_COVERAGE) warnings.push(`the surface broke into so many pieces that they fill only ${Math.round((inside / (size * size)) * 100)}% of the map, so most of its resolution is lost: lower the amplitude before the decimate, or keep more triangles`);if (from <= to) warnings.push(`the surface baked from has ${from} triangles and this one ${to}: there is no detail to carry. normalBake.from should name a step before the decimate`);return {- size, rgba, warnings, ...(colorRgba ? { colorRgba } : {}), ...(surfaceRgba ? { surfaceRgba } : {}),+ size, rgba, warnings, ...(colorRgba ? { colorRgba } : {}), ...(surfaceRgba ? { surfaceRgba } : {}), ...(occlusion ? { occlusion } : {}),report: {size, from, to,coverage: Math.round((inside / (size * size)) * 1000) / 1000,@@ -350,6 +417,97 @@ export function bakeNormalMap(model: BakedModel): NormalMap {};}+/**+ * Cosine-weighted rays from the dense surface, not the thin one: the thin one has no creases to be shaded by. A short reach+ * keeps it to contact and crevices, which is what a sky's light misses; a long one is what a light does already.+ */+function bakeOcclusion(+ tree: TriangleTree, corners: Float64Array, normals: Float64Array, indices: ArrayLike<number>,+ P: Float32Array, N: Float32Array, UV: Float32Array, mainSize: number, distance: number,+ options: { size?: number; reach?: number; samples?: number }, diagonal: number,+): { size: number; rgba: Uint8Array } {+ const size = options.size ?? Math.max(MIN_OCCLUSION_SIZE, Math.min(MAX_OCCLUSION_SIZE, mainSize >> 2));+ const reach = options.reach ?? diagonal * DEFAULT_OCCLUSION_REACH;+ const samples = options.samples ?? DEFAULT_OCCLUSION_SAMPLES;+ const bias = diagonal * 2e-4;+ const values = new Float32Array(size * size).fill(-1);+ const place = new Float64Array(2);+ // One set of directions, turned per texel: a pattern the blur below can take out, and a bake that comes out the same every time.+ const side = Math.ceil(Math.sqrt(samples));+ rasterTexels(indices, UV, size, (texel, a, b, c, w0, w1, w2) => {+ if (values[texel]! >= 0) return;+ const nx0 = w0 * N[a * 3]! + w1 * N[b * 3]! + w2 * N[c * 3]!;+ const ny0 = w0 * N[a * 3 + 1]! + w1 * N[b * 3 + 1]! + w2 * N[c * 3 + 1]!;+ const nz0 = w0 * N[a * 3 + 2]! + w1 * N[b * 3 + 2]! + w2 * N[c * 3 + 2]!;+ const nl = Math.hypot(nx0, ny0, nz0) || 1;+ const hit = tree.nearest(+ w0 * P[a * 3]! + w1 * P[b * 3]! + w2 * P[c * 3]!,+ w0 * P[a * 3 + 1]! + w1 * P[b * 3 + 1]! + w2 * P[c * 3 + 1]!,+ w0 * P[a * 3 + 2]! + w1 * P[b * 3 + 2]! + w2 * P[c * 3 + 2]!,+ nx0 / nl, ny0 / nl, nz0 / nl, distance, place,+ );+ if (hit < 0) { values[texel] = 1; return; }+ const [hv, hw] = [place[0]!, place[1]!];+ const hu = 1 - hv - hw;+ const at = hit * 9;+ const px = hu * corners[at]! + hv * corners[at + 3]! + hw * corners[at + 6]!;+ const py = hu * corners[at + 1]! + hv * corners[at + 4]! + hw * corners[at + 7]!;+ const pz = hu * corners[at + 2]! + hv * corners[at + 5]! + hw * corners[at + 8]!;+ let sx = hu * normals[at]! + hv * normals[at + 3]! + hw * normals[at + 6]!;+ let sy = hu * normals[at + 1]! + hv * normals[at + 4]! + hw * normals[at + 7]!;+ let sz = hu * normals[at + 2]! + hv * normals[at + 5]! + hw * normals[at + 8]!;+ const sl = Math.hypot(sx, sy, sz) || 1;+ sx /= sl; sy /= sl; sz /= sl;+ // Two axes square to the normal.+ let ux = Math.abs(sx) < 0.9 ? 0 : -sz, uy = Math.abs(sx) < 0.9 ? -sz : 0, uz = Math.abs(sx) < 0.9 ? sy : sx;+ const ul = Math.hypot(ux, uy, uz) || 1;+ ux /= ul; uy /= ul; uz /= ul;+ const vx = sy * uz - sz * uy, vy = sz * ux - sx * uz, vz = sx * uy - sy * ux;+ const turn = (texel * 0.61803398875) % 1;+ const [ox, oy, oz] = [px + sx * bias, py + sy * bias, pz + sz * bias];+ let open = 0, total = 0;+ for (let i = 0; i < side; i++) {+ for (let j = 0; j < side; j++) {+ const r = Math.sqrt((i + 0.5) / side);+ const phi = ((j + 0.5) / side + turn) * 2 * Math.PI;+ const [du, dv] = [r * Math.cos(phi), r * Math.sin(phi)];+ const dn = Math.sqrt(Math.max(0, 1 - r * r));+ total++;+ if (!tree.blocked(ox, oy, oz, ux * du + vx * dv + sx * dn, uy * du + vy * dv + sy * dn, uz * du + vz * dv + sz * dn, reach)) open++;+ }+ }+ values[texel] = open / total;+ });+ // A little blur, among texels on a surface: it takes out the turned pattern, and what this map holds has no edges to lose.+ const blurred = new Float32Array(size * size).fill(-1);+ const covered = new Uint8Array(size * size);+ for (let y = 0; y < size; y++) {+ for (let x = 0; x < size; x++) {+ const texel = y * size + x;+ if (values[texel]! < 0) continue;+ covered[texel] = 1;+ let sum = 0, n = 0;+ for (let dy = -1; dy <= 1; dy++) {+ for (let dx = -1; dx <= 1; dx++) {+ const [qx, qy] = [x + dx, y + dy];+ if (qx < 0 || qy < 0 || qx >= size || qy >= size) continue;+ const v = values[qy * size + qx]!;+ if (v >= 0) { sum += v; n++; }+ }+ }+ blurred[texel] = sum / n;+ }+ }+ const rgba = new Uint8Array(size * size * 4).fill(255);+ for (let i = 0; i < size * size; i++) {+ if (blurred[i]! < 0) continue;+ const v = Math.round(blurred[i]! * 255);+ rgba[i * 4] = v; rgba[i * 4 + 1] = v; rgba[i * 4 + 2] = v;+ }+ dilate([rgba], covered, size);+ return { size, rgba };+}+/** Spreads each chart's edge outward through every image, nearest first. */function dilate(images: readonly Uint8Array[], covered: Uint8Array, size: number): void {let edge: number[] = [];diff --git a/pipeline/tests/modeling-surface.test.ts b/pipeline/tests/modeling-surface.test.tsindex 9582242de..5ae0c0a96 100644--- a/pipeline/tests/modeling-surface.test.ts+++ b/pipeline/tests/modeling-surface.test.ts@@ -96,6 +96,74 @@ describe('normalBake surface', () => {});});+describe('normalBake occlusion', () => {+ /** The occlusion map at the point of the thin mesh nearest @p want. */+ function at(m: BakedModel, map: { size: number; rgba: Uint8Array }, want: readonly number[]): number {+ const { data } = m;+ const offset = (semantic: number) => data.channels.find((c) => c.semantic === semantic)!.offset;+ const view = new DataView(data.vertices.buffer, data.vertices.byteOffset, data.vertices.byteLength);+ let best = Infinity, u = 0, v = 0;+ for (let t = 0; t < data.indices.length / 3; t++) {+ const mean = (o: number, k: number): number => [0, 1, 2].reduce((sum, c) => sum + view.getFloat32(data.indices[t * 3 + c]! * data.vertexStride + o + k * 4, true), 0) / 3;+ const d = Math.hypot(mean(offset(MeshChannel.Position), 0) - want[0]!, mean(offset(MeshChannel.Position), 1) - want[1]!, mean(offset(MeshChannel.Position), 2) - want[2]!);+ if (d < best) { best = d; u = mean(offset(MeshChannel.TexCoord0), 0); v = mean(offset(MeshChannel.TexCoord0), 1); }+ }+ return map.rgba[(Math.floor((1 - v) * map.size) * map.size + Math.floor(u * map.size)) * 4]!;+ }++ it('is white over a ball, which nothing shades, and on any open flat', () => {+ const m = bakeModel({ version: 1, normalBake: { size: 256, occlusion: true }, steps: [ball, thin] });+ const { occlusion } = bakeNormalMap(m);+ expect(occlusion!.size).toBe(128);+ expect(at(m, occlusion!, [0, 1, 0])).toBeGreaterThan(245);+ expect(at(m, occlusion!, [1, 0, 0])).toBeGreaterThan(245);+ });++ it('is dark at the bottom of a deep slot, and light on the open face beside it', () => {+ const m = bakeModel({ version: 1, normalBake: { size: 512, from: 'form', occlusion: { reach: 1, samples: 64 } }, steps: [+ { id: 'form', op: 'organic', resolution: 80, blend: 0.01, parts: [+ { shape: 'box', size: [3, 2, 2] }, { shape: 'box', size: [0.3, 3, 1.4], at: [0, 0, 0.75], subtract: true, blend: 0.01 },+ ] },+ ] });+ const { occlusion } = bakeNormalMap(m);+ const inSlot = at(m, occlusion!, [0, 0, 0.05]);+ const open = at(m, occlusion!, [1, 0, 1]);+ expect(open).toBeGreaterThan(200);+ expect(inSlot).toBeLessThan(open * 0.6);+ }, 60000);++ it('sees where two parts meet, and not one far away', () => {+ const bake = (gap: number) => {+ const m = bakeModel({ version: 1, normalBake: { size: 256, from: 'high', occlusion: { reach: 0.5, samples: 36, size: 128 } }, steps: [+ { op: 'box', size: [4, 0.2, 4], at: [0, -0.1, 0], tag: 'floor' }, { op: 'sphere', radius: 0.5, rings: 24, sides: 48, at: [0, 0.5 + gap, 0], tag: 'ball' }, { id: 'high', op: 'tag', faces: 'floor', tag: 'floor' },+ ] });+ return at(m, bakeNormalMap(m).occlusion!, [0.2, 0, 0.1]);+ };+ expect(bake(0)).toBeLessThan(bake(3) * 0.75);+ }, 60000);++ it('fills past the charts\' edges, and is the same every time', () => {+ const source = { version: 1, normalBake: { size: 256, occlusion: { samples: 9 } }, steps: [ball, thin] };+ const a = bakeNormalMap(bakeModel(source)).occlusion!;+ const b = bakeNormalMap(bakeModel(source)).occlusion!;+ expect(a.rgba).toEqual(b.rgba);+ expect(a.rgba.every((_, i) => i % 4 !== 3 || a.rgba[i] === 255)).toBe(true);+ });++ it('is not worked out unless asked for', () => {+ expect(bakeNormalMap(bakeModel({ version: 1, normalBake: { size: 64 }, steps: [ball, thin] })).occlusion).toBeUndefined();+ });++ it('refuses what cannot be meant', () => {+ const steps = [ball, thin];+ expect(() => validateModelSource({ steps, normalBake: { occlusion: 1 } })).toThrow(/occlusion must be true, false or/);+ expect(() => validateModelSource({ steps, normalBake: { occlusion: { rays: 9 } } })).toThrow(/unknown field "rays"/);+ expect(() => validateModelSource({ steps, normalBake: { occlusion: { samples: 0 } } })).toThrow(/samples must be 1..256/);+ expect(() => validateModelSource({ steps, normalBake: { occlusion: { size: 100 } } })).toThrow(/power of two/);+ expect(() => validateModelSource({ steps, normalBake: { occlusion: { strength: 2 } } })).toThrow(/strength must be 0..1/);+ });+});+describe('the material a model writes', () => {it('is none where a MeshRenderer says it all', () => {expect(modelMaterialDoc(validateModelSource({ steps: [ball] }), true, {})).toBeNull();@@ -113,6 +181,11 @@ describe('the material a model writes', () => {expect(doc.properties).toEqual({ u_metallicRoughnessMap: 'a_surface.png', u_roughness: 1, u_metallic: 1 });});+ it('names an occlusion map and its strength, with constants left as they were', () => {+ const doc = modelMaterialDoc(validateModelSource({ steps: [ball, thin], normalBake: { occlusion: { strength: 0.5 } } }), true, { occlusion: 'a_occlusion.png' })!;+ expect(doc.properties).toEqual({ u_occlusionMap: 'a_occlusion.png', u_occlusionStrength: 0.5, u_roughness: 1, u_metallic: 0 });+ });+it('culls as the surface allows', () => {const source = validateModelSource({ steps: [ball], material: { metallic: 1 } });expect(modelMaterialDoc(source, true, {})!.cull).not.toBe(modelMaterialDoc(source, false, {})!.cull);diff --git a/tools/releaseNotes.mjs b/tools/releaseNotes.mjsindex 5723cbc02..b3dda4b62 100644--- a/tools/releaseNotes.mjs+++ b/tools/releaseNotes.mjs@@ -3882,6 +3882,10 @@ export const NOTED = {{"note":"A rotation given to an agent as a quaternion is that rotation."},"fix(asset): a map a material names keeps its import settings in Play, where it was decoded as colour":{"note":"A material's normal and metal-rough maps are read as data in the editor's Play."},+ "feat(modeling): a bake works out how much sky a dense surface sees, and shades its creases by it":+ {"note":"An AI building a model can shade its creases."},+ "feat(mcp): model_write bakes an occlusion map, and a check draws a groove with and without it":+ {"note":"An AI building a model can shade its creases."},"feat(modeling): a model says how rough and how metal it is, part by part, and a bake keeps that as a map":{"note":"An AI building a model can say what it is made of."},"feat(mcp): model_write writes a material and a surface map, and a check draws the map against its constant":