3D engine that AI agents drive
Pin the flush pass on glTF asset parts and the rank separation on both depth formats (#751)
Nguyen Xuan Anf17c51bf07c3
diff --git a/changes/751-asset-and-separation.md b/changes/751-asset-and-separation.mdnew file mode 100644index 00000000..7c773207--- /dev/null+++ b/changes/751-asset-and-separation.md@@ -0,0 +1 @@+- A recipe that places a glTF asset now loads the flush code on its own, so asset facades a few millimetres apart are cut or ranked in hosts that did not call `loadAllAgentSceneModules()`; tests pin both on asset parts and show that 24-bit and float depth separate all 16 ranks from 1 m to 1 km (#751).diff --git a/docs/learn/contact-detection.md b/docs/learn/contact-detection.mdindex e57182f9..8bd5c73c 100644--- a/docs/learn/contact-detection.md+++ b/docs/learn/contact-detection.md@@ -182,11 +182,11 @@ new ResizeObserver(() => {build(false);```-Both parts draw the plane `y = 0.1`, so the patch flickers against the slab. The diagnostic names the smaller part and suggests the fix. `decal: true` puts the part on layer 1 (`depthLayer` 0 to 4 orders stacked decals) and draws it with a scoped polygon offset in the colour pass and the render passes, so it never moves geometry. In an agent scene recipe, `environment.resolveCoplanar: 'offset'` biases the smaller part of every flagged pair in the same way.+Both parts draw the plane `y = 0.1`, so the patch flickers against the slab. The diagnostic names the smaller part and suggests the fix. `decal: true` puts the part on layer 1 (`depthLayer` 0 to 4 orders stacked decals) and draws it with a scoped polygon offset in the colour pass and the render passes, so it never moves geometry. An agent scene ranks flush faces by default (see the flush pass below), and `environment.resolveCoplanar: false` opts out. Skinned and morphed parts are skipped, because their triangles move.Use a real offset along the normal when the gap should be visible in exports or the path tracer.-The coplanar code is not in the core bundle. Call `installCoplanarDiagnostics(renderer)` from `67js/addons/coplanar-overlap` to add the warning to `renderer.diagnostics` (checked once per layout). To fix the flicker instead of naming it, pass `resolveCoplanar: resolveFlushParts` (same addon) to `new Renderer67`: it ranks flush and near-flush triangles, inside one part and across parts, and draws each rank with a small depth-only bias, so the face in front wins at every distance; ranked pairs leave the warning. Where an opaque face covers another on the same plane, or in front of it by at most a thousandth of the side of the shared area (never less than a millimetre or more than a centimetre), and they share 1 cm2 or more, the covered part is cut away instead. Seen at a grazing angle, that cut can open a strip no wider than the gap, which does not show, and it settles city models that nest facades of stacked blocks a few millimetres apart deeper than the ranks go: the part keeps its geometry, and the cut copy it draws with sits beside it (`flushOrderOf(part).geometry` in `depth-layer.js`), which the path tracer reads too. A face under a blended or masked face, and a double-sided face under a one-sided one, is ranked and not cut. Agent scenes do this by default wherever the code is loaded (the MCP preview always has it; elsewhere call `loadAllAgentSceneModules()`), and `environment.resolveCoplanar: false` opts out.+The coplanar code is not in the core bundle. Call `installCoplanarDiagnostics(renderer)` from `67js/addons/coplanar-overlap` to add the warning to `renderer.diagnostics` (checked once per layout). To fix the flicker instead of naming it, pass `resolveCoplanar: resolveFlushParts` (same addon) to `new Renderer67`: it ranks flush and near-flush triangles, inside one part and across parts, and draws each rank with a small depth-only bias, so the face in front wins at every distance; ranked pairs leave the warning. Where an opaque face covers another on the same plane, or in front of it by at most a thousandth of the side of the shared area (never less than a millimetre or more than a centimetre), and they share 1 cm2 or more, the covered part is cut away instead. Seen at a grazing angle, that cut can open a strip no wider than the gap, which does not show, and it settles city models that nest facades of stacked blocks a few millimetres apart deeper than the ranks go: the part keeps its geometry, and the cut copy it draws with sits beside it (`flushOrderOf(part).geometry` in `depth-layer.js`), which the path tracer reads too. A face under a blended or masked face, and a double-sided face under a one-sided one, is ranked and not cut. Agent scenes do this by default wherever the code is loaded (the MCP preview always has it, a recipe that places an asset or names `resolveCoplanar` loads it with `loadAgentSceneModules(recipe)`; for any other recipe call `loadAllAgentSceneModules()`), and `environment.resolveCoplanar: false` opts out. Skinned and morphed parts are skipped, because their triangles move.### Depth precision far from the cameradiff --git a/docs/test-checklist-vi.md b/docs/test-checklist-vi.mdindex fc3c9967..8a430aad 100644--- a/docs/test-checklist-vi.md+++ b/docs/test-checklist-vi.md@@ -91,11 +91,11 @@ Các recipe nằm ở cuối file (A1 đến A9). Mỗi recipe đã chạy qua `- Nhìn: test báo pass. Nạp một GLB dùng `KHR_materials_pbrSpecularGlossiness` bằng `scene_create_asset` thì `warnings` có dòng "converted to metallic-roughness".- Đạt: file nạp được, có warning và không báo lỗi. Trượt: nạp file bị throw.-## 12. Depth MSAA 16 bit (#387)+## 12. Depth MSAA 24 bit (#387)- Mở: `https://67js.codepawl.com/object.html`, hoặc `http://127.0.0.1:5173/object.html` khi chạy local. Mở Console và chạy `__studio.renderer.renderTargets.report()`.-- Nhìn: trong `targets`, mục `multisampleDepth` có `format` `DEPTH_COMPONENT16` và `samples` 4.-- Đạt: thấy `DEPTH_COMPONENT16` và ảnh bình thường. Trượt: `multisampleDepth` vẫn dùng `DEPTH_COMPONENT24`.+- Nhìn: trong `targets`, mục `multisampleDepth` có `format` `DEPTH_COMPONENT24` và `samples` 4.+- Đạt: thấy `DEPTH_COMPONENT24` và ảnh bình thường. Trượt: `multisampleDepth` dùng `DEPTH_COMPONENT16`.## Recipediff --git a/src/agent-scene-modules.js b/src/agent-scene-modules.jsindex 7bf097e1..a2e9971a 100644--- a/src/agent-scene-modules.js+++ b/src/agent-scene-modules.js@@ -82,8 +82,9 @@ const MODULE_MARKERS = {lineArt: ['"lines":{', '"toon":'],viewTransform: ['"viewTransform":"agx"', '"viewTransform":"neutral"', '"look":"'],// Flush faces are ranked whenever this is loaded (issue #751): the MCP preview registers it, loadAllAgentSceneModules loads it,- // and so does a recipe that names resolveCoplanar. A plain recipe in a host without it still needs no on-demand module.- coplanar: ['"resolveCoplanar"'],+ // and so does a recipe that names resolveCoplanar or places an asset, since glTF models nest facades a few millimetres apart.+ // A recipe of plain primitives in a host without it stays free of the 85 KB of flush code, so it needs no on-demand module.+ coplanar: ['"resolveCoplanar"', '"asset":{'],// A closed room a sun lights gets the floor sky trace by default. The recipe text can not say whether a room is closed, so a recipe that names its sun loads it and the module skips an open scene.floorSky: ['"keyColor"', '"keyIntensity"'],physicalSky: ['"type":"physical"'],diff --git a/src/flush-resolve.js b/src/flush-resolve.jsindex 0203b720..0b0d9d39 100644--- a/src/flush-resolve.js+++ b/src/flush-resolve.js@@ -13,7 +13,7 @@ const CACHED_ORDERS_PER_GEOMETRY = 8;// A face this close in front of another, facing the same way, is cut like a flush one. Seen at a grazing angle, the cut// can open a strip no wider than the gap at the front face's edge: a thousandth of the shared face, never less than a// millimetre and never more than a centimetre, does not show. A wider gap, such as a sign standing off a wall, is ranked-// instead. Without the cut, assets built as stacked blocks nest facades a few millimetres apart deeper than the 15 ranks+// instead. Without the cut, assets built as stacked blocks nest facades a few millimetres apart deeper than the 16 ranks// (the Melbourne city model nests 17).const NEAR_FLUSH_CUT_GAP_SHARE = 1e-3;const NEAR_FLUSH_CUT_MINIMUM_GAP = 1e-3;diff --git a/tests/flush-asset.test.js b/tests/flush-asset.test.jsnew file mode 100644index 00000000..96ceeca6--- /dev/null+++ b/tests/flush-asset.test.js@@ -0,0 +1,116 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { parseGltf } from '../src/gltf.js';+import { createAgentScene } from '../src/agent-scene.js';+import { agentSceneModulesNeeded, loadAgentSceneModules } from '../src/agent-scene-modules.js';+import { assetCopyKey } from '../src/agent-scene-assets.js';+import { flushOrderOf } from '../src/depth-layer.js';++const ASSET_ID = 'ef'.repeat(32);+const FLOAT = 5126;+const UNSIGNED_SHORT = 5123;++// An upward-facing square of `size` metres at height `height`, as glTF vertex data.+function squareData(size, height) {+ const half = size / 2;+ return {+ positions: new Float32Array([-half, height, -half, -half, height, half, half, height, half, half, height, -half]),+ normals: new Float32Array([0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0]),+ indices: new Uint16Array([0, 1, 2, 0, 2, 3]),+ };+}++// A glb with one mesh node per square, so a floor and a plaque `gap` metres above it become two parts of one asset.+function glbOfSquares(squares) {+ const document = {+ asset: { version: '2.0' }, scene: 0, scenes: [{ nodes: squares.map((_, index) => index) }],+ nodes: squares.map((_, index) => ({ name: `Square${index}`, mesh: index })),+ meshes: [], materials: [{ pbrMetallicRoughness: { baseColorFactor: [0.5, 0.5, 0.5, 1] } }],+ accessors: [], bufferViews: [], buffers: [{ byteLength: 0 }],+ };+ const chunks = [];+ let length = 0;+ function append(data, componentType, type) {+ document.bufferViews.push({ buffer: 0, byteOffset: length, byteLength: data.byteLength });+ document.accessors.push({ bufferView: document.bufferViews.length - 1, componentType, count: data.length / (type === 'VEC3' ? 3 : 1), type });+ chunks.push([length, new Uint8Array(data.buffer)]);+ length += Math.ceil(data.byteLength / 4) * 4;+ return document.accessors.length - 1;+ }+ for (const { positions, normals, indices } of squares) {+ document.meshes.push({ primitives: [{ attributes: { POSITION: append(positions, FLOAT, 'VEC3'), NORMAL: append(normals, FLOAT, 'VEC3') }, indices: append(indices, UNSIGNED_SHORT, 'SCALAR'), material: 0 }] });+ }+ document.buffers[0].byteLength = length;+ const json = new TextEncoder().encode(JSON.stringify(document));+ const jsonLength = Math.ceil(json.length / 4) * 4;+ const glb = new ArrayBuffer(28 + jsonLength + length);+ const header = new DataView(glb);+ header.setUint32(0, 0x46546c67, true);+ header.setUint32(4, 2, true);+ header.setUint32(8, glb.byteLength, true);+ header.setUint32(12, jsonLength, true);+ header.setUint32(16, 0x4e4f534a, true);+ const bytes = new Uint8Array(glb);+ bytes.fill(32, 20, 20 + jsonLength);+ bytes.set(json, 20);+ header.setUint32(20 + jsonLength, length, true);+ header.setUint32(24 + jsonLength, 0x004e4942, true);+ for (const [start, chunk] of chunks) bytes.set(chunk, 28 + jsonLength + start);+ return glb;+}++function facadeAsset(gap) {+ return glbOfSquares([squareData(2, 0), squareData(0.5, gap)]);+}++function recipe() {+ return {+ format: '67js.scene', schemaVersion: 1,+ geometries: [], materials: [],+ nodes: [{ id: 'building', asset: { id: ASSET_ID } }],+ };+}++async function sceneWith(gap, environment) {+ const assets = new Map([[assetCopyKey(ASSET_ID, 0), await parseGltf(facadeAsset(gap))]]);+ const source = recipe();+ if (environment) source.environment = environment;+ await loadAgentSceneModules(source);+ return createAgentScene(source, {}, { assets });+}++import { resolveFlushParts } from "../src/coplanar-overlap.js";+function partWithOrder(scene) {+ return scene.parts.find(part => flushOrderOf(part));+}++test('a recipe that places an asset loads the flush code, one of plain primitives does not', () => {+ assert.ok(agentSceneModulesNeeded(recipe()).includes('coplanar'));+ assert.deepEqual(agentSceneModulesNeeded({ geometries: [{ id: 'cube', type: 'box' }], nodes: [{ id: 'part', geometry: 'cube' }] }), []);+});++test('a glTF plaque half a millimetre above its floor has the floor cut under it', async () => {+ const scene = await sceneWith(0.0005);+ const floor = partWithOrder(scene);+ assert.ok(floor, 'the flush pass handles an asset part');+ assert.equal(floor.name, 'building/0');+ assert.ok(flushOrderOf(floor).geometry, 'the hidden floor draws a cut copy');+ assert.equal(scene.parts[1].geometry.indices.length, 6, 'the plaque keeps its geometry');+});++test('a glTF plaque 1.5 mm above its floor is ranked above it and not cut', async () => {+ const scene = await sceneWith(0.0015);+ const plaque = partWithOrder(scene);+ assert.ok(plaque, 'the flush pass handles an asset part');+ assert.equal(plaque.name, 'building/1');+ const order = flushOrderOf(plaque);+ assert.equal(order.geometry, undefined);+ assert.equal(Math.max(...order.ranges.map(range => range.rank)), 1);+});++test('resolveCoplanar false leaves an asset alone and reports the pair instead', async () => {+ const scene = await sceneWith(0.0005, { resolveCoplanar: false });+ assert.equal(partWithOrder(scene), undefined);+ const kinds = (scene.inspect().diagnostics ?? []).map(entry => entry.kind);+ assert.ok(kinds.includes('coplanar-overlap'));+});diff --git a/tests/flush-depth-separation.test.js b/tests/flush-depth-separation.test.jsnew file mode 100644index 00000000..450dc832--- /dev/null+++ b/tests/flush-depth-separation.test.js@@ -0,0 +1,76 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { MAXIMUM_FLUSH_RANK } from '../src/depth-layer.js';+import { reverseProjectionDepth } from '../src/reversed-depth.js';++// The constants applyDepthRank sends to the shader (src/renderer.js), checked against the rank test that pins them.+const METRES_PER_RANK = 1e-4;+const RELATIVE_PER_RANK = 2 ** -19;+const CLIP_PER_RANK = 2 ** -22;+const DEPTH_BITS = 24;++const DISTANCES = [1, 10, 100, 1000];+const CAMERAS = [{ near: 0.1, far: 2000 }, { near: 1, far: 2000 }, { near: 0.05, far: 5000 }];++function perspective({ near, far }) {+ return [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, (far + near) / (near - far), -1, 0, 0, 2 * far * near / (near - far), 0];+}++// The vertex shader's lift for a point straight ahead at `distance`: clip space moves `shift` metres toward the eye, then+// the 24-bit path adds the clip-space step. Returns the stored depth in 0..1 (standard) or 1..0 (reversed).+function storedDepth(camera, distance, rank, reversed) {+ const standard = perspective(camera);+ const matrix = reversed ? reverseProjectionDepth(new Array(16).fill(0), standard) : standard;+ const clipDepth = matrix[10] * -distance + matrix[14];+ const clipW = distance;+ const clipDepthPerMetre = -matrix[10];+ const shift = rank * (METRES_PER_RANK + RELATIVE_PER_RANK * distance);+ const lifted = (clipDepth - shift * clipDepthPerMetre) / (clipW - shift);+ const clipStep = !reversed && rank ? -rank * CLIP_PER_RANK : 0;+ const depth = reversed ? lifted : (lifted + clipStep + 1) / 2;+ return reversed ? Math.fround(depth) : Math.round(depth * (2 ** DEPTH_BITS - 1));+}++// How many metres toward the eye a rank moves the surface, read back through the unbiased projection.+function viewSeparation(camera, distance, rank, reversed) {+ const unbiased = storedDepth(camera, distance, 0, reversed);+ const biased = storedDepth(camera, distance, rank, reversed);+ const slope = (storedDepth(camera, distance + 1e-3, 0, reversed) - unbiased) / 1e-3;+ return (biased - unbiased) / -slope;+}++function everyRankDiffers(camera, distance, reversed) {+ const depths = Array.from({ length: MAXIMUM_FLUSH_RANK + 1 }, (_, rank) => storedDepth(camera, distance, rank, reversed));+ return depths.every((depth, rank) => rank === 0 || depth !== depths[rank - 1]);+}++test('on 24-bit and float depth every rank 1..15 stores a nearer depth than the rank below it, at 1 to 1000 m', () => {+ for (const camera of CAMERAS) {+ for (const distance of DISTANCES) {+ for (const reversed of [false, true]) {+ assert.ok(everyRankDiffers(camera, distance, reversed), `${reversed ? 'float' : '24-bit'} depth, ${distance} m, near ${camera.near}`);+ }+ }+ }+});++test('a rank moves float depth by the metric step and 24-bit depth by at least that step and two depth values', () => {+ for (const camera of CAMERAS) {+ for (const distance of DISTANCES) {+ const metric = METRES_PER_RANK + RELATIVE_PER_RANK * distance;+ const float = viewSeparation(camera, distance, 1, true);+ assert.ok(Math.abs(float - metric) / metric < 0.05, `float ${float} against ${metric} at ${distance} m`);+ const steps = viewSeparation(camera, distance, 1, false);+ assert.ok(steps >= metric * 0.98, `24-bit ${steps} against ${metric} at ${distance} m`);+ const depthStepsMoved = storedDepth(camera, distance, 0, false) - storedDepth(camera, distance, 1, false);+ assert.ok(depthStepsMoved >= 2, `24-bit moves ${depthStepsMoved} depth values per rank at ${distance} m`);+ }+ }+});++test('the accepted difference: far away a 24-bit rank is a depth-limited step, far wider than the float metric step', () => {+ const camera = { near: 0.1, far: 2000 };+ assert.ok(Math.abs(viewSeparation(camera, 1, 1, false) - viewSeparation(camera, 1, 1, true)) < 1e-5, 'the same at 1 m');+ assert.ok(viewSeparation(camera, 1000, 1, false) > 0.5, '24-bit cannot hold a step below its own depth resolution at 1 km');+ assert.ok(viewSeparation(camera, 1000, 1, true) < 0.003, 'float keeps the 2 mm metric step');+});