Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Pin the flush pass on glTF asset parts and the rank separation on both depth formats (#751)

Nguyen Xuan Anf17c51bf07c3
diff --git a/changes/751-asset-and-separation.md b/changes/751-asset-and-separation.md
new file mode 100644
index 00000000..7c773207
--- /dev/null
+++ b/changes/751-asset-and-separation.md
@@ -0,0 +1 @@
+- A recipe that places a glTF asset now loads the flush code on its own, so asset facades a few millimetres apart are cut or ranked in hosts that did not call `loadAllAgentSceneModules()`; tests pin both on asset parts and show that 24-bit and float depth separate all 16 ranks from 1 m to 1 km (#751).
diff --git a/docs/learn/contact-detection.md b/docs/learn/contact-detection.md
index e57182f9..8bd5c73c 100644
--- a/docs/learn/contact-detection.md
+++ b/docs/learn/contact-detection.md
@@ -182,11 +182,11 @@ new ResizeObserver(() => {
build(false);
```
-Both parts draw the plane `y = 0.1`, so the patch flickers against the slab. The diagnostic names the smaller part and suggests the fix. `decal: true` puts the part on layer 1 (`depthLayer` 0 to 4 orders stacked decals) and draws it with a scoped polygon offset in the colour pass and the render passes, so it never moves geometry. In an agent scene recipe, `environment.resolveCoplanar: 'offset'` biases the smaller part of every flagged pair in the same way.
+Both parts draw the plane `y = 0.1`, so the patch flickers against the slab. The diagnostic names the smaller part and suggests the fix. `decal: true` puts the part on layer 1 (`depthLayer` 0 to 4 orders stacked decals) and draws it with a scoped polygon offset in the colour pass and the render passes, so it never moves geometry. An agent scene ranks flush faces by default (see the flush pass below), and `environment.resolveCoplanar: false` opts out. Skinned and morphed parts are skipped, because their triangles move.
Use a real offset along the normal when the gap should be visible in exports or the path tracer.
-The coplanar code is not in the core bundle. Call `installCoplanarDiagnostics(renderer)` from `67js/addons/coplanar-overlap` to add the warning to `renderer.diagnostics` (checked once per layout). To fix the flicker instead of naming it, pass `resolveCoplanar: resolveFlushParts` (same addon) to `new Renderer67`: it ranks flush and near-flush triangles, inside one part and across parts, and draws each rank with a small depth-only bias, so the face in front wins at every distance; ranked pairs leave the warning. Where an opaque face covers another on the same plane, or in front of it by at most a thousandth of the side of the shared area (never less than a millimetre or more than a centimetre), and they share 1 cm2 or more, the covered part is cut away instead. Seen at a grazing angle, that cut can open a strip no wider than the gap, which does not show, and it settles city models that nest facades of stacked blocks a few millimetres apart deeper than the ranks go: the part keeps its geometry, and the cut copy it draws with sits beside it (`flushOrderOf(part).geometry` in `depth-layer.js`), which the path tracer reads too. A face under a blended or masked face, and a double-sided face under a one-sided one, is ranked and not cut. Agent scenes do this by default wherever the code is loaded (the MCP preview always has it; elsewhere call `loadAllAgentSceneModules()`), and `environment.resolveCoplanar: false` opts out.
+The coplanar code is not in the core bundle. Call `installCoplanarDiagnostics(renderer)` from `67js/addons/coplanar-overlap` to add the warning to `renderer.diagnostics` (checked once per layout). To fix the flicker instead of naming it, pass `resolveCoplanar: resolveFlushParts` (same addon) to `new Renderer67`: it ranks flush and near-flush triangles, inside one part and across parts, and draws each rank with a small depth-only bias, so the face in front wins at every distance; ranked pairs leave the warning. Where an opaque face covers another on the same plane, or in front of it by at most a thousandth of the side of the shared area (never less than a millimetre or more than a centimetre), and they share 1 cm2 or more, the covered part is cut away instead. Seen at a grazing angle, that cut can open a strip no wider than the gap, which does not show, and it settles city models that nest facades of stacked blocks a few millimetres apart deeper than the ranks go: the part keeps its geometry, and the cut copy it draws with sits beside it (`flushOrderOf(part).geometry` in `depth-layer.js`), which the path tracer reads too. A face under a blended or masked face, and a double-sided face under a one-sided one, is ranked and not cut. Agent scenes do this by default wherever the code is loaded (the MCP preview always has it, a recipe that places an asset or names `resolveCoplanar` loads it with `loadAgentSceneModules(recipe)`; for any other recipe call `loadAllAgentSceneModules()`), and `environment.resolveCoplanar: false` opts out. Skinned and morphed parts are skipped, because their triangles move.
### Depth precision far from the camera
diff --git a/docs/test-checklist-vi.md b/docs/test-checklist-vi.md
index fc3c9967..8a430aad 100644
--- a/docs/test-checklist-vi.md
+++ b/docs/test-checklist-vi.md
@@ -91,11 +91,11 @@ Các recipe nằm ở cuối file (A1 đến A9). Mỗi recipe đã chạy qua `
- Nhìn: test báo pass. Nạp một GLB dùng `KHR_materials_pbrSpecularGlossiness` bằng `scene_create_asset` thì `warnings` có dòng "converted to metallic-roughness".
- Đạt: file nạp được, có warning và không báo lỗi. Trượt: nạp file bị throw.
-## 12. Depth MSAA 16 bit (#387)
+## 12. Depth MSAA 24 bit (#387)
- Mở: `https://67js.codepawl.com/object.html`, hoặc `http://127.0.0.1:5173/object.html` khi chạy local. Mở Console và chạy `__studio.renderer.renderTargets.report()`.
-- Nhìn: trong `targets`, mục `multisampleDepth` có `format` `DEPTH_COMPONENT16` và `samples` 4.
-- Đạt: thấy `DEPTH_COMPONENT16` và ảnh bình thường. Trượt: `multisampleDepth` vẫn dùng `DEPTH_COMPONENT24`.
+- Nhìn: trong `targets`, mục `multisampleDepth` có `format` `DEPTH_COMPONENT24` và `samples` 4.
+- Đạt: thấy `DEPTH_COMPONENT24` và ảnh bình thường. Trượt: `multisampleDepth` dùng `DEPTH_COMPONENT16`.
## Recipe
diff --git a/src/agent-scene-modules.js b/src/agent-scene-modules.js
index 7bf097e1..a2e9971a 100644
--- a/src/agent-scene-modules.js
+++ b/src/agent-scene-modules.js
@@ -82,8 +82,9 @@ const MODULE_MARKERS = {
lineArt: ['"lines":{', '"toon":'],
viewTransform: ['"viewTransform":"agx"', '"viewTransform":"neutral"', '"look":"'],
// Flush faces are ranked whenever this is loaded (issue #751): the MCP preview registers it, loadAllAgentSceneModules loads it,
- // and so does a recipe that names resolveCoplanar. A plain recipe in a host without it still needs no on-demand module.
- coplanar: ['"resolveCoplanar"'],
+ // and so does a recipe that names resolveCoplanar or places an asset, since glTF models nest facades a few millimetres apart.
+ // A recipe of plain primitives in a host without it stays free of the 85 KB of flush code, so it needs no on-demand module.
+ coplanar: ['"resolveCoplanar"', '"asset":{'],
// A closed room a sun lights gets the floor sky trace by default. The recipe text can not say whether a room is closed, so a recipe that names its sun loads it and the module skips an open scene.
floorSky: ['"keyColor"', '"keyIntensity"'],
physicalSky: ['"type":"physical"'],
diff --git a/src/flush-resolve.js b/src/flush-resolve.js
index 0203b720..0b0d9d39 100644
--- a/src/flush-resolve.js
+++ b/src/flush-resolve.js
@@ -13,7 +13,7 @@ const CACHED_ORDERS_PER_GEOMETRY = 8;
// A face this close in front of another, facing the same way, is cut like a flush one. Seen at a grazing angle, the cut
// can open a strip no wider than the gap at the front face's edge: a thousandth of the shared face, never less than a
// millimetre and never more than a centimetre, does not show. A wider gap, such as a sign standing off a wall, is ranked
-// instead. Without the cut, assets built as stacked blocks nest facades a few millimetres apart deeper than the 15 ranks
+// instead. Without the cut, assets built as stacked blocks nest facades a few millimetres apart deeper than the 16 ranks
// (the Melbourne city model nests 17).
const NEAR_FLUSH_CUT_GAP_SHARE = 1e-3;
const NEAR_FLUSH_CUT_MINIMUM_GAP = 1e-3;
diff --git a/tests/flush-asset.test.js b/tests/flush-asset.test.js
new file mode 100644
index 00000000..96ceeca6
--- /dev/null
+++ b/tests/flush-asset.test.js
@@ -0,0 +1,116 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { parseGltf } from '../src/gltf.js';
+import { createAgentScene } from '../src/agent-scene.js';
+import { agentSceneModulesNeeded, loadAgentSceneModules } from '../src/agent-scene-modules.js';
+import { assetCopyKey } from '../src/agent-scene-assets.js';
+import { flushOrderOf } from '../src/depth-layer.js';
+
+const ASSET_ID = 'ef'.repeat(32);
+const FLOAT = 5126;
+const UNSIGNED_SHORT = 5123;
+
+// An upward-facing square of `size` metres at height `height`, as glTF vertex data.
+function squareData(size, height) {
+ const half = size / 2;
+ return {
+ positions: new Float32Array([-half, height, -half, -half, height, half, half, height, half, half, height, -half]),
+ normals: new Float32Array([0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0]),
+ indices: new Uint16Array([0, 1, 2, 0, 2, 3]),
+ };
+}
+
+// A glb with one mesh node per square, so a floor and a plaque `gap` metres above it become two parts of one asset.
+function glbOfSquares(squares) {
+ const document = {
+ asset: { version: '2.0' }, scene: 0, scenes: [{ nodes: squares.map((_, index) => index) }],
+ nodes: squares.map((_, index) => ({ name: `Square${index}`, mesh: index })),
+ meshes: [], materials: [{ pbrMetallicRoughness: { baseColorFactor: [0.5, 0.5, 0.5, 1] } }],
+ accessors: [], bufferViews: [], buffers: [{ byteLength: 0 }],
+ };
+ const chunks = [];
+ let length = 0;
+ function append(data, componentType, type) {
+ document.bufferViews.push({ buffer: 0, byteOffset: length, byteLength: data.byteLength });
+ document.accessors.push({ bufferView: document.bufferViews.length - 1, componentType, count: data.length / (type === 'VEC3' ? 3 : 1), type });
+ chunks.push([length, new Uint8Array(data.buffer)]);
+ length += Math.ceil(data.byteLength / 4) * 4;
+ return document.accessors.length - 1;
+ }
+ for (const { positions, normals, indices } of squares) {
+ document.meshes.push({ primitives: [{ attributes: { POSITION: append(positions, FLOAT, 'VEC3'), NORMAL: append(normals, FLOAT, 'VEC3') }, indices: append(indices, UNSIGNED_SHORT, 'SCALAR'), material: 0 }] });
+ }
+ document.buffers[0].byteLength = length;
+ const json = new TextEncoder().encode(JSON.stringify(document));
+ const jsonLength = Math.ceil(json.length / 4) * 4;
+ const glb = new ArrayBuffer(28 + jsonLength + length);
+ const header = new DataView(glb);
+ header.setUint32(0, 0x46546c67, true);
+ header.setUint32(4, 2, true);
+ header.setUint32(8, glb.byteLength, true);
+ header.setUint32(12, jsonLength, true);
+ header.setUint32(16, 0x4e4f534a, true);
+ const bytes = new Uint8Array(glb);
+ bytes.fill(32, 20, 20 + jsonLength);
+ bytes.set(json, 20);
+ header.setUint32(20 + jsonLength, length, true);
+ header.setUint32(24 + jsonLength, 0x004e4942, true);
+ for (const [start, chunk] of chunks) bytes.set(chunk, 28 + jsonLength + start);
+ return glb;
+}
+
+function facadeAsset(gap) {
+ return glbOfSquares([squareData(2, 0), squareData(0.5, gap)]);
+}
+
+function recipe() {
+ return {
+ format: '67js.scene', schemaVersion: 1,
+ geometries: [], materials: [],
+ nodes: [{ id: 'building', asset: { id: ASSET_ID } }],
+ };
+}
+
+async function sceneWith(gap, environment) {
+ const assets = new Map([[assetCopyKey(ASSET_ID, 0), await parseGltf(facadeAsset(gap))]]);
+ const source = recipe();
+ if (environment) source.environment = environment;
+ await loadAgentSceneModules(source);
+ return createAgentScene(source, {}, { assets });
+}
+
+import { resolveFlushParts } from "../src/coplanar-overlap.js";
+function partWithOrder(scene) {
+ return scene.parts.find(part => flushOrderOf(part));
+}
+
+test('a recipe that places an asset loads the flush code, one of plain primitives does not', () => {
+ assert.ok(agentSceneModulesNeeded(recipe()).includes('coplanar'));
+ assert.deepEqual(agentSceneModulesNeeded({ geometries: [{ id: 'cube', type: 'box' }], nodes: [{ id: 'part', geometry: 'cube' }] }), []);
+});
+
+test('a glTF plaque half a millimetre above its floor has the floor cut under it', async () => {
+ const scene = await sceneWith(0.0005);
+ const floor = partWithOrder(scene);
+ assert.ok(floor, 'the flush pass handles an asset part');
+ assert.equal(floor.name, 'building/0');
+ assert.ok(flushOrderOf(floor).geometry, 'the hidden floor draws a cut copy');
+ assert.equal(scene.parts[1].geometry.indices.length, 6, 'the plaque keeps its geometry');
+});
+
+test('a glTF plaque 1.5 mm above its floor is ranked above it and not cut', async () => {
+ const scene = await sceneWith(0.0015);
+ const plaque = partWithOrder(scene);
+ assert.ok(plaque, 'the flush pass handles an asset part');
+ assert.equal(plaque.name, 'building/1');
+ const order = flushOrderOf(plaque);
+ assert.equal(order.geometry, undefined);
+ assert.equal(Math.max(...order.ranges.map(range => range.rank)), 1);
+});
+
+test('resolveCoplanar false leaves an asset alone and reports the pair instead', async () => {
+ const scene = await sceneWith(0.0005, { resolveCoplanar: false });
+ assert.equal(partWithOrder(scene), undefined);
+ const kinds = (scene.inspect().diagnostics ?? []).map(entry => entry.kind);
+ assert.ok(kinds.includes('coplanar-overlap'));
+});
diff --git a/tests/flush-depth-separation.test.js b/tests/flush-depth-separation.test.js
new file mode 100644
index 00000000..450dc832
--- /dev/null
+++ b/tests/flush-depth-separation.test.js
@@ -0,0 +1,76 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { MAXIMUM_FLUSH_RANK } from '../src/depth-layer.js';
+import { reverseProjectionDepth } from '../src/reversed-depth.js';
+
+// The constants applyDepthRank sends to the shader (src/renderer.js), checked against the rank test that pins them.
+const METRES_PER_RANK = 1e-4;
+const RELATIVE_PER_RANK = 2 ** -19;
+const CLIP_PER_RANK = 2 ** -22;
+const DEPTH_BITS = 24;
+
+const DISTANCES = [1, 10, 100, 1000];
+const CAMERAS = [{ near: 0.1, far: 2000 }, { near: 1, far: 2000 }, { near: 0.05, far: 5000 }];
+
+function perspective({ near, far }) {
+ return [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, (far + near) / (near - far), -1, 0, 0, 2 * far * near / (near - far), 0];
+}
+
+// The vertex shader's lift for a point straight ahead at `distance`: clip space moves `shift` metres toward the eye, then
+// the 24-bit path adds the clip-space step. Returns the stored depth in 0..1 (standard) or 1..0 (reversed).
+function storedDepth(camera, distance, rank, reversed) {
+ const standard = perspective(camera);
+ const matrix = reversed ? reverseProjectionDepth(new Array(16).fill(0), standard) : standard;
+ const clipDepth = matrix[10] * -distance + matrix[14];
+ const clipW = distance;
+ const clipDepthPerMetre = -matrix[10];
+ const shift = rank * (METRES_PER_RANK + RELATIVE_PER_RANK * distance);
+ const lifted = (clipDepth - shift * clipDepthPerMetre) / (clipW - shift);
+ const clipStep = !reversed && rank ? -rank * CLIP_PER_RANK : 0;
+ const depth = reversed ? lifted : (lifted + clipStep + 1) / 2;
+ return reversed ? Math.fround(depth) : Math.round(depth * (2 ** DEPTH_BITS - 1));
+}
+
+// How many metres toward the eye a rank moves the surface, read back through the unbiased projection.
+function viewSeparation(camera, distance, rank, reversed) {
+ const unbiased = storedDepth(camera, distance, 0, reversed);
+ const biased = storedDepth(camera, distance, rank, reversed);
+ const slope = (storedDepth(camera, distance + 1e-3, 0, reversed) - unbiased) / 1e-3;
+ return (biased - unbiased) / -slope;
+}
+
+function everyRankDiffers(camera, distance, reversed) {
+ const depths = Array.from({ length: MAXIMUM_FLUSH_RANK + 1 }, (_, rank) => storedDepth(camera, distance, rank, reversed));
+ return depths.every((depth, rank) => rank === 0 || depth !== depths[rank - 1]);
+}
+
+test('on 24-bit and float depth every rank 1..15 stores a nearer depth than the rank below it, at 1 to 1000 m', () => {
+ for (const camera of CAMERAS) {
+ for (const distance of DISTANCES) {
+ for (const reversed of [false, true]) {
+ assert.ok(everyRankDiffers(camera, distance, reversed), `${reversed ? 'float' : '24-bit'} depth, ${distance} m, near ${camera.near}`);
+ }
+ }
+ }
+});
+
+test('a rank moves float depth by the metric step and 24-bit depth by at least that step and two depth values', () => {
+ for (const camera of CAMERAS) {
+ for (const distance of DISTANCES) {
+ const metric = METRES_PER_RANK + RELATIVE_PER_RANK * distance;
+ const float = viewSeparation(camera, distance, 1, true);
+ assert.ok(Math.abs(float - metric) / metric < 0.05, `float ${float} against ${metric} at ${distance} m`);
+ const steps = viewSeparation(camera, distance, 1, false);
+ assert.ok(steps >= metric * 0.98, `24-bit ${steps} against ${metric} at ${distance} m`);
+ const depthStepsMoved = storedDepth(camera, distance, 0, false) - storedDepth(camera, distance, 1, false);
+ assert.ok(depthStepsMoved >= 2, `24-bit moves ${depthStepsMoved} depth values per rank at ${distance} m`);
+ }
+ }
+});
+
+test('the accepted difference: far away a 24-bit rank is a depth-limited step, far wider than the float metric step', () => {
+ const camera = { near: 0.1, far: 2000 };
+ assert.ok(Math.abs(viewSeparation(camera, 1, 1, false) - viewSeparation(camera, 1, 1, true)) < 1e-5, 'the same at 1 m');
+ assert.ok(viewSeparation(camera, 1000, 1, false) > 0.5, '24-bit cannot hold a step below its own depth resolution at 1 km');
+ assert.ok(viewSeparation(camera, 1000, 1, true) < 0.003, 'float keeps the 2 mm metric step');
+});