Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Draw reversed-Z camera-relative so far flush pairs lose less depth (part of #310)

Nguyen Xuan An664c36c8e0e8
diff --git a/bench/reference/README.md b/bench/reference/README.md
index 902d3987..8730d375 100644
--- a/bench/reference/README.md
+++ b/bench/reference/README.md
@@ -1603,19 +1603,21 @@ Fix: `harness.js` renders the reference frame with `wind: 0`. Re-run with 67js o
`installReversedDepth(renderer, mode)` (`src/reversed-depth.js`, an addon: the renderer core carries none of it, so a scene that does not install it pays nothing in size) makes the colour pass draw with reversed-Z where `EXT_clip_control` exists. `mode` is `true` (on at once), `false` (off) or `'auto'`, which switches it on, once, at the start of a frame, when that frame's far / near passes 10,000, or when `renderer.requestReversedDepth()` is called; `installDepthPrecision(renderer, { reversedDepth })` installs it together with the coplanar warning and calls that request when the coplanar check finds a flush pair more than 40 units from the eye (a 24-bit buffer with a 0.1 near plane cannot separate a 1 mm gap much beyond that). Switching on respecifies the two depth stores as float and adds one stage that turns the resolved depth back to standard depth for the passes that read it. What it does: clip depth 0 to 1 with the near plane at 1, depth cleared to 0, `GREATER`, and a `DEPTH_COMPONENT32F` multisample buffer and scene depth texture. Everything else keeps the standard path: shadow maps (the sun, cascades and the local atlas), the sky and environment backgrounds and the post passes switch clip control back before they draw, so their projections do not change. The reversed projection is the standard one with its third row replaced by `(w - z) / 2` (`src/reversed-depth.js`), used only by the draw (`renderer.rasterProjection`, `rasterViewProjection`); culling, fog, inverse matrices and the CPU rasterizer keep the standard matrices. The passes that read the depth texture (ambient occlusion, depth of field and the other camera effects, volume fog, line art outlines and lines) are unchanged: a stage right after the resolve copies the depth and writes `1 - depth` back with `gl_FragDepth`, which is exactly the standard 0 to 1 value. The water mirror, render passes (their depth pass is eye distance, not buffer depth), liquid and foam, blended parts and Gaussian splats run inside the reversed window. A decal's polygon offset flips sign with it. `installReversedDepth(renderer, false)`, benchmark mode and a browser without the extension keep the old path (`renderer.reversedDepth` and `renderer.capabilities.reversedDepth` say which one is in use).
-Measured by `node bench/reference/depth-precision.mjs [gpu|cpu]` (scene in `depth-precision.js`): two 2.2-times-distance-wide planes at 1, 50 and 500 m, the near one `gap` closer (0, 0.1 and 1 mm), seen at about 30 degrees from their normal with the default near 0.1 and far 600, 8 camera positions about 1 to 10 cm apart per case. The number is the share of the red or blue pixels where the far (blue) plane wins, mean of the 8; 0 is right. Before is the renderer as it is, after is `installReversedDepth(renderer, true)`, same build. GPU is the RTX 3060 through Chrome (ANGLE), where the extension is present; the run was made without the GPU queue lock because it was busy and takes seconds, and is to be repeated through `gpu-lock.sh`.
+Measured by `node bench/reference/depth-precision.mjs [gpu|cpu]` (scene in `depth-precision.js`): two 2.2-times-distance-wide planes at 1, 50 and 500 m, the near one `gap` closer (0, 0.1 and 1 mm), seen at about 30 degrees from their normal with the default near 0.1 and far 600, 8 camera positions about 1 to 10 cm apart per case. The number is the share of the red or blue pixels where the far (blue) plane wins, mean of the 8; 0 is right. Before is the renderer as it is, "reversed, float" is `installReversedDepth(renderer, true)` without the camera-relative transform below (the first version of the addon), "camera-relative" is the addon as it is now, same build. GPU is the RTX 3060 through Chrome (ANGLE), where the extension is present; the camera-relative column was run through `gpu-lock.sh`.
-| Distance | Gap | Before (24-bit) | After (reversed, float) |
-| --- | --- | --- | --- |
-| 1 m | 0, 0.1, 1 mm | 0, 0, 0 | 0, 0, 0 |
-| 50 m | 0 mm | 0 | 0 |
-| 50 m | 0.1 mm | 0.150 | 0.009 |
-| 50 m | 1 mm | 0.079 | 0.000 |
-| 500 m | 0 mm | 0 | 0 |
-| 500 m | 0.1 mm | 0.190 | 0.134 |
-| 500 m | 1 mm | 0.188 | 0.015 |
-
-The gap-0 pair is two identical boxes, so every pixel ties and the first one drawn keeps it; it never flickers, and a coplanar pair of different size is what `decal` and `resolveCoplanar` are for. The wrong share falls to 1 percent or less at 50 m (0.1 and 1 mm) and at 500 m with a 1 mm gap, but not to zero: at 500 m the 0.1 mm case is still wrong on 13 percent of pixels, because a Float32 world position has a step of 0.03 mm there and the vertex transform adds more, which no depth buffer format removes. The next step, not done here, is a camera-relative vertex transform (the view translation applied to the instance matrix in double precision before the projection). SwiftShader also has the extension (the same page ran there). Its numbers are noisier and are not a precision measure: before 0.20 and 0.10 (50 m, 0.1 and 1 mm), 0.10 and 0.41 (500 m); after 0, 0.16, 0 and 0, with whole frames wrong in some jitters at 50 m and 1 mm. Both sets are 8 frames per case and move with the camera positions.
+| Distance | Gap | Before (24-bit) | Reversed, float | Reversed, camera-relative |
+| --- | --- | --- | --- | --- |
+| 1 m | 0, 0.1, 1 mm | 0, 0, 0 | 0, 0, 0 | 0, 0, 0 |
+| 50 m | 0 mm | 0 | 0 | 0 |
+| 50 m | 0.1 mm | 0.150 | 0.009 | 0.009 |
+| 50 m | 1 mm | 0.079 | 0.000 | 0.000 |
+| 500 m | 0 mm | 0 | 0 | 0 |
+| 500 m | 0.1 mm | 0.190 | 0.134 | 0.069 |
+| 500 m | 1 mm | 0.188 | 0.015 | 0.014 |
+
+The gap-0 pair is two identical boxes, so every pixel ties and the first one drawn keeps it; it never flickers, and a coplanar pair of different size is what `decal` and `resolveCoplanar` are for. The wrong share falls to 1 percent or less at 50 m (0.1 and 1 mm) and at 500 m with a 1 mm gap, but not to zero: at 500 m the 0.1 mm case was wrong on 13 percent of pixels with reversed-Z alone.
+
+**Camera-relative transform (2026-11-04, issue #310).** The vertex shader multiplied `viewProjection` by the world position, a sum of terms as large as the distance to the eye, so the view translation and the position cancelled in float32. When the addon switches on, `installReversedDepth` compiles the main vertex shader again through `shaderTransforms` (`src/shader-registry.js`) with `gl_Position = viewProjection * vec4(world.xyz - cameraOffset.xyz, 1.0 - cameraOffset.w) + cameraShift`: the eye is subtracted first, only the matrix's linear part multiplies it (w = 0), and the projection's own translation (`cameraShift`, its fourth column) is added. The 4 x 4 matrix stays whole, because the impostor cards and the water refraction read the eye or a world position through it; lighting, fog and shadows keep the absolute `worldPosition`; both uniforms are zero outside the reversed window, where the line is the old product. The renderer core is unchanged, so a scene without the addon draws the same bytes. The 500 m, 0.1 mm pair falls from 0.134 to 0.069 (an earlier trial that baked the camera offset into the planes' matrices in double precision, with the stock shader, gave the same 0.069, so the transform removes what it can); every other case is unchanged. What is left is the float32 step of the planes' own vertices: the matrices are Float32Array and the vertices sit 500 to 1000 m from the box centre, so each is rounded to 0.03 to 0.06 mm, a third to a half of the 0.1 mm gap, and no vertex transform removes that without subdividing the geometry or keeping a double-precision scene. SwiftShader has the extension too (the same page ran there), but this machine's SwiftShader build draws the standard path almost exactly (before 0 to 0.001 in all nine cases), so it is not a precision measure: after is 0 except 0.157 at 50 m with a 1 mm gap, where whole frames are wrong in some jitters, the same number on the base branch before this change.
Pictures: `glass`, `brick-flat`, `outdoor-hdri` and `mountain-valley` drawn by SwiftShader with `render-gpu.mjs --cpu` (standard depth, then reversed through the installed addon), with and without `--reversed-depth`: the first three are byte for byte the same; `mountain-valley` differs on 0.055 percent of its pixels by more than 2 of 255 (largest step 31), the edges of far trees where the depth test breaks a tie differently. With `--ambient-occlusion 1` (which reads the depth texture) `brick-flat` is identical and `mountain-valley` differs on 0.13 percent, largest step 28. The GPU renders of the same four scenes are queued behind a path-tracer job and are not in these numbers.
diff --git a/changes/310-camera-relative.md b/changes/310-camera-relative.md
new file mode 100644
index 00000000..64c36953
--- /dev/null
+++ b/changes/310-camera-relative.md
@@ -0,0 +1 @@
+- Make reversed-Z depth camera-relative (issue #310). `installReversedDepth` now compiles the main vertex shader with the eye subtracted from the world position before the view-projection multiply, so large world coordinates no longer meet the view translation in float32; the renderer core is unchanged and a scene without the addon draws byte for byte as before. On the depth-precision bench (GPU) the 500 m, 0.1 mm gap pair falls from 0.134 to 0.069 wrong pixels and the 500 m, 1 mm pair from 0.015 to 0.014; the rest of the 0.069 is the float32 step of the model's own vertices at that distance, which no vertex transform removes.
diff --git a/changes/310-camera-relative.size.md b/changes/310-camera-relative.size.md
new file mode 100644
index 00000000..58a802a9
--- /dev/null
+++ b/changes/310-camera-relative.size.md
@@ -0,0 +1 @@
+- entry:reversed-depth: the camera-relative vertex transform (a patched main vertex shader line, two uniforms and their per-frame upload) halves the wrong pixels of the 500 m, 0.1 mm flush pair, 343 bytes; the renderer core and every scene bundle are unchanged
diff --git a/src/reversed-depth.js b/src/reversed-depth.js
index b4eed936..4fc6c874 100644
--- a/src/reversed-depth.js
+++ b/src/reversed-depth.js
@@ -9,8 +9,10 @@
import { mat4 } from 'gl-matrix';
import { installCoplanarDiagnostics } from './coplanar-overlap.js';
import { compileProgram, fullscreenVertex } from './gl-program.js';
+import { shaderTransforms } from './shader-registry.js';
export const FAR_TO_NEAR_LIMIT = 1e4;
+const ZERO_VECTOR = new Float32Array(4);
const DEFAULT_NEAR = 0.1;
/**
@@ -30,6 +32,21 @@ export function reverseProjectionDepth(out, projection) {
return out;
}
+// Camera-relative vertex transform. The main vertex shader multiplies viewProjection by the world position, which is a sum of terms
+// as large as the distance to the eye, so at 500 m a float32 step of 0.03 mm already hides a 0.1 mm gap. The patched line subtracts
+// the eye first and sends w = 0 through the matrix (its linear part only), then adds the projection's own translation. The matrix
+// stays whole for the passes that read the eye out of it, and with both uniforms at zero the line is the old product.
+const WORLD_CLIP_LINE = 'gl_Position = viewProjection * world;';
+const CAMERA_RELATIVE_LINE = 'gl_Position = viewProjection * vec4(world.xyz - cameraOffset.xyz, 1.0 - cameraOffset.w) + cameraShift;';
+const CAMERA_UNIFORMS = 'uniform mat4 lightProjection;\nuniform vec4 cameraOffset;\nuniform vec4 cameraShift;';
+let patchingMainProgram = false;
+let transformInstalled = false;
+
+function makeVertexCameraRelative(source) {
+ if (!patchingMainProgram || !source.includes(WORLD_CLIP_LINE)) return source;
+ return source.replace('uniform mat4 lightProjection;', CAMERA_UNIFORMS).replace(WORLD_CLIP_LINE, CAMERA_RELATIVE_LINE);
+}
+
/** Whether a frame's lens is stretched enough that a 24-bit depth buffer cannot tell close surfaces apart far away. */
export function exceedsFarToNearLimit(frame) {
return frame.far / (frame.near ?? DEFAULT_NEAR) > FAR_TO_NEAR_LIMIT;
@@ -142,10 +159,14 @@ export function installReversedDepth(renderer, mode = 'auto') {
};
const rasterProjection = mat4.create();
const rasterViewProjection = mat4.create();
+ const cameraOffset = new Float32Array([0, 0, 0, 1]);
+ const cameraShift = new Float32Array(4);
let clipControl = null;
let requested = mode === true;
let inFrame = false;
let reversed = false;
+ let frameEye = null;
+ let cameraRelativeProgram = null;
// Back to the standard range before the backgrounds, the post passes and the shadow maps draw.
function endReversed() {
@@ -155,11 +176,29 @@ export function installReversedDepth(renderer, mode = 'auto') {
clipControl.active = false;
}
+ // The main program is in use. A program that a new shader feature builds starts with both uniforms at zero.
+ function setCameraRelative(offset, shift) {
+ context.uniform4fv(context.getUniformLocation(renderer.program, 'cameraOffset'), offset);
+ context.uniform4fv(context.getUniformLocation(renderer.program, 'cameraShift'), shift);
+ cameraRelativeProgram = renderer.program;
+ }
+
+ // The next frame, or a pass that draws with the standard matrix, expects the camera offset to be off.
+ function clearCameraRelative() {
+ if (cameraRelativeProgram !== renderer.program) return;
+ context.useProgram(renderer.program);
+ setCameraRelative(ZERO_VECTOR, ZERO_VECTOR);
+ cameraRelativeProgram = null;
+ }
+
// Called with the main program in use, after its standard matrix is set: swap in the reversed one and clear to the far value.
function beginReversed() {
reverseProjectionDepth(rasterProjection, renderer.projection);
mat4.multiply(rasterViewProjection, rasterProjection, renderer.view);
context.uniformMatrix4fv(renderer.uniforms.viewProjection, false, rasterViewProjection);
+ cameraOffset.set(frameEye);
+ cameraShift.set(rasterProjection.subarray(12, 16));
+ setCameraRelative(cameraOffset, cameraShift);
context.clearDepth(0);
context.clear(context.DEPTH_BUFFER_BIT);
context.clearDepth(1);
@@ -168,6 +207,25 @@ export function installReversedDepth(renderer, mode = 'auto') {
clipControl.active = true;
}
+ // Compiles the main program again with the patched line, now and whenever a new shader feature rebuilds it.
+ function useCameraRelativeProgram() {
+ if (!renderer.shaderFeatures) return;
+ if (!transformInstalled) {
+ shaderTransforms.push(makeVertexCameraRelative);
+ transformInstalled = true;
+ }
+ const startMainProgram = renderer.startMainProgram;
+ renderer.startMainProgram = function startCameraRelativeMainProgram() {
+ patchingMainProgram = true;
+ try {
+ return startMainProgram.call(this);
+ } finally {
+ patchingMainProgram = false;
+ }
+ };
+ renderer.rebuildMainProgram();
+ }
+
function enable() {
const extension = renderer.benchmark ? null : context.getExtension?.('EXT_clip_control');
requested = false;
@@ -200,6 +258,7 @@ export function installReversedDepth(renderer, mode = 'auto') {
};
renderer.postStages.unshift(createDepthConversionStage());
useFloatDepthTargets(renderer);
+ useCameraRelativeProgram();
return true;
}
@@ -209,12 +268,16 @@ export function installReversedDepth(renderer, mode = 'auto') {
renderer.render = function renderReversed(frame) {
if (!reversed && renderer.reversedDepth === undefined && (requested || exceedsFarToNearLimit(frame))) enable();
if (!reversed) return original.render.call(this, frame);
+ const outerEye = frameEye;
+ frameEye = frame.eye;
inFrame = true;
try {
return original.render.call(this, frame);
} finally {
inFrame = false;
+ frameEye = outerEye;
endReversed();
+ clearCameraRelative();
}
};
if (requested) enable();
diff --git a/tests/reversed-depth.test.js b/tests/reversed-depth.test.js
index 6dc3d64b..6923e744 100644
--- a/tests/reversed-depth.test.js
+++ b/tests/reversed-depth.test.js
@@ -102,3 +102,25 @@ test('a flush pair far from the eye asks for reversed depth, a near one does not
assert.equal(hasFarOverlap(pairAt(-100), view), true);
assert.equal(hasFarOverlap([], view), false);
});
+
+test('the main vertex shader is compiled camera-relative only through the addon, and only that program', async () => {
+ const { createVertexShader, SHADER_FEATURES } = await import('../src/shaders.js');
+ const { shaderTransforms } = await import('../src/shader-registry.js');
+ const compile = source => shaderTransforms.reduce((text, transform) => transform(text), source);
+ const mainSource = createVertexShader(SHADER_FEATURES.filter(feature => !['crowd', 'skin', 'morph'].includes(feature)));
+ assert.equal(compile(mainSource), mainSource);
+
+ const { renderer } = fakeRenderer();
+ const compiled = [];
+ renderer.shaderFeatures = new Set();
+ renderer.startMainProgram = () => compiled.push(compile(mainSource));
+ renderer.rebuildMainProgram = function rebuild() {
+ this.startMainProgram();
+ };
+ installReversedDepth(renderer, true);
+ const [cameraRelative] = compiled;
+ assert.match(cameraRelative, /uniform vec4 cameraOffset;/);
+ assert.match(cameraRelative, /gl_Position = viewProjection \* vec4\(world\.xyz - cameraOffset\.xyz, 1\.0 - cameraOffset\.w\) \+ cameraShift;/);
+ assert.doesNotMatch(cameraRelative, /gl_Position = viewProjection \* world;/);
+ assert.equal(compile(mainSource), mainSource);
+});