Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Add MIS compensation for the Cook environment map sampling table, off by default (#895)

Nguyen Xuan Ana345ecddca97
diff --git a/bench/cycles-parity/trace-native.mjs b/bench/cycles-parity/trace-native.mjs
index 97bf55c2..20848e0d 100644
--- a/bench/cycles-parity/trace-native.mjs
+++ b/bench/cycles-parity/trace-native.mjs
@@ -96,7 +96,7 @@ function parseLightSamplingOption(argumentList) {
function parseTraceArguments(argumentList) {
if (argumentList.length < 2 || argumentList[0].startsWith('--') || argumentList[1].startsWith('--')) {
- throw new RangeError('Usage: node bench/cycles-parity/trace-native.mjs <sceneJsonPath> <outputPrefix> [--kernel legacy|cycles] [--samples 256] [--bounces 8] [--seed 1] [--width 960] [--height 540] [--sampler white|sobol-burley] [--light-sampling auto|tree|uniform] [--path-regeneration] [--filter-type box|gaussian|blackman-harris] [--filter-width 1.0] [--clamp-direct 0] [--clamp-indirect 0] [--firefly-clamp 1e9] [--noise-threshold 0]');
+ throw new RangeError('Usage: node bench/cycles-parity/trace-native.mjs <sceneJsonPath> <outputPrefix> [--kernel legacy|cycles] [--samples 256] [--bounces 8] [--seed 1] [--width 960] [--height 540] [--sampler white|sobol-burley] [--light-sampling auto|tree|uniform] [--path-regeneration] [--env-compensation 0..1] [--filter-type box|gaussian|blackman-harris] [--filter-width 1.0] [--clamp-direct 0] [--clamp-indirect 0] [--firefly-clamp 1e9] [--noise-threshold 0]');
}
return {
sceneJsonPath: resolve(argumentList[0]),
@@ -110,6 +110,7 @@ function parseTraceArguments(argumentList) {
filterType: parseFilterTypeOption(argumentList),
sampler: parseSamplerOption(argumentList),
lightSampling: parseLightSamplingOption(argumentList),
+ envCompensation: parseNumberOption(argumentList, '--env-compensation', 0, 0),
directLighting: String(readFlagValue(argumentList, '--direct-lighting', 'both')),
lightTreeProbe: argumentList.includes('--light-tree-probe'),
pathRegeneration: argumentList.includes('--path-regeneration'),
diff --git a/changes/895-env-compensation.md b/changes/895-env-compensation.md
new file mode 100644
index 00000000..2fbfb5a7
--- /dev/null
+++ b/changes/895-env-compensation.md
@@ -0,0 +1 @@
+- The Cook `cycles` kernel can sample the environment map with MIS compensation (`envCompensation: true` or a fraction from 0 to 1; `--env-compensation` in `trace-native.mjs`), which lowers the error under a sunny HDRI and is off by default (#895)
diff --git a/docs/cycles-parity.md b/docs/cycles-parity.md
index f2d77698..e0b15c1b 100644
--- a/docs/cycles-parity.md
+++ b/docs/cycles-parity.md
@@ -496,6 +496,18 @@ Time to equal error, same interpolation as above `[measured and interpolated]`:
- 4096 spp parity of the `cycles` kernel at its defaults, seven scenes, seed 1, on a Modal NVIDIA A10: 7 of 7 pass, with the same mean ratios, tile counts, FLIP LDR and relMSE as the table below to the digits shown (worst per-channel mean ratio 1.0035 on `lit-interior`, against the 2 percent limit). Cook sampling seconds at 4096 spp: `spot-area` 24.3, `light-linking` 27.2, `metal` 17.8, `glass` 30.1, `outdoor-hdri` 9.4, `lit-interior` 49.4, `many-lights-500` 205.7.
- Checks on the same Modal container: `cargo test` 25 passed, 0 failed (no GPU needed; the napi symbols are left unresolved at link time on Linux, and the one new test covers a tree flag without emitter tables); `node --test tests/native-*.test.js tests/cook-kernel-parity.test.js tests/cycles-parity-many-lights.test.js` 76 tests, 75 passed, 1 skipped, 0 failed; lobe probes 22 of 22 and sphere furnace scenes 6 of 6 at 512 spp.
+### MIS compensation of the environment map (`envCompensation`, off by default, #895)
+
+`envCompensation: true` (or a fraction from 0 to 1; `trace-native.mjs --env-compensation 1`) builds the environment sampling table after Karlik et al. 2019 (the same step as pbrt-v4 `ImageInfiniteLight`, Apache-2.0): the host subtracts the sphere-mean radiance from every texel's weight and clamps at zero, so light sampling spends its samples where BSDF sampling does badly, and the BSDF-hit side reads the same table for its MIS weight. The kernel is unchanged: it reads its probability from the table it samples, so the pdf used in MIS is the pdf that was sampled, and a dim texel that gets zero probability is reached by BSDF sampling with weight 1. A map that would end up all zero (a uniform map) keeps the plain table. Radiance stays unbiased while every non-delta lobe has a positive pdf in the dropped directions, which holds for all closures of the kernel. `[measured]` on an NVIDIA A10, 960 x 540, 8 bounces, seed 3 against the Cycles 4096-spp seed-1 reference, baseline and trial interleaved in one container, three rounds (the image is deterministic, so the three rounds differ only in time):
+
+| Scene | relMSE at 16 / 64 / 256 / 1024 spp, off | on | Error ratio | Mean ratio off / on |
+| --- | --- | --- | --- | --- |
+| `outdoor-hdri` | 0.001168 / 0.000198 / 0.000051 / 0.000010 | 0.000661 / 0.000109 / 0.000023 / 0.000007 | 1.77 / 1.81 / 2.18 / 1.36 | 0.99965 / 0.99965 |
+| `lit-interior` | 0.065292 / 0.012832 / 0.002741 / 0.000656 | same to 5 digits | 1.00 | 1.00334 / 1.00334 |
+| `metal` | 0.004500 / 0.280629 / 0.018116 / 0.008311 | 0.003695 / 0.280517 / 0.018077 / 0.008307 | 1.22 / 1.00 / 1.00 / 1.00 | 0.99650 / 0.99651 at 64 spp |
+
+The sampling time per sample is the same with the option on or off (ratio 0.90 to 1.03 over the rungs, inside the run to run spread, since only the table changes), so the error ratio at equal samples is also the ratio at equal time. At 1024 spp `outdoor-hdri` sits close to the reference's own noise floor (0.000007 at 4096 spp), which compresses the last ratio. The `metal` 64 spp value is one firefly-dominated image of the baseline and the trial alike. `lit-interior` has no sunny environment, so it does not change. At 4096 spp, seed 1, the seven scenes give the same mean ratios and relMSE with the option off as the table above and with it on (`glass` relMSE 0.000058 to 0.000056, `outdoor-hdri` 0.000005 both, the mean ratio of every scene equal to five digits). Tests: `tests/native-pack-v2.test.js` (default off, dim texels dropped with valid CDFs, uniform map fallback, option validation).
+
### Final gates for #895
Release addon built; on the RTX 3060 `cargo test --target x86_64-pc-windows-msvc`: 24 pass; `node --test tests/native-*.test.js`: 53 pass; lobe probes 22 of 22 and sphere furnace scenes 6 of 6 pass (512 spp); 4096 spp parity of the `cycles` kernel at its defaults on all seven scenes, 7 of 7 pass `[measured]`. The Modal run after the review fixes gives the same table (see above) with 25 cargo tests and 75 node tests passing:
diff --git a/packages/render-native/README.md b/packages/render-native/README.md
index 6cb1144a..1fb0b994 100644
--- a/packages/render-native/README.md
+++ b/packages/render-native/README.md
@@ -32,7 +32,7 @@ const { radiance, escaped } = traceNativeTexels(input, { count: texelCount, poin
`radiance` is the mean light that reaches each texel (three floats per texel) and `escaped` the share of first rays that left the scene. The agent runtime uses it for recipe bakes (`bake.backend: "native"`).
-Cook's `cycles` kernel samples with Cycles' Sobol-Burley blue-noise pattern (`sampler: 'sobol-burley'`, default; `'white'` keeps the earlier hash sampler, and `'pcg'` and `'sobol'` are other names for `'white'` and `'sobol-burley'`). It picks lights through the Cycles light tree over the analytic lights, a sun and the emissive triangles (`lightSampling: 'auto'`, default: tree from more than one emitter; `'tree'`; `'uniform'` for the flat pick, and `useLightTree: true|false` is another name for `'tree'|'uniform'`). Naming an option twice with different values is an error. The pixel filter is `box` unless `filterType` says otherwise. Details and measurements: [docs/cycles-parity.md](../../docs/cycles-parity.md), layout: [FORMAT-v2.md](./FORMAT-v2.md).
+Cook's `cycles` kernel samples with Cycles' Sobol-Burley blue-noise pattern (`sampler: 'sobol-burley'`, default; `'white'` keeps the earlier hash sampler, and `'pcg'` and `'sobol'` are other names for `'white'` and `'sobol-burley'`). It picks lights through the Cycles light tree over the analytic lights, a sun and the emissive triangles (`lightSampling: 'auto'`, default: tree from more than one emitter; `'tree'`; `'uniform'` for the flat pick, and `useLightTree: true|false` is another name for `'tree'|'uniform'`). Naming an option twice with different values is an error. `envCompensation: true` (a fraction from 0 to 1 also works; off by default) subtracts the mean radiance from the environment sampling table so light sampling covers only the bright part of the map, which lowers the error under a sunny HDRI. The pixel filter is `box` unless `filterType` says otherwise. Details and measurements: [docs/cycles-parity.md](../../docs/cycles-parity.md), layout: [FORMAT-v2.md](./FORMAT-v2.md).
## Dispatches
diff --git a/packages/render-native/pack-input-v2.mjs b/packages/render-native/pack-input-v2.mjs
index 90aa0ad2..d284cb4f 100644
--- a/packages/render-native/pack-input-v2.mjs
+++ b/packages/render-native/pack-input-v2.mjs
@@ -1330,6 +1330,65 @@ function packLightTree(packedLights, packedEmissives) {
};
}
+/**
+ * Reads the `envCompensation` option: `undefined`, `false` or `0` (off, the default), `true` (the full mean) or a
+ * fraction in (0, 1] of the mean radiance to subtract from the sampling table. Returns the fraction, 0 when off.
+ */
+export function resolveEnvCompensationFraction(envCompensation, callerName) {
+ if (envCompensation === undefined || envCompensation === false) {
+ return 0;
+ }
+ if (envCompensation === true) {
+ return 1;
+ }
+ if (typeof envCompensation !== 'number' || !Number.isFinite(envCompensation) || envCompensation < 0 || envCompensation > 1) {
+ throw new RangeError(
+ `${callerName}: \`envCompensation\` must be true, false or a number from 0 to 1 (the fraction of the mean radiance to subtract), got ${JSON.stringify(envCompensation)}.`
+ );
+ }
+ return envCompensation;
+}
+
+/**
+ * MIS compensation of the environment sampling table (Karlik et al. 2019; pbrt-v4 ImageInfiniteLight, Apache-2.0).
+ * Subtracts `fraction` times the sphere-mean radiance from every texel's weight and clamps at zero, so light sampling
+ * spends its samples where BSDF sampling does badly. The kernel reads the pdf from the same table, so the pdf used in
+ * MIS stays the pdf that was sampled. A map that would end up all zero keeps its original table.
+ * Rewrites `pixelWeights` and `rowTotals` in place and returns the new total.
+ */
+function compensateEnvironmentWeights(pixelWeights, rowTotals, width, height, fraction, originalTotal) {
+ const deltaPhi = (2 * Math.PI) / width;
+ const deltaTheta = Math.PI / height;
+ let solidAngleSum = 0;
+ for (let y = 0; y < height; y++) {
+ solidAngleSum += Math.sin(Math.PI * (y + 0.5) / height) * deltaPhi * deltaTheta * width;
+ }
+ const meanRadiance = originalTotal / solidAngleSum;
+
+ const compensatedWeights = new Float64Array(pixelWeights.length);
+ const compensatedRowTotals = new Float64Array(height);
+ let compensatedTotal = 0;
+ for (let y = 0; y < height; y++) {
+ const texelSolidAngle = Math.sin(Math.PI * (y + 0.5) / height) * deltaPhi * deltaTheta;
+ const subtractedWeight = fraction * meanRadiance * texelSolidAngle;
+ let rowSum = 0;
+ for (let x = 0; x < width; x++) {
+ const pixelIndex = y * width + x;
+ const weight = Math.max(0, pixelWeights[pixelIndex] - subtractedWeight);
+ compensatedWeights[pixelIndex] = weight;
+ rowSum += weight;
+ }
+ compensatedRowTotals[y] = rowSum;
+ compensatedTotal += rowSum;
+ }
+ if (!(compensatedTotal > 0)) {
+ return originalTotal;
+ }
+ pixelWeights.set(compensatedWeights);
+ rowTotals.set(compensatedRowTotals);
+ return compensatedTotal;
+}
+
function packEnvironmentTexelsV2(environment, options = {}) {
if (!environment || !(environment.width > 0 && environment.height > 0)) {
const bg = options.backgroundColor ?? [0, 0, 0];
@@ -1356,6 +1415,7 @@ function packEnvironmentTexelsV2(environment, options = {}) {
? Number(environment.rotation)
: Math.atan2(Number(environment.sine ?? 0), Number(environment.cosine ?? 1));
+ const compensationFraction = resolveEnvCompensationFraction(options.envCompensation, 'buildNativeInputV2');
const pixelCount = width * height;
const totalVec4 = 2 * pixelCount + height;
const texels = new Float32Array(totalVec4 * 4);
@@ -1388,6 +1448,10 @@ function packEnvironmentTexelsV2(environment, options = {}) {
totalWeight += rowSum;
}
+ if (compensationFraction > 0) {
+ totalWeight = compensateEnvironmentWeights(pixelWeights, rowTotals, width, height, compensationFraction, totalWeight);
+ }
+
const envCdfOffset = pixelCount;
let marginalRunning = 0;
for (let y = 0; y < height; y++) {
@@ -1588,6 +1652,7 @@ export function validatePackOptions(options, callerName, requireDimensionsAndSam
requireFiniteIntegerInRangeIfPresent(samplesPerDispatch, 'samplesPerDispatch', 1, 65536, callerName);
validateMaxSeconds(options.maxSeconds, callerName);
resolveSamplingOptions(options, callerName);
+ resolveEnvCompensationFraction(options.envCompensation, callerName);
validateFilterOptions(options, callerName);
if (width !== undefined && height !== undefined) {
validateImagePixelCount(width, height, callerName);
diff --git a/tests/native-pack-v2.test.js b/tests/native-pack-v2.test.js
index e0f1053f..d0762c87 100644
--- a/tests/native-pack-v2.test.js
+++ b/tests/native-pack-v2.test.js
@@ -774,3 +774,76 @@ test('pack options reject strings, booleans, arrays and null instead of coercing
}
}
});
+
+function packSunnyEnvironment(options) {
+ const width = 16;
+ const height = 8;
+ const pixels = new Float32Array(width * height * 3).fill(0.2);
+ const sunTexelIndex = 2 * width + 5;
+ pixels[sunTexelIndex * 3] = 12;
+ pixels[sunTexelIndex * 3 + 1] = 12;
+ pixels[sunTexelIndex * 3 + 2] = 12;
+ const environment = createEnvironmentMap({ width, height, data: pixels, intensity: 1 });
+ const parts = [{ name: 'floor', geometry: quadGeometry(-2), material: { color: [0.5, 0.5, 0.5] } }];
+ const scene = buildTraceScene(parts, { map: environment });
+ const camera = createCamera({ eye: [0, 0, 5], target: [0, 0, 0], fov: 50 }, 16, 16);
+ return buildNativeInputV2(scene, camera, { width: 16, height: 16, samples: 1, ...options });
+}
+
+function readEnvironmentConditionalProbabilities(packedInput, width, height) {
+ const cdfStart = width * height;
+ const probabilities = [];
+ for (let pixelIndex = 0; pixelIndex < width * height; pixelIndex++) {
+ probabilities.push(packedInput.environmentTexels[(cdfStart + height + pixelIndex) * 4 + 1]);
+ }
+ return probabilities;
+}
+
+test('envCompensation is off by default and leaves the environment table unchanged', () => {
+ const withoutOption = packSunnyEnvironment({});
+ const withFalse = packSunnyEnvironment({ envCompensation: false });
+ const withZero = packSunnyEnvironment({ envCompensation: 0 });
+ assert.deepEqual(withFalse.environmentTexels, withoutOption.environmentTexels);
+ assert.deepEqual(withZero.environmentTexels, withoutOption.environmentTexels);
+});
+
+test('envCompensation removes the dim texels from the sampling table and keeps the CDFs valid', () => {
+ const plain = packSunnyEnvironment({});
+ const compensated = packSunnyEnvironment({ envCompensation: true });
+ const width = 16;
+ const height = 8;
+ const plainProbabilities = readEnvironmentConditionalProbabilities(plain, width, height);
+ const compensatedProbabilities = readEnvironmentConditionalProbabilities(compensated, width, height);
+
+ const sunRow = 2;
+ const sunColumn = 5;
+ assert.ok(plainProbabilities[sunRow * width + 0] > 0, 'dim texels are sampled without compensation');
+ assert.equal(compensatedProbabilities[sunRow * width + 0], 0, 'dim texels get zero probability');
+ assert.equal(compensatedProbabilities[sunRow * width + sunColumn], 1, 'the sun is the only texel left in its row');
+
+ const cdfStart = width * height;
+ let marginalProbabilitySum = 0;
+ for (let row = 0; row < height; row++) {
+ marginalProbabilitySum += compensated.environmentTexels[(cdfStart + row) * 4 + 1];
+ }
+ assert.ok(Math.abs(marginalProbabilitySum - 1) < 1e-5, 'marginal probabilities sum to 1');
+ assert.equal(compensated.environmentTexels[(cdfStart + height - 1) * 4], 1, 'marginal CDF ends at 1');
+ assert.ok(compensated.environmentTexels[0] === plain.environmentTexels[0], 'radiance texels are untouched');
+});
+
+test('envCompensation keeps the original table for a uniform map and rejects bad values', () => {
+ const width = 8;
+ const height = 4;
+ const uniformMap = createEnvironmentMap({ width, height, data: new Float32Array(width * height * 3).fill(0.5), intensity: 1 });
+ const parts = [{ name: 'floor', geometry: quadGeometry(-2), material: { color: [0.5, 0.5, 0.5] } }];
+ const scene = buildTraceScene(parts, { map: uniformMap });
+ const camera = createCamera({ eye: [0, 0, 5], target: [0, 0, 0], fov: 50 }, 16, 16);
+ const base = { width: 16, height: 16, samples: 1 };
+ const plain = buildNativeInputV2(scene, camera, base);
+ const compensated = buildNativeInputV2(scene, camera, { ...base, envCompensation: true });
+ assert.deepEqual(compensated.environmentTexels, plain.environmentTexels, 'an all-zero table falls back to the plain table');
+ for (const invalid of [-0.1, 1.5, NaN, '1', null]) {
+ assert.throws(() => validatePackOptions({ ...base, envCompensation: invalid }, 'traceNative', true), /envCompensation/, `envCompensation ${JSON.stringify(invalid)}`);
+ }
+ assert.doesNotThrow(() => validatePackOptions({ ...base, envCompensation: 0.5 }, 'traceNative', true));
+});