Skip to content

LightCraft: An open-source non-destructive photo editor written by scratch in pure Rust

LR-PROF-CAMERACOLOR: cover the held-out toe error gate

Alex Chambers1926c9029fb6
diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs
index 50d439eac..7f68911e0 100644
--- a/crates/engine/src/camera_preview.rs
+++ b/crates/engine/src/camera_preview.rs
@@ -1458,6 +1458,64 @@ mod tests {
}
}
+ #[test]
+ fn camera_toe_requires_lower_independent_dark_error() {
+ for holdout_matches_original in [true, false] {
+ let mut sensor = Rgb32f::new(96, 64);
+ let mut reference = sensor.clone();
+ for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() {
+ let y = if i < 600 { 0.0015 + (i % 200) as f32 * 0.00001 } else { 0.02 + ((i * 7919) % 6144) as f32 / 6144.0 * 0.23 };
+ *src = [y; 3];
+ // Midtones teach the original slope of 3. Dark training teaches 0.3,
+ // but the independent dark holdout can agree with either slope.
+ let slope = if i < 600 && !(holdout_matches_original && i % 3 == 0) { 0.3 } else { 3.0 };
+ *dst = [y * slope; 3];
+ assert_eq!(colour_pair(*src, *dst), i >= 600);
+ }
+ let (pairs, _) = collect_pairs(&sensor, &reference, f32::NEG_INFINITY, None).unwrap();
+ let original = fit_tone(pairs.iter().enumerate().filter(|(i, _)| i % 3 != 0).map(|(_, (x, y))| (luma(*x), luma(*y))).collect()).unwrap();
+ // Both sample sets start at a multiple of three, so this split matches
+ // the helper's separate colour and extra-sample indices.
+ let candidate = fit_tone(
+ sensor
+ .data
+ .iter()
+ .zip(&reference.data)
+ .enumerate()
+ .filter(|(i, _)| i % 3 != 0)
+ .map(|(_, (x, y))| (luma(x.map(f64::from)), luma(y.map(f64::from))))
+ .collect(),
+ )
+ .unwrap();
+ let held: Pairs =
+ sensor.data.iter().zip(&reference.data).take(600).step_by(3).map(|(x, y)| (x.map(f64::from), y.map(f64::from))).collect();
+ assert_eq!(held.len(), 200);
+ assert!(held.iter().all(|(_, y)| (0.0003..0.015).contains(&luma(*y))));
+ assert!(rank_correlation_ties(&held) > 0.99);
+ let error = |tone: &CameraTone| {
+ held.iter().map(|(x, y)| (f64::from(tone.apply(luma(*x) as f32)).powf(1.0 / 2.2) - luma(*y).powf(1.0 / 2.2)).powi(2)).sum::<f64>()
+ };
+ assert_ne!(candidate, original);
+ let selected = fit_tone_with_shadows(ToneFit::Quantile, &pairs, &sensor, &reference, None, |x| x).unwrap();
+ if holdout_matches_original {
+ assert!(error(&candidate) > error(&original) + 0.1);
+ assert_eq!(selected, original, "a correlated but misleading training set must keep the original curve");
+ } else {
+ assert!(error(&candidate) < error(&original) * 0.99);
+ assert_eq!(selected, candidate, "matching dark training and holdout must accept the candidate");
+ }
+ let mut previous = 0.0;
+ assert_eq!(selected.apply(0.0), 0.0);
+ for i in 0..10_000 {
+ let y = selected.apply(10f32.powf(-8.0 + i as f32 / 1000.0));
+ assert!(y.is_finite() && y >= previous && y <= 1.0);
+ previous = y;
+ }
+ assert_eq!(selected.apply(f32::NAN), 0.0);
+ assert_eq!(selected.apply(f32::INFINITY), 0.0);
+ }
+ }
+
#[test]
fn unreliable_dark_pairs_keep_the_original_tone_and_colour() {
let (sensor, reference) = steep_toe_scene();
diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md
index 1feef2e17..8c544854b 100644
--- a/docs/camera-preview-colour.md
+++ b/docs/camera-preview-colour.md
@@ -28,10 +28,14 @@ The colour pairs exclude JPEG luminance below 0.015 (about L* 13). Using those s
Tone fitting now also tries a separate luminance sample set from the same binned proxies. Each sensor channel must be finite and between 0.0001 and 1.5; JPEG channels must be finite and nonnegative, with luminance between 0.0003 and 0.85. The lower JPEG floor is about one 8-bit sRGB code above black. It leaves compression-clipped black samples out of the fit. Luminance does not need the colour fit's per-channel ratios or its JPEG channel floor. The existing clip mask and, on an attempt away from edges, the same edge exclusion still apply.
-The wider set needs at least 96 dark samples (JPEG luminance below 0.015), including at least 32 held-out dark samples. Two thirds train the curve, with the existing quantile medians or conditional-median fallback. Extra samples are split separately, so the wider set cannot put original colour holdouts into tone training. The held-out dark samples must have luminance rank correlation of at least 0.5, with ties sharing their rank. The candidate must reduce their squared gamma-encoded luminance error by at least 1% against the original curve. Sparse, flat, uncorrelated or invalid shadow samples keep the original curve. The completed look must still pass its existing held-out gates. Colour-pair selection, colour fitting and the original colour training split are unchanged.
+The wider set needs at least 96 dark samples (JPEG luminance below 0.015), including at least 32 held-out dark samples. Two thirds train the curve, with the existing quantile medians or conditional-median fallback. Extra samples are split separately, so the wider set cannot put original colour holdouts into tone training. The held-out dark samples must have luminance rank correlation of at least 0.5, with ties sharing their rank. When the original curve can be fitted, the candidate must reduce its held-out dark error by at least 1%, measured as squared gamma-encoded luminance error. Sparse, flat, uncorrelated or invalid shadow samples keep the original curve.
+
+If the original curve cannot be fitted, the wider curve can still be accepted after the sample-count and rank checks. In either case, the completed look must pass its existing held-out gates: a full look needs finite errors, linear-display squared error below 70% of the neutral fallback's and per-channel RMS no greater than 0.10. Partial and monochrome tone-only looks need luminance rank correlation of at least 0.5 and finite squared errors below 70% of the fallback's in both linear and gamma-encoded values. The monochrome comparison uses luminance; the full and partial comparisons use colour channels. Colour-pair selection, colour fitting and the original colour training split are unchanged.
The extra dark knots move the straight-line extension down toward black without a new serialized curve format or a different CPU/GPU interpolation rule. `LOOK_VERSION` 5 refreshes older fitted smart previews when their originals are online; `RENDER_CACHE_VERSION` 62 redraws cached renders.
+No process-version bump is needed under the [rule for changes to source data](process-versions.md#the-rule). The decoded source carries the fitted camera tone in `SourceInfo`; both V1 and V2 use it in `finish::base_tone`. Existing raws edited under V1 or V2 can therefore change when decoded again. Offline smart previews keep their stored pixels and tone until rebuilt from the original. The scan on library opening runs once per look version for each smart previews folder. If an original was offline at that scan, run Build Smart Previews after reconnecting it.
+
Compared with main `701f78ea` at a 1200-pixel long edge, all 15 public raw.pixls.us files across six makes and the private #1021 file retain their accepted look. The darkest 10% moves closer to the JPEG on all seven public deep-shadow files (at least 10% below L* 13) and #1021. On Sony RX100 V sample 1324, its mean L* changes from 4.79 to 2.13 against the JPEG's 2.14; on a6700 sample 6735, 7.20 to 5.23 against 4.60. On #1021 (numbers only), 9.28 to 3.25 against 3.50; the 10th percentile changes from 11.94 to 5.34 against 5.45, and mean CIEDE2000 at 200 pixels changes from 7.25 to 6.37. Five of eight controls are byte-identical. The largest control changes are on a7 IV sample 6928, which still has 6.8% below L* 13: mean CIEDE2000 improves by 0.048, whole-frame mean L* falls by 0.149, and the 5th percentile falls by 1.56 L*. This is a limited camera-JPEG comparison, not a claim of colour calibration or Lightroom parity.
## Source metadata and white balance