Skip to content

LightCraft: An open-source non-destructive photo editor written by scratch in pure Rust

M11.4: merge main; spectral matrices only replace the neutral fallback

Brandon Thomas9f73f086c905
diff --cc ROADMAP.md
index c4bd73cef,dad5a432d..0986772bd
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@@ -23,13 -23,13 +23,13 @@@ duplicate Local entries #22, black GPU
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79.7% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P0: full-resolution zoom detail; P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
- | **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, uncompressed Samsung SRW (NX5 to NX1100, EX1, WB2000), RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, compressed Samsung SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini), Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
- | **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); 52 models start from matrices fitted to measured spectral sensitivities; other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
+ | **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, uncompressed Samsung SRW (NX5 to NX1100, EX1, WB2000), RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, compressed Samsung SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini), Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Nikon High Efficiency NEF (HE / HE★: Z 8, Z 9, Z 6III, Z f; JPEG XS-based, preview only, #193). Per-model verification is thin (~55 corpus files vs >1,000 models) |
-| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7CR, ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
++| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7CR, ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); 52 models whose JPEG fit is rejected start from matrices fitted to measured spectral sensitivities; other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics, or the photo's own segmentation mattes when it ships them (DNG semantic masks, e.g. iPhone ProRAW's sky matte); non-destructive Bayer RAW denoise with opt-in weights on pure-Rust CPU/GPU; **faces (MVP, [docs/faces.md](docs/faces.md))**: a pure-Rust detector (a 232 KB opt-in download), opt-in recognition models (one-click download or add a file; they run on the shared pure-Rust CPU interpreter; pinned SFace/AuraFace embeddings agree with an independent reference runtime on 40 public crops; one-thread medians 34–36 / 233–252 ms (AuraFace remains expensive)), a background scan that finds and embeds faces, name suggestions, a person's page with look-alike faces to confirm, an Unnamed faces section (look-alikes together, select a group and name it at once) | Real segmentation masks (subject, sky, people, objects, landscape, depth), denoise model policy / linear RGB / quality verification, super resolution, lens blur, generative remove, natural-language search; for faces: clusters with separators between groups, small faces in large photos (the detector looks at 640 px), thresholds calibrated on live faces, writing face regions back to XMP. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
- | **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ |
- | **HDR & video** | 0% | | HDR edit/display/export; video play/trim/edit/export |
- | **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese, Japanese, Brazilian Portuguese and German UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
+ | **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic), paginated contact-sheet PDF export (A4/Letter, layout/captions, current edits, 150 dpi sRGB) | **Map view, Book, full Print module, Slideshow module, Web, publish services**: contact sheets work, custom print packages and printer colour management remain missing |
+ | **HDR & video** | ~25% | HDR editing (headroom limit, SDR rendition, Visualize HDR); gain map JPEG, PQ AVIF and float TIFF export | On-screen HDR display, HLG, HDR settings from crs XMP; video play/trim/edit/export |
+ | **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese, Japanese, Brazilian Portuguese, Spanish, German, Russian and Ukrainian UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
### By kind of user
@@@ -48,9 -48,8 +48,9 @@@ Priorities, in order. Each points at tr
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2,
- Pentax PEF and Samsung SRW have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF since issue #310, not yet
- checked on the corpus); 52 models start from matrices fitted to measured spectral sensitivities (rawtoaces-data);
- then ORF, then validate fidelity.
+ Pentax PEF, Samsung SRW and Olympus ORF have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF
- since issue #310, not yet checked on the corpus); then validate fidelity.
++ since issue #310, not yet checked on the corpus); 52 models whose fit is rejected fall back to matrices fitted to
++ measured spectral sensitivities (rawtoaces-data); then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): Remaining **unverified CR3 CRX** variants, compressed ORF, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
@@@ -183,13 -188,9 +189,9 @@@ packed formats 2, 5 and 7, the 16-bit w
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Supported containers yield their embedded JPEG preview when present (including CR3). The engine uses it for RAW variants it cannot decode yet; HEVC-only CR3 previews are not decoded.
Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
- - **Nikon "lossy after split" NEF** (non-zero split row in maker note `0x0096`, e.g. some D3400/D5000/D5200/D5500/D5600
- files): rows above the split decode with the regular lossy table; from the split row on a different code is used
- that our black-box analysis has not recovered yet (none of the four regular tables fits, also not with byte
- alignment or reset predictors at the split row). The other Nikon Huffman variants are decoded (`crates/raw/src/vendor/nefc.rs` documents the analysis).
- **Olympus compressed ORF**, **Canon CR3 unverified CRX variants** (M11.1), **Canon sRAW/mRAW**.
- **Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF and Samsung SRW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. 52 models without a camera profile start from matrices fitted to their measured spectral sensitivities instead (rawtoaces-data, Apache-2.0; `crates/raw/src/spectral.rs`). Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
-**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF, Samsung SRW and Olympus ORF files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
++**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF, Samsung SRW and Olympus ORF files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. 52 models whose fit is rejected (or that have no usable JPEG) start from matrices fitted to their measured spectral sensitivities instead (rawtoaces-data, Apache-2.0; `crates/raw/src/spectral.rs`). Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
diff --cc assets/ATTRIBUTION.md
index 30709cb12,0fc883bdd..7ade9dac2
--- a/assets/ATTRIBUTION.md
+++ b/assets/ATTRIBUTION.md
@@@ -12,8 -12,9 +12,10 @@@ covers a whole directory), or if a lice
| `assets/app-icon/` (`lightcraft.svg` and the PNG/ICNS/ICO/SVG files derived from it) | LightCraft app icon (lynx) | LightCraft project owner (drawn in ArtCraft) | original work (vectorised from the owner's drawing; see `assets/app-icon/README.md`) | MIT OR Apache-2.0 (`assets/app-icon/LICENSE.txt`) | 2026-10-02 | vectorised to SVG; rendered to PNG/ICNS/ICO by `packaging/icons.sh` |
| `assets/camera-profiles/*.json` | Camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | ILCE-7M4: Harald Hoyer | original work: fitted by `lightcraft-cli calibrate` to the contributor's own raw photos and their in-camera JPEGs (597 ILCE-7M4 photos, 2026); aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-07 | none |
| `packaging/macos/dmg/` (`background.svg`, the `background.tiff` rendered from it by `generate.py`, and the `dmg-layout.DS_Store` layout file) | LightCraft macOS DMG window background | @XusBadia | original work for this repository, derived from the LightCraft app icon (`assets/app-icon/lightcraft.svg`, referenced via `<image>`, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0 | 2026-10-08 | none (original); used for the Finder window of the macOS DMG, `packaging/macos/dmg/generate.py` renders it to `background.tiff` |
+ | `assets/camera-profiles/ILCE-7CR.json` | Sony ILCE-7CR camera colour profile (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by @AlexChambers | original work: fitted by `lightcraft-cli calibrate` to 125 usable ILCE-7CR ARWs (of 131, September 2025 to September 2026, Standard creative style) and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-09 | none |
| `assets/camera-profiles/X-H2S.json`, `assets/camera-profiles/X-T4.json` | Fujifilm camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by Sebastian Pfluegelmeier | original work: fitted by `lightcraft-cli calibrate` to 201 X-H2S and 545 X-T4 RAFs and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-08 | none |
+| `crates/raw/src/spectral_table.rs` | Camera colour matrices (DNG ColorMatrix1/2 and ForwardMatrix1/2 for Standard Illuminant A and D65) for 52 camera models, and the model alias list | Derived by LightCraft contributors from rawtoaces-data (Academy Software Foundation): camera spectral sensitivities measured by Weta Digital (44 cameras), scitech (7) and Jonathon Irons at the National Physical Laboratory (Nikon D5100); training reflectances; CIE 1931 colour-matching functions (CIE, CC-BY-4.0, as shipped in rawtoaces-data) | https://github.com/AcademySoftwareFoundation/rawtoaces-data, tag v1.1.0, commit 5cfa53707b053594b2d0d48e8019fb56e6582d6c | Apache-2.0 (`LICENSE-APACHE`); the colour-matching functions CC-BY-4.0 | 2026-10-08 | generated by `tools/spectral_camera_table.py`: a 3×3 matrix per illuminant fitted to the spectral data (no measurement copied verbatim); aliases copied from `data/aliases.json`; no Adobe or third-party camera matrix |
+ | `assets/camera-tone/sony-dro-off.json` | Sony global tone curve with the Dynamic Range Optimizer off (32 knots), built into LightCraft for ARWs shot with DRO on | LightCraft contributors; source photographs by raw.pixls.us contributors | original work: tone curve quantile-matched over the pooled pixel pairs of 12 CC0 raw.pixls.us ARWs (DRO off, Standard creative style; ILCE-3500, -5000, -6600, -7CR, -7M3 ×3, -7S, -9M2, ILCA-77M2, SLT-A57, NEX-F3) and their embedded camera JPEGs; the file lists each source URL and SHA-256 and how the curve was derived (see also `docs/camera-preview-colour.md`); aggregate tone statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 (sources CC0) | 2026-10-09 | none |
| `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `docs/images/*-tetons.jpg`, `docs/images/grid-pd.jpg` (screenshots containing it) | "The Tetons and the Snake River" | Ansel Adams (1942), U.S. National Archives | https://commons.wikimedia.org/wiki/File:Adams_The_Tetons_and_the_Snake_River.jpg | Public domain (U.S. federal government work) | 2026-09-30 | developed in LightCraft; shown inside app screenshots |
diff --cc crates/engine/src/camera_preview.rs
index 582580217,fd7c58f8a..8f7789591
--- a/crates/engine/src/camera_preview.rs
+++ b/crates/engine/src/camera_preview.rs
@@@ -4,8 -4,7 +4,7 @@@
//! A global matrix can't follow the camera's hue-dependent rendering (the best matrix rendered a
//! lime shirt olive that the camera kept lime): a hue/saturation table fitted to the residuals
//! (applied like a DNG `ProfileHueSatMap`) corrects that when it also improves the held-out pixels.
- use crate::camera_profiles::CameraProfile;
-use lightcraft_color::{D50, D65, Mat3, PROPHOTO, REC2020, bradford, luminance_2020};
+use lightcraft_color::{D50, D65, Mat3, PROPHOTO, REC2020, Xy, bradford, luminance_2020};
use lightcraft_pipeline::tone::{CameraTone, ToneMap};
use lightcraft_raster::{
Rgb32f,
@@@ -46,55 -46,23 +46,60 @@@ pub(crate) fn file_local_look(format: R
)
}
- /// The colour a raw starts from, first available first: the file's own colour matrices (DNG); the camera profile for
- /// its model (bundled or local, `camera_profiles`), with tone and chroma fitted to the file's JPEG; the model's colour
- /// matrices from measured spectral sensitivities (`lightcraft_raw::spectral`); a fit to the file's own JPEG; the
- /// neutral fallback. The spectral matrices are written into `raw.color`. Returns the as-shot white, the camera
- /// transform at it and the fitted look.
++/// The colour a raw starts from, first available first: the file's own colour matrices (DNG); a look fitted to the
++/// file's JPEG ([`fit_preview`]: with the camera profile for its model when there is one, else with colour fitted to
++/// this photo alone); the model's colour matrices from measured spectral sensitivities (`lightcraft_raw::spectral`);
++/// the neutral fallback. The spectral matrices only stand in for the neutral fallback, so a photo whose JPEG fit is
++/// accepted renders as without them; when used they are written into `raw.color`. Returns the as-shot white, the
++/// camera transform at it and the fitted look.
+pub(crate) fn starting_colour(raw: &mut RawImage, bytes: &[u8]) -> (Xy, CameraTransform, Option<CameraLook>) {
++ starting_colour_with(raw, |raw, t| fit_preview(raw, bytes, t))
++}
++
++/// [`starting_colour`] with the JPEG fit passed in (tests pin the order with it).
++fn starting_colour_with(
++ raw: &mut RawImage,
++ fit: impl FnOnce(&RawImage, &CameraTransform) -> Option<CameraLook>,
++) -> (Xy, CameraTransform, Option<CameraLook>) {
+ let start = |raw: &RawImage| {
+ let xy = lightcraft_raw::color::as_shot_white_xy(raw);
+ (xy, lightcraft_raw::color::camera_transform(raw, xy))
+ };
+ let (xy, t) = start(raw);
- let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get);
- let spectral = match raw.metadata.model.as_deref() {
- Some(model) if t.matrix_is_fallback => lightcraft_raw::spectral::find(raw.metadata.make.as_deref(), model),
- _ => None,
- };
- let Some(camera) = spectral else { return (xy, t, fit_preview(raw, bytes, &t, profile.as_deref(), true)) };
- if let Some(look) = fit_preview(raw, bytes, &t, profile.as_deref(), false) {
++ if !t.matrix_is_fallback {
++ return (xy, t, None);
++ }
++ if let Some(look) = fit(raw, &t) {
+ return (xy, t, Some(look));
+ }
++ let Some(camera) = raw.metadata.model.as_deref().and_then(|model| lightcraft_raw::spectral::find(raw.metadata.make.as_deref(), model)) else {
++ return (xy, t, None);
++ };
+ camera.fill(&mut raw.color);
+ if lightcraft_pipeline::profiling() {
+ eprintln!("[profile] {:?} camera colour from spectral sensitivities: {} {}", raw.format, camera.make, camera.model);
+ }
+ let (xy, t) = start(raw);
+ (xy, t, None)
+}
+
- /// The look fitted to the file's JPEG: with `profile`'s colour when there is one, else (or when that fails the
- /// gates, if `own_colour`) with colour fitted to this photo alone.
- fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform, profile: Option<&CameraProfile>, own_colour: bool) -> Option<CameraLook> {
- if !transform.matrix_is_fallback || !file_local_look(raw.format) || (profile.is_none() && !own_colour) {
+ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option<CameraLook> {
+ if !transform.matrix_is_fallback || !file_local_look(raw.format) {
return None;
}
+ let (sensor, reference, clipped) = proxies(raw, bytes, transform, PROXY)?;
// A camera profile pooled from many photos knows colours this photo shows too little of;
- // try its colour first and fit tone/chroma per photo (DRO and picture styles vary).
- let colour = profile.and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone())));
- if colour.is_none() && !own_colour {
- return None;
- }
- let (sensor, reference) = proxies(raw, bytes, transform, PROXY)?;
- let look = if own_colour {
- fit_pairs_with(&sensor, &reference, colour)?
- } else {
- TONE_FITS.iter().find_map(|&tone| fit_candidate(&sensor, &reference, colour.clone(), tone))?
- };
+ // try its colour first and fit tone/chroma per photo (picture styles vary).
+ let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get);
+ let colour = profile.as_ref().and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone())));
+ let look = fit_look(&sensor, &reference, &clipped, colour)?;
+ // Sony's Dynamic Range Optimizer (on by default) brightens the camera JPEG's darker regions,
+ // not the raw: keep the colour fitted to the JPEG, take the tone curve without DRO.
+ let dro = lightcraft_raw::embedded_preview_dynamic_range_optimized(bytes) == Some(true);
+ let look = if dro { with_dro_off_tone(look) } else { look };
if lightcraft_pipeline::profiling() {
eprintln!(
- "[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}",
+ "[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}, DRO tone replaced {dro}",
raw.format,
look.matrix.0,
look.tone,
@@@ -1142,84 -1389,6 +1426,130 @@@ mod tests
assert!(fit_hue_sat(&[], &Mat3::IDENTITY).is_none(), "too few samples");
}
+ fn raw_of(format: RawFormat, make: &str, model: &str, color: lightcraft_raw::ColorData) -> RawImage {
+ let area = lightcraft_raw::Rect::new(0, 0, 2, 2);
+ RawImage {
+ format,
+ width: 2,
+ height: 2,
+ cpp: 1,
+ data: lightcraft_raw::RawData::U16(vec![0; 4]),
+ cfa: None,
+ bits: 14,
+ black: Default::default(),
+ white: vec![16383.0],
+ active_area: area,
+ crop: area,
+ orientation: Default::default(),
+ color,
+ wb_multipliers: Some([2.0, 1.0, 1.5]),
+ linearized: false,
+ opcodes: Default::default(),
+ metadata: lightcraft_raw::Metadata { make: Some(make.to_owned()), model: Some(model.to_owned()), ..Default::default() },
+ }
+ }
+
- /// Step 1: a file's own colour matrices win over everything, including a spectral row for its model.
++ /// Step 1: a file's own colour matrices win over everything, including a spectral row for its model; no JPEG fit
++ /// is tried.
+ #[test]
+ fn a_files_own_matrices_come_first() {
+ let own = Mat3([[0.6, -0.1, -0.05], [-0.4, 1.3, 0.1], [-0.1, 0.2, 0.7]]);
+ let color = lightcraft_raw::ColorData { illuminant: [21, 0], color_matrix: [Some(own), None], ..Default::default() };
+ let mut raw = raw_of(RawFormat::Cr2, "Canon", "Canon EOS 600D", color.clone());
- let (_, t, look) = starting_colour(&mut raw, &[]);
++ let (_, t, look) = starting_colour_with(&mut raw, |_, _| panic!("no JPEG fit for a raw with its own matrices"));
+ assert_eq!(raw.color, color);
+ assert!(!t.matrix_is_fallback && look.is_none());
+ }
+
- /// Step 2: a camera profile comes before the spectral matrices; when its fit can't run (here no camera JPEG) the
- /// spectral matrices follow. The ILCE-7M4 has a bundled profile and a spectral row.
++ /// Steps 2 and 3 before step 4: an accepted JPEG fit (with or without a camera profile) is kept for a model that
++ /// has spectral matrices, and the spectral matrices are not written. Photos that had a fitted look before the
++ /// spectral table render exactly as they did.
+ #[test]
- fn a_camera_profile_is_tried_before_the_spectral_matrices() {
++ fn an_accepted_jpeg_fit_comes_before_the_spectral_matrices() {
++ for (make, model) in [("SONY", "ILCE-7M4"), ("Canon", "Canon EOS 600D")] {
++ assert!(lightcraft_raw::spectral::find(Some(make), model).is_some(), "{model}");
++ let mut raw = raw_of(RawFormat::Cr2, make, model, Default::default());
++ let fitted = dro_test_look(&std::array::from_fn(|i| 0.006 * 1.13f32.powi(i as i32)), &|x| x.sqrt());
++ let (xy, t, look) = starting_colour_with(&mut raw, |raw, t| {
++ assert!(t.matrix_is_fallback && !lightcraft_raw::color::has_matrix(&raw.color), "the fit sees the raw without spectral matrices");
++ Some(fitted.clone())
++ });
++ assert_eq!(look.map(|l| l.matrix), Some(fitted.matrix), "{model}");
++ assert_eq!(raw.color, lightcraft_raw::ColorData::default(), "{model}: spectral matrices left out");
++ assert!(t.matrix_is_fallback, "{model}");
++ assert_eq!(xy, lightcraft_raw::color::as_shot_white_xy(&raw));
++ }
++ }
++
++ /// Step 4: when the JPEG fit fails (here: no camera JPEG at all), a model with a spectral row takes its matrices,
++ /// whether or not it has a camera profile (the ILCE-7M4 has a bundled one).
++ #[test]
++ fn a_failed_jpeg_fit_falls_back_to_the_spectral_matrices() {
+ assert!(crate::camera_profiles::get("ILCE-7M4").is_some());
+ let camera = lightcraft_raw::spectral::find(Some("SONY"), "ILCE-7M4").unwrap();
+ let mut raw = raw_of(RawFormat::Arw, "SONY", "ILCE-7M4", Default::default());
+ let (_, t, look) = starting_colour(&mut raw, &[]);
+ assert!(look.is_none());
+ assert_eq!(raw.color.color_matrix, camera.color_matrix.map(Some));
+ assert!(!t.matrix_is_fallback);
- }
+
- /// Step 3: a camera without a profile takes its spectral matrices, and no JPEG fit is tried: the look is `None`
- /// and the transform is the DNG model's, at the as-shot white the matrices give the vendor multipliers.
- #[test]
- fn spectral_matrices_come_before_the_photos_own_fit() {
+ let camera = lightcraft_raw::spectral::find(Some("Canon"), "Canon EOS 600D").unwrap();
+ let mut raw = raw_of(RawFormat::Cr2, "Canon", "Canon EOS 600D", Default::default());
- let (xy, t, look) = starting_colour(&mut raw, &[]);
++ let (xy, t, look) = starting_colour_with(&mut raw, |_, _| None);
+ assert!(look.is_none());
+ assert_eq!(raw.color.illuminant, [17, 21]);
+ assert_eq!(raw.color.forward_matrix, camera.forward_matrix.map(Some));
+ assert!(!t.matrix_is_fallback);
+ assert_eq!(xy, lightcraft_raw::color::as_shot_white_xy(&raw));
+ // white-balanced white stays white
+ let w = t.matrix.apply([1.0; 3]);
+ assert!(w.iter().all(|v| (v - 1.0).abs() < 1e-4), "{w:?}");
+ }
+
- /// Steps 4 and 5: a camera outside the table keeps today's path, the photo's own JPEG fit and else the neutral
- /// fallback (here: no camera JPEG, so the fallback).
++ /// Step 5: a camera outside the table whose fit fails keeps the neutral fallback.
+ #[test]
- fn a_camera_outside_the_table_keeps_the_fit_or_the_fallback() {
++ fn a_camera_outside_the_table_keeps_the_fallback() {
+ let mut raw = raw_of(RawFormat::Cr2, "Canon", "Canon EOS 7D", Default::default());
+ let (_, t, look) = starting_colour(&mut raw, &[]);
+ assert!(look.is_none() && t.matrix_is_fallback);
+ assert_eq!(raw.color, lightcraft_raw::ColorData::default());
+ // without a model there is no table row
+ let mut raw = raw_of(RawFormat::Cr2, "Canon", "", Default::default());
+ raw.metadata.model = None;
+ assert!(starting_colour(&mut raw, &[]).1.matrix_is_fallback);
+ }
+
++ /// The colour precedence on public raws (each skipped without its corpus file): a photo whose JPEG fit is
++ /// accepted keeps it and its colour data; one whose fit fails takes its spectral matrices when its model has
++ /// them; a DNG keeps its own matrices. White balance stays relative to the as-shot look for every raw without
++ /// matrices of its own.
++ #[test]
++ fn corpus_colour_precedence() {
++ let dir = std::env::var_os("LIGHTCRAFT_CORPUS")
++ .map(std::path::PathBuf::from)
++ .unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
++ .join("raw");
++ for name in ["arw-sony-a7m4-14bit.arw", "cr2-canon-5d3.cr2", "nef-nikon-d5100-lossless.nef", "cr2-canon-7d.cr2", "dng-canon-5d3-16bit.dng"] {
++ let Ok(bytes) = std::fs::read(dir.join(name)) else {
++ eprintln!("skip: {name} absent");
++ continue;
++ };
++ let mut raw = lightcraft_raw::decode(&bytes).unwrap();
++ raw.opcodes.list3.retain(|op| !op.is_lens_correction());
++ let own = lightcraft_raw::color::has_matrix(&raw.color);
++ let covered = raw.metadata.model.as_deref().and_then(|m| lightcraft_raw::spectral::find(raw.metadata.make.as_deref(), m)).is_some();
++ let before = raw.color.clone();
++ let (_, t, fitted) = starting_colour(&mut raw, &bytes);
++ let spectral = !own && fitted.is_none() && covered;
++ eprintln!("{name}: own matrices {own}, JPEG fit {}, spectral {spectral}", fitted.is_some());
++ assert_eq!(raw.color != before, spectral, "{name}");
++ assert_eq!(t.matrix_is_fallback, !own && !spectral, "{name}");
++ let (_, info) = crate::files::load_bytes(&bytes, 400).unwrap();
++ assert_eq!(info.camera_tone.is_some(), fitted.is_some(), "{name}");
++ assert_eq!(info.relative_wb, !own, "{name}");
++ }
++ }
++
#[test]
fn supported_raws_get_a_file_local_look() {
// every decoded raw without a colour matrix of its own (issue #310: CR2 rendered flat with the fallback)
diff --cc crates/engine/src/media.rs
index fcfbbe208,1e2a7d3e5..a14093de8
--- a/crates/engine/src/media.rs
+++ b/crates/engine/src/media.rs
@@@ -62,7 -62,7 +62,7 @@@ impl SettingsHashes
}
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
- pub const RENDER_CACHE_VERSION: u64 = 23;
-pub const RENDER_CACHE_VERSION: u64 = 37;
++pub const RENDER_CACHE_VERSION: u64 = 38;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
diff --cc crates/raw/src/lib.rs
index 7f725090a,ed17107e9..4f9821c09
--- a/crates/raw/src/lib.rs
+++ b/crates/raw/src/lib.rs
@@@ -16,12 -16,10 +16,11 @@@
//! `jxl` feature, on by default), tiled/stripped, CFA and LinearRaw),
//! Canon CR2 / CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), Nikon NEF/NRW (uncompressed, Huffman lossless / lossy compressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
//! and X-Trans, lossless and lossy compressed), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format: compressed 4 and 6, the prefix-coded strips of 8,
- //! packed 2/5/7, the 16-bit words of the oldest bodies), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
- //! [`embedded_preview`] covers these containers' JPEG previews. Variants we can't decode yet (Nikon "lossy after split" NEF,
- //! compressed ORF, CR3 unverified marker families / C-RAW configurations) return [`RawError::Unsupported`]; each vendor module documents its sources
+ //! packed 2/5/7, the 16-bit words of the oldest bodies), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed: 16-bit words, 12-bit XZ-2 words, 12-bit 16-byte blocks of the E-300/E-330/E-500).
+ //! [`embedded_preview`] covers these containers' JPEG previews. Variants we can't decode yet (compressed ORF, CR3 unverified marker families / C-RAW configurations) return [`RawError::Unsupported`]; each vendor module documents its sources
//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
-//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
+//! colour matrix: [`spectral`] has matrices fitted to measured spectral sensitivities for 52 models, and
+//! otherwise [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
@@@ -38,8 -36,8 +37,9 @@@ pub mod ljpeg
pub mod opcodes;
mod preview;
pub mod profile;
+ mod saturation;
pub mod semantic;
+pub mod spectral;
mod tiffraw;
mod unpack;
mod vendor;
diff --cc docs/camera-preview-colour.md
index ca529a001,813b1e162..afa48dfa6
--- a/docs/camera-preview-colour.md
+++ b/docs/camera-preview-colour.md
@@@ -74,37 -78,27 +78,56 @@@ This is a per-file camera-look estimate
One photo can show too little of a colour for its own fit to learn it: in a second shot of the lime shirt only a few dozen proxy pixels show it, next to a hillside of foliage at the same hue that the camera renders differently, and the shirt stayed yellow (hue 68° against the JPEG's 82°; a 192-pixel proxy only reached 74°). `lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS…` therefore pools the colour pairs of many ARWs, NEFs or RAFs per camera model — each against its own embedded JPEG, at a 192-pixel proxy, at most 4000 pairs per photo, files spread evenly over the folders — and fits one matrix (from white-balanced camera RGB, so no single photo's white point is baked in) and one hue/saturation/value table. Models need at least 5 usable photos.
- Profiles are JSON files `<model>.json` in `$LIGHTCRAFT_CAMERA_PROFILES`, else `<config>/camera-profiles` (macOS `~/Library/Application Support/LightCraft/camera-profiles`); a local profile replaces a built-in one. Built-in profiles live in `assets/camera-profiles/` (listed in `assets/ATTRIBUTION.md`) and are compiled in, so the app, CLI and web build share them: ILCE-7M4, fitted to 597 photos (2.2 million colour pairs) shot in 2026, mostly with the Standard creative style and DRO Auto; X-H2S, 201 photos; X-T4, 545 photos. They hold aggregate colour statistics only. A photo of a profiled model takes its colour from the profile and fits only its own tone and chroma curves (DRO and picture styles vary per shot); the acceptance gates still apply. A rejected profile fit retries the photo's own colour and tone fit before falling back to neutral rendering. Files are read once per process and validated (version, bounded invertible matrix, table shape and finite data); a damaged file is ignored with a warning. The profiles folder's contents are part of the render cache keys, so thumbnails rendered before a profile existed are redone; smart previews built before keep their colour until rebuilt.
+ Profiles are JSON files `<model>.json` in `$LIGHTCRAFT_CAMERA_PROFILES`, else `<config>/camera-profiles` (macOS `~/Library/Application Support/LightCraft/camera-profiles`); a local profile replaces a built-in one. Built-in profiles live in `assets/camera-profiles/` (listed in `assets/ATTRIBUTION.md`) and are compiled in, so the app, CLI and web build share them: ILCE-7M4, fitted to 597 photos (2.2 million colour pairs) shot in 2026, mostly with the Standard creative style and DRO Auto; ILCE-7CR, 125 photos (below); X-H2S, 201 photos; X-T4, 545 photos. They hold aggregate colour statistics only. A photo of a profiled model takes its colour from the profile and fits only its own tone and chroma curves (picture styles vary per shot; with Sony's DRO on, that tone curve is then lowered as described below); the acceptance gates still apply. A rejected profile fit retries the photo's own colour and tone fit before falling back to neutral rendering. Files are read once per process and validated (version, bounded invertible matrix, table shape and finite data); a damaged file is ignored with a warning. The profiles folder's contents are part of the render cache keys, so thumbnails rendered before a profile existed are redone; smart previews built before keep their colour until rebuilt.
+
+
+ ### Sony ILCE-7CR
+
+ Fitted from 125 usable photos of 131 ILCE-7CR ARWs (one photographer, September 2025 to September 2026, Standard creative style with DRO off, Lv3 or Auto; 127 uncompressed, 2 compressed, 2 lossless compressed), 448,230 colour pairs. Measured as above: rendered at 1200 px with `lightcraft-cli render`, mean CIE76 ΔE against each file's embedded JPEG at 200 px.
+
+ Without a profile, 123 of the 131 photos accept their own fit; the other 8 open with the neutral fallback at ΔE 6–26. For validation every fifth photo in sorted name order (26) was withheld and a profile fitted to the other 105 (100 usable). On the withheld photos mean ΔE went from 4.07 with per-file fitting to 3.29 with the profile. Both withheld photos whose own fit is rejected are accepted (ΔE 26.0 → 6.1 on a night scene, 6.2 → 5.5). Withheld photos whose own fit was accepted are about unchanged and move both ways: 12 of 24 improve, mean ΔE 3.06 → 3.08, largest regression 1.8. On that photo (gold tinsel bokeh) the profile turns a blue light blue again where the per-file fit had made it teal, but renders the gold slightly more saturated than the camera. With the final profile all 131 photos are accepted, and the 8 previously rejected ones land at ΔE 1.3–5.7; that comparison is in-sample.
+
+ The nine public raw.pixls.us ILCE-7CR files are one scene in every coding the camera writes (uncompressed, lossless compressed L, M and S, compressed, full frame and APS-C crop). They were not used for fitting. All nine accept their own fit at ΔE 1.3–2.1 and stay accepted with the profile, at ΔE 1.5–2.9 (mean 1.59 → 1.99). On this colourful daylight scene the per-file fit is already close. The subsampled M and S codings, which decode with unity white-balance gains, move most (2.0 → 2.8). This is a camera-JPEG look estimate like the others, not measured calibration or Lightroom parity.
+ ## Sony Dynamic Range Optimizer (DRO)
+
+ Sony bodies apply DRO, a local tone operator that brightens darker regions, to their JPEG (and the preview embedded in the ARW) only; the raw data are unchanged, and the factory default is DRO Auto. The tone curve fitted to that JPEG carried the brightening into the raw's starting look, so DRO photos opened lighter than in Lightroom, darktable, Photomator or Affinity, which develop the raw without it (issue #244). On 24 ILCE-7CR photos (Standard creative style) with Lightroom 9.5.1 default exports, mean L* at 200 px: with DRO off, Lightroom matches the camera JPEG (+0.15 L*, all within ±1.4) and LightCraft matches Lightroom (−0.19 L*, median ΔE76 3.6); with DRO Lv3 or Auto (8 photos), Lightroom is 2.4–10.7 L* darker than the camera JPEG and LightCraft followed the JPEG, +6.4 L* from Lightroom (median ΔE76 7.5). The tone curves fitted to 131 ILCE-7CR photos show where: at scene luminance 0.01, 0.02, 0.05 and 0.1 the median DRO Lv3/Auto curve is 8, 10, 9 and 6 L* above the DRO-off one, and from 0.3 up they agree.
+
+ The ARW decoder reads the setting from maker-note tag `0xb025` (ExifTool's Sony tag documentation: 0 Off; 1 Standard, 2 Advanced Auto, 3 Auto, 8–12 Advanced Lv1–5, 16–23 Lv1–8; other values are treated as unknown). It was present on all 264 Sony raws checked (98 bodies, DSLR-A100 to ILCE-7CR); the second tag of that name, `0xb04f`, only says off or on and is missing on the older bodies. With DRO on, the look keeps the colour fitted to the JPEG (matrix, hue/saturation table, chroma curve) and its tone curve is lowered, knot by knot, to Sony's curve without DRO wherever that is darker (`assets/camera-tone/sony-dro-off.json`, listed in `assets/ATTRIBUTION.md`). Past its last knot a curve continues as a shoulder whose rate comes from its last two knots, so the second-last knot is lowered only as far as keeps that shoulder at or below the fitted one; the result is never brighter than the fitted curve at any scene luminance. Sony's curve is quantile-matched, as one photo's tone is, over the pooled pairs of 12 CC0 raw.pixls.us ARWs shot with DRO off and the Standard creative style on 10 interchangeable-lens bodies (ILCE-3500, -5000, -6600, -7M3, -7S, -7CR, -9M2, ILCA-77M2, SLT-A57, NEX-F3; the file lists their URLs and SHA-256); pooled the same way from 55 private ILCE-7CR photos with DRO off, the curve differs by at most 1.2 L* between scene luminance 0.01 and 0.5. DRO only brightens, so the photo's own curve bounds the curve without it from above: the 1″ compacts map raw values darker (with DRO off, the DSC-RX100M2, -RX100M5 and -RX10M4 curves lie 8–16 L* below Sony's at scene luminance 0.18) and keep their own curve where it is the darker one. Photos with DRO off or unknown, and other makers, are unchanged.
+
+ Measured before → after (mean L* and CIE76 ΔE at 200 px): on the 24 ILCE-7CR photos with Lightroom references, the 8 with DRO on go from +6.4 L* (+4.4 to +10.5) to +0.8 L* (−0.4 to +1.9) against Lightroom's default, median ΔE 7.5 → 3.3; by Lightroom lightness band (L* 0–25, 25–50, 50–75, 75–100) they now differ from it by +2.3, +1.4, −0.8 and −3.0 L* (before +7.8, +9.4, +3.8, −0.8), like the 16 with DRO off (+2.5, −0.1, −2.0, −3.2), which render byte-identically. Of all 131 ILCE-7CR photos, the 56 fitted with DRO on now render 6.1 L* darker than their camera JPEGs on average (0.3 lighter before), as Lightroom renders the measured ones 5.7 L* darker; the 71 with DRO off are byte-identical. Of 116 CC0 Sony files from 100 bodies, 79 were shot with DRO on (71 of them Auto); the 57 changed of the 69 fitted ones render 2.1 L* darker on average (at most 10.8), the 12 others (six compacts, four ILCE-7M4 shots in Vivid, an ILCE-1 and an ILCE-7RM4, whose own curves are already darker) and all 35 DRO-off files are unchanged. The issue's ILCE-7M3 file goes from +0.8 to −6.5 L* against its camera JPEG. Applied to photos shot with DRO off, whose JPEG shows the camera's own curve, the rule would move the 68 fitted ILCE-7CR photos by −0.4 L* (median; at most −4.1) and 25 of 28 fitted CC0 ones by at most −1.8 L*; the exceptions are an ILCE-3500 (−4.7) and the cine bodies FX3 and FX30 (−7.2 and −5.1), whose picture profiles are not the Standard style.
+
+ Picture styles other than Standard are lowered to the Standard curve too where it is darker; Lightroom ignores them as well. Nikon Active D-Lighting, Canon Auto Lighting Optimizer and Fujifilm D-Range settings are likely to have the same effect and are not handled yet. Render cache version 23 redraws thumbnails; smart previews built before keep the brightened tone curve in their header until rebuilt.
+## Spectral camera matrices and the order of precedence
+
+For 52 camera models LightCraft has colour matrices fitted to the camera's measured spectral sensitivities
+(`crates/raw/src/spectral.rs`, generated into `spectral_table.rs` by `tools/spectral_camera_table.py` from
+[rawtoaces-data](https://github.com/AcademySoftwareFoundation/rawtoaces-data) v1.1.0, Apache-2.0): 22 Canon, 11 Nikon,
+11 Sony, 3 Fujifilm and 5 others, plus the other names rawtoaces-data lists for them (EOS Rebel T3i and Kiss X5 for
+the EOS 600D; bodies it lists as sharing a sensor, such as the GFX100S with the GFX 100). Per model the generator fits,
+under CIE Standard Illuminant A and under D65, a white-preserving 3×3 from white-balanced camera RGB to XYZ by the
+least mean CIE76 error over rawtoaces-data's 190 training reflectances, and writes DNG `ColorMatrix1/2` and
+`ForwardMatrix1/2`. The existing DNG colour model then interpolates them for the file's as-shot white, which the
+vendor white-balance multipliers give through the same matrices. No number comes from Adobe or from a raw decoder.
+
+A raw starts from the first of these that is available (`camera_preview::starting_colour`):
+
+1. the file's own colour matrices (DNG);
+2. a camera profile for its model (bundled or local), with tone and chroma fitted to the file's JPEG;
- 3. the model's spectral matrices, with LightCraft's default tone curve and no JPEG fit;
- 4. the file-local fit to its own JPEG described above;
++3. the file-local fit to its own JPEG described above;
++4. the model's spectral matrices, with LightCraft's default tone curve and no JPEG fit;
+5. the neutral fallback.
+
- A profiled model whose profile fit fails the gates (or that has no usable JPEG) takes its spectral matrices when it
- has them. White balance stays relative to the as-shot look for every raw without matrices of its own, as before, so
- the catalog's `Photo::relative_wb` and the controls are unchanged. `lightcraft-cli calibrate` still pools the
- uncorrected camera colour of covered models, so a profile can be fitted for them and then takes precedence.
++The spectral matrices only replace the neutral fallback: a photo whose JPEG fit is accepted (with or without a
++profile) renders exactly as before, and a covered model takes its spectral matrices only when that fit fails the
++gates or the file has no usable JPEG. White balance stays relative to the as-shot look for every raw without matrices
++of its own, as before, so the catalog's `Photo::relative_wb` and the controls are unchanged. `lightcraft-cli
++calibrate` still pools the uncorrected camera colour of covered models, so a profile can be fitted for them.
+
+No colour-difference measurement of the spectral matrices against the camera JPEG, or against the file-local fit,
- is recorded here yet. Moving step 3 after step 4 is a one-line change in `starting_colour`.
++is recorded here yet. Whether they should come before the file-local fit for these models is open until one is.
+
## Nikon crop and preview colour metadata
Nikon maker-note `CropArea` (0x0045) supplies the default `[left, top, width, height]` crop. The decoder validates the rectangle against the active sensor area and falls back to that area for missing or invalid values. The CFA origin is unchanged; cropping follows demosaicing.
diff --cc docs/parity.md
index 67ab95d73,ad82e3d25..3860e0439
--- a/docs/parity.md
+++ b/docs/parity.md
@@@ -296,7 -297,7 +297,7 @@@ under the UI command that wraps it (a n
| LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome |
| LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | |
| LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | |
- | LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF and Samsung SRW get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10), kept with smart previews (see docs/camera-preview-colour.md); 52 models without a profile start from DNG-style matrices fitted to their measured spectral sensitivities instead of the file-local fit (`crates/raw/src/spectral.rs`, rawtoaces-data v1.1.0, Apache-2.0, aliases included; not yet measured against the camera JPEG); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25) (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today |
-| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF, Samsung SRW and Olympus ORF get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10; bundled ILCE-7CR from 125 photos: 26 held-out 4.07 → 3.29, and every one of the 131 photos accepted, 8 of which fell back to neutral without it), kept with smart previews (see docs/camera-preview-colour.md); Sony ARWs shot with DRO on (maker note `0xb025`, on by default) keep that colour but their tone curve is lowered to Sony's curve without DRO, whose lifted shadows are in the camera JPEG only (issue #244: 8 ILCE-7CR photos +6.4 → +0.8 L* from Lightroom's default, median ΔE 7.5 → 3.3); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25); ORF: the 3 decodable corpus files (E-1, E-400, XZ-2) accepted (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today |
++| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/raw/src/profile.rs`, `crates/raw/src/gaintable.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices and the camera-profile look they carry (`ProfileHueSatMap`, `ProfileLookTable`, `ProfileToneCurve`, read from the file at run time per DNG spec ch. 6 — Lightroom-converted DNGs rendered flat and muted without them, issue #138). A DNG's `ProfileGainTableMap` / `ProfileGainTableMap2` (Apple ProRAW's local tone map; both versions, DNG 1.7 precedence, the file's byte order with a big-endian fallback) is read and written back by DNG export and not rendered by default, as Lightroom Classic renders ProRAW without it (measured on one CC0 iPhone 12 Pro ProRAW); the Profile option "Camera local tone mapping" (`profile.camera_local_tone`, per photo, off by default, `crates/pipeline/src/local_tone.rs`) renders it as the camera does, per output pixel in the scene-linear stage (CPU, also under the GPU path); Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3 and CR2, Pentax PEF, Samsung SRW and Olympus ORF get a guarded file-local fit to their own embedded JPEG — chromaticity matrix + hue/saturation/value table + tone and chroma curves, relative WB — or, with a bundled or local per-model profile pooled from many photos (`lightcraft-cli calibrate`, ARW, NEF and RAF so far, `crates/engine/src/camera_profiles.rs`), the profile's colour and only their own tone/chroma curves (on 29 held-out ILCE-7M4 photos mean ΔE vs the camera JPEG 3.54 → 3.10; bundled ILCE-7CR from 125 photos: 26 held-out 4.07 → 3.29, and every one of the 131 photos accepted, 8 of which fell back to neutral without it), kept with smart previews (see docs/camera-preview-colour.md); Sony ARWs shot with DRO on (maker note `0xb025`, on by default) keep that colour but their tone curve is lowered to Sony's curve without DRO, whose lifted shadows are in the camera JPEG only (issue #244: 8 ILCE-7CR photos +6.4 → +0.8 L* from Lightroom's default, median ΔE 7.5 → 3.3); 52 models without a profile whose JPEG fit fails (or that have no usable JPEG) start from DNG-style matrices fitted to their measured spectral sensitivities instead of the neutral fallback (`crates/raw/src/spectral.rs`, rawtoaces-data v1.1.0, Apache-2.0, aliases included; not yet measured against the camera JPEG); ARW: on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; NEF: 13 of 13 decodable samples (D750, D780, D850, D7500, Z 50 and local D7500 shots) accepted, ΔE 13–46 → 3–8 (issue #150); RW2 / RWL: 140 of 174 public samples accepted, from 93 of the 114 bodies with a preview, median ΔE 5.0 with matching lightness (13 of the 14 corpus files with a preview: ΔE 9.6–27 → 2.8–8.9; most rejections are compacts and kit zooms whose camera JPEG is distortion-corrected); a fit that fails the gates on all pixels is retried away from edges, where a lens-corrected camera JPEG still lines up (issue #232: the six distortion-corrected public ILCE-7RM2 samples ΔE 16.6–16.9 → 3.7–3.8, the corpus GX80 19.1 → 7.9); RAF: bundled X-H2S/X-T4 profiles from 201/545 photos; on 150 withheld files mean ΔE improves slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); 23 of 24 references from 13 bodies accepted, including the dark X-H2S samples (8.74 → 2.25); ORF: the 3 decodable corpus files (E-1, E-400, XZ-2) accepted (`docs/camera-preview-colour.md`); this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today |
| LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched |
| LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | |
| LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF |
diff --cc docs/whats-new.md
index 61c554767,8ae60a7e4..a0edaef77
--- a/docs/whats-new.md
+++ b/docs/whats-new.md
@@@ -40,10 -157,10 +157,15 @@@
- Samsung SRW files without compression now open as raws: NX5, NX10, NX11, NX20, NX200, NX210, NX1000, NX1100, EX1
and WB2000. The compressed ones (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini) still open from their
camera JPEG. Photos already imported pick the change up on Reload.
- - Raws from 52 camera models (22 Canon, 11 Nikon, 11 Sony, 3 Fujifilm and 5 others, plus their other names) start
- from colour matrices fitted to the camera's measured spectral sensitivities, from the Academy Software
- Foundation's rawtoaces-data. A camera profile for the model still comes first. See
++- Raws from 52 camera models (22 Canon, 11 Nikon, 11 Sony, 3 Fujifilm and 5 others, plus their other names) whose
++ camera JPEG can't be fitted no longer open with the neutral fallback: they start from colour matrices fitted to the
++ camera's measured spectral sensitivities, from the Academy Software Foundation's rawtoaces-data. A camera profile
++ and the fit to the photo's own JPEG still come first, so photos that had a fitted look keep it. See
+ [camera preview colour](camera-preview-colour.md#spectral-camera-matrices-and-the-order-of-precedence).
+ - Canon CR2 raws no longer open too dark with blocked-up shadows: the black level was measured over border columns
+ that light already reaches (left of the image area Canon records), 4 % of the range too high on the EOS 6D. It is
+ now measured on the masked columns alone (8 corpus bodies, EOS 40D to 5DS R). On a 6D photo the starting render
+ moved from ΔE00 7.6 to 5.2 against Lightroom's.
- Canon CR2 and Pentax PEF raws get the same starting look fitted to the camera's own JPEG as ARW, NEF, RW2, RAF
and CR3, instead of opening flat and desaturated (issue #310). Photos already imported pick it up when re-rendered.
- JPEG XL compressed DNGs (DNG 1.7) now open: lossless tiles decode sample for sample (checked on synthetic files);