LightCraft: An open-source non-destructive photo editor written by scratch in pure Rust
LR-MASK-SKY: AI masks use DNG semantic mattes (iPhone ProRAW sky matte)
Exalted5373ed59d78e837e
diff --git a/ROADMAP.md b/ROADMAP.mdindex 26acc1765..2ad5884ea 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -25,7 +25,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).| **Feature checklist** | 79.7% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P0: full-resolution zoom detail; P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim || **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) || **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |-| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |+| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics, or the photo's own segmentation mattes when it ships them (DNG semantic masks, e.g. iPhone ProRAW's sky matte) | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) || **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering || **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ || **HDR & video** | 0% | | HDR edit/display/export; video play/trim/edit/export |diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rsindex bad30e149..8e14c68f7 100644--- a/crates/engine/src/files.rs+++ b/crates/engine/src/files.rs@@ -245,7 +245,8 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<raw.opcodes.list3.retain(|op| !op.is_lens_correction());let xy = lightcraft_raw::color::as_shot_white_xy(&raw);let t = lightcraft_raw::color::camera_transform(&raw, xy);- let camera_look = crate::camera_preview::fit_preview(&raw, &bytes, &t);+ // the source's segmentation mattes (DNG semantic masks), read while the preview is fitted+ let (camera_look, mattes) = rayon::join(|| crate::camera_preview::fit_preview(&raw, &bytes, &t), || dng_mattes(&bytes, &raw));drop(bytes);// Previews and thumbnails bin the mosaic straight to (about) the size they need; only// larger levels (exports, 1:1) demosaic the whole sensor.@@ -308,7 +309,7 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<let relative = crate::camera_preview::file_local_look(raw.format) && t.matrix_is_fallback;let camera_tone = camera_look.as_ref().map(|p| p.tone).or_else(|| raw.color.profile.tone_curve.as_ref().and_then(dng_tone_curve));let (temp, tint) = if relative { (6500.0, 0.0) } else { (temp.round(), tint.round()) };- return Ok((img, SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_tone }));+ return Ok((img, SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_tone, mattes }));}let d = lightcraft_codecs::decode(&bytes, fit_box(max_edge)).map_err(|e| e.to_string())?;drop(bytes);@@ -317,6 +318,42 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result<Ok((img.into_oriented(Orientation::from_exif(d.orientation)), SourceInfo::default()))}+/// The semantic masks of a DNG that AI masks understand, over the developed image (default crop,+/// oriented like it). Person mattes without a confidently selected pixel are left out (the+/// Subject mask then falls back to its heuristic); a sky matte counts even when empty (no sky).+fn dng_mattes(bytes: &[u8], raw: &lightcraft_raw::RawImage) -> Option<Arc<lightcraft_pipeline::masks::Mattes>> {+ use lightcraft_pipeline::masks::{MatteKind, Mattes};+ if raw.format != lightcraft_raw::RawFormat::Dng {+ return None;+ }+ let mut mattes = Mattes::default();+ for m in lightcraft_raw::semantic_masks(bytes) {+ let Some(kind) = matte_kind(&m.name) else { continue };+ let Some(img) = m.developed(raw.active_area, raw.crop) else { continue };+ if kind != MatteKind::Sky && !img.data.iter().any(|&v| v >= 128) {+ continue;+ }+ mattes.push(kind, img.into_oriented(raw.orientation));+ }+ (!mattes.is_empty()).then(|| Arc::new(mattes))+}++/// What a semantic mask selects, by its `SemanticName`: Apple's (iPhone ProRAW) are named after+/// their auxiliary image types, `urn:com:apple:photo:<year>:aux:<type>`. Other names: `None`.+fn matte_kind(name: &str) -> Option<lightcraft_pipeline::masks::MatteKind> {+ use lightcraft_pipeline::masks::MatteKind::*;+ let kind = name.strip_prefix("urn:com:apple:photo:")?.rsplit(':').next()?;+ Some(match kind {+ "semanticskymatte" => Sky,+ "semanticskinmatte" => Skin,+ "semantichairmatte" => Hair,+ "semanticteethmatte" => Teeth,+ "semanticglassesmatte" => Glasses,+ "portraiteffectsmatte" => Person,+ _ => return None,+ })+}+/// A DNG `ProfileToneCurve` (linear in, linear out, 1.0 = white after exposure compensation) as the/// finish stage's camera tone curve: 32 knots, log-spaced over 12 stops below white; above white/// the camera tone's shoulder continues it.@@ -699,6 +736,70 @@ mod tests {assert!((mean(&dim) / mean(&before) - 0.5).abs() < 0.02, "{} vs {}", mean(&dim), mean(&before));}+ /// iPhone ProRAW-style semantic masks drive the Sky mask: Apple's sky matte (here the right+ /// half of the sensor, i.e. the bottom of the photo once turned 90° clockwise — where the sky+ /// heuristic would never look) is used; a matte with an unknown name is ignored.+ #[test]+ fn dng_sky_matte_drives_the_sky_mask() {+ use lightcraft_develop::{DevelopSettings, LocalAdjustments, Mask, MaskComponent, MaskOp, MaskShape};+ use lightcraft_pipeline::{RenderRequest, render};+ use lightcraft_tiff::tags::{self as t, photometric};+ use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, Value};+ let (w, h) = (16u32, 8u32);+ let image = |ifd: &mut IfdBuilder, w: u32, h: u32, cpp: u16, bits: u16, data: Vec<u8>| {+ ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));+ ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));+ ifd.set(t::BITS_PER_SAMPLE, Value::Short(vec![bits; cpp as usize]));+ ifd.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![cpp]));+ ifd.set(t::COMPRESSION, Value::Short(vec![1]));+ ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![data] });+ };+ let mut raw = IfdBuilder::new();+ raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));+ raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::LINEAR_RAW]));+ image(&mut raw, w, h, 3, 16, 12000u16.to_le_bytes().repeat((w * h * 3) as usize));+ let matte = |name: &str, right: bool| {+ let mut m = IfdBuilder::new();+ m.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![t::SUBFILE_SEMANTIC_MASK]));+ m.set(t::PHOTOMETRIC, Value::Short(vec![photometric::MASK]));+ m.set(t::SEMANTIC_NAME, Value::Ascii(name.into()));+ image(&mut m, w / 2, h / 2, 1, 8, (0..w / 2 * h / 2).map(|i| if (i % (w / 2) >= w / 4) == right { 255 } else { 0 }).collect());+ m+ };+ let mut ifd0 = IfdBuilder::new();+ ifd0.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![1]));+ ifd0.set(t::DNG_VERSION, Value::Byte(vec![1, 6, 0, 0]));+ ifd0.set(t::MAKE, Value::Ascii("Apple".into()));+ ifd0.set(t::ORIENTATION, Value::Short(vec![6]));+ ifd0.set(t::COLOR_MATRIX_1, Value::SRational(vec![(1, 1), (0, 1), (0, 1), (0, 1), (1, 1), (0, 1), (0, 1), (0, 1), (1, 1)]));+ ifd0.set(t::CALIBRATION_ILLUMINANT_1, Value::Short(vec![21]));+ ifd0.set(t::PHOTOMETRIC, Value::Short(vec![photometric::RGB]));+ image(&mut ifd0, 4, 2, 3, 8, vec![128; 24]);+ ifd0.add_sub_ifd(raw);+ ifd0.add_sub_ifd(matte("urn:com:apple:photo:2020:aux:semanticskymatte", true));+ ifd0.add_sub_ifd(matte("urn:com:apple:photo:2020:aux:semanticsomethingelse", false));+ let bytes = TiffWriter::new(ByteOrder::Little, false).write(&[ifd0]).unwrap();++ let (img, info) = load_bytes(&bytes, 64).unwrap();+ assert_eq!((img.width, img.height), (8, 16));+ assert!(info.mattes.is_some());+ let sky = Mask {+ components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::Sky }],+ adjust: LocalAdjustments { exposure: -3.0, ..Default::default() },+ ..Default::default()+ };+ let s = DevelopSettings { masks: vec![sky], ..Default::default() };+ let req = RenderRequest::fit(8, 16);+ let green = |s: &DevelopSettings, info: &SourceInfo, y: usize| render(&img, info, s, &req).image.get(4, y)[1] as i32;+ let plain = DevelopSettings::default();+ assert_eq!(green(&s, &info, 2), green(&plain, &info, 2), "no sky at the top");+ assert!(green(&s, &info, 13) + 40 < green(&plain, &info, 13), "the sky at the bottom is darkened");+ // without the matte the heuristic looks at the top of the frame instead+ let heuristic = SourceInfo { mattes: None, ..info.clone() };+ assert!(green(&s, &heuristic, 2) < green(&plain, &heuristic, 2));+ assert_eq!(green(&s, &heuristic, 13), green(&plain, &heuristic, 13));+ }+/// Apple ProRAW: like Lightroom Classic, the default render ignores the file's/// `ProfileGainTableMap` (Apple's local tone mapping); a map that would double every pixel/// changes nothing.diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rsindex 1ed89e473..56ad62d4a 100644--- a/crates/engine/src/media.rs+++ b/crates/engine/src/media.rs@@ -143,7 +143,7 @@ impl DecodedSource {/// What to render these pixels against: the decoder's facts, else `header` (the catalog's)/// with any stored camera tone curve.pub fn info_or(&self, header: SourceInfo) -> SourceInfo {- self.info.unwrap_or(SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header })+ self.info.clone().unwrap_or_else(|| SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header })}}@@ -253,8 +253,9 @@ fn rendered_budget(share: usize) -> usize {RENDERED_MEM_BYTES.min(share / 4)}-fn source_bytes(img: &Rgb32f) -> usize {- img.data.len() * 12 + 64 + std::mem::size_of::<SourceInfo>()+fn source_bytes(s: &DecodedSource) -> usize {+ let mattes = s.info.as_ref().and_then(|i| i.mattes.as_ref()).map_or(0, |m| m.bytes());+ s.image.data.len() * 12 + 64 + std::mem::size_of::<SourceInfo>() + mattes}impl MediaCache {@@ -319,8 +320,8 @@ impl MediaCache {let Some((tick, which)) = candidates.into_iter().flatten().min() else { break };let freed = match which {0 => self.thumbs.pop_oldest(),- 1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1.image)),- 2 => self.full.take().map(|e| source_bytes(&e.1.image)),+ 1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1)),+ 2 => self.full.take().map(|e| source_bytes(&e.1)),_ => self.rendered.evict_oldest(),};match freed {@@ -372,7 +373,7 @@ impl MediaCache {let tick = lightcraft_preview::next_tick();match level {SourceLevel::Thumb => {- let cost = source_bytes(&img.image);+ let cost = source_bytes(&img);self.thumbs.insert(id, img, cost);}SourceLevel::Preview => {@@ -415,8 +416,8 @@ impl MediaCache {/// Decoded sources held: (thumbnail level, preview level, full size).pub fn usage(&self) -> (crate::memory::Usage, crate::memory::Usage, crate::memory::Usage) {use crate::memory::Usage;- let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1.image)).sum());- let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1.image))).unwrap_or_default();+ let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1)).sum());+ let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1))).unwrap_or_default();(Usage::new(self.thumbs.len(), self.thumbs.cost()), previews, full)}@@ -1210,20 +1211,21 @@ mod tests {let mut s = crate::Session::with_demo();let id = s.active().unwrap();let mut job = s.render_job(id, 64, 64, false, true).unwrap();+ let loaded_info = info.clone();job.source = SourceRef::File {path: "synthetic.arw".into(),max_edge: 64,loader: Some(Arc::new(move |_, _| {let mut image = Rgb32f::new(64, 64);image.data.fill([0.1; 3]);- Ok((image, info))+ Ok((image, loaded_info.clone()))})),fallback: None,};job.info = SourceInfo::default(); // Header facts cannot override decoder facts.job.settings = Arc::new(DevelopSettings::default());let r = job.clone().run();- assert_eq!(r.loaded.as_ref().unwrap().info, Some(info));+ assert_eq!(r.loaded.as_ref().unwrap().info.as_ref(), Some(&info));let expected = lightcraft_pipeline::render(&r.loaded.as_ref().unwrap().image, &info, &job.settings, &job.request);assert_eq!(r.rendered.as_ref().unwrap().image.data, expected.image.data);s.accept(&r);diff --git a/crates/gpu/src/render.rs b/crates/gpu/src/render.rsindex 60ffd8065..7e7ac8a3e 100644--- a/crates/gpu/src/render.rs+++ b/crates/gpu/src/render.rs@@ -717,7 +717,7 @@ pub fn render(Err(m) => {let img = host.lin.get_or_insert_with(|| Arc::new(cx.read_rgb(&lin, w, h))).clone();let l = host.log_l.get_or_insert_with(|| Arc::new(cx.read_plane(&prep.log_l, w, h))).clone();- lightcraft_pipeline::masks::evaluate_one(m, &plan.frame, w, h, &img, &l, s.light.exposure as f32)+ lightcraft_pipeline::masks::evaluate_one(m, &plan.frame, w, h, &img, &l, s.light.exposure as f32, plan.mattes.as_deref())}});lightcraft_pipeline::visualize::apply(&mut image, req.overlay, &plan, overlay_mask.as_ref());@@ -1018,7 +1018,7 @@ fn masks(cx: &mut Cx<'_>, lin: &Buf, prep: &Prep, plan: &Plan<'_>, host: &mut Ho// no kernel: evaluate on the CPUlet img = host.lin.get_or_insert_with(|| Arc::new(cx.read_rgb(lin, w, h))).clone();let l = host.log_l.get_or_insert_with(|| Arc::new(cx.read_plane(&prep.log_l, w, h))).clone();- let mut v = lightcraft_pipeline::masks::shape_alpha(&comp.shape, frame, w, h, &img, &l, ev);+ let mut v = lightcraft_pipeline::masks::shape_alpha(&comp.shape, frame, w, h, &img, &l, ev, plan.mattes.as_deref());if comp.invert {v.data.iter_mut().for_each(|x| *x = 1.0 - *x);}diff --git a/crates/pipeline/src/lib.rs b/crates/pipeline/src/lib.rsindex 057fb42fa..76e80ed22 100644--- a/crates/pipeline/src/lib.rs+++ b/crates/pipeline/src/lib.rs@@ -55,7 +55,7 @@ use lightcraft_raster::{Histogram, Plane, Rgb32f, Rgba8, par_rows};pub use tone::ToneMap;/// Facts about the source the settings are interpreted against.-#[derive(Clone, Copy, Debug, PartialEq)]+#[derive(Clone, Debug, PartialEq)]pub struct SourceInfo {/// Lens corrections embedded in the file (DNG opcodes), relative to the EXIF-oriented source.pub lens: Option<lightcraft_develop::EmbeddedLens>,@@ -67,11 +67,13 @@ pub struct SourceInfo {/// No measured camera illuminant: WB adjustments are relative to the camera's rendered look.pub relative_wb: bool,pub camera_tone: Option<tone::CameraTone>,+ /// Segmentation mattes stored in the file (DNG semantic masks): AI masks use them.+ pub mattes: Option<Arc<masks::Mattes>>,}impl Default for SourceInfo {fn default() -> Self {- Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, relative_wb: false, camera_tone: None }+ Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, relative_wb: false, camera_tone: None, mattes: None }}}@@ -332,6 +334,8 @@ pub struct Plan<'a> {/// [`spots::window_for_reads`]); this is the requested window inside the rendered one, which/// the result is cut to.pub keep: Option<PixelWindow>,+ /// The source's segmentation mattes ([`SourceInfo::mattes`]).+ pub mattes: Option<Arc<masks::Mattes>>,}/// Long edge of the small render a windowed render estimates the whole frame's airlight from.@@ -392,7 +396,7 @@ pub fn plan<'a>(src: &Rgb32f, info: &SourceInfo, s: &'a DevelopSettings, req: &R[d.nr_luminance, d.nr_detail, d.nr_color, d.nr_color_detail, d.nr_color_smoothness].map(f64::to_bits),src_long,));- Plan { settings, frame, w, h, px_per_long, src_long, geo, lin_key, eyes, fixed_air, keep }+ Plan { settings, frame, w, h, px_per_long, src_long, geo, lin_key, eyes, fixed_air, keep, mattes: info.mattes.clone() }}/// Whether the scene-linear stage needs work only the CPU does (defringe, spot removal).@@ -474,7 +478,7 @@ fn render_impl(src: Src<'_>, info: &SourceInfo, s: &DevelopSettings, req: &RendeSome(p) if p.key == lin_key => p,_ => local::Planes { key: lin_key, ..Default::default() },};- let mut prep = local::prepare(lin.clone(), s, frame, px_per_long, req.quality, &mut planes);+ let mut prep = local::prepare(lin.clone(), s, frame, px_per_long, req.quality, &mut planes, plan.mattes.as_deref());if let Some(a) = plan.fixed_air {prep.air = a;}@@ -543,7 +547,7 @@ fn overlay_alpha(o: Overlay, plan: &Plan<'_>, prep: &Prepared) -> Option<Plane>return Some(e.alpha.clone());}let ev = plan.settings.light.exposure as f32;- Some(masks::evaluate_one(m, &plan.frame, plan.w, plan.h, &prep.img, &prep.log_l, ev))+ Some(masks::evaluate_one(m, &plan.frame, plan.w, plan.h, &prep.img, &prep.log_l, ev, plan.mattes.as_deref()))}/// Convenience: render a before/after pair side by side is up to the UI; this renders "before"diff --git a/crates/pipeline/src/local.rs b/crates/pipeline/src/local.rsindex 61ec84820..898e69cfd 100644--- a/crates/pipeline/src/local.rs+++ b/crates/pipeline/src/local.rs@@ -279,7 +279,16 @@ pub fn plane_sigmas(s: &DevelopSettings, px_per_long: f64, q: Quality) -> PlaneS/// The spatial planes the per-pixel stage needs for `img` (white-balanced, before exposure),/// reusing whatever `planes` already holds for it. Missing planes are computed side by side (each/// one alone scales poorly: the guided filters work on small subsampled grids).-pub(crate) fn prepare(img: Arc<Rgb32f>, s: &DevelopSettings, frame: &Frame, px_per_long: f64, q: Quality, planes: &mut Planes) -> Prepared {+#[allow(clippy::too_many_arguments)]+pub(crate) fn prepare(+ img: Arc<Rgb32f>,+ s: &DevelopSettings,+ frame: &Frame,+ px_per_long: f64,+ q: Quality,+ planes: &mut Planes,+ mattes: Option<&masks::Mattes>,+) -> Prepared {let log_l = planes.log_l.get_or_insert_with(|| Arc::new(timed("log_l", || img.map(log_lum)))).clone();let PlaneSigmas { base: base_sigma, clarity: clarity_sigma, texture: texture_sigma, dark: dark_sigma, chroma: chroma_sigma } =plane_sigmas(s, px_per_long, q);@@ -324,7 +333,7 @@ pub(crate) fn prepare(img: Arc<Rgb32f>, s: &DevelopSettings, frame: &Frame, px_pNone => (None, 1.0),};let ev = s.light.exposure as f32;- let masks = timed("masks", || masks::evaluate(&s.masks, frame, img.width, img.height, &img, &log_l, ev));+ let masks = timed("masks", || masks::evaluate(&s.masks, frame, img.width, img.height, &img, &log_l, ev, mattes));Prepared { img, log_l, base, clarity_blur, texture_blur, dark, chroma_blur, air, masks, px_per_long }}diff --git a/crates/pipeline/src/masks.rs b/crates/pipeline/src/masks.rsindex aae261c5a..4777c8782 100644--- a/crates/pipeline/src/masks.rs+++ b/crates/pipeline/src/masks.rs@@ -2,14 +2,96 @@//!//! Shapes are defined in normalized oriented-image coordinates; sizes are in "long-edge units".//! Components combine with Add (max), Subtract (a·(1−c)) and Intersect (a·c).+//!+//! AI shapes (Sky, Subject, Background, People) use the segmentation [`Mattes`] the source carries+//! (DNG semantic masks, e.g. iPhone ProRAW's) when it has a matching one, else a heuristic.use lightcraft_develop::{BrushStroke, LocalAdjustments, Mask, MaskOp, MaskShape, SegMask};-use lightcraft_geom::Point;-use lightcraft_raster::{Plane, Rgb32f};+use lightcraft_geom::{Orientation, Point};+use lightcraft_raster::{Image, Plane, Rgb32f};use crate::for_rows;use crate::geometry::Frame;+/// What an embedded segmentation matte selects.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub enum MatteKind {+ Sky,+ /// A whole person (all people in the photo).+ Person,+ Skin,+ Hair,+ Teeth,+ Glasses,+}++/// Segmentation mattes that came with the source: 0..255 alpha planes, each covering the whole+/// (EXIF-oriented) source image at its own resolution. Several mattes of one kind (e.g. one per+/// person) are combined with max.+#[derive(Clone, Default, PartialEq)]+pub struct Mattes {+ mattes: Vec<(MatteKind, Image<u8>)>,+}++impl std::fmt::Debug for Mattes {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.debug_list().entries(self.mattes.iter().map(|(k, m)| (k, m.width, m.height))).finish()+ }+}++impl Mattes {+ /// Add a matte (empty ones are ignored).+ pub fn push(&mut self, kind: MatteKind, matte: Image<u8>) {+ if matte.width > 0 && matte.height > 0 && matte.data.len() == matte.width * matte.height {+ self.mattes.push((kind, matte));+ }+ }++ pub fn is_empty(&self) -> bool {+ self.mattes.is_empty()+ }++ /// Memory held (bytes).+ pub fn bytes(&self) -> usize {+ self.mattes.iter().map(|(_, m)| m.data.len()).sum()+ }++ fn of(&self, kinds: &[MatteKind]) -> Vec<&Image<u8>> {+ self.mattes.iter().filter(|(k, _)| kinds.contains(k)).map(|(_, m)| m).collect()+ }++ /// The mattes a shape uses instead of its heuristic (empty: none fits).+ fn for_shape(&self, shape: &MaskShape) -> Vec<&Image<u8>> {+ use MatteKind::*;+ // the whole person: a person matte, else the union of the parts+ let person = || {+ let p = self.of(&[Person]);+ if p.is_empty() { self.of(&[Skin, Hair, Teeth, Glasses]) } else { p }+ };+ match shape {+ MaskShape::Sky => self.of(&[Sky]),+ MaskShape::Subject => person(),+ MaskShape::People { parts, .. } => {+ let wanted: Vec<MatteKind> = parts+ .iter()+ .filter_map(|p| {+ let p = p.to_ascii_lowercase();+ [("skin", Skin), ("hair", Hair), ("teeth", Teeth), ("glasses", Glasses)]+ .into_iter()+ .find(|(n, _)| p.contains(n))+ .map(|(_, k)| k)+ })+ .collect();+ let whole =+ parts.is_empty() || parts.iter().any(|p| matches!(p.to_ascii_lowercase().as_str(), "person" | "entire person" | "entireperson"));+ // parts without a matte (lips, clothes, …) keep the heuristic+ if whole { person() } else { self.of(&wanted) }+ }+ _ => Vec::new(),+ }+ }+}+pub struct Evaluated {/// The mask's id ([`Mask::id`]).pub id: u32,@@ -23,22 +105,24 @@ fn smooth(e0: f32, e1: f32, x: f32) -> f32 {t * t * (3.0 - 2.0 * t)}-/// The visible masks with components, evaluated in order.-pub fn evaluate(masks: &[Mask], frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32) -> Vec<Evaluated> {+/// The visible masks with components, evaluated in order (`mattes`: the source's, see [`Mattes`]).+#[allow(clippy::too_many_arguments)]+pub fn evaluate(masks: &[Mask], frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32, mattes: Option<&Mattes>) -> Vec<Evaluated> {masks.iter().filter(|m| m.visible && !m.components.is_empty())- .map(|m| Evaluated { id: m.id, alpha: evaluate_one(m, frame, w, h, img, log_l, ev), adjust: m.adjust })+ .map(|m| Evaluated { id: m.id, alpha: evaluate_one(m, frame, w, h, img, log_l, ev, mattes), adjust: m.adjust }).collect()}/// The alpha plane of mask `m` (whether visible or not): its components combined, inverted and/// scaled by its amount.-pub fn evaluate_one(m: &Mask, frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32) -> Plane {+#[allow(clippy::too_many_arguments)]+pub fn evaluate_one(m: &Mask, frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32, mattes: Option<&Mattes>) -> Plane {let mut alpha = Plane::new(w, h);let mut first = true;for comp in &m.components {- let mut c = shape_alpha(&comp.shape, frame, w, h, img, log_l, ev);+ let mut c = shape_alpha(&comp.shape, frame, w, h, img, log_l, ev, mattes);if comp.invert {c.data.iter_mut().for_each(|v| *v = 1.0 - *v);}@@ -88,7 +172,13 @@ fn for_each_pos(frame: &Frame, w: usize, h: usize, out: &mut Plane, f: impl Fn(P});}-pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32) -> Plane {+/// The alpha plane of one component shape. AI shapes use the source's matching `mattes`, if any.+#[allow(clippy::too_many_arguments)]+pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32, mattes: Option<&Mattes>) -> Plane {+ let fitting = mattes.map(|m| m.for_shape(shape)).unwrap_or_default();+ if !fitting.is_empty() {+ return matte_alpha(&fitting, frame, w, h);+ }let mut out = Plane::new(w, h);let to_long = |p: Point| frame.norm_to_long(p);// `img`/`log_l` are before exposure: range masks select on the exposed values.@@ -180,7 +270,7 @@ pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &Rsmooth_plane(&mut out, 0.015 * frame_px(frame, w));}MaskShape::Background => {- let mut s = shape_alpha(&MaskShape::Subject, frame, w, h, img, log_l, ev);+ let mut s = shape_alpha(&MaskShape::Subject, frame, w, h, img, log_l, ev, mattes);s.data.iter_mut().for_each(|v| *v = 1.0 - *v);out = s;}@@ -252,6 +342,71 @@ fn smooth_plane(p: &mut Plane, sigma: f32) {*p = lightcraft_raster::blur::gaussian(p, sigma.max(0.5));}+/// The union (max) of `mattes` (0..255, over the EXIF-oriented source) at output resolution,+/// sampled like the image: user orientation, lens / perspective warp, crop and flips; area-averaged+/// first when the output is much smaller than the matte, bilinear otherwise.+fn matte_alpha(mattes: &[&Image<u8>], frame: &Frame, w: usize, h: usize) -> Plane {+ let mut out = Plane::new(w, h);+ let o2t = frame.out_to_oriented(w, h);+ let ppl = frame.px_per_long(w).max(1e-9);+ for m in mattes {+ let oriented =+ if frame.orient == Orientation::Normal { std::borrow::Cow::Borrowed(*m) } else { std::borrow::Cow::Owned(m.oriented(frame.orient)) };+ // matte px per output px+ let k = oriented.width as f64 / frame.ow * frame.ow.max(frame.oh) / ppl;+ let base = box_down(&oriented, if k >= 2.0 { (k.floor() as usize).min(64) } else { 1 });+ let (sx, sy) = ((base.width as f64 / frame.ow) as f32, (base.height as f64 / frame.oh) as f32);+ for_rows(&mut out.data, w, |y, row| {+ for (x, v) in row.iter_mut().enumerate() {+ let s = match &frame.warp {+ Some(wp) => {+ let Some((_, s)) = wp.frame(&o2t, x, y) else {+ continue;+ };+ s+ }+ None => o2t.apply(Point::new(x as f64 + 0.5, y as f64 + 0.5)),+ };+ *v = v.max(bilinear(&base, s.x as f32 * sx, s.y as f32 * sy) / 255.0);+ }+ });+ }+ out+}++/// `m` area-averaged by `f × f` (0..255 values kept); `f == 1` converts only.+fn box_down(m: &Image<u8>, f: usize) -> Plane {+ let f = f.max(1);+ let (bw, bh) = (m.width.div_ceil(f).max(1), m.height.div_ceil(f).max(1));+ let mut out = Plane::new(bw, bh);+ for_rows(&mut out.data, bw, |by, row| {+ let rows = by * f..((by + 1) * f).min(m.height);+ for (bx, v) in row.iter_mut().enumerate() {+ let cols = bx * f..((bx + 1) * f).min(m.width);+ let (mut sum, mut n) = (0u32, 0u32);+ for y in rows.clone() {+ for x in cols.clone() {+ sum += m.data.get(y * m.width + x).copied().unwrap_or(0) as u32;+ n += 1;+ }+ }+ *v = sum as f32 / n.max(1) as f32;+ }+ });+ out+}++/// Bilinear sample at continuous pixel coordinates (pixel centres at +0.5), edges extended.+fn bilinear(p: &Plane, x: f32, y: f32) -> f32 {+ let (fx, fy) = (x - 0.5, y - 0.5);+ let (x0, y0) = (fx.floor(), fy.floor());+ let (tx, ty) = (fx - x0, fy - y0);+ let (x0, y0) = (x0 as isize, y0 as isize);+ let a = p.get_clamped(x0, y0) + (p.get_clamped(x0 + 1, y0) - p.get_clamped(x0, y0)) * tx;+ let b = p.get_clamped(x0, y0 + 1) + (p.get_clamped(x0 + 1, y0 + 1) - p.get_clamped(x0, y0 + 1)) * tx;+ a + (b - a) * ty+}+/// A rough display mapping for colour picking (so samples taken on screen match).pub(crate) fn tonemap_for_select(c: [f32; 3]) -> [f32; 3] {c.map(|v| v / (1.0 + v))@@ -436,7 +591,7 @@ mod tests {let f = frame(100, 50);let img = Rgb32f::new(100, 50);let l = Plane::new(100, 50);- let a = shape_alpha(&MaskShape::Linear { start: Point::new(0.0, 0.0), end: Point::new(1.0, 0.0) }, &f, 100, 50, &img, &l, 0.0);+ let a = shape_alpha(&MaskShape::Linear { start: Point::new(0.0, 0.0), end: Point::new(1.0, 0.0) }, &f, 100, 50, &img, &l, 0.0, None);assert!(a.get(1, 25) > 0.99 && a.get(98, 25) < 0.01);assert!((a.get(50, 25) - 0.5).abs() < 0.05);}@@ -452,7 +607,7 @@ mod tests {MaskShape::Prompt { text: "left".into(), seg: Some(seg.clone()), detail: vec![], edge: 0.0 },] {for (w, h) in [(40, 20), (400, 200)] {- let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0);+ let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0, None);assert!(a.get(1, h / 2) > 0.99 && a.get(w - 2, h / 2) < 0.01, "{w}×{h}");if w >= 400 {let soft = (0..w).filter(|x| (0.05..0.95).contains(&a.get(*x, h / 2))).count();@@ -465,7 +620,7 @@ mod tests {d.crop.geometry.rect = lightcraft_geom::Rect::from_center(Point::new(0.75, 0.5), 0.5, 1.0);let f = Frame::new(40, 20, &d, true);let shape = MaskShape::Prompt { text: "left".into(), seg: Some(seg), detail: vec![], edge: 0.0 };- let a = shape_alpha(&shape, &f, 20, 20, &Rgb32f::new(20, 20), &Plane::new(20, 20), 0.0);+ let a = shape_alpha(&shape, &f, 20, 20, &Rgb32f::new(20, 20), &Plane::new(20, 20), 0.0, None);assert!(a.data.iter().all(|v| *v < 0.05));}@@ -478,7 +633,7 @@ mod tests {let patch = SegMask::from_logits_in(side, &l, [0.5, 0.0, 1.0, 1.0]);let shape = MaskShape::Object { hint: vec![Point::new(0.6, 0.5)], exclude: vec![], seg: Some(coarse), detail: vec![patch], edge: 0.0 };let (w, h) = (200, 100);- let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0);+ let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0, None);assert!(a.get(20, 50) < 0.01, "outside the patch: the coarse mask");assert!(a.get(120, 50) > 0.99, "inside the patch, its selected half");assert!(a.get(190, 50) < 0.01, "inside the patch, its unselected half");@@ -494,7 +649,7 @@ mod tests {let shape = MaskShape::Object { hint: vec![Point::new(0.2, 0.5)], exclude: vec![], seg: None, detail: vec![patch; 20_000], edge: 0.0 };let (w, h) = (64, 32);let t = std::time::Instant::now();- let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0);+ let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0, None);assert!(a.get(5, 16) > 0.99 && a.get(60, 16) < 0.01);assert!(t.elapsed() < std::time::Duration::from_secs(20), "{:?}", t.elapsed());}@@ -507,7 +662,7 @@ mod tests {let (w, h) = (400, 40);let soft_px = |edge: f64| {let shape = MaskShape::Prompt { text: "x".into(), seg: Some(seg.clone()), detail: vec![], edge };- let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0);+ let a = shape_alpha(&shape, &frame(w, h), w, h, &Rgb32f::new(w, h), &Plane::new(w, h), 0.0, None);assert!(a.get(2, h / 2) > 0.9 && a.get(w - 3, h / 2) < 0.1, "still the same selection at {edge}");(0..w).filter(|x| (0.1..0.9).contains(&a.get(*x, h / 2))).count()};@@ -524,7 +679,7 @@ mod tests {MaskShape::Prompt { text: "sky".into(), seg: None, detail: vec![], edge: 0.0 },MaskShape::Prompt { text: "sky".into(), seg: Some(SegMask { side: 4, data: "damaged!".into(), rect: None }), detail: vec![], edge: 0.0 },] {- assert!(shape_alpha(&shape, &f, 30, 30, &img, &l, 0.0).data.iter().all(|v| *v == 0.0));+ assert!(shape_alpha(&shape, &f, 30, 30, &img, &l, 0.0, None).data.iter().all(|v| *v == 0.0));}}@@ -534,10 +689,10 @@ mod tests {let img = Rgb32f::new(100, 100);let l = Plane::new(100, 100);let shape = MaskShape::Radial { center: Point::new(0.5, 0.5), rx: 0.2, ry: 0.2, angle: 0.0, feather: 20.0, invert: false };- let a = shape_alpha(&shape, &f, 100, 100, &img, &l, 0.0);+ let a = shape_alpha(&shape, &f, 100, 100, &img, &l, 0.0, None);assert!(a.get(50, 50) > 0.99 && a.get(5, 5) < 0.01);let m = Mask { components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: true, shape }], ..Default::default() };- let e = evaluate(&[m], &f, 100, 100, &img, &l, 0.0);+ let e = evaluate(&[m], &f, 100, 100, &img, &l, 0.0, None);assert!(e[0].alpha.get(50, 50) < 0.01);}@@ -552,7 +707,7 @@ mod tests {components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::Brush { strokes: vec![stroke, erase] } }],..Default::default()};- let e = evaluate(&[m], &f, 200, 100, &img, &l, 0.0);+ let e = evaluate(&[m], &f, 200, 100, &img, &l, 0.0, None);let a = &e[0].alpha;assert!(a.get(40, 50) > 0.9, "{}", a.get(40, 50));assert!(a.get(100, 50) < 0.05, "erased centre {}", a.get(100, 50));@@ -574,8 +729,8 @@ mod tests {..Default::default()};let comp = |auto| MaskShape::Brush { strokes: vec![stroke(auto)] };- let plain = shape_alpha(&comp(false), &f, w, h, &img, &l, 0.0);- let auto = shape_alpha(&comp(true), &f, w, h, &img, &l, 0.0);+ let plain = shape_alpha(&comp(false), &f, w, h, &img, &l, 0.0, None);+ let auto = shape_alpha(&comp(true), &f, w, h, &img, &l, 0.0, None);// without Auto Mask the brush spills over the edge; with it, it stays on the dark sideassert!(plain.get(105, 50) > 0.9, "{}", plain.get(105, 50));assert!(auto.get(105, 50) < 0.05, "spill {}", auto.get(105, 50));@@ -585,8 +740,8 @@ mod tests {// on a flat area Auto Mask paints like the plain brushlet flat = Rgb32f::from_fn(w, h, |_, _| [0.2; 3]);let fl = flat.map(crate::local::log_lum);- let a = shape_alpha(&comp(true), &f, w, h, &flat, &fl, 0.0);- let b = shape_alpha(&comp(false), &f, w, h, &flat, &fl, 0.0);+ let a = shape_alpha(&comp(true), &f, w, h, &flat, &fl, 0.0, None);+ let b = shape_alpha(&comp(false), &f, w, h, &flat, &fl, 0.0, None);assert!((a.get(94, 50) - b.get(94, 50)).abs() < 0.02, "{} vs {}", a.get(94, 50), b.get(94, 50));}@@ -609,7 +764,7 @@ mod tests {};let refined = Mask { refine: 100.0, ..soft.clone() };let step = |m: &Mask| {- let a = evaluate_one(m, &f, w, h, &img, &log_l, 0.0);+ let a = evaluate_one(m, &f, w, h, &img, &log_l, 0.0, None);a.data[40 * w + 63] - a.data[40 * w + 56]};let (plain, sharp) = (step(&soft), step(&refined));@@ -640,7 +795,7 @@ mod tests {}let alpha = |refine| {let shape = MaskShape::ColorRange { samples: vec![sample], refine };- let a = shape_alpha(&shape, &f, w, h, &img, &log_l, 0.0);+ let a = shape_alpha(&shape, &f, w, h, &img, &log_l, 0.0, None);(a.get(30, 20), a.get(90, 20))};for refine in [0.0, 50.0, 85.0] {@@ -653,4 +808,63 @@ mod tests {let (l, r) = alpha(100.0);assert!(l > 0.95 && r < 0.3, "refine 100: {l} vs {r}");}++ /// A half-resolution matte selecting the left `frac` of a `w × h` source.+ fn left_matte(w: usize, h: usize, frac: f64) -> Image<u8> {+ Image::from_fn(w / 2, h / 2, |x, _| if (x as f64 + 0.5) < frac * (w / 2) as f64 { 255 } else { 0 })+ }++ /// The Sky mask follows the source's sky matte where it has one (here: the bottom-left of a+ /// uniformly bright blue photo, where the heuristic sees no sky, and not its top right, where+ /// it does), through user orientation, at any output size; without one it keeps the heuristic.+ #[test]+ fn sky_uses_the_sources_matte() {+ let (w, h) = (200, 100);+ let img = Rgb32f::filled(w, h, [0.5, 0.7, 1.4]);+ let l = img.map(crate::local::log_lum);+ let mut mattes = Mattes::default();+ mattes.push(MatteKind::Sky, left_matte(w, h, 0.5));+ let f = frame(w, h);+ let heuristic = shape_alpha(&MaskShape::Sky, &f, w, h, &img, &l, 0.0, None);+ let matte = shape_alpha(&MaskShape::Sky, &f, w, h, &img, &l, 0.0, Some(&mattes));+ assert!(heuristic.get(150, 5) > 0.9 && heuristic.get(25, 90) < 0.1, "{} {}", heuristic.get(150, 5), heuristic.get(25, 90));+ assert!(matte.get(150, 5) < 0.01 && matte.get(25, 90) > 0.99, "{} {}", matte.get(150, 5), matte.get(25, 90));+ // mattes of other kinds leave the heuristic alone+ let mut hair = Mattes::default();+ hair.push(MatteKind::Hair, left_matte(w, h, 0.5));+ assert_eq!(shape_alpha(&MaskShape::Sky, &f, w, h, &img, &l, 0.0, Some(&hair)), heuristic);+ // rotated 90° clockwise, the source's left half is the top half; a preview and a larger+ // render agree+ let s = lightcraft_develop::DevelopSettings { orientation: Orientation::Rotate90, ..Default::default() };+ let f = Frame::new(w, h, &s, true);+ for (ow, oh) in [(100, 200), (25, 50)] {+ let small = Rgb32f::filled(ow, oh, [0.5, 0.7, 1.4]);+ let a = shape_alpha(&MaskShape::Sky, &f, ow, oh, &small, &small.map(crate::local::log_lum), 0.0, Some(&mattes));+ let at = |x: f64, y: f64| a.get((x * ow as f64) as usize, (y * oh as f64) as usize);+ assert!(at(0.5, 0.2) > 0.99 && at(0.5, 0.8) < 0.01, "{ow}×{oh}: {} {}", at(0.5, 0.2), at(0.5, 0.8));+ }+ }++ /// Subject, Background and People use person mattes: the whole person is the union of the+ /// parts; a People part without a matte keeps the heuristic.+ #[test]+ fn people_and_subject_use_person_mattes() {+ let (w, h) = (200, 100);+ let img = Rgb32f::filled(w, h, [0.2; 3]);+ let l = img.map(crate::local::log_lum);+ let mut mattes = Mattes::default();+ mattes.push(MatteKind::Skin, left_matte(w, h, 0.25));+ let right: Image<u8> = Image::from_fn(w / 2, h / 2, |x, _| if x >= 75 { 255 } else { 0 });+ mattes.push(MatteKind::Hair, right);+ let f = frame(w, h);+ let eval = |shape: &MaskShape| shape_alpha(shape, &f, w, h, &img, &l, 0.0, Some(&mattes));+ let subject = eval(&MaskShape::Subject);+ assert!(subject.get(10, 50) > 0.99 && subject.get(190, 50) > 0.99 && subject.get(100, 50) < 0.01);+ let background = eval(&MaskShape::Background);+ assert!(background.get(100, 50) > 0.99 && background.get(10, 50) < 0.01);+ let hair = eval(&MaskShape::People { person: 0, parts: vec!["Hair".into()] });+ assert!(hair.get(190, 50) > 0.99 && hair.get(10, 50) < 0.01);+ let lips = MaskShape::People { person: 0, parts: vec!["Lips".into()] };+ assert_eq!(eval(&lips), shape_alpha(&lips, &f, w, h, &img, &l, 0.0, None));+ }}diff --git a/crates/raw/src/lib.rs b/crates/raw/src/lib.rsindex b9bc029ad..82f1bdf8d 100644--- a/crates/raw/src/lib.rs+++ b/crates/raw/src/lib.rs@@ -2,7 +2,8 @@//!//! - [`probe`] recognises raw containers; [`decode`] turns a file into a [`RawImage`] (sensor data + everything//! needed to render it: CFA, black/white levels, active area, default crop, orientation, DNG colour tags,-//! opcode lists, [`Metadata`]); [`embedded_preview`] returns the largest embedded JPEG (or DNG 1.7 JPEG XL) preview.+//! opcode lists, [`Metadata`]); [`embedded_preview`] returns the largest embedded JPEG (or DNG 1.7 JPEG XL) preview;+//! [`semantic_masks`] reads a DNG's semantic masks (segmentation mattes, e.g. iPhone ProRAW's sky matte).//! - [`RawImage::normalized`] subtracts black, scales white to 1.0 and crops to the active area (applying DNG//! `OpcodeList1`/`OpcodeList2`); [`demosaic`] turns CFA data into camera-RGB [`Rgb32f`];//! [`RawImage::develop`] does all of it plus `OpcodeList3` and the default crop; [`RawImage::develop_binned`]@@ -36,6 +37,7 @@ pub mod ljpeg;pub mod opcodes;mod preview;pub mod profile;+pub mod semantic;mod tiffraw;mod unpack;mod vendor;@@ -48,6 +50,7 @@ pub use lightcraft_meta::Metadata;pub use lightcraft_raster::Rgb32f;pub use opcodes::{Opcode, OpcodeLists};pub use preview::{PreviewColorSpace, embedded_preview, embedded_preview_color_space};+pub use semantic::{SemanticMask, semantic_masks};use lightcraft_color::Xy;use lightcraft_tiff::{Tiff, TiffError};diff --git a/crates/raw/src/semantic.rs b/crates/raw/src/semantic.rsnew file mode 100644index 000000000..e9af87c08--- /dev/null+++ b/crates/raw/src/semantic.rs@@ -0,0 +1,136 @@+//! DNG semantic masks (DNG 1.6+, chapter "Semantic Masks"): single-channel segmentation mattes+//! stored in their own IFDs (NewSubFileType 10004.H, PhotometricInterpretation 52527) with a+//! `SemanticName`, an optional `SemanticInstanceID` and an optional `MaskSubArea`. iPhone ProRAW+//! stores e.g. a sky matte this way.+//!+//! [`semantic_masks`] reads them separately from [`crate::decode`] (only callers that use them+//! pay for them); malformed or unsupported mask IFDs are skipped.++use crate::Rect;+use crate::tiffraw::{Packing, read_image};+use crate::{RawData, Result};+use lightcraft_raster::Image;+use lightcraft_tiff::Tiff;+use lightcraft_tiff::tags::{self as t, photometric};++/// At most this many mask IFDs are read from one file.+pub const MAX_MASKS: usize = 16;+/// Masks with more samples than this are skipped (a mask is a low-resolution matte; this is+/// 8192 × 4096).+pub const MAX_MASK_SAMPLES: usize = 1 << 25;++/// Where a cropped mask sits in the uncropped mask (DNG `MaskSubArea`). The uncropped mask+/// (`full_width × full_height`) spans the main image's active area.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub struct MaskSubArea {+ pub top: u32,+ pub left: u32,+ pub full_width: u32,+ pub full_height: u32,+}++/// One semantic mask: `width × height` values, 0 = not part of the attribute, 65535 = fully part.+#[derive(Clone, Debug, PartialEq)]+pub struct SemanticMask {+ /// `SemanticName`, e.g. `urn:com:apple:photo:2020:aux:semanticskymatte`.+ pub name: String,+ pub instance_id: Option<String>,+ pub width: usize,+ pub height: usize,+ /// `None`: the mask covers the whole active area.+ pub sub_area: Option<MaskSubArea>,+ pub data: Vec<u16>,+}++/// The semantic masks of a DNG, in file order (none for other files). Only the first+/// [`MAX_MASKS`] mask IFDs are looked at, malformed ones included, so a file full of mask IFDs+/// costs at most that many decodes.+pub fn semantic_masks(bytes: &[u8]) -> Vec<SemanticMask> {+ let Ok(tiff) = Tiff::parse(bytes) else { return Vec::new() };+ tiff.all_ifds()+ .into_iter()+ .filter(|i| i.u16(t::PHOTOMETRIC) == Some(photometric::MASK) || i.u32(t::NEW_SUBFILE_TYPE) == Some(t::SUBFILE_SEMANTIC_MASK))+ .take(MAX_MASKS)+ .filter_map(|ifd| read_mask(bytes, &tiff, ifd).ok().flatten())+ .collect()+}++fn read_mask(bytes: &[u8], tiff: &Tiff, ifd: &lightcraft_tiff::Ifd) -> Result<Option<SemanticMask>> {+ let Some(name) = ifd.string(t::SEMANTIC_NAME).filter(|n| !n.is_empty()) else { return Ok(None) };+ let info = ifd.image()?;+ let (w, h) = (info.width as usize, info.height as usize);+ if info.samples_per_pixel != 1 || w.checked_mul(h).is_none_or(|n| n > MAX_MASK_SAMPLES) {+ return Ok(None);+ }+ let bits = info.bits() as u32;+ let data: Vec<u16> = match read_image(bytes, &info, tiff.order, Packing::Msb)? {+ RawData::U16(v) if (1..=16).contains(&bits) => {+ let max = ((1u32 << bits) - 1) as f32;+ if bits == 16 { v } else { v.into_iter().map(|s| (s as f32 / max * 65535.0).round().min(65535.0) as u16).collect() }+ }+ RawData::F32(v) => v.into_iter().map(|s| if s.is_finite() { (s.clamp(0.0, 1.0) * 65535.0).round() as u16 } else { 0 }).collect(),+ RawData::U16(_) => return Ok(None),+ };+ if data.len() != w * h {+ return Ok(None);+ }+ // the tag is ignored when the stored mask does not fit inside the uncropped one+ let sub_area = match ifd.u64s(t::MASK_SUB_AREA).as_deref() {+ Some(&[top, left, fw, fh]) => {+ let fits = fw > 0 && fh > 0 && left.saturating_add(w as u64) <= fw && top.saturating_add(h as u64) <= fh && fw.max(fh) <= u32::MAX as u64;+ fits.then_some(MaskSubArea { top: top as u32, left: left as u32, full_width: fw as u32, full_height: fh as u32 })+ }+ _ => None,+ };+ Ok(Some(SemanticMask { name, instance_id: ifd.string(t::SEMANTIC_INSTANCE_ID).filter(|s| !s.is_empty()), width: w, height: h, sub_area, data }))+}++impl SemanticMask {+ /// The mask value (0..1) at a continuous position of the uncropped mask (pixel centres at+ /// +0.5); 0 outside the stored (cropped) mask.+ pub fn value_at(&self, x: f64, y: f64) -> f32 {+ let (left, top) = self.sub_area.map_or((0.0, 0.0), |a| (a.left as f64, a.top as f64));+ let (x, y) = (x - left, y - top);+ if self.width == 0 || self.height == 0 || !(x >= 0.0 && y >= 0.0 && x <= self.width as f64 && y <= self.height as f64) {+ return 0.0;+ }+ let (fx, fy) = ((x - 0.5).max(0.0), (y - 0.5).max(0.0));+ let (x0, y0) = (fx.floor() as usize, fy.floor() as usize);+ let (tx, ty) = ((fx - x0 as f64) as f32, (fy - y0 as f64) as f32);+ let at = |x: usize, y: usize| {+ let (x, y) = (x.min(self.width - 1), y.min(self.height - 1));+ self.data.get(y * self.width + x).copied().unwrap_or(0) as f32 / 65535.0+ };+ let top = at(x0, y0) + (at(x0 + 1, y0) - at(x0, y0)) * tx;+ let bottom = at(x0, y0 + 1) + (at(x0 + 1, y0 + 1) - at(x0, y0 + 1)) * tx;+ top + (bottom - top) * ty+ }++ /// The mask over the developed image ([`crate::RawImage::develop`]: the default `crop` of the+ /// `active_area`, before orientation), at the mask's own resolution, as 0..255 alpha. `None`+ /// for an empty crop.+ pub fn developed(&self, active_area: Rect, crop: Rect) -> Option<Image<u8>> {+ let c = crop.clipped(active_area.width, active_area.height);+ if c.width == 0 || c.height == 0 || self.width == 0 || self.height == 0 {+ return None;+ }+ let (fw, fh) = self.sub_area.map_or((self.width, self.height), |a| (a.full_width as usize, a.full_height as usize));+ // uncropped-mask px per active-area px+ let (sx, sy) = (fw as f64 / active_area.width as f64, fh as f64 / active_area.height as f64);+ let ow = ((c.width as f64 * sx).round() as usize).clamp(1, self.width.max(fw));+ let oh = ((c.height as f64 * sy).round() as usize).clamp(1, self.height.max(fh));+ if ow.checked_mul(oh).is_none_or(|n| n > MAX_MASK_SAMPLES) {+ return None;+ }+ let (kx, ky) = (c.width as f64 / ow as f64, c.height as f64 / oh as f64);+ let mut out = Image::new(ow, oh);+ lightcraft_raster::par_rows(&mut out.data, ow, |y, row| {+ let ay = c.y as f64 + (y as f64 + 0.5) * ky;+ for (x, v) in row.iter_mut().enumerate() {+ let ax = c.x as f64 + (x as f64 + 0.5) * kx;+ *v = (self.value_at(ax * sx, ay * sy) * 255.0).round() as u8;+ }+ });+ Some(out)+ }+}diff --git a/crates/raw/tests/corpus.rs b/crates/raw/tests/corpus.rsindex 683381c40..53193f1f1 100644--- a/crates/raw/tests/corpus.rs+++ b/crates/raw/tests/corpus.rs@@ -300,6 +300,33 @@ fn corpus_dngs_carry_profile_looks() {eprintln!("{seen} DNGs with profile looks checked");}+/// iPhone ProRAW (`dng-apple-iphone12pro-proraw.dng`, iPhone 12 Pro, CC0 from raw.pixls.us, see+/// `cargo xtask corpus`) carries a lossy-JPEG sky matte at half resolution as a DNG semantic mask:+/// it decodes, and it is a sky. The matte is stored like the raw (landscape; the photo is shown+/// rotated 90° clockwise), so the sky is on its left and the lake and shore on its right.+#[test]+fn corpus_proraw_sky_matte() {+ let path = corpus_root().join("raw/dng-apple-iphone12pro-proraw.dng");+ let Ok(bytes) = std::fs::read(&path) else {+ eprintln!("skip: {} absent", path.display());+ return;+ };+ let t0 = Instant::now();+ let masks = lightcraft_raw::semantic_masks(&bytes);+ eprintln!("semantic masks read in {:.1} ms", t0.elapsed().as_secs_f64() * 1e3);+ let names: Vec<&str> = masks.iter().map(|m| m.name.as_str()).collect();+ assert_eq!(names, ["urn:com:apple:photo:2020:aux:semanticskymatte"]);+ let m = &masks[0];+ assert_eq!((m.width, m.height, m.sub_area), (2016, 1512, None));+ let cols = |x0: usize, x1: usize| {+ let sum: f64 = (0..m.height).flat_map(|y| m.data[y * m.width + x0..y * m.width + x1].iter()).map(|&v| v as f64 / 65535.0).sum();+ sum / ((x1 - x0) * m.height) as f64+ };+ let (left, right) = (cols(0, 100), cols(1916, 2016));+ eprintln!("sky matte: left columns {left:.3}, right columns {right:.3}");+ assert!(left > 0.5 && right < 0.02, "left {left}, right {right}");+}+/// Issue #148: Sony ARWs from before ~2017 carry no plain white-balance, black-level or crop tags in the raw IFD./// White balance comes from the maker note's enciphered `Tag2010`, the black level from the encrypted `SR2SubIFD`/// and the crop from `FullImageSize`; without them the RX100 III opened bright green. Expected values: the blackdiff --git a/crates/raw/tests/dng.rs b/crates/raw/tests/dng.rsindex 9cbbb0ea3..02d4f82a3 100644--- a/crates/raw/tests/dng.rs+++ b/crates/raw/tests/dng.rs@@ -824,3 +824,130 @@ fn gain_table_map_byte_orders() {assert_eq!(img.color.profile.gain_table_map.map(|m| m.gains), read.then(|| vec![1.0, 2.0]), "{data:?} payload, {file:?} file");}}++/// A semantic mask IFD (8-bit, uncompressed) named `name`.+fn mask_ifd(w: usize, h: usize, name: &str, px: Vec<u8>) -> IfdBuilder {+ let mut m = IfdBuilder::new();+ m.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![t::SUBFILE_SEMANTIC_MASK]));+ m.set(t::IMAGE_WIDTH, Value::Long(vec![w as u32]));+ m.set(t::IMAGE_LENGTH, Value::Long(vec![h as u32]));+ m.set(t::BITS_PER_SAMPLE, Value::Short(vec![8]));+ m.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![1]));+ m.set(t::PHOTOMETRIC, Value::Short(vec![photometric::MASK]));+ m.set(t::COMPRESSION, Value::Short(vec![1]));+ if !name.is_empty() {+ m.set(t::SEMANTIC_NAME, Value::Ascii(name.into()));+ }+ m.set_image(ImageData::Strips { rows_per_strip: h as u32, strips: vec![px] });+ m+}++/// Semantic masks: a cropped mask is placed by its `MaskSubArea` (zero outside it), mapped onto the+/// developed image (default crop of the active area), and doesn't disturb the raw decode.+#[test]+fn semantic_masks_are_read_and_placed() {+ use lightcraft_raw::semantic::MaskSubArea;+ for order in [ByteOrder::Little, ByteOrder::Big] {+ let bytes = profile_dng(order, |ifd0| {+ // a 4 × 2 crop at (top 1, left 2) of an 8 × 4 mask over the 8 × 6 image+ let mut sky = mask_ifd(4, 2, "sky", vec![10, 20, 30, 40, 50, 60, 70, 255]);+ sky.set(t::MASK_SUB_AREA, Value::Long(vec![1, 2, 8, 4]));+ ifd0.add_sub_ifd(sky);+ let mut skin = mask_ifd(2, 2, "skin", vec![0, 255, 255, 0]);+ skin.set(t::SEMANTIC_INSTANCE_ID, Value::Ascii("person_a".into()));+ ifd0.add_sub_ifd(skin);+ });+ let raw = decode(&bytes).unwrap();+ assert_eq!((raw.width, raw.height, raw.cpp), (8, 6, 3), "the masks are not the raw image");+ let masks = lightcraft_raw::semantic_masks(&bytes);+ assert_eq!(masks.len(), 2);+ let (sky, skin) = (&masks[0], &masks[1]);+ assert_eq!((sky.name.as_str(), sky.instance_id.as_deref(), sky.width, sky.height), ("sky", None, 4, 2));+ assert_eq!(sky.sub_area, Some(MaskSubArea { top: 1, left: 2, full_width: 8, full_height: 4 }));+ assert_eq!((sky.data[0], sky.data[7]), (10 * 257, 65535));+ assert_eq!((skin.name.as_str(), skin.instance_id.as_deref(), skin.sub_area), ("skin", Some("person_a"), None));+ // positions are in uncropped-mask pixels: zero outside the stored crop+ assert!((sky.value_at(2.5, 1.5) - 10.0 / 255.0).abs() < 1e-4);+ assert!((sky.value_at(5.5, 2.5) - 1.0).abs() < 1e-4);+ assert_eq!((sky.value_at(1.5, 1.5), sky.value_at(2.5, 0.5), sky.value_at(6.5, 1.5)), (0.0, 0.0, 0.0));+ // over the whole image: the uncropped mask's 8 × 4 grid+ let full = sky.developed(Rect::new(0, 0, 8, 6), Rect::new(0, 0, 8, 6)).unwrap();+ assert_eq!((full.width, full.height), (8, 4));+ assert_eq!((full.get(2, 1), full.get(5, 2), full.get(1, 1), full.get(6, 2), full.get(2, 0)), (10, 255, 0, 0, 0));+ // a default crop keeping columns 2..6 of the image: the stored crop fills it+ let cropped = sky.developed(Rect::new(0, 0, 8, 6), Rect::new(2, 0, 4, 6)).unwrap();+ assert_eq!((cropped.width, cropped.height), (4, 4));+ assert_eq!((cropped.get(0, 1), cropped.get(3, 2), cropped.get(0, 0)), (10, 255, 0));+ }+}++/// Malformed semantic masks are skipped (never a panic), and so is the `MaskSubArea` of a mask it+/// doesn't fit; the raw image still decodes.+#[test]+fn malformed_semantic_masks_are_skipped() {+ let bytes = profile_dng(ByteOrder::Little, |ifd0| {+ ifd0.add_sub_ifd(mask_ifd(2, 2, "", vec![1, 2, 3, 4])); // no SemanticName+ let mut rgb = mask_ifd(2, 2, "rgb", vec![0; 12]);+ rgb.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![3]));+ rgb.set(t::BITS_PER_SAMPLE, Value::Short(vec![8, 8, 8]));+ ifd0.add_sub_ifd(rgb);+ let mut huge = mask_ifd(2, 2, "huge", vec![0; 4]);+ huge.set(t::IMAGE_WIDTH, Value::Long(vec![100_000]));+ huge.set(t::IMAGE_LENGTH, Value::Long(vec![100_000]));+ ifd0.add_sub_ifd(huge);+ let mut tiny = mask_ifd(2, 2, "tiny data", vec![0; 4]);+ tiny.set(t::IMAGE_WIDTH, Value::Long(vec![4000]));+ tiny.set(t::IMAGE_LENGTH, Value::Long(vec![3000]));+ ifd0.add_sub_ifd(tiny);+ let mut jxl = mask_ifd(2, 2, "unknown coding", vec![0; 4]);+ jxl.set(t::COMPRESSION, Value::Short(vec![60_000]));+ ifd0.add_sub_ifd(jxl);+ let mut zero = mask_ifd(2, 2, "zero size", vec![0; 4]);+ zero.set(t::IMAGE_WIDTH, Value::Long(vec![0]));+ ifd0.add_sub_ifd(zero);+ let mut short = mask_ifd(4, 4, "short strip", vec![200; 3]);+ short.set(t::MASK_SUB_AREA, Value::Long(vec![u32::MAX, u32::MAX, u32::MAX, u32::MAX]));+ ifd0.add_sub_ifd(short);+ let mut outside = mask_ifd(2, 2, "sub area too small", vec![255; 4]);+ outside.set(t::MASK_SUB_AREA, Value::Long(vec![3, 0, 2, 4]));+ ifd0.add_sub_ifd(outside);+ });+ assert_eq!(decode(&bytes).unwrap().cpp, 3);+ let masks = lightcraft_raw::semantic_masks(&bytes);+ let names: Vec<&str> = masks.iter().map(|m| m.name.as_str()).collect();+ assert_eq!(names, ["short strip", "sub area too small"]);+ assert!(masks.iter().all(|m| m.sub_area.is_none()), "sub areas the mask doesn't fit are ignored");+ for m in &masks {+ let d = m.developed(Rect::new(0, 0, 8, 6), Rect::new(0, 0, 8, 6)).unwrap();+ assert_eq!((d.width, d.height), (m.width, m.height));+ assert!(m.developed(Rect::new(0, 0, 8, 6), Rect::new(8, 6, 0, 0)).is_none());+ }+ // not a TIFF at all+ assert!(lightcraft_raw::semantic_masks(b"not a dng").is_empty());+}++/// Only the first `MAX_MASKS` mask IFDs are decoded, malformed ones included, so a file full of+/// mask IFDs costs a bounded number of decodes: here the first `MAX_MASKS` can't be decoded, and+/// the valid masks after them are never looked at.+#[test]+fn semantic_mask_ifds_past_the_cap_are_not_decoded() {+ use lightcraft_raw::semantic::MAX_MASKS;+ let with = |undecodable: usize, valid: usize| {+ profile_dng(ByteOrder::Little, |ifd0| {+ for i in 0..undecodable {+ let mut m = mask_ifd(2, 2, &format!("undecodable {i}"), vec![0; 4]);+ m.set(t::COMPRESSION, Value::Short(vec![60_000]));+ ifd0.add_sub_ifd(m);+ }+ for i in 0..valid {+ ifd0.add_sub_ifd(mask_ifd(2, 2, &format!("valid {i}"), vec![255; 4]));+ }+ })+ };+ let bytes = with(MAX_MASKS, 3 * MAX_MASKS);+ assert_eq!(decode(&bytes).unwrap().cpp, 3);+ assert!(lightcraft_raw::semantic_masks(&bytes).is_empty());+ // one fewer undecodable mask leaves room for exactly one valid one+ let names: Vec<String> = lightcraft_raw::semantic_masks(&with(MAX_MASKS - 1, 3 * MAX_MASKS)).into_iter().map(|m| m.name).collect();+ assert_eq!(names, ["valid 0"]);+}diff --git a/crates/tiff/src/tags.rs b/crates/tiff/src/tags.rsindex acdc76697..5771316a5 100644--- a/crates/tiff/src/tags.rs+++ b/crates/tiff/src/tags.rs@@ -180,6 +180,12 @@ pub const FORWARD_MATRIX_3: u16 = 52532;pub const ILLUMINANT_DATA_1: u16 = 52533;pub const ILLUMINANT_DATA_2: u16 = 52534;pub const ILLUMINANT_DATA_3: u16 = 52535;+// DNG 1.6 semantic masks (spec chapter 4, "Semantic Masks")+pub const SEMANTIC_NAME: u16 = 52526;+pub const SEMANTIC_INSTANCE_ID: u16 = 52528;+pub const MASK_SUB_AREA: u16 = 52536;+/// NewSubFileType of a semantic mask IFD.+pub const SUBFILE_SEMANTIC_MASK: u32 = 0x10004;pub const PROFILE_GAIN_TABLE_MAP_2: u16 = 52544;// --- Compression codes (TIFF 6.0, DNG, and widely used registrations)@@ -208,4 +214,6 @@ pub mod photometric {pub const YCBCR: u16 = 6;pub const CFA: u16 = 32803;pub const LINEAR_RAW: u16 = 34892;+ /// DNG 1.6 semantic mask.+ pub const MASK: u16 = 52527;}diff --git a/docs/parity.md b/docs/parity.mdindex 628cd8629..8e6c6f5a1 100644--- a/docs/parity.md+++ b/docs/parity.md@@ -341,11 +341,11 @@ under the UI command that wraps it (a no-op error); the UI command now wins.| Id | Feature | Tier | Status | Evidence | Missing / notes ||---|---|---|---|---|---|| LR-MASK-PANEL | Masks panel | P0 | ✅ | `cmd:panel.masking`, `cmd:mask.add`, `cmd:mask.select`, `cmd:mask.rename`, `cmd:mask.duplicate` (`invert`), `cmd:mask.move`, `cmd:mask.visible`, `cmd:mask.delete`, `crates/ui-egui/src/panels/masking.rs` | list with per-mask show/hide eye, double-click rename, right-click menu (duplicate, duplicate and invert, invert, hide, move up/down, rename, delete); fits any panel width (create tiles go to three columns, mask actions wrap, long component names are cut short); no drag-to-reorder |-| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | saliency heuristic, no segmentation model |-| LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`) | heuristic |-| LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | inverse of the subject heuristic |+| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | saliency heuristic, no segmentation model; a photo that carries person mattes (DNG semantic masks: iPhone ProRAW portrait / skin / hair / teeth / glasses mattes) uses them instead (synthetic files only: no corpus file carries person mattes) |+| LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`), `crates/pipeline/src/masks.rs` (`Mattes`), `crates/raw/src/semantic.rs` | heuristic; a photo that carries a sky matte (DNG semantic mask, e.g. iPhone ProRAW) uses it instead, through orientation / crop / lens corrections, preview = export (checked on one CC0 iPhone 12 Pro ProRAW and synthetic files) |+| LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | inverse of the subject mask (heuristic, or the photo's person mattes) || LR-MASK-OBJECTS | Object selection | P2 | 🟡 | `cmd:mask.add` (`object`, `prompt`), `cmd:mask.objectPoint`, `cmd:mask.refineDetail`, `cmd:segment.prepare`, `cmd:segment.model.status`, `cmd:segment.model.download`, `cmd:segment.model.cancel`, `crates/segment`, `crates/engine/src/segment/mod.rs` | SAM 3 in pure Rust (candle; Metal on macOS, CPU elsewhere): Object tile → click to include, ⌥-click to leave out; Describe tile → a text prompt selects every instance ("sky", "the red car"); both also as Add/Subtract/Intersect components; + / − under the selected mask; comma lists (`car, road`); a zoomed-in detail pass for 5–10× finer edges on small objects; per-selection Edge (hard ↔ soft). The model runs on its own worker thread (the UI never waits; panics become errors), is unloaded after 10 min idle, and is optional: the segmentation is stored with the mask (288² logits), so renders and exports never need it. Weights are never bundled (SAM License): without them the app offers a consented background download (mirrors, resume, timeouts, SHA-256) — **but no default download location is configured yet** (users set `LIGHTCRAFT_SAM3_MIRRORS` or install by hand: the dialog names the three files, shows the model folder and links docs/ai-masks.md). Not verified against Lightroom's Select Object; no brush/box object mode; the first click on a photo waits for its analysis (~4 s on an M4 Pro, much longer on CPU) |-| LR-MASK-PEOPLE | People parts | P2 | ⬜ | | |+| LR-MASK-PEOPLE | People parts | P2 | ⬜ | | shape exists (no UI): uses the photo's person / skin / hair / teeth / glasses mattes when it carries them (DNG semantic masks), else the subject heuristic || LR-MASK-LANDSCAPE | Landscape classes | P2 | ⬜ | | shape exists, evaluates empty || LR-MASK-BRUSH | Brush mask | P0 | ✅ | `cmd:tool.brush`, `cmd:mask.brushStroke` (`autoMask`), `crates/pipeline/src/masks.rs` | size/feather/flow/density/erase; Auto Mask: dabs weighted by similarity to the colour under the dab centre, refined by a guided filter on luminance (CPU + GPU); no A/B brushes, no pressure || LR-MASK-LINEAR | Linear gradient | P0 | ✅ | `cmd:tool.linear`, `cmd:mask.update` | |diff --git a/docs/whats-new.md b/docs/whats-new.mdindex f6361b048..ca5758bf4 100644--- a/docs/whats-new.md+++ b/docs/whats-new.md@@ -22,6 +22,9 @@embedded JPEG.- Apple ProRAW's gain table map (its local tone mapping, `ProfileGainTableMap`) is now read and kept when a photo isexported or converted to DNG. It is not applied to the render: Lightroom Classic renders ProRAW without it.+- iPhone ProRAW and other DNGs that carry their own segmentation mattes (DNG semantic masks) use them for the Select+ Sky, Subject and Background masks instead of our heuristics, so the sky is selected where the camera found it+ (checked on one CC0 iPhone 12 Pro ProRAW). Photos without mattes are unchanged.- Panasonic and Leica raws (RW2, RWL) are now corrected for lens distortion the way the camera corrects its own JPEG(issue #256): the correction the camera records in the file is applied under Lens Corrections ▸ Enable ProfileCorrections, on by default for newly imported photos, with the same framing as the camera's JPEG. At 12 mm the