LightCraft: An open-source non-destructive photo editor written by scratch in pure Rust
M3.8: embedded previews for every raw container + engine fallback; raw corpus
Brandon Thomas07359a735e25
diff --git a/ROADMAP.md b/ROADMAP.mdindex 584ed3e7c..91d7f54df 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -11,7 +11,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | 🚧 (UI, pipeline, engine, control channel ✅; MCP/CLI, web 🚧) || M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | 🚧 (import, codecs, EXIF/XMP, albums, search ✅; persistence 🚧) || M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ⬜ |-| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW ✅) |+| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed, embedded previews ✅; NEF Huffman ⬜) || M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ⬜ || M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | ⬜ || M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | ⬜ |@@ -19,7 +19,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | ⬜ || M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ⬜ || M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | ⬜ |-| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | ⬜ |+| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅) || M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ || M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ⬜ || M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |@@ -44,6 +44,30 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are reviHEVC (HEIC).- **Look parity** with Adobe's default rendering is subjective tuning against our own reference targets.+## Raw format coverage and known gaps++Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,+LJ92, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed, RAF uncompressed (Bayer and+X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every+supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants+it can't decode yet.++Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):+- **Nikon Huffman NEF** (lossless / lossy). Black-box findings so far: maker note `0x0096` holds per-parity predictor+ seeds (2048 for 14-bit lossless); ≈ 9 bits per pixel; the category-0 code word is a 6-bit rotation of `011111`. The+ Huffman tables are not stored in the files, and an automatic table search (beam search over prefix codes scored by+ smoothness and bit rate, validated on Pentax files whose tables *are* stored) did not converge.+- **Panasonic RW2 raw format 4** (quantised): the block layout is known (0x4000-byte chunks rotated by 0x1ff8; 128-bit+ blocks of two 12-bit seeds plus four groups of a 2-bit scale and three 8-bit codes; scales 0/1 are ×1/×2 differences+ from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.+- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.++**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged+`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer+matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our+own chart-based calibration (M11.4). Adobe matrices are never used.+## Log- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.+- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rsindex f4a4a08a9..a5cbad040 100644--- a/crates/engine/src/files.rs+++ b/crates/engine/src/files.rs@@ -44,7 +44,25 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {let m = lightcraft_meta::extract(bytes);let (meta, captured) = meta_of(&m);if lightcraft_raw::probe(bytes).is_some() {- let raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;+ let raw = match lightcraft_raw::decode(bytes) {+ Ok(r) => r,+ Err(lightcraft_raw::RawError::Unsupported(why)) => {+ // a raw variant we can't decode yet: describe it from its embedded preview+ let (w, h) = embedded_preview_size(bytes).ok_or(format!("unsupported raw ({why}) without an embedded preview"))?;+ return Ok(ProbeInfo {+ width: w,+ height: h,+ format: ext_upper(name),+ kind: MediaKind::Raw,+ file_size: bytes.len() as u64,+ captured,+ meta,+ as_shot_wb: None,+ content_hash,+ });+ }+ Err(e) => return Err(e.to_string()),+ };let (mut w, mut h) = (raw.crop.width.max(1) as u32, raw.crop.height.max(1) as u32);if w <= 1 || h <= 1 {(w, h) = (raw.active_area.width as u32, raw.active_area.height as u32);@@ -97,7 +115,14 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {/// Decode a file into a linear Rec.2020 image no larger than `max_edge`, oriented.pub fn load_bytes(bytes: &[u8], max_edge: usize) -> Result<(Rgb32f, SourceInfo), String> {if lightcraft_raw::probe(bytes).is_some() {- let raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;+ let raw = match lightcraft_raw::decode(bytes) {+ Ok(r) => r,+ Err(lightcraft_raw::RawError::Unsupported(why)) => {+ // show the camera's embedded JPEG (rendered, not raw) until the variant is supported+ return load_embedded_preview(bytes, max_edge).ok_or(format!("unsupported raw ({why}) without an embedded preview"));+ }+ Err(e) => return Err(e.to_string()),+ };let method = if max_edge <= 600 { lightcraft_raw::Method::Bilinear } else { lightcraft_raw::Method::Ahd };let mut img = raw.develop(method).map_err(|e| e.to_string())?;let xy = lightcraft_raw::color::as_shot_white_xy(&raw);@@ -124,6 +149,34 @@ pub fn load_bytes(bytes: &[u8], max_edge: usize) -> Result<(Rgb32f, SourceInfo),Ok((img.oriented(Orientation::from_exif(d.orientation)), SourceInfo::default()))}+/// Orientation for an embedded preview: its own EXIF orientation when it has one, else the raw file's.+fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {+ if jpeg_orientation > 1 {+ return Orientation::from_exif(jpeg_orientation);+ }+ lightcraft_meta::extract(raw_bytes).orientation.unwrap_or(Orientation::Normal)+}++/// Oriented size of the embedded preview of a raw file.+fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {+ let jpeg = lightcraft_raw::embedded_preview(bytes)?;+ let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(64, 64)).ok()?;+ let (mut w, mut h) = (d.source_width, d.source_height);+ if preview_orientation(bytes, d.orientation).swaps_axes() {+ std::mem::swap(&mut w, &mut h);+ }+ Some((w, h))+}++/// The embedded preview of a raw file as a working-space image no larger than `max_edge`, oriented.+pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, SourceInfo)> {+ let jpeg = lightcraft_raw::embedded_preview(bytes)?;+ let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).ok()?;+ let img = d.to_working();+ let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };+ Some((img.oriented(preview_orientation(bytes, d.orientation)), SourceInfo::default()))+}+/// Filesystem-backed hooks (native). On the web the host installs bytes-based hooks instead.pub fn fs_hooks() -> (FileLoader, FileProbe) {let loader: FileLoader = Arc::new(|path: &str, max_edge: usize| {@@ -146,3 +199,36 @@ impl crate::Session {self}}++#[cfg(test)]+mod tests {+ use super::*;+ use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};++ /// A CR3-shaped file (not decodable yet) whose only content is a `PRVW` preview box.+ fn cr3_with_preview(w: u32, h: u32) -> Vec<u8> {+ let px: Vec<u8> = (0..w * h).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();+ let jpeg = encode_jpeg(&EncodeImage::new(w, h, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();+ let mut b = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();+ b.extend_from_slice(&((24 + jpeg.len()) as u32).to_be_bytes());+ b.extend_from_slice(b"PRVW\0\0\0\0\0\x01");+ b.extend_from_slice(&(w as u16).to_be_bytes());+ b.extend_from_slice(&(h as u16).to_be_bytes());+ b.extend_from_slice(b"\0\x01");+ b.extend_from_slice(&(jpeg.len() as u32).to_be_bytes());+ b.extend_from_slice(&jpeg);+ b+ }++ #[test]+ fn unsupported_raw_falls_back_to_embedded_preview() {+ let b = cr3_with_preview(48, 32);+ let p = probe_bytes("x.cr3", &b).unwrap();+ assert_eq!((p.width, p.height, p.kind, p.format.as_str()), (48, 32, MediaKind::Raw, "CR3"));+ let (img, src) = load_bytes(&b, 24).unwrap();+ assert_eq!((img.width, img.height), (24, 16));+ assert!(!src.raw);+ // no preview at all: a clear error, not a panic+ assert!(load_bytes(b"\0\0\0\x18ftypcrx \0\0\0\x01", 24).is_err());+ }+}diff --git a/crates/raw/src/lib.rs b/crates/raw/src/lib.rsindex 60375fbcb..2803d4b5c 100644--- a/crates/raw/src/lib.rs+++ b/crates/raw/src/lib.rs@@ -10,10 +10,12 @@//! and produces camera → linear Rec.2020 D65 matrices.//!//! Formats: DNG (uncompressed, lossless JPEG, Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),-//! Canon CR2, Nikon NEF (uncompressed), Sony ARW (uncompressed, lossless), Fujifilm RAF (uncompressed Bayer and-//! X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF-//! (uncompressed, Huffman), Olympus ORF (uncompressed). See the crate README for sources and-//! gaps. The decoders never panic on malformed input.+//! Canon CR2, Nikon NEF/NRW (uncompressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer+//! and X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).+//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon Huffman NEF, Panasonic+//! quantised RW2, compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources+//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no+//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.#![forbid(unsafe_code)]pub mod color;diff --git a/crates/raw/src/preview.rs b/crates/raw/src/preview.rsindex a89512ba2..050870ce1 100644--- a/crates/raw/src/preview.rs+++ b/crates/raw/src/preview.rs@@ -67,6 +67,9 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {let j = crate::vendor::raf::header(bytes).ok()?.jpeg?;return is_dct_jpeg(j).then(|| trim_eoi(j).to_vec());}+ if crate::probe(bytes) == Some(crate::RawFormat::Cr3) {+ return cr3_preview(bytes).map(|j| trim_eoi(j).to_vec());+ }let tiff = Tiff::parse(bytes).ok()?;let mut found: Vec<&[u8]> = Vec::new();for ifd in tiff.all_ifds() {@@ -93,6 +96,34 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {found.into_iter().filter(|s| is_dct_jpeg(s)).max_by_key(|s| s.len()).map(|s| trim_eoi(s).to_vec())}+/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0+/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller+/// `THMB` box has the same shape. Returns the larger valid one.+fn cr3_preview(bytes: &[u8]) -> Option<&[u8]> {+ let mut best: Option<&[u8]> = None;+ for tag in [b"PRVW", b"THMB"] {+ let mut from = 0;+ while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) {+ from = i + 4;+ let Some(start) = i.checked_sub(4) else { continue };+ let be32 = |at: usize| bytes.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize);+ let (Some(size), Some(len)) = (be32(start), be32(start + 20)) else { continue };+ let j = start + 24;+ if len < 4 || j + len > start + size.max(24) || j + len > bytes.len() {+ continue;+ }+ let jpeg = &bytes[j..j + len];+ if is_dct_jpeg(jpeg) {+ if best.is_none_or(|b| b.len() < jpeg.len()) {+ best = Some(jpeg);+ }+ break;+ }+ }+ }+ best+}+/// Trim trailing garbage after the last EOI when a stored length over-reports.fn trim_eoi(s: &[u8]) -> &[u8] {let end = s.windows(2).rposition(|w| w == [0xff, 0xd9]).map(|p| p + 2).unwrap_or(s.len());@@ -137,5 +168,16 @@ mod tests {let bytes = TiffWriter::default().write(&[ifd0]).unwrap();assert_eq!(embedded_preview(&bytes).unwrap(), big);assert!(embedded_preview(b"nope").is_none());+ // CR3: a PRVW box after the ftyp+ let mut cr3 = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();+ let j = fake_jpeg(300);+ cr3.extend_from_slice(&((24 + j.len()) as u32).to_be_bytes());+ cr3.extend_from_slice(b"PRVW\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01");+ cr3.extend_from_slice(&(j.len() as u32).to_be_bytes());+ cr3.extend_from_slice(&j);+ assert_eq!(embedded_preview(&cr3).unwrap(), j);+ for n in 0..cr3.len() {+ let _ = embedded_preview(&cr3[..n]);+ }}}diff --git a/crates/raw/src/vendor/nef.rs b/crates/raw/src/vendor/nef.rsindex 3f2055d49..bcd8dd1ac 100644--- a/crates/raw/src/vendor/nef.rs+++ b/crates/raw/src/vendor/nef.rs@@ -4,7 +4,9 @@//! layout, SubIFDs, maker note header) and the ExifTool Nikon tag-name documentation (`0x000c` WB_RBLevels,//! `0x003d` BlackLevel). Nikon's Huffman-compressed variants (compression 34713: lossless, lossy type 1/2) are//! **not** implemented: the only descriptions we found are derived from GPL code, which our clean-room rules-//! forbid. They are reported as [`RawError::Unsupported`]; their embedded previews still work.+//! forbid. The coding is T.81-style (difference category + additional bits, as in Pentax PEF), but unlike PEF the+//! Huffman tables are not stored in the files and our black-box table search has not converged (see ROADMAP.md).+//! They are reported as [`RawError::Unsupported`]; their embedded previews still work.use super::white_from_data;use crate::tiffraw::{Packing, read_image};diff --git a/crates/raw/src/vendor/orf.rs b/crates/raw/src/vendor/orf.rsindex c9451cccf..2e2eebb38 100644--- a/crates/raw/src/vendor/orf.rs+++ b/crates/raw/src/vendor/orf.rs@@ -105,7 +105,7 @@ pub(crate) fn decode(bytes: &[u8]) -> Result<RawImage> {return Err(RawError::Unsupported("Olympus compressed ORF".into()));};let RawData::U16(ref mut samples) = data else { return Err(RawError::Unsupported("float ORF".into())) };- let bits = if bits == 16 && samples.par_iter().filter(|v| *v & 15 != 0).count() * 100 <= n {+ let bits = if bits == 16 && samples.iter().step_by(7).filter(|v| *v & 15 != 0).count() * 700 <= n {// 12-bit values stored in the top bitssamples.par_iter_mut().for_each(|v| *v >>= 4);12diff --git a/crates/raw/tests/corpus.rs b/crates/raw/tests/corpus.rsnew file mode 100644index 000000000..e8f055cff--- /dev/null+++ b/crates/raw/tests/corpus.rs@@ -0,0 +1,84 @@+//! Corpus test over `corpus/raw/**` (git-ignored CC0 samples from raw.pixls.us, fetched with+//! `cargo xtask corpus --download`; `LIGHTCRAFT_CORPUS` overrides the corpus root). Skips cleanly when absent.+//!+//! Every file must be recognised, carry an embedded JPEG preview (DNG: optional), and either decode to a valid image or report+//! `Unsupported` for one of the variants we know we don't decode yet. Prints decode times+//! (`cargo test -p lightcraft-raw --release --test corpus -- --nocapture`).++use lightcraft_raw::{RawError, RawFormat, decode, embedded_preview, probe};+use std::path::{Path, PathBuf};+use std::time::Instant;++fn corpus_root() -> PathBuf {+ std::env::var_os("LIGHTCRAFT_CORPUS").map(PathBuf::from).unwrap_or_else(|| Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))+}++/// Variants known not to decode yet (see the crate docs): matched against the lower-case file name.+const KNOWN_UNSUPPORTED: &[&str] = &[+ "cr3-", // CR3 / CRX (M11.1)+ "raf-fuji-xt20-compressed", // Fujifilm compressed RAF+ "rw2-panasonic-gh5.", // Panasonic raw format 4 (quantised)+ "rw2-panasonic-gx80", // "+ "rw2-panasonic-g9-b", // "+ "orf-olympus-em", // Olympus compressed ORF+ "nef-nikon-d5100-lossless", // Nikon Huffman NEF+ "nef-nikon-d7000-lossy", // "+ "sraw", // Canon sRAW / mRAW+ "-lossy.dng", // lossy (DCT) DNG+];++#[test]+fn corpus_raw_decodes() {+ let dir = corpus_root().join("raw");+ let Ok(rd) = std::fs::read_dir(&dir) else {+ eprintln!("skip: {} absent", dir.display());+ return;+ };+ let mut paths: Vec<PathBuf> = rd.flatten().map(|e| e.path()).filter(|p| p.is_file()).collect();+ paths.sort();+ let (mut ok, mut unsupported) = (0, 0);+ for p in paths {+ let name = p.file_name().unwrap().to_string_lossy().to_lowercase();+ if name.ends_with(".part") || name.ends_with(".txt") || name.ends_with(".md") {+ continue;+ }+ let bytes = std::fs::read(&p).unwrap();+ let fmt = probe(&bytes).unwrap_or_else(|| panic!("{name}: not recognised"));+ let t0 = Instant::now();+ let preview = embedded_preview(&bytes);+ let tp = t0.elapsed().as_secs_f64() * 1e3;+ // DNG previews are optional (and some carry only an uncompressed RGB thumbnail); vendor raws always embed a JPEG+ if let Some(p) = &preview {+ assert!(p.starts_with(&[0xff, 0xd8]) && p.ends_with(&[0xff, 0xd9]), "{name}: preview is not a JPEG");+ } else {+ assert_eq!(fmt, RawFormat::Dng, "{name}: no embedded preview");+ }+ let preview_kb = preview.as_ref().map_or(0, |p| p.len() / 1024);+ let t1 = Instant::now();+ match decode(&bytes) {+ Ok(img) => {+ let dt = t1.elapsed().as_secs_f64() * 1e3;+ img.validate().unwrap();+ assert!(img.white_at(0) > img.black.mean(), "{name}: white {} <= black {}", img.white_at(0), img.black.mean());+ let mp = (img.width * img.height) as f64 / 1e6;+ eprintln!(+ "{name:44} {fmt:?} {}x{} {}-bit {:?}: decode {dt:.0} ms ({:.0} MP/s), preview {} KB in {tp:.1} ms",+ img.width,+ img.height,+ img.bits,+ img.cfa.as_ref().map(|c| c.name()),+ mp / (dt / 1e3),+ preview_kb+ );+ ok += 1;+ }+ Err(RawError::Unsupported(why)) => {+ assert!(KNOWN_UNSUPPORTED.iter().any(|k| name.contains(k)), "{name}: unexpectedly unsupported: {why}");+ eprintln!("{name:44} {fmt:?} unsupported ({why}); preview {preview_kb} KB in {tp:.1} ms");+ unsupported += 1;+ }+ Err(e) => panic!("{name}: {e}"),+ }+ }+ eprintln!("corpus/raw: {ok} decoded, {unsupported} known-unsupported (preview only)");+}diff --git a/xtask/src/main.rs b/xtask/src/main.rsindex 0218d4a96..6a537e081 100644--- a/xtask/src/main.rs+++ b/xtask/src/main.rs@@ -23,7 +23,7 @@ commands:--serve serves it on http://127.0.0.1:<port> (default 8080)ci fmt --check, clippy -D warnings, test, layers, assets, wasm (stops at first failure)corpus [--download]- show where test corpora live; --download fetches PngSuite into corpus/pngsuite+ show where test corpora live; --download fetches PngSuite and CC0 raw samples (raw.pixls.us) into corpus/stats [--exact] count tests and lines per crate (--exact: ask the test harness via `-- --list`)";@@ -197,7 +197,86 @@ fn cmd_ci() -> Result<(), String> {}/// CC0 raw samples (raw.pixls.us). Small, representative set; extend freely (CC0 only).-const RAW_SAMPLES: &[(&str, &str)] = &[];+/// CC0 raw samples from raw.pixls.us (each verified CC0 on the site; files matched by their published SHA-256).+/// One per format / compression variant we decode or deliberately report as unsupported (preview only).+const RAW_SAMPLES: &[(&str, &str)] = &[+ (+ "arw-sony-a7m3-compressed.arw",+ "https://raw.pixls.us/getfile.php/2414/nice/Sony%20-%20ILCE-7M3%20-%2014bit%2014bit%20compressed%20%283:2%29.ARW",+ ),+ (+ "arw-sony-a7m3-uncompressed.arw",+ "https://raw.pixls.us/getfile.php/2418/nice/Sony%20-%20ILCE-7M3%20-%2014bit%2014bit%20uncompressed%20%283:2%29.ARW",+ ),+ ("arw-sony-a7m4-14bit.arw", "https://raw.pixls.us/getfile.php/6936/nice/Sony%20-%20ILCE-7M4%20-%2014bit%20%283:2%29.ARW"),+ ("cr2-canon-5d3-sraw2.cr2", "https://raw.pixls.us/getfile.php/773/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%20sRAW2%20%28sRAW%29.CR2"),+ ("cr2-canon-5d3.cr2", "https://raw.pixls.us/getfile.php/771/nice/Canon%20-%20EOS%205D%20Mark%20III.CR2"),+ ("cr2-canon-80d.cr2", "https://raw.pixls.us/getfile.php/1294/nice/Canon%20-%20EOS%2080D%20-%20RAW%20%283:2%29.CR2"),+ ("cr3-canon-m50-craw.cr3", "https://raw.pixls.us/getfile.php/2663/nice/Canon%20-%20EOS%20M50%20-%20CRAW%20%283:2%29.CR3"),+ (+ "dng-adobe-canon-5d3-linear-lj92.dng",+ "https://raw.pixls.us/getfile.php/1032/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%20Lossless%20JPEG%20compression%2C%20rgb%20%283:2%29.DNG",+ ),+ (+ "dng-adobe-canon-5d3-lj92.dng",+ "https://raw.pixls.us/getfile.php/1024/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%2016bit%2016bit%20Lossless%20JPEG%20compression%20%283:2%29.DNG",+ ),+ (+ "dng-adobe-canon-5d3-lossy.dng",+ "https://raw.pixls.us/getfile.php/1023/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%20Lossy%20JPEG%20compression%20%283:2%29.DNG",+ ),+ (+ "dng-canon-5d3-14bit-small.dng",+ "https://raw.pixls.us/getfile.php/2204/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2014bit%2014bit%20%282.3471882640587%29.dng",+ ),+ ("dng-canon-5d3-16bit-169.dng", "https://raw.pixls.us/getfile.php/2649/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2016bit%20%2816:9%29.dng"),+ ("dng-canon-5d3-16bit.dng", "https://raw.pixls.us/getfile.php/885/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2016bit%2016bit%20RAW.dng"),+ ("dng-google-pixel2xl.dng", "https://raw.pixls.us/getfile.php/2206/nice/Google%20-%20Pixel%202%20XL%20-%2016bit%20%284:3%29.dng"),+ ("dng-ricoh-gr3.dng", "https://raw.pixls.us/getfile.php/3115/nice/Ricoh%20-%20GR%20III%20-%2014bit%20%283:2%29.DNG"),+ (+ "nef-nikon-d5100-lossless.nef",+ "https://raw.pixls.us/getfile.php/1597/nice/Nikon%20-%20D5100%20-%2014bit%2014bit%20compressed%20%28Lossless%29%20%283:2%29.nef",+ ),+ (+ "nef-nikon-d5100-uncompressed.nef",+ "https://raw.pixls.us/getfile.php/1598/nice/Nikon%20-%20D5100%20-%2014bit%2014bit%20uncompressed%20%283:2%29.nef",+ ),+ (+ "nef-nikon-d7000-lossy12.nef",+ "https://raw.pixls.us/getfile.php/961/nice/Nikon%20-%20D7000%20-%2012bit%2012bit%20compressed%20%28Lossy%20%28type%202%29%29%20%283:2%29.NEF",+ ),+ (+ "nrw-nikon-b700-uncompressed.nrw",+ "https://raw.pixls.us/getfile.php/1621/nice/Nikon%20-%20COOLPIX%20B700%20-%2012bit%2012bit%20uncompressed%20%284:3%29.NRW",+ ),+ ("orf-olympus-e1.orf", "https://raw.pixls.us/getfile.php/1800/nice/Olympus%20-%20E-1%20-%2016bit%20%284:3%29.ORF"),+ ("orf-olympus-e400.orf", "https://raw.pixls.us/getfile.php/2151/nice/Olympus%20-%20E-400%20-%2016bit%20%284:3%29.ORF"),+ ("orf-olympus-em1.orf", "https://raw.pixls.us/getfile.php/1051/nice/Olympus%20-%20E-M1%20-%2016bit%20%284:3%29.orf"),+ ("orf-olympus-em10iii.orf", "https://raw.pixls.us/getfile.php/1787/nice/Olympus%20-%20E-M10%20Mark%20III%20-%2016bit%20%284:3%29.ORF"),+ ("orf-olympus-xz2.orf", "https://raw.pixls.us/getfile.php/1432/nice/Olympus%20-%20XZ-2%20-%2012bit%20%284:3%29.orf"),+ ("pef-pentax-k10d.pef", "https://raw.pixls.us/getfile.php/2239/nice/Pentax%20-%20K10D%20-%2012bit%2012bit%20compressed%20%283:2%29.PEF"),+ ("pef-pentax-k3.pef", "https://raw.pixls.us/getfile.php/1075/nice/Pentax%20-%20K-3%20-%2014bit%20%283:2%29.PEF"),+ ("pef-pentax-k5iis.pef", "https://raw.pixls.us/getfile.php/1198/nice/Pentax%20-%20K-5%20II%20s%20-%2014bit%20%283:2%29.PEF"),+ (+ "raf-fuji-xa2-12bit-bayer.raf",+ "https://raw.pixls.us/getfile.php/2883/nice/Fujifilm%20-%20X-A2%20-%2012bit%2012bit%20uncompressed%20%283:2%29.RAF",+ ),+ (+ "raf-fuji-xa5-14bit-bayer.raf",+ "https://raw.pixls.us/getfile.php/2526/nice/Fujifilm%20-%20X-A5%20-%2014bit%2014bit%20uncompressed%20%283:2%29.RAF",+ ),+ ("raf-fuji-xe1-12bit.raf", "https://raw.pixls.us/getfile.php/3098/nice/Fujifilm%20-%20X-E1%20-%2012bit%2012bit%20uncompressed%20%283:2%29.RAF"),+ ("raf-fuji-xt20-14bit.raf", "https://raw.pixls.us/getfile.php/1177/nice/Fujifilm%20-%20X-T20%20-%2014bit%2014bit%20uncompressed%20%283:2%29.RAF"),+ (+ "raf-fuji-xt20-compressed.raf",+ "https://raw.pixls.us/getfile.php/1178/nice/Fujifilm%20-%20X-T20%20-%2014bit%2014bit%20compressed%20%283:2%29.RAF",+ ),+ ("rw2-panasonic-g9-b.rw2", "https://raw.pixls.us/getfile.php/2348/nice/Panasonic%20-%20DC-G9%20-%204:3.RW2"),+ ("rw2-panasonic-g9.rw2", "https://raw.pixls.us/getfile.php/2585/nice/Panasonic%20-%20DC-G9%20-%204:3.RW2"),+ ("rw2-panasonic-gh5.rw2", "https://raw.pixls.us/getfile.php/1517/nice/Panasonic%20-%20DC-GH5%20-%204:3.RW2"),+ ("rw2-panasonic-gh5s.rw2", "https://raw.pixls.us/getfile.php/2603/nice/Panasonic%20-%20DC-GH5S%20-%204:3.RW2"),+ ("rw2-panasonic-gx80.rw2", "https://raw.pixls.us/getfile.php/1569/nice/Panasonic%20-%20DMC-GX80%20-%204:3.RW2"),+];fn cmd_corpus(download: bool) -> Result<(), String> {let corpus = root().join("corpus");