Skip to content

LightCraft: An open-source non-destructive photo editor written by scratch in pure Rust

M3.8: embedded previews for every raw container + engine fallback; raw corpus

Brandon Thomas07359a735e25
diff --git a/ROADMAP.md b/ROADMAP.md
index 584ed3e7c..91d7f54df 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -11,7 +11,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | 🚧 (UI, pipeline, engine, control channel ✅; MCP/CLI, web 🚧) |
| M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | 🚧 (import, codecs, EXIF/XMP, albums, search ✅; persistence 🚧) |
| M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ⬜ |
-| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW ✅) |
+| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed, embedded previews ✅; NEF Huffman ⬜) |
| M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ⬜ |
| M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | ⬜ |
| M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | ⬜ |
@@ -19,7 +19,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | ⬜ |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ⬜ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | ⬜ |
-| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | ⬜ |
+| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ⬜ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
@@ -44,6 +44,30 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
HEVC (HEIC).
- **Look parity** with Adobe's default rendering is subjective tuning against our own reference targets.
+## Raw format coverage and known gaps
+
+Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
+LJ92, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed, RAF uncompressed (Bayer and
+X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
+supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
+it can't decode yet.
+
+Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
+- **Nikon Huffman NEF** (lossless / lossy). Black-box findings so far: maker note `0x0096` holds per-parity predictor
+ seeds (2048 for 14-bit lossless); ≈ 9 bits per pixel; the category-0 code word is a 6-bit rotation of `011111`. The
+ Huffman tables are not stored in the files, and an automatic table search (beam search over prefix codes scored by
+ smoothness and bit rate, validated on Pentax files whose tables *are* stored) did not converge.
+- **Panasonic RW2 raw format 4** (quantised): the block layout is known (0x4000-byte chunks rotated by 0x1ff8; 128-bit
+ blocks of two 12-bit seeds plus four groups of a 2-bit scale and three 8-bit codes; scales 0/1 are ×1/×2 differences
+ from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.
+- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
+
+**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
+`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
+matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our
+own chart-based calibration (M11.4). Adobe matrices are never used.
+
## Log
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.
+- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.
diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rs
index f4a4a08a9..a5cbad040 100644
--- a/crates/engine/src/files.rs
+++ b/crates/engine/src/files.rs
@@ -44,7 +44,25 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
let m = lightcraft_meta::extract(bytes);
let (meta, captured) = meta_of(&m);
if lightcraft_raw::probe(bytes).is_some() {
- let raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
+ let raw = match lightcraft_raw::decode(bytes) {
+ Ok(r) => r,
+ Err(lightcraft_raw::RawError::Unsupported(why)) => {
+ // a raw variant we can't decode yet: describe it from its embedded preview
+ let (w, h) = embedded_preview_size(bytes).ok_or(format!("unsupported raw ({why}) without an embedded preview"))?;
+ return Ok(ProbeInfo {
+ width: w,
+ height: h,
+ format: ext_upper(name),
+ kind: MediaKind::Raw,
+ file_size: bytes.len() as u64,
+ captured,
+ meta,
+ as_shot_wb: None,
+ content_hash,
+ });
+ }
+ Err(e) => return Err(e.to_string()),
+ };
let (mut w, mut h) = (raw.crop.width.max(1) as u32, raw.crop.height.max(1) as u32);
if w <= 1 || h <= 1 {
(w, h) = (raw.active_area.width as u32, raw.active_area.height as u32);
@@ -97,7 +115,14 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result<ProbeInfo, String> {
/// Decode a file into a linear Rec.2020 image no larger than `max_edge`, oriented.
pub fn load_bytes(bytes: &[u8], max_edge: usize) -> Result<(Rgb32f, SourceInfo), String> {
if lightcraft_raw::probe(bytes).is_some() {
- let raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
+ let raw = match lightcraft_raw::decode(bytes) {
+ Ok(r) => r,
+ Err(lightcraft_raw::RawError::Unsupported(why)) => {
+ // show the camera's embedded JPEG (rendered, not raw) until the variant is supported
+ return load_embedded_preview(bytes, max_edge).ok_or(format!("unsupported raw ({why}) without an embedded preview"));
+ }
+ Err(e) => return Err(e.to_string()),
+ };
let method = if max_edge <= 600 { lightcraft_raw::Method::Bilinear } else { lightcraft_raw::Method::Ahd };
let mut img = raw.develop(method).map_err(|e| e.to_string())?;
let xy = lightcraft_raw::color::as_shot_white_xy(&raw);
@@ -124,6 +149,34 @@ pub fn load_bytes(bytes: &[u8], max_edge: usize) -> Result<(Rgb32f, SourceInfo),
Ok((img.oriented(Orientation::from_exif(d.orientation)), SourceInfo::default()))
}
+/// Orientation for an embedded preview: its own EXIF orientation when it has one, else the raw file's.
+fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
+ if jpeg_orientation > 1 {
+ return Orientation::from_exif(jpeg_orientation);
+ }
+ lightcraft_meta::extract(raw_bytes).orientation.unwrap_or(Orientation::Normal)
+}
+
+/// Oriented size of the embedded preview of a raw file.
+fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
+ let jpeg = lightcraft_raw::embedded_preview(bytes)?;
+ let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(64, 64)).ok()?;
+ let (mut w, mut h) = (d.source_width, d.source_height);
+ if preview_orientation(bytes, d.orientation).swaps_axes() {
+ std::mem::swap(&mut w, &mut h);
+ }
+ Some((w, h))
+}
+
+/// The embedded preview of a raw file as a working-space image no larger than `max_edge`, oriented.
+pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, SourceInfo)> {
+ let jpeg = lightcraft_raw::embedded_preview(bytes)?;
+ let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).ok()?;
+ let img = d.to_working();
+ let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };
+ Some((img.oriented(preview_orientation(bytes, d.orientation)), SourceInfo::default()))
+}
+
/// Filesystem-backed hooks (native). On the web the host installs bytes-based hooks instead.
pub fn fs_hooks() -> (FileLoader, FileProbe) {
let loader: FileLoader = Arc::new(|path: &str, max_edge: usize| {
@@ -146,3 +199,36 @@ impl crate::Session {
self
}
}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
+
+ /// A CR3-shaped file (not decodable yet) whose only content is a `PRVW` preview box.
+ fn cr3_with_preview(w: u32, h: u32) -> Vec<u8> {
+ let px: Vec<u8> = (0..w * h).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
+ let jpeg = encode_jpeg(&EncodeImage::new(w, h, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
+ let mut b = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();
+ b.extend_from_slice(&((24 + jpeg.len()) as u32).to_be_bytes());
+ b.extend_from_slice(b"PRVW\0\0\0\0\0\x01");
+ b.extend_from_slice(&(w as u16).to_be_bytes());
+ b.extend_from_slice(&(h as u16).to_be_bytes());
+ b.extend_from_slice(b"\0\x01");
+ b.extend_from_slice(&(jpeg.len() as u32).to_be_bytes());
+ b.extend_from_slice(&jpeg);
+ b
+ }
+
+ #[test]
+ fn unsupported_raw_falls_back_to_embedded_preview() {
+ let b = cr3_with_preview(48, 32);
+ let p = probe_bytes("x.cr3", &b).unwrap();
+ assert_eq!((p.width, p.height, p.kind, p.format.as_str()), (48, 32, MediaKind::Raw, "CR3"));
+ let (img, src) = load_bytes(&b, 24).unwrap();
+ assert_eq!((img.width, img.height), (24, 16));
+ assert!(!src.raw);
+ // no preview at all: a clear error, not a panic
+ assert!(load_bytes(b"\0\0\0\x18ftypcrx \0\0\0\x01", 24).is_err());
+ }
+}
diff --git a/crates/raw/src/lib.rs b/crates/raw/src/lib.rs
index 60375fbcb..2803d4b5c 100644
--- a/crates/raw/src/lib.rs
+++ b/crates/raw/src/lib.rs
@@ -10,10 +10,12 @@
//! and produces camera → linear Rec.2020 D65 matrices.
//!
//! Formats: DNG (uncompressed, lossless JPEG, Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),
-//! Canon CR2, Nikon NEF (uncompressed), Sony ARW (uncompressed, lossless), Fujifilm RAF (uncompressed Bayer and
-//! X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF
-//! (uncompressed, Huffman), Olympus ORF (uncompressed). See the crate README for sources and
-//! gaps. The decoders never panic on malformed input.
+//! Canon CR2, Nikon NEF/NRW (uncompressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
+//! and X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
+//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon Huffman NEF, Panasonic
+//! quantised RW2, compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
+//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
+//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]
pub mod color;
diff --git a/crates/raw/src/preview.rs b/crates/raw/src/preview.rs
index a89512ba2..050870ce1 100644
--- a/crates/raw/src/preview.rs
+++ b/crates/raw/src/preview.rs
@@ -67,6 +67,9 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
let j = crate::vendor::raf::header(bytes).ok()?.jpeg?;
return is_dct_jpeg(j).then(|| trim_eoi(j).to_vec());
}
+ if crate::probe(bytes) == Some(crate::RawFormat::Cr3) {
+ return cr3_preview(bytes).map(|j| trim_eoi(j).to_vec());
+ }
let tiff = Tiff::parse(bytes).ok()?;
let mut found: Vec<&[u8]> = Vec::new();
for ifd in tiff.all_ifds() {
@@ -93,6 +96,34 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
found.into_iter().filter(|s| is_dct_jpeg(s)).max_by_key(|s| s.len()).map(|s| trim_eoi(s).to_vec())
}
+/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0
+/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller
+/// `THMB` box has the same shape. Returns the larger valid one.
+fn cr3_preview(bytes: &[u8]) -> Option<&[u8]> {
+ let mut best: Option<&[u8]> = None;
+ for tag in [b"PRVW", b"THMB"] {
+ let mut from = 0;
+ while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) {
+ from = i + 4;
+ let Some(start) = i.checked_sub(4) else { continue };
+ let be32 = |at: usize| bytes.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize);
+ let (Some(size), Some(len)) = (be32(start), be32(start + 20)) else { continue };
+ let j = start + 24;
+ if len < 4 || j + len > start + size.max(24) || j + len > bytes.len() {
+ continue;
+ }
+ let jpeg = &bytes[j..j + len];
+ if is_dct_jpeg(jpeg) {
+ if best.is_none_or(|b| b.len() < jpeg.len()) {
+ best = Some(jpeg);
+ }
+ break;
+ }
+ }
+ }
+ best
+}
+
/// Trim trailing garbage after the last EOI when a stored length over-reports.
fn trim_eoi(s: &[u8]) -> &[u8] {
let end = s.windows(2).rposition(|w| w == [0xff, 0xd9]).map(|p| p + 2).unwrap_or(s.len());
@@ -137,5 +168,16 @@ mod tests {
let bytes = TiffWriter::default().write(&[ifd0]).unwrap();
assert_eq!(embedded_preview(&bytes).unwrap(), big);
assert!(embedded_preview(b"nope").is_none());
+ // CR3: a PRVW box after the ftyp
+ let mut cr3 = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();
+ let j = fake_jpeg(300);
+ cr3.extend_from_slice(&((24 + j.len()) as u32).to_be_bytes());
+ cr3.extend_from_slice(b"PRVW\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01");
+ cr3.extend_from_slice(&(j.len() as u32).to_be_bytes());
+ cr3.extend_from_slice(&j);
+ assert_eq!(embedded_preview(&cr3).unwrap(), j);
+ for n in 0..cr3.len() {
+ let _ = embedded_preview(&cr3[..n]);
+ }
}
}
diff --git a/crates/raw/src/vendor/nef.rs b/crates/raw/src/vendor/nef.rs
index 3f2055d49..bcd8dd1ac 100644
--- a/crates/raw/src/vendor/nef.rs
+++ b/crates/raw/src/vendor/nef.rs
@@ -4,7 +4,9 @@
//! layout, SubIFDs, maker note header) and the ExifTool Nikon tag-name documentation (`0x000c` WB_RBLevels,
//! `0x003d` BlackLevel). Nikon's Huffman-compressed variants (compression 34713: lossless, lossy type 1/2) are
//! **not** implemented: the only descriptions we found are derived from GPL code, which our clean-room rules
-//! forbid. They are reported as [`RawError::Unsupported`]; their embedded previews still work.
+//! forbid. The coding is T.81-style (difference category + additional bits, as in Pentax PEF), but unlike PEF the
+//! Huffman tables are not stored in the files and our black-box table search has not converged (see ROADMAP.md).
+//! They are reported as [`RawError::Unsupported`]; their embedded previews still work.
use super::white_from_data;
use crate::tiffraw::{Packing, read_image};
diff --git a/crates/raw/src/vendor/orf.rs b/crates/raw/src/vendor/orf.rs
index c9451cccf..2e2eebb38 100644
--- a/crates/raw/src/vendor/orf.rs
+++ b/crates/raw/src/vendor/orf.rs
@@ -105,7 +105,7 @@ pub(crate) fn decode(bytes: &[u8]) -> Result<RawImage> {
return Err(RawError::Unsupported("Olympus compressed ORF".into()));
};
let RawData::U16(ref mut samples) = data else { return Err(RawError::Unsupported("float ORF".into())) };
- let bits = if bits == 16 && samples.par_iter().filter(|v| *v & 15 != 0).count() * 100 <= n {
+ let bits = if bits == 16 && samples.iter().step_by(7).filter(|v| *v & 15 != 0).count() * 700 <= n {
// 12-bit values stored in the top bits
samples.par_iter_mut().for_each(|v| *v >>= 4);
12
diff --git a/crates/raw/tests/corpus.rs b/crates/raw/tests/corpus.rs
new file mode 100644
index 000000000..e8f055cff
--- /dev/null
+++ b/crates/raw/tests/corpus.rs
@@ -0,0 +1,84 @@
+//! Corpus test over `corpus/raw/**` (git-ignored CC0 samples from raw.pixls.us, fetched with
+//! `cargo xtask corpus --download`; `LIGHTCRAFT_CORPUS` overrides the corpus root). Skips cleanly when absent.
+//!
+//! Every file must be recognised, carry an embedded JPEG preview (DNG: optional), and either decode to a valid image or report
+//! `Unsupported` for one of the variants we know we don't decode yet. Prints decode times
+//! (`cargo test -p lightcraft-raw --release --test corpus -- --nocapture`).
+
+use lightcraft_raw::{RawError, RawFormat, decode, embedded_preview, probe};
+use std::path::{Path, PathBuf};
+use std::time::Instant;
+
+fn corpus_root() -> PathBuf {
+ std::env::var_os("LIGHTCRAFT_CORPUS").map(PathBuf::from).unwrap_or_else(|| Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
+}
+
+/// Variants known not to decode yet (see the crate docs): matched against the lower-case file name.
+const KNOWN_UNSUPPORTED: &[&str] = &[
+ "cr3-", // CR3 / CRX (M11.1)
+ "raf-fuji-xt20-compressed", // Fujifilm compressed RAF
+ "rw2-panasonic-gh5.", // Panasonic raw format 4 (quantised)
+ "rw2-panasonic-gx80", // "
+ "rw2-panasonic-g9-b", // "
+ "orf-olympus-em", // Olympus compressed ORF
+ "nef-nikon-d5100-lossless", // Nikon Huffman NEF
+ "nef-nikon-d7000-lossy", // "
+ "sraw", // Canon sRAW / mRAW
+ "-lossy.dng", // lossy (DCT) DNG
+];
+
+#[test]
+fn corpus_raw_decodes() {
+ let dir = corpus_root().join("raw");
+ let Ok(rd) = std::fs::read_dir(&dir) else {
+ eprintln!("skip: {} absent", dir.display());
+ return;
+ };
+ let mut paths: Vec<PathBuf> = rd.flatten().map(|e| e.path()).filter(|p| p.is_file()).collect();
+ paths.sort();
+ let (mut ok, mut unsupported) = (0, 0);
+ for p in paths {
+ let name = p.file_name().unwrap().to_string_lossy().to_lowercase();
+ if name.ends_with(".part") || name.ends_with(".txt") || name.ends_with(".md") {
+ continue;
+ }
+ let bytes = std::fs::read(&p).unwrap();
+ let fmt = probe(&bytes).unwrap_or_else(|| panic!("{name}: not recognised"));
+ let t0 = Instant::now();
+ let preview = embedded_preview(&bytes);
+ let tp = t0.elapsed().as_secs_f64() * 1e3;
+ // DNG previews are optional (and some carry only an uncompressed RGB thumbnail); vendor raws always embed a JPEG
+ if let Some(p) = &preview {
+ assert!(p.starts_with(&[0xff, 0xd8]) && p.ends_with(&[0xff, 0xd9]), "{name}: preview is not a JPEG");
+ } else {
+ assert_eq!(fmt, RawFormat::Dng, "{name}: no embedded preview");
+ }
+ let preview_kb = preview.as_ref().map_or(0, |p| p.len() / 1024);
+ let t1 = Instant::now();
+ match decode(&bytes) {
+ Ok(img) => {
+ let dt = t1.elapsed().as_secs_f64() * 1e3;
+ img.validate().unwrap();
+ assert!(img.white_at(0) > img.black.mean(), "{name}: white {} <= black {}", img.white_at(0), img.black.mean());
+ let mp = (img.width * img.height) as f64 / 1e6;
+ eprintln!(
+ "{name:44} {fmt:?} {}x{} {}-bit {:?}: decode {dt:.0} ms ({:.0} MP/s), preview {} KB in {tp:.1} ms",
+ img.width,
+ img.height,
+ img.bits,
+ img.cfa.as_ref().map(|c| c.name()),
+ mp / (dt / 1e3),
+ preview_kb
+ );
+ ok += 1;
+ }
+ Err(RawError::Unsupported(why)) => {
+ assert!(KNOWN_UNSUPPORTED.iter().any(|k| name.contains(k)), "{name}: unexpectedly unsupported: {why}");
+ eprintln!("{name:44} {fmt:?} unsupported ({why}); preview {preview_kb} KB in {tp:.1} ms");
+ unsupported += 1;
+ }
+ Err(e) => panic!("{name}: {e}"),
+ }
+ }
+ eprintln!("corpus/raw: {ok} decoded, {unsupported} known-unsupported (preview only)");
+}
diff --git a/xtask/src/main.rs b/xtask/src/main.rs
index 0218d4a96..6a537e081 100644
--- a/xtask/src/main.rs
+++ b/xtask/src/main.rs
@@ -23,7 +23,7 @@ commands:
--serve serves it on http://127.0.0.1:<port> (default 8080)
ci fmt --check, clippy -D warnings, test, layers, assets, wasm (stops at first failure)
corpus [--download]
- show where test corpora live; --download fetches PngSuite into corpus/pngsuite
+ show where test corpora live; --download fetches PngSuite and CC0 raw samples (raw.pixls.us) into corpus/
stats [--exact] count tests and lines per crate (--exact: ask the test harness via `-- --list`)
";
@@ -197,7 +197,86 @@ fn cmd_ci() -> Result<(), String> {
}
/// CC0 raw samples (raw.pixls.us). Small, representative set; extend freely (CC0 only).
-const RAW_SAMPLES: &[(&str, &str)] = &[];
+/// CC0 raw samples from raw.pixls.us (each verified CC0 on the site; files matched by their published SHA-256).
+/// One per format / compression variant we decode or deliberately report as unsupported (preview only).
+const RAW_SAMPLES: &[(&str, &str)] = &[
+ (
+ "arw-sony-a7m3-compressed.arw",
+ "https://raw.pixls.us/getfile.php/2414/nice/Sony%20-%20ILCE-7M3%20-%2014bit%2014bit%20compressed%20%283:2%29.ARW",
+ ),
+ (
+ "arw-sony-a7m3-uncompressed.arw",
+ "https://raw.pixls.us/getfile.php/2418/nice/Sony%20-%20ILCE-7M3%20-%2014bit%2014bit%20uncompressed%20%283:2%29.ARW",
+ ),
+ ("arw-sony-a7m4-14bit.arw", "https://raw.pixls.us/getfile.php/6936/nice/Sony%20-%20ILCE-7M4%20-%2014bit%20%283:2%29.ARW"),
+ ("cr2-canon-5d3-sraw2.cr2", "https://raw.pixls.us/getfile.php/773/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%20sRAW2%20%28sRAW%29.CR2"),
+ ("cr2-canon-5d3.cr2", "https://raw.pixls.us/getfile.php/771/nice/Canon%20-%20EOS%205D%20Mark%20III.CR2"),
+ ("cr2-canon-80d.cr2", "https://raw.pixls.us/getfile.php/1294/nice/Canon%20-%20EOS%2080D%20-%20RAW%20%283:2%29.CR2"),
+ ("cr3-canon-m50-craw.cr3", "https://raw.pixls.us/getfile.php/2663/nice/Canon%20-%20EOS%20M50%20-%20CRAW%20%283:2%29.CR3"),
+ (
+ "dng-adobe-canon-5d3-linear-lj92.dng",
+ "https://raw.pixls.us/getfile.php/1032/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%20Lossless%20JPEG%20compression%2C%20rgb%20%283:2%29.DNG",
+ ),
+ (
+ "dng-adobe-canon-5d3-lj92.dng",
+ "https://raw.pixls.us/getfile.php/1024/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%2016bit%2016bit%20Lossless%20JPEG%20compression%20%283:2%29.DNG",
+ ),
+ (
+ "dng-adobe-canon-5d3-lossy.dng",
+ "https://raw.pixls.us/getfile.php/1023/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%20Lossy%20JPEG%20compression%20%283:2%29.DNG",
+ ),
+ (
+ "dng-canon-5d3-14bit-small.dng",
+ "https://raw.pixls.us/getfile.php/2204/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2014bit%2014bit%20%282.3471882640587%29.dng",
+ ),
+ ("dng-canon-5d3-16bit-169.dng", "https://raw.pixls.us/getfile.php/2649/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2016bit%20%2816:9%29.dng"),
+ ("dng-canon-5d3-16bit.dng", "https://raw.pixls.us/getfile.php/885/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2016bit%2016bit%20RAW.dng"),
+ ("dng-google-pixel2xl.dng", "https://raw.pixls.us/getfile.php/2206/nice/Google%20-%20Pixel%202%20XL%20-%2016bit%20%284:3%29.dng"),
+ ("dng-ricoh-gr3.dng", "https://raw.pixls.us/getfile.php/3115/nice/Ricoh%20-%20GR%20III%20-%2014bit%20%283:2%29.DNG"),
+ (
+ "nef-nikon-d5100-lossless.nef",
+ "https://raw.pixls.us/getfile.php/1597/nice/Nikon%20-%20D5100%20-%2014bit%2014bit%20compressed%20%28Lossless%29%20%283:2%29.nef",
+ ),
+ (
+ "nef-nikon-d5100-uncompressed.nef",
+ "https://raw.pixls.us/getfile.php/1598/nice/Nikon%20-%20D5100%20-%2014bit%2014bit%20uncompressed%20%283:2%29.nef",
+ ),
+ (
+ "nef-nikon-d7000-lossy12.nef",
+ "https://raw.pixls.us/getfile.php/961/nice/Nikon%20-%20D7000%20-%2012bit%2012bit%20compressed%20%28Lossy%20%28type%202%29%29%20%283:2%29.NEF",
+ ),
+ (
+ "nrw-nikon-b700-uncompressed.nrw",
+ "https://raw.pixls.us/getfile.php/1621/nice/Nikon%20-%20COOLPIX%20B700%20-%2012bit%2012bit%20uncompressed%20%284:3%29.NRW",
+ ),
+ ("orf-olympus-e1.orf", "https://raw.pixls.us/getfile.php/1800/nice/Olympus%20-%20E-1%20-%2016bit%20%284:3%29.ORF"),
+ ("orf-olympus-e400.orf", "https://raw.pixls.us/getfile.php/2151/nice/Olympus%20-%20E-400%20-%2016bit%20%284:3%29.ORF"),
+ ("orf-olympus-em1.orf", "https://raw.pixls.us/getfile.php/1051/nice/Olympus%20-%20E-M1%20-%2016bit%20%284:3%29.orf"),
+ ("orf-olympus-em10iii.orf", "https://raw.pixls.us/getfile.php/1787/nice/Olympus%20-%20E-M10%20Mark%20III%20-%2016bit%20%284:3%29.ORF"),
+ ("orf-olympus-xz2.orf", "https://raw.pixls.us/getfile.php/1432/nice/Olympus%20-%20XZ-2%20-%2012bit%20%284:3%29.orf"),
+ ("pef-pentax-k10d.pef", "https://raw.pixls.us/getfile.php/2239/nice/Pentax%20-%20K10D%20-%2012bit%2012bit%20compressed%20%283:2%29.PEF"),
+ ("pef-pentax-k3.pef", "https://raw.pixls.us/getfile.php/1075/nice/Pentax%20-%20K-3%20-%2014bit%20%283:2%29.PEF"),
+ ("pef-pentax-k5iis.pef", "https://raw.pixls.us/getfile.php/1198/nice/Pentax%20-%20K-5%20II%20s%20-%2014bit%20%283:2%29.PEF"),
+ (
+ "raf-fuji-xa2-12bit-bayer.raf",
+ "https://raw.pixls.us/getfile.php/2883/nice/Fujifilm%20-%20X-A2%20-%2012bit%2012bit%20uncompressed%20%283:2%29.RAF",
+ ),
+ (
+ "raf-fuji-xa5-14bit-bayer.raf",
+ "https://raw.pixls.us/getfile.php/2526/nice/Fujifilm%20-%20X-A5%20-%2014bit%2014bit%20uncompressed%20%283:2%29.RAF",
+ ),
+ ("raf-fuji-xe1-12bit.raf", "https://raw.pixls.us/getfile.php/3098/nice/Fujifilm%20-%20X-E1%20-%2012bit%2012bit%20uncompressed%20%283:2%29.RAF"),
+ ("raf-fuji-xt20-14bit.raf", "https://raw.pixls.us/getfile.php/1177/nice/Fujifilm%20-%20X-T20%20-%2014bit%2014bit%20uncompressed%20%283:2%29.RAF"),
+ (
+ "raf-fuji-xt20-compressed.raf",
+ "https://raw.pixls.us/getfile.php/1178/nice/Fujifilm%20-%20X-T20%20-%2014bit%2014bit%20compressed%20%283:2%29.RAF",
+ ),
+ ("rw2-panasonic-g9-b.rw2", "https://raw.pixls.us/getfile.php/2348/nice/Panasonic%20-%20DC-G9%20-%204:3.RW2"),
+ ("rw2-panasonic-g9.rw2", "https://raw.pixls.us/getfile.php/2585/nice/Panasonic%20-%20DC-G9%20-%204:3.RW2"),
+ ("rw2-panasonic-gh5.rw2", "https://raw.pixls.us/getfile.php/1517/nice/Panasonic%20-%20DC-GH5%20-%204:3.RW2"),
+ ("rw2-panasonic-gh5s.rw2", "https://raw.pixls.us/getfile.php/2603/nice/Panasonic%20-%20DC-GH5S%20-%204:3.RW2"),
+ ("rw2-panasonic-gx80.rw2", "https://raw.pixls.us/getfile.php/1569/nice/Panasonic%20-%20DMC-GX80%20-%204:3.RW2"),
+];
fn cmd_corpus(download: bool) -> Result<(), String> {
let corpus = root().join("corpus");