PdfCraft: An open-source PDF reader and editor written by scratch in pure Rust
M11: JBIG2 symbol coding for black-and-white images
Grzegorz Martens9edd08ce0758
diff --git a/ROADMAP.md b/ROADMAP.mdindex f2fa3e25a..6ae5a40a2 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -200,6 +200,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md## Progress log+- **2026-10-26 (M11, JBIG2 symbols):** Lossless JBIG2 now stores each distinct mark of a black-and-white image once, as a symbol, and places its exact copies, when that is smaller than the generic region: on six pages of rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's, and the same bits back through two independent decoders (hayro-jbig2 and Poppler). Scans, whose letters are seldom identical, gain little until refinement coding; a dictionary shared between pages (`/JBIG2Globals`) is next. Estimates unchanged.- **2026-10-27 (M10, OCR models):** Model staging downloads and verifies the ONNX pair after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, retains complete RTEN fallback and preserves directory precedence.- **2026-10-24 (M10, #4158 partial):** Word, HTML, RTF and PowerPoint export keep a block that continues another on its line (an italic case name, a link, a superscript) as a run in its own style inside that paragraph, with the source's word space, instead of a paragraph of its own. Style changes that wrap onto the next line still split paragraphs (the reading blocks keep style boundaries). Estimates unchanged.- **2026-10-24 (M3, window move on Wayland, #3970 follow-up):** After a move or resize by PdfCraft's own title bar or edges, the button release given back to egui goes before the pointer leaving the window, so hover no longer sticks at the old place until the pointer moves. Found by CodeRabbit in review of #3970; a unit test covers it. Estimates unchanged.diff --git a/crates/filters/README.md b/crates/filters/README.mdindex 98363e33d..0113e2217 100644--- a/crates/filters/README.md+++ b/crates/filters/README.md@@ -3,7 +3,7 @@PDF stream filters: the decoders and encoders behind `/Filter` and `/DecodeParms`.- **Layer:** L0, standalone. No dependency on any other PdfCraft crate. `cos` depends on it (architecture §3 rule 3).-- **Licence:** MIT OR Apache-2.0. Clean-room: written from ISO 32000-2 §7.4, RFC 1950/1951 (through `flate2` with the pure-Rust `miniz_oxide` backend), TIFF 6.0 §14 (predictor 2) and RFC 2083 §6 (PNG filters). CCITT Group 4 encoding (ITU-T T.6) is the `fax` crate's (MIT, already in the tree through `tiff`), checked here by round trips through an independent decoder. JBIG2 generic region encoding is written here from the JBIG committee's ISO/IEC 14492 Final Committee Draft (WG1 N1359, 1999), the public draft of ITU-T T.88; its Annex H.2 test sequence checks the arithmetic coder byte for byte.+- **Licence:** MIT OR Apache-2.0. Clean-room: written from ISO 32000-2 §7.4, RFC 1950/1951 (through `flate2` with the pure-Rust `miniz_oxide` backend), TIFF 6.0 §14 (predictor 2) and RFC 2083 §6 (PNG filters). CCITT Group 4 encoding (ITU-T T.6) is the `fax` crate's (MIT, already in the tree through `tiff`), checked here by round trips through an independent decoder. JBIG2 encoding (generic regions, symbol dictionaries and text regions) is written here from the JBIG committee's ISO/IEC 14492 Final Committee Draft (WG1 N1359, 1999), the public draft of ITU-T T.88; its Annex H.2 test sequence checks the arithmetic coder byte for byte, and its Annex A example the integer coders.- `#![forbid(unsafe_code)]`. Builds for `wasm32-unknown-unknown`.## API@@ -23,7 +23,8 @@ decode_tolerant_counted(chain, data, max_output) -> Result<(Vec<u8>, bool /*partencode(filter, params, data) -> Result<Vec<u8>, FilterError>encode_flate(data) -> Vec<u8> // zlib, level 6encode_g4(data, columns, rows) -> Result<Vec<u8>, FilterError> // 1-bit rows as CCITT Group 4; columns <= MAX_G4_COLUMNS-encode_jbig2_generic(data, columns, rows) -> Result<Vec<u8>, FilterError> // 1-bit rows as lossless JBIG2 for JBIG2Decode; columns <= MAX_JBIG2_COLUMNS+encode_jbig2(data, columns, rows) -> Result<Vec<u8>, FilterError> // 1-bit rows as lossless JBIG2 for JBIG2Decode, symbol coded when that is smaller; columns <= MAX_JBIG2_COLUMNS+encode_jbig2_generic(data, columns, rows) -> Result<Vec<u8>, FilterError> // the same as one generic region```- `chain` is applied first to last, as in a `/Filter` array.@@ -43,7 +44,7 @@ encode_jbig2_generic(data, columns, rows) -> Result<Vec<u8>, FilterError> // 1| ASCII85Decode | `<~` prefix, `z`, white space, `~>` (or bare `~`, or none) EOD, final partial group; `z` inside a group, a 1-char final group, overflow or bad characters → Corrupt | `z` for zero groups, newline every 75 characters, `~>` || RunLengthDecode | 0–127 literal, 129–255 repeat, 128 EOD (optional); truncated run → Corrupt | runs of ≥2 repeat, literals up to 128, 128 || CCITTFaxDecode | not here (`Unsupported`; renderers decode it) | `encode_g4`: Group 4 (`/K -1`) with an end-of-block mark, 0 bits coded black so `/BlackIs1 false` decodes the same bits; rows packed MSB first, each on a new byte, padding bits and trailing bytes ignored; up to 32,767 samples a row (the encoder counts positions in 16 bits); no rows or columns, or too wide → Unsupported, too few bytes → Corrupt |-| JBIG2Decode | not here (`Unsupported`; the renderer and image export decode it with `hayro-jbig2`) | `encode_jbig2_generic`: the embedded organisation (D.3) with a page information segment and an immediate lossless generic region (7.4.6), no `/JBIG2Globals`; template 0 with its nominal AT pixels and typical prediction (6.2.5), coded with the MQ arithmetic coder (Annex E, Table E.1) and ended by its 0xFF 0xAC marker. A 0 sample is coded as a JBIG2 1 (black), since `JBIG2Decode` gives black as 0, so the stream decodes to the same bits; rows packed MSB first, each on a new byte, padding bits and trailing bytes ignored; up to 1,048,576 samples a row; no rows or columns, or too wide → Unsupported, too few bytes → Corrupt |+| JBIG2Decode | not here (`Unsupported`; the renderer and image export decode it with `hayro-jbig2`) | `encode_jbig2_generic`: the embedded organisation (D.3) with a page information segment and an immediate lossless generic region (7.4.6), no `/JBIG2Globals`; template 0 with its nominal AT pixels and typical prediction (6.2.5), coded with the MQ arithmetic coder (Annex E, Table E.1) and ended by its 0xFF 0xAC marker. A 0 sample is coded as a JBIG2 1 (black), since `JBIG2Decode` gives black as 0, so the stream decodes to the same bits; rows packed MSB first, each on a new byte, padding bits and trailing bytes ignored; up to 1,048,576 samples a row; no rows or columns, or too wide → Unsupported, too few bytes → Corrupt. `encode_jbig2`: the smaller of that and symbol coding (`jbig2/symbols.rs`): each distinct mark (8-connected black pixels, at most 256 a side) stored once in a symbol dictionary (7.4.2; arithmetic coded, each bitmap a template 0 generic region without typical prediction, the bitmaps sharing their contexts) and placed by an immediate lossless text region referring to it (7.4.3; bottom-left reference corner, strips of 1, 2, 4 or 8 rows, whichever is smallest), with larger marks, and in a second plan marks that occur once, in a generic region over their box. Only exact copies share a symbol (no refinement), so the stream decodes to the same bits; fewer than two symbols, more than 2^20 black runs, 65,536 symbols or 16 MiB of symbol pixels, or more than 2^24 rows, and the generic coding is used |## Tests@@ -53,8 +54,9 @@ encode_jbig2_generic(data, columns, rows) -> Result<Vec<u8>, FilterError> // 1- `ccitt.rs`: Group 4 output decoded by `hayro-ccitt` (dev-dependency, an independent decoder) matches the input bits: a text-like page, padding bits, the widest row, and property tests over bitmaps of noise and of runs; any input returns rather than panics.- `mq.rs`: the arithmetic coder reproduces Annex H.2's 30 bytes from its 256 decisions.- `jbig2.rs`: output decoded by `hayro-jbig2` (dev-dependency, an independent decoder) matches the input bits: a text-like page (smaller than Group 4), edges, padding, all-white and all-black regions (typical rows), and property tests over bitmaps of noise and of runs; any input returns rather than panics.+- `jbig2/symbols.rs`: the integer coders against Annex A's example and Table A.1; symbol-coded output decoded by `hayro-jbig2` matches the input bits for every plan and strip size: text whose letters repeat (under half the generic coding's size, the dictionary retained and referred to), large and lone marks left to a generic region, marks on the edges, touching at corners and inside each other's boxes, and property tests over glyphs scattered over each other and the edges with stray pixels (never larger than the generic coding); one shape, a blank page and noise are coded generically; any input returns rather than panics.- `fuzz/`: a detached cargo-fuzz crate (`cargo +nightly fuzz run decode`, run from `crates/filters/`).## Status-M1.1/M1.2 for the general-purpose filters are done. Not yet here: the image codecs (DCT via zune-jpeg, JPX via hayro-jpeg2000, JBIG2 and CCITT decode, DCT encode), CCITT Group 3 encoding, JBIG2 symbol coding (text regions, symbol dictionaries, lossy) and Flate encode levels other than 6.+M1.1/M1.2 for the general-purpose filters are done. Not yet here: the image codecs (DCT via zune-jpeg, JPX via hayro-jpeg2000, JBIG2 and CCITT decode, DCT encode), CCITT Group 3 encoding, JBIG2 refinement coding (near-copies of a symbol), symbol dictionaries shared between images (`/JBIG2Globals`) and lossy JBIG2, and Flate encode levels other than 6.diff --git a/crates/filters/src/jbig2.rs b/crates/filters/src/jbig2.rsindex 8983c335f..d3b37e5ae 100644--- a/crates/filters/src/jbig2.rs+++ b/crates/filters/src/jbig2.rs@@ -1,8 +1,10 @@-//! JBIG2 generic region encoding (ITU-T T.88 | ISO/IEC 14492), for `JBIG2Decode`-//! (ISO 32000-2 §7.4.7): lossless, template 0 with its nominal adaptive pixels, typical prediction-//! on, MQ arithmetic coding ([`crate::mq`]). Written from the standard's 6.2.5 (coding a generic-//! region), 7.2 (segment headers), 7.4.1, 7.4.6 and 7.4.8 (the segments) and D.3 (the embedded-//! organisation PDF uses).+//! JBIG2 encoding (ITU-T T.88 | ISO/IEC 14492), for `JBIG2Decode` (ISO 32000-2 §7.4.7), lossless:+//! a generic region, template 0 with its nominal adaptive pixels, typical prediction on, MQ+//! arithmetic coding ([`crate::mq`]); or the image's marks as symbols ([`symbols`]). Written from+//! the standard's 6.2.5 (coding a generic region), 7.2 (segment headers), 7.4.1, 7.4.6 and 7.4.8+//! (the segments) and D.3 (the embedded organisation PDF uses).++mod symbols;use crate::FilterError;use crate::mq::{Context, MqEncoder};@@ -41,6 +43,23 @@ const SLTP: [u8; 16] = [1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1];/// Columns of padding either side of a row, so every template pixel of a row's pixels is in it.const PAD: usize = 4;+/// `rows` rows of `columns` 1-bit PDF samples as lossless JBIG2, the smaller of two codings:+/// [`encode_jbig2_generic`]'s, or one that stores each distinct mark of the image once, as a+/// symbol, and places its copies (see [`symbols`]). Either way the stream is in the embedded+/// organisation and decodes to the same bits without `/JBIG2Globals`.+///+/// Errors: as [`encode_jbig2_generic`].+pub fn encode_jbig2(data: &[u8], columns: u32, rows: u32) -> Result<Vec<u8>, FilterError> {+ let generic = encode_jbig2_generic(data, columns, rows)?;+ // `encode_jbig2_generic` checked that the rows are there.+ let stride = (columns as usize).div_ceil(8);+ let samples = usize::try_from(rows).ok().and_then(|r| r.checked_mul(stride)).and_then(|n| data.get(..n));+ Ok(match samples.and_then(|s| symbols::encode(s, columns, rows)) {+ Some(coded) if coded.len() < generic.len() => coded,+ _ => generic,+ })+}+/// `rows` rows of `columns` 1-bit PDF samples (most significant bit first, each row starting on/// a new byte), as a JBIG2 stream in the embedded organisation: a page information segment and an/// immediate lossless generic region segment, which `JBIG2Decode` decodes to the same bits@@ -62,69 +81,104 @@ pub fn encode_jbig2_generic(data: &[u8], columns: u32, rows: u32) -> Result<Vec<filter: "JBIG2Decode",detail: format!("{} bytes for {rows} rows of {columns} samples", data.len()),})?;- let region = generic_region(samples, width, stride);+ let mut contexts: Vec<Context> = vec![0; 1 << 16];+ let mut mq = MqEncoder::new();+ let height = samples.len() / stride;+ code_generic(&mut mq, &mut contexts, width, height, true, |y, row| {+ let packed = samples.get(y * stride..).unwrap_or_default();+ for (x, pixel) in row.iter_mut().enumerate() {+ let sample = packed.get(x / 8).is_some_and(|b| b & (0x80 >> (x % 8)) != 0);+ *pixel = u8::from(!sample);+ }+ });+ let region = mq.finish();let mut out = Vec::with_capacity(region.len() + 64);- // Page information (7.4.8): the region's size, resolution unknown, eventually lossless,- // default pixel 0 (white), OR combination, not striped.+ segment(&mut out, 0, 48, &[], false, &page_information(columns, rows));+ segment(&mut out, 1, 39, &[], false, &generic_region(columns, rows, 0, 0, ®ion));+ Ok(out)+}++/// A page information segment's data (7.4.8): the page's size, resolution unknown, eventually+/// lossless, default pixel 0 (white), OR combination, not striped.+fn page_information(columns: u32, rows: u32) -> Vec<u8> {let mut page = Vec::with_capacity(19);page.extend_from_slice(&columns.to_be_bytes());page.extend_from_slice(&rows.to_be_bytes());page.extend_from_slice(&[0; 8]);page.push(0x01);page.extend_from_slice(&[0, 0]);- segment(&mut out, 0, 48, &page);- // Immediate lossless generic region (7.4.6): region information (7.4.1) at (0, 0) with OR,- // flags (MMR 0, template 0, TPGDON 1), the AT pixels at their nominal places, then the data.- let mut generic = Vec::with_capacity(26 + region.len());- generic.extend_from_slice(&columns.to_be_bytes());- generic.extend_from_slice(&rows.to_be_bytes());- generic.extend_from_slice(&[0; 8]);- generic.push(0);- generic.push(0x08);+ page+}++/// A region segment information field (7.4.1): a `width` × `height` region at (`x`, `y`) on the+/// page, combined with it by OR.+fn region_information(out: &mut Vec<u8>, width: u32, height: u32, x: u32, y: u32) {+ for v in [width, height, x, y] {+ out.extend_from_slice(&v.to_be_bytes());+ }+ out.push(0);+}++/// An immediate lossless generic region segment's data (7.4.6): its region information, flags+/// (MMR 0, template 0, TPGDON 1), the AT pixels at their nominal places, then `coded`.+fn generic_region(width: u32, height: u32, x: u32, y: u32, coded: &[u8]) -> Vec<u8> {+ let mut data = Vec::with_capacity(26 + coded.len());+ region_information(&mut data, width, height, x, y);+ data.push(0x08);+ at_pixels(&mut data);+ data.extend_from_slice(coded);+ data+}++/// Template 0's four AT pixels at their nominal places, as AT flags (7.4.6.3, 7.4.2.1.2).+fn at_pixels(out: &mut Vec<u8>) {for (x, y) in &TEMPLATE[12..] {- generic.push(*x as i8 as u8);- generic.push(*y as i8 as u8);+ out.push(*x as i8 as u8);+ out.push(*y as i8 as u8);}- generic.extend_from_slice(®ion);- segment(&mut out, 1, 39, &generic);- Ok(out)}-/// A segment (7.2): its header (number, type, no referred-to segments, page 1, data length), then-/// its data.-fn segment(out: &mut Vec<u8>, number: u32, kind: u8, data: &[u8]) {+/// A segment (7.2): its header (number, type, the segments it refers to, page 1, data length),+/// then its data. `retained`: a later segment refers to this one. Segment numbers here are below+/// 256, so each referred-to one takes a byte (7.2.5); a segment refers to at most four, so their+/// count and retain bits take one byte too (7.2.4), and the retain bits of the referred-to ones+/// are 0: nothing after this segment refers to them.+fn segment(out: &mut Vec<u8>, number: u32, kind: u8, refers_to: &[u8], retained: bool, data: &[u8]) {out.extend_from_slice(&number.to_be_bytes());out.push(kind);- out.push(0);+ let count = refers_to.len().min(4);+ out.push((count as u8) << 5 | u8::from(retained));+ out.extend_from_slice(refers_to.get(..count).unwrap_or_default());out.push(1);// A segment's data never comes near 4 GiB here: a row is at most 1 MiB wide.out.extend_from_slice(&u32::try_from(data.len()).unwrap_or(u32::MAX).to_be_bytes());out.extend_from_slice(data);}-/// The arithmetic-coded data of a generic region (6.2.5.7) with template 0, the nominal AT pixels-/// and typical prediction.-fn generic_region(samples: &[u8], width: usize, stride: usize) -> Vec<u8> {- let mut contexts: Vec<Context> = vec![0; 1 << 16];- let mut mq = MqEncoder::new();+/// Codes a `width` × `height` bitmap as a generic region (6.2.5.7) with template 0 and the nominal+/// AT pixels, its decisions in `contexts` (2^16 of them), with typical prediction when `typical`+/// (TPGDON). `fill(y, row)` sets row `y`'s `width` pixels, 1 for black. Pixels outside the bitmap+/// count as 0 (white), as the decoder takes them.+fn code_generic(mq: &mut MqEncoder, contexts: &mut [Context], width: usize, height: usize, typical: bool, mut fill: impl FnMut(usize, &mut [u8])) {let sltp = SLTP.iter().enumerate().fold(0usize, |cx, (k, v)| cx | usize::from(*v) << k);// The two rows above the current one and the current one, as JBIG2 pixels (1 black), padded.let blank = vec![0u8; width + 2 * PAD];let (mut above2, mut above1) = (blank.clone(), blank.clone());let mut row = blank;let mut ltp = false;- for packed in samples.chunks_exact(stride) {- for (x, pixel) in row.iter_mut().skip(PAD).take(width).enumerate() {- let sample = packed.get(x / 8).is_some_and(|b| b & (0x80 >> (x % 8)) != 0);- *pixel = u8::from(!sample);+ for y in 0..height {+ if let Some(pixels) = row.get_mut(PAD..PAD + width) {+ fill(y, pixels);}- // Typical prediction (6.2.5.5): a row the same as the one above is coded as one decision.- let typical = row == above1;- if let Some(cx) = contexts.get_mut(sltp) {- mq.encode(cx, typical != ltp);+ if typical {+ // Typical prediction (6.2.5.5): a row the same as the one above is coded as one decision.+ let same = row == above1;+ if let Some(cx) = contexts.get_mut(sltp) {+ mq.encode(cx, same != ltp);+ }+ ltp = same;}- ltp = typical;- if !typical {+ if !ltp {for x in 0..width {let at = |dx: i32, dy: i32| {let line = match dy {@@ -144,7 +198,6 @@ fn generic_region(samples: &[u8], width: usize, stride: usize) -> Vec<u8> {std::mem::swap(&mut above2, &mut above1);std::mem::swap(&mut above1, &mut row);}- mq.finish()}#[cfg(test)]@@ -153,7 +206,7 @@ mod tests {/// The rows an independent decoder (`hayro-jbig2`) reads from `data`, as PDF samples: true/// for white (a JBIG2 0).- fn decode(data: &[u8]) -> Vec<Vec<bool>> {+ pub(super) fn decode(data: &[u8]) -> Vec<Vec<bool>> {struct Rows(Vec<Vec<bool>>, Vec<bool>, usize);impl hayro_jbig2::Decoder for Rows {fn push_pixel(&mut self, black: bool) {@@ -175,7 +228,7 @@ mod tests {}/// The rows of packed samples `data`, true for a 1 bit.- fn unpack(data: &[u8], columns: u32, rows: u32) -> Vec<Vec<bool>> {+ pub(super) fn unpack(data: &[u8], columns: u32, rows: u32) -> Vec<Vec<bool>> {let stride = (columns as usize).div_ceil(8);(0..rows as usize).map(|y| (0..columns as usize).map(|x| data[y * stride + x / 8] & (0x80 >> (x % 8)) != 0).collect()).collect()}@@ -242,7 +295,7 @@ mod tests {}/// Packed samples from JBIG2 pixels (1 black): a 0 sample for each 1.- fn samples(pixels: &[Vec<u8>]) -> Vec<u8> {+ pub(super) fn samples(pixels: &[Vec<u8>]) -> Vec<u8> {let width = pixels.first().map_or(0, Vec::len);let stride = width.div_ceil(8);let mut data = vec![0xFF; stride * pixels.len()];diff --git a/crates/filters/src/jbig2/symbols.rs b/crates/filters/src/jbig2/symbols.rsnew file mode 100644index 000000000..00aa18760--- /dev/null+++ b/crates/filters/src/jbig2/symbols.rs@@ -0,0 +1,776 @@+//! JBIG2 symbol coding, lossless: each distinct mark of an image (a group of black pixels that+//! touch, side by side or at a corner) is stored once, as a symbol in a symbol dictionary, and a+//! text region places a copy of it wherever it occurs. Marks too large to be symbols (rules,+//! frames, pictures) are left to a generic region beside it, and so, in [`encode`]'s second plan,+//! are marks that occur only once. Only exact copies share a symbol, so the image decodes to the+//! same bits. Written from the committee draft's 6.4 (the text region decoding procedure), 6.5+//! (the symbol dictionary decoding procedure), 7.4.2 and 7.4.3 (their segments) and Annex A (the+//! arithmetic integer coders).++use std::collections::HashMap;+use std::hash::{DefaultHasher, Hash, Hasher};+use std::ops::Range;++use super::{at_pixels, code_generic, generic_region, page_information, region_information, segment};+use crate::mq::{Context, MqEncoder};++/// The longest side of a mark stored as a symbol. A larger one is left to the generic region.+const MAX_SYMBOL_SIDE: u32 = 256;++/// The most black runs looked at. Past it (halftones, noise) the image is left to generic coding,+/// which suits it better anyway; this bounds the memory the marks take.+const MAX_RUNS: usize = 1 << 20;++/// The most distinct symbols, and the most pixels their bitmaps may hold together.+const MAX_SYMBOLS: usize = 1 << 16;+const MAX_SYMBOL_PIXELS: usize = 1 << 24;++/// The most rows symbol coded. With rows at most 2^20 samples wide, every coordinate and+/// difference coded then fits the 32-bit integers Annex A codes.+const MAX_ROWS: u32 = 1 << 24;++/// The integer coders' ranges (Table A.1): how many 1 bits pick the range, how many bits give the+/// offset into it, and its first value. The last range has no 0 after its prefix.+const RANGES: [(u32, u32, u32); 6] = [(0, 2, 0), (1, 4, 4), (2, 6, 20), (3, 8, 84), (4, 12, 340), (5, 32, 4436)];++/// A run of black pixels on row `y`, from `x0` up to but not including `x1`.+#[derive(Clone, Copy, Debug)]+struct Run {+ y: u32,+ x0: u32,+ x1: u32,+}++/// A mark: its box (`x1` and `y1` just past it) and its runs, as a range of the run order+/// [`marks`] gives.+#[derive(Debug)]+struct Mark {+ x0: u32,+ x1: u32,+ y0: u32,+ y1: u32,+ runs: Range<usize>,+}++/// A distinct mark's bitmap, a byte a pixel (1 black), and how many marks are exact copies of it.+#[derive(Debug)]+struct Symbol {+ width: u32,+ height: u32,+ pixels: Vec<u8>,+ uses: u32,+}++/// An image's marks, sorted out: its black runs, its marks with the run order they index, the+/// distinct marks of at most [`MAX_SYMBOL_SIDE`] a side as symbols, each such mark as (its+/// symbol, the mark), and the marks too large to be symbols.+struct Page {+ runs: Vec<Run>,+ marks: Vec<Mark>,+ order: Vec<u32>,+ symbols: Vec<Symbol>,+ copies: Vec<(u32, u32)>,+ large: Vec<u32>,+}++/// A symbol instance (6.4.5): the symbol, and where its bottom-left pixel goes, as S (the column)+/// and T (the row). `width` is the symbol's.+#[derive(Clone, Copy, Debug)]+struct Instance {+ id: u32,+ s: u32,+ t: u32,+ width: u32,+}++/// `rows` rows of `columns` 1-bit PDF samples (black 0), symbol coded: the smaller of two plans,+/// every mark of at most [`MAX_SYMBOL_SIDE`] a side as a symbol, or only those with a copy.+/// `None` when neither plan has two symbols, or the image is too large or too busy to look at.+pub(super) fn encode(samples: &[u8], columns: u32, rows: u32) -> Option<Vec<u8>> {+ let page = page(samples, columns, rows)?;+ [1, 2].into_iter().filter_map(|copies| plan(&page, columns, rows, copies, None)).min_by_key(Vec::len)+}++/// The image's marks and symbols; `None` past the limits above.+fn page(samples: &[u8], columns: u32, rows: u32) -> Option<Page> {+ if rows > MAX_ROWS {+ return None;+ }+ let width = usize::try_from(columns).ok()?;+ let stride = width.div_ceil(8);+ if stride == 0 {+ return None;+ }+ let runs = black_runs(samples, width, stride)?;+ let (marks, order) = marks(&runs);+ let mut page = Page { runs, marks, order, symbols: Vec::new(), copies: Vec::new(), large: Vec::new() };+ symbols(&mut page)?;+ Some(page)+}++/// The black runs of each row (0 samples), row by row and left to right; `None` past [`MAX_RUNS`].+fn black_runs(samples: &[u8], width: usize, stride: usize) -> Option<Vec<Run>> {+ let black = |row: &[u8], x: usize| row.get(x / 8).is_some_and(|b| b & (0x80 >> (x % 8)) == 0);+ let mut runs = Vec::new();+ for (y, row) in samples.chunks_exact(stride).enumerate() {+ let y = u32::try_from(y).ok()?;+ let mut x = 0;+ while x < width {+ // Whole white bytes at once.+ if x % 8 == 0 && row.get(x / 8) == Some(&0xFF) {+ x += 8;+ continue;+ }+ if !black(row, x) {+ x += 1;+ continue;+ }+ let start = x;+ while x < width && black(row, x) {+ x += 1;+ }+ if runs.len() == MAX_RUNS {+ return None;+ }+ runs.push(Run { y, x0: u32::try_from(start).ok()?, x1: u32::try_from(x).ok()? });+ }+ }+ Some(runs)+}++/// The marks the runs make, in the order their first runs come (so from the top), and the run+/// order: the runs' indexes, mark by mark, each mark's in row order.+fn marks(runs: &[Run]) -> (Vec<Mark>, Vec<u32>) {+ // Union–find over the runs; a set's root is its first run, as the larger root always joins+ // the smaller. Runs on adjacent rows join when they overlap, counting a corner.+ let mut parent: Vec<u32> = (0..runs.len()).map(|i| i as u32).collect();+ let (mut above, mut above_at, mut above_y): (&[Run], usize, Option<u32>) = (&[], 0, None);+ let mut at = 0;+ for row in runs.chunk_by(|a, b| a.y == b.y) {+ let y = row.first().map(|r| r.y);+ if above_y.and_then(|a| a.checked_add(1)) == y {+ let mut i = 0;+ for (j, b) in row.iter().enumerate() {+ while above.get(i).is_some_and(|a| a.x1 < b.x0) {+ i += 1;+ }+ let mut k = i;+ while above.get(k).is_some_and(|a| a.x0 <= b.x1) {+ union(&mut parent, above_at + k, at + j);+ k += 1;+ }+ }+ }+ (above, above_at, above_y) = (row, at, y);+ at += row.len();+ }+ let mut mark_of = vec![0u32; runs.len()];+ let mut marks: Vec<Mark> = Vec::new();+ let mut counts: Vec<usize> = Vec::new();+ for (i, r) in runs.iter().enumerate() {+ let root = find_root(&mut parent, i);+ let m = if root == i {+ marks.push(Mark { x0: r.x0, x1: r.x1, y0: r.y, y1: r.y.saturating_add(1), runs: 0..0 });+ counts.push(0);+ marks.len() - 1+ } else {+ mark_of.get(root).map_or(0, |m| *m as usize)+ };+ if let Some(slot) = mark_of.get_mut(i) {+ *slot = m as u32;+ }+ if let (Some(mark), Some(count)) = (marks.get_mut(m), counts.get_mut(m)) {+ mark.x0 = mark.x0.min(r.x0);+ mark.x1 = mark.x1.max(r.x1);+ mark.y1 = mark.y1.max(r.y.saturating_add(1));+ *count += 1;+ }+ }+ let mut next = 0;+ for (mark, count) in marks.iter_mut().zip(&counts) {+ mark.runs = next..next + count;+ next += count;+ }+ let mut order = vec![0u32; runs.len()];+ let mut fill: Vec<usize> = marks.iter().map(|m| m.runs.start).collect();+ for (i, m) in mark_of.iter().enumerate() {+ if let Some(n) = fill.get_mut(*m as usize) {+ if let Some(slot) = order.get_mut(*n) {+ *slot = i as u32;+ }+ *n += 1;+ }+ }+ (marks, order)+}++/// The root of `i`'s set, halving the path to it as it goes. Parents are never above their+/// children, so the walk ends.+fn find_root(parent: &mut [u32], mut i: usize) -> usize {+ loop {+ let Some(&p) = parent.get(i) else { return i };+ let p = p as usize;+ if p == i {+ return i;+ }+ let grandparent = parent.get(p).copied().unwrap_or(p as u32);+ if let Some(slot) = parent.get_mut(i) {+ *slot = grandparent;+ }+ i = grandparent as usize;+ }+}++fn union(parent: &mut [u32], a: usize, b: usize) {+ let (a, b) = (find_root(parent, a), find_root(parent, b));+ if let Some(slot) = parent.get_mut(a.max(b)) {+ *slot = a.min(b) as u32;+ }+}++/// Each mark's own pixels in its box: those of its runs, not of other marks the box overlaps.+fn pixels(mark: &Mark, runs: &[Run], order: &[u32]) -> Vec<u8> {+ // A mark's box holds its runs, so these don't go below 0; a symbol is at most 256 a side.+ let width = mark.x1.saturating_sub(mark.x0) as usize;+ let height = mark.y1.saturating_sub(mark.y0) as usize;+ let mut pixels = vec![0u8; width * height];+ for r in order.get(mark.runs.clone()).unwrap_or_default().iter().filter_map(|i| runs.get(*i as usize)) {+ let row = r.y.saturating_sub(mark.y0) as usize * width;+ let from = row + r.x0.saturating_sub(mark.x0) as usize;+ let to = row + r.x1.saturating_sub(mark.x0) as usize;+ pixels.get_mut(from..to).unwrap_or_default().fill(1);+ }+ pixels+}++/// Sorts the marks into symbols: exact copies share one. `None` past [`MAX_SYMBOLS`] or+/// [`MAX_SYMBOL_PIXELS`].+fn symbols(page: &mut Page) -> Option<()> {+ let mut by_hash: HashMap<u64, Vec<u32>> = HashMap::new();+ let mut kept = 0usize;+ for (m, mark) in page.marks.iter().enumerate() {+ let m = u32::try_from(m).ok()?;+ let (width, height) = (mark.x1.saturating_sub(mark.x0), mark.y1.saturating_sub(mark.y0));+ if width > MAX_SYMBOL_SIDE || height > MAX_SYMBOL_SIDE {+ page.large.push(m);+ continue;+ }+ let pixels = pixels(mark, &page.runs, &page.order);+ let mut hasher = DefaultHasher::new();+ (width, height, &pixels).hash(&mut hasher);+ let same = by_hash.entry(hasher.finish()).or_default();+ let symbols = &mut page.symbols;+ let known =+ same.iter().copied().find(|s| symbols.get(*s as usize).is_some_and(|s| (s.width, s.height) == (width, height) && s.pixels == pixels));+ let id = match known {+ Some(id) => id,+ None => {+ kept = kept.checked_add(pixels.len())?;+ if kept > MAX_SYMBOL_PIXELS || symbols.len() == MAX_SYMBOLS {+ return None;+ }+ let id = u32::try_from(symbols.len()).ok()?;+ symbols.push(Symbol { width, height, pixels, uses: 0 });+ same.push(id);+ id+ }+ };+ if let Some(s) = symbols.get_mut(id as usize) {+ s.uses = s.uses.saturating_add(1);+ }+ page.copies.push((id, m));+ }+ Some(())+}++/// One plan's stream: the symbols with at least `copies` uses in a symbol dictionary, a text+/// region placing them (in strips of 2^`log_strips` rows, or whichever size codes smallest), and+/// the other marks in a generic region. `None` with fewer than two such symbols: their IDs would+/// take no bits at all (SBSYMCODELEN 0), a corner of the standard left alone here.+fn plan(page: &Page, columns: u32, rows: u32, copies: u32, log_strips: Option<u32>) -> Option<Vec<u8>> {+ let symbol = |s: u32| page.symbols.get(s as usize);+ // The dictionary's order (6.5.5): height classes from the shortest, each narrowest first.+ let mut kept: Vec<u32> = (0..page.symbols.len() as u32).filter(|s| symbol(*s).is_some_and(|s| s.uses >= copies)).collect();+ if kept.len() < 2 {+ return None;+ }+ kept.sort_by_key(|s| symbol(*s).map(|s| (s.height, s.width)));+ let mut id = vec![None; page.symbols.len()];+ for (k, s) in kept.iter().enumerate() {+ if let Some(slot) = id.get_mut(*s as usize) {+ *slot = u32::try_from(k).ok();+ }+ }+ let mut instances = Vec::new();+ let mut left = page.large.clone();+ for (s, m) in &page.copies {+ match (id.get(*s as usize).copied().flatten(), page.marks.get(*m as usize), symbol(*s)) {+ (Some(id), Some(mark), Some(symbol)) => instances.push(Instance { id, s: mark.x0, t: mark.y1.saturating_sub(1), width: symbol.width }),+ _ => left.push(*m),+ }+ }+ let kept: Vec<&Symbol> = kept.iter().filter_map(|s| symbol(*s)).collect();+ let dictionary = symbol_dictionary(&kept)?;+ let count = u32::try_from(kept.len()).ok()?;+ let text = match log_strips {+ Some(l) => text_region(&mut instances, count, columns, rows, l)?,+ None => (0..4).filter_map(|l| text_region(&mut instances, count, columns, rows, l)).min_by_key(Vec::len)?,+ };+ let mut out = Vec::new();+ segment(&mut out, 0, 48, &[], false, &page_information(columns, rows));+ segment(&mut out, 1, 0, &[], true, &dictionary);+ segment(&mut out, 2, 7, &[1], false, &text);+ if !left.is_empty() {+ segment(&mut out, 3, 39, &[], false, &leftover_region(&left, page)?);+ }+ Some(out)+}++/// A symbol dictionary segment's data (7.4.2) defining `symbols` (in dictionary order) and+/// exporting them all: arithmetic coded, no refinement or aggregation, each bitmap a generic+/// region with template 0, its nominal AT pixels and no typical prediction (Table 16), the+/// bitmaps sharing one set of contexts.+fn symbol_dictionary(symbols: &[&Symbol]) -> Option<Vec<u8>> {+ let count = u32::try_from(symbols.len()).ok()?;+ let mut data = vec![0, 0];+ at_pixels(&mut data);+ data.extend_from_slice(&count.to_be_bytes());+ data.extend_from_slice(&count.to_be_bytes());+ let mut mq = MqEncoder::new();+ let mut contexts: Vec<Context> = vec![0; 1 << 16];+ let (mut iadh, mut iadw, mut iaex) = (IntCoder::new(), IntCoder::new(), IntCoder::new());+ let mut height = 0;+ for class in symbols.chunk_by(|a, b| a.height == b.height) {+ let h = class.first()?.height;+ iadh.encode(&mut mq, Some(int(i64::from(h) - i64::from(height))?));+ height = h;+ let mut width = 0;+ for s in class {+ iadw.encode(&mut mq, Some(int(i64::from(s.width) - i64::from(width))?));+ width = s.width;+ let w = s.width as usize;+ code_generic(&mut mq, &mut contexts, w, s.height as usize, false, |y, row| {+ let line = s.pixels.get(y * w..).unwrap_or_default();+ row.iter_mut().zip(line).for_each(|(p, v)| *p = *v);+ });+ }+ iadw.encode(&mut mq, None);+ }+ // The export flags (6.5.10): a run of no symbols not exported, then a run of all of them.+ iaex.encode(&mut mq, Some(0));+ iaex.encode(&mut mq, Some(int(i64::from(count))?));+ data.extend_from_slice(&mq.finish());+ Some(data)+}++/// An immediate lossless text region segment's data (7.4.3) placing `instances` over the whole+/// page: arithmetic coded, no refinement, strips of 2^`log_strips` rows, each symbol's+/// bottom-left pixel at (S, T) (REFCORNER BOTTOMLEFT, so a line's letters share their T),+/// not transposed, OR, default pixel 0, SBDSOFFSET 0. Sorts `instances` into strips, each+/// left to right.+fn text_region(instances: &mut [Instance], symbols: u32, columns: u32, rows: u32, log_strips: u32) -> Option<Vec<u8>> {+ // LOGSBSTRIPS has two bits: strips of 1, 2, 4 or 8 rows.+ if log_strips > 3 {+ return None;+ }+ let strips = 1i64 << log_strips;+ instances.sort_by_key(|i| (i.t >> log_strips, i.s));+ let mut data = Vec::new();+ region_information(&mut data, columns, rows, 0, 0);+ data.extend_from_slice(&((log_strips as u16) << 2).to_be_bytes());+ data.extend_from_slice(&u32::try_from(instances.len()).ok()?.to_be_bytes());+ // SBSYMCODELEN (Table 31): ⌈log2 SBNUMSYMS⌉.+ let length = symbols.checked_next_power_of_two()?.trailing_zeros();+ let mut ids: Vec<Context> = vec![0; 1 << length];+ let (mut iadt, mut iafs, mut iads, mut iait) = (IntCoder::new(), IntCoder::new(), IntCoder::new(), IntCoder::new());+ let mut mq = MqEncoder::new();+ // The initial STRIPT (6.4.5 step 2) is 0, so each strip's delta T is its first row over the+ // strip size, from the strip before.+ iadt.encode(&mut mq, Some(0));+ let (mut strip_t, mut first_s) = (0i64, 0i64);+ for strip in instances.chunk_by(|a, b| a.t >> log_strips == b.t >> log_strips) {+ let t = i64::from(strip.first()?.t >> log_strips << log_strips);+ iadt.encode(&mut mq, Some(int((t - strip_t) / strips)?));+ strip_t = t;+ let mut cur_s = 0;+ for (k, instance) in strip.iter().enumerate() {+ let s = i64::from(instance.s);+ if k == 0 {+ iafs.encode(&mut mq, Some(int(s - first_s)?));+ first_s = s;+ } else {+ iads.encode(&mut mq, Some(int(s - cur_s)?));+ }+ if strips > 1 {+ iait.encode(&mut mq, Some(int(i64::from(instance.t) - strip_t)?));+ }+ encode_id(&mut mq, &mut ids, length, instance.id);+ // The next S counts from this symbol's last column (6.4.5 step 3c x).+ cur_s = s + i64::from(instance.width) - 1;+ }+ iads.encode(&mut mq, None);+ }+ data.extend_from_slice(&mq.finish());+ Some(data)+}++/// A generic region segment's data holding the marks `left`, over their box.+fn leftover_region(left: &[u32], page: &Page) -> Option<Vec<u8>> {+ let left: Vec<&Mark> = left.iter().filter_map(|m| page.marks.get(*m as usize)).collect();+ let x0 = left.iter().map(|m| m.x0).min()?;+ let y0 = left.iter().map(|m| m.y0).min()?;+ let x1 = left.iter().map(|m| m.x1).max()?;+ let y1 = left.iter().map(|m| m.y1).max()?;+ let (width, height) = (x1.saturating_sub(x0) as usize, y1.saturating_sub(y0) as usize);+ let stride = width.div_ceil(8);+ // Packed, 1 for black: no larger than the image's own samples.+ let mut packed = vec![0u8; stride.checked_mul(height)?];+ for mark in &left {+ for r in page.order.get(mark.runs.clone()).unwrap_or_default().iter().filter_map(|i| page.runs.get(*i as usize)) {+ let row = r.y.saturating_sub(y0) as usize * stride;+ for x in r.x0.saturating_sub(x0) as usize..r.x1.saturating_sub(x0) as usize {+ if let Some(b) = packed.get_mut(row + x / 8) {+ *b |= 0x80 >> (x % 8);+ }+ }+ }+ }+ let mut mq = MqEncoder::new();+ let mut contexts: Vec<Context> = vec![0; 1 << 16];+ code_generic(&mut mq, &mut contexts, width, height, true, |y, row| {+ let line = packed.get(y * stride..).unwrap_or_default();+ for (x, p) in row.iter_mut().enumerate() {+ *p = u8::from(line.get(x / 8).is_some_and(|b| b & (0x80 >> (x % 8)) != 0));+ }+ });+ Some(generic_region(u32::try_from(width).ok()?, u32::try_from(height).ok()?, x0, y0, &mq.finish()))+}++fn int(v: i64) -> Option<i32> {+ i32::try_from(v).ok()+}++/// An arithmetic integer coder other than IAID (A.2): its 512 contexts, indexed by PREV.+struct IntCoder([Context; 512]);++impl IntCoder {+ fn new() -> Self {+ IntCoder([0; 512])+ }++ /// Codes `value`, or OOB for `None`.+ fn encode(&mut self, mq: &mut MqEncoder, value: Option<i32>) {+ for (cx, d) in decisions(value) {+ if let Some(state) = self.0.get_mut(cx) {+ mq.encode(state, d);+ }+ }+ }+}++/// The decisions that code `value`, or OOB for `None`, with their contexts (A.2, Table A.1): the+/// sign, the prefix that picks the range, then the offset into it, most significant bit first;+/// OOB is a negative 0. Each decision's context is PREV: a leading 1 and the decisions before it,+/// at most the last eight once there are more.+fn decisions(value: Option<i32>) -> Vec<(usize, bool)> {+ let (negative, magnitude) = value.map_or((true, 0), |v| (v < 0, v.unsigned_abs()));+ let (ones, bits, first) =+ RANGES.into_iter().find(|(_, bits, first)| *bits == 32 || u64::from(magnitude) < u64::from(*first) + (1 << bits)).unwrap_or(RANGES[5]);+ let offset = magnitude - first;+ let mut d = vec![negative];+ d.extend(std::iter::repeat_n(true, ones as usize));+ if ones < 5 {+ d.push(false);+ }+ d.extend((0..bits).rev().map(|i| offset >> i & 1 == 1));+ let mut prev = 1usize;+ d.into_iter()+ .map(|bit| {+ let cx = prev;+ prev = if prev < 256 { prev << 1 | usize::from(bit) } else { ((prev << 1 | usize::from(bit)) & 511) | 256 };+ (cx, bit)+ })+ .collect()+}++/// Codes symbol ID `id` in `length` bits, most significant first, through IAID (A.3): each+/// decision's context is a leading 1 and all the bits before it.+fn encode_id(mq: &mut MqEncoder, contexts: &mut [Context], length: u32, id: u32) {+ let mut prev = 1usize;+ for i in (0..length).rev() {+ let d = id >> i & 1 == 1;+ if let Some(state) = contexts.get_mut(prev) {+ mq.encode(state, d);+ }+ prev = prev << 1 | usize::from(d);+ }+}++#[cfg(test)]+mod tests {+ use super::*;+ use crate::jbig2::tests::{decode, samples, unpack};+ use crate::jbig2::{encode_jbig2, encode_jbig2_generic};++ type Bitmap = Vec<Vec<u8>>;++ /// A small linear congruential generator, so the pictures are the same every run.+ struct Lcg(u64);++ impl Lcg {+ fn below(&mut self, n: usize) -> usize {+ self.0 = self.0.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1_442_695_040_888_963_407);+ (self.0 >> 33) as usize % n.max(1)+ }+ }++ /// `count` different letter-like shapes: three strokes two pixels thick in a box of 6 to 18+ /// pixels a side, each after the first starting on one before it, so a shape is one mark;+ /// trimmed to its pixels.+ fn glyphs(count: usize, seed: u64) -> Vec<Bitmap> {+ let mut r = Lcg(seed);+ let mut out: Vec<Bitmap> = Vec::new();+ while out.len() < count {+ let (w, h) = (6 + r.below(13), 6 + r.below(13));+ let mut g = vec![vec![0u8; w]; h];+ let (mut x, mut y) = (r.below(w), r.below(h));+ for _ in 0..3 {+ let (dx, dy) = [(1, 0), (0, 1), (1, 1)][r.below(3)];+ let mut stroke = Vec::new();+ while x < w && y < h {+ g[y][x] = 1;+ g[y][(x + 1).min(w - 1)] = 1;+ stroke.push((x, y));+ x += dx;+ y += dy;+ }+ (x, y) = stroke[r.below(stroke.len())];+ }+ let rows: Vec<usize> = (0..h).filter(|y| g[*y].contains(&1)).collect();+ let columns: Vec<usize> = (0..w).filter(|x| g.iter().any(|row| row[*x] == 1)).collect();+ let g: Bitmap = g[rows[0]..=rows[rows.len() - 1]].iter().map(|row| row[columns[0]..=columns[columns.len() - 1]].to_vec()).collect();+ if !out.contains(&g) {+ out.push(g);+ }+ }+ out+ }++ /// Draws `glyph` with its top-left pixel at (`x`, `y`), clipped to the page, by OR.+ fn draw(page: &mut Bitmap, glyph: &Bitmap, x: i64, y: i64) {+ for (gy, row) in glyph.iter().enumerate() {+ for (gx, p) in row.iter().enumerate() {+ let (px, py) = (x + gx as i64, y + gy as i64);+ if *p == 1+ && px >= 0+ && py >= 0+ && let Some(cell) = page.get_mut(py as usize).and_then(|r| r.get_mut(px as usize))+ {+ *cell = 1;+ }+ }+ }+ }++ /// Lines of text: glyphs side by side, one to four pixels apart, each line's glyphs on its+ /// baseline, the lines 30 rows apart; the first glyphs are the most common, as letters are.+ fn text(columns: usize, rows: usize, glyphs: &[Bitmap], seed: u64) -> Bitmap {+ let mut page = vec![vec![0u8; columns]; rows];+ let mut r = Lcg(seed);+ for baseline in (24..rows).step_by(30) {+ let mut x = 4 + r.below(8);+ loop {+ let common = r.below(glyphs.len()) + 1;+ let g = &glyphs[r.below(common)];+ if x + g[0].len() > columns {+ break;+ }+ draw(&mut page, g, x as i64, (baseline + 1 - g.len()) as i64);+ x += g[0].len() + 1 + r.below(4);+ }+ }+ page+ }++ /// `n` glyphs anywhere, some over the edges, some over each other.+ fn scatter(columns: usize, rows: usize, glyphs: &[Bitmap], seed: u64, n: usize) -> Bitmap {+ let mut page = vec![vec![0u8; columns]; rows];+ let mut r = Lcg(seed);+ for _ in 0..n {+ let g = &glyphs[r.below(glyphs.len())];+ draw(&mut page, g, r.below(columns + 12) as i64 - 6, r.below(rows + 12) as i64 - 6);+ }+ page+ }++ /// The segments of a stream in the embedded organisation, from their headers (7.2): each one's+ /// type, whether it is retained, and the segments it refers to.+ fn segments(data: &[u8]) -> Vec<(u8, bool, Vec<u8>)> {+ let (mut segments, mut at) = (Vec::new(), 0);+ while at < data.len() {+ let refers = usize::from(data[at + 5] >> 5);+ let length_at = at + 6 + refers + 1;+ let length = u32::from_be_bytes(data[length_at..length_at + 4].try_into().unwrap()) as usize;+ segments.push((data[at + 4] & 0x3F, data[at + 5] & 1 == 1, data[at + 6..at + 6 + refers].to_vec()));+ at = length_at + 4 + length;+ }+ segments+ }++ fn segment_types(data: &[u8]) -> Vec<u8> {+ segments(data).into_iter().map(|(kind, ..)| kind).collect()+ }++ /// Every plan, at every strip size, decodes to the picture; returns how many plans there were.+ fn every_plan_round_trips(picture: &Bitmap) -> usize {+ let (columns, rows) = (picture[0].len() as u32, picture.len() as u32);+ let data = samples(picture);+ let page = page(&data, columns, rows).unwrap();+ let mut plans = 0;+ for copies in [1, 2] {+ for log_strips in 0..4 {+ if let Some(coded) = plan(&page, columns, rows, copies, Some(log_strips)) {+ assert_eq!(decode(&coded), unpack(&data, columns, rows), "{copies} copies, strips of {}", 1 << log_strips);+ plans += 1;+ }+ }+ }+ plans+ }++ /// Annex A.2's example (12 through IADW: the decisions 0101000, in contexts 1, 10, 101, …)+ /// and Table A.1's codes.+ #[test]+ fn integers_are_coded_as_annex_a_says() {+ assert_eq!(decisions(Some(12)), [(1, false), (2, true), (5, false), (10, true), (21, false), (42, false), (84, false)]);+ let bits = |v| decisions(v).iter().map(|(_, d)| u8::from(*d)).collect::<Vec<u8>>();+ assert_eq!(bits(None), [1, 0, 0, 0], "OOB");+ assert_eq!(bits(Some(0)), [0, 0, 0, 0]);+ assert_eq!(bits(Some(-1)), [1, 0, 0, 1]);+ assert_eq!(bits(Some(-3)), [1, 0, 1, 1]);+ assert_eq!(bits(Some(4)), [0, 1, 0, 0, 0, 0, 0]);+ assert_eq!(bits(Some(-19)), [1, 1, 0, 1, 1, 1, 1]);+ assert_eq!(bits(Some(83)), [0, 1, 1, 0, 1, 1, 1, 1, 1, 1]);+ assert_eq!(bits(Some(4435))[..6], [0, 1, 1, 1, 1, 0]);+ let big = bits(Some(4436));+ assert_eq!((big.len(), &big[..6]), (6 + 32, &[0, 1, 1, 1, 1, 1][..]));+ assert!(big[6..].iter().all(|b| *b == 0));+ // PREV keeps a leading 1 and the last eight decisions.+ let contexts: Vec<usize> = decisions(Some(i32::MIN)).iter().map(|(cx, _)| *cx).collect();+ assert!(contexts[..8].iter().all(|cx| *cx < 256) && contexts[8..].iter().all(|cx| (256..512).contains(cx)));+ }++ /// A page of text whose letters repeat exactly, as rendered text does, keeps each letter once:+ /// a symbol dictionary and a text region, much smaller than the generic coding.+ #[test]+ fn repeated_marks_are_stored_once() {+ let picture = text(1700, 400, &glyphs(40, 7), 11);+ let data = samples(&picture);+ let coded = encode_jbig2(&data, 1700, 400).unwrap();+ let generic = encode_jbig2_generic(&data, 1700, 400).unwrap();+ // The dictionary is retained for the text region, which refers to it (7.2.4, 7.2.5).+ assert_eq!(segments(&coded), [(48, false, vec![]), (0, true, vec![]), (7, false, vec![1])]);+ assert_eq!(decode(&coded), unpack(&data, 1700, 400));+ assert!(coded.len() * 2 < generic.len(), "symbols {} bytes, generic {}", coded.len(), generic.len());+ assert_eq!(every_plan_round_trips(&picture), 8);+ }++ /// Marks too large to be symbols (a frame, a rule) go to a generic region over their box, and+ /// with the second plan so do marks that occur once; the page still decodes to its bits.+ #[test]+ fn large_and_lone_marks_are_left_to_a_generic_region() {+ let mut picture = text(900, 300, &glyphs(25, 3), 5);+ picture[10][100..800].fill(1);+ picture[290][100..800].fill(1);+ for row in &mut picture[10..=290] {+ row[100] = 1;+ row[799] = 1;+ }+ // A lone blob, unlike any letter.+ for (y, row) in picture[150..170].iter_mut().enumerate() {+ row[850 + y % 7] = 1;+ }+ let data = samples(&picture);+ let coded = encode_jbig2(&data, 900, 300).unwrap();+ assert_eq!(segment_types(&coded), [48, 0, 7, 39]);+ assert_eq!(decode(&coded), unpack(&data, 900, 300));+ assert_eq!(every_plan_round_trips(&picture), 8);+ }++ /// Marks on every edge, marks touching only at a corner (one mark), a mark inside another's+ /// box, and marks over each other's boxes.+ #[test]+ fn marks_at_the_edges_and_corners_round_trip() {+ let dot = vec![vec![1u8, 1], vec![1, 1]];+ let ell = vec![vec![1u8, 0, 0], vec![1, 0, 0], vec![1, 1, 1]];+ let mut picture = vec![vec![0u8; 41]; 23];+ for (x, y) in [(0, 0), (39, 0), (0, 21), (39, 21), (20, 0), (0, 10), (39, 10), (20, 21)] {+ draw(&mut picture, &dot, x, y);+ }+ for (x, y) in [(10, 5), (28, 14), (12, 14)] {+ draw(&mut picture, &ell, x, y);+ }+ // Corners touching: one mark.+ picture[3][6] = 1;+ picture[4][7] = 1;+ picture[5][8] = 1;+ // A ring with a dot inside, and an ell over the ring's box.+ for i in 0..7 {+ picture[8][20 + i] = 1;+ picture[14][20 + i] = 1;+ picture[8 + i][20] = 1;+ picture[8 + i][26] = 1;+ }+ draw(&mut picture, &dot, 22, 10);+ draw(&mut picture, &ell, 25, 15);+ assert_eq!(every_plan_round_trips(&picture), 8);+ }++ /// One shape, however often, would need symbol IDs of no bits: that image is coded as a generic+ /// region, as are a blank page and pure noise.+ #[test]+ fn one_shape_blank_pages_and_noise_are_coded_generically() {+ let letter = &glyphs(1, 9)[..];+ let mut r = Lcg(1);+ let noise: Bitmap = (0..60).map(|_| (0..200).map(|_| u8::from(r.below(2) == 1)).collect()).collect();+ for picture in [text(600, 200, letter, 2), vec![vec![0u8; 300]; 40], noise] {+ let (columns, rows) = (picture[0].len() as u32, picture.len() as u32);+ let data = samples(&picture);+ assert_eq!(encode_jbig2(&data, columns, rows).unwrap(), encode_jbig2_generic(&data, columns, rows).unwrap());+ }+ }++ proptest::proptest! {+ /// Glyphs anywhere, over the edges and each other, with stray pixels: every plan decodes to+ /// the picture, and `encode_jbig2` is never larger than the generic coding.+ #[test]+ fn scattered_marks_round_trip(+ columns in 1usize..240,+ rows in 1usize..120,+ shapes in 2usize..10,+ n in 0usize..80,+ seed in proptest::num::u64::ANY,+ stray in proptest::collection::vec((0usize..240, 0usize..120), 0..20),+ ) {+ let mut picture = scatter(columns, rows, &glyphs(shapes, seed), seed, n);+ for (x, y) in stray {+ if let Some(cell) = picture.get_mut(y).and_then(|r| r.get_mut(x)) {+ *cell ^= 1;+ }+ }+ every_plan_round_trips(&picture);+ let data = samples(&picture);+ let (c, r) = (columns as u32, rows as u32);+ let coded = encode_jbig2(&data, c, r).unwrap();+ proptest::prop_assert!(coded.len() <= encode_jbig2_generic(&data, c, r).unwrap().len());+ proptest::prop_assert_eq!(decode(&coded), unpack(&data, c, r));+ }++ /// Whatever it is given, it returns.+ #[test]+ fn any_input_returns(data in proptest::collection::vec(0u8..=255, 0..600), columns in 0u32..5000, rows in 0u32..80) {+ let _ = encode_jbig2(&data, columns, rows);+ let _ = encode(&data, columns, rows);+ }+ }+}diff --git a/crates/filters/src/lib.rs b/crates/filters/src/lib.rsindex da675a024..30094e652 100644--- a/crates/filters/src/lib.rs+++ b/crates/filters/src/lib.rs@@ -7,7 +7,7 @@//! Image codecs (`DCTDecode`, `JPXDecode`, `JBIG2Decode`, `CCITTFaxDecode`),//! `Crypt` and unknown filters are reported as [`FilterError::Unsupported`] so//! callers keep such data encoded. 1-bit samples can be encoded as CCITT Group 4-//! ([`encode_g4`]).+//! ([`encode_g4`]) or lossless JBIG2 ([`encode_jbig2`]).//!//! Every decoder is bounded by a caller-supplied `max_output` (decompression-bomb//! defence) and never panics on malformed input.@@ -27,7 +27,7 @@ mod runlength;use std::borrow::Cow;pub use ccitt::{MAX_G4_COLUMNS, encode_g4};-pub use jbig2::{MAX_JBIG2_COLUMNS, encode_jbig2_generic};+pub use jbig2::{MAX_JBIG2_COLUMNS, encode_jbig2, encode_jbig2_generic};/// A PDF stream filter, identified by its `/Filter` name.#[derive(Clone, Debug, PartialEq, Eq, Hash)]diff --git a/crates/optimize/README.md b/crates/optimize/README.mdindex a1c47697b..4c8ae1cd3 100644--- a/crates/optimize/README.md+++ b/crates/optimize/README.md@@ -37,12 +37,15 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going- **Monochrome images** (`Settings::mono`, CCITT Group 4 by default, or JBIG2): a 1-bit imagewith one colour component (grey, indexed, a separation) or an image mask, stored with no filteror with general-purpose ones, is stored again as CCITT Group 4 (`pdfcraft_filters::encode_g4`)- or lossless JBIG2 (`encode_jbig2_generic`; a document below PDF 1.4 gets `/Version /1.4` in- its catalog) when that is smaller as stored. On 300 dpi text JBIG2 is about a third smaller- than Group 4. It is lossless: the samples decode to the same bits, so the colour+ or lossless JBIG2 (`encode_jbig2`; a document below PDF 1.4 gets `/Version /1.4` in+ its catalog) when that is smaller as stored. JBIG2 stores each distinct mark once, as a+ symbol, when the image's marks repeat exactly, as rendered text's do: on six pages of+ 300 dpi text that came to 28% of a generic region's size and 17% of Group 4's. On scans, whose copies of a letter are seldom identical, it is a generic region,+ about a third smaller than Group 4. It is lossless: the samples decode to the same bits, so the colourspace, decode array and masks stay. Group 4 wins on scans, whose ragged edges defeat Flate;on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Not yet:- JBIG2 symbol coding (lossy or lossless), monochrome downsampling, inline images, soft masks and+ JBIG2 refinement coding (lossless gains on scans), a symbol dictionary shared between images+ (`/JBIG2Globals`), lossy JBIG2, monochrome downsampling, inline images, soft masks and`/Mask` images, and Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an imageremoves the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an imagediff --git a/crates/optimize/src/images.rs b/crates/optimize/src/images.rsindex cf668d0fb..392e9145e 100644--- a/crates/optimize/src/images.rs+++ b/crates/optimize/src/images.rs@@ -596,9 +596,10 @@ fn mono(doc: &Document, s: &Stream, compression: MonoCompression) -> Option<Strenew.remove(k);}let data = if compression == MonoCompression::Jbig2 {- // One embedded JBIG2 stream with its own page; no /JBIG2Globals.+ // One embedded JBIG2 stream with its own page (and symbol dictionary, when the image's+ // marks repeat); no /JBIG2Globals.new.set(b"Filter".to_vec(), Object::name("JBIG2Decode"));- pdfcraft_filters::encode_jbig2_generic(&samples, w, h).ok()?+ pdfcraft_filters::encode_jbig2(&samples, w, h).ok()?} else {let mut parms = Dict::new();parms.set(b"K".to_vec(), Object::Int(-1));diff --git a/crates/optimize/src/lib.rs b/crates/optimize/src/lib.rsindex 09c487989..7ef159d22 100644--- a/crates/optimize/src/lib.rs+++ b/crates/optimize/src/lib.rs@@ -124,8 +124,9 @@ pub enum Compression {pub enum MonoCompression {/// CCITT Group 4 (lossless), when smaller than the image as it is.CcittG4,- /// JBIG2, lossless (a generic region), when smaller than the image as it is. JBIG2Decode is- /// PDF 1.4: an older document gets `/Version /1.4` in its catalog.+ /// JBIG2, lossless (a generic region, or symbols when the image's marks repeat), when smaller+ /// than the image as it is. JBIG2Decode is PDF 1.4: an older document gets `/Version /1.4` in+ /// its catalog.Jbig2,/// Keep each image's own compression.Retain,diff --git a/crates/optimize/tests/optimize.rs b/crates/optimize/tests/optimize.rsindex 9a0990286..a4315e239 100644--- a/crates/optimize/tests/optimize.rs+++ b/crates/optimize/tests/optimize.rs@@ -674,6 +674,66 @@ fn a_catalog_version_given_by_reference_is_read() {}}+/// Text as a renderer draws it at 300 dpi: lines 48 rows apart of a few letter shapes, each+/// `#` a 3 × 3 block, three columns apart with a wider gap between words, every copy of a letter+/// the same to the pixel.+fn rendered_text(w: u32, h: u32) -> Vec<u8> {+ const LETTERS: [&[&str]; 5] = [+ &["##", "##", "##", "##", "##", "##", "##", "##", "##", "##"],+ &["......", ".####.", "##..##", "##..##", "##..##", "##..##", ".####."],+ &["##....", "##....", "##....", "#####.", "##..##", "##..##", "##..##", "#####."],+ &["##..##", "##..##", "##..##", "##..##", ".#####"],+ &["##", "##", "..", "##", "##", "##", "##", "##"],+ ];+ let stride = (w as usize).div_ceil(8);+ let mut data = vec![0xFF; stride * h as usize];+ let mut seed = 0x2545_F491_4F6C_DD1Du64;+ let mut next = move || {+ seed ^= seed << 13;+ seed ^= seed >> 7;+ seed ^= seed << 17;+ seed+ };+ for baseline in (32..h as usize).step_by(48) {+ let mut x = 8;+ while x + 20 < w as usize {+ let letter = LETTERS[(next() % 5) as usize];+ for (dy, row) in letter.iter().enumerate() {+ for (dx, c) in row.bytes().enumerate() {+ for (py, px) in (0..9).map(|k| (baseline + 1 + 3 * dy + k / 3 - 3 * letter.len(), x + 3 * dx + k % 3)) {+ if c == b'#' {+ data[py * stride + px / 8] &= !(0x80 >> (px % 8));+ }+ }+ }+ }+ x += 3 * letter[0].len() + 3 + if next() % 5 == 0 { 12 } else { 0 };+ }+ }+ data+}++/// Text whose letters repeat exactly, as rendered text does, becomes JBIG2 symbols: each letter+/// stored once and its copies placed, several times smaller than JBIG2's generic coding of the+/// same image, and the page looks the same.+#[test]+fn rendered_text_becomes_jbig2_symbols() {+ let mut doc = Document::new_empty();+ let (w, h) = (600, 480);+ let text = rendered_text(w, h);+ let im = mono_image(&mut doc, w, h, &text, None, &[("BitsPerComponent", 1.into()), ("ColorSpace", Object::name("DeviceGray"))]);+ // One image pixel a device pixel at scale 1, so a wrong pixel shows.+ page(&mut doc, &[("Im", im)], "q 600 0 0 480 6 300 cm /Im Do Q");+ let before = render(&doc);+ let r = optimize(&mut doc, &Settings { mono: MonoCompression::Jbig2, ..Settings::default() }).unwrap();+ assert_eq!(r.images_recompressed, 1);+ let s = stream(&doc, im);+ assert_eq!(s.dict.name(b"Filter"), Some(&b"JBIG2Decode"[..]));+ let generic = pdfcraft_filters::encode_jbig2_generic(&text, w, h).unwrap();+ assert!(s.raw.len() * 2 < generic.len(), "symbols {} bytes, generic {}", s.raw.len(), generic.len());+ assert!(render(&doc) == before, "the page looks the same");+}+#[test]fn settings_choose_what_happens() {let mut doc = Document::new_empty();diff --git a/docs/pdf-spec-parity.md b/docs/pdf-spec-parity.mdindex a1c26eefb..68d26bcce 100644--- a/docs/pdf-spec-parity.md+++ b/docs/pdf-spec-parity.md@@ -1,6 +1,6 @@# PDF specification and standards parity-> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-26 · **Change:** minor (§7.4: lossless JBIG2 encoding in the Optimizer; JBIG2 images export; earlier, §7.4: CCITT Group 4 encoding in the Optimizer) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-26 · **Change:** minor (§7.4: JBIG2 symbol coding in the Optimizer; earlier, lossless JBIG2 encoding, JBIG2 images export and CCITT Group 4 encoding) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)How much of ISO 32000-2 (PDF 2.0) and the PDF subset standards PdfCraft implements, set againstwhat Acrobat Pro implements. This is PdfCraft's equivalent of a "geometry" or "codec" checklist:@@ -21,7 +21,7 @@ are borrowed; PDF 2.0 additions and most subset standards are missing. **150–2|---|---|---|---|---|| §7.2–7.5 | Syntax, objects, file structure, xref tables and streams, object streams, hybrid files, incremental updates | full | **own**, tolerant, repairs damaged files | `crates/cos`; corpus 958 files; fuzzed nightly || §7.4 | Stream filters: Flate, LZW, ASCII85, ASCIIHex, RunLength, predictors | full | **own** | `crates/filters` |-| §7.4 | Image filters: DCT, JPX, JBIG2, CCITTFax | full | **borrowed** (decode); CCITT Group 4 encoding; **own** JBIG2 encoding | `core.image-filters` partial; own codecs M1.1/M1.2; the Optimizer encodes CCITT Group 4 (`pdfcraft_filters::encode_g4`) and lossless JBIG2 (a generic region; no symbol coding), `optimize.mono-compression` |+| §7.4 | Image filters: DCT, JPX, JBIG2, CCITTFax | full | **borrowed** (decode); CCITT Group 4 encoding; **own** JBIG2 encoding | `core.image-filters` partial; own codecs M1.1/M1.2; the Optimizer encodes CCITT Group 4 (`pdfcraft_filters::encode_g4`) and lossless JBIG2 (a generic region, or symbols for exact copies; no refinement, `/JBIG2Globals` or lossy coding), `optimize.mono-compression` || §7.5.8 | Linearization | R W | R only | `core.linearization` planned || §7.6.4 | Standard security handler R2–R6 (RC4 40/128, AES-128, AES-256), crypt filters, `/EncryptMetadata`, SASLprep | full | **own**, read and write | `crates/crypt`; written files open in Acrobat DC since #836 (#774: Acrobat needs the catalog outside object streams) || §7.6.5 | Public-key security handler | full | none | `core.open-public-key`, `protect.certificate-encryption` planned |@@ -130,6 +130,7 @@ are borrowed; PDF 2.0 additions and most subset standards are missing. **150–2| Date | Change | Summary ||---|---|---|+| 2026-10-26 | minor | §7.4: lossless JBIG2 symbol coding (symbol dictionaries and text regions, own, from the JBIG2 draft standard) in the Optimizer, used when smaller than the generic region || 2026-10-26 | minor | §7.4: lossless JBIG2 generic-region encoding (own, from the JBIG2 draft standard) in the Optimizer; Export ▸ Images decodes JBIG2 || 2026-10-26 | minor | §7.4: CCITT Group 4 encoding (`fax`) in the Optimizer || 2026-10-14 | minor | §12.4.4: `/Trans` and `/Dur` read into `DocInfo::transitions` and reported by `doc_info` |diff --git a/docs/roadmap.md b/docs/roadmap.mdindex 4f376cfb7..804af9717 100644--- a/docs/roadmap.md+++ b/docs/roadmap.md@@ -2,6 +2,7 @@> **Last reviewed:** 2026-10-27 | **Last updated:** 2026-10-27 | **Change:** minor (M10: OCR model downloads use verified ONNX pairs with legacy discovery fallback; estimates unchanged)> **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M14 localization current state: Arabic logical catalog, explicit RTL label direction and mirrored stock chrome; native/browser RTL workflows unqualified. Native release Hans face via LXGW WenKai Lite; estimates unchanged)+> **Last reviewed:** 2026-10-26 · **Last updated:** 2026-10-26 · **Change:** minor (M11: lossless JBIG2 stores each distinct mark once, as a symbol; estimates unchanged)> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-24 · **Change:** minor (M11: the PDF Optimizer can store black-and-white images as lossless JBIG2, and Export ▸ Images reads JBIG2 images; estimates unchanged)> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: the PDF Optimizer and Reduce File Size store black-and-white images as CCITT Group 4 when smaller; estimates unchanged)> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged)@@ -257,6 +258,7 @@ Estimates unchanged.| Date | Change | Summary ||---|---|---|+| 2026-10-26 | minor | M11: lossless JBIG2 symbol coding (`pdfcraft_filters::encode_jbig2`): each distinct mark stored once in a symbol dictionary and its exact copies placed by a text region, larger and lone marks in a generic region, used when smaller than the generic coding; on six pages of rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's. Scans gain little until refinement coding; `/JBIG2Globals` and lossy coding not yet. Estimates unchanged. || 2026-10-23 | minor | M11: lossless JBIG2 for black-and-white images (`pdfcraft_filters::encode_jbig2_generic`, a generic region coded with the MQ coder, written from the JBIG2 draft standard), as Monochrome Images ▸ JBIG2 (lossless) and `doc_optimize` `mono: jbig2`; about a third smaller than Group 4 on 300 dpi text. Export ▸ Images reads JBIG2 images (`hayro-jbig2`, with `/JBIG2Globals`). Symbol coding not yet. Estimates unchanged. || 2026-10-23 | minor | M11: black-and-white images (1-bit, and image masks) are stored losslessly as CCITT Group 4 when that is smaller: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size too), `doc_optimize` `mono`; `pdfcraft_filters::encode_g4`. JBIG2 and monochrome downsampling not yet. Estimates unchanged. || 2026-10-27 | minor | M10: OCR stages SHA-256-verified ONNX models after the upstream RTEN URL brownout; discovery prefers a complete ONNX pair per directory, falls back to a complete RTEN pair, and never mixes formats or directories; language/web limits and estimates unchanged. |diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.tomlindex 7bdf89eca..c18856e3a 100644--- a/parity/acrobat-features.toml+++ b/parity/acrobat-features.toml@@ -7384,8 +7384,8 @@ milestone = "M11"status = "partial"commands = ["optimize.advanced", "optimize.reduce"]tools = ["doc_optimize", "doc_reduce"]-evidence = ["crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy"]-notes = "CCITT Group 4 (the default, so Reduce File Size does it too) or JBIG2, both lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. JBIG2 is a generic region (template 0, typical prediction), about a third smaller than Group 4 on 300 dpi text; a document below PDF 1.4 gets /Version 1.4. Not yet: JBIG2 symbol coding (text regions with symbol dictionaries, where JBIG2's larger gains on text come from, and its lossy mode), JBIG2Globals, monochrome downsampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to 1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word, PowerPoint and HTML export leave the picture out."+evidence = ["crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy"]+notes = "CCITT Group 4 (the default, so Reduce File Size does it too) or JBIG2, both lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. JBIG2 stores each distinct mark once in a symbol dictionary and places its exact copies with a text region, larger and lone marks in a generic region (template 0, typical prediction), or the whole image as a generic region, whichever is smaller: on rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's; on scans, whose copies of a letter are seldom identical, the generic region, about a third smaller than Group 4. A document below PDF 1.4 gets /Version 1.4. Not yet: JBIG2 refinement coding of near-copies (where lossless gains on scans would come from), a symbol dictionary shared between images (/JBIG2Globals), JBIG2's lossy mode, monochrome downsampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to 1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word, PowerPoint and HTML export leave the picture out."[[feature]]id = "optimize.only-if-smaller"