Skip to content

PdfCraft: An open-source PDF reader and editor written by scratch in pure Rust

M11: say where Group 3 with no code before the first row is written back as read

Grzegorz Martensaaa3d815c18b
diff --git a/crates/filters/Cargo.toml b/crates/filters/Cargo.toml
index 91b5d3d57..fd63ac2b0 100644
--- a/crates/filters/Cargo.toml
+++ b/crates/filters/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "pdfcraft-filters"
-description = "PDF stream filters (ISO 32000-2 §7.4): Flate, LZW, predictors, ASCII85, ASCIIHex, RunLength, CCITT Group 4 encoding"
+description = "PDF stream filters (ISO 32000-2 §7.4): Flate, LZW, predictors, ASCII85, ASCIIHex, RunLength, CCITT Group 4 encoding and Group 3 and 4 decoding"
version.workspace = true
edition.workspace = true
license.workspace = true
diff --git a/crates/filters/README.md b/crates/filters/README.md
index b8a4c5372..b8c1d2281 100644
--- a/crates/filters/README.md
+++ b/crates/filters/README.md
@@ -43,14 +43,14 @@ decode_ccitt(data, &params, max_rows) -> Result<Vec<u8>, FilterError> // CCITT
| ASCIIHexDecode | white space ignored, odd digit padded with 0, `>` EOD (optional), other characters → Corrupt | upper case, newline every 64 digits, `>` |
| ASCII85Decode | `<~` prefix, `z`, white space, `~>` (or bare `~`, or none) EOD, final partial group; `z` inside a group, a 1-char final group, overflow or bad characters → Corrupt | `z` for zero groups, newline every 75 characters, `~>` |
| RunLengthDecode | 0–127 literal, 129–255 repeat, 128 EOD (optional); truncated run → Corrupt | runs of ≥2 repeat, literals up to 128, 128 |
-| CCITTFaxDecode | not in a chain (`Unsupported`); `decode_ccitt` decodes Group 3 and Group 4 on its own, as rows of 1-bit samples each on a new byte, black 0 unless `black_is_1`, at most `max_rows` rows (fewer with `/Rows`), the rows before any damage, up to 1,048,576 samples a row; Group 3 rows on bytes are read by their end-of-line codes when the data starts with one and on bytes otherwise, whatever `end_of_line` says (`pdfcraft_cos::Document::decoded_image` calls it for a fax image, after the filters before it) | `encode_g4`: Group 4 (`/K -1`) with an end-of-block mark, 0 bits coded black so `/BlackIs1 false` decodes the same bits; rows packed MSB first, each on a new byte, padding bits and trailing bytes ignored; up to 32,767 samples a row (the encoder counts positions in 16 bits); no rows or columns, or too wide → Unsupported, too few bytes → Corrupt |
+| CCITTFaxDecode | not in a chain (`Unsupported`); `decode_ccitt` decodes Group 3 and Group 4 on its own, as rows of 1-bit samples each on a new byte, black 0 unless `black_is_1`, at most `max_rows` rows (fewer with `/Rows`), the rows before any damage, up to 1,048,576 samples a row; Group 3 rows on bytes are read by their end-of-line codes when the data starts with one and on bytes otherwise, whatever `end_of_line` says, so data with codes but none before the first row (not as T.4 sends them) is read on bytes, as the renderer draws it, and at `k` 0 may come out as other rows (`pdfcraft_cos::Document::decoded_image` calls it for a fax image, after the filters before it) | `encode_g4`: Group 4 (`/K -1`) with an end-of-block mark, 0 bits coded black so `/BlackIs1 false` decodes the same bits; rows packed MSB first, each on a new byte, padding bits and trailing bytes ignored; up to 32,767 samples a row (the encoder counts positions in 16 bits); no rows or columns, or too wide → Unsupported, too few bytes → Corrupt |
## Tests
- Unit tests per filter with hand-computed vectors (the ISO 32000-2 LZW example, RFC 1951 stored blocks, PNG/TIFF rows).
- LZW is cross-checked against `weezl` (dev-dependency) in both directions for both early-change settings.
- `tests/proptests.rs`: encode→decode round trips for every encodable filter × predictor × colors × bpc × columns, two-filter chains, output-limit checks, and fuzz-style no-panic tests (random bytes, random/nonsense params, bit-flipped valid streams).
-- `ccitt.rs`: `decode_ccitt` gives back the rows `encode_g4` was given (with `/BlackIs1`, `/Rows` and a row limit), refuses bad widths, and bounds its output for any input. Group 4 output decoded by `hayro-ccitt` (an independent decoder) matches the input bits: a text-like page, padding bits, the widest row, and property tests over bitmaps of noise and of runs; any input returns rather than panics.
+- `ccitt.rs`: `decode_ccitt` gives back the rows `encode_g4` was given (with `/BlackIs1`, `/Rows` and a row limit), refuses bad widths, and bounds its output for any input; byte-aligned Group 3 in each layout, over 32 seeded pictures, decodes to its picture, and a byte short to fewer rows. Group 4 output decoded by `hayro-ccitt` (an independent decoder) matches the input bits: a text-like page, padding bits, the widest row, and property tests over bitmaps of noise and of runs; any input returns rather than panics.
- `fuzz/`: a detached cargo-fuzz crate (`cargo +nightly fuzz run decode`, run from `crates/filters/`).
## Status
diff --git a/crates/filters/src/ccitt.rs b/crates/filters/src/ccitt.rs
index 7a2a31077..e87a82fe1 100644
--- a/crates/filters/src/ccitt.rs
+++ b/crates/filters/src/ccitt.rs
@@ -84,7 +84,9 @@ impl Default for CcittParams {
/// Group 3 rows on bytes (`k` from 0, `encoded_byte_align`) are read by their end-of-line codes,
/// whether the fill starts each code on a byte or ends it on one, when `data` starts with one (T.4
/// sends one before the first row when it sends them at all), and on bytes otherwise, whatever
-/// `end_of_line` says. Group 4 is read as its parameters say.
+/// `end_of_line` says. Group 4 is read as its parameters say. Data with codes but none before the
+/// first row is read on bytes, as PdfCraft's renderer draws it; with `k` 0 and the fill ending each
+/// code on a byte, that may be other rows than were coded.
///
/// Errors: `columns` that aren't from 1 to [`MAX_CCITT_COLUMNS`] ([`FilterError::Unsupported`]).
pub fn decode_ccitt(data: &[u8], p: &CcittParams, max_rows: u32) -> Result<Vec<u8>, FilterError> {
diff --git a/crates/optimize/README.md b/crates/optimize/README.md
index 470911127..2c9098177 100644
--- a/crates/optimize/README.md
+++ b/crates/optimize/README.md
@@ -41,7 +41,10 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
450 ppi, CCITT Group 4). Black-and-white images are a 1-bit image with one colour component
(grey, indexed, a separation) or an image mask, stored with no filter, with general-purpose ones
or with CCITT fax after any of those (decoded by `Document::decoded_image_within`, no further
- than its rows can need), and an eight-bit grey image of
+ than its rows can need; byte-aligned Group 3 with end-of-line codes but none before the first
+ row, not as T.4 sends them, is read as PdfCraft draws it, which at `/K 0` may be other rows than
+ were coded, and an image resampled, or stored again as Group 4 from behind another filter,
+ keeps them), and an eight-bit grey image of
only 0 and 255, packed to one bit (not an indexed image, whose samples are palette indices, nor
one with a colour-key `/Mask`) unless the grey settings make it smaller still, its soft mask counted
(resampled with it there, kept as it is here), as they can a dithered scan by downsampling it to
@@ -64,8 +67,8 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
settings. Every change is kept only when it is smaller as stored.
- Apart from the downsampling it is lossless: the samples keep their meaning, so the colour
space, decode array and `/Mask` stay. Group 4 wins on scans, whose ragged edges defeat Flate;
- on clean computer-drawn 1-bit art Flate can be smaller. Not yet: JBIG2, Group 3 to Group 4
- without resampling, inline images, an image's own `/SMask` or `/Mask` image (only images drawn
+ on clean computer-drawn 1-bit art Flate can be smaller. Not yet: JBIG2, a lone Group 3 image to
+ Group 4 without resampling (one behind another filter is stored again as Group 4), inline images, an image's own `/SMask` or `/Mask` image (only images drawn
on a page are visited), and Group 4 for rows wider than 32,767 samples.
- **Discard objects:** thumbnails, alternate images, document tags, print settings.
- **Clean up:** Flate for streams with no filter; for streams with an outer ASCII85 or ASCIIHex
diff --git a/crates/redact/README.md b/crates/redact/README.md
index 6d273c234..74509a5d0 100644
--- a/crates/redact/README.md
+++ b/crates/redact/README.md
@@ -20,7 +20,9 @@ clear_marks(&mut doc, None)?; // remove marks without app
- **images**: fully covered → removed; partly covered → a copy with the covered pixels cleared
(8/16-bit and 1/2/4-bit images, image masks; Flate/LZW/RL/A85/AHx, and CCITT fax as the last
filter, decoded through `Document::decoded_image_within`; `/DecodeParms` given by reference,
- numbers inside included, count). Codecs redaction doesn't decode (DCT, JPX, JBIG2) are removed
+ numbers inside included, count; byte-aligned Group 3 with end-of-line codes but none before the
+ first row, not as T.4 sends them, is cleared as PdfCraft draws it, which at `/K 0` may be other
+ rows than were coded). Codecs redaction doesn't decode (DCT, JPX, JBIG2) are removed
whole (fail-closed); inline images under an area are removed. Images with separate `/SMask`
or `/Mask` data, and images whose data decodes to more than three times their declared raster
and a byte per row, plus 4 KiB (ASCIIHex text broken into lines under Flate is a little over
diff --git a/docs/roadmap.md b/docs/roadmap.md
index 6d12e8756..56b10853a 100644
--- a/docs/roadmap.md
+++ b/docs/roadmap.md
@@ -3,6 +3,7 @@
> **Last reviewed:** 2026-10-27 | **Last updated:** 2026-10-27 | **Change:** minor (M10: OCR model downloads use verified ONNX pairs with legacy discovery fallback; estimates unchanged)
> **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M14 localization current state: Arabic logical catalog, explicit RTL label direction and mirrored stock chrome; native/browser RTL workflows unqualified. Native release Hans face via LXGW WenKai Lite; estimates unchanged)
> **Last reviewed:** 2026-10-24 · **Last updated:** 2026-10-24 · **Change:** minor (M11: black-and-white images above 450 ppi are averaged down to 300 ppi by the optimizer; estimates unchanged)
+> **Last reviewed:** 2026-10-24 · **Last updated:** 2026-10-24 · **Change:** minor (M11: black-and-white pictures stay one bit when PDFs are created and optimized; CCITT images export; estimates unchanged)
> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: the PDF Optimizer and Reduce File Size store black-and-white images as CCITT Group 4 when smaller; estimates unchanged)
> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged)
> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: image optimization accounts for page-local UserUnit; estimates unchanged)
diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml
index d656b51a2..8f2271476 100644
--- a/parity/acrobat-features.toml
+++ b/parity/acrobat-features.toml
@@ -130,8 +130,8 @@ milestone = "M1"
status = "partial"
commands = []
tools = []
-evidence = ["crates/filters/src/ccitt.rs::decoding_gives_the_rows_back", "crates/cos/src/document/tests.rs::fax_images_decode_with_their_parameters"]
-notes = "decoded through the hayro bootstrap for display; own codecs are M1.1/M1.2. CCITT fax decoding is in filters (decode_ccitt, through hayro-ccitt), which Document::decoded_image uses for Export ▸ Images, the optimizer and redaction"
+evidence = ["crates/filters/src/ccitt.rs::decoding_gives_the_rows_back", "crates/filters/src/ccitt.rs::group_3_rows_on_bytes_read_by_the_codes_in_the_data", "crates/cos/src/document/tests.rs::fax_images_decode_with_their_parameters"]
+notes = "decoded through the hayro bootstrap for display; own codecs are M1.1/M1.2. CCITT fax decoding is in filters (decode_ccitt, through hayro-ccitt), which Document::decoded_image uses for Export ▸ Images, the optimizer and redaction. Byte-aligned Group 3 is read by its end-of-line codes when the data starts with one and on bytes otherwise; still limited: data with codes but none before the first row (not as T.4 sends them) is read on bytes, as the renderer draws it, and at /K 0 may come out as other rows, which the optimizer and redaction then write back"
[[feature]]
id = "core.decompression-limits"
@@ -7385,7 +7385,7 @@ status = "partial"
commands = ["optimize.advanced", "optimize.reduce"]
tools = ["doc_optimize", "doc_reduce"]
evidence = ["crates/optimize/tests/optimize.rs::eight_bit_black_and_white_images_become_one_bit", "crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy"]
-notes = "CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. An eight-bit grey image of only black and white is packed to one bit first, then stored the same way (indexed images and colour-key masks aside), unless the grey settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Not yet: JBIG2 (lossless or lossy, which Adobe's help says beats CCITT for monochrome), Group 3 to Group 4 without resampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples."
+notes = "CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. An eight-bit grey image of only black and white is packed to one bit first, then stored the same way (indexed images and colour-key masks aside), unless the grey settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Not yet: JBIG2 (lossless or lossy, which Adobe's help says beats CCITT for monochrome), a lone Group 3 image to Group 4 without resampling (one behind another filter is stored again as Group 4; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded), inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples."
[[feature]]
id = "optimize.only-if-smaller"