A fast, helpful, and open-source document parser
docs: record round 4 of the table/TEDS work
Logan Markewich18be0de7ae7d
diff --git a/TABLE_TEDS_PROGRESS.md b/TABLE_TEDS_PROGRESS.mdindex 291f1f2..e5e79eb 100644--- a/TABLE_TEDS_PROGRESS.md+++ b/TABLE_TEDS_PROGRESS.md@@ -5,22 +5,22 @@ Goal: close the table-fidelity (TEDS) gap against **pdf-inspector** (firecrawl)## Standing-| metric | baseline (2.10.1) | round 1 (`093ddba`) | round 2 (`520413d`) | round 3 | Δ total | pdf-inspector |-|---|---|---|---|---|---|---|-| **overall** | 0.8732 | 0.8757 | 0.8783 | **0.8786** | +0.0054 | 0.8753 |-| TEDS | 0.6929 | 0.7128 | 0.7441 | **0.7471** | **+0.0542** | 0.8141 |-| MHS | 0.8114 | 0.8175 | 0.8179 | **0.8179** | +0.0065 | 0.7879 |-| NID | 0.9127 | 0.9129 | 0.9136 | **0.9136** | +0.0009 | 0.9147 |+| metric | baseline (2.10.1) | r1 (`093ddba`) | r2 (`520413d`) | r3 (`08225ac`) | r4 | Δ total | pdf-inspector |+|---|---|---|---|---|---|---|---|+| **overall** | 0.8732 | 0.8757 | 0.8783 | 0.8786 | **0.8816** | +0.0084 | 0.8753 |+| TEDS | 0.6929 | 0.7128 | 0.7441 | 0.7471 | **0.7817** | **+0.0888** | 0.8141 |+| MHS | 0.8114 | 0.8175 | 0.8179 | 0.8179 | **0.8203** | +0.0089 | 0.7879 |+| NID | 0.9127 | 0.9129 | 0.9136 | 0.9136 | **0.9143** | +0.0016 | 0.9147 |-ParseBench table composite (the mandatory cross-check): 0.4034 → **0.4063** in-round 3, so this round is positive on *both* benchmarks.+ParseBench table composite (the mandatory cross-check): 0.4034 → 0.4063 (r3) →+**0.4074** (r4), so every round so far is positive-or-flat on *both* benchmarks.-We lead on overall + MHS, trail on TEDS by **0.067** (was 0.101) and NID by 0.001.-**TEDS is scored on only 42 of the 200 docs**, so one doc = 0.024 of the mean.+We lead on overall + MHS, trail on TEDS by **0.032** (was 0.101) and NID by+0.0004. **TEDS is scored on only 42 of the 200 docs**, so one doc = 0.024 of the+mean.-Round 1 shipped as `093ddba`, round 2 as `520413d`; round 3 is ~120 lines in-`crates/liteparse/src/markdown_layout/tables.rs` plus one line in `parser.rs`.-303 lib tests pass.+Round 4 is three commits (`87d54ef`, `46e95b4`, `8aad56b`), ~300 lines in+`tables.rs` plus one line in `classify.rs`. 308 lib tests pass.## What shipped@@ -104,6 +104,77 @@ Round 1 shipped as `093ddba`, round 2 as `520413d`; round 3 is ~120 lines inRequires ≥3 words, since two words give one gap with nothing to compare itto. Unit tests cover both the firing case and the justified-prose case.+## What shipped — round 4 (rowspans, the veto share, booktabs bands)++6. **Rowspan-aware density gate** (`87d54ef`; +0.0055 odl TEDS, +0.0011+ ParseBench; doc 146 0.358 → 0.588, nothing down).++ The ruled-grid density gate counted a rowspan continuation cell as a *failed+ text assignment*, so a table with a merged label column ("1. Embodying+ sustainability values" beside three competence rows) died as mostly-empty and+ its text spilled into prose. This is the general form of the "empty-frac is a+ symptom, not a threshold" note below — the empties were real, they just had a+ geometric explanation.++ `rowspan_mask` reads the merge straight off the rules: a cell whose top+ boundary carries **no horizontal stroke over its own centre**, at a boundary+ **some vertical rule runs through**, is merged with the cell above. The mask+ rides on `CellGrid` so it survives `collapse_phantom_rows`/`_cols` (and is+ realigned across `merge_stacked_header`) — an early scalar version forgave+ *pre*-collapse spanned cells against a *post*-collapse empty count and let a+ junk grid through on the scale mismatch alone.++ Both guards are load-bearing, each found by a document it broke:+ - **vertical must continue through the boundary** — otherwise the grid has+ merely *ended* there, and a page-frame component shredded body prose into a+ 2-column table.+ - **the merged head must hold text** (walk up the run of spanned cells; the+ first unspanned cell must be filled) — otherwise a chart's vertical+ gridlines made every cell "spanned" and produced a 27-column table,+ −0.0072 NID over 4 docs.++ Test the column *centre*, never containment of the whole column: `xs` comes+ from clustered + gutter-collapsed boundaries and the outer ones can sit tens+ of points off the drawn border (doc 146: `xs[0]=42.9` vs a border at 79.7).++7. **`already_handled` needs a material share** (`46e95b4`; +0.0053 odl TEDS,+ ParseBench flat; doc 200 0.054 → 0.276, NID +0.097, MHS −0.143).++ The global ruled pass builds doc 200's table whole, then `classify.rs` throws+ it away because *some* xy_cut leaf tables on its own. That leaf was the+ landscape slide's title band — **3 of the run's 58 lines**, "tabling" as a row+ of gutter-split title fragments. Require the vetoing leaf to hold ≥¼ of the+ run's lines. This is the mechanism behind the two dead `flatten_header_band`+ guards recorded below: that component was rejected here, downstream of the+ flatten, which is why both guards measured exactly zero.++8. **Booktabs rule-band 2-column pass** (`8aad56b`; +0.0238 odl TEDS,+ ParseBench flat; doc 165 0.000 → **1.000**, matching pdf-inspector, NID+ +0.046, *one* doc changed).++ Item 4's "bordered worksheet" diagnosis was wrong. Doc 165's table is two+ hairlines with **no vertical rules anywhere on the page**, so `extract_h_v_segments`+ yields 0 VSegs (they're gated on `height > 1.0`) and `find_grid_components`+ never forms a component — `build_ruled_table` is never called, and no density+ gate is ever reached. The real killer is `TABLE_MIN_COLUMNS = 3` on a+ genuinely 2-column table. Round 3's splitter already recovers both tracks+ (`ratio=3.42`, xs=[60,118]) from the merged header run; they were discarded+ one line later.++ `rule_bands` pairs two horizontal rules with ≥60% x-overlap, 10pt < gap <+ ½ page, and **no vertical crossing between them** — a band with verticals is+ a real grid and the ruled detector owns it. Inside a band, in gaps the normal+ pass left empty, retry with `allow_two_col`: column minimum 2, and+ single-piece body rows allowed (a blank second column makes *every* body row+ one piece — `cells_from_raw_items_with_tracks` rejects those by default, and+ that guard must stay default-off).++ Detection runs **against the band's lines alone** (sub-slice, indices shifted+ back), so a run can't reach past the rules that justify the relaxation. Doing+ it in-place instead failed on `row spacing cv 0.53` because the run swallowed+ the paragraph below the band. Further gates: band lines contiguous, ≥3 of+ them, seed line reads as exactly 2 pieces.+## Cross-benchmark check is mandatory for table work**opendataloader-bench alone is not enough.** Round 2 looked perfect on it — four@@ -160,12 +231,23 @@ that report in place**, so copy it aside before running a variant.- **Never rebuild the binary while a bench run is in flight** — both harnessesshell out to `target/release/lit`, so a mid-run `cargo build` silently mixestwo variants into one score.+- **Attribution runs are cheap and mandatory.** Two changes bundled into one+ ParseBench run cost a day of ambiguity in round 4; one run per variant+ (~12 min) is always worth it. `ab.sh` on odl is 1-2 min, so bundle there and+ split on ParseBench.- **pdf-inspector repro**: `git clone --depth 1 https://github.com/firecrawl/pdf-inspector`,`cargo build --release --bin pdf2md` (~28s), run `pdf2md <pdf> --raw`. Its outputs andscores are already committed at `opendataloader-bench/prediction/pdf-inspector/`.Re-evaluate with `uv run src/evaluator.py --engine pdf-inspector`.-## Where the remaining 0.101 sits+## Where the remaining 0.032 sits++Top per-doc gaps after round 4 (`perdoc.py`-style, ours vs pdf-inspector):+200 −0.515, 83 −0.327, 119 −0.276, 197 −0.211, 182 −0.206, 121 −0.173,+81 −0.162, 46 −0.156, 84 −0.154, 190 −0.127, 146 −0.126, 170 −0.122.+Docs 81/83/84 are the same source family (item 8 in the old ranking, below).++## Where the 0.101 sat before round 4Measured by GT-cell-text retention (liteparse **0.747** vs pdf-inspector **0.870**):@@ -188,16 +270,23 @@ So the dominant problem is **misfiling**, not dropping, text.column *is* populated. Worth trying next, but the empty-first-cell rule is what makesthe current version regression-free — relax it only behind an A/B.-3. **Doc 200 (−0.737 alone, ~+0.018).** Columns are now correct after the gutter fix,- but the body rows come out empty. Cause: `flatten_header_band` absorbs the landscape- slide's full-width title rows into the colspan header, so rows 0..3 become a junk- header band. Note the obvious guard (skip rows whose `cells_repl` is uniform across- all columns) **does not fire** — PDFium splits the title into several spans.--4. **Doc 165 worksheet (~+0.024).** Bordered worksheet whose second column is entirely- blank; the ruled grid is rejected (`rows-after-collapse 1`, `no-lines-consumed`).- Related: `TABLE_MAX_EMPTY_CELL_FRACTION` cannot be relaxed (see below), so this needs- a different route — likely the tagged struct tree (item 5).+3. **Doc 200 is still the biggest single item (0.276 vs 0.791, ~+0.012).** The+ table now reaches the output (item 7), so `flatten_header_band` is finally on a+ live path. Two problems remain, and the *first* is the real one:+ - **Column over-segmentation**: `xs` has 6 columns where GT has 4.+ `336.9/358.5` is a padding-pair sliver that `collapse_gutter_columns` keeps+ only because a *title* span centre lands in it, and `347.7` is a spurious+ split inside the `Explanation` column. This also causes the second problem,+ by pushing the real header row below `TABLE_ROW_MIN_FILL`.+ - The flatten's anchor then lands on the first *data* row and eats it along+ with the two title rows above. Relaxing the anchor to accept a half-filled+ row of short, column-distinct labels wins +0.009 TEDS here and **costs+ −0.0043 on ParseBench** — measured, not guessed. Do not ship it without+ fixing the columns first.++4. ~~**Doc 165 worksheet**~~ — **DONE in round 4** (item 8), 0.000 → 1.000. The+ "bordered worksheet / `TABLE_MAX_EMPTY_CELL_FRACTION`" diagnosis in this slot+ was wrong; see item 8 for what it actually was.5. ~~**Tagged `/Table` structure tree**~~ — **DEAD. Do not build this.** Measuredbefore writing any code: **0 of the 200 opendataloader-bench PDFs carry a@@ -210,10 +299,15 @@ So the dominant problem is **misfiling**, not dropping, text.nodes under `pages[].structure_tree.roots`. Still worth having for *realuser* documents someday, but it can never show up in these numbers.-6. **Sub-word tracks are still on the table.** Round 3 uses word boxes only to- split runs into pieces. `split_text_at_x_anchors` still locates a split by- *linear interpolation over character index*, which is why doc 83 emits- `10 .1%`. Feeding real word x's into that function is the obvious next step.+6. ~~**Sub-word tracks**~~ — **built and measured inert.** `split_words_at_x_anchors`+ (real word x's) and `bucket_words_into_columns` (per-word ruled-column+ assignment) now front both anchor-split paths, and the 200-doc output is+ **byte-identical**. The borderless multi-track branch fires only **12 times in+ the whole corpus** and word/char agree every time; the ruled branch fires 434+ times and buckets 219 of them differently, but every one of those components+ is rejected later anyway. Round 3's gutter split already consumes this class+ upstream. Kept (geometry beats interpolation where it does fire), but don't+ expect a number from it — doc 83's `10 .1%` comes from somewhere else.7. **Doc 190's last row (`Merge v4 | SLERP`, −0.126).** The header row is fixed;the final body row still falls out because its second cell (`SLERP`, x=182.9)@@ -258,6 +352,12 @@ So the dominant problem is **misfiling**, not dropping, text.(`Number | of clauses` split across two columns) over a correct borderlesstable. Fixing this class needs either detector-priority plumbing or splittingthe *component* — not a post-hoc row trim. Reverted; not in the shipped diff.+- **Relaxing the `flatten_header_band` anchor** to accept a half-filled row of+ short (≤40 char), column-distinct labels when column over-segmentation pushes+ the real header below `TABLE_ROW_MIN_FILL`. Doc 200 +0.009 TEDS / +0.020 NID+ with no odl collateral — and **−0.0043 on ParseBench**, attributed by a+ one-variable run. Reverted. This is the second time odl-bench alone would have+ shipped a table regression.- **A second full-width-spanner guard in `flatten_header_band`** (reject the flattenwhen a produced header cell exceeds 100 chars, aimed at doc 200): **exactly zerodelta** — this is the second guard to die here. The `[ruled] colspan header flatten`