RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.
Go agent-harnessagent-native-databaseagentic-aiagentic-retrievalagentic-searchaiai-agentscontext-enginecontext-engineeringcontext-managementharness-engineeringknowledge-compilationragretrieval-augmented-generationretrieval-harnesssearch-harness
chore: remove the DeepDoc Go alignment report (#19311)
xugangqiang1dce16760291
diff --git a/deepdoc_go_alignment_report.md b/deepdoc_go_alignment_report.mddeleted file mode 100644index 409b916744..0000000000--- a/deepdoc_go_alignment_report.md+++ /dev/null@@ -1,1139 +0,0 @@-# DeepDoc Go In-Process Backend vs Python Inference Service--> Status: completed.-> Scope: `internal/deepdoc/native`, `internal/deepdoc/parser/pdf`,-> `internal/deepdoc/parser/pdf/inference/native_analyzer`.--## Contents--- [1. Background](#1-background)-- [2. Python inference service dependency inventory](#2-python-inference-service-dependency-inventory)-- [3. Go-side migration](#3-go-side-migration)-- [4. Caller perspective: are the two semantically consistent?](#4-caller-perspective-are-the-two-semantically-consistent)-- [5. Real OCR batch + removal of the external Python service dependency](#5-real-ocr-batch--removal-of-the-external-python-service-dependency)-- [6. Implementation notes / pitfalls](#6-implementation-notes--pitfalls)-- [7. Open items](#7-open-items)-- [Key file index](#key-file-index)-- [The native verification harness (design record)](#the-native-verification-harness-design-record)-- [Equivalence proof](#equivalence-proof)--## 1. Background--RAGFlow PDF parsing relies on DeepDoc inference (OCR text detection / layout-analysis DLA / table structure TSR / text recognition OCR). The original-deployment depended on a standalone **Python inference service**-(`deepdoc_server`). This project reimplements an **in-process Go backend**:--- Pure Go + ONNX Runtime reimplements the same models and the same- post-processing;-- it is packaged into the single server binary (`bin/ragflow_server`) — no- separate Python service to build or deploy;-- both implementations satisfy the same `DocAnalyzer` interface, so callers are- unaware of which one is active.--## 2. Python inference service dependency inventory--### Runtime inference (used directly by `deepdoc/vision`)--| Package | Purpose |-|---|---|-| `onnxruntime>=1.20.0` | runs the det / layout / tsr / rec ONNX models |-| `opencv-python-headless` (cv2) | image decode, resize, findContours, minAreaRect+boxPoints, fillPoly, approxPolyDP, NMS |-| `numpy` | tensor / array math |-| `pillow` (PIL) | server-side image decode / encode |-| `pyclipper>=1.4.0` | det box unclip (polygon dilation) |-| `shapely` | only `Polygon.area/length` for unclip |-| `six` | py2/3 compat shim |--### Service layer (HTTP)--`litserve>=0.2.17`, `python-multipart`--### Rasterization (outside the inference boundary)--`pdfplumber` (PDF → image @216 DPI)--### Model download--`huggingface_hub` (`snapshot_download`)--## 3. Go-side migration--| Python dependency | Go migration | Status / residual |-|---|---|---|-| `onnxruntime` | `onnxruntime_go` (cgo; `InitORT` resolves ORT via `dlopen(NULL)` from the binary; bounded pool in `session.go`/`session_pool.go`) | same `.onnx` models, sha256-locked |-| `cv2.resize (INTER_LINEAR)` | pure-Go `bilinearResize` | float vs fixed-point → **~3px detection floor** (accepted) |-| `cv2.findContours (RETR_LIST)` | hand-written Moore-neighbour (Suzuki-Abe) `findContours` (`det.go`) | residual = box geometry drift (1:1 box set, mean IoU 0.969) |-| `cv2.minAreaRect + boxPoints` | pure-Go rotated-rect `minAreaRect` (`geometry.go`) | aligned |-| `cv2.fillPoly` | pure-Go scanline `fillPoly` (`box_score_fast`) | bit-level identical to cv2 |-| `cv2.approxPolyDP / arcLength` | not ported | only used in Python `poly` path; quad path does not use it |-| `cv2` NMS | `nms.go` (only DLA-table 0.45 / TSR 0.2; det has no NMS) | aligned |-| `cv2 imdecode` / PIL | Go `image` `Decode` (auto format detect + size/pixel caps) | decode contributes ~0 |-| `pyclipper` (unclip, JT_ROUND) | `clipper_offset.go` (Clipper1 port) | 0px-aligned with pyclipper |-| `shapely Polygon.area/length` | computed directly inside `clipper_offset.go` | no dependency needed |-| `numpy` | Go `[]float32` | — |-| `pdfplumber` (PDF→raster @216 DPI) | production uses Go `pdfium.RenderPage` @216 DPI + `FPDF_LCD_TEXT` | measured DLA ≤0.03px |-| `litserve` / `python-multipart` / `six` (HTTP service) | not migrated | replaced by in-process `NativeAnalyzer`; no external HTTP client path |-| `huggingface_hub` (model download) | not migrated | `ragflow_deps/download_deps.py` fetches the same snapshot, sha256-locked |--**Summary:** every inference dependency is migrated — `onnxruntime`→`onnxruntime_go`,-`cv2`→pure-Go geometry reimplementation (resize / findContours / minAreaRect /-fillPoly / NMS), `pyclipper`→`clipper_offset.go`, `shapely`/`PIL`/`numpy`→native-Go. The only things **not** migrated are three categories: the HTTP service layer-(replaced by in-process), model download (Python script retained), and-`pdfplumber` (replaced by `pdfium`, measured-aligned).--## 4. Caller perspective: are the two semantically consistent?--**Conclusion: semantically aligned; bounded numeric differences that do not-affect upper layers.**--Callers consume four `DocAnalyzer` outputs:--| Output | Semantic consistency | Measured evidence |-|---|---|---|-| **Text content** (OCRRecognize) | consistent | on the 35-PDF baseline only 2/35 differ, and only by confidence 0.008 — the text itself is identical |-| **Layout** (DLA) | consistent | 171/174 (98.3%) match <1px; end-to-end raster-aligned text pages ≤0.03px |-| **Table structure** (TSR) | consistent | 85/85 (100%) match |-| **Text-region set** (OCRDetect) | consistent | 1559 boxes 1:1 match, only **1 orphan box** (Go over-detects) |--Callers see "where is text, what is the text, how is layout/table arranged" —-**semantically consistent**. The `drop_score` contract is also aligned (both blank-text but keep the real score when confidence < 0.5).--"Fully consistent" ≠ byte-identical: coordinates jitter by a few pixels-(mean IoU 0.969, worst 21px), confidence has ~1e-5 float noise, and there is that-1 orphan box. This is not a uniform "every box off by a few pixels" error — it is-geometry drift on a small number of boxes.--### Do the 35-PDF intermediate results stay semantically consistent for upper layers?--**Yes, semantically; with one negligible exception.**--- Final output (chunk / table-HTML / markdown) is driven by **text content +- layout/table structure**, both of which are consistent → upper-layer output is- semantically consistent.-- Coordinate jitter does not affect upper layers: reading order / grouping uses- proximity, and a few pixels do not change grouping; table HTML is driven by TSR- cells (100% consistent).-- **Only exception:** the 1 Go-only orphan box in `10_numbering_patterns.pdf`. If- that box is recognized and merged into a chunk it adds a small text span —- **1/1559 boxes, negligible**, but strictly not 100% identical.--> Honest boundary: the equivalence proof covers the **inference boundary** (same-> image, same model); `PdfParser` downstream (split/assemble) is out of scope.-> Because its inputs (text/structure) are consistent, upper-layer semantic-> consistency is a strong inference, not an exhaustive measurement.--### Why not fully identical (the "why" at the numeric layer)--All residuals come from **detection geometry**, unrelated to `drop_score` or the-ORT version (controlled experiments ruled out ORT version):--1. **Contour tracing algorithm:** Go hand-written Moore-neighbour vs- `cv2.findContours` → different boundary pixels → `minAreaRect` geometry- slightly off;-2. **resize interpolation:** Go float `bilinearResize` vs cv2 fixed-point- `INTER_LINEAR` → ≤1/255 pixel noise → seg map differs by 634px → contour- boundary shifts accordingly;-3. combined, near-threshold (0.5 box_thresh) regions produce a "score-flip"- orphan (that 1 box).--## 5. Real OCR batch + removal of the external Python service dependency--### 5.1 Real batch integration (completed)--- **Low-level batch** (`internal/deepdoc/native`):- - `ocr_rec.go`: `RunOCRRecBatchReal(ctx, modelDir, []*Image)` concatenates each- line's preprocessed blob into a `{N,3,48,imgW}` tensor, runs ONNX **once**,- and CTC-decodes line by line. Numerically identical to per-image `RunOCRRec`.- - `image.go`: `FromImages(imgs []image.Image) ([]*Image, error)` batches- `FromImage`.-- **Production analyzer** (`internal/deepdoc/parser/pdf/inference/native_analyzer/native_analyzer.go`):- - `OCRRecognizeBatch(ctx, []image.Image) ([][]deepdoctype.OCRText, error)`:- `n==0` → nil; `n==1` → falls back to single `OCRRecognize` (no batch-width- dilation); `n>1` → `native.FromImages` + `native.RunOCRRecBatchReal` (true- batch). Applies the `dropScore` blanking contract, consistent with the- single-image path.- - Fixed a compile error where the file had **two duplicate `OCRRecognizeBatch`- methods**; kept only the `FromImages`-based version.-- **Call site** (`internal/deepdoc/parser/pdf/parser_concurrency.go` +- `parser_ocr.go`):- - `parser_concurrency.go`: optional `batchRecognizer` interface- (`OCRRecognizeBatch`); `inferOCRRecognizeBatch` occupies one limiter slot for- the whole batch; `docSupportsBatchOCR` detects batch capability. Analyzers- that do not implement the interface (MockDocAnalyzer / replay analyzer)- fall back to the per-crop path — zero interface breakage.- - `parser_ocr.go`: `ocrDetectAndRecognize` collects each box's layer-2 rotation- candidates, accumulates a 1-D `cropAcc`, and if `docSupportsBatchOCR` issues- a single `inferOCRRecognizeBatch` (one Run), otherwise falls back per-crop;- then selects the highest-confidence candidate per box and emits `TextBox`- concurrently. `buildTextBoxes` wires the OCR re-recognition branch into batch- as well. The replay analyzer never implements `batchRecognizer`, so it always- takes the per-crop fallback and `ocrBoxIdxCtxKey` box-index routing stays- intact.--### 5.2 Removal of PyOracle / external Python service dependency (completed)--`internal/deepdoc/parser/pdf/inference/py_oracle.go` was a **non-test file**-implementing a full HTTP inference client (`PyOracle`) for the external Python-service, violating the "production uses in-process only" architecture. It has-been **removed**, along with the pure Python comparison tests that referenced it:--- Removed: `py_oracle.go`; `inprocess_vs_service_compare_test.go`;- `inprocess_vs_service_iou_test.go`; the `tmpcheck/` directory (3 temporary- debug/compare tests + `EQUIVALENCE_REPORT.md`); `ocr_merge_test.go`;- `native_integration_test.go`'s `TestWireVsLiveServer`.-- Rewritten to use `NativeAnalyzer` (with `cgo` tag, so they become real-inference- regression tests that run only in the ORT-deployed `--test-native` build):- `helpers_test.go` (`mustConnectInferenceClient` → `mustConnectInProcessAnalyzer`)- and the call sites `dla_real_world_test.go`, `dla_tsr_compare_test.go`,- `inference_client_integration_test.go`, `parser_parallel_integration_test.go`,- `parser_pipeline_integration_test.go`, `parser_pipeline_manual_test.go`,- `production_smoke_test.go`, `scan_all_pdfs_test.go`, `batch_smoke_test.go`,- `table_crop_ab_manual_test.go`, `table_rotate_integration_test.go`,- `parser_ocr_rotate_integration_test.go`.-- `DefaultDLALabels` / `DefaultTSRLabels` moved to `cache.go`; 12 tests switched- to the in-process `NativeAnalyzer`.-- Further cleanup: deleted `pdf_parser_resolver_test.go`; removed 6 redundant- `t.Setenv("DEEPDOC_URL","")` calls; removed dead config in `environments.go`- (`GetDeepDocURL`/`GetOSSDeepDocURL` getters, `DeepDocURL`/`OSSDeepDocURL` fields- and env reads, `EnvDeepDocURL`/`EnvOSSDeepDocURL` constants); `resolveDocAnalyzer`- dropped the dead `baseURL` parameter.-- Docs synced: `AGENTS.md`/`CLAUDE.md` (removed "needs external service at- startup" misdescription), this document's **Equivalence proof** section (removed `TestWireVsLiveServer` /- live-oracle stale description), `docker/.env` comment.--### 5.3 Verification--| Check | Command | Result |-|---|---|---|-| `parser/pdf` main package compile+link | `go build -tags cgo` (build.sh cgo env) | ✅ EXIT=0 |-| `native_analyzer` package (no models) | `build.sh --test-native` | ✅ ok |-| native real-batch inference | `TestOCRRecBatchIntegration` (ORT+models) | ✅ ok 11.3s |-| `NativeAnalyzer` end-to-end | `build.sh --test-native` (ORT+models) | ✅ ok 387s |-| call-site batch interface | `TestAnalyzerOCRRecBatchIntegration` (ORT+models) | ✅ ok 7.6s |--## 6. Implementation notes / pitfalls--- **pdfium is present:** the static lib is at- `~/ragflow-native-libs/pdfium-static/lib/libpdfium.a`. `build.sh`'s- `setup_cgo_env` exports the correct cgo flags — do not assume it cannot link.-- **cgo is mandatory:** `NativeAnalyzer` lives in a `//go:build cgo` package whose- import chain pulls in the onnxruntime cgo binding. Any test that uses it must- carry the `cgo` tag and can only run where ORT is deployed (`--test-native`).-- **batch width semantics:** `RunOCRRecBatchReal` resizes all lines to the- batch-max width, so batch text **differs from** per-image `OCRRecognize` (single- image uses fixed recW=320). This is by design — assert against- `RunOCRRecBatchReal`, not the single-image path.-- **`native.InitORT` is process-global** and idempotent; multiple calls must be- safe (follow the `analyzerWithModels` pattern).--## 7. Open items--- `docker/docker-compose.yml` still defines a standalone `deepdoc` service- container (port 9390), and `docker/README.md` still mentions `DEEPDOC_URL`. The- Go code no longer connects to it, but whether to remove the deployment-layer- container is a separate scope.--## Key file index--| File | Role |-|---|---|-| `internal/deepdoc/native/ocr_rec.go` | `RunOCRRecBatchReal` / `RunOCRRec` / `RunOCRRecBatch` |-| `internal/deepdoc/native/image.go` | `FromImage` / `FromImages` |-| `internal/deepdoc/parser/pdf/inference/native_analyzer/native_analyzer.go` | `NativeAnalyzer` (incl. `OCRRecognizeBatch`) |-| `internal/deepdoc/parser/pdf/parser_concurrency.go` | `batchRecognizer` interface / `inferOCRRecognizeBatch` / `docSupportsBatchOCR` |-| `internal/deepdoc/parser/pdf/parser_ocr.go` | `ocrDetectAndRecognize` / `buildTextBoxes` (batch call site) |-| `internal/deepdoc/parser/type/labels.go` | `DefaultDLALabels` / `DefaultTSRLabels` (DLA/TSR wire-contract taxonomies; moved here from the removed `inference` HTTP-service cache package) |-| `internal/parser/parser/pdf_parser_common.go` | `deepDocAnalyzerFromEnv` (ignores `DEEPDOC_URL`) |-| `build.sh` | `setup_cgo_env` (cgo flags), `--test-native` |-| `cmd/ragflow_server.go` | `registerNativeDeepDoc` (registers in-process only) |-------## The native verification harness (design record)--> This section is the authoritative design record (ADR style) for `native`.-> The equivalence proof follows below in **Equivalence proof**; the historical-> session handoff notes (HANDOFF*) were deleted — their still-valid points are-> folded into this document.--### 1. Positioning and boundary--`native` is a **standalone verification harness**, not a production path:--- It reimplements the DeepDoc Python inference pipeline (OCR text detection- `det` / layout analysis `DLA` / table structure `TSR` / text recognition- `OCR-rec`) in **pure Go + ONNX Runtime**, running on CPU.-- It compares against the Python reference implementation (`ref_*.py`, which- imports the repo-root `deepdoc`) to catch regressions in the port.-- **Wired into production as the sole backend**: `internal/deepdoc` no longer- talks to the remote Python service over HTTP — production serves DeepDoc- entirely in-process via `infnative.NativeAnalyzer` (built with `-tags- cgo`). The Python service survives only as a read-only equivalence- oracle in the `inprocess_vs_service_*` tests (via the test-only `PyOracle`- client), never as a production path. The `native` harness and the- production backend share the same ONNX Runtime port, so `native`'s- regression checks directly guard the production inference.--It is a regular package inside the main module-(`ragflow/internal/deepdoc/native`), gated by the `cgo` build tag so the-ONNX Runtime (`onnxruntime_go`) cgo binding stays out of the default (no-cgo)-build — the same isolation used for `office_oxide` / `pdfium` / `pdf_oxide`.--### 2. Why pure Go (P3 decision)--A `gocv` / `nogocv` dual build once existed: the gocv path used cv2 decode +-resize and reached 1:1 parity, while pure Go had a ~3 px floor. The code has-**converged to a single pure-Go path** (`image_gocv.go` / `det_gocv.go` /-`dla_gocv.go` deleted, `gocv.io/x/gocv` removed from `go.mod`, the CI-`go-native-gocv` job removed).--Trade-off: give up cv2 1:1 parity in exchange for **zero OpenCV / CGO-dependency**.--### 3. Where the 3px floor comes from (known hard floor)--The port's maximum coordinate residual vs the Python reference is stable at-**~3 px**, from:--- `bilinearResize` (Go float weights) vs cv2 fixed-point `INTER_LINEAR`- implementation difference;-- the contour minimum-area rect in the `box#8` post-process step- (`minAreaRect`).--The floor is **input-format independent** (measured 3.0 px for both JPG and-PNG; decode contributes ~0 because PNG losslessly wraps JPEG-decoded pixels).-The geometry core is not touched unless a decision is made to emulate cv2's-bit-exact float `minAreaRect` (which would require reintroducing OpenCV).--### 4. How goldens are generated / what the drift gate proves--- **golden**: output of `ref_*.py` (the Python oracle) on fixed fixtures,- frozen to `testdata/<stem>.<task>.golden.json`. Fixtures are now **PNG**- (losslessly transcoded from JPG, pixel-equivalent, verified 47/47 zero- difference), matching the production `EncodePNG` wire.-- **`python-drift` job**: re-runs the Python oracle and compares against the- golden — it alerts only when **the Python logic drifts and the golden is- regenerated and committed**. It does **not** independently prove Python is- correct (Python is the trust anchor / oracle).-- **`go-native` job**: runs the Go port against the same golden — catches- **Go regressions**.--> Conclusion: the drift gate is a "Go vs frozen Python snapshot" comparison.-> It reliably catches Go-side regressions but does **not** prove the Python-> side is correct — this is inherent to the "re-implement-to-verify" pattern.--### 5. Test tiers and safety--- **unit** (no tag): pure geometry/post-process unit tests- (`clipper_offset_test.go` cross-checked against pyclipper at 0 px,- `minAreaRect` cross-check, `image_test.go` decode limits); run by default- via `go test ./native/`.-- **integration** (`//go:build integration`, needs `MODEL_DIR`; ORT is always- resolved from the statically-linked binary via dlopen(NULL), so no `ORT_LIB`- is needed and self-skips only when the binary was not built with static ORT):- full-component Go-vs-golden comparison.-- **decode safety**: `Decode` validates the decoded raster's size/pixel- limits (`maxImageDim=16384`, `maxImagePixels=100MP`) to defend against- decompression bombs. It currently runs only on fixed fixtures with no- production exposure; the limits activate if untrusted input is ever wired- in.--### 6. Comparison tolerance--Coordinate tolerance = **`coordFloor(3.0) + coordTolMargin(0.5)` = 3.5**,-computed from constants rather than a literal (`native_integration_test.go`).-`coordFloor` is the 3 px hard floor of §3; `coordTolMargin` lifts the-tolerance just above the floor so that a regression crossing the floor trips-the gate instead of hiding under it.--Because the tolerance derives from `coordFloor`, adjusting the floor later-**follows automatically** with no manual sync. The gate can only catch-regressions **> 3 px (the floor)**, which is enough for "prevent large-breakage"; finer regressions (below the floor) are inherently-indistinguishable — this is the tool's sensitivity floor, not a defect.--### 7. How to regenerate goldens--Goldens are the `ref_*.py` (Python oracle) output on fixed fixtures, frozen to-`testdata/<stem>.<task>.golden.json`. After a deepdoc Python-logic change,-re-run the oracle and write the output back into the goldens, then commit —-otherwise the `python-drift` job alerts (`check_drift.py` only compares, never-writes).--Prerequisites (some venv with `deepdoc` + `onnxruntime` + `opencv-python`):--```bash-export MODEL_DIR=<deepdoc model dir>-export PYTHONPATH=<ragflow repo root> # so ref_det.py can import deepdoc-```--Each `ref_*.py` prints the wire JSON to stdout; redirect to write it to disk-(format identical to the existing goldens):--```bash-cd internal/deepdoc/native-# single fixture + single task-python ref_det.py testdata/page0.png "$MODEL_DIR" > testdata/page0.det.golden.json-python ref_dla.py testdata/page0.png "$MODEL_DIR" > testdata/page0.dla.golden.json-python ref_tsr.py testdata/table0.png "$MODEL_DIR" > testdata/table0.tsr.golden.json-python ref_ocr_rec.py testdata/line0.png "$MODEL_DIR" > testdata/line0.ocr_rec.golden.json--# all fixtures: enumerate the existing goldens in testdata, auto-match task and image stem-for f in testdata/*.golden.json; do- bn=$(basename "$f" .golden.json) # e.g. page0.dla- task=${bn##*.} # dla / det / tsr / ocr_rec- imgstem=${bn%.*} # page0- case "$task" in- det) python ref_det.py "testdata/$imgstem.png" "$MODEL_DIR" > "$f" ;;- dla) python ref_dla.py "testdata/$imgstem.png" "$MODEL_DIR" > "$f" ;;- tsr) python ref_tsr.py "testdata/$imgstem.png" "$MODEL_DIR" > "$f" ;;- ocr_rec) python ref_ocr_rec.py "testdata/$imgstem.png" "$MODEL_DIR" > "$f" ;;- esac-done-```--After regeneration, run `check_drift.py` to confirm no other drift, then-commit the changed goldens together.-------## Equivalence proof--> Design rationale, positioning, and the pure-Go decision live in **The native-> verification harness (design record)** above. This section is the equivalence-> proof; it builds on that record's documented 3 px coordinate floor.--### Executive summary (for decision makers)--**Bottom line.** The Go in-process DeepDoc backend (`infnative.NativeAnalyzer`,-backed by `native`) is functionally equivalent to the Python `deepdoc_server`-inference service for all four tasks it covers — proven by *measured* golden-comparison, not by code inspection.--**What is actually replaced.** `deepdoc_server` is a thin HTTP wrapper over four-recognizers (`dla_adapter` → `LayoutRecognizer4YOLOv10`, `ocr_adapter` →-`TextDetector` / `TextRecognizer`, `tsr_adapter` → `TableStructureRecognizer`).-The caller depends only on the `DocAnalyzer` interface and its wire format. In-the Go server (`-tags cgo`), `infnative.Register` registers-`NativeAnalyzer` as the `DocAnalyzer`, and `resolveDocAnalyzer` serves it-**in-process only** — the external `DEEPDOC_URL` HTTP client has been removed-from the production path entirely. The deployment is in-process, single binary,-no separate Python service to operate. The Python `deepdoc` service now serves-only as an offline alignment baseline (it generates the reference goldens used-by the Go integration tests); it is no longer a runtime backend, a client, or-an equivalence oracle reached over HTTP.--**Three pillars of the proof.**-1. **Boundary equivalence** — DLA / TSR / OCR-rec / Det outputs match the Python- reference goldens within documented, bounded floors (see *Evidence*).-2. **Wire isomorphism** — Go's `Wire()` JSON is structurally identical to the- `deepdoc_server` output for DLA / TSR / Det / OCR-rec, validated against the- re-serialized golden shape (`TestWireSchemaMatchesGolden`) and the Python- reference scripts (`ref_dla.py` / `ref_tsr.py` / `ref_ocr_rec.py`) that- generate the goldens. The goldens are produced by the same Python recognizers- the service wraps, so the Go backend is compared field-for-field against the- Python baseline it replaces.-3. **Measured, not inferred** — every claim is verified by running the real ONNX- models on committed fixtures; the tests *self-skip* (rather than fake-pass)- when ORT / models are absent.--**Honest boundaries (condensed).**-- Not bit-identical; "equivalent within bounded, accepted floors" is the correct- claim. Known floors: Det IoU orphan **3/5** (benign, OCR-adjudicated), TSR- **≤ 3.5 px** on ordinary/moderate real tables (worst measured 2.70 px; dense- annual-report ≤ 1.21 px), **≤ 10 px** on a 4:1 aspect crop (structure- preserved); dense technical-standard full-page tables (15K606 p40) can break- the strict floor on BOTH coordinate drift and cell count (17/30, documented- exception — see *Known model-floor limits*).-- **Production rasterization is now measured, not assumed.** Go rasterizes PDF- pages with pdfium @216 DPI (LCD text AA enabled, matching pdfplumber); Python- deepdoc with pdfplumber @216 DPI. The end-to-end raster-alignment harness- (Methodology §8) renders the *same* real PDF page with both paths and compares- boxes: DLA **≤ 0.03 px on text pages** (≤ 0.72 px on the one figure page), Det- IoU orphan 0/0–1/2, TSR on real-table pages ≤ 2.70 px. The "same raster bytes- in" premise for layout/text detection is empirically closed (the prior Scope- note about pdfium-vs-poppler is superseded: deepdoc uses pdfplumber, also- @216 DPI). **Caveat (measured):** enabling AA tightened DLA but did not change- Det — and on closer measurement the Det `corner-maxd 8–12 px` in the test log- is the *max per-corner* difference on 1–2 skewed outlier boxes, not a center drift;- per-box center distance is sub-pixel (no render-origin offset). See Scope note.-- Depends on the `-tags cgo` build path **and** ONNX Runtime **1.23.x**- **and** the same `InfiniFlow/deepdoc` model snapshot as the (frozen) Python- side.-- No standalone HTTP service mirrors `deepdoc_server`; only the in-process- library and a CLI exist.-- Requires CI to keep running the native tests (see *How to reproduce*) so the- equivalence does not silently regress as the Go side evolves.--**Prerequisites for the claim to hold.** Python side (including models) frozen;-`MODEL_DIR` pinned to the same snapshot; native tests wired into CI.--**Runtime version (concretely recorded).** The Go side is pinned to ONNX-Runtime **1.23.2** (`internal/common.DeepDocORTVersion = "1.23.2"`). The Python-reference `deepdoc_server` was validated against this proof with `onnxruntime-1.23.2` at golden-generation time; the **currently running** Python service in-this environment resolves `onnxruntime` to **1.28.0** (`deepdoc/server/pyproject.toml`-pins `onnxruntime>=1.20.0`). This cross-version state is **measured, not-assumed**:--- The `onnxruntime_go` binding (forked to `github.com/xugangqiang/onnxruntime_go`)- resolves ONNX Runtime at runtime through the C-API (`native.InitORT`): the- in-process backend is built with `libonnxruntime.a` **statically linked** and- finds `OrtGetApiBase` via dlopen(NULL) from the running process, so no- `libonnxruntime.so` is needed at runtime and there is no dynamic `.so`- deployment. A historical experiment once loaded the Python service's 1.28.0- `.so` into the Go backend to prove ABI compatibility; that path has since been- removed. The binding is ABI-compatible across 1.23.x/1.28.x.-- A controlled experiment (Go backend loaded with **1.28.0**'s `.so`) reproduced- the **identical** detection matching as the default 1.23.2 (588/868 `<1px`- matches on both, per-PDF box counts identical), so **the ORT version is not a- sensitivity** for the detection boundary. The two runtimes are numerically- aligned in behaviour, not just ABI-compatible.-- The guard for future drift stays the golden-comparison integration tests- (`TestWireSchemaMatchesGolden` and the `TestAnalyzer*Golden` suites) and the- per-page parity harness (Methodology §9): if the Python side is ever moved to- another ORT build, regenerate the goldens and re-run those before trusting- the claim.--### Scope--This document proves that the Go `native` inference library produces output-**equivalent** to the Python `deepdoc` inference service (the `deepdoc_server`-HTTP service backed by the `deepdoc/vision` recognizers) for the four tasks it-covers:--- **Det** — DB text detection-- **DLA** — document layout analysis-- **TSR** — table structure recognition-- **OCR** — text-line recognition--Both sides load the **same ONNX models** from `InfiniFlow/deepdoc`-(`det.onnx`, `layout.onnx`, `tsr.onnx`, `rec.onnx`) and the same `ocr.res`-character dictionary, and consume the **same raster image bytes**. The Python-service decodes request bytes with PIL / cv2; the Go side decodes with Go's-`image` package — both are format-agnostic raster decoders, so the inference-boundary is identical (raster in, boxes/text out). PDF rasterization happens-upstream of both, outside this boundary.--**Boundary of this proof (read carefully).** This document proves the two-backends are equivalent **at the inference-service boundary** — given the *same-raster image bytes*, they emit the same boxes / text / wire JSON. It does-**not** prove end-to-end equivalence of the full ingestion pipeline (PDF →-raster → recognizers → `PdfParser` post-processing → chunks / table-HTML /-markdown). Two things live outside this boundary and are out of scope here:-1. **PDF → raster.** The Go pipeline rasterizes via `pdfium` (@216 DPI) and the- Python deepdoc pipeline via `pdfplumber` (`to_image(resolution=72*zoomin=216,- antialias=True)`) — **both at 216 DPI**. This is no longer an open scope cut:- the end-to-end raster-alignment harness (Methodology §8) renders the *same*- real PDF page with both paths and compares the recognizer output in- source-pixel space. Result: DLA ≤ 0.03 px on text pages (≤ 0.72 px on the one- figure-heavy page), Det IoU orphan 0/0–1/2, TSR on real-table pages ≤ 2.70 px- — i.e. the "same raster bytes in" premise for layout/text detection holds- empirically through the actual production render paths.-- **AA status (measured).** The Go pdfium render now sets `FPDF_LCD_TEXT (0x02)`- in `pdfium.go`, upgrading text from pdfium's *default* grayscale anti-aliasing- to LCD sub-pixel AA to match Python pdfplumber's `antialias=True` text. (Note:- pdfium anti-aliases by default — `0x02` only refines text AA; there is no- "pdfium has no AA" regime, and `pdf_oxide` is not in the render path at all —- it only does text/char extraction. The AA flag is a pdfium C-API flag applied- directly to `FPDF_RenderPageBitmap`.) Re-running the raster-alignment harness- after this change produced a split result:- - *DLA tightened sharply on text pages*: worst max Δ dropped from ~0.72 px to- ≤ 0.03 px on annual report/ZoomNeXt/ZH-TW/Three-Kingdoms pages (effectively pixel-identical).- The one technical-standard figure page (15K606 p10) stayed at 0.721 px —- that residual is **not** an AA artifact (it survived the text-AA change), so- it comes from a different source (vector/figure rendering, not text- smoothing).- - *Det: no render-origin offset (measured)*: the test log's `corner-maxd 8.0 / 12.0- px` on annual report p2/p8 is the **max per-corner** coordinate difference, not a- center drift. A nearest-center (greedy + Hungarian) analysis of the dumped- boxes shows per-box center distance is sub-pixel — median **0 px**, mean- **< 0.5 px**, p90 **< 2.2 px**, max **< 5 px** — with IoU orphan unchanged- 0/0–1/2. So there is **no translation / coordinate-origin offset** between- the two render paths; the earlier "8–12 px is a render-origin translation"- guess is **falsified**. The ~8–12 px corner figure is concentrated in 1–2- outlier text boxes whose quadrilaterals are slightly skewed, from the same- contour-boundary geometry that produces the 3/5 IoU orphan floor- (`bilinearResize` vs cv2.resize at text edges + Moore-neighbour vs- cv2.findContours). AA did not change it (identical before/after) because it- is not an antialias artifact. The IoU ≥ 0.5 floor still holds, so it- remains a benign quad-skew, not a detection divergence.-2. **`PdfParser` downstream logic** — caption/figure/table association, table- cell extraction, equation handling, chunk assembly. These consume the- recognizer outputs but are not inference endpoints, so they are not covered- by "inference service equivalence".--### How to reproduce--The proof is a reproducible test harness. From-`internal/deepdoc/native`:--```bash-# ORT is resolved from the statically-linked binary via dlopen(NULL); no-# libonnxruntime.so is needed or supported.-MODEL_DIR=<path/to/InfiniFlow/deepdoc> \-go test -tags integration -run 'TestEquivalenceReport|TestDetMembershipAllFixtures' -v ./...-```--- `TestEquivalenceReport` prints the consolidated summary below to the test- log (visible in CI).-- `TestDetMembershipAllFixtures` guards the full-fixture detection floor.--The same model boundary is exercised through the **`DocAnalyzer` seam the PDF-parser actually consumes** (`infnative.NativeAnalyzer`, package-`internal/deepdoc/parser/pdf/inference/native_analyzer`), so the equivalence is proven-at the integration point rather than only inside the standalone library.--> **Use `build.sh`, not raw `go test`.** The Go native paths need CGO flags and-> the static native libs (`office_oxide` / `pdfium` / `pdf_oxide`) that-> `build.sh` wires automatically. From the repo root:--```bash-# ORT is resolved from the statically-linked binary via dlopen(NULL); no-# libonnxruntime.so is needed or supported.-MODEL_DIR=<path/to/InfiniFlow/deepdoc> \-bash build.sh --test-native \- -run 'TestAnalyzerDLAGolden|TestAnalyzerTSRGolden|TestAnalyzerOCRRecGolden|TestAnalyzerDetGolden' \- ./internal/deepdoc/parser/pdf/inference/native_analyzer/...-```--`bash build.sh --test-native` (no `-run` filter) runs the **entire** native-tier in one shot: the analyzer golden suite above **plus** the `native`-integration suite (`TestEquivalenceReport` / `TestDetMembershipAllFixtures`).-That single command is what CI should gate on.--> **Env-var note.** `MODEL_DIR` (tests) / `DEEPDOC_MODEL_DIR` (server) select-> the model snapshot and are required. ONNX Runtime is statically linked into-> the binary and resolved via dlopen(NULL); `ORT_LIB` / `DEEPDOC_ORT_LIB` are no-> longer read (there is no dynamic `.so` deployment). Both point at the same-> `InfiniFlow/deepdoc` snapshot.--These four tests reuse the comparison helpers in `native/golden.go`-(coordinate / score tolerances, IoU box-membership) and compare the analyzer's-`DLA` / `TSR` / `OCRRecognize` / `OCRDetect` output against the same Python-reference goldens.--Models are fetched once with `ragflow_deps/download_deps.py` (a snapshot of-`InfiniFlow/deepdoc`), which also downloads the ONNX Runtime **static**-archives (`libonnxruntime*.a`) into `~/ragflow-native-libs/onnxruntime/static_lib`.-Those archives are linked into the binary and resolved at runtime via-dlopen(NULL), so no `libonnxruntime.so` is required. A historical controlled-experiment once loaded a dynamic `libonnxruntime.so` (default `DeepDocORTVersion`-**1.23.2**, also validated with **1.28.0** — the version the currently running-Python service resolves) to show identical detection matching across builds-(see *Runtime version* in the summary); that dynamic path has since been removed.-The `onnxruntime_go` v1.23.0 binding is C-API ABI-compatible with both builds,-so the validated parity applies to the static linking used now.--### How to deploy / switch backends--The server chooses its DeepDoc backend **once at startup**; there is no-per-request hot-swap.--**In-process (no Python service to operate).** Build the server with the-native backend and point it at the models:--```bash-bash build.sh --go # server binary is built with -tags cgo-export DEEPDOC_MODEL_DIR=<path/to/InfiniFlow/deepdoc> # default: ragflow_deps/download_deps.py snapshot-```--**Startup guarantee (fail-fast).** `cmd/ragflow_server.go` requires the-in-process backend (ORT + models present, built with `-tags cgo`) and-exits with an error if it cannot serve, so a misconfigured deployment cannot-silently produce empty parses. The CLI binary is built without `cgo`-(no backend at all) and therefore fails fast on startup.--**Switching is safe because the outputs are interchangeable.** Both backends-implement the same `deepdoctype.DocAnalyzer` interface with the same wire JSON,-and the content (text, layout, tables) is the same — see the equivalence above.-The only observable difference when switching is a few-pixel wobble in-detected-box coordinates (typically <5 px, worst ~21 px) and a rare extra box-(1/1559), which do not change downstream chunking or retrieval. Consumers that-assert exact box coordinates, or hash the parsed JSON as a cache key, must not-switch backends blindly.--### How the underlying dependencies align--Equivalence rests on both sides running the **same model artifacts** and the-**equivalent runtime stack**. Dependency by dependency:--| Dependency | Go in-process | Python deepdoc | Alignment |-|---|---|---|---|-| Models (`det.onnx`/`layout.onnx`/`tsr.onnx`/`rec.onnx`/`ocr.res`) | `ragflow_deps/download_deps.py` snapshot of `InfiniFlow/deepdoc` | same frozen snapshot | **Same; sha256-pinned** (`modelSnapshotHashes`, `TestModelSnapshotHash` fails on drift) |-| ONNX Runtime | statically linked `libonnxruntime.a` (dlopen(NULL) via the `onnxruntime_go` fork) — no dynamic `.so` | `onnxruntime` PyPI, currently resolves 1.28.0 | **ABI-compatible + measured version-insensitive** (Runtime version / §9) |-| Char dictionary | `ocr.res` | `ocr.res` | **Same file** |-| Image decode (PNG/JPEG → raster) | Go `image` package | PIL / cv2 | **Format-agnostic; decode contributes ~0 px (measured on PNG/JPEG)** |-| PDF → raster | pdfium @216 DPI, `FPDF_LCD_TEXT` AA | pdfplumber @216 DPI, `antialias=True` | **Measured via §8: DLA ≤ 0.03 px on text pages** |-| Det post-process constants | thresh 0.3 / box_thresh 0.5 / unclip 1.5 / min_size 3·5 / round+clamp scaling / no NMS / no approxPolyDP on the quad path | identical (`DBPostProcess` values) | **Identical** |-| Resize | Go `bilinearResize` (float) | cv2 `INTER_LINEAR` (fixed-point) | **Residual = the ~3 px det floor** (documented, accepted) |-| Contour extraction | hand-rolled Moore-neighbour `findContours` | cv2 `findContours` (RETR_LIST, CHAIN_APPROX_SIMPLE) | **Residual = box-geometry drift** (documented, accepted) |-| Go-native CGO libs (`office_oxide`/`pdfium`/`pdf_oxide`) | static-linked via `build.sh` | n/a (Python uses pdfplumber etc.) | **Go-only build deps; not inference-equivalence deps** |--Bottom line: everything *inference-relevant* (models, char dict, post-process-constants) is identical; everything *implementation* (decoder, resize, contour-tracer, ORT build) is equivalent within the measured, documented floors in-*Evidence* and *Det IoU orphan 3/5*.--### Methodology: how the proof is constructed--The proof is not a hand-written mirror of the Python output — it compares the-Go port against **production deepdoc inference**, captured as frozen "golden"-fixtures. Two independent test tiers run the same comparison math against the-same goldens, at two different integration depths.--#### 1. Reference goldens are real deepdoc output--Each golden JSON is produced by a Python oracle in this directory that calls-`deepdoc.vision` directly, then re-maps the OSS labels to the Go class ids and-serializes into the Go `DocAnalyzer` wire shape. The oracles invoke the exact-same recognizers and post-processing the running server uses:--| Task | Oracle | Production recognizer / config it pins |-|------|--------|----------------------------------------|-| DLA | `ref_dla.py` | `LayoutRecognizer4YOLOv10("layout").forward(thr=0.2)` → OSS labels re-mapped via `DLA_CLASS_MAP` to Go DLA class ids |-| TSR | `ref_tsr.py` | `TableStructureRecognizer()([img], thr=0.2)` incl. `alignTSR` (mean/median when >4 rows/cols) → `TSR_CLASS_MAP` (6 classes) |-| Det | `ref_det.py` | `TextDetector`: `DetResizeForTest → NormalizeImage → det.onnx → DBPostProcess(thresh=0.3, box_thresh=0.5, unclip_ratio=1.5) → filter_tag_det_res` |-| OCR | `ref_ocr_rec.py` | `TextRecognizer` with **batch-wide max `wh_ratio`** resize (matches production batch semantics); wire score pinned to `1.0` to match the Go `DocAnalyzer` OCR-rec wire |--Because the reference is the actual production code path, a Go/Python mismatch-surfaces a genuine divergence, not a divergence between two reimplementations.-The fixtures are frozen and committed under `internal/deepdoc/native/testdata`-(`*.dla.golden.json`, `*.tsr.golden.json`, `*.det.golden.json`,-`*.ocr_rec.golden.json`). They were regenerated from the **current** live-detectors after the `normalizeCHW` RGB/BGR channel-order fix, so the det-baseline (3/5 IoU orphans) reflects the true Go-vs-cv2 gap rather than a stale-or swapped-channel oracle.--#### 2. Comparison math (shared by both tiers)--All matching lives in `internal/deepdoc/native/golden.go` and is-imported by both the `native` integration suite and the `infnative`-analyzer suite, so the two tiers cannot drift apart.--- **Axis-aligned boxes (DLA / TSR / OCR):** `CompareBoxes` / `MatchBoxesRelaxed`- match every golden box to its **nearest same-class** Go box by center- distance, then assert each coordinate is within `CmpTolCoord` and the score- within `CmpTolScore`. The relaxed variant returns the unmatched count instead- of failing, so callers can express structural assertions (e.g. "only a- near-threshold row may be dropped").-- **Label alignment:** goldens and the Go analyzer both key on the **first**- index of a label string, so DLA's duplicate labels (indices 4/7/9 share- source text with neighbours) map identically on both sides. The analyzer test- re-derives each box's class from `labelKey(labels, r.Label)` and rewrites the- golden's integer class to the same key before matching.-- **Quads (Det):** `MatchBothDirections` and `MatchIoUBothDirections` match the- rotated text quads **in both directions** by (a) nearest-center within- `CmpTolCoord` and (b) greedy best-IoU ≥ 0.5. IoU membership isolates true- box divergence — a split (1→2), a merge (2→1), or a hallucination — from mere- coordinate drift: a box shifted 20 px but still overlapping its twin scores- high IoU and is **not** an orphan. Reporting both directions surfaces Go boxes- that have no golden counterpart (extra detections) as well as golden boxes Go- missed.--#### 3. Tolerances and floors--| Constant | Value | Meaning |-|----------|-------|---------|-| `CoordFloor` | `3.0` | Documented hard accuracy floor of the comparison pipeline (bilinear resize + `box#8` postprocess for det; DLA/TSR are tighter). |-| `CoordTolMargin` | `0.5` | Lifts the tolerance just above `CoordFloor` so a regression trips the gate instead of hiding under it. |-| `CmpTolCoord` | `3.5` | Coordinate tolerance (px) for golden comparisons. |-| `CmpTolScore` | `0.05` | Score tolerance for detection boxes. |--Two special floors are documented, not hidden:--- **Det hard floor ≈ 3 px.** The pure-Go geometry stabilizes at ~3 px; the- `3.5` tolerance sits just above it. `TestDetIntegration` asserts every golden- quad is within `3.5` px of a Go quad.-- **Extreme-aspect TSR floor ≈ 10 px.** On a ~4:1 crop the model's 640×640 input- squishes x by ~1.45×, amplifying the residual Go-vs-PIL decode difference to- ~8 px. `TestTSRExtremeAspect` therefore uses a relaxed `10` px tolerance and- asserts only that **structure survives** (table + all columns must match,- row count within ±1, only a near-threshold row may be dropped, no hallucinated- boxes). The analyzer TSR golden (`tsr_table_rotation`, a 1:6.3 tall table) sits- comfortably under the 3.5 px tolerance.--Because the analyzer does not expose a TSR score, `TestAnalyzerTSRGolden` widens-the score tolerance to `1.0` and asserts only class + coordinates.--##### Known model-floor limits (full-page real tables) — measured--The 3.5 px coordinate tolerance is **not** universal across every real table — it-is a property of the committed fixtures, not a guarantee for all inputs. The-end-to-end raster-alignment harness (Methodology §8 / `TestTSRFloorFullPageTables`)-quantifies this on whole-page real tables through BOTH production raster paths:--| Full-page table | Match | Max Δ | Verdict |-|-----------------|-------|-------|---------|-| Xiamen Xiangyu annual report p8 (moderate) | **34/34** | **1.21 px** | inside 3.5px floor |-| Xiamen Xiangyu annual report p12 (dense) | **25/25** | **0.37 px** | inside 3.5px floor |-| 15K606 "Building Smoke Exhaust System Technical Standard" p40 (dense technical-standard) | **17/30** | 3.36 px | **documented exception** — model-level cell-count divergence (Go emits 31 vs 30 golden; 13 cells unmatched), not a rasterization floor |--So the empirical upper bound is:--> TSR equivalence is proven within **≤ 3.5 px** on ordinary/moderate real tables-> (worst measured 2.70 px on Three Kingdoms p1; dense annual-report tables ≤ 1.21 px).-> **Dense technical-standard tables** can break the strict floor — both-> coordinate drift AND cell-count disagreement (17/30 on 15K606 p40). This is a-> model-floor effect (the TSR model itself disagrees on a hard table under-> pdfium-vs-pdfplumber rasterization), not a Go logic bug. It is recorded as a-> known-hard exception in `TestTSRFloorFullPageTables` with a regression guard-> (must not get worse than the 17/30 baseline), and is **not** in the strict-> 3.5px fixture suite.--If a future caller needs pixel-exact TSR on dense technical-standard tables, the-fix is a decode/resize alignment on large inputs plus a harder table in the-corpus, not a logic change.--#### 4. Per-task proof--| Task | What is compared | Proving tests | Result |-|------|------------------|---------------|--------|-| DLA | Layout boxes vs `ref_dla.py` goldens (**11 fixtures**: EN textbook, CN whitepaper, eq-heavy paper, ZH-TW enterprise, baseline, 2 figure-caption pages, 1 equation page, **+ CN annual-report page (Xiamen Xiangyu annual report p2), ZH-TW migration doc p3, EN paper (ZoomNeXt p1)**) | `TestDLAIntegration` / `TestAnalyzerDLAGolden`, `TestEquivalenceReport` | 148/148 boxes, max Δ < 0.13 px |-| TSR | Table-structure boxes vs `ref_tsr.py` goldens (**8 fixtures**: table0, normal 2.65:1, rotated 1:6.3, content, caption, cross-page, text-interleaved — covers projected-row-header; **+ real annual-report table (Xiamen Xiangyu annual report p8)**) | `TestTSRIntegration` / `TestAnalyzerTSRGolden`, `TestTSRExtremeAspect` | 190/190 matched, 2.155 px (≤10 px @4:1 aspect, structure preserved) |-| OCR | Recognized text vs `ref_ocr_rec.py` goldens (EN incl. bold/italic/serif, CJK, mixed, digits; plus batch semantics; **+ 3 real text-line crops: Three Kingdoms p1, ZH-TW migration p2, ZoomNeXt p1**) | `TestOCRRecIntegration` / `TestAnalyzerOCRRecGolden`, `TestOCRRecBatchIntegration` | exact text; batch-wide resize reproduced |-| Det | Text quads vs `ref_det.py` goldens (all fixtures) | `TestDetIntegration` / `TestAnalyzerDetGolden`, `TestDetMembershipAllFixtures`, `TestDetOCRAdjudication` | IoU orphan floor 3/5, adjudicated benign via OCR |--#### 5. Two proof tiers (single source of truth)--1. **Standalone library tier** — `internal/deepdoc/native`, run with- `-tags integration`. Exercises `RunDLA` / `RunTSR` / `RunDet` /- `RunOCRRec` / `RunOCRRecBatch` directly, proving the inference library- itself is equivalent.-2. **`DocAnalyzer` seam tier** — `internal/deepdoc/parser/pdf/inference/native_analyzer`,- run with `-tags "cgo integration"`. Exercises `infnative.NativeAnalyzer`- — the exact `deepdoctype.DocAnalyzer` implementation the PDF parser consumes- in production — proving equivalence at the integration point rather than only- inside the library. These are `TestAnalyzerDLAGolden`, `TestAnalyzerTSRGolden`,- `TestAnalyzerOCRRecGolden`, `TestAnalyzerDetGolden`.--Both tiers call the same helpers in `golden.go`, so a change to the matching-math applies identically to library and seam.--#### 6. Schema and determinism guards--Equivalence is not only value-level; the **wire contract** is guarded too:--- `TestWireSchemaMatchesGolden` asserts Go's `Wire()` JSON structure (top-level- key, nesting depth, leaf types) is identical to the deepdoc server adapter- shape for DLA / TSR / Det / OCR-rec, so a caller parsing Go output sees the- same schema as the Python service.-- `TestDLASessionReuse` / `TestTSRSessionReuse` / `TestOCRRecSessionReuse`- assert byte-identical `Wire()` across pooled-session runs (no stale tensor,- no cross-call contamination).-- `TestDetSessionPoolBounded` asserts the session pool set stays bounded under- many distinct page sizes (regression guard for a prior unbounded `sync.Map`- leak).--These guards mean the Go port is a drop-in wire-compatible replacement, not just-numerically close on a fixed fixture set.--#### 7. Live-service field diff (proves "Go == running service", not just "Go == golden")--`TestWireSchemaMatchesGolden` (section 6) compares Go's `Wire()` against the-**re-serialized golden shape** — i.e. against a Python-shaped artifact. Because-the goldens are generated by the same Python `deepdoc.vision` recognizers the-service wraps (`ref_dla.py` / `ref_tsr.py` / `ref_ocr_rec.py`), the comparison-is field-for-field against the Python baseline the Go backend replaces. (The-former `TestWireVsLiveServer` live-HTTP diff was removed when the external-service path was dropped; the equivalence it established is preserved here as a-recorded result.)--**Why this matters.** It answers the reviewer question "you proved Go == a-frozen Python snapshot, not Go == the running service" directly: the goldens-are produced by the actual service's recognizers. It also re-confirmed the-server is a thin wrapper —-its adapters only decode bytes, convert color space, run the same-`deepdoc.vision` recognizers with the same config the goldens were generated-against (DLA/TSR `thr=0.2`, OCR default pipeline), clamp bboxes, and map-label→class_id; there is no extra resize / DPI / rotation / server-side-rasterization.--**Measured (Go ORT 1.23.2 vs the reference `deepdoc_server`; the same field-diff re-validated against the currently running 1.28.0 service by the parity-harness in §9):**--| Task | Fixture | Server boxes | Go boxes | Match / Δ |-|------|---------|--------------|----------|-----------|-| DLA | page0 | 4 | 4 | matched 4/4, max Δ 0.006 px |-| DLA | mp_textbook_en_p0 | 13 | 13 | matched 13/13, max Δ 0.023 px |-| DLA | dla_2510_eq | 24 | 24 | matched 24/24, max Δ 0.007 px |-| TSR | table0 | 11 | 11 | matched 11/11, max Δ 0.509 px |-| TSR | tsr_table_rotation | 15 | 15 | matched 15/15, max Δ 0.886 px |-| Det | page0 | 15 | 15 | IoU orphan 0/0 |-| Det | mp_en_dense_p0 | 93 | 95 | IoU orphan 2/4 (within 3/5 floor) |-| OCR-rec | line0, line_cn | — | — | exact text match |--Conclusion: the Go backend and the live `deepdoc_server` are field-for-field-equivalent on these fixtures; the only Det divergence is the same documented-3/5 contour floor already covered above.--#### 8. End-to-end raster alignment (closes the "same-bytes-in" gap)--Sections 1–7 prove "**given the same raster image bytes**, Go == Python". But in-production neither side receives a pre-rendered PNG: the Go server rasterizes PDF-pages with **pdfium** (`pdfium.RenderPage` @ **216 DPI**) and the Python deepdoc-pipeline rasterizes with **pdfplumber** (`page.to_image(resolution=72*zoomin,-antialias=True)`, `zoomin=3` ⇒ **216 DPI**). §Scope noted this as the one gap-outside the proved boundary. This section closes it by rasterizing the **same-real PDF page with both paths** and comparing the resulting boxes in source-pixel-coordinates — so the "same-bytes-in" assumption becomes a *measured* fact, not a-declaration.--- **Go side:** `pdfium.RenderPage(pdf, page, 216)` → `NativeAnalyzer.DLA` /- `OCRDetect` / `TSR` (the exact `DocAnalyzer` seam production consumes).-- **Python side:** `ref_raster.py` renders the same page at 216 DPI via deepdoc's- OWN pdfplumber path (so it matches what the live `deepdoc_server` would rasterize),- then runs the real `deepdoc.vision` recognizers.-- Both render at **216 DPI**, so box coordinates land in the **same pixel space**- and are compared directly. Harness: `TestRasterAlignmentDLA` /- `TestRasterAlignmentDet` / `TestRasterAlignmentTSR` (analyzer suite,- `//go:build cgo && integration`).--**Measured (6 real documents, 216 DPI both sides, Go pdfium render with-`FPDF_LCD_TEXT (0x02)` set — LCD sub-pixel text AA on top of pdfium's default-grayscale AA, to match pdfplumber's `antialias=True`):**--| Task | Pages | Result |-|------|-------|--------|-| DLA | annual report p2 | **15/15**, worst max Δ **0.017 px** |-| DLA | annual report p8 | **14/14**, worst max Δ **0.025 px** |-| DLA | ZoomNeXt p1 | **15/15**, worst max Δ **0.011 px** |-| DLA | ZH-TW migration p3 | **24/24**, worst max Δ **0.031 px** |-| DLA | Three Kingdoms p1 | **18/18**, worst max Δ **0.006 px** |-| DLA | 15K606 p10 (figure/table) | **18/18**, worst max Δ **0.721 px** (AA-invariant — see note) |-| Det | annual report p2 | matched **93/93**, center-max **3.20 px**, corner-maxd **8.0 px**; IoU orphan 0/0 |-| Det | annual report p8 | matched **42/42**, center-max **3.35 px**, corner-maxd **12.0 px**; IoU orphan 0/0 |-| Det | ZoomNeXt p1 | matched **140/140**, center-max **3.35 px**, corner-maxd **3.0 px**; IoU orphan 0/0 |-| Det | ZH-TW migration p3 | matched **30/30**, center-max **0.50 px**, corner-maxd **1.0 px**; IoU orphan 0/0 |-| Det | Three Kingdoms p1 | matched **32/32**, center-max **1.50 px**, corner-maxd **3.0 px**; IoU orphan 0/0 |-| Det | 15K606 p10 | matched **55/55**, center-max **3.20 px**, corner-maxd **3.0 px**; IoU orphan 1/2 |-| TSR | annual report p2/p8, ZH-TW migration p3, Three Kingdoms p1 | **matched 100%** (117/117 cells), worst max Δ **2.700 px** (≤ 3.5px floor) |--**Interpretation.**-- **DLA is fully closed end-to-end, and now near-pixel-exact on text pages**:- pdfium-vs-pdfplumber rasterization with AA enabled leaves ≤ 0.03 px of- coordinate drift on the four text/table pages (far inside the 3.5px floor), so- the "same-bytes-in" assumption for layout detection is effectively exact there.- The single technical-standard figure page (15K606 p10) stays at 0.721 px even- *with* AA on — confirming that residual is a vector/figure-render difference,- not a text-smoothing artifact.-- **Det is closed end-to-end on structure** (IoU orphan 0/0–1/2, no loss /- no hallucination). The per-box *center* distance is **sub-pixel**: a- greedy/Hungarian nearest-center analysis of the dumped boxes on annual report p2/p8- gives median **0 px**, mean **< 0.5 px**, p90 **< 2.2 px**, max **< 5 px** —- i.e. there is **no translation / coordinate-origin offset** between the two- render paths (the earlier "8–12 px is a render-origin translation" guess,- inferred from the test log's `corner-maxd`, is falsified). What the test log- reports as `corner-maxd 8.0 / 12.0 px` is the *max per-corner* coordinate difference- (`MatchBothDirections` measures the worst of the 4 corners, not the center,- golden.go:191), and it is concentrated in **1–2 outlier text boxes per page**- whose quadrilaterals are slightly rotated/skewed differently. The source is the- same documented contour-boundary geometry already behind the 3/5 IoU orphan- floor: Go's `bilinearResize` vs cv2.resize interpolation at high-contrast text- edges, plus the Moore-neighbour vs cv2.findContours contour trace. AA does not- change it (identical before/after) because it is not an antialias artifact.- Every box still overlaps its twin at IoU ≥ 0.5, so it is a benign quad-skew- inside the documented Det floor, not a detection divergence.-- **TSR is closed end-to-end on ordinary/moderate real tables** (≤ 2.7 px). For- *dense full-page technical-standard tables* the coordinate floor and the model- itself can diverge further — see *Known model-floor limits* below for the- quantified bound and the one documented exception.--The harness requires `uv run python3` with `deepdoc` + `pdfplumber` available;-if the Python oracle is absent the alignment tests **skip** (not fail), so CI-without the Python oracle still passes the rest of the native suite.--#### 9. Per-page IoU diff harness (full-corpus scale)--§1–8 measure equivalence on committed PNG/PDF fixtures. The per-page IoU harness-(formerly `inprocess_vs_service_iou_test.go`, `TestInProcessVsServiceIoUDiff`)-extended the measurement to **every page of every real PDF in the corpus** by-comparing the in-process `NativeAnalyzer` against the Python `deepdoc` service-output. That live-comparison test was removed with the external service path;-the recorded 35-PDF result below is retained as historical evidence.--- renders each page with the production Go path (pdfium @216 DPI),-- runs `OCRDetect` on that exact image through the in-process- `NativeAnalyzer`, and compares the box set against the Python reference- (`deepdoc/vision` recognizer) output,-- greedily matches the two box sets by **rotated-quad polygon IoU**- (Sutherland–Hodgman intersection, matched at IoU ≥ 0.5), so the comparison no- longer depends on either backend emitting boxes in the same order or count,-- classifies every unmatched box as `split_merge` / `threshold_fragment` /- `partial_overlap` / `one_sided`, flags matched boxes that fail a tight-overlap- test (IoU < 0.9 or per-corner distance > 3 px) as `coord_drift`, and writes a- per-page overlay PNG (Go=red, Py=blue, drifted=magenta) plus a JSON report- under `testdata/output/render_compare/iou/`.--**Measured (35 real PDFs, every page; Go ORT pinned to 1.28.0 to match the-Python-side runtime at measurement time):**--| Metric | Value |-|---|---|-| Matched boxes (IoU ≥ 0.5) | **1,559** |-| Orphans (`one_sided`) | **1** (Go-only box in `10_numbering_patterns.pdf`) |-| `split_merge` / `threshold_fragment` / `partial_overlap` | **0** |-| Mean IoU over matched boxes | **0.969** |-| Worst IoU | 0.668 |-| Max per-corner distance | **21 px** (`16_dense_cjk.pdf`) |-| `coord_drift` (IoU < 0.9 or corner > 3 px) | **227 / 1,559 (14.6%)** |-| PDFs with zero drift | **12 / 35** (of which 7 are bit-exact, max corner 0 px) |-| Drift concentration | ~60% in `13_crosspage_table.pdf` |--**Interpretation.** The two detectors agree on box membership for **>99.9%** of-all text regions across the full corpus — the fixture-level Det 3/5 orphan-floor (0.27%/0.44%) holds at larger scale (1/1559 ≈ 0.06%). There are **no**-split/merge/threshold-fragment differences; the entire divergence is-`coord_drift` on matched boxes, i.e. the same contour-boundary/resize geometry-quantified in §8 and *Why the divergence is bounded*. This harness doubles as-the cross-ORT guard: it passes with Go loading either the 1.23.2 or the 1.28.0-runtime against the same models, corroborating *Runtime version*.--### Evidence (measured)--Latest `TestEquivalenceReport` (both tiers green; summary printed to the test log):--| Task | Fixtures | Match | Max Δ | Status |-|------|----------|-------|-------|--------|-| DLA | 11 | 148/148 boxes (incl. equation + figure-caption classes, +3 real-document pages: CN annual report / ZH-TW migration / EN paper) | < 0.13 px | OK |-| TSR | 8 | 190/190 boxes (incl. projected-row-header class, +1 real annual-report table) | 2.155 px (≤ 10 px on a 4:1 aspect crop, structure preserved) | OK |-| OCR | 11 | exact text (EN / CJK / mixed / digits, + font variants, +3 real text-line crops: CN / ZH-TW / EN) | — | OK |-| Det | all | IoU box-membership orphan **3/5** (gold 1125 / go 1127) | — | OK (accepted floor) |--The detection orphan boxes were adjudicated with OCR: cropping each orphan and-running OCR shows the regions still resolve to real text. Concretely, the-orphans split as **Python-only real text 0/3** (those 3 regions are empty-furniture on both sides) and **Go-only real text 3/5** (Go emits 3 text regions-the Python side does not). So the two outputs are **not byte-identical**: Go is-a *superset* on those few regions. This is a small, one-directional divergence-(Go finds extra text), not a loss — the downstream text consumer (dedup /-align) absorbs it, and no content is dropped. "Benign" here means "no content-loss", not "identical output".--#### Det IoU orphan 3/5 — what it means, and how small it is--**What "IoU orphan" means.** For Det we compare two *sets* of text boxes — the-Python golden set and the Go set — over each fixture. "IoU" (Intersection-over--Union) here is computed on the **axis-aligned bounding box (AABB)** of each-quadrilateral (`iou()` in `golden.go:249`): `IoU = area(intersection) /-area(union)`. Two boxes are considered a match only if their *best* greedy IoU-is `≥ 0.5` (`MatchIoUBothDirections`, `golden.go:269`), matched independently in-both directions. An **orphan** is a box on one side whose best counterpart on-the other side has IoU `< 0.5` — i.e. no overlapping twin at all. So "3/5"-means: **3 Python golden boxes** have no Go twin at IoU ≥ 0.5 (Go "missed"-them), and **5 Go boxes** have no Python twin at IoU ≥ 0.5 (Go "extra"). This-isolates genuine box-membership divergence (a box split, merged, or-hallucinated) from mere coordinate drift — a box shifted 20 px but still-overlapping its twin scores high IoU and is **not** an orphan.--**Sample volume.** The 3/5 is accumulated across **every** committed Det-fixture, not a cherry-picked subset. The corpus is **35 fixtures**, totalling-**1,125 golden text boxes** vs **1,127 Go text boxes** (logged by-`TestDetMembershipAllFixtures` as `TOTAL gold=1125 go=1127`). This spans blank-pages, degradation variants (noise / low-contrast / skew / tiny-text / CJK-vertical), and mixed-language dense pages.--**Proportion (how dominant the agreement is).**--| Side | Boxes | Matched (IoU ≥ 0.5) | Orphan | Orphan rate |-|------|-------|---------------------|--------|-------------|-| Python golden | 1,125 | 1,122 | 3 | **0.27%** |-| Go | 1,127 | 1,122 | 5 | **0.44%** |-| Both combined | 2,252 | 2,244 | 8 | **0.36%** |--In other words, the two detectors agree on box membership for **>99.6%** of all-text regions; the residual is a handful of boxes whose contour geometry differs-just enough to drop below the 0.5 IoU hinge.--**Why it is accepted, not a defect.** It is a *regression guard*, not a-zero-target: the test fails only if Go gets *worse* than the baseline (gold-orphan `> 6`, go orphan `> 8`, with slack 3 — see `native_integration_test.go:559`).-The 3/5 is the residue left after every real bug was fixed — stale goldens-(37/20, 42/13) and an R/B channel-swap in `normalizeCHW` (23/9) — leaving only-contour-tracer geometry (explained below). And the orphan boxes are-OCR-adjudicated benign (above): cropping each orphan and running OCR still-resolves real text, so the divergence never drops content.--### Why the divergence is bounded and deterministic--- **Det 3/5** — contour-tracer geometry. Go's pure-Go boundary follower- (Moore-neighbour, Suzuki-Abe style, `RETR_LIST`) selects boundary pixels- slightly differently from cv2's `findContours` at 8-connected diagonal / hole- junctions. 100% of the orphans are SCORE-FLIPs: a different convex hull →- different `minAreaRect` → `box_score_fast` crosses 0.5 at a handful of- regions on dense pages. It does not flip any segmentation.-- **≤ 1/255 residual** — ONNX Runtime's fixed-point `uint8` resize introduces a- pixel-level noise ceiling (max |Δ| = 1 gray level vs Go's float bilinear). It- is irreducible for any pure-float implementation and is far below the 0.5 /- 0.3 score thresholds, so it never changes a detection or score.--Both effects are **deterministic and reproducible**, not random accuracy loss.--### What is NOT proven here (honest boundaries)--1. **Not bit-identical.** "Equivalent within bounded, accepted floors" is the- correct claim; "perfect / pixel-identical" is not. The Det 3/5 floor is a- known, accepted divergence.-2. **Production caller is wired (server only).** The server binary built with- `-tags cgo` registers the in-process backend via- `infnative.Register(...)` in `cmd/ragflow_server.go`; it fails fast at- startup unless ORT + models are available. The in-process backend is the only- DeepDoc backend — there is no external service path. The CLI binary is built- without `cgo` (no-op path).-3. **Coverage confirmation required.** Go implements {det, dla, tsr, ocr}.- Confirm the Python path being replaced uses only these recognizers (e.g. a- separate table-cell recognizer would not be covered).-4. **Runtime version.** Validated with onnxruntime 1.23.2 (see- `internal/common.DeepDocORTVersion`) **and** 1.28.0 (the version the running- Python service resolves); a controlled experiment showed identical detection- matching across both, so the ORT version is **not a sensitivity**. Re-run the- golden-comparison tests (§6 / §7) / parity harness (§9) only if the Python- side moves to a materially different ORT line.-5. **HTTP server shape.** If the goal is a standalone HTTP service mirroring- `deepdoc_server`, that surface is not built — only the in-process library and- a CLI (`main.go`) exist.--### Known model-floor limits (documented, not hidden)--The full-page real-table TSR floor is **measured end-to-end** (both production-raster paths) in Methodology §8 / `TestTSRFloorFullPageTables` and recorded in-the *Known model-floor limits (full-page real tables) — measured* subsection-under §3. That is the authoritative, quantified statement: ordinary/moderate-real tables ≤ 3.5px (worst 2.70px; dense annual-report ≤ 1.21px), and dense-technical-standard tables (15K606 p40) breaking the floor on both coordinate-drift **and** cell count (17/30, documented exception).--The two excluded pages (`15K606` p40, `Xiamen Xiangyu annual report` p12) are retained in the-generator (`/tmp/gen_corpus.py`) for future investigation but are **not** in the-strict 3.5px fixture suite — p12 actually measures 0.37px (25/25) once rasterized-through the real production path, so only p40 remains a genuine hard case.--### Reviewer follow-ups (prioritized, with status)--These are the items raised by an independent review of this proof, ranked by-impact. "Closed" items are done in code or in this document.--| ID | Item | Status |-|----|------|--------|-| P1-1 | **Live-service field diff** — confirm server is a thin wrapper (no extra preprocessing) and diff Go `Wire()` against the *real* `deepdoc_server` HTTP response. | **CLOSED** — server verified thin (adapters only decode + color-convert + clamp + label→class_id; config matches goldens: DLA/TSR `thr=0.2`, OCR default pipeline). The equivalence was established and recorded in Methodology §7 against the reference server (Go ORT 1.23.2 vs reference server 1.23.2; re-validated cross-version against the 1.28.0 runtime by §9). The `TestWireVsLiveServer` live-HTTP diff was later removed when the external service path was dropped from the codebase, but the verified-thin conclusion stands. |-| P1-5 | **ORT version** — record the Python-side ORT build, not just assume ABI compatibility. | **CLOSED** — Go runs **1.23.2** (`DeepDocORTVersion`); the currently running Python service resolves **1.28.0**. A controlled experiment (Go loading 1.28.0's `.so`) reproduced identical detection matching, so the version is not a sensitivity. Recorded in *Runtime version* and Methodology §9. |-| P2-2 | **Scope wording** — state that inference-boundary equivalence ≠ end-to-end PDF→chunk pipeline equivalence. | **CLOSED** — explicit "Boundary of this proof" paragraph added to Scope; PDF→raster and `PdfParser` downstream named as out-of-scope. |-| P2-3 | **Det 3/5 "benign" wording** — it is not identical output (Go emits 3 extra text regions). | **CLOSED** — reworded to "no content loss, not identical output"; Go is a *superset* on those regions, absorbed by downstream dedup. |-| P2-8 | **Corpus** — DLA/TSR/OCR coverage is thin (8 / 7 / 8; OCR are line crops, not full pages). | **CLOSED** — expanded with diverse full-page real-document fixtures: DLA +3 (`dla_real_cn_report` Xiamen Xiangyu annual report p2, `dla_real_zhtw` ZH-TW migration doc p3, `dla_real_en_paper` ZoomNeXt paper p1), TSR +1 (`tsr_real_report` Xiamen Xiangyu annual report p8), OCR +3 (`line_real_cn` Three Kingdoms p1, `line_real_zhtw` ZH-TW migration p2, `line_real_en` ZoomNeXt p1). All pass sub-pixel / exact-text. The one genuine hard case (15K606 p40 dense technical-standard table) is a documented exception, not in the strict 3.5px suite — see *Known model-floor limits*. |-| P0 | **Model snapshot hash lock** — `MODEL_DIR` must be pinned to the same `InfiniFlow/deepdoc` snapshot as the frozen Python side; the proof must fail if it drifts. | **CLOSED** — enforced. `modelSnapshotHashes` (sha256 of `det.onnx`, `layout.onnx`, `tsr.onnx`, `rec.onnx`, `ocr.res`) is checked by `TestModelSnapshotHash` and at the top of `TestEquivalenceReport`; Fatal on any mismatch. Both repo copies verified byte-identical. Update the table only when the snapshot is intentionally upgraded, and regenerate every golden in the same change. |-| P3 | **Concurrency correctness** — parallel vs serial inference must give identical results (thread-safety is correctness, not performance). | **CLOSED** — `TestInferenceConcurrencyConsistent` drives DLA/TSR/OCR-rec/Det once serially (baseline wire) then 8× concurrently and asserts every concurrent run is byte-identical to the serial baseline. Proves the shared model-session pool is race-free and contamination-free under parallel load (complementing `TestDetSessionPoolBounded` which guards pool *size*). |-| **E2E-1** | **End-to-end raster alignment** — the "same raster bytes in" premise is a declaration: production rasterizes via different engines (Go pdfium @216 DPI vs Python pdfplumber @216 DPI). Prove the two render paths yield equivalent boxes. | **CLOSED** — `TestRasterAlignmentDLA/Det/TSR` rasterize the *same* real PDF page with both paths (pdfium vs deepdoc's own pdfplumber) at 216 DPI and compare boxes in source-pixel space. After enabling LCD text AA in the Go pdfium render to match pdfplumber: DLA **104/104** matched with worst max Δ **0.721 px** on the figure page and **≤ 0.03 px** on the four text pages (near-pixel-exact); Det IoU orphan 0/0–1/2 (inside 3/5); TSR on real-table pages **117/117** matched (worst 2.700px). The "same-bytes-in" premise is now **measured**, not assumed. Measured numbers in Methodology §8. Note (measured): the Det test-log `corner-maxd 8–12 px` is the max *per-corner* difference on 1–2 skewed outlier boxes; per-box **center** distance is sub-pixel (median 0, mean <0.5px, p90 <2.2px, max <5px) — there is **no render-origin translation**; the residual is contour-boundary quad-skew, the same source as the 3/5 IoU orphan floor. |-| **E2E-2** | **Quantify TSR floor on full-page real tables** — give an empirical upper bound ("N pages, worst X px") instead of two hand-picked excluded examples. | **CLOSED** — `TestTSRFloorFullPageTables` runs TSR on whole-page real tables through both raster paths: moderate tables ≤ 3.5px (Xiamen Xiangyu annual report p8 1.21px, p12 0.37px; Three Kingdoms p1 2.70px); dense technical-standard 15K606 p40 is the documented exception (17/30, model-level cell-count divergence, regression-guarded). Empirical bound recorded in *Known model-floor limits (full-page real tables) — measured*. |--**Withdrawn critique.** One review point claimed the two proof tiers "use-different inference code paths / decoders, so preprocessing may differ." This is-**factually incorrect**: `infnative.NativeAnalyzer.DLA/TSR/OCRDetect/OCRRecognize`-all delegate to the *same* `native.RunDLA` / `RunDet` / `RunTSR` / `RunOCRRec`-(`native_analyzer.go:101-181`); only the initial PNG→pixel decode differs, which-is the already-documented `≤ 1/255` residual. The standalone and seam tiers-therefore share one inference path.diff --git a/internal/deepdoc/parser/pdf/inference/native_analyzer/native_analyzer_test.go b/internal/deepdoc/parser/pdf/inference/native_analyzer/native_analyzer_test.goindex 42653a2a9f..61c5c21d85 100644--- a/internal/deepdoc/parser/pdf/inference/native_analyzer/native_analyzer_test.go+++ b/internal/deepdoc/parser/pdf/inference/native_analyzer/native_analyzer_test.go@@ -213,10 +213,9 @@ func cropBox(src image.Image, b deepdoctype.OCRBox) image.Image {// The tests below prove the NativeAnalyzer (the DocAnalyzer the PDF parser// actually consumes) produces output equivalent to the Python deepdoc// reference, reusing the SAME Python-reference goldens as the native-// integration suite. This closes the gap noted in deepdoc_go_alignment_report.md-// (Equivalence proof section): the in--// process backend previously only had a smoke test (non-empty, in-bounds);-// these tests assert value-level parity through the analyzer's public API.+// integration suite. Before these tests the in-process backend only had a smoke+// test (non-empty, in-bounds); these tests assert value-level parity through+// the analyzer's public API.// goldenPath resolves a native testdata fixture from this package's test// directory (internal/deepdoc/parser/pdf/inference/native_analyzer). Four ".." climb todiff --git a/internal/deepdoc/parser/pdf/inference/native_analyzer/raster_alignment_test.go b/internal/deepdoc/parser/pdf/inference/native_analyzer/raster_alignment_test.goindex 805747f6cd..d8b0f70b44 100644--- a/internal/deepdoc/parser/pdf/inference/native_analyzer/raster_alignment_test.go+++ b/internal/deepdoc/parser/pdf/inference/native_analyzer/raster_alignment_test.go@@ -4,14 +4,13 @@ package infnative// End-to-end raster alignment tests (reviewer gap #1 and #2).//-// The native proof establishes "given the same raster image bytes, Go == Python"-// (deepdoc_go_alignment_report.md → Equivalence proof / "Boundary of this proof"). But in production neither-// side receives a pre-rendered PNG: the Go server rasterizes PDF pages with-// pdfium (pdfium.RenderPage @ 216 DPI) and the Python deepdoc pipeline-// rasterizes with pdfplumber (page.to_image(resolution=72*zoomin, antialias=True),-// zoomin=3 => 216 DPI). These tests close the remaining gap by rasterizing the-// SAME PDF page with BOTH paths and comparing the resulting boxes in source--// pixel coordinates:+// The native proof establishes "given the same raster image bytes, Go == Python".+// But in production neither side receives a pre-rendered PNG: the Go server+// rasterizes PDF pages with pdfium (pdfium.RenderPage @ 216 DPI) and the Python+// deepdoc pipeline rasterizes with pdfplumber+// (page.to_image(resolution=72*zoomin, antialias=True), zoomin=3 => 216 DPI).+// These tests close the remaining gap by rasterizing the SAME PDF page with BOTH+// paths and comparing the resulting boxes in source-pixel coordinates://// Go side : pdfium.RenderPage(pdf, page, 216) -> a.DLA / a.OCRDetect / a.TSR// Py side : ref_raster.py renders the same page at 216 DPI via deepdoc's own