Skip to content

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

Merge dev into PR #1669 and preserve integrated behavior

Bingxi Zhao (Frank)335301bff32f
diff --cc docs-for-user/SOURCE_VISUALS.md
index e27a10fbc,9a9a00326..10c2553a0
--- a/docs-for-user/SOURCE_VISUALS.md
+++ b/docs-for-user/SOURCE_VISUALS.md
@@@ -27,32 -27,32 +27,62 @@@ that require OCR when no usable OCR eng
Other RAG providers do not yet index this visual manifest. Interactive
exercises, visual annotations, and mastery updates are separate future work.
+## Tiny scanned PDF pages with MinerU
+
+Some scanned PDFs encode a full-resolution page in an unusually small physical
+page box. MinerU's official cloud backend can optionally enlarge these pages in
+a temporary upload copy. The original file, page order, compressed image bytes,
+and soft masks are preserved; language, OCR, model, formula, and table settings
+remain as configured. This is a geometry workaround, not a guarantee of better
+OCR or preservation of arbitrary interactive PDF semantics.
+
+The option defaults to off. An administrator can enable it through the existing
+`PUT /api/settings/document-parsing` endpoint with this partial payload:
+
+```json
+{"engines": {"mineru": {"normalize_tiny_scans": true}}}
+```
+
+Use `false` to disable it. The option also lives at
+`engines.mineru.normalize_tiny_scans` in `document_parsing.json`. Enabling it
+changes the cloud parse-cache signature; old parses remain intact. Existing
+indexes are not rebuilt automatically. The optional
+`deeptutor[parse-pymupdf4llm]` extra provides the required PyMuPDF dependency.
+
+Only text-free pages below 144 points on their longest side, with a
+high-resolution image covering at least 80% of the page at an apparent density
+of at least 1200 DPI, qualify. The longest side is scaled to 768 points.
+Rotated, cropped, annotated, vector-bearing, or non-default UserUnit pages are
+left unchanged. Local MinerU, custom cloud endpoints, normal PDFs, and non-PDF
+inputs keep their existing behavior. Temporary copies are removed after success
+or failure, and upload is refused if compressed image streams change.
++
+ ## Optional image-description batches
+
+ The existing LlamaIndex image-description pass can send multiple images per
+ vision request. Set `image_description_batch_size` through
+ `PUT /api/knowledge-bases/rag-pipelines/llamaindex/config`, for example:
+
+ ```json
+ {"image_description_batch_size": 4}
+ ```
+
+ The default is `1`, preserving individual requests; accepted values are clamped
+ to 1–8. Concurrency limits count batches when enabled, and the existing timeout
+ covers the whole batch including any split attempts. Returned captions are
+ matched by explicit IDs, never by response order. Malformed JSON/ID maps and
+ explicit context overflow split into smaller groups, eventually using the
+ existing single-image prompt. A group of N images makes at most 2N−1 completion
+ calls, with provider retries disabled for this mode. Authentication and rate
+ limits stop queued groups in the job; other API/transport errors do not split.
+
+ This only affects subsequently processed images in the existing LlamaIndex
+ description pass. It does not enable descriptions for structured source visuals
+ or alter reading-material captions. Batches use a different structured prompt
+ and cache complete, successful groups separately from independent single-image
+ captions. The digest includes ordered image contents and metadata, prompts,
+ model identity, and retry policy. Failed, incomplete, or canceled groups are
+ not cached; successful split groups can be reused. Setting the size
+ back to `1` restores the normal single-image path. The model must support
+ multiple image blocks; unsupported API responses are reported without a burst
+ of fallback requests.