A fast, helpful, and open-source document parser
implement new docs
Logan Markewichcba3fa1f6631
diff --git a/docs/src/content/docs/liteparse/cli-reference.md b/docs/src/content/docs/liteparse/cli-reference.mdindex 0716989..c5bb850 100644--- a/docs/src/content/docs/liteparse/cli-reference.md+++ b/docs/src/content/docs/liteparse/cli-reference.md@@ -5,7 +5,7 @@ sidebar:order: 5----LiteParse provides the `lit` CLI with three commands: `parse`, `batch-parse`, and `screenshot`.+LiteParse provides the `lit` CLI with three commands: `parse`, `batch-parse`, and `screenshot`. The CLI is the same whether installed via `npm`, `pip`, or built from Rust source.## `lit parse`@@ -27,17 +27,16 @@ lit parse [options] <file>|--------|-------------|---------|| `-o, --output <file>` | Write output to a file instead of stdout | — || `--format <format>` | Output format: `json` or `text` | `text` |-| `--ocr-server-url <url>` | HTTP OCR server URL | — (uses Tesseract) || `--no-ocr` | Disable OCR entirely | — |-| `--ocr-language <lang>` | OCR language code | `en` |+| `--ocr-language <lang>` | OCR language code (Tesseract format) | `eng` |+| `--ocr-server-url <url>` | HTTP OCR server URL | — (uses Tesseract) |+| `--tessdata-path <path>` | Path to tessdata directory | — (uses `TESSDATA_PREFIX` env var) || `--num-workers <n>` | Pages to OCR in parallel | CPU cores - 1 |-| `--max-pages <n>` | Maximum pages to parse | `10000` |+| `--max-pages <n>` | Maximum pages to parse | `1000` || `--target-pages <pages>` | Pages to parse (e.g., `"1-5,10"`) | — (all pages) || `--dpi <dpi>` | Rendering DPI | `150` |-| `--no-precise-bbox` | **Deprecated:** Disable populating the output boundingBoxes array. Will be removed in v2.0. Text item coordinates (`x`, `y`, `width`, `height`) are always present regardless. | — || `--preserve-small-text` | Keep very small text | — || `--password <password>` | Password for encrypted/protected documents | — |-| `--config <file>` | JSON config file path | — || `-q, --quiet` | Suppress progress output | — |### Examples@@ -87,17 +86,16 @@ lit batch-parse [options] <input-dir> <output-dir>| Option | Description | Default ||--------|-------------|---------|| `--format <format>` | Output format: `json` or `text` | `text` |-| `--ocr-server-url <url>` | HTTP OCR server URL | — (uses Tesseract) || `--no-ocr` | Disable OCR entirely | — |-| `--ocr-language <lang>` | OCR language code | `en` |+| `--ocr-language <lang>` | OCR language code | `eng` |+| `--ocr-server-url <url>` | HTTP OCR server URL | — (uses Tesseract) |+| `--tessdata-path <path>` | Path to tessdata directory | — || `--num-workers <n>` | Pages to OCR in parallel | CPU cores - 1 |-| `--max-pages <n>` | Maximum pages per file | `10000` |+| `--max-pages <n>` | Maximum pages per file | `1000` || `--dpi <dpi>` | Rendering DPI | `150` |-| `--no-precise-bbox` | **Deprecated:** Disable populating the output boundingBoxes array. Will be removed in v2.0. Text item coordinates (`x`, `y`, `width`, `height`) are always present regardless. | — || `--recursive` | Search subdirectories | — || `--extension <ext>` | Only process this extension (e.g., `".pdf"`) | — (all supported) || `--password <password>` | Password for encrypted/protected documents (applied to all files) | — |-| `--config <file>` | JSON config file path | — || `-q, --quiet` | Suppress progress output | — |### Examples@@ -111,9 +109,6 @@ lit batch-parse ./documents ./output --recursive --extension ".pdf"# Batch parse with JSON output and no OCRlit batch-parse ./documents ./output --format json --no-ocr--# Use a config file for consistent settings-lit batch-parse ./documents ./output --config liteparse.config.json```---@@ -139,9 +134,7 @@ lit screenshot [options] <file>| `-o, --output-dir <dir>` | Output directory | `./screenshots` || `--target-pages <pages>` | Pages to screenshot (e.g., `"1,3,5"` or `"1-5"`) | — (all pages) || `--dpi <dpi>` | Rendering DPI | `150` |-| `--format <format>` | Image format: `png` or `jpg` | `png` || `--password <password>` | Password for encrypted/protected documents | — |-| `--config <file>` | JSON config file path | — || `-q, --quiet` | Suppress progress output | — |### Examples@@ -151,13 +144,10 @@ lit screenshot [options] <file>lit screenshot document.pdf -o ./pages# First 5 pages at high DPI-lit screenshot document.pdf --pages "1-5" --dpi 300 -o ./pages--# JPG format for smaller files-lit screenshot document.pdf --format jpg -o ./pages+lit screenshot document.pdf --target-pages "1-5" --dpi 300 -o ./pages# Specific pages only-lit screenshot document.pdf --pages "1,5,10" -o ./pages+lit screenshot document.pdf --target-pages "1,5,10" -o ./pages```---diff --git a/docs/src/content/docs/liteparse/getting_started.md b/docs/src/content/docs/liteparse/getting_started.mdxsimilarity index 57%rename from docs/src/content/docs/liteparse/getting_started.mdrename to docs/src/content/docs/liteparse/getting_started.mdxindex c9cde22..b1076e1 100644--- a/docs/src/content/docs/liteparse/getting_started.md+++ b/docs/src/content/docs/liteparse/getting_started.mdx@@ -5,17 +5,48 @@ sidebar:order: 1---+LiteParse is available for Node.js, Python, and as a standalone Rust binary.++All versions (except WASM) ship the same CLI and core library capabilities.+## Installation-Install LiteParse globally via npm to use the `lit` command anywhere:+<Tabs>+<TabItem label="Node.js / TypeScript" value="node">+```bashnpm i -g @llamaindex/liteparse```+</TabItem>+<TabItem label="Python" value="python">++```bash+pip install liteparse+```+</TabItem>+<TabItem label="Rust" value="rust">++```bash+cargo install liteparse+```++</TabItem>+<TabItem label="Browser (WASM)" value="wasm">++```bash+npm i @llamaindex/liteparse-wasm+```++Since the WASM package is designed for browser usage, it does not include the `lit` CLI command. Instead, you can use the exported functions directly from JavaScript to parse documents in the browser environment.++For full browser usage and limitations, see the [WASM package guide](/liteparse/guides/browser-usage/).+</TabItem>+</Tabs>## Quick start-Once installed, you can start parsing from the command line:+Once installed, parse from the command line:```bash# Parse a PDF and print text to stdout@@ -52,5 +83,6 @@ lit screenshot document.pdf -o ./screenshots- [Library usage](/liteparse/guides/library-usage/): Use LiteParse programmatically from TypeScript or Python.- [OCR configuration](/liteparse/guides/ocr/): Configure Tesseract, use an external OCR server, or bring your own.- [Multi-format support](/liteparse/guides/multi-format/): Parse DOCX, XLSX, PPTX, images, and more.+- [Browser usage (WASM)](/liteparse/guides/browser-usage/): Run LiteParse entirely in the browser.- [Agent skill](/liteparse/guides/agent-skill/): Add LiteParse as a skill for coding agents.- [CLI reference](/liteparse/cli-reference/): Complete command and option reference.diff --git a/docs/src/content/docs/liteparse/guides/agent-skill.md b/docs/src/content/docs/liteparse/guides/agent-skill.mdindex 11a8827..2337360 100644--- a/docs/src/content/docs/liteparse/guides/agent-skill.md+++ b/docs/src/content/docs/liteparse/guides/agent-skill.md@@ -28,29 +28,3 @@ Once the skill is installed, you can ask your coding agent things like:- "Screenshot pages 1-5 of this PDF at 300 DPI"- "Parse this scanned document using the PaddleOCR server on localhost:8828"- "Get the bounding boxes for all text on page 3"---## Configuring Defaults--You might want to configure some defaults so that your agent doesn't have to specify them in every prompt. You can create a `liteparse.config.json` file in the root of your project with settings like:--```json-{- "ocrLanguage": "en",- "ocrEnabled": true,- "maxPages": 1000,- "dpi": 150,- "outputFormat": "json",- "preserveVerySmallText": false-}-```--This is especially useful for custom OCR servers. Just add the `ocrServerUrl` to your config:--```json-{- "ocrServerUrl": "http://localhost:8828/ocr",- "ocrLanguage": "en",- "outputFormat": "json"-}-```diff --git a/docs/src/content/docs/liteparse/guides/browser-usage.md b/docs/src/content/docs/liteparse/guides/browser-usage.mdindex 6432136..de53be9 100644--- a/docs/src/content/docs/liteparse/guides/browser-usage.md+++ b/docs/src/content/docs/liteparse/guides/browser-usage.md@@ -1,207 +1,93 @@----title: Browser Usage-description: Run LiteParse in the browser with Vite.+title: Browser Usage (WASM)+description: Run LiteParse entirely in the browser with the WASM package.sidebar:order: 6----LiteParse's core parsing engine (PDF.js text extraction, spatial grid projection, and OCR via Tesseract.js) can run entirely in the browser. Since the library includes Node-only dependencies (sharp, fs, child_process, etc.), you need a bundler like [Vite](https://vite.dev/) to swap those out with browser-safe stubs at build time.+LiteParse ships a WebAssembly package that runs entirely in the browser — no server, no cloud calls. It supports PDF parsing and custom OCR engines implemented in JavaScript.-## What works in the browser--- **PDF parsing** from `Uint8Array` input (use `file.arrayBuffer()` to get bytes from a file picker)-- **OCR** via Tesseract.js (runs in Web Workers, fetches language data from CDN on first use)-- **Text and JSON output formats**--## What doesn't work--- **File path input** — pass `Uint8Array` instead-- **DOCX/XLSX/PPTX/image conversion** — requires LibreOffice/ImageMagick which aren't available in the browser-- **HTTP OCR server** — irrelevant for client-side use; Tesseract.js runs directly in the browser-- **Screenshots** — these use PDFium + sharp, which are native Node addons--## Vite setup+## Install-The setup requires two things:--1. **A Vite plugin** that redirects Node-only source files to browser-safe stub replacements-2. **`resolve.alias` entries** that stub out Node built-in modules (`fs`, `path`, `child_process`, etc.)+```bash+npm install @llamaindex/liteparse-wasm+```-### Full Vite config+## Quick start```typescript-// vite.config.ts-import { defineConfig, type Plugin } from "vite";-import { resolve, dirname } from "node:path";-import { fileURLToPath } from "node:url";--const here = (p: string) =>- resolve(fileURLToPath(new URL(".", import.meta.url)), p);--// These Node-only files need browser replacements.-// Point each target at your own stub file (see below).-const FILE_REDIRECTS: Array<{ match: RegExp; target: string }> = [- {- match: /\/engines\/pdf\/pdfium-renderer(\.js|\.ts)?$/,- target: here("stubs/pdfium-renderer.ts"),- },- {- match: /\/engines\/pdf\/pdfjsImporter(\.js|\.ts)?$/,- target: here("stubs/pdfjsImporter.ts"),- },- {- match: /\/engines\/ocr\/http-simple(\.js|\.ts)?$/,- target: here("stubs/http-simple.ts"),- },- {- match: /\/conversion\/convertToPdf(\.js|\.ts)?$/,- target: here("stubs/convertToPdf.ts"),- },- {- match: /\/processing\/gridDebugLogger(\.js|\.ts)?$/,- target: here("stubs/gridDebugLogger.ts"),- },- {- match: /\/processing\/gridVisualizer(\.js|\.ts)?$/,- target: here("stubs/gridVisualizer.ts"),- },-];--function liteparseNodeRedirects(): Plugin {- return {- name: "liteparse-node-redirects",- enforce: "pre",- async resolveId(source, importer) {- if (!importer) return null;- const importerDir = dirname(importer);- const absolutePath = source.startsWith(".")- ? resolve(importerDir, source)- : source;- for (const { match, target } of FILE_REDIRECTS) {- if (match.test(absolutePath) || match.test(source)) {- return target;- }- }- return null;- },- };-}--export default defineConfig({- plugins: [liteparseNodeRedirects()],- optimizeDeps: {- include: ["tesseract.js"],- },- resolve: {- alias: [- { find: "node:fs/promises", replacement: here("stubs/empty.ts") },- { find: "node:fs", replacement: here("stubs/empty.ts") },- { find: "node:url", replacement: here("stubs/empty.ts") },- { find: "node:path", replacement: here("stubs/empty.ts") },- { find: "node:os", replacement: here("stubs/empty.ts") },- { find: "node:child_process", replacement: here("stubs/empty.ts") },- { find: /^fs$/, replacement: here("stubs/empty.ts") },- { find: /^fs\/promises$/, replacement: here("stubs/empty.ts") },- { find: /^path$/, replacement: here("stubs/empty.ts") },- { find: /^os$/, replacement: here("stubs/empty.ts") },- { find: /^child_process$/, replacement: here("stubs/empty.ts") },- { find: "form-data", replacement: here("stubs/empty.ts") },- { find: "axios", replacement: here("stubs/empty.ts") },- { find: "file-type", replacement: here("stubs/file-type.ts") },- ],- },- define: {- "process.env.NODE_ENV": JSON.stringify("production"),- },-});-```+import init, { LiteParse } from "@llamaindex/liteparse-wasm";-### Stub files+// Load the WASM module+await init();-You need to create stub files that export the same interfaces as the Node-only modules but with browser-safe (or no-op) implementations. A complete working example with all stubs is available in the [`scripts/browser-compat/`](https://github.com/run-llama/liteparse/tree/main/scripts/browser-compat) directory.+const parser = new LiteParse({+ ocrEnabled: false,+ outputFormat: "json",+});-The key stubs:+// data is a Uint8Array (e.g. from <input type="file"> or fetch)+const bytes = new Uint8Array(await file.arrayBuffer());+const result = await parser.parse(bytes);-**`stubs/empty.ts`** — catch-all for Node built-ins:-```typescript-export default {};+console.log(result.text);+console.log(result.pages[0]);```-**`stubs/convertToPdf.ts`** — only PDF magic-byte detection is needed:-```typescript-export async function convertToPdf() {- throw new Error("File conversion is not supported in browser environments.");-}-export async function convertBufferToPdf() {- throw new Error("File conversion is not supported in browser environments.");-}-export async function cleanupConversionFiles() {}-export async function guessExtensionFromBuffer(data: Uint8Array) {- if (data[0] === 0x25 && data[1] === 0x50 && data[2] === 0x44 && data[3] === 0x46) {- return ".pdf";- }- return null;-}-```+## What works in the browser-**`stubs/pdfjsImporter.ts`** — browser-safe PDF.js loader (no `node:url`/`node:path`):-```typescript-// @ts-expect-error vendored ESM build has no types-import * as pdfjs from "@llamaindex/liteparse/vendor/pdfjs/pdf.mjs";--export async function importPdfJs() {- return {- fn: (pdfjs as any).getDocument,- dir: new URL("@llamaindex/liteparse/vendor/pdfjs", import.meta.url).href,- };-}-```+- **PDF parsing** from `Uint8Array` input (use `file.arrayBuffer()` to get bytes from a file picker for example)+- **Custom OCR** via the `ocrEngine` callback interface (see below)+- **Text and JSON output formats**-**`stubs/file-type.ts`** — minimal file type detection:-```typescript-export async function fileTypeFromBuffer(buf: Uint8Array) {- if (buf[0] === 0x25 && buf[1] === 0x50 && buf[2] === 0x44 && buf[3] === 0x46) {- return { ext: "pdf", mime: "application/pdf" };- }- return undefined;-}-export async function fileTypeFromFile() {- return undefined;-}-```+## What doesn't work-The remaining stubs (`pdfium-renderer`, `http-simple`, `gridDebugLogger`, `gridVisualizer`) just need to export the same class/function names with no-op implementations. See the full example in `scripts/browser-compat/stubs/`.+- **File path input** — pass `Uint8Array` instead+- **DOCX/XLSX/PPTX/image conversion** — requires LibreOffice/ImageMagick which aren't available in the browser+- **Built-in Tesseract or HTTP OCR** — use the custom `ocrEngine` interface instead+- **Screenshots** — not available in the WASM build-## Usage in your app+## OCR in the browser-Once the bundler is configured, use LiteParse the same way as in Node — just pass `Uint8Array` instead of file paths:+The native Tesseract and HTTP OCR backends are not available in WASM. To use OCR, pass a custom `ocrEngine` object with a `recognize` method:```typescript-import { LiteParse } from "@llamaindex/liteparse";-const parser = new LiteParse({ocrEnabled: true,- outputFormat: "json",+ ocrLanguage: "eng",+ ocrEngine: {+ /**+ * @param imageData PNG-encoded image bytes+ * @param width rendered page width in pixels+ * @param height rendered page height in pixels+ * @param language e.g. "eng"+ * @returns array of { text, bbox: [x1, y1, x2, y2], confidence }+ */+ async recognize(imageData, width, height, language) {+ // e.g. call a Web Worker wrapping tesseract.js, or a remote OCR service+ return [+ { text: "Hello", bbox: [10, 20, 80, 40], confidence: 0.98 },+ ];+ },+ },});+```-// From a file picker-const fileInput = document.querySelector<HTMLInputElement>("#file")!;-fileInput.addEventListener("change", async () => {- const file = fileInput.files?.[0];- if (!file) return;-- const bytes = new Uint8Array(await file.arrayBuffer());- const result = await parser.parse(bytes);+This lets you plug in any OCR implementation — a Web Worker running tesseract.js, a cloud OCR API, or anything else that returns text with bounding boxes.- console.log(result.text);- console.log(JSON.stringify(result.json, null, 2));-});-```+## Config options-## Known gotchas+All optional, camelCase:-- **First OCR run is slow.** Tesseract.js fetches `eng.traineddata` (~10 MB) from the jsdelivr CDN on first use. Subsequent runs use the browser cache.-- **`file://` doesn't work.** The PDF.js worker and WASM assets need to be served over HTTP. Use `vite dev` or a static server.-- **Safari < 17.** Older Safari/iOS WebKit ships `ReadableStream` without `Symbol.asyncIterator`, which PDF.js needs. You may need a polyfill — see the [MDN docs](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#browser_compatibility).-- **ArrayBuffer detachment.** PDF.js transfers the input buffer to its worker, which detaches the original `ArrayBuffer`. If you need the bytes again after parsing (e.g., for screenshots), read the file a second time with `file.arrayBuffer()`.-- **Bundle size.** Expect ~500 KB for the app bundle plus ~2.3 MB for the PDF.js worker. Tesseract loads lazily on first OCR run.+| Option | Type | Default | Description |+|---|---|---|---|+| `ocrLanguage` | `string` | `"eng"` | Language code passed to the OCR engine |+| `ocrEnabled` | `boolean` | `true` | Run OCR on text-sparse pages |+| `maxPages` | `number` | `1000` | Stop after this many pages |+| `targetPages` | `string` | — | e.g. `"1-5,10,15-20"` |+| `dpi` | `number` | `150` | Render DPI for OCR |+| `outputFormat` | `"json" \| "text"` | `"json"` | Format used by `parser.format(...)` |+| `preserveVerySmallText` | `boolean` | `false` | Keep tiny text that's normally filtered |+| `password` | `string` | — | Password for protected PDFs |+| `quiet` | `boolean` | `false` | Suppress progress logging |+| `ocrEngine` | `object` | — | Custom JS-side OCR engine (see above) |diff --git a/docs/src/content/docs/liteparse/guides/library-usage.md b/docs/src/content/docs/liteparse/guides/library-usage.mdxsimilarity index 55%rename from docs/src/content/docs/liteparse/guides/library-usage.mdrename to docs/src/content/docs/liteparse/guides/library-usage.mdxindex 6f3b1e7..f06a1ad 100644--- a/docs/src/content/docs/liteparse/guides/library-usage.md+++ b/docs/src/content/docs/liteparse/guides/library-usage.mdx@@ -5,9 +5,13 @@ sidebar:order: 1----LiteParse can be used as a library in your own code, not just from the CLI. There are packages for both TypeScript and Python.+LiteParse can be used as a library in your own code, not just from the CLI. There are native packages for both TypeScript and Python.-## TypeScript++<Tabs>+<TabItem label="TypeScript" value="typescript">++## TypeScript / Node.js usageInstall as a project dependency:@@ -36,13 +40,15 @@ for (const page of result.pages) {### JSON output with bounding boxes+Text items include spatial coordinates (`x`, `y`, `width`, `height`) in PDF points:+```typescriptconst parser = new LiteParse({ outputFormat: "json" });const result = await parser.parse("document.pdf");-for (const page of result.json?.pages || []) {+for (const page of result.pages) {for (const item of page.textItems) {- console.log(`[${item.x}, ${item.y}] → [${item.x + item.width}, ${item.y + item.height}] ${item.text}`);+ console.log(`[${item.x}, ${item.y}] ${item.width}×${item.height} "${item.text}"`);}}```@@ -59,13 +65,13 @@ const parser = new LiteParse({dpi: 300,outputFormat: "json",targetPages: "1-10",- password: "secret", // for encrypted/protected documents+ password: "secret",});```### Buffer / Uint8Array input-You can pass raw bytes directly instead of a file path. PDF buffers are parsed with **zero disk I/O** — no temp files are written:+You can pass raw bytes directly instead of a file path:```typescriptimport { readFile } from "fs/promises";@@ -82,24 +88,21 @@ const buffer = Buffer.from(await response.arrayBuffer());const result2 = await parser.parse(buffer);```-Non-PDF buffers (images, Office documents) are written to a temp directory for format conversion. You can control the temp directory with the `LITEPARSE_TMPDIR` environment variable.-### Screenshots-Generate page images as buffers — useful for sending to LLMs or saving to disk. Accepts file paths, `Buffer`, or `Uint8Array`:+Generate page images as buffers — useful for sending to LLMs or saving to disk:```typescriptconst parser = new LiteParse();-const screenshots = await parser.screenshot("document.pdf");+const screenshots = parser.screenshot("document.pdf");for (const shot of screenshots) {console.log(`Page ${shot.pageNum}: ${shot.width}x${shot.height}`);- // shot.imageBuffer contains the raw PNG/JPG data+ // shot.imageBuffer contains the raw PNG data}-// Also works with buffer input-const pdfBytes = await readFile("document.pdf");-const shots = await parser.screenshot(pdfBytes, [1, 2, 3]);+// Screenshot specific pages+const shots = parser.screenshot("document.pdf", [1, 2, 3]);```### Environment variables@@ -107,34 +110,18 @@ const shots = await parser.screenshot(pdfBytes, [1, 2, 3]);| Variable | Description ||----------|-------------|| `TESSDATA_PREFIX` | Path to a directory containing Tesseract `.traineddata` files. For offline/air-gapped environments. Also available as the `tessdataPath` config option. |-| `LITEPARSE_TMPDIR` | Override the temp directory for format conversion. Defaults to `os.tmpdir()`. |-See the [API reference](/liteparse/api/) for full type details.-----+</TabItem>+<TabItem label="Python" value="python">-## Python+## Python usage-The Python package is a wrapper around the LiteParse Node.js CLI. **Node.js (>= 18) must be installed** on your system.--### Installation--First, install the LiteParse CLI:--```bash-npm install -g @llamaindex/liteparse-```--Then install the Python package:+Install the package from PyPI:```bashpip install liteparse```-<Aside type="caution">- The Python package calls the LiteParse CLI under the hood via subprocess. Node.js (>= 18) is required. While the package can auto-install the CLI via `npm install -g @llamaindex/liteparse` on first use, it is recommended to install it separately beforehand.-</Aside>-### Parsing a document```python@@ -148,25 +135,23 @@ print(result.text)# Per-page datafor page in result.pages:- print(f"Page {page.pageNum}: {len(page.textItems)} text items")+ print(f"Page {page.page_num}: {len(page.text_items)} text items")```### Configuration-The `LiteParse` constructor accepts `cli_path` (to override CLI auto-detection) and `install_if_not_available` (to control auto-install behavior). All parsing options are passed per-call:+All options are passed to the constructor:```python-parser = LiteParse()--result = parser.parse(- "document.pdf",+parser = LiteParse(ocr_enabled=True,ocr_server_url="http://localhost:8828/ocr",ocr_language="fra",dpi=300,target_pages="1-5",- password="secret", # for encrypted/protected documents+ password="secret",)+result = parser.parse("document.pdf")```### Parsing from bytes@@ -181,17 +166,25 @@ result = parser.parse(pdf_bytes)print(result.text)```-### Batch parsing--For multiple files, batch mode reuses the PDF engine and is significantly faster:+### Screenshots```python-result = parser.batch_parse(- input_dir="./documents",- output_dir="./output",- recursive=True,- extension_filter=".pdf",-)--print(f"Output written to: {result.output_dir}")+screenshots = parser.screenshot("document.pdf", page_numbers=[1, 2, 3])+for s in screenshots:+ print(f"Page {s.page_num}: {s.width}x{s.height}")+ # s.image_bytes contains PNG data```++### Environment variables++| Variable | Description |+|----------|-------------|+| `TESSDATA_PREFIX` | Path to a directory containing Tesseract `.traineddata` files. For offline/air-gapped environments. Also available as the `tessdata_path` constructor option. |+</TabItem>+<TabItem label="Rust" value="rust">++</TabItem>+<TabItem label="Browser (WASM)" value="wasm">++</TabItem>+</Tabs>\ No newline at end of filediff --git a/docs/src/content/docs/liteparse/guides/multi-format.md b/docs/src/content/docs/liteparse/guides/multi-format.mdindex c2a66ba..1a816c0 100644--- a/docs/src/content/docs/liteparse/guides/multi-format.md+++ b/docs/src/content/docs/liteparse/guides/multi-format.md@@ -13,9 +13,9 @@ LiteParse automatically converts non-PDF formats to PDF before parsing. This let| Category | Extensions ||----------|-----------|-| Word | `.doc`, `.docx`, `.docm`, `.odt`, `.rtf` |-| PowerPoint | `.ppt`, `.pptx`, `.pptm`, `.odp` |-| Spreadsheets | `.xls`, `.xlsx`, `.xlsm`, `.ods`, `.csv`, `.tsv` |+| Word | `.doc`, `.docx`, `.docm`, `.odt`, `.rtf`, `.pages` |+| PowerPoint | `.ppt`, `.pptx`, `.pptm`, `.odp`, `.key` |+| Spreadsheets | `.xls`, `.xlsx`, `.xlsm`, `.ods`, `.csv`, `.tsv`, `.numbers` |### Images (via ImageMagick)diff --git a/docs/src/content/docs/liteparse/guides/ocr.md b/docs/src/content/docs/liteparse/guides/ocr.mdindex 3fced8b..a9e5722 100644--- a/docs/src/content/docs/liteparse/guides/ocr.md+++ b/docs/src/content/docs/liteparse/guides/ocr.md@@ -9,14 +9,12 @@ LiteParse uses OCR selectively — only on embedded images or pages where native## Built-in Tesseract (default)-Tesseract.js is bundled with LiteParse. The only setup is the automatic download of the Tesseract model files on first use. Just run:+Tesseract is bundled with LiteParse and works out of the box. Just run:```bashlit parse document.pdf```-If bundling LiteParse into a docker container or server environment, you might want to pre-download the Tesseract files to avoid network calls at runtime with the above command or similar.-### Language supportSpecify the OCR language for better accuracy on non-English documents:@@ -29,6 +27,21 @@ lit parse document.pdf --ocr-language jpn # JapaneseTesseract uses [ISO 639-3](https://tesseract-ocr.github.io/tessdoc/Data-Files-in-different-versions.html) language codes (`eng`, `fra`, `deu`, etc.).+### Offline / air-gapped environments++For environments without internet access, point Tesseract at a local directory containing pre-downloaded `.traineddata` files:++```bash+# Via environment variable+export TESSDATA_PREFIX=/path/to/tessdata+lit parse document.pdf --ocr-language eng++# Or via CLI flag+lit parse document.pdf --tessdata-path /path/to/tessdata+```++The `tessdata_path` / `tessdataPath` option is also available in the library APIs.+### Disabling OCRIf you don't need OCR (pure native-text PDFs, or you don't care about images), disable it for faster parsing:@@ -136,10 +149,14 @@ lit parse document.pdf --ocr-server-url http://localhost:8080/ocr- Results should be in reading order (top-to-bottom, left-to-right)- Cache OCR models in memory rather than reloading per request+## OCR in the browser (WASM)++The built-in Tesseract and HTTP OCR backends are not available in the WASM build. Instead, you can pass a custom `ocrEngine` object with a `recognize` method. See the [browser usage guide](/liteparse/guides/browser-usage/) for details.+### A note on OCR approaches-These days, its common to apply the term "OCR" to both traditional approaches and newer LLM-based document understanding models.+These days, its common to apply the term "OCR" to both traditional approaches and newer LLM-based document understanding models.-The LiteParse OCR API is designed specifically for approaches that return text with bounding boxes.+The LiteParse OCR API is designed specifically for approaches that return text with bounding boxes.If you are trying to integrate a method that doesn't return bounding boxes, you will have to generate dummy bounding boxes.diff --git a/docs/src/content/docs/liteparse/guides/parsing-urls.md b/docs/src/content/docs/liteparse/guides/parsing-urls.mdindex 9852fb5..47048a3 100644--- a/docs/src/content/docs/liteparse/guides/parsing-urls.md+++ b/docs/src/content/docs/liteparse/guides/parsing-urls.md@@ -5,7 +5,7 @@ sidebar:order: 5----To parse remote files, LiteParse support both CLI and library usage for reading bytes and streams. This means the CLI can download them with any tool you like and pipe the bytes to `lit parse` using `-` as the file argument, while the libraries can fetch the bytes directly from the URL and pass them to the parser.+To parse remote files, LiteParse supports both CLI and library usage for reading bytes and streams. The CLI can download them with any tool you like and pipe the bytes to `lit parse` using `-` as the file argument, while the libraries can fetch the bytes directly and pass them to the parser.## CLI usage@@ -24,7 +24,9 @@ The `-` argument tells LiteParse to read from stdin instead of a file path. Any## Library usage-The TypeScript library accepts `Buffer`/`Uint8Array` directly, so you can handle the download however you like:+The TypeScript library accepts `Buffer`/`Uint8Array` directly, so you can handle the download however you like.++For example, using `fetch` in a Node.js environment:```typescriptimport { LiteParse } from "@llamaindex/liteparse";diff --git a/docs/src/content/docs/liteparse/guides/visual-citations.md b/docs/src/content/docs/liteparse/guides/visual-citations.mdxsimilarity index 68%rename from docs/src/content/docs/liteparse/guides/visual-citations.mdrename to docs/src/content/docs/liteparse/guides/visual-citations.mdxindex 3811c60..2936951 100644--- a/docs/src/content/docs/liteparse/guides/visual-citations.md+++ b/docs/src/content/docs/liteparse/guides/visual-citations.mdx@@ -5,13 +5,13 @@ sidebar:order: 4----When building agents or RAG workflows, it is often not enough to parse text and call it done. Frequently, users and applications will require you to show _where_ that text came from.+When building agents or RAG workflows, it is often not enough to parse text and call it done. Frequently, users and applications will require you to show _where_ that text came from.LiteParse gives you spatial coordinates for every text item, plus page screenshots, so you can highlight exact regions on the rendered page.## How bounding boxes work-When you parse a document with JSON output, each page includes a key data source for visual citations: **`textItems`**. Every extracted text element with its position (`x`, `y`, `width`, `height`) and content.+When you parse a document with JSON output, each page includes **`textItems`** — every extracted text element with its position (`x`, `y`, `width`, `height`) and content.```json$ lit parse document.pdf --format json@@ -32,32 +32,51 @@ Coordinates are in **PDF points** (1 point = 1/72 inch). Origin is the top-left## Library usage-The library lets you do both in a single script, parse for bboxes and generate screenshots. For example, you might be looking for specific information like "Revenue" and want to show exactly where it appears on the page:+The library lets you parse for bboxes and generate screenshots in a single script. For example, searching for "0°C to 70°C" and showing where it appears:-```typescript-import { LiteParse } from "@llamaindex/liteparse";+<Tabs>+<TabItem label="TypeScript" value="typescript">+### TypeScript-const parser = new LiteParse({ outputFormat: "json", dpi: 150 });+```typescript+import { LiteParse, searchItems } from "@llamaindex/liteparse";+const parser = new LiteParse({ outputFormat: "json" });const result = await parser.parse("report.pdf");-const screenshots = await parser.screenshot("report.pdf");--// Find a text item by its content-for (const page of result.json?.pages || []) {- for (const item of page.textItems) {- if (item.text.includes("Revenue")) {- console.log(`Found on page ${page.page}: (${item.x}, ${item.y}) ${item.width}×${item.height}`);- }++for (const page of result.pages) {+ const matches = searchItems(page.textItems, { phrase: "0°C to 70°C" });+ for (const match of matches) {+ console.log(`Found "${match.text}" at (${match.x}, ${match.y}) ${match.width}×${match.height}`);}}```+</TabItem>++<TabItem label="Python" value="python">++### Python++```python+from liteparse import LiteParse, search_items++parser = LiteParse()+result = parser.parse("report.pdf")++for page in result.pages:+ matches = search_items(page.text_items, "0°C to 70°C")+ for match in matches:+ print(f'Found "{match.text}" at ({match.x}, {match.y}) {match.width}×{match.height}')+```+</TabItem>+</Tabs>## Converting coordinates to image pixelsText item coordinates are in PDF points, but screenshots are in pixels. To draw highlights on a screenshot, you need to scale the coordinates:```typescript-const scaleFactor = dpi / 72; // PDF points → pixels at your chosen DPI+const scaleFactor = dpi / 72; // PDF points -> pixels at your chosen DPIfunction itemToPixels(item, dpi = 150) {const scale = dpi / 72;@@ -70,29 +89,7 @@ function itemToPixels(item, dpi = 150) {}```-For example, at the default 150 DPI the scale factor is `150 / 72 ≈ 2.08`, so a text item at `(72, 200)` maps to pixel `(150, 416)`.--## Searching for phrases with `searchItems`--A single text item often contains just one word or fragment. A phrase like `"0°C to 70°C"` may span several adjacent items. The `searchItems` utility handles this — it concatenates consecutive items, finds matches, and returns merged text items with combined bounding boxes:--```typescript-import { LiteParse, searchItems } from "@llamaindex/liteparse";--const parser = new LiteParse({ outputFormat: "json" });-const result = await parser.parse("report.pdf");--for (const page of result.json.pages) {- const matches = searchItems(page.textItems, { phrase: "0°C to 70°C" });- for (const match of matches) {- console.log(`Found "${match.text}" at (${match.x}, ${match.y}) ${match.width}×${match.height}`);- }-}-```--Each returned item has the same shape as a regular text item, with merged coordinates spanning all the items that contributed to the match.--For single-word searches, iterating `textItems` individually (as shown in the library usage section above) is simpler and works fine.+For example, at the default 150 DPI the scale factor is `150 / 72 ~ 2.08`, so a text item at `(72, 200)` maps to pixel `(150, 416)`.## Full example: highlighting citations with sharp@@ -182,11 +179,8 @@ lit screenshot document.pdf -o ./screenshotsFrom there, you (or an agent) can process the resulting JSON and screenshots as needed using any tools available.-## Deprecated: `boundingBoxes`--The `boundingBoxes` array in JSON output is **deprecated** and will be removed in **v2.0**. It is a redundant representation of the same spatial data already available on each text item (`x`, `y`, `width`, `height`). Use `textItems` directly instead — it has the same coordinates plus text content, font metadata, and consistent indexing.-## Tips- Use the same `dpi` value for both `parse()` and `screenshot()`. The default is `150` for both.- Page `width` and `height` in the JSON are in PDF points, matching the coordinate space. Use these if you need to normalize coordinates to percentages.+- For multi-word phrase search, iterate `textItems` and check adjacent items — a phrase may span multiple text items.diff --git a/docs/src/content/docs/liteparse/index.md b/docs/src/content/docs/liteparse/index.mdindex f02edff..6b9a2c1 100644--- a/docs/src/content/docs/liteparse/index.md+++ b/docs/src/content/docs/liteparse/index.md@@ -5,22 +5,25 @@ sidebar:order: 0----LiteParse is an open-source document parsing library that parses text with spatial layout information and bounding boxes. It runs entirely on your machine, with no cloud dependencies, no LLMs, no API keys.+LiteParse is an open-source document parsing library that parses text with spatial layout information and bounding boxes. Written in Rust for speed and reliability, it runs entirely on your machine with no cloud dependencies, no LLMs, and no API keys.LiteParse is designed specifically for use cases that require fast, accurate text parsing: real-time applications, coding agents, and local workflows. It provides a simple CLI and library API for parsing PDFs, Office documents, and images, with built-in OCR support.+<img src="https://github.com/user-attachments/assets/07ba6a82-6bb1-4dea-b0ef-cad7df7d1622" alt="out">+## What can LiteParse do?- **Parse PDFs** with precise spatial layout. Text comes back positioned where it appears on the page- **Extract bounding boxes** for every text line, ready for downstream processing or visualization-- **OCR scanned documents** using built-in Tesseract.js or plug in your own OCR server+- **OCR scanned documents** using built-in Tesseract or plug in your own OCR server- **Parse Office files and images** with support for DOCX, XLSX, PPTX, PNG, JPG, and more via automatic conversion- **Screenshot PDF pages** as high-quality images for LLM-based workflows-- **Use from TypeScript, Python, or the CLI** — whatever fits your stack+- **Use from Node.js/TypeScript, Python, Rust, or the browser (WASM)** — whatever fits your stack## Get started- [Getting started](/liteparse/getting_started/): Install LiteParse and parse your first document.- [Library usage](/liteparse/guides/library-usage/): Use LiteParse from TypeScript or Python code.+- [Browser usage (WASM)](/liteparse/guides/browser-usage/): Run LiteParse in the browser with zero server dependencies.- [CLI reference](/liteparse/cli-reference/): Complete command and option reference.-- [API reference](/liteparse/api/): TypeScript library types and methods.+- [API reference](/liteparse/api/): Detailed API documentation (rust) for all public types and functions. The same types apply across all language bindings.