A fast, helpful, and open-source document parser
add the actual docs
Logan Markewichc72defaea8bb
diff --git a/docs/src/content/docs/liteparse/_meta.yml b/docs/src/content/docs/liteparse/_meta.ymlnew file mode 100644index 0000000..7d25541--- /dev/null+++ b/docs/src/content/docs/liteparse/_meta.yml@@ -0,0 +1,3 @@+label: LiteParse+order: 1+collapsed: falsediff --git a/docs/src/content/docs/liteparse/agent-skill.md b/docs/src/content/docs/liteparse/agent-skill.mdnew file mode 100644index 0000000..5a3d970--- /dev/null+++ b/docs/src/content/docs/liteparse/agent-skill.md@@ -0,0 +1,57 @@+---+title: Agent Skill+description: Add LiteParse as a skill for coding agents like Claude Code, Cursor, and others.+sidebar:+ order: 4+---++LiteParse can be installed as a **coding agent skill** using Vercel's [skills](https://github.com/vercel-labs/skills) utility. This gives your coding agent the ability to parse documents, generate screenshots, and extract text from files — all locally.++## Installation++Add the LiteParse skill to your project:++```bash+npx skills add run-llama/llamaparse-agent-skills --skill liteparse+```++This downloads a skill file that compatible coding agents (Claude Code, Cursor, etc.) will automatically pick up.++Once configured, your agent will be able to call the LiteParse CLI commands directly from its code execution environment. This means you can have your agent parse PDFs, extract text, and generate screenshots on the fly as part of its reasoning process.++## Example prompts++Once the skill is installed, you can ask your coding agent things like:++- "Parse this PDF and extract the text as JSON"+- "Extract text from all the DOCX files in the `./contracts` folder"+- "Screenshot pages 1-5 of this PDF at 300 DPI"+- "Parse this scanned document using the PaddleOCR server on localhost:8828"+- "Get the bounding boxes for all text on page 3"+++## Configuring Defaults++You might want to configure some defaults so that your agent doesn't have to specify them in every prompt. You can create a `liteparse.config.json` file in the root of your project with settings like:++```json+{+ "ocrLanguage": "en",+ "ocrEnabled": true,+ "maxPages": 1000,+ "dpi": 150,+ "outputFormat": "json",+ "preciseBoundingBox": true,+ "preserveVerySmallText": false+}+```++This is especially useful for custom OCR servers. Just add the `ocrServerUrl` to your config:++```json+{+ "ocrServerUrl": "http://localhost:8828/ocr",+ "ocrLanguage": "en",+ "outputFormat": "json"+}+```diff --git a/docs/src/content/docs/liteparse/cli-reference.md b/docs/src/content/docs/liteparse/cli-reference.mdnew file mode 100644index 0000000..824ccd0--- /dev/null+++ b/docs/src/content/docs/liteparse/cli-reference.md@@ -0,0 +1,166 @@+---+title: CLI Reference+description: Complete reference for all LiteParse CLI commands and options.+sidebar:+ order: 5+---++LiteParse provides the `lit` CLI with three commands: `parse`, `batch-parse`, and `screenshot`.++## `lit parse`++Parse a single document.++```+lit parse [options] <file>+```++### Arguments++| Argument | Description |+|----------|-------------|+| `file` | Path to the document file |++### Options++| Option | Description | Default |+|--------|-------------|---------|+| `-o, --output <file>` | Write output to a file instead of stdout | — |+| `--format <format>` | Output format: `json` or `text` | `text` |+| `--ocr-server-url <url>` | HTTP OCR server URL | — (uses Tesseract) |+| `--no-ocr` | Disable OCR entirely | — |+| `--ocr-language <lang>` | OCR language code | `en` |+| `--num-workers <n>` | Pages to OCR in parallel | CPU cores - 1 |+| `--max-pages <n>` | Maximum pages to parse | `10000` |+| `--target-pages <pages>` | Pages to parse (e.g., `"1-5,10"`) | — (all pages) |+| `--dpi <dpi>` | Rendering DPI | `150` |+| `--no-precise-bbox` | Disable precise bounding boxes | — |+| `--preserve-small-text` | Keep very small text | — |+| `--config <file>` | JSON config file path | — |+| `-q, --quiet` | Suppress progress output | — |++### Examples++```bash+# Basic text extraction+lit parse report.pdf++# JSON output with bounding boxes+lit parse report.pdf --format json -o report.json++# Parse pages 1-5 only, no OCR+lit parse report.pdf --target-pages "1-5" --no-ocr++# High-DPI rendering with French OCR+lit parse report.pdf --dpi 300 --ocr-language fra++# Use an external OCR server+lit parse report.pdf --ocr-server-url http://localhost:8828/ocr++# Pipe output to another tool+lit parse report.pdf -q | wc -l+```++---++## `lit batch-parse`++Parse multiple documents in a directory.++```+lit batch-parse [options] <input-dir> <output-dir>+```++### Arguments++| Argument | Description |+|----------|-------------|+| `input-dir` | Directory containing documents to parse |+| `output-dir` | Directory for output files |++### Options++| Option | Description | Default |+|--------|-------------|---------|+| `--format <format>` | Output format: `json` or `text` | `text` |+| `--ocr-server-url <url>` | HTTP OCR server URL | — (uses Tesseract) |+| `--no-ocr` | Disable OCR entirely | — |+| `--ocr-language <lang>` | OCR language code | `en` |+| `--num-workers <n>` | Pages to OCR in parallel | CPU cores - 1 |+| `--max-pages <n>` | Maximum pages per file | `10000` |+| `--dpi <dpi>` | Rendering DPI | `150` |+| `--no-precise-bbox` | Disable precise bounding boxes | — |+| `--recursive` | Search subdirectories | — |+| `--extension <ext>` | Only process this extension (e.g., `".pdf"`) | — (all supported) |+| `--config <file>` | JSON config file path | — |+| `-q, --quiet` | Suppress progress output | — |++### Examples++```bash+# Parse all supported files in a directory+lit batch-parse ./documents ./output++# Recursively parse only PDFs+lit batch-parse ./documents ./output --recursive --extension ".pdf"++# Batch parse with JSON output and no OCR+lit batch-parse ./documents ./output --format json --no-ocr++# Use a config file for consistent settings+lit batch-parse ./documents ./output --config liteparse.config.json+```++---++## `lit screenshot`++Generate page images from a PDF.++```+lit screenshot [options] <file>+```++### Arguments++| Argument | Description |+|----------|-------------|+| `file` | Path to the PDF file |++### Options++| Option | Description | Default |+|--------|-------------|---------|+| `-o, --output-dir <dir>` | Output directory | `./screenshots` |+| `--target-pages <pages>` | Pages to screenshot (e.g., `"1,3,5"` or `"1-5"`) | — (all pages) |+| `--dpi <dpi>` | Rendering DPI | `150` |+| `--format <format>` | Image format: `png` or `jpg` | `png` |+| `--config <file>` | JSON config file path | — |+| `-q, --quiet` | Suppress progress output | — |++### Examples++```bash+# Screenshot all pages+lit screenshot document.pdf -o ./pages++# First 5 pages at high DPI+lit screenshot document.pdf --pages "1-5" --dpi 300 -o ./pages++# JPG format for smaller files+lit screenshot document.pdf --format jpg -o ./pages++# Specific pages only+lit screenshot document.pdf --pages "1,5,10" -o ./pages+```++---++## Global options++These options are available on all commands:++| Option | Description |+|--------|-------------|+| `-h, --help` | Show help for a command |+| `-V, --version` | Show version number |diff --git a/docs/src/content/docs/liteparse/index.md b/docs/src/content/docs/liteparse/index.mdnew file mode 100644index 0000000..91083ef--- /dev/null+++ b/docs/src/content/docs/liteparse/index.md@@ -0,0 +1,50 @@+---+title: Getting Started+description: Install LiteParse and parse your first document in under a minute.+sidebar:+ order: 1+---++LiteParse is an open-source PDF parsing tool that extracts text with spatial layout information and bounding boxes. It runs entirely on your machine, which means no cloud dependencies, no LLMs, no API keys.++## Installation++```bash+# npm (recommended)+npm i -g @llamaindex/liteparse++# brew (macOS/Linux)+brew tap run-llama/liteparse+brew install llamaindex-liteparse+```++## Quick start++Once installed, you can start parsing immediately from the command line:++```bash+lit parse document.pdf++# Save output to a text file+lit parse document.pdf -o output.txt++# Get structured JSON with bounding boxes+lit parse document.pdf --format json -o output.json++# Parse only specific pages+lit parse document.pdf --target-pages "1-5,10,15-20"++# Parse an entire directory of PDFs into an output directory+lit batch-parse ./pdfs ./outputs++# Screenshot the pages of a PDF for LLM agents+lit screenshot document.pdf -o ./screenshots+```++## Next steps++- [OCR Servers](./ocr) — configure Tesseract, EasyOCR, PaddleOCR, or bring your own+- [Multi-format support](./multi-format) — parse DOCX, XLSX, PPTX, images, and more+- [Agent skill](./agent-skill) — use LiteParse as a coding agent skill+- [CLI reference](./cli-reference) — complete command and option reference+- [API reference](./api) — TypeScript library types and methodsdiff --git a/docs/src/content/docs/liteparse/multi-format.md b/docs/src/content/docs/liteparse/multi-format.mdnew file mode 100644index 0000000..c2a66ba--- /dev/null+++ b/docs/src/content/docs/liteparse/multi-format.md@@ -0,0 +1,82 @@+---+title: Multi-Format Support+description: Parse Word documents, spreadsheets, presentations, and images with LiteParse.+sidebar:+ order: 3+---++LiteParse automatically converts non-PDF formats to PDF before parsing. This lets you use the same parsing pipeline for Office documents, images, and more.++## Supported formats++### Office documents (via LibreOffice)++| Category | Extensions |+|----------|-----------|+| Word | `.doc`, `.docx`, `.docm`, `.odt`, `.rtf` |+| PowerPoint | `.ppt`, `.pptx`, `.pptm`, `.odp` |+| Spreadsheets | `.xls`, `.xlsx`, `.xlsm`, `.ods`, `.csv`, `.tsv` |++### Images (via ImageMagick)++`.jpg`, `.jpeg`, `.png`, `.gif`, `.bmp`, `.tiff`, `.webp`, `.svg`++Images are converted to PDF and then parsed with OCR to extract text.++## Installing dependencies++Format conversion uses standard system tools. Install the ones you need:++### LibreOffice (for Office documents)++```bash+# macOS+brew install --cask libreoffice++# Ubuntu/Debian+apt-get install libreoffice++# Windows+choco install libreoffice-fresh+```++> On Windows, you may need to add the LibreOffice CLI directory (typically `C:\Program Files\LibreOffice\program`) to your PATH and restart.++### ImageMagick (for images)++```bash+# macOS+brew install imagemagick++# Ubuntu/Debian+apt-get install imagemagick++# Windows+choco install imagemagick.app+```++## Usage++Once the dependencies are installed, just pass any supported file to `lit parse`:++```bash+lit parse report.docx+lit parse slides.pptx --format json+lit parse spreadsheet.xlsx -o output.txt+lit parse scan.png+```++Batch mode also handles mixed formats:++```bash+lit batch-parse ./documents ./output --recursive+```++## How it works++1. LiteParse detects the file extension+2. If it's not a PDF, it converts to PDF using the appropriate tool (LibreOffice or ImageMagick)+3. The resulting PDF is parsed normally+4. Temporary conversion files are cleaned up automatically++If the required conversion tool isn't installed, LiteParse will return an error explaining which dependency is needed.diff --git a/docs/src/content/docs/liteparse/ocr.md b/docs/src/content/docs/liteparse/ocr.mdnew file mode 100644index 0000000..3fced8b--- /dev/null+++ b/docs/src/content/docs/liteparse/ocr.md@@ -0,0 +1,145 @@+---+title: OCR Configuration+description: Configure OCR in LiteParse — built-in Tesseract, or bring your own via HTTP servers.+sidebar:+ order: 2+---++LiteParse uses OCR selectively — only on embedded images or pages where native text extraction didn't find text. This keeps parsing fast while still capturing text from scanned pages and embedded images.++## Built-in Tesseract (default)++Tesseract.js is bundled with LiteParse. The only setup is the automatic download of the Tesseract model files on first use. Just run:++```bash+lit parse document.pdf+```++If bundling LiteParse into a docker container or server environment, you might want to pre-download the Tesseract files to avoid network calls at runtime with the above command or similar.++### Language support++Specify the OCR language for better accuracy on non-English documents:++```bash+lit parse document.pdf --ocr-language fra # French+lit parse document.pdf --ocr-language deu # German+lit parse document.pdf --ocr-language jpn # Japanese+```++Tesseract uses [ISO 639-3](https://tesseract-ocr.github.io/tessdoc/Data-Files-in-different-versions.html) language codes (`eng`, `fra`, `deu`, etc.).++### Disabling OCR++If you don't need OCR (pure native-text PDFs, or you don't care about images), disable it for faster parsing:++```bash+lit parse document.pdf --no-ocr+```++## HTTP OCR servers++For higher accuracy or GPU-accelerated OCR, you can point LiteParse at an HTTP OCR server. LiteParse ships with ready-to-use examples for popular OCR engines.++### EasyOCR++```bash+# Start the EasyOCR server (requires Python)+git clone https://github.com/run-llama/liteparse.git+cd liteparse/ocr/easyocr+pip install -r requirements.txt+python server.py++# Parse with EasyOCR in another terminal+lit parse document.pdf --ocr-server-url http://localhost:8828/ocr+```++### PaddleOCR++```bash+# Start the PaddleOCR server (requires Python)+git clone https://github.com/run-llama/liteparse.git+cd liteparse/ocr/paddleocr+pip install -r requirements.txt+python server.py++# Parse with PaddleOCR in another terminal+lit parse document.pdf --ocr-server-url http://localhost:8828/ocr+```++### Parallel OCR workers++LiteParse OCRs multiple pages in parallel. By default, it uses one fewer worker than your CPU core count. Override this with:++```bash+lit parse document.pdf --num-workers 8+```++This is useful if you need to slow down OCR requests to an external server or if your OCR engine is GPU-accelerated and can handle more concurrency.++## Custom OCR servers++You can integrate any OCR engine by implementing the LiteParse OCR API. Your server needs a single endpoint:++```+POST /ocr+Content-Type: multipart/form-data+```++**Request fields:**++| Field | Type | Required | Description |+|-------|------|----------|-------------|+| `file` | binary | Yes | Image file (PNG, JPG, etc.) |+| `language` | string | No | ISO 639-1 language code (default: `en`) |++**Response format:**++```json+{+ "results": [+ {+ "text": "recognized text",+ "bbox": [x1, y1, x2, y2],+ "confidence": 0.95+ }+ ]+}+```++Each result contains:++| Field | Type | Description |+|-------|------|-------------|+| `text` | string | Recognized text |+| `bbox` | `[x1, y1, x2, y2]` | Bounding box in pixels. Origin is top-left, x goes right, y goes down |+| `confidence` | number | Score from 0.0 to 1.0 |++### Testing your server++```bash+# Quick test with curl+curl -X POST http://localhost:8080/ocr \+ -F "[email protected]" \+ -F "language=en" | jq .++# Use with LiteParse+lit parse document.pdf --ocr-server-url http://localhost:8080/ocr+```++### Common Gotchas++- Return `{"results": []}` if no text is detected+- Bounding boxes must be axis-aligned (`[x1, y1, x2, y2]` where top-left to bottom-right)+- If your engine returns rotated boxes, convert to axis-aligned by taking min/max coordinates+- If your engine doesn't provide confidence scores, return `1.0`+- Results should be in reading order (top-to-bottom, left-to-right)+- Cache OCR models in memory rather than reloading per request++### A note on OCR approaches++These days, its common to apply the term "OCR" to both traditional approaches and newer LLM-based document understanding models.++The LiteParse OCR API is designed specifically for approaches that return text with bounding boxes.++If you are trying to integrate a method that doesn't return bounding boxes, you will have to generate dummy bounding boxes.