Skip to content

A fast, helpful, and open-source document parser

wip: tentative optimizations

Clelia (Astra) Bertellie562cea6eeba
diff --git a/crates/copilot-session-edfc05f0-b306-47a0-b299-e68c0df2e82a.md b/crates/copilot-session-edfc05f0-b306-47a0-b299-e68c0df2e82a.md
new file mode 100644
index 0000000..e5fea6a
--- /dev/null
+++ b/crates/copilot-session-edfc05f0-b306-47a0-b299-e68c0df2e82a.md
@@ -0,0 +1,1104 @@
+# πŸ€– Copilot CLI Session
+
+> [!NOTE]
+> - **Session ID:** `edfc05f0-b306-47a0-b299-e68c0df2e82a`
+> - **Started:** 5/19/2026, 11:17:00 AM
+> - **Duration:** 7m 45s
+> - **Exported:** 5/19/2026, 11:24:46 AM
+
+---
+
+<sub>⏱️ 0s</sub>
+
+### ℹ️ Info
+
+πŸ’‘ No copilot instructions found. Run /init to generate a copilot-instructions.md file for this project.
+
+---
+
+<sub>⏱️ 1s</sub>
+
+### ℹ️ Info
+
+Environment loaded: 2 custom instructions, 2 skills, 1 MCP server
+
+---
+
+<sub>⏱️ 38s</sub>
+
+### πŸ‘€ User
+
+Read @liteparse-python/OPTIMIZATION_PLAN.md and let's discuss if the optimizations are feasible in @liteparse-python/src/lib.rs
+
+---
+
+<sub>⏱️ 41s</sub>
+
+### πŸ’­ Reasoning
+
+*I'm going to read through both files to understand what we're working with.*
+
+---
+
+<sub>⏱️ 42s</sub>
+
+### βœ… `view`
+
+**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/OPTIMIZATION_PLAN.md.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/OPTIMIZATION_PLAN.md
+
+<details>
+<summary>31 lines</summary>
+
+```
+1. Looking at your Scalene profile, here are the bottlenecks I can identify:
+2.
+3. ## Key Observations
+4.
+5. **1. I/O is the dominant cost (~0–40ms block)**
+6. The timeline shows a solid blue I/O bar consuming roughly the first 40ms of your 170ms total run. This suggests the file read itself is a significant chunk β€” likely `__fread` / `_srefill` / `_sread` / `_aread` in the call stack. If you're doing unbuffered or small-chunk reads, that's low-hanging fruit.
+7.
+8. **2. GIL contention is visible**
+9. The GIL row in the timeline shows activity, which is notable for a Rust binding. This usually means your Rust extension is calling back into Python too frequently (e.g. building Python objects incrementally rather than bulk-returning), or you're not releasing the GIL during the heavy native work with `py.allow_threads(|| ...)`.
+10.
+11. **3. `_Py_HandlePending` / `PyErr_CheckSignalsState` in the hot path**
+12. These appear mid-stack and indicate Python's signal-checking mechanism is being hit repeatedly. This happens when Rust calls back into the Python C API in a tight loop β€” each re-entry triggers signal checks. Batching your Python object construction would eliminate most of these.
+13.
+14. **4. Thread/semaphore wait (`_PySemaphore_Wait`, `pthread_cond_wait`)**
+15. The right-hand call stack shows a thread sitting in `_PySemaphore_Wait` β†’ `pthread_cond_wait`. This is a waiting thread, likely a GIL waiter or a thread pool worker stalled. If you're spawning worker threads in Rust, they may be contending more than helping for a single-file parse.
+16.
+17. **5. `infinite_lru_cache_wrapper`**
+18. This appears in the Python call stack. If you have a Python-side LRU cache wrapping something in the hot path, that's adding overhead β€” especially if it's being hit per-page or per-token.
+19.
+20. ---
+21.
+22. ## Suggested Fixes
+23.
+24. - **Release the GIL** during your Rust parsing work: wrap the core parse loop in `py.allow_threads(|| { ... })` in PyO3. This is the single highest-impact change if you aren't already doing it.
+25. - **Batch Python object construction** β€” parse everything in Rust, return a single large structure (Vec of structs), then convert to Python objects once at the end rather than per-element.
+26. - **Buffered reads** β€” if you're opening the file in Rust, make sure you're using `BufReader` rather than raw file reads, which maps to the `_srefill`/`_sread` pattern you're seeing.
+27. - **Profile the LRU cache** β€” check what `infinite_lru_cache_wrapper` is wrapping on the Python side and whether it can be moved into Rust or eliminated.
+28. - **Avoid thread spawning per call** β€” if Rayon or a thread pool is being initialized per `parse()` invocation, use a lazy static pool instead.
+29.
+30. The biggest win is almost certainly the GIL release + batched return pattern. pymupdf and pdftotext win here largely because their Python bindings do exactly that β€” all work happens natively, Python only touches the final result.
+31.
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 42s</sub>
+
+### βœ… `view`
+
+**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs
+
+<details>
+<summary>366 lines</summary>
+
+```
+1. use pyo3::prelude::*;
+2. use pyo3::types::PyBytes;
+3.
+4. use liteparse::config::{LiteParseConfig, OutputFormat};
+5. use liteparse::types::PdfInput;
+6.
+7. mod cli;
+8.
+9. // ---------------------------------------------------------------------------
+10. // Python type wrappers
+11. // ---------------------------------------------------------------------------
+12.
+13. #[pyclass(frozen, from_py_object)]
+14. #[derive(Clone)]
+15. struct PyTextItem {
+16. #[pyo3(get)]
+17. text: String,
+18. #[pyo3(get)]
+19. x: f64,
+20. #[pyo3(get)]
+21. y: f64,
+22. #[pyo3(get)]
+23. width: f64,
+24. #[pyo3(get)]
+25. height: f64,
+26. #[pyo3(get)]
+27. font_name: Option<String>,
+28. #[pyo3(get)]
+29. font_size: Option<f64>,
+30. #[pyo3(get)]
+31. confidence: Option<f64>,
+32. }
+33.
+34. #[pymethods]
+35. impl PyTextItem {
+36. fn __repr__(&self) -> String {
+37. format!(
+38. "TextItem(text={:?}, x={}, y={}, width={}, height={})",
+39. self.text, self.x, self.y, self.width, self.height
+40. )
+41. }
+42. }
+43.
+44. impl PyTextItem {
+45. fn from_rust(item: &liteparse::types::TextItem) -> Self {
+46. Self {
+47. text: item.text.clone(),
+48. x: item.x as f64,
+49. y: item.y as f64,
+50. width: item.width as f64,
+51. height: item.height as f64,
+52. font_name: item.font_name.clone(),
+53. font_size: item.font_size.map(|v| v as f64),
+54. confidence: item.confidence.map(|v| v as f64).or(Some(1.0)),
+55. }
+56. }
+57. }
+58.
+59. #[pyclass(frozen, from_py_object)]
+60. #[derive(Clone)]
+61. struct PyParsedPage {
+62. #[pyo3(get)]
+63. page_num: u32,
+64. #[pyo3(get)]
+65. width: f64,
+66. #[pyo3(get)]
+67. height: f64,
+68. #[pyo3(get)]
+69. text: String,
+70. #[pyo3(get)]
+71. text_items: Vec<PyTextItem>,
+72. }
+73.
+74. #[pymethods]
+75. impl PyParsedPage {
+76. fn __repr__(&self) -> String {
+77. format!(
+78. "ParsedPage(page_num={}, width={}, height={}, text_items={})",
+79. self.page_num,
+80. self.width,
+81. self.height,
+82. self.text_items.len()
+83. )
+84. }
+85. }
+86.
+87. impl PyParsedPage {
+88. fn from_rust(page: &liteparse::types::ParsedPage) -> Self {
+89. Self {
+90. page_num: page.page_number as u32,
+91. width: page.page_width as f64,
+92. height: page.page_height as f64,
+93. text: page.text.clone(),
+94. text_items: page.text_items.iter().map(PyTextItem::from_rust).collect(),
+95. }
+96. }
+97. }
+98.
+99. #[pyclass(frozen, from_py_object)]
+100. #[derive(Clone)]
+101. struct PyParseResult {
+102. #[pyo3(get)]
+103. pages: Vec<PyParsedPage>,
+104. #[pyo3(get)]
+105. text: String,
+106. }
+107.
+108. #[pymethods]
+109. impl PyParseResult {
+110. #[getter]
+111. fn num_pages(&self) -> usize {
+112. self.pages.len()
+113. }
+114.
+115. fn get_page(&self, page_num: u32) -> Option<PyParsedPage> {
+116. self.pages.iter().find(|p| p.page_num == page_num).cloned()
+117. }
+118.
+119. fn __repr__(&self) -> String {
+120. format!(
+121. "ParseResult(pages={}, text_len={})",
+122. self.pages.len(),
+123. self.text.len()
+124. )
+125. }
+126. }
+127.
+128. impl PyParseResult {
+129. fn from_rust(result: &liteparse::parser::ParseResult) -> Self {
+130. Self {
+131. pages: result.pages.iter().map(PyParsedPage::from_rust).collect(),
+132. text: result.text.clone(),
+133. }
+134. }
+135. }
+136.
+137. #[pyclass(frozen, from_py_object)]
+138. #[derive(Clone)]
+139. struct PyScreenshotResult {
+140. #[pyo3(get)]
+141. page_num: u32,
+142. #[pyo3(get)]
+143. width: u32,
+144. #[pyo3(get)]
+145. height: u32,
+146. image_buffer: Vec<u8>,
+147. }
+148.
+149. #[pymethods]
+150. impl PyScreenshotResult {
+151. #[getter]
+152. fn image_bytes<'py>(&self, py: Python<'py>) -> Bound<'py, PyBytes> {
+153. PyBytes::new(py, &self.image_buffer)
+154. }
+155.
+156. fn __repr__(&self) -> String {
+157. format!(
+158. "ScreenshotResult(page_num={}, width={}, height={})",
+159. self.page_num, self.width, self.height
+160. )
+161. }
+162. }
+163.
+164. // ---------------------------------------------------------------------------
+165. // Main LiteParse class
+166. // ---------------------------------------------------------------------------
+167.
+168. #[pyclass]
+169. struct LiteParse {
+170. inner: liteparse::parser::LiteParse,
+171. config: LiteParseConfig,
+172. runtime: tokio::runtime::Runtime,
+173. }
+174.
+175. #[pymethods]
+176. impl LiteParse {
+177. #[new]
+178. #[pyo3(signature = (
+179. *,
+180. ocr_language = None,
+181. ocr_enabled = None,
+182. ocr_server_url = None,
+183. tessdata_path = None,
+184. max_pages = None,
+185. target_pages = None,
+186. dpi = None,
+187. output_format = None,
+188. preserve_very_small_text = None,
+189. password = None,
+190. quiet = None,
+191. num_workers = None,
+192. ))]
+193. fn new(
+194. ocr_language: Option<String>,
+195. ocr_enabled: Option<bool>,
+196. ocr_server_url: Option<String>,
+197. tessdata_path: Option<String>,
+198. max_pages: Option<usize>,
+199. target_pages: Option<String>,
+200. dpi: Option<f32>,
+201. output_format: Option<String>,
+202. preserve_very_small_text: Option<bool>,
+203. password: Option<String>,
+204. quiet: Option<bool>,
+205. num_workers: Option<usize>,
+206. ) -> PyResult<Self> {
+207. let mut cfg = LiteParseConfig::default();
+208. if let Some(v) = ocr_language {
+209. cfg.ocr_language = v;
+210. }
+211. if let Some(v) = ocr_enabled {
+212. cfg.ocr_enabled = v;
+213. }
+214. if let Some(v) = ocr_server_url {
+215. cfg.ocr_server_url = Some(v);
+216. }
+217. if let Some(v) = tessdata_path {
+218. cfg.tessdata_path = Some(v);
+219. }
+220. if let Some(v) = max_pages {
+221. cfg.max_pages = v;
+222. }
+223. if let Some(v) = target_pages {
+224. cfg.target_pages = Some(v);
+225. }
+226. if let Some(v) = dpi {
+227. cfg.dpi = v;
+228. }
+229. if let Some(v) = output_format {
+230. cfg.output_format = match v.as_str() {
+231. "text" => OutputFormat::Text,
+232. _ => OutputFormat::Json,
+233. };
+234. }
+235. if let Some(v) = preserve_very_small_text {
+236. cfg.preserve_very_small_text = v;
+237. }
+238. if let Some(v) = password {
+239. cfg.password = Some(v);
+240. }
+241. if let Some(v) = quiet {
+242. cfg.quiet = v;
+243. }
+244. if let Some(v) = num_workers {
+245. cfg.num_workers = v;
+246. }
+247.
+248. let inner = liteparse::parser::LiteParse::new(cfg.clone());
+249. let runtime = tokio::runtime::Runtime::new()
+250. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+251.
+252. Ok(Self {
+253. inner,
+254. config: cfg,
+255. runtime,
+256. })
+257. }
+258.
+259. /// Parse a document from a file path.
+260. fn parse(&self, input: String) -> PyResult<PyParseResult> {
+261. let pdf_input = PdfInput::Path(input);
+262. let result = self
+263. .runtime
+264. .block_on(self.inner.parse_input(pdf_input))
+265. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+266. Ok(PyParseResult::from_rust(&result))
+267. }
+268.
+269. /// Parse a document from raw bytes.
+270. fn parse_bytes(&self, data: Vec<u8>) -> PyResult<PyParseResult> {
+271. let pdf_input = PdfInput::Bytes(data);
+272. let result = self
+273. .runtime
+274. .block_on(self.inner.parse_input(pdf_input))
+275. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+276. Ok(PyParseResult::from_rust(&result))
+277. }
+278.
+279. /// Take screenshots of document pages. Returns a list of ScreenshotResult.
+280. #[pyo3(signature = (input, page_numbers = None))]
+281. fn screenshot(
+282. &self,
+283. input: String,
+284. page_numbers: Option<Vec<u32>>,
+285. ) -> PyResult<Vec<PyScreenshotResult>> {
+286. let dpi = self.config.dpi;
+287. let lib = pdfium::Library::init();
+288. let document = lib
+289. .load_document(&input, self.config.password.as_deref())
+290. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+291. let page_count = document.page_count() as u32;
+292.
+293. let pages: Vec<u32> = match page_numbers {
+294. Some(nums) => nums,
+295. None => (1..=page_count).collect(),
+296. };
+297.
+298. let mut results = Vec::with_capacity(pages.len());
+299. for page_num in pages {
+300. if page_num < 1 || page_num > page_count {
+301. return Err(PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(
+302. "page {page_num} out of range (document has {page_count} pages)"
+303. )));
+304. }
+305. let page = document
+306. .page((page_num - 1) as i32)
+307. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+308. let bitmap = page
+309. .render(dpi)
+310. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+311.
+312. let width = bitmap.width() as u32;
+313. let height = bitmap.height() as u32;
+314. let rgba = bitmap.to_rgba();
+315.
+316. let mut png_buf: Vec<u8> = Vec::new();
+317. let encoder = image::codecs::png::PngEncoder::new(&mut png_buf);
+318. use image::ImageEncoder;
+319. encoder
+320. .write_image(&rgba, width, height, image::ColorType::Rgba8.into())
+321. .map_err(|e| {
+322. PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(format!(
+323. "PNG encode failed: {e}"
+324. ))
+325. })?;
+326.
+327. results.push(PyScreenshotResult {
+328. page_num,
+329. width,
+330. height,
+331. image_buffer: png_buf,
+332. });
+333. }
+334.
+335. Ok(results)
+336. }
+337.
+338. fn __repr__(&self) -> String {
+339. format!(
+340. "LiteParse(ocr_enabled={}, dpi={}, max_pages={})",
+341. self.config.ocr_enabled, self.config.dpi, self.config.max_pages
+342. )
+343. }
+344. }
+345.
+346. // ---------------------------------------------------------------------------
+347. // Module
+348. // ---------------------------------------------------------------------------
+349.
+350. /// Run the `lit` CLI with the given arguments.
+351. #[pyfunction]
+352. fn run_cli(args: Vec<String>) -> PyResult<()> {
+353. cli::run_cli(args).map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))
+354. }
+355.
+356. #[pymodule]
+357. fn _liteparse(m: &Bound<'_, PyModule>) -> PyResult<()> {
+358. m.add_class::<LiteParse>()?;
+359. m.add_class::<PyParseResult>()?;
+360. m.add_class::<PyParsedPage>()?;
+361. m.add_class::<PyTextItem>()?;
+362. m.add_class::<PyScreenshotResult>()?;
+363. m.add_function(wrap_pyfunction!(run_cli, m)?)?;
+364. Ok(())
+365. }
+366.
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 46s</sub>
+
+### πŸ’­ Reasoning
+
+*I'm checking each suggestion against the actual code to see if they're accurate.*
+
+---
+
+<sub>⏱️ 46s</sub>
+
+### πŸ’¬ Copilot
+
+Here's my assessment of the OPTIMIZATION_PLAN suggestions against the current `lib.rs`:
+
+## Feasibility analysis
+
+**1. Release the GIL with `py.allow_threads(...)` β€” βœ… Highly feasible, biggest win**
+
+Currently `parse`, `parse_bytes`, and `screenshot` all run heavy native work (including `runtime.block_on(...)` on a Tokio runtime) while holding the GIL. None of these methods accept `py: Python\<'_\>` or wrap work in `allow_threads`. This is genuinely low-hanging fruit:
+
+```rust
+fn parse(&self, py: Python\<'_\>, input: String) -\> PyResult\<PyParseResult\> {
+ let pdf_input = PdfInput::Path(input);
+ let result = py.allow_threads(|| {
+ self.runtime.block_on(self.inner.parse_input(pdf_input))
+ }).map_err(...)?;
+ Ok(PyParseResult::from_rust(&result))
+}
+```
+
+This alone should eliminate the GIL contention, `_Py_HandlePending`, and `_PySemaphore_Wait` symptoms β€” any Python thread waiting on the GIL is blocked while we sit in `block_on` for ~100ms+.
+
+**2. Batch Python object construction β€” ⚠️ Partially applicable**
+
+The plan describes "Rust calling back into Python in a tight loop." Looking at `from_rust`, we don't actually do that β€” `PyTextItem`/`PyParsedPage` are plain Rust structs (`#[pyclass(frozen)]`) built in Rust, with no Python API calls per element. The conversion happens after parsing is complete. So this concern is largely already addressed.
+
+However, `PyParseResult::from_rust` clones every `TextItem`, `String`, and `Vec` from the underlying `ParseResult`. We could **move** instead of clone by taking ownership of `result`:
+
+```rust
+fn from_rust(result: liteparse::parser::ParseResult) -\> Self { ... }
+```
+
+Modest win, but worth doing.
+
+**3. Buffered reads (`BufReader`) β€” ❓ Out of scope here**
+
+`lib.rs` doesn't do file I/O itself β€” it hands a `PdfInput::Path` to `liteparse::parser`. The `_fread`/`_srefill` cost is inside the core crate (or pdfium). Fixing it requires changes in the `liteparse` crate, not the Python binding. Worth investigating separately.
+
+**4. `infinite_lru_cache_wrapper` β€” ❌ Not in this file**
+
+That's Python-side. Likely in `liteparse-python`'s Python wrapper code (if any) or in user code. Not addressable in `lib.rs`.
+
+**5. Avoid per-call thread pool / runtime spawning β€” βœ… Already done, but could be better**
+
+Good news: the Tokio `Runtime` is created once in `LiteParse::new()` and reused. However, `screenshot` calls `pdfium::Library::init()` **per invocation**, which may have init cost. Worth checking if pdfium can be initialized once and stored on `self` (depends on pdfium's API and `Send`/`Sync` guarantees).
+
+## Recommended order of changes
+
+1. **Wrap all three methods in `py.allow_threads`** β€” highest ROI, ~5-line change. Frees the GIL for ~95% of wall-clock time.
+2. **Move (not clone) in `from_rust` conversions** β€” small but easy win once we restructure ownership.
+3. **Hoist `pdfium::Library::init()` to `LiteParse::new`** β€” if API permits.
+4. **Investigate `BufReader` / I/O** in the core `liteparse` crate (separate task).
+
+Want me to implement #1 (and optionally #2, #3)?
+
+---
+
+<sub>⏱️ 2m 14s</sub>
+
+### πŸ‘€ User
+
+let's go with 1-3
+
+---
+
+<sub>⏱️ 2m 18s</sub>
+
+### πŸ’­ Reasoning
+
+*I'm checking the pdfium API documentation to see whether the Library object can be stored as an instance variable.*
+
+---
+
+<sub>⏱️ 2m 21s</sub>
+
+### βœ… `grep`
+
+**pub (fn|struct) (init|Library|load_document)**
+
+"pub (fn|struct) (init|Library|load_document)" (pdfium)
+
+```
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:9:pub struct Library {
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:14: pub fn init() -> Library {
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:21: pub fn load_document(
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:45: pub fn load_document_from_bytes(
+```
+
+
+---
+
+<sub>⏱️ 2m 21s</sub>
+
+### βœ… `grep`
+
+**struct Library|impl Library|unsafe impl (Send|Sync) for Library**
+
+"struct Library|impl Library|unsafe impl (Send|Sync) for Library" (pdfium)
+
+```
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:9:pub struct Library {
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:13:impl Library {
+```
+
+
+---
+
+<sub>⏱️ 2m 24s</sub>
+
+### βœ… `view`
+
+**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs (lines 1-60)
+
+<details>
+<summary>60 lines</summary>
+
+```
+1. use std::ffi::CString;
+2. use std::sync::Once;
+3.
+4. use crate::document::Document;
+5. use crate::error::PdfiumError;
+6.
+7. static INIT: Once = Once::new();
+8.
+9. pub struct Library {
+10. _private: (),
+11. }
+12.
+13. impl Library {
+14. pub fn init() -> Library {
+15. INIT.call_once(|| {
+16. unsafe { pdfium_sys::FPDF_InitLibrary() };
+17. });
+18. Library { _private: () }
+19. }
+20.
+21. pub fn load_document(
+22. &self,
+23. path: &str,
+24. password: Option<&str>,
+25. ) -> Result<Document, PdfiumError> {
+26. let c_path = CString::new(path).map_err(|_| PdfiumError::FileNotFound)?;
+27. let c_password = password
+28. .map(|p| CString::new(p).map_err(|_| PdfiumError::OperationFailed))
+29. .transpose()?;
+30.
+31. let handle = unsafe {
+32. pdfium_sys::FPDF_LoadDocument(
+33. c_path.as_ptr(),
+34. c_password.as_ref().map_or(std::ptr::null(), |p| p.as_ptr()),
+35. )
+36. };
+37.
+38. if handle.is_null() {
+39. return Err(PdfiumError::from_last_error());
+40. }
+41.
+42. Ok(Document { handle })
+43. }
+44.
+45. pub fn load_document_from_bytes(
+46. &self,
+47. data: &[u8],
+48. password: Option<&str>,
+49. ) -> Result<Document, PdfiumError> {
+50. let c_password = password
+51. .map(|p| CString::new(p).map_err(|_| PdfiumError::OperationFailed))
+52. .transpose()?;
+53.
+54. let handle = unsafe {
+55. pdfium_sys::FPDF_LoadMemDocument(
+56. data.as_ptr() as *const std::ffi::c_void,
+57. data.len() as i32,
+58. c_password.as_ref().map_or(std::ptr::null(), |p| p.as_ptr()),
+59. )
+60. };
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 2m 25s</sub>
+
+### ℹ️ Info
+
+All permissions are now enabled. Tool, path, and URL requests will be automatically approved.
+
+---
+
+<sub>⏱️ 2m 29s</sub>
+
+### πŸ’­ Reasoning
+
+*`Library` being a zero-sized type wrapped in `Once` means reinitializing it is cheap since there's no real allocation happeningβ€”the actual overhead comes from loading the document itself. Moving it to `self` would be cleaner and eliminate any per-call costs. The tricky part is that while `Library` should automatically be `Send + Sync` as a ZST containing just `()`, I need to verify it actually compiles with pyclass's `Send` requirement.
+
+Now I'm checking the shape of `ParseResult` to see if I can optimize the move semantics there.*
+
+---
+
+<sub>⏱️ 2m 31s</sub>
+
+### βœ… `grep`
+
+**pub struct (ParseResult|ParsedPage|TextItem)**
+
+"pub struct (ParseResult|ParsedPage|TextItem)" (liteparse/src)
+
+```
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/parser.rs:16:pub struct ParseResult {
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/output/json.rs:29:pub struct ParseResultJson {
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs:28:pub struct TextItem {
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs:81:pub struct ParsedPage {
+```
+
+
+---
+
+<sub>⏱️ 2m 35s</sub>
+
+### βœ… `view`
+
+**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/parser.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/parser.rs (lines 16-30)
+
+<details>
+<summary>15 lines</summary>
+
+```
+16. pub struct ParseResult {
+17. /// Parsed pages with projected text layout.
+18. pub pages: Vec<ParsedPage>,
+19. /// Full document text, concatenated from all pages.
+20. pub text: String,
+21. }
+22.
+23. /// Main LiteParse orchestrator.
+24. pub struct LiteParse {
+25. config: LiteParseConfig,
+26. /// Optional caller-provided OCR engine. When set, this overrides the
+27. /// built-in selection logic (HTTP OCR / Tesseract). This is the primary
+28. /// mechanism for plugging an OCR engine in environments without the
+29. /// built-ins (e.g. WASM, where the JS side supplies a callback engine).
+30. ocr_engine_override: Option<std::sync::Arc<dyn OcrEngine>>,
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 2m 35s</sub>
+
+### βœ… `view`
+
+**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs (lines 28-105)
+
+<details>
+<summary>78 lines</summary>
+
+```
+28. pub struct TextItem {
+29. pub text: String,
+30. /// Viewport-space coordinates (top-left origin, 72 DPI).
+31. pub x: f32,
+32. pub y: f32,
+33. pub width: f32,
+34. pub height: f32,
+35. /// Rotation in degrees (counter-clockwise, adjusted for page rotation).
+36. pub rotation: f32,
+37. pub font_name: Option<String>,
+38. pub font_size: Option<f32>,
+39. /// Font size * scale_y from the text matrix β€” accounts for CTM scaling.
+40. #[serde(skip_serializing_if = "Option::is_none")]
+41. pub font_height: Option<f32>,
+42. #[serde(skip_serializing_if = "Option::is_none")]
+43. pub font_ascent: Option<f32>,
+44. #[serde(skip_serializing_if = "Option::is_none")]
+45. pub font_descent: Option<f32>,
+46. #[serde(skip_serializing_if = "Option::is_none")]
+47. pub font_weight: Option<i32>,
+48. #[serde(skip_serializing_if = "Option::is_none")]
+49. pub font_flags: Option<i32>,
+50. /// Sum of glyph widths (using charcode-based lookup when possible).
+51. #[serde(skip_serializing_if = "Option::is_none")]
+52. pub text_width: Option<f32>,
+53. /// Whether the font has buggy encoding (private-use codepoints, TT subset, etc.)
+54. #[serde(skip_serializing_if = "std::ops::Not::not")]
+55. pub font_is_buggy: bool,
+56. /// Marked content ID from the PDF structure tree.
+57. #[serde(skip_serializing_if = "Option::is_none")]
+58. pub mcid: Option<i32>,
+59. /// Fill color as ARGB hex string (e.g. "ff000000").
+60. #[serde(skip_serializing_if = "Option::is_none")]
+61. pub fill_color: Option<String>,
+62. /// Stroke color as ARGB hex string.
+63. #[serde(skip_serializing_if = "Option::is_none")]
+64. pub stroke_color: Option<String>,
+65. /// OCR confidence score (0.0–1.0). None for native PDF text.
+66. #[serde(skip_serializing_if = "Option::is_none")]
+67. pub confidence: Option<f32>,
+68. }
+69.
+70. /// Represents a single page in a PDF document, including its dimensions and extracted text items.
+71. #[derive(Debug, Serialize)]
+72. pub struct Page {
+73. pub page_number: usize,
+74. pub page_width: f32,
+75. pub page_height: f32,
+76. pub text_items: Vec<TextItem>,
+77. }
+78.
+79. /// Represents a fully parsed page with projected text layout.
+80. #[derive(Debug, Serialize)]
+81. pub struct ParsedPage {
+82. pub page_number: usize,
+83. pub page_width: f32,
+84. pub page_height: f32,
+85. pub text: String,
+86. pub text_items: Vec<TextItem>,
+87. }
+88.
+89. #[derive(Debug, Serialize)]
+90. pub enum Snap {
+91. Left,
+92. Right,
+93. Center,
+94. }
+95.
+96. #[derive(Debug, Serialize)]
+97. pub enum Anchor {
+98. Left,
+99. Right,
+100. Center,
+101. }
+102.
+103. /// Represents a Projected piece of text, responsible for keeping track of projection related data
+104. #[derive(Debug, Serialize)]
+105. pub struct ProjectedTextItem {
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 2m 39s</sub>
+
+### πŸ’­ Reasoning
+
+*Now I'm ready to implement the fixes: wrapping the main functions in thread-safe blocks, refactoring the ownership model to move fields directly, and initializing the PDF library at the struct level rather than per-operation. Since the library is a zero-sized type that's already thread-safe, I just need to store it in the struct and ensure the Python class can handle it.
+
+Writing the implementation...*
+
+---
+
+<sub>⏱️ 3m 17s</sub>
+
+### βœ… `edit`
+
+**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs
+
+```
+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.
+```
+
+
+---
+
+<sub>⏱️ 3m 17s</sub>
+
+### βœ… `edit`
+
+**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs
+
+```
+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.
+```
+
+
+---
+
+<sub>⏱️ 3m 17s</sub>
+
+### βœ… `edit`
+
+**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs
+
+```
+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.
+```
+
+
+---
+
+<sub>⏱️ 3m 17s</sub>
+
+### βœ… `edit`
+
+**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs
+
+```
+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.
+```
+
+
+---
+
+<sub>⏱️ 3m 17s</sub>
+
+### βœ… `edit`
+
+**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**
+
+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs
+
+```
+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.
+```
+
+
+---
+
+<sub>⏱️ 3m 20s</sub>
+
+### πŸ’¬ Copilot
+
+Now let's verify it builds:
+
+---
+
+<sub>⏱️ 3m 26s</sub>
+
+### βœ… `bash`
+
+**cargo check liteparse-python**
+
+$ cd /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python && cargo check 2>&1 | tail -60
+
+<details>
+<summary>35 lines</summary>
+
+```
+Compiling pyo3-build-config v0.28.3
+ Checking liteparse-pdfium-sys v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium-sys)
+ Compiling liteparse v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse)
+ Compiling liteparse-python v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python)
+ Checking liteparse-pdfium v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium)
+ Compiling pyo3-macros-backend v0.28.3
+ Compiling pyo3-ffi v0.28.3
+ Compiling pyo3 v0.28.3
+ Compiling pyo3-macros v0.28.3
+error[E0599]: no method named `allow_threads` found for struct `pyo3::Python<'py>` in the current scope
+ --> crates/liteparse-python/src/lib.rs:274:14
+ |
+273 | let result = py
+ | ______________________-
+274 | | .allow_threads(|| self.runtime.block_on(self.inner.parse_input(pdf_input)))
+ | | -^^^^^^^^^^^^^ method not found in `pyo3::Python<'_>`
+ | |_____________|
+ |
+
+error[E0599]: no method named `allow_threads` found for struct `pyo3::Python<'py>` in the current scope
+ --> crates/liteparse-python/src/lib.rs:283:14
+ |
+282 | let result = py
+ | ______________________-
+283 | | .allow_threads(|| self.runtime.block_on(self.inner.parse_input(pdf_input)))
+ | | -^^^^^^^^^^^^^ method not found in `pyo3::Python<'_>`
+ | |_____________|
+ |
+
+error[E0599]: no method named `allow_threads` found for struct `pyo3::Python<'py>` in the current scope
+ --> crates/liteparse-python/src/lib.rs:298:12
+ |
+298 | py.allow_threads(move || {
+ | ---^^^^^^^^^^^^^ method not found in `pyo3::Python<'_>`
+
+For more information about this error, try `rustc --explain E0599`.
+error: could not compile `liteparse-python` (lib) due to 3 previous errors
+<exited with exit code 0>
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 3m 32s</sub>
+
+### πŸ’­ Reasoning
+
+*I need to figure out which version of pyo3 introduced changes to `allow_threads` and whether it's behind a feature flag or has been renamed in the newer releases.*
+
+---
+
+<sub>⏱️ 3m 33s</sub>
+
+### βœ… `bash`
+
+**find allow_threads in pyo3**
+
+$ cd /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python && grep -r "allow_threads\|fn detach" ~/.cargo/registry/src/*pyo3-0.28*/src/marker.rs 2>/dev/null | head -20
+
+```
+
+<exited with exit code 0>
+```
+
+
+---
+
+<sub>⏱️ 3m 39s</sub>
+
+### βœ… `bash`
+
+**find pyo3 marker.rs**
+
+$ find ~/.cargo/registry/src -name "marker.rs" -path "*pyo3-0.28*" 2>/dev/null | head -3
+
+```
+/Users/clee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-0.28.3/src/marker.rs
+<exited with exit code 0>
+```
+
+
+---
+
+<sub>⏱️ 3m 43s</sub>
+
+### βœ… `bash`
+
+**find detach/allow_threads**
+
+$ grep -n "allow_threads\|pub fn detach\|fn detach<" /Users/clee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-0.28.3/src/marker.rs | head -20
+
+```
+558: pub fn detach<T, F>(self, f: F) -> T
+<exited with exit code 0>
+```
+
+
+---
+
+<sub>⏱️ 3m 46s</sub>
+
+### πŸ’­ Reasoning
+
+*I need to update the PyO3 code to use `detach` instead of `allow_threads` since that's the new API in version 0.28.*
+
+---
+
+<sub>⏱️ 3m 46s</sub>
+
+### πŸ’¬ Copilot
+
+In pyo3 0.28, `allow_threads` was renamed to `detach`. Fixing:
+
+---
+
+<sub>⏱️ 3m 49s</sub>
+
+### βœ… `bash`
+
+**rename to detach and recheck**
+
+$ sed -i '' 's/\.allow_threads(/.detach(/g; s/py\.allow_threads(/py.detach(/g' /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs && cd /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python && cargo check 2>&1 | tail -30
+
+<details>
+<summary>6 lines</summary>
+
+```
+Checking liteparse-pdfium-sys v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium-sys)
+ Compiling liteparse-python v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python)
+ Checking liteparse-pdfium v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium)
+ Checking liteparse v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse)
+ Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.44s
+<exited with exit code 0>
+```
+
+</details>
+
+
+---
+
+<sub>⏱️ 3m 51s</sub>
+
+### πŸ’¬ Copilot
+
+Clean build. Summary of changes to `crates/liteparse-python/src/lib.rs`:
+
+1. **GIL released** in `parse`, `parse_bytes`, `screenshot` via `py.detach(|| ...)` (pyo3 0.28's renamed `allow_threads`). All native work β€” Tokio `block_on`, pdfium rendering, PNG encoding β€” now runs without holding the GIL.
+
+2. **Eliminated clones** in conversion path: `PyTextItem::from_rust`, `PyParsedPage::from_rust`, `PyParseResult::from_rust` now take ownership and move `String`/`Vec` fields instead of cloning. Saves N string allocs per page.
+
+3. **Hoisted `pdfium::Library::init()`** onto `LiteParse` struct so it's done once at construction, not per `screenshot()` call. (It's a ZST guarded by `Once`, so the saving is small, but cleaner.)
+
+Build passes. The #1 change is the high-impact one β€” your Scalene profile's GIL contention, `_Py_HandlePending`, and `_PySemaphore_Wait` bars should largely disappear, and any other Python threads can make progress during parsing.
+
+---
+
+<sub>Generated by [GitHub Copilot CLI](https://github.com/features/copilot/cli)</sub>
diff --git a/crates/liteparse-python/OPTIMIZATION_PLAN.md b/crates/liteparse-python/OPTIMIZATION_PLAN.md
new file mode 100644
index 0000000..3c308de
--- /dev/null
+++ b/crates/liteparse-python/OPTIMIZATION_PLAN.md
@@ -0,0 +1,30 @@
+Looking at your Scalene profile, here are the bottlenecks I can identify:
+
+## Key Observations
+
+**1. I/O is the dominant cost (~0–40ms block)**
+The timeline shows a solid blue I/O bar consuming roughly the first 40ms of your 170ms total run. This suggests the file read itself is a significant chunk β€” likely `__fread` / `_srefill` / `_sread` / `_aread` in the call stack. If you're doing unbuffered or small-chunk reads, that's low-hanging fruit.
+
+**2. GIL contention is visible**
+The GIL row in the timeline shows activity, which is notable for a Rust binding. This usually means your Rust extension is calling back into Python too frequently (e.g. building Python objects incrementally rather than bulk-returning), or you're not releasing the GIL during the heavy native work with `py.allow_threads(|| ...)`.
+
+**3. `_Py_HandlePending` / `PyErr_CheckSignalsState` in the hot path**
+These appear mid-stack and indicate Python's signal-checking mechanism is being hit repeatedly. This happens when Rust calls back into the Python C API in a tight loop β€” each re-entry triggers signal checks. Batching your Python object construction would eliminate most of these.
+
+**4. Thread/semaphore wait (`_PySemaphore_Wait`, `pthread_cond_wait`)**
+The right-hand call stack shows a thread sitting in `_PySemaphore_Wait` β†’ `pthread_cond_wait`. This is a waiting thread, likely a GIL waiter or a thread pool worker stalled. If you're spawning worker threads in Rust, they may be contending more than helping for a single-file parse.
+
+**5. `infinite_lru_cache_wrapper`**
+This appears in the Python call stack. If you have a Python-side LRU cache wrapping something in the hot path, that's adding overhead β€” especially if it's being hit per-page or per-token.
+
+---
+
+## Suggested Fixes
+
+- **Release the GIL** during your Rust parsing work: wrap the core parse loop in `py.allow_threads(|| { ... })` in PyO3. This is the single highest-impact change if you aren't already doing it.
+- **Batch Python object construction** β€” parse everything in Rust, return a single large structure (Vec of structs), then convert to Python objects once at the end rather than per-element.
+- **Buffered reads** β€” if you're opening the file in Rust, make sure you're using `BufReader` rather than raw file reads, which maps to the `_srefill`/`_sread` pattern you're seeing.
+- **Profile the LRU cache** β€” check what `infinite_lru_cache_wrapper` is wrapping on the Python side and whether it can be moved into Rust or eliminated.
+- **Avoid thread spawning per call** β€” if Rayon or a thread pool is being initialized per `parse()` invocation, use a lazy static pool instead.
+
+The biggest win is almost certainly the GIL release + batched return pattern. pymupdf and pdftotext win here largely because their Python bindings do exactly that β€” all work happens natively, Python only touches the final result.
diff --git a/crates/liteparse-python/src/lib.rs b/crates/liteparse-python/src/lib.rs
index 43dd255..ce171de 100644
--- a/crates/liteparse-python/src/lib.rs
+++ b/crates/liteparse-python/src/lib.rs
@@ -56,14 +56,14 @@ impl PyTextItem {
}
}
- fn from_rust(item: &liteparse::types::TextItem) -> Self {
+ fn from_rust(item: liteparse::types::TextItem) -> Self {
Self {
- text: item.text.clone(),
+ text: item.text,
x: item.x as f64,
y: item.y as f64,
width: item.width as f64,
height: item.height as f64,
- font_name: item.font_name.clone(),
+ font_name: item.font_name,
font_size: item.font_size.map(|v| v as f64),
confidence: item.confidence.map(|v| v as f64).or(Some(1.0)),
}
@@ -99,13 +99,17 @@ impl PyParsedPage {
}
impl PyParsedPage {
- fn from_rust(page: &liteparse::types::ParsedPage) -> Self {
+ fn from_rust(page: liteparse::types::ParsedPage) -> Self {
Self {
page_num: page.page_number as u32,
width: page.page_width as f64,
height: page.page_height as f64,
- text: page.text.clone(),
- text_items: page.text_items.iter().map(PyTextItem::from_rust).collect(),
+ text: page.text,
+ text_items: page
+ .text_items
+ .into_iter()
+ .map(PyTextItem::from_rust)
+ .collect(),
}
}
}
@@ -140,10 +144,14 @@ impl PyParseResult {
}
impl PyParseResult {
- fn from_rust(result: &liteparse::parser::ParseResult) -> Self {
+ fn from_rust(result: liteparse::parser::ParseResult) -> Self {
Self {
- pages: result.pages.iter().map(PyParsedPage::from_rust).collect(),
- text: result.text.clone(),
+ pages: result
+ .pages
+ .into_iter()
+ .map(PyParsedPage::from_rust)
+ .collect(),
+ text: result.text,
}
}
}
@@ -184,6 +192,7 @@ struct LiteParse {
inner: liteparse::parser::LiteParse,
config: LiteParseConfig,
runtime: tokio::runtime::Runtime,
+ pdfium: pdfium::Library,
}
#[pymethods]
@@ -262,91 +271,95 @@ impl LiteParse {
let inner = liteparse::parser::LiteParse::new(cfg.clone());
let runtime = tokio::runtime::Runtime::new()
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+ let pdfium = pdfium::Library::init();
Ok(Self {
inner,
config: cfg,
runtime,
+ pdfium,
})
}
/// Parse a document from a file path.
- fn parse(&self, input: String) -> PyResult<PyParseResult> {
+ fn parse(&self, py: Python<'_>, input: String) -> PyResult<PyParseResult> {
let pdf_input = PdfInput::Path(input);
- let result = self
- .runtime
- .block_on(self.inner.parse_input(pdf_input))
+ let result = py
+ .detach(|| self.runtime.block_on(self.inner.parse_input(pdf_input)))
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
- Ok(PyParseResult::from_rust(&result))
+ Ok(PyParseResult::from_rust(result))
}
/// Parse a document from raw bytes.
- fn parse_bytes(&self, data: Vec<u8>) -> PyResult<PyParseResult> {
+ fn parse_bytes(&self, py: Python<'_>, data: Vec<u8>) -> PyResult<PyParseResult> {
let pdf_input = PdfInput::Bytes(data);
- let result = self
- .runtime
- .block_on(self.inner.parse_input(pdf_input))
+ let result = py
+ .detach(|| self.runtime.block_on(self.inner.parse_input(pdf_input)))
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
- Ok(PyParseResult::from_rust(&result))
+ Ok(PyParseResult::from_rust(result))
}
/// Take screenshots of document pages. Returns a list of ScreenshotResult.
#[pyo3(signature = (input, page_numbers = None))]
fn screenshot(
&self,
+ py: Python<'_>,
input: String,
page_numbers: Option<Vec<u32>>,
) -> PyResult<Vec<PyScreenshotResult>> {
let dpi = self.config.dpi;
- let lib = pdfium::Library::init();
- let document = lib
- .load_document(&input, self.config.password.as_deref())
- .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
- let page_count = document.page_count() as u32;
-
- let pages: Vec<u32> = match page_numbers {
- Some(nums) => nums,
- None => (1..=page_count).collect(),
- };
-
- let mut results = Vec::with_capacity(pages.len());
- for page_num in pages {
- if page_num < 1 || page_num > page_count {
- return Err(PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(
- "page {page_num} out of range (document has {page_count} pages)"
- )));
- }
- let page = document
- .page((page_num - 1) as i32)
- .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
- let bitmap = page
- .render(dpi)
+ let password = self.config.password.clone();
+ py.detach(move || {
+ let document = self
+ .pdfium
+ .load_document(&input, password.as_deref())
.map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;
+ let page_count = document.page_count() as u32;
+
+ let pages: Vec<u32> = match page_numbers {
+ Some(nums) => nums,
+ None => (1..=page_count).collect(),
+ };
- let width = bitmap.width() as u32;
- let height = bitmap.height() as u32;
- let rgba = bitmap.to_rgba();
-
- let mut png_buf: Vec<u8> = Vec::new();
- let encoder = image::codecs::png::PngEncoder::new(&mut png_buf);
- use image::ImageEncoder;
- encoder
- .write_image(&rgba, width, height, image::ColorType::Rgba8.into())
- .map_err(|e| {
- PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(format!(
- "PNG encode failed: {e}"
- ))
+ let mut results = Vec::with_capacity(pages.len());
+ for page_num in pages {
+ if page_num < 1 || page_num > page_count {
+ return Err(PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(
+ "page {page_num} out of range (document has {page_count} pages)"
+ )));
+ }
+ let page = document.page((page_num - 1) as i32).map_err(|e| {
+ PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string())
+ })?;
+ let bitmap = page.render(dpi).map_err(|e| {
+ PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string())
})?;
- results.push(PyScreenshotResult {
- page_num,
- width,
- height,
- image_buffer: png_buf,
- });
- }
+ let width = bitmap.width() as u32;
+ let height = bitmap.height() as u32;
+ let rgba = bitmap.to_rgba();
+
+ let mut png_buf: Vec<u8> = Vec::new();
+ let encoder = image::codecs::png::PngEncoder::new(&mut png_buf);
+ use image::ImageEncoder;
+ encoder
+ .write_image(&rgba, width, height, image::ColorType::Rgba8.into())
+ .map_err(|e| {
+ PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(format!(
+ "PNG encode failed: {e}"
+ ))
+ })?;
+
+ results.push(PyScreenshotResult {
+ page_num,
+ width,
+ height,
+ image_buffer: png_buf,
+ });
+ }
- Ok(results)
+ Ok(results)
+ })
}
fn __repr__(&self) -> String {