A fast, helpful, and open-source document parser
wip: tentative optimizations
Clelia (Astra) Bertellie562cea6eeba
diff --git a/crates/copilot-session-edfc05f0-b306-47a0-b299-e68c0df2e82a.md b/crates/copilot-session-edfc05f0-b306-47a0-b299-e68c0df2e82a.mdnew file mode 100644index 0000000..e5fea6a--- /dev/null+++ b/crates/copilot-session-edfc05f0-b306-47a0-b299-e68c0df2e82a.md@@ -0,0 +1,1104 @@+# π€ Copilot CLI Session++> [!NOTE]+> - **Session ID:** `edfc05f0-b306-47a0-b299-e68c0df2e82a`+> - **Started:** 5/19/2026, 11:17:00 AM+> - **Duration:** 7m 45s+> - **Exported:** 5/19/2026, 11:24:46 AM++---++<sub>β±οΈ 0s</sub>++### βΉοΈ Info++π‘ No copilot instructions found. Run /init to generate a copilot-instructions.md file for this project.++---++<sub>β±οΈ 1s</sub>++### βΉοΈ Info++Environment loaded: 2 custom instructions, 2 skills, 1 MCP server++---++<sub>β±οΈ 38s</sub>++### π€ User++Read @liteparse-python/OPTIMIZATION_PLAN.md and let's discuss if the optimizations are feasible in @liteparse-python/src/lib.rs++---++<sub>β±οΈ 41s</sub>++### π Reasoning++*I'm going to read through both files to understand what we're working with.*++---++<sub>β±οΈ 42s</sub>++### β `view`++**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/OPTIMIZATION_PLAN.md.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/OPTIMIZATION_PLAN.md++<details>+<summary>31 lines</summary>++```+1. Looking at your Scalene profile, here are the bottlenecks I can identify:+2.+3. ## Key Observations+4.+5. **1. I/O is the dominant cost (~0β40ms block)**+6. The timeline shows a solid blue I/O bar consuming roughly the first 40ms of your 170ms total run. This suggests the file read itself is a significant chunk β likely `__fread` / `_srefill` / `_sread` / `_aread` in the call stack. If you're doing unbuffered or small-chunk reads, that's low-hanging fruit.+7.+8. **2. GIL contention is visible**+9. The GIL row in the timeline shows activity, which is notable for a Rust binding. This usually means your Rust extension is calling back into Python too frequently (e.g. building Python objects incrementally rather than bulk-returning), or you're not releasing the GIL during the heavy native work with `py.allow_threads(|| ...)`.+10.+11. **3. `_Py_HandlePending` / `PyErr_CheckSignalsState` in the hot path**+12. These appear mid-stack and indicate Python's signal-checking mechanism is being hit repeatedly. This happens when Rust calls back into the Python C API in a tight loop β each re-entry triggers signal checks. Batching your Python object construction would eliminate most of these.+13.+14. **4. Thread/semaphore wait (`_PySemaphore_Wait`, `pthread_cond_wait`)**+15. The right-hand call stack shows a thread sitting in `_PySemaphore_Wait` β `pthread_cond_wait`. This is a waiting thread, likely a GIL waiter or a thread pool worker stalled. If you're spawning worker threads in Rust, they may be contending more than helping for a single-file parse.+16.+17. **5. `infinite_lru_cache_wrapper`**+18. This appears in the Python call stack. If you have a Python-side LRU cache wrapping something in the hot path, that's adding overhead β especially if it's being hit per-page or per-token.+19.+20. ---+21.+22. ## Suggested Fixes+23.+24. - **Release the GIL** during your Rust parsing work: wrap the core parse loop in `py.allow_threads(|| { ... })` in PyO3. This is the single highest-impact change if you aren't already doing it.+25. - **Batch Python object construction** β parse everything in Rust, return a single large structure (Vec of structs), then convert to Python objects once at the end rather than per-element.+26. - **Buffered reads** β if you're opening the file in Rust, make sure you're using `BufReader` rather than raw file reads, which maps to the `_srefill`/`_sread` pattern you're seeing.+27. - **Profile the LRU cache** β check what `infinite_lru_cache_wrapper` is wrapping on the Python side and whether it can be moved into Rust or eliminated.+28. - **Avoid thread spawning per call** β if Rayon or a thread pool is being initialized per `parse()` invocation, use a lazy static pool instead.+29.+30. The biggest win is almost certainly the GIL release + batched return pattern. pymupdf and pdftotext win here largely because their Python bindings do exactly that β all work happens natively, Python only touches the final result.+31.+```++</details>+++---++<sub>β±οΈ 42s</sub>++### β `view`++**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs++<details>+<summary>366 lines</summary>++```+1. use pyo3::prelude::*;+2. use pyo3::types::PyBytes;+3.+4. use liteparse::config::{LiteParseConfig, OutputFormat};+5. use liteparse::types::PdfInput;+6.+7. mod cli;+8.+9. // ---------------------------------------------------------------------------+10. // Python type wrappers+11. // ---------------------------------------------------------------------------+12.+13. #[pyclass(frozen, from_py_object)]+14. #[derive(Clone)]+15. struct PyTextItem {+16. #[pyo3(get)]+17. text: String,+18. #[pyo3(get)]+19. x: f64,+20. #[pyo3(get)]+21. y: f64,+22. #[pyo3(get)]+23. width: f64,+24. #[pyo3(get)]+25. height: f64,+26. #[pyo3(get)]+27. font_name: Option<String>,+28. #[pyo3(get)]+29. font_size: Option<f64>,+30. #[pyo3(get)]+31. confidence: Option<f64>,+32. }+33.+34. #[pymethods]+35. impl PyTextItem {+36. fn __repr__(&self) -> String {+37. format!(+38. "TextItem(text={:?}, x={}, y={}, width={}, height={})",+39. self.text, self.x, self.y, self.width, self.height+40. )+41. }+42. }+43.+44. impl PyTextItem {+45. fn from_rust(item: &liteparse::types::TextItem) -> Self {+46. Self {+47. text: item.text.clone(),+48. x: item.x as f64,+49. y: item.y as f64,+50. width: item.width as f64,+51. height: item.height as f64,+52. font_name: item.font_name.clone(),+53. font_size: item.font_size.map(|v| v as f64),+54. confidence: item.confidence.map(|v| v as f64).or(Some(1.0)),+55. }+56. }+57. }+58.+59. #[pyclass(frozen, from_py_object)]+60. #[derive(Clone)]+61. struct PyParsedPage {+62. #[pyo3(get)]+63. page_num: u32,+64. #[pyo3(get)]+65. width: f64,+66. #[pyo3(get)]+67. height: f64,+68. #[pyo3(get)]+69. text: String,+70. #[pyo3(get)]+71. text_items: Vec<PyTextItem>,+72. }+73.+74. #[pymethods]+75. impl PyParsedPage {+76. fn __repr__(&self) -> String {+77. format!(+78. "ParsedPage(page_num={}, width={}, height={}, text_items={})",+79. self.page_num,+80. self.width,+81. self.height,+82. self.text_items.len()+83. )+84. }+85. }+86.+87. impl PyParsedPage {+88. fn from_rust(page: &liteparse::types::ParsedPage) -> Self {+89. Self {+90. page_num: page.page_number as u32,+91. width: page.page_width as f64,+92. height: page.page_height as f64,+93. text: page.text.clone(),+94. text_items: page.text_items.iter().map(PyTextItem::from_rust).collect(),+95. }+96. }+97. }+98.+99. #[pyclass(frozen, from_py_object)]+100. #[derive(Clone)]+101. struct PyParseResult {+102. #[pyo3(get)]+103. pages: Vec<PyParsedPage>,+104. #[pyo3(get)]+105. text: String,+106. }+107.+108. #[pymethods]+109. impl PyParseResult {+110. #[getter]+111. fn num_pages(&self) -> usize {+112. self.pages.len()+113. }+114.+115. fn get_page(&self, page_num: u32) -> Option<PyParsedPage> {+116. self.pages.iter().find(|p| p.page_num == page_num).cloned()+117. }+118.+119. fn __repr__(&self) -> String {+120. format!(+121. "ParseResult(pages={}, text_len={})",+122. self.pages.len(),+123. self.text.len()+124. )+125. }+126. }+127.+128. impl PyParseResult {+129. fn from_rust(result: &liteparse::parser::ParseResult) -> Self {+130. Self {+131. pages: result.pages.iter().map(PyParsedPage::from_rust).collect(),+132. text: result.text.clone(),+133. }+134. }+135. }+136.+137. #[pyclass(frozen, from_py_object)]+138. #[derive(Clone)]+139. struct PyScreenshotResult {+140. #[pyo3(get)]+141. page_num: u32,+142. #[pyo3(get)]+143. width: u32,+144. #[pyo3(get)]+145. height: u32,+146. image_buffer: Vec<u8>,+147. }+148.+149. #[pymethods]+150. impl PyScreenshotResult {+151. #[getter]+152. fn image_bytes<'py>(&self, py: Python<'py>) -> Bound<'py, PyBytes> {+153. PyBytes::new(py, &self.image_buffer)+154. }+155.+156. fn __repr__(&self) -> String {+157. format!(+158. "ScreenshotResult(page_num={}, width={}, height={})",+159. self.page_num, self.width, self.height+160. )+161. }+162. }+163.+164. // ---------------------------------------------------------------------------+165. // Main LiteParse class+166. // ---------------------------------------------------------------------------+167.+168. #[pyclass]+169. struct LiteParse {+170. inner: liteparse::parser::LiteParse,+171. config: LiteParseConfig,+172. runtime: tokio::runtime::Runtime,+173. }+174.+175. #[pymethods]+176. impl LiteParse {+177. #[new]+178. #[pyo3(signature = (+179. *,+180. ocr_language = None,+181. ocr_enabled = None,+182. ocr_server_url = None,+183. tessdata_path = None,+184. max_pages = None,+185. target_pages = None,+186. dpi = None,+187. output_format = None,+188. preserve_very_small_text = None,+189. password = None,+190. quiet = None,+191. num_workers = None,+192. ))]+193. fn new(+194. ocr_language: Option<String>,+195. ocr_enabled: Option<bool>,+196. ocr_server_url: Option<String>,+197. tessdata_path: Option<String>,+198. max_pages: Option<usize>,+199. target_pages: Option<String>,+200. dpi: Option<f32>,+201. output_format: Option<String>,+202. preserve_very_small_text: Option<bool>,+203. password: Option<String>,+204. quiet: Option<bool>,+205. num_workers: Option<usize>,+206. ) -> PyResult<Self> {+207. let mut cfg = LiteParseConfig::default();+208. if let Some(v) = ocr_language {+209. cfg.ocr_language = v;+210. }+211. if let Some(v) = ocr_enabled {+212. cfg.ocr_enabled = v;+213. }+214. if let Some(v) = ocr_server_url {+215. cfg.ocr_server_url = Some(v);+216. }+217. if let Some(v) = tessdata_path {+218. cfg.tessdata_path = Some(v);+219. }+220. if let Some(v) = max_pages {+221. cfg.max_pages = v;+222. }+223. if let Some(v) = target_pages {+224. cfg.target_pages = Some(v);+225. }+226. if let Some(v) = dpi {+227. cfg.dpi = v;+228. }+229. if let Some(v) = output_format {+230. cfg.output_format = match v.as_str() {+231. "text" => OutputFormat::Text,+232. _ => OutputFormat::Json,+233. };+234. }+235. if let Some(v) = preserve_very_small_text {+236. cfg.preserve_very_small_text = v;+237. }+238. if let Some(v) = password {+239. cfg.password = Some(v);+240. }+241. if let Some(v) = quiet {+242. cfg.quiet = v;+243. }+244. if let Some(v) = num_workers {+245. cfg.num_workers = v;+246. }+247.+248. let inner = liteparse::parser::LiteParse::new(cfg.clone());+249. let runtime = tokio::runtime::Runtime::new()+250. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+251.+252. Ok(Self {+253. inner,+254. config: cfg,+255. runtime,+256. })+257. }+258.+259. /// Parse a document from a file path.+260. fn parse(&self, input: String) -> PyResult<PyParseResult> {+261. let pdf_input = PdfInput::Path(input);+262. let result = self+263. .runtime+264. .block_on(self.inner.parse_input(pdf_input))+265. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+266. Ok(PyParseResult::from_rust(&result))+267. }+268.+269. /// Parse a document from raw bytes.+270. fn parse_bytes(&self, data: Vec<u8>) -> PyResult<PyParseResult> {+271. let pdf_input = PdfInput::Bytes(data);+272. let result = self+273. .runtime+274. .block_on(self.inner.parse_input(pdf_input))+275. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+276. Ok(PyParseResult::from_rust(&result))+277. }+278.+279. /// Take screenshots of document pages. Returns a list of ScreenshotResult.+280. #[pyo3(signature = (input, page_numbers = None))]+281. fn screenshot(+282. &self,+283. input: String,+284. page_numbers: Option<Vec<u32>>,+285. ) -> PyResult<Vec<PyScreenshotResult>> {+286. let dpi = self.config.dpi;+287. let lib = pdfium::Library::init();+288. let document = lib+289. .load_document(&input, self.config.password.as_deref())+290. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+291. let page_count = document.page_count() as u32;+292.+293. let pages: Vec<u32> = match page_numbers {+294. Some(nums) => nums,+295. None => (1..=page_count).collect(),+296. };+297.+298. let mut results = Vec::with_capacity(pages.len());+299. for page_num in pages {+300. if page_num < 1 || page_num > page_count {+301. return Err(PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(+302. "page {page_num} out of range (document has {page_count} pages)"+303. )));+304. }+305. let page = document+306. .page((page_num - 1) as i32)+307. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+308. let bitmap = page+309. .render(dpi)+310. .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+311.+312. let width = bitmap.width() as u32;+313. let height = bitmap.height() as u32;+314. let rgba = bitmap.to_rgba();+315.+316. let mut png_buf: Vec<u8> = Vec::new();+317. let encoder = image::codecs::png::PngEncoder::new(&mut png_buf);+318. use image::ImageEncoder;+319. encoder+320. .write_image(&rgba, width, height, image::ColorType::Rgba8.into())+321. .map_err(|e| {+322. PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(format!(+323. "PNG encode failed: {e}"+324. ))+325. })?;+326.+327. results.push(PyScreenshotResult {+328. page_num,+329. width,+330. height,+331. image_buffer: png_buf,+332. });+333. }+334.+335. Ok(results)+336. }+337.+338. fn __repr__(&self) -> String {+339. format!(+340. "LiteParse(ocr_enabled={}, dpi={}, max_pages={})",+341. self.config.ocr_enabled, self.config.dpi, self.config.max_pages+342. )+343. }+344. }+345.+346. // ---------------------------------------------------------------------------+347. // Module+348. // ---------------------------------------------------------------------------+349.+350. /// Run the `lit` CLI with the given arguments.+351. #[pyfunction]+352. fn run_cli(args: Vec<String>) -> PyResult<()> {+353. cli::run_cli(args).map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))+354. }+355.+356. #[pymodule]+357. fn _liteparse(m: &Bound<'_, PyModule>) -> PyResult<()> {+358. m.add_class::<LiteParse>()?;+359. m.add_class::<PyParseResult>()?;+360. m.add_class::<PyParsedPage>()?;+361. m.add_class::<PyTextItem>()?;+362. m.add_class::<PyScreenshotResult>()?;+363. m.add_function(wrap_pyfunction!(run_cli, m)?)?;+364. Ok(())+365. }+366.+```++</details>+++---++<sub>β±οΈ 46s</sub>++### π Reasoning++*I'm checking each suggestion against the actual code to see if they're accurate.*++---++<sub>β±οΈ 46s</sub>++### π¬ Copilot++Here's my assessment of the OPTIMIZATION_PLAN suggestions against the current `lib.rs`:++## Feasibility analysis++**1. Release the GIL with `py.allow_threads(...)` β β Highly feasible, biggest win**++Currently `parse`, `parse_bytes`, and `screenshot` all run heavy native work (including `runtime.block_on(...)` on a Tokio runtime) while holding the GIL. None of these methods accept `py: Python\<'_\>` or wrap work in `allow_threads`. This is genuinely low-hanging fruit:++```rust+fn parse(&self, py: Python\<'_\>, input: String) -\> PyResult\<PyParseResult\> {+ let pdf_input = PdfInput::Path(input);+ let result = py.allow_threads(|| {+ self.runtime.block_on(self.inner.parse_input(pdf_input))+ }).map_err(...)?;+ Ok(PyParseResult::from_rust(&result))+}+```++This alone should eliminate the GIL contention, `_Py_HandlePending`, and `_PySemaphore_Wait` symptoms β any Python thread waiting on the GIL is blocked while we sit in `block_on` for ~100ms+.++**2. Batch Python object construction β β οΈ Partially applicable**++The plan describes "Rust calling back into Python in a tight loop." Looking at `from_rust`, we don't actually do that β `PyTextItem`/`PyParsedPage` are plain Rust structs (`#[pyclass(frozen)]`) built in Rust, with no Python API calls per element. The conversion happens after parsing is complete. So this concern is largely already addressed.++However, `PyParseResult::from_rust` clones every `TextItem`, `String`, and `Vec` from the underlying `ParseResult`. We could **move** instead of clone by taking ownership of `result`:++```rust+fn from_rust(result: liteparse::parser::ParseResult) -\> Self { ... }+```++Modest win, but worth doing.++**3. Buffered reads (`BufReader`) β β Out of scope here**++`lib.rs` doesn't do file I/O itself β it hands a `PdfInput::Path` to `liteparse::parser`. The `_fread`/`_srefill` cost is inside the core crate (or pdfium). Fixing it requires changes in the `liteparse` crate, not the Python binding. Worth investigating separately.++**4. `infinite_lru_cache_wrapper` β β Not in this file**++That's Python-side. Likely in `liteparse-python`'s Python wrapper code (if any) or in user code. Not addressable in `lib.rs`.++**5. Avoid per-call thread pool / runtime spawning β β Already done, but could be better**++Good news: the Tokio `Runtime` is created once in `LiteParse::new()` and reused. However, `screenshot` calls `pdfium::Library::init()` **per invocation**, which may have init cost. Worth checking if pdfium can be initialized once and stored on `self` (depends on pdfium's API and `Send`/`Sync` guarantees).++## Recommended order of changes++1. **Wrap all three methods in `py.allow_threads`** β highest ROI, ~5-line change. Frees the GIL for ~95% of wall-clock time.+2. **Move (not clone) in `from_rust` conversions** β small but easy win once we restructure ownership.+3. **Hoist `pdfium::Library::init()` to `LiteParse::new`** β if API permits.+4. **Investigate `BufReader` / I/O** in the core `liteparse` crate (separate task).++Want me to implement #1 (and optionally #2, #3)?++---++<sub>β±οΈ 2m 14s</sub>++### π€ User++let's go with 1-3++---++<sub>β±οΈ 2m 18s</sub>++### π Reasoning++*I'm checking the pdfium API documentation to see whether the Library object can be stored as an instance variable.*++---++<sub>β±οΈ 2m 21s</sub>++### β `grep`++**pub (fn|struct) (init|Library|load_document)**++"pub (fn|struct) (init|Library|load_document)" (pdfium)++```+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:9:pub struct Library {+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:14: pub fn init() -> Library {+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:21: pub fn load_document(+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:45: pub fn load_document_from_bytes(+```+++---++<sub>β±οΈ 2m 21s</sub>++### β `grep`++**struct Library|impl Library|unsafe impl (Send|Sync) for Library**++"struct Library|impl Library|unsafe impl (Send|Sync) for Library" (pdfium)++```+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:9:pub struct Library {+/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs:13:impl Library {+```+++---++<sub>β±οΈ 2m 24s</sub>++### β `view`++**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium/src/library.rs (lines 1-60)++<details>+<summary>60 lines</summary>++```+1. use std::ffi::CString;+2. use std::sync::Once;+3.+4. use crate::document::Document;+5. use crate::error::PdfiumError;+6.+7. static INIT: Once = Once::new();+8.+9. pub struct Library {+10. _private: (),+11. }+12.+13. impl Library {+14. pub fn init() -> Library {+15. INIT.call_once(|| {+16. unsafe { pdfium_sys::FPDF_InitLibrary() };+17. });+18. Library { _private: () }+19. }+20.+21. pub fn load_document(+22. &self,+23. path: &str,+24. password: Option<&str>,+25. ) -> Result<Document, PdfiumError> {+26. let c_path = CString::new(path).map_err(|_| PdfiumError::FileNotFound)?;+27. let c_password = password+28. .map(|p| CString::new(p).map_err(|_| PdfiumError::OperationFailed))+29. .transpose()?;+30.+31. let handle = unsafe {+32. pdfium_sys::FPDF_LoadDocument(+33. c_path.as_ptr(),+34. c_password.as_ref().map_or(std::ptr::null(), |p| p.as_ptr()),+35. )+36. };+37.+38. if handle.is_null() {+39. return Err(PdfiumError::from_last_error());+40. }+41.+42. Ok(Document { handle })+43. }+44.+45. pub fn load_document_from_bytes(+46. &self,+47. data: &[u8],+48. password: Option<&str>,+49. ) -> Result<Document, PdfiumError> {+50. let c_password = password+51. .map(|p| CString::new(p).map_err(|_| PdfiumError::OperationFailed))+52. .transpose()?;+53.+54. let handle = unsafe {+55. pdfium_sys::FPDF_LoadMemDocument(+56. data.as_ptr() as *const std::ffi::c_void,+57. data.len() as i32,+58. c_password.as_ref().map_or(std::ptr::null(), |p| p.as_ptr()),+59. )+60. };+```++</details>+++---++<sub>β±οΈ 2m 25s</sub>++### βΉοΈ Info++All permissions are now enabled. Tool, path, and URL requests will be automatically approved.++---++<sub>β±οΈ 2m 29s</sub>++### π Reasoning++*`Library` being a zero-sized type wrapped in `Once` means reinitializing it is cheap since there's no real allocation happeningβthe actual overhead comes from loading the document itself. Moving it to `self` would be cleaner and eliminate any per-call costs. The tricky part is that while `Library` should automatically be `Send + Sync` as a ZST containing just `()`, I need to verify it actually compiles with pyclass's `Send` requirement.++Now I'm checking the shape of `ParseResult` to see if I can optimize the move semantics there.*++---++<sub>β±οΈ 2m 31s</sub>++### β `grep`++**pub struct (ParseResult|ParsedPage|TextItem)**++"pub struct (ParseResult|ParsedPage|TextItem)" (liteparse/src)++```+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/parser.rs:16:pub struct ParseResult {+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/output/json.rs:29:pub struct ParseResultJson {+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs:28:pub struct TextItem {+/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs:81:pub struct ParsedPage {+```+++---++<sub>β±οΈ 2m 35s</sub>++### β `view`++**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/parser.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/parser.rs (lines 16-30)++<details>+<summary>15 lines</summary>++```+16. pub struct ParseResult {+17. /// Parsed pages with projected text layout.+18. pub pages: Vec<ParsedPage>,+19. /// Full document text, concatenated from all pages.+20. pub text: String,+21. }+22.+23. /// Main LiteParse orchestrator.+24. pub struct LiteParse {+25. config: LiteParseConfig,+26. /// Optional caller-provided OCR engine. When set, this overrides the+27. /// built-in selection logic (HTTP OCR / Tesseract). This is the primary+28. /// mechanism for plugging an OCR engine in environments without the+29. /// built-ins (e.g. WASM, where the JS side supplies a callback engine).+30. ocr_engine_override: Option<std::sync::Arc<dyn OcrEngine>>,+```++</details>+++---++<sub>β±οΈ 2m 35s</sub>++### β `view`++**view the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse/src/types.rs (lines 28-105)++<details>+<summary>78 lines</summary>++```+28. pub struct TextItem {+29. pub text: String,+30. /// Viewport-space coordinates (top-left origin, 72 DPI).+31. pub x: f32,+32. pub y: f32,+33. pub width: f32,+34. pub height: f32,+35. /// Rotation in degrees (counter-clockwise, adjusted for page rotation).+36. pub rotation: f32,+37. pub font_name: Option<String>,+38. pub font_size: Option<f32>,+39. /// Font size * scale_y from the text matrix β accounts for CTM scaling.+40. #[serde(skip_serializing_if = "Option::is_none")]+41. pub font_height: Option<f32>,+42. #[serde(skip_serializing_if = "Option::is_none")]+43. pub font_ascent: Option<f32>,+44. #[serde(skip_serializing_if = "Option::is_none")]+45. pub font_descent: Option<f32>,+46. #[serde(skip_serializing_if = "Option::is_none")]+47. pub font_weight: Option<i32>,+48. #[serde(skip_serializing_if = "Option::is_none")]+49. pub font_flags: Option<i32>,+50. /// Sum of glyph widths (using charcode-based lookup when possible).+51. #[serde(skip_serializing_if = "Option::is_none")]+52. pub text_width: Option<f32>,+53. /// Whether the font has buggy encoding (private-use codepoints, TT subset, etc.)+54. #[serde(skip_serializing_if = "std::ops::Not::not")]+55. pub font_is_buggy: bool,+56. /// Marked content ID from the PDF structure tree.+57. #[serde(skip_serializing_if = "Option::is_none")]+58. pub mcid: Option<i32>,+59. /// Fill color as ARGB hex string (e.g. "ff000000").+60. #[serde(skip_serializing_if = "Option::is_none")]+61. pub fill_color: Option<String>,+62. /// Stroke color as ARGB hex string.+63. #[serde(skip_serializing_if = "Option::is_none")]+64. pub stroke_color: Option<String>,+65. /// OCR confidence score (0.0β1.0). None for native PDF text.+66. #[serde(skip_serializing_if = "Option::is_none")]+67. pub confidence: Option<f32>,+68. }+69.+70. /// Represents a single page in a PDF document, including its dimensions and extracted text items.+71. #[derive(Debug, Serialize)]+72. pub struct Page {+73. pub page_number: usize,+74. pub page_width: f32,+75. pub page_height: f32,+76. pub text_items: Vec<TextItem>,+77. }+78.+79. /// Represents a fully parsed page with projected text layout.+80. #[derive(Debug, Serialize)]+81. pub struct ParsedPage {+82. pub page_number: usize,+83. pub page_width: f32,+84. pub page_height: f32,+85. pub text: String,+86. pub text_items: Vec<TextItem>,+87. }+88.+89. #[derive(Debug, Serialize)]+90. pub enum Snap {+91. Left,+92. Right,+93. Center,+94. }+95.+96. #[derive(Debug, Serialize)]+97. pub enum Anchor {+98. Left,+99. Right,+100. Center,+101. }+102.+103. /// Represents a Projected piece of text, responsible for keeping track of projection related data+104. #[derive(Debug, Serialize)]+105. pub struct ProjectedTextItem {+```++</details>+++---++<sub>β±οΈ 2m 39s</sub>++### π Reasoning++*Now I'm ready to implement the fixes: wrapping the main functions in thread-safe blocks, refactoring the ownership model to move fields directly, and initializing the PDF library at the struct level rather than per-operation. Since the library is a zero-sized type that's already thread-safe, I just need to store it in the struct and ensure the Python class can handle it.++Writing the implementation...*++---++<sub>β±οΈ 3m 17s</sub>++### β `edit`++**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs++```+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.+```+++---++<sub>β±οΈ 3m 17s</sub>++### β `edit`++**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs++```+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.+```+++---++<sub>β±οΈ 3m 17s</sub>++### β `edit`++**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs++```+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.+```+++---++<sub>β±οΈ 3m 17s</sub>++### β `edit`++**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs++```+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.+```+++---++<sub>β±οΈ 3m 17s</sub>++### β `edit`++**edit the file at /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs.**++/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs++```+File /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs updated with changes.+```+++---++<sub>β±οΈ 3m 20s</sub>++### π¬ Copilot++Now let's verify it builds:++---++<sub>β±οΈ 3m 26s</sub>++### β `bash`++**cargo check liteparse-python**++$ cd /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python && cargo check 2>&1 | tail -60++<details>+<summary>35 lines</summary>++```+Compiling pyo3-build-config v0.28.3+ Checking liteparse-pdfium-sys v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium-sys)+ Compiling liteparse v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse)+ Compiling liteparse-python v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python)+ Checking liteparse-pdfium v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium)+ Compiling pyo3-macros-backend v0.28.3+ Compiling pyo3-ffi v0.28.3+ Compiling pyo3 v0.28.3+ Compiling pyo3-macros v0.28.3+error[E0599]: no method named `allow_threads` found for struct `pyo3::Python<'py>` in the current scope+ --> crates/liteparse-python/src/lib.rs:274:14+ |+273 | let result = py+ | ______________________-+274 | | .allow_threads(|| self.runtime.block_on(self.inner.parse_input(pdf_input)))+ | | -^^^^^^^^^^^^^ method not found in `pyo3::Python<'_>`+ | |_____________|+ |++error[E0599]: no method named `allow_threads` found for struct `pyo3::Python<'py>` in the current scope+ --> crates/liteparse-python/src/lib.rs:283:14+ |+282 | let result = py+ | ______________________-+283 | | .allow_threads(|| self.runtime.block_on(self.inner.parse_input(pdf_input)))+ | | -^^^^^^^^^^^^^ method not found in `pyo3::Python<'_>`+ | |_____________|+ |++error[E0599]: no method named `allow_threads` found for struct `pyo3::Python<'py>` in the current scope+ --> crates/liteparse-python/src/lib.rs:298:12+ |+298 | py.allow_threads(move || {+ | ---^^^^^^^^^^^^^ method not found in `pyo3::Python<'_>`++For more information about this error, try `rustc --explain E0599`.+error: could not compile `liteparse-python` (lib) due to 3 previous errors+<exited with exit code 0>+```++</details>+++---++<sub>β±οΈ 3m 32s</sub>++### π Reasoning++*I need to figure out which version of pyo3 introduced changes to `allow_threads` and whether it's behind a feature flag or has been renamed in the newer releases.*++---++<sub>β±οΈ 3m 33s</sub>++### β `bash`++**find allow_threads in pyo3**++$ cd /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python && grep -r "allow_threads\|fn detach" ~/.cargo/registry/src/*pyo3-0.28*/src/marker.rs 2>/dev/null | head -20++```++<exited with exit code 0>+```+++---++<sub>β±οΈ 3m 39s</sub>++### β `bash`++**find pyo3 marker.rs**++$ find ~/.cargo/registry/src -name "marker.rs" -path "*pyo3-0.28*" 2>/dev/null | head -3++```+/Users/clee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-0.28.3/src/marker.rs+<exited with exit code 0>+```+++---++<sub>β±οΈ 3m 43s</sub>++### β `bash`++**find detach/allow_threads**++$ grep -n "allow_threads\|pub fn detach\|fn detach<" /Users/clee/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/pyo3-0.28.3/src/marker.rs | head -20++```+558: pub fn detach<T, F>(self, f: F) -> T+<exited with exit code 0>+```+++---++<sub>β±οΈ 3m 46s</sub>++### π Reasoning++*I need to update the PyO3 code to use `detach` instead of `allow_threads` since that's the new API in version 0.28.*++---++<sub>β±οΈ 3m 46s</sub>++### π¬ Copilot++In pyo3 0.28, `allow_threads` was renamed to `detach`. Fixing:++---++<sub>β±οΈ 3m 49s</sub>++### β `bash`++**rename to detach and recheck**++$ sed -i '' 's/\.allow_threads(/.detach(/g; s/py\.allow_threads(/py.detach(/g' /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python/src/lib.rs && cd /Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python && cargo check 2>&1 | tail -30++<details>+<summary>6 lines</summary>++```+Checking liteparse-pdfium-sys v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium-sys)+ Compiling liteparse-python v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse-python)+ Checking liteparse-pdfium v1.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/pdfium)+ Checking liteparse v2.0.0 (/Users/clee/Desktop/llama_index_work/liteparse/crates/liteparse)+ Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.44s+<exited with exit code 0>+```++</details>+++---++<sub>β±οΈ 3m 51s</sub>++### π¬ Copilot++Clean build. Summary of changes to `crates/liteparse-python/src/lib.rs`:++1. **GIL released** in `parse`, `parse_bytes`, `screenshot` via `py.detach(|| ...)` (pyo3 0.28's renamed `allow_threads`). All native work β Tokio `block_on`, pdfium rendering, PNG encoding β now runs without holding the GIL.++2. **Eliminated clones** in conversion path: `PyTextItem::from_rust`, `PyParsedPage::from_rust`, `PyParseResult::from_rust` now take ownership and move `String`/`Vec` fields instead of cloning. Saves N string allocs per page.++3. **Hoisted `pdfium::Library::init()`** onto `LiteParse` struct so it's done once at construction, not per `screenshot()` call. (It's a ZST guarded by `Once`, so the saving is small, but cleaner.)++Build passes. The #1 change is the high-impact one β your Scalene profile's GIL contention, `_Py_HandlePending`, and `_PySemaphore_Wait` bars should largely disappear, and any other Python threads can make progress during parsing.++---++<sub>Generated by [GitHub Copilot CLI](https://github.com/features/copilot/cli)</sub>diff --git a/crates/liteparse-python/OPTIMIZATION_PLAN.md b/crates/liteparse-python/OPTIMIZATION_PLAN.mdnew file mode 100644index 0000000..3c308de--- /dev/null+++ b/crates/liteparse-python/OPTIMIZATION_PLAN.md@@ -0,0 +1,30 @@+Looking at your Scalene profile, here are the bottlenecks I can identify:++## Key Observations++**1. I/O is the dominant cost (~0β40ms block)**+The timeline shows a solid blue I/O bar consuming roughly the first 40ms of your 170ms total run. This suggests the file read itself is a significant chunk β likely `__fread` / `_srefill` / `_sread` / `_aread` in the call stack. If you're doing unbuffered or small-chunk reads, that's low-hanging fruit.++**2. GIL contention is visible**+The GIL row in the timeline shows activity, which is notable for a Rust binding. This usually means your Rust extension is calling back into Python too frequently (e.g. building Python objects incrementally rather than bulk-returning), or you're not releasing the GIL during the heavy native work with `py.allow_threads(|| ...)`.++**3. `_Py_HandlePending` / `PyErr_CheckSignalsState` in the hot path**+These appear mid-stack and indicate Python's signal-checking mechanism is being hit repeatedly. This happens when Rust calls back into the Python C API in a tight loop β each re-entry triggers signal checks. Batching your Python object construction would eliminate most of these.++**4. Thread/semaphore wait (`_PySemaphore_Wait`, `pthread_cond_wait`)**+The right-hand call stack shows a thread sitting in `_PySemaphore_Wait` β `pthread_cond_wait`. This is a waiting thread, likely a GIL waiter or a thread pool worker stalled. If you're spawning worker threads in Rust, they may be contending more than helping for a single-file parse.++**5. `infinite_lru_cache_wrapper`**+This appears in the Python call stack. If you have a Python-side LRU cache wrapping something in the hot path, that's adding overhead β especially if it's being hit per-page or per-token.++---++## Suggested Fixes++- **Release the GIL** during your Rust parsing work: wrap the core parse loop in `py.allow_threads(|| { ... })` in PyO3. This is the single highest-impact change if you aren't already doing it.+- **Batch Python object construction** β parse everything in Rust, return a single large structure (Vec of structs), then convert to Python objects once at the end rather than per-element.+- **Buffered reads** β if you're opening the file in Rust, make sure you're using `BufReader` rather than raw file reads, which maps to the `_srefill`/`_sread` pattern you're seeing.+- **Profile the LRU cache** β check what `infinite_lru_cache_wrapper` is wrapping on the Python side and whether it can be moved into Rust or eliminated.+- **Avoid thread spawning per call** β if Rayon or a thread pool is being initialized per `parse()` invocation, use a lazy static pool instead.++The biggest win is almost certainly the GIL release + batched return pattern. pymupdf and pdftotext win here largely because their Python bindings do exactly that β all work happens natively, Python only touches the final result.diff --git a/crates/liteparse-python/src/lib.rs b/crates/liteparse-python/src/lib.rsindex 43dd255..ce171de 100644--- a/crates/liteparse-python/src/lib.rs+++ b/crates/liteparse-python/src/lib.rs@@ -56,14 +56,14 @@ impl PyTextItem {}}- fn from_rust(item: &liteparse::types::TextItem) -> Self {+ fn from_rust(item: liteparse::types::TextItem) -> Self {Self {- text: item.text.clone(),+ text: item.text,x: item.x as f64,y: item.y as f64,width: item.width as f64,height: item.height as f64,- font_name: item.font_name.clone(),+ font_name: item.font_name,font_size: item.font_size.map(|v| v as f64),confidence: item.confidence.map(|v| v as f64).or(Some(1.0)),}@@ -99,13 +99,17 @@ impl PyParsedPage {}impl PyParsedPage {- fn from_rust(page: &liteparse::types::ParsedPage) -> Self {+ fn from_rust(page: liteparse::types::ParsedPage) -> Self {Self {page_num: page.page_number as u32,width: page.page_width as f64,height: page.page_height as f64,- text: page.text.clone(),- text_items: page.text_items.iter().map(PyTextItem::from_rust).collect(),+ text: page.text,+ text_items: page+ .text_items+ .into_iter()+ .map(PyTextItem::from_rust)+ .collect(),}}}@@ -140,10 +144,14 @@ impl PyParseResult {}impl PyParseResult {- fn from_rust(result: &liteparse::parser::ParseResult) -> Self {+ fn from_rust(result: liteparse::parser::ParseResult) -> Self {Self {- pages: result.pages.iter().map(PyParsedPage::from_rust).collect(),- text: result.text.clone(),+ pages: result+ .pages+ .into_iter()+ .map(PyParsedPage::from_rust)+ .collect(),+ text: result.text,}}}@@ -184,6 +192,7 @@ struct LiteParse {inner: liteparse::parser::LiteParse,config: LiteParseConfig,runtime: tokio::runtime::Runtime,+ pdfium: pdfium::Library,}#[pymethods]@@ -262,91 +271,95 @@ impl LiteParse {let inner = liteparse::parser::LiteParse::new(cfg.clone());let runtime = tokio::runtime::Runtime::new().map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+ let pdfium = pdfium::Library::init();Ok(Self {inner,config: cfg,runtime,+ pdfium,})}/// Parse a document from a file path.- fn parse(&self, input: String) -> PyResult<PyParseResult> {+ fn parse(&self, py: Python<'_>, input: String) -> PyResult<PyParseResult> {let pdf_input = PdfInput::Path(input);- let result = self- .runtime- .block_on(self.inner.parse_input(pdf_input))+ let result = py+ .detach(|| self.runtime.block_on(self.inner.parse_input(pdf_input))).map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;- Ok(PyParseResult::from_rust(&result))+ Ok(PyParseResult::from_rust(result))}/// Parse a document from raw bytes.- fn parse_bytes(&self, data: Vec<u8>) -> PyResult<PyParseResult> {+ fn parse_bytes(&self, py: Python<'_>, data: Vec<u8>) -> PyResult<PyParseResult> {let pdf_input = PdfInput::Bytes(data);- let result = self- .runtime- .block_on(self.inner.parse_input(pdf_input))+ let result = py+ .detach(|| self.runtime.block_on(self.inner.parse_input(pdf_input))).map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;- Ok(PyParseResult::from_rust(&result))+ Ok(PyParseResult::from_rust(result))}/// Take screenshots of document pages. Returns a list of ScreenshotResult.#[pyo3(signature = (input, page_numbers = None))]fn screenshot(&self,+ py: Python<'_>,input: String,page_numbers: Option<Vec<u32>>,) -> PyResult<Vec<PyScreenshotResult>> {let dpi = self.config.dpi;- let lib = pdfium::Library::init();- let document = lib- .load_document(&input, self.config.password.as_deref())- .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;- let page_count = document.page_count() as u32;-- let pages: Vec<u32> = match page_numbers {- Some(nums) => nums,- None => (1..=page_count).collect(),- };-- let mut results = Vec::with_capacity(pages.len());- for page_num in pages {- if page_num < 1 || page_num > page_count {- return Err(PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(- "page {page_num} out of range (document has {page_count} pages)"- )));- }- let page = document- .page((page_num - 1) as i32)- .map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;- let bitmap = page- .render(dpi)+ let password = self.config.password.clone();+ py.detach(move || {+ let document = self+ .pdfium+ .load_document(&input, password.as_deref()).map_err(|e| PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string()))?;+ let page_count = document.page_count() as u32;++ let pages: Vec<u32> = match page_numbers {+ Some(nums) => nums,+ None => (1..=page_count).collect(),+ };- let width = bitmap.width() as u32;- let height = bitmap.height() as u32;- let rgba = bitmap.to_rgba();-- let mut png_buf: Vec<u8> = Vec::new();- let encoder = image::codecs::png::PngEncoder::new(&mut png_buf);- use image::ImageEncoder;- encoder- .write_image(&rgba, width, height, image::ColorType::Rgba8.into())- .map_err(|e| {- PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(format!(- "PNG encode failed: {e}"- ))+ let mut results = Vec::with_capacity(pages.len());+ for page_num in pages {+ if page_num < 1 || page_num > page_count {+ return Err(PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(+ "page {page_num} out of range (document has {page_count} pages)"+ )));+ }+ let page = document.page((page_num - 1) as i32).map_err(|e| {+ PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string())+ })?;+ let bitmap = page.render(dpi).map_err(|e| {+ PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(e.to_string())})?;- results.push(PyScreenshotResult {- page_num,- width,- height,- image_buffer: png_buf,- });- }+ let width = bitmap.width() as u32;+ let height = bitmap.height() as u32;+ let rgba = bitmap.to_rgba();++ let mut png_buf: Vec<u8> = Vec::new();+ let encoder = image::codecs::png::PngEncoder::new(&mut png_buf);+ use image::ImageEncoder;+ encoder+ .write_image(&rgba, width, height, image::ColorType::Rgba8.into())+ .map_err(|e| {+ PyErr::new::<pyo3::exceptions::PyRuntimeError, _>(format!(+ "PNG encode failed: {e}"+ ))+ })?;++ results.push(PyScreenshotResult {+ page_num,+ width,+ height,+ image_buffer: png_buf,+ });+ }- Ok(results)+ Ok(results)+ })}fn __repr__(&self) -> String {