Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
Merge remote-tracking branch 'origin/main' into feat/ocr-scanned-pdfs
Elian Doran33c8a84ad021
diff --cc apps/server/src/assets/doc_notes/en/User Guide/User Guide/Advanced Usage/Text Extraction (OCR).htmlindex 8fb65f552e,a68f9a30e5..4f60f7581b--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Advanced Usage/Text Extraction (OCR).html+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Advanced Usage/Text Extraction (OCR).html@@@ -31,104 -31,98 +31,106 @@@<li>The underlying technology is Tesseract.js.</li></ul><h3>PDFs</h3>-<p>Currently only text extraction is supported and not OCR.</p>+<p>The PDF extraction supports:</p><ul>- <li>This means that the PDF needs to have proper text information in it (i.e.- the text can be selected in a PDF viewer), whereas scanned documents are- not yet supported.</li>- <li>There are plans to integrate the same OCR-based recognition for PDFs used- for images, but this is not yet implemented.</li>+ <li>PDFs with text information (e.g. a PDF created by a browser or a word+ processing application), in which the text is extracted.</li>+ <li>Scanned PDFs, in which the text is extracted through an OCR process.</li>+ <li>Mixed PDFs, containing both text and images with text content.</li></ul>+<p>Note that OCR extraction is limited to 50 pages per PDF, pages containing+ text information (not as an image) are not subject to this limitation.</p>+<aside+class="admonition note">- <p>Text extraction for scanned PDFs using OCR was introduced in v0.104.0.++ <p>Text extraction for scanned PDFs using OCR was introduced in v0.106.0.+ If a scanned PDF was already processed by an earlier version, reprocess+ it individually via <em>Process OCR</em>; batch processing only handles- files that haven't been OCR'd yet.</p>++ files that haven't been OCR'd yet.</p>+ </aside>- <h3>Office documents</h3>- <p>The text will be extracted from the following file formats:</p>+ <h3>Office documents</h3>+ <p>The text will be extracted from the following file formats:</p>+ <ul>+ <li>Microsoft Word documents</li>+ <li>Microsoft Excel documents+ <ul>+ <li>Only the raw text information, the cell structure is not maintained.</li>+ <li>Values are extracted raw, so searching for dates will not work. The OpenDocument+ alternative will actually extract the properly formatted value.</li>+ </ul>+ </li>+ <li>Microsoft PowerPoint documents</li>+ <li>The OpenDocument alternatives to the previous formats (Text, Spreadsheet,+ Presentation), created by editors such as LibreOffice and OpenOffice.</li>+ <li><a href="https://en.wikipedia.org/wiki/Rich_Text_Format">Rich Text Format (RTF)</a>,+ since v0.104.0.</li>+ <li><a href="https://en.wikipedia.org/wiki/EPUB">EPUB</a>, since v0.104.1.</li>+ </ul>+ <h2>Configuring and triggering OCR</h2>+ <p>The OCR can be configured by going to <a class="reference-link" href="#root/_help_4TIF1oA4VQRO">Options</a> → + <a+ class="reference-link" href="#root/_hidden/_options/_optionsMedia">Media</a> and looking for the <em>Text Extraction (OCR)</em> section.</p>+ <p>There are three ways to trigger the OCR:</p>+ <ul>+ <li>By enabling <em>Auto-process new files</em> which will process only the+ notes or attachments created after enabling the option, existing files+ will remain unprocessed.</li>+ <li>By pressing <em>Start Batch Processing</em> which will process all the existing+ notes.</li>+ <li>By manually requesting for an image or file to have its text extracted,+ regardless of whether the automatic processing is enabled or not.</li>+ </ul>+ <h3>Minimum confidence</h3>+ <p>When extracting text from an image, there is a certain level of confidence+ which indicates whether the extracted text appears relevant.</p>+ <p>When the minimum confidence is set to a low percentage, the text extraction+ can interpret symbols and drawings incorrectly resulting in garbled text.</p>+ <p>If the extracted text for a note or an attachment quality is lower than+ the minimum confidence, the OCR is disregarded.</p>+ <h2>Language management</h2>+ <p>OCR needs to be aware of the language of the content in order for it to+ work correctly. The reason is that each language has its own data which+ needs to be downloaded, and accents or other symbols will not be supported+ by the default language.</p>+ <p>To configure the languages that are supported by the OCR, simply go to + <a+ class="reference-link" href="#root/_help_4TIF1oA4VQRO">Options</a> → <a class="reference-link" href="#root/_hidden/_options/_optionsLocalization">Language & Region</a> and+ adjust the <em>Content languages</em>.</p>+ <p>When there are no content languages defined, the user interface <em>Language</em> is+ used instead.</p>+ <p>After making this change, the automatic processing or manual reprocessing+ will take into consideration the new languages.</p>+ <p>To enforce the detection in a particular language for a given note, use+ the <code spellcheck="false">language</code> <a href="#root/_help_zEY4DaJG4YT5">attribute</a>,+ similar to <a href="#root/_help_veGu4faJErEM">text content language</a>. For + <a+ class="reference-link" href="#root/_help_0vhv7lsOLy82">Attachments</a>, it's not possible to manually adjust the language.</p>+ <aside+ class="admonition note">+ <p>The trained data for each language is not packaged with Trilium, as that+ would require a significant amount of space that might not be otherwise+ needed. As such, when the trained data will be downloaded automatically+ via <a href="https://github.com/naptha/tesseract.js/">Tesseract.js</a>.</p>+ <p>The downloaded trained data is located in the <a class="reference-link"+ href="#root/_help_tAassRL4RSQL">Data directory</a>, in the <code spellcheck="false">ocr-cache</code> directory.</p>+ </aside>+ <h2>Viewing extracted content for a single note</h2>+ <p>To access the extracted content of a note:</p><ul>- <li>Microsoft Word documents</li>- <li>Microsoft Excel documents- <ul>- <li>Only the raw text information, the cell structure is not maintained.</li>- <li>Values are extracted raw, so searching for dates will not work. The OpenDocument- alternative will actually extract the properly formatted value.</li>- </ul>- </li>- <li>Microsoft PowerPoint documents</li>- <li>The OpenDocument alternatives to the previous formats (Text, Spreadsheet,- Presentation), created by editors such as LibreOffice and OpenOffice.</li>- <li>Rich Text Format (RTF), since v0.104.0.</li>+ <li>For <a class="reference-link" href="#root/_help_W8vYD3Q1zjCR">File</a> notes,+ go to the <a class="reference-link" href="#root/_help_8YBEPzcpUgxw">Note buttons</a> → <em>Advanced</em> → <em>View OCR Text</em>.</li>+ <li>For <a class="reference-link" href="#root/_help_0vhv7lsOLy82">Attachments</a> (e.g. + <a+ class="reference-link" href="#root/_help_mT0HEkOsz6i1">Images</a> in <a class="reference-link" href="#root/_help_iPIMuisry3hd">Text</a> notes),+ double-click the attachment to view the details, press the […] button at+ the left and select <em>View extracted text (OCR)</em>.</li></ul>- <h2>Configuring and triggering OCR</h2>- <p>The OCR can be configured by going to <a class="reference-link" href="#root/_help_4TIF1oA4VQRO">Options</a> → - <a- class="reference-link" href="#root/_hidden/_options/_optionsMedia">Media</a> and looking for the <em>Text Extraction (OCR)</em> section.</p>- <p>There are three ways to trigger the OCR:</p>+ <p>This section allows:</p><ul>- <li>By enabling <em>Auto-process new files</em> which will process only the- notes or attachments created after enabling the option, existing files- will remain unprocessed.</li>- <li>By pressing <em>Start Batch Processing</em> which will process all the existing- notes.</li>- <li>By manually requesting for an image or file to have its text extracted,- regardless of whether the automatic processing is enabled or not.</li>- </ul>- <h3>Minimum confidence</h3>- <p>When extracting text from an image, there is a certain level of confidence- which indicates whether the extracted text appears relevant.</p>- <p>When the minimum confidence is set to a low percentage, the text extraction- can interpret symbols and drawings incorrectly resulting in garbled text.</p>- <p>If the extracted text for a note or an attachment quality is lower than- the minimum confidence, the OCR is disregarded.</p>- <h2>Language management</h2>- <p>OCR needs to be aware of the language of the content in order for it to- work correctly. The reason is that each language has its own data which- needs to be downloaded, and accents or other symbols will not be supported- by the default language.</p>- <p>To configure the languages that are supported by the OCR, simply go to - <a- class="reference-link" href="#root/_help_4TIF1oA4VQRO">Options</a> → <a class="reference-link" href="#root/_hidden/_options/_optionsLocalization">Language & Region</a> and- adjust the <em>Content languages</em>.</p>- <p>When there are no content languages defined, the user interface <em>Language</em> is- used instead.</p>- <p>After making this change, the automatic processing or manual reprocessing- will take into consideration the new languages.</p>- <p>To enforce the detection in a particular language for a given note, use- the <code spellcheck="false">language</code> <a href="#root/_help_zEY4DaJG4YT5">attribute</a>,- similar to <a href="#root/_help_veGu4faJErEM">text content language</a>. For - <a- class="reference-link" href="#root/_help_0vhv7lsOLy82">Attachments</a>, it's not possible to manually adjust the language.</p>- <aside- class="admonition note">- <p>The trained data for each language is not packaged with Trilium, as that- would require a significant amount of space that might not be otherwise- needed. As such, when the trained data will be downloaded automatically- via <a href="https://github.com/naptha/tesseract.js/">Tesseract.js</a>.</p>- <p>The downloaded trained data is located in the <a class="reference-link"- href="#root/_help_tAassRL4RSQL">Data directory</a>, in the <code spellcheck="false">ocr-cache</code> directory.</p>- </aside>- <h2>Viewing extracted content for a single note</h2>- <p>To access the extracted content of a note:</p>- <ul>- <li>For <a class="reference-link" href="#root/_help_W8vYD3Q1zjCR">File</a> notes,- go to the <a class="reference-link" href="#root/_help_8YBEPzcpUgxw">Note buttons</a> → <em>Advanced</em> → <em>View OCR Text</em>.</li>- <li>For <a class="reference-link" href="#root/_help_0vhv7lsOLy82">Attachments</a> (e.g. - <a- class="reference-link" href="#root/_help_mT0HEkOsz6i1">Images</a> in <a class="reference-link" href="#root/_help_iPIMuisry3hd">Text</a> notes),- double-click the attachment to view the details, press the […] button at- the left and select <em>View extracted text (OCR)</em>.</li>- </ul>- <p>This section allows:</p>- <ul>- <li>Viewing the extracted text, which can be copied elsewhere if needed or- just to check the quality of the extraction.</li>- <li>If the note has not been extracted yet, pressing <em>Process OCR</em> will- process it in the background. If the extraction confidence is lower than- the minimum confidence, there will be a notification.</li>- <li>Similarly, if the minimum confidence was changed in settings, it is possible- to press the <em>Process OCR</em> button again to extract the text again.</li>- </ul>+ <li>Viewing the extracted text, which can be copied elsewhere if needed or+ just to check the quality of the extraction.</li>+ <li>If the note has not been extracted yet, pressing <em>Process OCR</em> will+ process it in the background. If the extraction confidence is lower than+ the minimum confidence, there will be a notification.</li>+ <li>Similarly, if the minimum confidence was changed in settings, it is possible+ to press the <em>Process OCR</em> button again to extract the text again.</li>+ </ul>diff --cc apps/server/src/services/ocr/processors/pdf_processor.tsindex 70ccd8ccd7,1ca486dc7e..ef84c2b8b4--- a/apps/server/src/services/ocr/processors/pdf_processor.ts+++ b/apps/server/src/services/ocr/processors/pdf_processor.ts@@@ -1,41 -1,9 +1,39 @@@import { getLog } from "@triliumnext/core";- import { Jimp } from "jimp";- import { extractImages, extractText, getDocumentProxy } from 'unpdf';+import { OCRProcessingOptions, OCRResult } from '../ocr_service.js';+import recognizer from '../tesseract_recognizer.js';import { FileProcessor } from './file_processor.js';- type PdfDocument = Awaited<ReturnType<typeof getDocumentProxy>>;++type PdfDocument = Awaited<ReturnType<typeof import('unpdf').getDocumentProxy>>;++/** A page whose embedded text is shorter than this (after trimming) is treated as scanned and sent to OCR. */+const MIN_EMBEDDED_PAGE_CHARS = 16;++/**+ * Upper bound on how many scanned pages a single PDF may OCR. Rasterizing and+ * recognizing a page costs seconds and non-trivial memory, so an unbounded scan+ * of a large document could stall the OCR queue. Pages beyond the cap keep+ * whatever embedded text they have (usually none) and are logged, never silently+ * dropped.+ */+const MAX_OCR_PAGES = 50;++/** An embedded-text page is exact, so it is reported at the same high confidence the previous text-only path used. */+const EMBEDDED_TEXT_CONFIDENCE = 0.99;+/**- * PDF processor for extracting embedded text from PDF files using unpdf.+ * Embedded images smaller than this on either side are skipped: scanned pages are+ * full-page raster, whereas tiny images are almost always icons, bullets or rules+ * that hold no recognizable text and would only waste an OCR pass.+ */+const MIN_OCR_IMAGE_DIM = 50;++/**+ * PDF processor. Prefers the PDF's embedded text layer (fast and exact) and+ * falls back to OCR for scanned, image-only pages by extracting each page's+ * embedded images and running them through the shared Tesseract recognizer.+ * Detection is per page, so mixed PDFs (some real-text pages, some scans) are+ * handled correctly.*/export class PDFProcessor extends FileProcessor {@@@ -50,59 -18,17 +48,61 @@@async extractText(buffer: Buffer, options: OCRProcessingOptions = {}): Promise<OCRResult> {getLog().info('Starting PDF text extraction...');+ const language = options.language || "eng";+ // Dynamically imported so unpdf only loads when a PDF is actually processed.+ const { extractText, getDocumentProxy } = await import('unpdf');const pdf = await getDocumentProxy(new Uint8Array(buffer));- const { totalPages, text } = await extractText(pdf, { mergePages: true });- const trimmed = text.trim();+ const { totalPages, text: pageTexts } = await extractText(pdf, { mergePages: false });++ const pageResults: string[] = [];+ const pageConfidences: number[] = [];+ let ocrPagesProcessed = 0;+ let ocrPagesSkipped = 0;++ for (let pageNum = 1; pageNum <= totalPages; pageNum++) {+ const embedded = (pageTexts[pageNum - 1] ?? "").trim();++ if (embedded.length >= MIN_EMBEDDED_PAGE_CHARS) {+ pageResults.push(embedded);+ pageConfidences.push(EMBEDDED_TEXT_CONFIDENCE);+ continue;+ }++ if (ocrPagesProcessed >= MAX_OCR_PAGES) {+ ocrPagesSkipped++;+ if (embedded.length > 0) {+ pageResults.push(embedded);+ pageConfidences.push(EMBEDDED_TEXT_CONFIDENCE);+ }+ continue;+ }++ ocrPagesProcessed++;+ const ocr = await this.ocrPage(pdf, pageNum, language);+ if (ocr.text.length > 0) {+ pageResults.push(ocr.text);+ pageConfidences.push(ocr.confidence);+ } else if (embedded.length > 0) {+ // Nothing OCR-able on the page; keep whatever little embedded text it had.+ pageResults.push(embedded);+ pageConfidences.push(EMBEDDED_TEXT_CONFIDENCE);+ }+ }++ if (ocrPagesSkipped > 0) {+ getLog().info(`PDF OCR page cap reached: OCR'd ${MAX_OCR_PAGES} page(s), skipped ${ocrPagesSkipped} further scanned page(s).`);+ }++ const text = pageResults.join("\n\n").trim();+ const confidence = text.length === 0 || pageConfidences.length === 0+ ? 0+ : pageConfidences.reduce((sum, c) => sum + c, 0) / pageConfidences.length;return {- text: trimmed,- confidence: trimmed.length > 0 ? 0.99 : 0,+ text,+ confidence,extractedAt: new Date().toISOString(),- language: options.language || "eng",+ language,pageCount: totalPages};}@@@ -111,90 -37,4 +111,93 @@@return 'pdf';}+ /**+ * OCR a single scanned page by recognizing each embedded image it paints and+ * concatenating the results. Failures on an individual page are logged and+ * treated as "no text" so one bad page never aborts the whole document.+ */+ private async ocrPage(pdf: PdfDocument, pageNum: number, language: string): Promise<{ text: string; confidence: number }> {+ try {++ const { extractImages } = await import('unpdf');+ const images = await extractImages(pdf, pageNum);+ if (images.length === 0) {+ return { text: "", confidence: 0 };+ }++ const parts: string[] = [];+ const confidences: number[] = [];++ for (const image of images) {+ if (image.width < MIN_OCR_IMAGE_DIM || image.height < MIN_OCR_IMAGE_DIM) {+ continue;+ }+ const png = await toPngBuffer(image);+ const { text, confidence } = await recognizer.recognize(png, language);+ if (text.length > 0) {+ parts.push(text);+ confidences.push(confidence);+ }+ }++ return {+ text: parts.join("\n"),+ confidence: confidences.length > 0+ ? confidences.reduce((sum, c) => sum + c, 0) / confidences.length+ : 0+ };+ } catch (error) {+ getLog().error(`PDF OCR failed for page ${pageNum}: ${error}`);+ return { text: "", confidence: 0 };+ }+ }+}++interface ExtractedImage {+ data: Uint8ClampedArray;+ width: number;+ height: number;+ // Typed as a plain number rather than `1 | 3 | 4`: the value is whatever unpdf+ // hands us at runtime, so it is validated below rather than trusted here.+ channels: number;+}++/**+ * Encode a raw image extracted from a PDF into a PNG buffer that Tesseract can+ * decode. unpdf returns 1- (grayscale), 3- (RGB) or 4-channel (RGBA) pixel data;+ * Jimp bitmaps are always RGBA, so narrower formats are expanded here. An+ * unexpected channel count throws rather than silently producing garbled pixels.+ */+async function toPngBuffer(image: ExtractedImage): Promise<Buffer> {+ const { data, width, height, channels } = image;+ const pixelCount = width * height;++ let rgba: Buffer;+ if (channels === 4) {+ // Already RGBA — reuse the underlying bytes instead of copying pixel by pixel.+ rgba = Buffer.from(data.buffer, data.byteOffset, data.byteLength);+ } else if (channels === 3 || channels === 1) {+ rgba = Buffer.alloc(pixelCount * 4);+ for (let i = 0; i < pixelCount; i++) {+ const out = i * 4;+ if (channels === 1) {+ const value = data[i];+ rgba[out] = value;+ rgba[out + 1] = value;+ rgba[out + 2] = value;+ } else {+ const src = i * 3;+ rgba[out] = data[src];+ rgba[out + 1] = data[src + 1];+ rgba[out + 2] = data[src + 2];+ }+ rgba[out + 3] = 255;+ }+ } else {+ throw new Error(`Unsupported image channel count: ${channels}`);+ }+++ // Dynamically imported so jimp only loads when a scanned page is actually rasterized.++ const { Jimp } = await import("jimp");+ const jimpImage = Jimp.fromBitmap({ data: rgba, width, height });+ return jimpImage.getBuffer("image/png");}diff --cc apps/server/src/services/ocr/tesseract_recognizer.spec.tsindex f0c179efdf,0000000000..39734ce0a4mode 100644,000000..100644--- a/apps/server/src/services/ocr/tesseract_recognizer.spec.ts+++ b/apps/server/src/services/ocr/tesseract_recognizer.spec.ts@@@ -1,251 -1,0 +1,294 @@@+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';++// Mock Tesseract.js so no real OCR model is ever loaded.+const mockWorker = {+ recognize: vi.fn(),+ terminate: vi.fn().mockResolvedValue(undefined)+};++const mockTesseract = {+ createWorker: vi.fn()+};++vi.mock('tesseract.js', () => ({+ default: mockTesseract+}));++// Avoid touching the real filesystem for the worker cache directory.+vi.mock('fs', () => ({+ default: {+ mkdirSync: vi.fn()+ }+}));++vi.mock('../data_dir.js', () => ({+ default: {+ OCR_CACHE_DIR: '/tmp/trilium-ocr-test-cache'+ }+}));++const mockOptions = {+ getOption: vi.fn()+};++const mockLog = {+ info: vi.fn(),+ error: vi.fn()+};++vi.mock('@triliumnext/core', async (importOriginal) => {+ const actual = await importOriginal<typeof import('@triliumnext/core')>();+ return {+ ...actual,+ options: mockOptions,+ getLog: () => mockLog+ };+});++// A fresh singleton is imported per test so worker state never leaks between them.+let recognizer: typeof import('./tesseract_recognizer.js').default;++beforeEach(async () => {+ vi.resetModules();+ vi.clearAllMocks();+ mockOptions.getOption.mockReturnValue('0');+ mockTesseract.createWorker.mockResolvedValue(mockWorker);+ recognizer = (await import('./tesseract_recognizer.js')).default;+});++afterEach(() => {+ vi.restoreAllMocks();+});++const image = Buffer.from('fake-image');+++/**++ * A recognition result in the shape tesseract.js actually answers with when the `blocks` output++ * format is requested: the words are nested under blocks → paragraphs → lines, and there is no++ * top-level `words` array to read them from.++ */++function pageWithLines(confidence: number, lines: [ text: string, confidence: number ][][]) {++ return {++ text: lines.map(line => line.map(([ text ]) => text).join(' ')).join('\n'),++ confidence,++ blocks: [ {++ paragraphs: [ {++ lines: lines.map(line => ({++ text: line.map(([ text ]) => text).join(' '),++ words: line.map(([ text, wordConfidence ]) => ({ text, confidence: wordConfidence }))++ }))++ } ]++ } ]++ };++}++++/** What a recognition with no readable text answers with: text, a score, and `blocks: null`. */++function pageWithoutBlocks(text: string, confidence: number) {++ return { text, confidence, blocks: null };++}+++describe('TesseractRecognizer', () => {+ it('recognizes text and reports overall confidence when no threshold is set', async () => {+ mockWorker.recognize.mockResolvedValue({- data: { text: ' hello world ', confidence: 88, words: [] }++ data: pageWithoutBlocks(' hello world ', 88)+ });++ const result = await recognizer.recognize(image, 'eng');++ expect(result.text).toBe('hello world');+ expect(result.confidence).toBeCloseTo(0.88);+ expect(mockTesseract.createWorker).toHaveBeenCalledWith(+ 'eng',+ 1,+ expect.objectContaining({ cachePath: '/tmp/trilium-ocr-test-cache' })+ );+ });+++ it('asks tesseract for the per-word breakdown the confidence filter needs', async () => {++ mockWorker.recognize.mockResolvedValue({ data: pageWithLines(80, [ [ [ 'x', 80 ] ] ]) });++++ await recognizer.recognize(image, 'eng');++++ // Without this output format the result carries the plain text alone, and every image is++ // judged by its mean confidence instead of word by word.++ expect(mockWorker.recognize).toHaveBeenCalledWith(image, {}, { blocks: true });++ });+++ it('reuses the worker for the same language and recreates it when the language changes', async () => {+ mockWorker.recognize.mockResolvedValue({- data: { text: 'a', confidence: 50, words: [] }++ data: pageWithLines(50, [ [ [ 'a', 50 ] ] ])+ });++ await recognizer.recognize(image, 'eng');+ await recognizer.recognize(image, 'eng');+ expect(mockTesseract.createWorker).toHaveBeenCalledTimes(1);+ expect(mockWorker.terminate).not.toHaveBeenCalled();++ await recognizer.recognize(image, 'deu');+ expect(mockWorker.terminate).toHaveBeenCalledTimes(1);+ expect(mockTesseract.createWorker).toHaveBeenCalledTimes(2);+ });++ it('serializes overlapping calls so a language switch cannot tear down an in-flight worker', async () => {+ // Hold the first recognition open so the second call is issued while it runs.+ let releaseFirst: (value: unknown) => void = () => {};+ const firstRecognition = new Promise((resolve) => { releaseFirst = resolve; });+ mockWorker.recognize+ .mockReturnValueOnce(firstRecognition)- .mockResolvedValue({ data: { text: 'second', confidence: 50, words: [] } });++ .mockResolvedValue({ data: pageWithLines(50, [ [ [ 'second', 50 ] ] ]) });++ const p1 = recognizer.recognize(image, 'eng');+ const p2 = recognizer.recognize(image, 'deu');++ // Let microtasks settle while the first job is still running.+ await new Promise((resolve) => setTimeout(resolve, 0));++ // The second (different-language) job must not have started: no second worker+ // was created and the in-flight worker was not terminated mid-recognition.+ expect(mockTesseract.createWorker).toHaveBeenCalledTimes(1);+ expect(mockWorker.terminate).not.toHaveBeenCalled();+- releaseFirst({ data: { text: 'first', confidence: 50, words: [] } });++ releaseFirst({ data: pageWithLines(50, [ [ [ 'first', 50 ] ] ]) });+ const [r1, r2] = await Promise.all([p1, p2]);++ expect(r1.text).toBe('first');+ expect(r2.text).toBe('second');+ // Only after the first job finished did the language switch tear down and rebuild.+ expect(mockWorker.terminate).toHaveBeenCalledTimes(1);+ expect(mockTesseract.createWorker).toHaveBeenCalledTimes(2);+ });++ it('invokes the recognizing-text logger callback', async () => {+ mockWorker.recognize.mockResolvedValue({- data: { text: 'a', confidence: 50, words: [] }++ data: pageWithLines(50, [ [ [ 'a', 50 ] ] ])+ });++ await recognizer.recognize(image, 'eng');++ const config = mockTesseract.createWorker.mock.calls[0][2];+ config.logger({ status: 'recognizing text', progress: 0.5 });+ config.logger({ status: 'loading', progress: 0.1 });++ expect(mockLog.info).toHaveBeenCalledWith(+ expect.stringContaining('OCR progress')+ );+ });++ it('passes an errorHandler that logs worker errors instead of rethrowing them', async () => {+ mockWorker.recognize.mockResolvedValue({- data: { text: 'a', confidence: 50, words: [] }++ data: pageWithLines(50, [ [ [ 'a', 50 ] ] ])+ });++ await recognizer.recognize(image, 'eng');++ // Without an errorHandler, tesseract.js turns job failures into uncaught+ // exceptions, which surface as Electron's "JavaScript error" dialog (#9754).+ const config = mockTesseract.createWorker.mock.calls[0][2];+ expect(() => config.errorHandler('Error attempting to read image.')).not.toThrow();+ expect(mockLog.error).toHaveBeenCalledWith(+ 'Tesseract worker error: Error attempting to read image.'+ );+ });++ it('propagates recognition errors to the caller', async () => {+ mockWorker.recognize.mockRejectedValue(new Error('recognize failed'));++ await expect(recognizer.recognize(image, 'eng')).rejects.toThrow('recognize failed');+ });++ describe('confidence filtering', () => {- it('keeps only words above the configured threshold', async () => {++ it('keeps the words above the threshold on the lines they were read on', async () => {+ mockOptions.getOption.mockReturnValue('0.8');+ mockWorker.recognize.mockResolvedValue({- data: {- text: 'good bad good',- confidence: 70,- words: [- { text: 'good', confidence: 90 },- { text: 'bad', confidence: 50 },- { text: 'good', confidence: 95 }- ]- }++ data: pageWithLines(70, [++ [ [ 'good', 90 ], [ 'bad', 50 ] ],++ [ [ 'also', 95 ], [ 'good', 85 ] ]++ ])+ });++ const result = await recognizer.recognize(image, 'eng');+- expect(result.text).toBe('good good');- expect(result.confidence).toBeCloseTo((0.9 + 0.95) / 2);++ expect(result.text).toBe('good\nalso good');++ expect(result.confidence).toBeCloseTo((0.9 + 0.95 + 0.85) / 3);+ });+- it('returns empty confidence when no words pass the threshold', async () => {- mockOptions.getOption.mockReturnValue('0.99');++ it('keeps well-read words on an image whose mean confidence is below the threshold', async () => {++ // The mean is dragged under the line by the marks Tesseract misread as text, which is++ // exactly what the per-word filter is for — judging the image by that mean instead++ // would throw away every word on it.++ mockOptions.getOption.mockReturnValue('0.75');+ mockWorker.recognize.mockResolvedValue({- data: {- text: 'low',- confidence: 10,- words: [{ text: 'low', confidence: 10 }]- }++ data: pageWithLines(60, [ [ [ 'Invoice', 96 ], [ '®', 0 ], [ 'total', 94 ] ] ])+ });++ const result = await recognizer.recognize(image, 'eng');+- expect(result.text).toBe('');- expect(result.confidence).toBe(0);++ expect(result.text).toBe('Invoice total');+ });+- it('handles an empty word array with a threshold set', async () => {- mockOptions.getOption.mockReturnValue('0.5');++ it('leaves no blank line where every word of one was dropped', async () => {++ mockOptions.getOption.mockReturnValue('0.8');++ mockWorker.recognize.mockResolvedValue({++ data: pageWithLines(70, [++ [ [ 'kept', 90 ] ],++ [ [ 'noise', 10 ] ],++ [ [ 'also-kept', 90 ] ]++ ])++ });++++ const result = await recognizer.recognize(image, 'eng');++++ expect(result.text).toBe('kept\nalso-kept');++ });++++ it('returns empty text and no confidence when no word passes the threshold', async () => {++ mockOptions.getOption.mockReturnValue('0.99');+ mockWorker.recognize.mockResolvedValue({- data: { text: 'ignored', confidence: 80, words: [] }++ data: pageWithLines(10, [ [ [ 'low', 10 ] ] ])+ });++ const result = await recognizer.recognize(image, 'eng');++ expect(result.text).toBe('');+ expect(result.confidence).toBe(0);+ });+- it('falls back to overall confidence when there is no word-level data', async () => {++ it('falls back to overall confidence when nothing was broken down into words', async () => {+ mockOptions.getOption.mockReturnValue('0.5');+ mockWorker.recognize.mockResolvedValue({- data: { text: ' whole text ', confidence: 80, words: undefined }++ data: pageWithoutBlocks(' whole text ', 80)+ });++ const result = await recognizer.recognize(image, 'eng');++ expect(result.text).toBe('whole text');+ expect(result.confidence).toBeCloseTo(0.8);+ });++ it('drops all text via the fallback when overall confidence is too low', async () => {+ mockOptions.getOption.mockReturnValue('0.9');+ mockWorker.recognize.mockResolvedValue({- data: { text: 'whole text', confidence: 40, words: undefined }++ data: pageWithoutBlocks('whole text', 40)+ });++ const result = await recognizer.recognize(image, 'eng');++ expect(result.text).toBe('');+ expect(result.confidence).toBeCloseTo(0.4);+ expect(mockLog.info).toHaveBeenCalledWith(+ expect.stringContaining('Entire text filtered out')+ );+ });++ it('defaults the threshold to 0 when the option is null', async () => {+ mockOptions.getOption.mockReturnValue(null);+ mockWorker.recognize.mockResolvedValue({- data: { text: 'kept', confidence: 30, words: [{ text: 'kept', confidence: 30 }] }++ data: pageWithLines(30, [ [ [ 'kept', 30 ] ] ])+ });++ const result = await recognizer.recognize(image, 'eng');++ expect(result.text).toBe('kept');+ });+ });+});diff --cc apps/server/src/services/ocr/tesseract_recognizer.tsindex 850e71d2a3,0000000000..145b93dc5bmode 100644,000000..100644--- a/apps/server/src/services/ocr/tesseract_recognizer.ts+++ b/apps/server/src/services/ocr/tesseract_recognizer.ts@@@ -1,161 -1,0 +1,186 @@@+import { getLog, options } from '@triliumnext/core';+import fs from 'fs';+import Tesseract from 'tesseract.js';++import dataDirs from '../data_dir.js';++export interface RecognitionResult {+ /** Recognized text after per-word confidence filtering. */+ text: string;+ /** Overall confidence in the range [0, 1] of the kept text. */+ confidence: number;+}+- /** The subset of Tesseract's page result this recognizer relies on. */- interface RecognizedData {- text: string;- /** Overall page confidence, 0–100. */- confidence: number;- /** Per-word data; absent in some Tesseract output configurations. */- words?: Array<{ text: string; confidence: number }>;- }-+/**+ * Owns a single long-lived Tesseract.js worker and the confidence filtering+ * shared by every OCR path that recognizes raster images — standalone image+ * notes/attachments and, page by page, scanned PDFs. Keeping one worker (rather+ * than one per processor) avoids paying the model-load cost repeatedly and lets+ * an image and a PDF page reuse the same loaded language data.+ */+class TesseractRecognizer {+ private worker: Tesseract.Worker | null = null;+ private currentLanguage: string | null = null;+ // Serializes recognition jobs. The worker is a single shared resource and a+ // language change tears it down and rebuilds it, so two overlapping calls (e.g.+ // a manual reprocess arriving mid-batch) could otherwise terminate a worker that+ // another call is mid-recognition with. Chaining also bounds OCR to one CPU-heavy+ // job at a time.+ private queue: Promise<unknown> = Promise.resolve();++ /**+ * Recognize text in an encoded image buffer (PNG/JPEG/etc.) for the given+ * Tesseract language code(s), applying the configured confidence filtering.+ * Calls are queued so only one recognition runs at a time.+ */+ recognize(image: Buffer, language: string): Promise<RecognitionResult> {+ const result = this.queue.then(async () => {+ const worker = await this.ensureWorker(language);- const { data } = await worker.recognize(image);++ // The per-word breakdown the confidence filter runs on is an opt-in output format:++ // by default `recognize` answers with the plain text and nothing else, leaving the++ // filter with only the whole image's mean confidence to judge by.++ const { data } = await worker.recognize(image, {}, { blocks: true });+ return this.filterTextByConfidence(data);+ });+ // Keep the chain going regardless of this job's outcome, without swallowing+ // the result/error the caller receives.+ this.queue = result.catch(() => {});+ return result;+ }++ /**+ * Ensures a Tesseract worker is ready for the given language.+ * Creates a new worker if none exists or if the language has changed.+ */+ private async ensureWorker(language: string): Promise<Tesseract.Worker> {+ if (this.worker && this.currentLanguage === language) {+ return this.worker;+ }++ if (this.worker) {+ await this.worker.terminate();+ this.worker = null;+ }++ fs.mkdirSync(dataDirs.OCR_CACHE_DIR, { recursive: true });++ getLog().info(`Initializing Tesseract worker for language(s): ${language}`);+ const worker = await Tesseract.createWorker(language, 1, {+ cachePath: dataDirs.OCR_CACHE_DIR,+ // Without an errorHandler, tesseract.js rethrows job failures (e.g. undecodable+ // images) from its worker message handler as uncaught exceptions — in the desktop+ // app that surfaces as Electron's blocking "JavaScript error" dialog (#9754).+ // The job promise still rejects, so callers handle the failure normally.+ errorHandler: (error: unknown) => {+ getLog().error(`Tesseract worker error: ${error}`);+ },+ logger: (m: { status: string; progress: number }) => {+ if (m.status === 'recognizing text') {+ getLog().info(`OCR progress (${language}): ${Math.round(m.progress * 100)}%`);+ }+ }+ });+ this.worker = worker;+ this.currentLanguage = language;+ return worker;+ }++ /**- * Filter text based on minimum confidence threshold++ * Filter text based on minimum confidence threshold.++ *++ * Word by word, because the words Tesseract is least sure of are usually not words at all —++ * a border read as punctuation, a speck read as a comma — and it is those that drag the mean++ * for the image down. Judging the image by that mean throws away every well-read word on it++ * along with them, so what is kept is decided per word and reassembled line by line, leaving++ * the text laid out as it is on the picture.+ */- private filterTextByConfidence(data: RecognizedData): RecognitionResult {++ private filterTextByConfidence(data: Tesseract.Page): RecognitionResult {+ const minConfidence = this.getMinConfidenceThreshold();++ // If no minimum confidence set, return original text+ if (minConfidence <= 0) {+ return {+ text: data.text.trim(),+ confidence: data.confidence / 100+ };+ }+- const filteredWords: string[] = [];- const validConfidences: number[] = [];++ const lines = getRecognizedLines(data);+- // Tesseract provides word-level data- if (data.words && Array.isArray(data.words)) {- for (const word of data.words) {- const wordConfidence = word.confidence / 100; // Convert to decimal-- if (wordConfidence >= minConfidence) {- filteredWords.push(word.text);- validConfidences.push(wordConfidence);- }- }- } else {- // Fallback: if word-level data not available, use overall confidence++ // Nothing was broken down into words — an image nothing could be read from. There is only++ // the one score to go by, so the text stands or falls as a whole.++ if (lines.length === 0) {+ const overallConfidence = data.confidence / 100;+ if (overallConfidence >= minConfidence) {+ return {+ text: data.text.trim(),+ confidence: overallConfidence+ };+ }+ getLog().info(`Entire text filtered out due to low confidence ${overallConfidence} (below threshold ${minConfidence})`);+ return {+ text: '',+ confidence: overallConfidence+ };+ }+++ const keptLines: string[] = [];++ const keptConfidences: number[] = [];++ let totalWords = 0;++++ for (const line of lines) {++ const keptWords: string[] = [];++++ for (const word of line.words ?? []) {++ totalWords++;++ const wordConfidence = word.confidence / 100; // Convert to decimal++++ if (wordConfidence >= minConfidence) {++ keptWords.push(word.text);++ keptConfidences.push(wordConfidence);++ }++ }++++ // A line every word of which was dropped leaves no blank behind: the gap would read as++ // spacing on the picture that isn't there.++ if (keptWords.length > 0) {++ keptLines.push(keptWords.join(' '));++ }++ }+++ // Calculate average confidence of accepted words- const averageConfidence = validConfidences.length > 0- ? validConfidences.reduce((sum, conf) => sum + conf, 0) / validConfidences.length++ const averageConfidence = keptConfidences.length > 0++ ? keptConfidences.reduce((sum, conf) => sum + conf, 0) / keptConfidences.length+ : 0;+- const filteredText = filteredWords.join(' ').trim();-- getLog().info(`Filtered OCR text: ${filteredWords.length} words kept out of ${data.words?.length || 0} total words (min confidence: ${minConfidence})`);++ getLog().info(`Filtered OCR text: ${keptConfidences.length} words kept out of ${totalWords} total words (min confidence: ${minConfidence})`);++ return {- text: filteredText,++ text: keptLines.join('\n').trim(),+ confidence: averageConfidence+ };+ }++ /**+ * Get minimum confidence threshold from options+ */+ private getMinConfidenceThreshold(): number {+ const minConfidence = options.getOption('ocrMinConfidence') ?? 0;+ return parseFloat(minConfidence);+ }+}++export default new TesseractRecognizer();++++/**++ * The words Tesseract read, in the lines it read them on.++ *++ * They sit three levels down the page it describes, and that description is only present when the++ * `blocks` output format was asked for — see {@link TesseractRecognizer.recognize}. Without it, and++ * for an image nothing was read from, this is empty.++ */++function getRecognizedLines(data: Tesseract.Page): Tesseract.Line[] {++ return (data.blocks ?? []).flatMap(++ block => (block.paragraphs ?? []).flatMap(paragraph => paragraph.lines ?? [])++ );++}diff --cc docs/User Guide/User Guide/Advanced Usage/Text Extraction (OCR).mdindex 5a18cfd006,b45f484795..14f0faac78--- a/docs/User Guide/User Guide/Advanced Usage/Text Extraction (OCR).md+++ b/docs/User Guide/User Guide/Advanced Usage/Text Extraction (OCR).md@@@ -27,16 -27,10 +27,16 @@@ OCR in Trilium supports the following f### PDFs-Currently only text extraction is supported and not OCR.+The PDF extraction supports:-* This means that the PDF needs to have proper text information in it (i.e. the text can be selected in a PDF viewer), whereas scanned documents are not yet supported.-* There are plans to integrate the same OCR-based recognition for PDFs used for images, but this is not yet implemented.+* PDFs with text information (e.g. a PDF created by a browser or a word processing application), in which the text is extracted.+* Scanned PDFs, in which the text is extracted through an OCR process.+* Mixed PDFs, containing both text and images with text content.++Note that OCR extraction is limited to 50 pages per PDF, pages containing text information (not as an image) are not subject to this limitation.++> [!NOTE]- > Text extraction for scanned PDFs using OCR was introduced in v0.104.0. If a scanned PDF was already processed by an earlier version, reprocess it individually via _Process OCR_; batch processing only handles files that haven't been OCR'd yet.++> Text extraction for scanned PDFs using OCR was introduced in v0.106.0. If a scanned PDF was already processed by an earlier version, reprocess it individually via _Process OCR_; batch processing only handles files that haven't been OCR'd yet.### Office documents