The minimalistic document archiving platform.
feat(lecture): normalize image rotation before OCR (#1469)
Corentin Thomasset347029dbd1ae
diff --git a/.changeset/five-donkeys-wonder.md b/.changeset/five-donkeys-wonder.mdnew file mode 100644index 00000000..736ccd92--- /dev/null+++ b/.changeset/five-donkeys-wonder.md@@ -0,0 +1,6 @@+---+'@papra/app': patch+'@papra/lecture': patch+---++Fixed OCR extraction for images whose rotation is stored in EXIF metadata.diff --git a/packages/lecture/fixtures/022-jpg-exif-rotation/022.expected.txt b/packages/lecture/fixtures/022-jpg-exif-rotation/022.expected.txtnew file mode 100644index 00000000..85877744--- /dev/null+++ b/packages/lecture/fixtures/022-jpg-exif-rotation/022.expected.txt@@ -0,0 +1,41 @@+at his touch of a certain icy pang along my blood. “Come, sir,” said I.+“You forget that I have not yet the pleasure of your acquaintance. Be+seated, if you please.” And I showed him an example, and sat down+myself in my customary seat and with as fair an imitation of my or-+dinary manner to a patient, as the lateness of the hour, the nature of+my preoccupations, and the horror I had of my visitor, would suffer+me to muster.++“I beg your pardon, Dr. Lanyon,” he replied civilly enough. “What+you say is very well founded; and my impatience has shown its heels+to my politeness. I come here at the instance of your colleague, Dr.+Henry Jekyll, on a piece of business of some moment; and I under-+stood...” He paused and put his hand to his throat, and I could see,+in spite of his collected manner, that he was wrestling against the+approaches of the hysteria—“T understood, a drawer...”++But here I took pity on my visitor's suspense, and some perhaps+on my own growing curiosity.++“There it is, sir,” said I, pointing to the drawer, where it lay on the+floor behind a table and still covered with the sheet.++He sprang to it, and then paused, and laid his hand upon his+heart: I could hear his teeth grate with the convulsive action of his+jaws; and his face was so ghastly to see that I grew alarmed both for+his life and reason.++“Compose yourself,” said I.++He turned a dreadful smile to me, and as if with the decision of+despair, plucked away the sheet. At sight of the contents, he uttered+one loud sob of such immense relief that I sat petrified. And the+next moment, in a voice that was already fairly well under control,+“Have you a graduated glass?” he asked.++I rose from my place with something of an effort and gave him+what he asked.++He thanked me with a smiling nod, measured out a few min-+ims of the red tincture and added one of the powders. The mix-+ture, which was at first of a reddish hue, began, in proportion as the\ No newline at end of filediff --git a/packages/lecture/fixtures/022-jpg-exif-rotation/022.input.jpg b/packages/lecture/fixtures/022-jpg-exif-rotation/022.input.jpgnew file mode 100644index 00000000..3c54d13eBinary files /dev/null and b/packages/lecture/fixtures/022-jpg-exif-rotation/022.input.jpg differdiff --git a/packages/lecture/src/extractors/img.extractor.ts b/packages/lecture/src/extractors/img.extractor.tsindex e05a1155..7ba27907 100644--- a/packages/lecture/src/extractors/img.extractor.ts+++ b/packages/lecture/src/extractors/img.extractor.ts@@ -1,13 +1,35 @@+import sharp from 'sharp';import { defineTextExtractor } from '../extractors.models';import { createTesseractExtractor } from '../tesseract/tesseract.usecases';+import type { Logger } from '../types';++async function normalizeImageRotation(+ arrayBuffer: ArrayBuffer,+ { logger }: { logger?: Logger },+): Promise<Buffer | ArrayBuffer> {+ try {+ const image = sharp(arrayBuffer);+ const { orientation } = await image.metadata();+ const shouldNormalizeRotation = orientation !== undefined && orientation !== 1;+ return shouldNormalizeRotation ? await image.rotate().toBuffer() : arrayBuffer;+ } catch (error) {+ logger?.error(+ { error },+ 'Failed to normalize image rotation, trying to extract text from the original image',+ );+ return arrayBuffer;+ }+}export const imageExtractorDefinition = defineTextExtractor({name: 'image',mimeTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'],- extract: async ({ arrayBuffer, config }) => {+ extract: async ({ arrayBuffer, config, logger }) => {const { extract, extractorType } = await createTesseractExtractor(config.tesseract);- const content = await extract(arrayBuffer);+ const normalizedImage = await normalizeImageRotation(arrayBuffer, { logger });++ const content = await extract(normalizedImage);return {content,