Skip to content

armgit / papra

Public Imported from GitHub
0

The minimalistic document archiving platform.

feat(lecture): normalize image rotation before OCR (#1469)

Corentin Thomasset347029dbd1ae
diff --git a/.changeset/five-donkeys-wonder.md b/.changeset/five-donkeys-wonder.md
new file mode 100644
index 00000000..736ccd92
--- /dev/null
+++ b/.changeset/five-donkeys-wonder.md
@@ -0,0 +1,6 @@
+---
+'@papra/app': patch
+'@papra/lecture': patch
+---
+
+Fixed OCR extraction for images whose rotation is stored in EXIF metadata.
diff --git a/packages/lecture/fixtures/022-jpg-exif-rotation/022.expected.txt b/packages/lecture/fixtures/022-jpg-exif-rotation/022.expected.txt
new file mode 100644
index 00000000..85877744
--- /dev/null
+++ b/packages/lecture/fixtures/022-jpg-exif-rotation/022.expected.txt
@@ -0,0 +1,41 @@
+at his touch of a certain icy pang along my blood. “Come, sir,” said I.
+“You forget that I have not yet the pleasure of your acquaintance. Be
+seated, if you please.” And I showed him an example, and sat down
+myself in my customary seat and with as fair an imitation of my or-
+dinary manner to a patient, as the lateness of the hour, the nature of
+my preoccupations, and the horror I had of my visitor, would suffer
+me to muster.
+
+“I beg your pardon, Dr. Lanyon,” he replied civilly enough. “What
+you say is very well founded; and my impatience has shown its heels
+to my politeness. I come here at the instance of your colleague, Dr.
+Henry Jekyll, on a piece of business of some moment; and I under-
+stood...” He paused and put his hand to his throat, and I could see,
+in spite of his collected manner, that he was wrestling against the
+approaches of the hysteria—“T understood, a drawer...”
+
+But here I took pity on my visitor's suspense, and some perhaps
+on my own growing curiosity.
+
+“There it is, sir,” said I, pointing to the drawer, where it lay on the
+floor behind a table and still covered with the sheet.
+
+He sprang to it, and then paused, and laid his hand upon his
+heart: I could hear his teeth grate with the convulsive action of his
+jaws; and his face was so ghastly to see that I grew alarmed both for
+his life and reason.
+
+“Compose yourself,” said I.
+
+He turned a dreadful smile to me, and as if with the decision of
+despair, plucked away the sheet. At sight of the contents, he uttered
+one loud sob of such immense relief that I sat petrified. And the
+next moment, in a voice that was already fairly well under control,
+“Have you a graduated glass?” he asked.
+
+I rose from my place with something of an effort and gave him
+what he asked.
+
+He thanked me with a smiling nod, measured out a few min-
+ims of the red tincture and added one of the powders. The mix-
+ture, which was at first of a reddish hue, began, in proportion as the
\ No newline at end of file
diff --git a/packages/lecture/fixtures/022-jpg-exif-rotation/022.input.jpg b/packages/lecture/fixtures/022-jpg-exif-rotation/022.input.jpg
new file mode 100644
index 00000000..3c54d13e
Binary files /dev/null and b/packages/lecture/fixtures/022-jpg-exif-rotation/022.input.jpg differ
diff --git a/packages/lecture/src/extractors/img.extractor.ts b/packages/lecture/src/extractors/img.extractor.ts
index e05a1155..7ba27907 100644
--- a/packages/lecture/src/extractors/img.extractor.ts
+++ b/packages/lecture/src/extractors/img.extractor.ts
@@ -1,13 +1,35 @@
+import sharp from 'sharp';
import { defineTextExtractor } from '../extractors.models';
import { createTesseractExtractor } from '../tesseract/tesseract.usecases';
+import type { Logger } from '../types';
+
+async function normalizeImageRotation(
+ arrayBuffer: ArrayBuffer,
+ { logger }: { logger?: Logger },
+): Promise<Buffer | ArrayBuffer> {
+ try {
+ const image = sharp(arrayBuffer);
+ const { orientation } = await image.metadata();
+ const shouldNormalizeRotation = orientation !== undefined && orientation !== 1;
+ return shouldNormalizeRotation ? await image.rotate().toBuffer() : arrayBuffer;
+ } catch (error) {
+ logger?.error(
+ { error },
+ 'Failed to normalize image rotation, trying to extract text from the original image',
+ );
+ return arrayBuffer;
+ }
+}
export const imageExtractorDefinition = defineTextExtractor({
name: 'image',
mimeTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'],
- extract: async ({ arrayBuffer, config }) => {
+ extract: async ({ arrayBuffer, config, logger }) => {
const { extract, extractorType } = await createTesseractExtractor(config.tesseract);
- const content = await extract(arrayBuffer);
+ const normalizedImage = await normalizeImageRotation(arrayBuffer, { logger });
+
+ const content = await extract(normalizedImage);
return {
content,