LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Python ai-agentsdata-engineeringdocument-aigenerative-aiidpjson-extractionllmmcp-serverocrpdf-extractionprompt-engineeringstructured-output
ocr tool with adapter support
vishnuszipstackf9921f8705c3
diff --git a/tools/ocr/.dockerignore b/tools/ocr/.dockerignorenew file mode 100644index 000000000..c26352afc--- /dev/null+++ b/tools/ocr/.dockerignore@@ -0,0 +1,3 @@+venv/+.venv/+.envdiff --git a/tools/ocr/Dockerfile b/tools/ocr/Dockerfilenew file mode 100644index 000000000..4c95f3d4d--- /dev/null+++ b/tools/ocr/Dockerfile@@ -0,0 +1,21 @@+FROM python:3.9-slim++LABEL maintainer="Zipstack Inc."+ENV UNSTRACT_ENTRYPOINT "python /app/src/main.py"++# Install dependencies for unstructured library's partition+RUN apt-get update && apt-get --no-install-recommends -y install libmagic-dev poppler-utils tesseract-ocr libreoffice pandoc\+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/*++RUN pip install --no-cache-dir -U pip+# Set the working directory in the container+WORKDIR /app+COPY requirements.txt /app/+RUN pip install --no-cache-dir -r requirements.txt+# Copy the contents of your project directory into the container at /app+COPY src /app/src/+WORKDIR /app/src+++ENTRYPOINT ["python", "main.py"]diff --git a/tools/ocr/README.md b/tools/ocr/README.mdnew file mode 100644index 000000000..4ba66babb--- /dev/null+++ b/tools/ocr/README.md@@ -0,0 +1,112 @@+## Document OCR processor++This is tool which can be used to extract text from images. Perform OCR operations on images.++### Required environment variables++| Variable | Description |+| -------------------------- | --------------------------------------------------------------------- |+| `PLATFORM_SERVICE_HOST` | The host in which the platform service is running |+| `PLATFORM_SERVICE_PORT` | The port in which the service is listening |+| `PLATFORM_SERVICE_API_KEY` | The API key for the platform |+| `TOOL_DATA_DIR` | The directory in the filesystem which has contents for tool execution |++### Testing the tool locally++#### Setting up a dev environment+Setup a virtual environment and activate it++```commandline+python -m venv .venv+source .venv/bin/activate+```++Install the dependencies for the tool++```commandline+pip install -r requirements.txt+```++To use the local development version of the [unstract-sdk](https://pypi.org/project/unstract-sdk/) install it from the local repository.+Replace the path with the path to your local repository++```commandline+pip install -e ~/path_to_repo/sdks/.+```++#### Tool execution preparation++Load the environment variables for the tool.+Make a copy of the `sample.env` file and name it `.env`. Fill in the required values.+They get loaded with [python-dotenv](https://pypi.org/project/python-dotenv/) through the SDK.++Update the tool's `data_dir` marked by the `TOOL_DATA_DIR` env. This has to be done before each tool execution since the tool updates the `INFILE` and `METADATA.json`.++#### Run SPEC command++Represents the JSON schema for the runtime configurable `settings` of a tool+```commandline+python main.py --command SPEC+```++#### Run PROPERTIES command++Describes some metadata for the tool such as its `version`, `description`, `inputs` and `outputs`+```commandline+python main.py --command PROPERTIES+```++#### Run ICON command++Returns the SVG icon for the tool, used by Unstract's frontend+```commandline+python main.py --command ICON+```++#### Run VARIABLES command++Represents the runtime variables or envs that will be used by the tool+```commandline+python main.py --command VARIABLES+```++#### Run RUN command++The format of the jsons required for settings and params can be found by running the SPEC command and the PROPERTIES+command respectively. Alternatively if you have access to the code base, it is located in the `config` folder+as `spec.json` and `properties.json`.+++```commandline+python main.py \+ --command RUN \+ --settings '{+ }' \+ --workflow-id '00000000-0000-0000-0000-000000000000' \+ --log-level DEBUG++```+### Testing the tool from its docker image++Build the tool docker image from the folder containing the `Dockerfile` with+```commandline+docker build -t unstract/tool-example:0.0.1 .+```++Make sure the directory pointed by `TOOL_DATA_DIR` has the required information for the tool to run and+necessary services like the `unstract-platform-service` is up.+To test the tool from its docker image, run the following command++```commandline+docker run -it \+ --network unstract-network \+ --env-file .env \+ -v "$(pwd)"/data_dir:/app/data_dir \+ unstract/tool-example:0.0.1 \+ --command RUN \+ --settings '{+ }' \+ --workflow-id '00000000-0000-0000-0000-000000000000' \+ --log-level DEBUG++```diff --git a/tools/ocr/requirements.txt b/tools/ocr/requirements.txtnew file mode 100644index 000000000..cfbb98360--- /dev/null+++ b/tools/ocr/requirements.txt@@ -0,0 +1,9 @@+# Add your dependencies here+pypdf~=3.11.1+argparse==1.4.0+filetype==1.2.0+requests==2.31.0+pillow~=10.2.0++# Required for all unstract tools+unstract-sdk==0.12.0diff --git a/tools/ocr/sample.env b/tools/ocr/sample.envnew file mode 100644index 000000000..a0caeece0--- /dev/null+++ b/tools/ocr/sample.env@@ -0,0 +1,4 @@+PLATFORM_SERVICE_HOST=http://unstract-platform-service+PLATFORM_SERVICE_PORT=3001+PLATFORM_SERVICE_API_KEY=<add_platform_key_from_Unstract_frontend>+TOOL_DATA_DIR=../data_dirdiff --git a/tools/ocr/src/config/properties.json b/tools/ocr/src/config/properties.jsonnew file mode 100644index 000000000..a34a5bc81--- /dev/null+++ b/tools/ocr/src/config/properties.json@@ -0,0 +1,59 @@+{+ "schemaVersion": "0.0.1",+ "displayName": "OCR",+ "functionName": "ocr",+ "toolVersion": "0.0.1",+ "description": "This is tool which can be used to extract text from images. Perform OCR operations on images.",+ "input": {+ "description": "The file on which OCR operation is to be performed"+ },+ "output": {+ "description": "Places the file into a folder (bin) that its OCR opertation performed into."+ },+ "result": {+ "type": "JSON",+ "description": "JSON response containing the bin to which the file was classified into",+ "schema": {}+ },+ "adapter": {+ "ocrs": [+ {+ "isEnabled": true,+ "adapterId": "ocrAdapterId",+ "title": "OCR to use",+ "isRequired": true,+ "description": "OCR used to extract text from images"+ }+ ]+ },+ "ioCompatibility": {+ "api": {+ "sourceSupport": true,+ "destinationSupport": true,+ "additionalArgs": {+ "sync": true+ }+ },+ "file": {+ "sourceSupport": true,+ "destinationSupport": true,+ "additionalArgs": {}+ },+ "db": {+ "destinationSupport": true,+ "additionalArgs": {}+ }+ },+ "restrictions": {+ "maxFileSize": "10MB",+ "allowedFileTypes": [+ "jpeg",+ "png",+ "tiff",+ "bmp",+ "gif",+ "webp",+ "pdf"+ ]+ }+}diff --git a/tools/ocr/src/config/spec.json b/tools/ocr/src/config/spec.jsonnew file mode 100644index 000000000..918d03237--- /dev/null+++ b/tools/ocr/src/config/spec.json@@ -0,0 +1,14 @@+{+ "title": "OCR Tool",+ "description": "Setup the OCR tool",+ "type": "object",+ "required": [],+ "properties": {+ "useCache": {+ "type": "boolean",+ "title": "Cache and use cached results",+ "default": true,+ "description": "Use cached results"+ }+ }+}\ No newline at end of filediff --git a/tools/ocr/src/constants.py b/tools/ocr/src/constants.pynew file mode 100644index 000000000..87e75f366--- /dev/null+++ b/tools/ocr/src/constants.py@@ -0,0 +1,18 @@+class FileType:+ TEXT_PLAIN = "text/plain"+ IMAGE_JPEG = "image/jpeg"+ IMAGE_PNG = "image/png"+ IMAGE_TIFF = "image/tiff"+ IMAGE_BMP = "image/bmp"+ IMAGE_GIF = "image/gif"+ IMAGE_WEBP = "image/webp"+ APPLICATION_PDF = "application/pdf"+ ALLOWED_TYPES = [+ IMAGE_JPEG,+ IMAGE_PNG,+ IMAGE_TIFF,+ IMAGE_BMP,+ IMAGE_GIF,+ IMAGE_WEBP,+ APPLICATION_PDF,+ ]diff --git a/tools/ocr/src/enums.py b/tools/ocr/src/enums.pynew file mode 100644index 000000000..da748ef17--- /dev/null+++ b/tools/ocr/src/enums.py@@ -0,0 +1,7 @@+from enum import Enum+++class CostUnits(Enum):+ CACHE = "Cache"+ GOOGLE_PAGES = "google/pages"+diff --git a/tools/ocr/src/helper.py b/tools/ocr/src/helper.pynew file mode 100644index 000000000..bb8a5b3b4--- /dev/null+++ b/tools/ocr/src/helper.py@@ -0,0 +1,129 @@+import pypdf+from constants import FileType+from enums import CostUnits+from unstract.sdk.cache import ToolCache+from unstract.sdk.constants import LogLevel, ToolEnv+from unstract.sdk.tool.base import BaseTool+++class OcrHelper:+ """Helper functions for Ocr tool."""++ def __init__(self, tool: BaseTool, use_cache: bool = False) -> None:+ self.cache: ToolCache = None+ self.use_cache = False+ self.tool = tool++ if use_cache:+ platform_host = self.tool.get_env_or_die(ToolEnv.PLATFORM_HOST)+ platform_port = self.tool.get_env_or_die(ToolEnv.PLATFORM_PORT)+ self.tool.stream_log("Check result in cache")++ self.cache = ToolCache(+ tool=self.tool,+ platform_host=platform_host,+ platform_port=int(platform_port),+ )+ self.use_cache = True++ def stream_error_and_exit(self, message: str) -> None:+ """Stream error log and exit.++ Args:+ message (str): Error message+ """+ self.tool.stream_log(message, level=LogLevel.ERROR)+ exit(1)++ def get_page_count(self, file: bytes, file_type_mime: str) -> int:+ """Count pages for billing purposes.++ Args:+ file (str): The path to the input file+ file_type_mime (str): The MIME type of the file++ Returns:+ int: page count+ """+ page_count = 1+ # Count pages in case of PDF for billing purposes+ if file_type_mime == FileType.APPLICATION_PDF:+ pdf_page_count = 0+ with open(file, mode="rb") as input_file_obj:+ pdf_reader = pypdf.PdfReader(input_file_obj)+ pdf_page_count = len(pdf_reader.pages)+ self.tool.stream_log(f"PDF page count: {pdf_page_count}")+ page_count = pdf_page_count+ return page_count++ def calculate_cost(+ self,+ file: bytes,+ file_type_mime: str,+ cached_result: bool = False,+ ) -> None:+ """Get cost and stream cost.++ Args:+ file (bytes): _description_+ file_type_mime (str): _description_+ """+ if cached_result:+ self.tool.stream_cost(+ cost=0.0,+ cost_units=CostUnits.CACHE.value,+ )+ else:+ page_count = self.get_page_count(+ file=file, file_type_mime=file_type_mime+ )+ self.tool.stream_cost(+ cost=float(page_count),+ cost_units=CostUnits.GOOGLE_PAGES.value,+ )++ def set_result_in_cache(+ self,+ key: str,+ result: str,+ cached_result: bool = False,+ ) -> None:+ """Get result from cache by the help of unstract Cache tool.++ Args:+ key (str): Cache key++ Required env variables:+ PLATFORM_HOST: Host of platform service+ PLATFORM_PORT: Port of platform service+ Returns:+ Optional[str]: result+ """++ if not self.use_cache:+ return None+ if not cached_result:+ self.cache.set(key, result)++ def stream_output_text_log(self, data: str) -> None:+ """Stream document text.++ Args:+ sql (str): _description_+ """+ data_text_for_log = "### OCR Output\n\n"+ if len(data) > 500:+ data_text_for_log = data[:500] + "...(truncated)"+ self.tool.stream_single_step_message(+ f"```json\n{data_text_for_log}\n```"+ )++ def time_taken(self, start_time: float, end_time: float) -> None:+ """Calculate Time difference.++ Args:+ start_time (float): _description_+ end_time (float): _description_+ """+ time_taken = end_time - start_time+ self.tool.stream_log(f"Time taken: {time_taken}")diff --git a/tools/ocr/src/main.py b/tools/ocr/src/main.pynew file mode 100644index 000000000..dd2127917--- /dev/null+++ b/tools/ocr/src/main.py@@ -0,0 +1,109 @@+import sys+import time+from pathlib import Path+from typing import Any++from helper import OcrHelper+from unstract.sdk.cache import ToolCache+from unstract.sdk.constants import LogLevel, LogState, MetadataKey, ToolEnv+from unstract.sdk.ocr import OCR+from unstract.sdk.tool.base import BaseTool+from unstract.sdk.tool.entrypoint import ToolEntrypoint+from unstract.sdk.tool.validator import ToolValidator+from unstract.sdk.utils.tool_utils import ToolUtils+++class UnstractOCR(BaseTool):+ def __init__(self, log_level: str = LogLevel.INFO) -> None:+ super().__init__(log_level)+ self.helper = OcrHelper(tool=self)+ self.validator = ToolValidator(tool=self)++ def run(+ self,+ settings: dict[str, Any],+ input_file: str,+ output_dir: str,+ ) -> None:+ # Initializing Function Arguments+ use_cache = settings["useCache"]+ ocr_adapter_id = settings["ocrAdapterId"]++ # Set adapter+ tool_ocr = OCR(tool=self)+ ocr_adapter = tool_ocr.get_ocr(adapter_instance_id=ocr_adapter_id)++ # Read image file into memory+ with open(input_file, "rb") as image_file:+ image_content = image_file.read()++ input_file_type_mime = ToolUtils.get_file_mime_type(Path(input_file))++ # Construct an image object+ output_log = ""+ input_log = f"Input file: `{input_file}`\n\n"+ self.stream_update(input_log, state=LogState.INPUT_UPDATE)++ # Check cache+ content_hash = ToolUtils.get_hash_from_file(file_path=input_file)+ cache_key = (+ f"cache:{self.workflow_id}:{ocr_adapter.get_name()}:{content_hash}"+ )++ result_text = None+ if use_cache:+ cache = ToolCache(+ tool=self,+ platform_host=self.get_env_or_die(ToolEnv.PLATFORM_HOST),+ platform_port=int(self.get_env_or_die(ToolEnv.PLATFORM_PORT)),+ )+ cached_response = cache.get(cache_key)+ if cached_response is not None:+ result_text = cached_response+ self.stream_cost(cost=0.0, cost_units="cache")+ cached_result = True if result_text else False+ # Process if not cached+ if not result_text:+ result_text = ""+ t1 = time.time()+ result_text = ocr_adapter.process(input_file_path=input_file)+ t2 = time.time()+ self.helper.time_taken(start_time=t1, end_time=t2)++ # Pre-process and cache result+ self.helper.set_result_in_cache(+ key=cache_key, result=result_text, cached_result=cached_result+ )+ self.helper.calculate_cost(+ file=image_content,+ file_type_mime=input_file_type_mime,+ cached_result=cached_result,+ )+ self.helper.stream_output_text_log(result_text)++ # Write result to file+ self.stream_log("Writing tool output")+ source_name = self.get_exec_metadata.get(MetadataKey.SOURCE_NAME)+ output_path = Path(output_dir) / f"{Path(source_name).stem}.txt"+ try:+ with open(output_path, "w", encoding="utf-8") as f:+ f.write(result_text)+ except Exception as e:+ self.stream_error_and_exit(f"Error creating output file: {e}")++ # Log output+ if len(result_text) > 1000:+ output_log = (+ f"```text\n{result_text[:1000]}... (truncated)\n```\n\n"+ )+ else:+ output_log = f"```text\n{result_text}\n```\n\n"+ self.stream_update(output_log, state=LogState.OUTPUT_UPDATE)+ self.stream_single_step_message(output_log)+ self.write_tool_result(data=result_text)+++if __name__ == "__main__":+ args = sys.argv[1:]+ tool = UnstractOCR.from_tool_args(args=args)+ ToolEntrypoint.launch(tool=tool, args=args)diff --git a/unstract/tool-registry/src/unstract/tool_registry/public_tools.json b/unstract/tool-registry/src/unstract/tool_registry/public_tools.jsonindex d5e04f633..31b00467b 100644--- a/unstract/tool-registry/src/unstract/tool_registry/public_tools.json+++ b/unstract/tool-registry/src/unstract/tool_registry/public_tools.json@@ -646,5 +646,106 @@"image_url": "docker:unstract/tool-text-extractor:0.0.1","image_name": "unstract/tool-text-extractor","image_tag": "0.0.1"+ },+ "ocr": {+ "tool_uid": "ocr",+ "properties": {+ "schemaVersion": "0.0.1",+ "displayName": "OCR",+ "functionName": "ocr",+ "toolVersion": "0.0.1",+ "description": "This is tool which can be used to extract text from images. Perform OCR operations on images.",+ "input": {+ "description": "The file on which OCR operation is to be performed"+ },+ "output": {+ "description": "Places the file into a folder (bin) that its OCR opertation performed into."+ },+ "result": {+ "type": "JSON",+ "description": "JSON response containing the bin to which the file was classified into",+ "schema": {}+ },+ "adapter": {+ "ocrs": [+ {+ "isEnabled": true,+ "adapterId": "ocrAdapterId",+ "title": "OCR to use",+ "isRequired": true,+ "description": "OCR used to extract text from images"+ }+ ]+ },+ "ioCompatibility": {+ "api": {+ "sourceSupport": true,+ "destinationSupport": true,+ "additionalArgs": {+ "sync": true+ }+ },+ "file": {+ "sourceSupport": true,+ "destinationSupport": true,+ "additionalArgs": {}+ },+ "db": {+ "destinationSupport": true,+ "additionalArgs": {}+ }+ },+ "restrictions": {+ "maxFileSize": "10MB",+ "allowedFileTypes": [+ "jpeg",+ "png",+ "tiff",+ "bmp",+ "gif",+ "webp",+ "pdf"+ ]+ }+ },+ "spec": {+ "title": "OCR Tool",+ "description": "Setup the OCR tool",+ "type": "object",+ "required": [],+ "properties": {+ "useCache": {+ "type": "boolean",+ "title": "Cache and use cached results",+ "default": true,+ "description": "Use cached results"+ }+ }+ },+ "variables": {+ "title": "Runtime Variables",+ "description": "Runtime Variables for ocr",+ "type": "object",+ "required": [+ "GOOGLE_SERVICE_ACCOUNT",+ "GOOGLE_DOCUMENT_AI_ENDPOINT"+ ],+ "properties": {+ "GOOGLE_SERVICE_ACCOUNT": {+ "type": "string",+ "title": "Google Service Account",+ "description": "Google Service account"+ },+ "GOOGLE_DOCUMENT_AI_ENDPOINT": {+ "type": "string",+ "title": "Google Document API Base URL",+ "description": "The base URL for the Google Document service"+ }+ }+ },+ "icon": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<svg\n enable-background=\"new 0 0 20 20\"\n height=\"48\"\n viewBox=\"0 0 20 20\"\n width=\"48\"\n fill=\"#000000\"\n version=\"1.1\"\n id=\"svg8109\"\n sodipodi:docname=\"folder_copy_black_48dp.svg\"\n xmlns:inkscape=\"http://www.inkscape.org/namespaces/inkscape\"\n xmlns:sodipodi=\"http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd\"\n xmlns=\"http://www.w3.org/2000/svg\"\n xmlns:svg=\"http://www.w3.org/2000/svg\">\n <defs\n id=\"defs8113\" />\n <sodipodi:namedview\n id=\"namedview8111\"\n pagecolor=\"#ffffff\"\n bordercolor=\"#000000\"\n borderopacity=\"0.25\"\n inkscape:showpageshadow=\"2\"\n inkscape:pageopacity=\"0.0\"\n inkscape:pagecheckerboard=\"0\"\n inkscape:deskcolor=\"#d1d1d1\"\n showgrid=\"false\" />\n <g\n id=\"g8099\">\n <rect\n fill=\"none\"\n height=\"20\"\n width=\"20\"\n x=\"0\"\n id=\"rect8097\"\n y=\"0\" />\n </g>\n <g\n id=\"g8107\"\n style=\"fill:#ff4d6d;fill-opacity:1\">\n <g\n id=\"g8105\"\n style=\"fill:#ff4d6d;fill-opacity:1\">\n <path\n d=\"M 2.5,5 H 1 V 15.5 C 1,16.33 1.67,17 2.5,17 H 15.68 V 15.5 H 2.5 Z\"\n id=\"path8101\"\n style=\"fill:#ff4d6d;fill-opacity:1\" />\n <path\n d=\"M 16.5,4 H 11 L 9,2 H 5.5 C 4.67,2 4,2.67 4,3.5 v 9 C 4,13.33 4.67,14 5.5,14 h 11 c 0.83,0 1.5,-0.67 1.5,-1.5 v -7 C 18,4.67 17.33,4 16.5,4 Z m 0,8.5 h -11 v -9 h 2.88 l 2,2 h 6.12 z\"\n id=\"path8103\"\n style=\"fill:#ff4d6d;fill-opacity:1\" />\n </g>\n </g>\n</svg>\n",+ "image_url": "docker:unstract/tool-ocr:0.0.1",+ "image_name": "unstract/tool-ocr",+ "image_tag": "0.0.1"}-}\ No newline at end of file+}