LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Python ai-agentsdata-engineeringdocument-aigenerative-aiidpjson-extractionllmmcp-serverocrpdf-extractionprompt-engineeringstructured-output
[MISC] Dockerfile enhancement to cache layers better (#1377)
Chandrasekharan M8980ce59fb49
diff --git a/docker/README.md b/docker/README.mdindex f19834dfe..843fc67d6 100644--- a/docker/README.md+++ b/docker/README.md@@ -70,9 +70,11 @@ This can be useful during development to:3. Start services with watch mode enabled```bash- VERSION=dev docker compose -f docker-compose.yaml -f compose.override.yaml watch+ VERSION=dev docker compose -f docker-compose.yaml -f compose.override.yaml -f docker-compose.build.yaml watch```+> **NOTE**: Make sure to specify the build definitions also in your `compose.override.yaml` file or specify [docker-compose.build.yaml](/docker/docker-compose.build.yaml) while running the above command.+### Example Workflow1. Start services with watch mode:diff --git a/docker/dockerfiles/backend.Dockerfile b/docker/dockerfiles/backend.Dockerfileindex e0da7658c..21c6ff7c8 100644--- a/docker/dockerfiles/backend.Dockerfile+++ b/docker/dockerfiles/backend.Dockerfile@@ -1,51 +1,69 @@-FROM python:3.12.9-slim+# Use a specific version of Python slim image+FROM python:3.12.9-slim AS base-LABEL maintainer="Zipstack Inc."+ARG VERSION=dev+LABEL maintainer="Zipstack Inc." \+ description="Backend Service Container" \+ version="${VERSION}"-ENV \- # Keeps Python from generating .pyc files in the container- PYTHONDONTWRITEBYTECODE=1 \- # Set to immediately flush stdout and stderr streams without first buffering+# Set environment variables+ENV PYTHONDONTWRITEBYTECODE=1 \PYTHONUNBUFFERED=1 \PYTHONPATH=/unstract \BUILD_CONTEXT_PATH=backend \BUILD_PACKAGES_PATH=unstract \DJANGO_SETTINGS_MODULE="backend.settings.dev" \+ APP_HOME=/app \# OpenTelemetry configuration (disabled by default, enable in docker-compose)OTEL_TRACES_EXPORTER=none \OTEL_LOGS_EXPORTER=none \OTEL_SERVICE_NAME=unstract_backend# Install system dependencies-RUN apt-get update && \- apt-get --no-install-recommends install -y \- build-essential \- freetds-dev \- git \- libkrb5-dev \- libmagic-dev \- libssl-dev \- pkg-config && \- apt-get clean && \- rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*-+RUN apt-get update \+ && apt-get --no-install-recommends install -y \+ build-essential \+ freetds-dev \+ git \+ libkrb5-dev \+ libmagic-dev \+ libssl-dev \+ pkg-config \+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*+# Install uv package managerCOPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/-WORKDIR /app+# Create working directory+WORKDIR ${APP_HOME}++# -----------------------------------------------+# EXTERNAL DEPENDENCIES STAGE - This layer gets cached if external dependencies don't change+# -----------------------------------------------+FROM base AS ext-dependencies-# Copy only what's needed for dependency installation first-COPY pyproject.toml uv.lock /app/+# Copy dependency-related files+COPY ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ${BUILD_CONTEXT_PATH}/README.md ./-# Copy source code and dependencies-COPY ${BUILD_CONTEXT_PATH}/ /app/+# Copy local package dependenciesCOPY ${BUILD_PACKAGES_PATH}/ /unstract/-# Install dependencies in one layer-RUN uv sync --frozen && \- uv sync --group deploy && \- .venv/bin/python3 -m ensurepip --upgrade && \+# Install external dependencies from pyproject.toml+RUN uv sync --group deploy --locked --no-install-project --no-dev && \uv run opentelemetry-bootstrap -a install+# -----------------------------------------------+# FINAL STAGE - Minimal image for production+# -----------------------------------------------+FROM ext-dependencies AS production++# Copy application code (this layer changes most frequently)+COPY ${BUILD_CONTEXT_PATH}/ ./++# Install the application+RUN uv sync --group deploy --locked && \+ chmod +x ./entrypoint.sh+EXPOSE 8000ENTRYPOINT [ "./entrypoint.sh" ]diff --git a/docker/dockerfiles/backend.Dockerfile.dockerignore b/docker/dockerfiles/backend.Dockerfile.dockerignoreindex 04276a19f..cd5828c03 100644--- a/docker/dockerfiles/backend.Dockerfile.dockerignore+++ b/docker/dockerfiles/backend.Dockerfile.dockerignore@@ -1,4 +1,5 @@**/__pycache__+**/.mypy_cache**/.pytest_cache**/.python-version**/.pycdiff --git a/docker/dockerfiles/platform.Dockerfile b/docker/dockerfiles/platform.Dockerfileindex 460ad8516..6e04ccdb4 100644--- a/docker/dockerfiles/platform.Dockerfile+++ b/docker/dockerfiles/platform.Dockerfile@@ -1,14 +1,12 @@# Use a specific version of Python slim image-FROM python:3.12.9-slim+FROM python:3.12.9-slim AS base+ARG VERSION=devLABEL maintainer="Zipstack Inc." \description="Platform Service Container" \- version="1.0"+ version="${VERSION}"-ENV \- # Keeps Python from generating .pyc files in the container- PYTHONDONTWRITEBYTECODE=1 \- # Set to immediately flush stdout and stderr streams without first buffering+ENV PYTHONDONTWRITEBYTECODE=1 \PYTHONUNBUFFERED=1 \PYTHONPATH=/unstract \BUILD_CONTEXT_PATH=platform-service \@@ -22,42 +20,54 @@ ENV \OTEL_SERVICE_NAME=unstract_platform# Install system dependencies and create user in one layer-RUN apt-get update && \- apt-get --no-install-recommends install -y build-essential libmagic-dev git && \- apt-get clean && \- rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* && \- adduser -u 5678 --disabled-password --gecos "" ${APP_USER} && \- mkdir -p ${APP_HOME} && \- chown -R ${APP_USER}:${APP_USER} ${APP_HOME}+RUN apt-get update \+ && apt-get --no-install-recommends install -y \+ build-essential \+ libmagic-dev \+ git \+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* \+ && adduser -u 5678 --disabled-password --gecos "" ${APP_USER} \+ && mkdir -p ${APP_HOME} \+ && chown -R ${APP_USER}:${APP_USER} ${APP_HOME}# Install uv package managerCOPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/+# Create working directoryWORKDIR ${APP_HOME}-# Copy dependency files first to leverage Docker cache-COPY --chown=${APP_USER} ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ./+# -----------------------------------------------+# EXTERNAL DEPENDENCIES STAGE - This layer gets cached if external dependencies don't change+# -----------------------------------------------+FROM base AS ext-dependencies-# Read and execute access to non-root user to avoid security hotspot-# Write access to specific sub-directory need to be explicitly provided if required-COPY --chown=${APP_USER} ${BUILD_CONTEXT_PATH} /app/-# Copy local dependency packages-COPY --chown=${APP_USER} ${BUILD_PACKAGES_PATH}/core /unstract/core-COPY --chown=${APP_USER} ${BUILD_PACKAGES_PATH}/flags /unstract/flags+# Copy dependency-related files+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ${BUILD_CONTEXT_PATH}/README.md ./-# Copy application files-COPY --chown=${APP_USER} ${BUILD_CONTEXT_PATH} ./+# Copy local package dependencies+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_PACKAGES_PATH}/core /unstract/core+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_PACKAGES_PATH}/flags /unstract/flags-# Switch to non-root user-USER ${APP_USER}+# Install external dependencies from pyproject.toml+RUN uv sync --group deploy --locked --no-install-project --no-dev && \+ uv run opentelemetry-bootstrap -a install++# -----------------------------------------------+# FINAL STAGE - Minimal image for production+# -----------------------------------------------+FROM ext-dependencies AS production++# Copy application code (this layer changes most frequently)+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH} ./-# Create virtual environment and install dependencies in one layer-RUN uv sync --frozen \- && uv sync --group deploy \- && .venv/bin/python3 -m ensurepip --upgrade \- && uv run opentelemetry-bootstrap -a install+# Install the application in non-editable mode to avoid permission issues+RUN uv sync --group deploy --locked++# Switch to non-root user for the rest of the operations+USER ${APP_USER}EXPOSE 3001-# During debugging, this entry point will be overridden. For more information, please refer to https://aka.ms/vscode-docker-python-debug+# During debugging, this entry point will be overriddenCMD [".venv/bin/gunicorn", "--bind", "0.0.0.0:3001", "--timeout", "300", "unstract.platform_service.run:app"]diff --git a/docker/dockerfiles/platform.Dockerfile.dockerignore b/docker/dockerfiles/platform.Dockerfile.dockerignoreindex 8e455ee73..e1c970ee8 100644--- a/docker/dockerfiles/platform.Dockerfile.dockerignore+++ b/docker/dockerfiles/platform.Dockerfile.dockerignore@@ -1,4 +1,5 @@**/__pycache__+**/.mypy_cache**/.pytest_cache**/.python-version**/.pycdiff --git a/docker/dockerfiles/prompt.Dockerfile b/docker/dockerfiles/prompt.Dockerfileindex 88149fead..4ab51553e 100644--- a/docker/dockerfiles/prompt.Dockerfile+++ b/docker/dockerfiles/prompt.Dockerfile@@ -1,12 +1,12 @@# Use a specific version of Python slim image-FROM python:3.12.9-slim+FROM python:3.12.9-slim AS base+ARG VERSION=devLABEL maintainer="Zipstack Inc." \description="Prompt Service Container" \- version="1.0"+ version="${VERSION}"ENV PYTHONDONTWRITEBYTECODE=1 \- # Set to immediately flush stdout and stderr streams without first bufferingPYTHONUNBUFFERED=1 \PYTHONPATH=/unstract \BUILD_CONTEXT_PATH=prompt-service \@@ -21,42 +21,53 @@ ENV PYTHONDONTWRITEBYTECODE=1 \OTEL_SERVICE_NAME=unstract_prompt# Install system dependencies, create user, and setup directories in one layer-RUN apt-get update && \- apt-get --no-install-recommends install -y \- build-essential \- libmagic-dev \- pkg-config \- git && \- apt-get clean && \- rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* && \- adduser -u 5678 --disabled-password --gecos "" ${APP_USER} && \- mkdir -p ${APP_HOME} && \- chown -R ${APP_USER}:${APP_USER} ${APP_HOME}+RUN apt-get update \+ && apt-get --no-install-recommends install -y \+ build-essential \+ libmagic-dev \+ pkg-config \+ git \+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* \+ && adduser -u 5678 --disabled-password --gecos "" ${APP_USER} \+ && mkdir -p ${APP_HOME} \+ && chown -R ${APP_USER}:${APP_USER} ${APP_HOME}# Install uv package managerCOPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/+# Create working directoryWORKDIR ${APP_HOME}-# Copy dependency files first for better caching-COPY --chown=${APP_USER} ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ./+# -----------------------------------------------+# EXTERNAL DEPENDENCIES STAGE - This layer gets cached if external dependencies don't change+# -----------------------------------------------+FROM base AS ext-dependencies-# Copy required packages-COPY --chown=${APP_USER} ${BUILD_PACKAGES_PATH}/core /unstract/core-COPY --chown=${APP_USER} ${BUILD_PACKAGES_PATH}/flags /unstract/flags+# Copy dependency-related files+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ${BUILD_CONTEXT_PATH}/README.md ./-# Copy application code-COPY --chown=${APP_USER} ${BUILD_CONTEXT_PATH} /app/+# Copy local package dependencies+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_PACKAGES_PATH}/core /unstract/core+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_PACKAGES_PATH}/flags /unstract/flags-# Switch to non-root user-USER ${APP_USER}+# Install external dependencies from pyproject.toml+RUN uv sync --group deploy --locked --no-install-project --no-dev && \+ uv run opentelemetry-bootstrap -a install++# -----------------------------------------------+# FINAL STAGE - Minimal image for production+# -----------------------------------------------+FROM ext-dependencies AS production++# Copy application code (this layer changes most frequently)+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH} ./-# Install dependencies in a single layer-RUN uv sync --frozen && \- . .venv/bin/activate && \- uv sync --group deploy && \- # Install plugins in development mode from root directory- for dir in "${TARGET_PLUGINS_PATH}"/*/; do \+# Install just the application in editable mode+RUN uv sync --group deploy --locked++# Install plugins after copying source code+RUN for dir in "${TARGET_PLUGINS_PATH}"/*/; do \dirpath=${dir%*/}; \dirname=${dirpath##*/}; \if [ "${dirname}" != "*" ]; then \@@ -64,11 +75,11 @@ RUN uv sync --frozen && \uv pip install "${TARGET_PLUGINS_PATH}/${dirname}"; \fi; \done && \- .venv/bin/python3 -m ensurepip --upgrade && \- uv run opentelemetry-bootstrap -a install && \mkdir -p prompt-studio-data+# Switch to non-root user+USER ${APP_USER}+EXPOSE 3003-# Default commandCMD ["./entrypoint.sh"]diff --git a/docker/dockerfiles/prompt.Dockerfile.dockerignore b/docker/dockerfiles/prompt.Dockerfile.dockerignoreindex e49108bb6..999f293e8 100644--- a/docker/dockerfiles/prompt.Dockerfile.dockerignore+++ b/docker/dockerfiles/prompt.Dockerfile.dockerignore@@ -1,4 +1,5 @@**/__pycache__+**/.mypy_cache**/.pytest_cache**/.python-version**/.pycdiff --git a/docker/dockerfiles/runner.Dockerfile b/docker/dockerfiles/runner.Dockerfileindex b61d2ef4a..7f788c418 100644--- a/docker/dockerfiles/runner.Dockerfile+++ b/docker/dockerfiles/runner.Dockerfile@@ -1,13 +1,13 @@# Use a specific version of Python slim image-FROM python:3.12.9-slim+FROM python:3.12.9-slim AS base-LABEL maintainer="Zipstack Inc."-LABEL description="Runner Service Container"-LABEL version="1.0"+ARG VERSION=dev+LABEL maintainer="Zipstack Inc." \+ description="Runner Service Container" \+ version="${VERSION}"# Set environment variablesENV PYTHONDONTWRITEBYTECODE=1 \- # Set to immediately flush stdout and stderr streams without first bufferingPYTHONUNBUFFERED=1 \PYTHONPATH=/unstract \BUILD_CONTEXT_PATH=runner \@@ -19,41 +19,54 @@ ENV PYTHONDONTWRITEBYTECODE=1 \OTEL_LOGS_EXPORTER=none \OTEL_SERVICE_NAME=unstract_runner+# Install system dependenciesRUN apt-get update \- && apt-get --no-install-recommends install -y docker git\- && apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*-+ && apt-get --no-install-recommends install -y \+ docker \+ git \+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*# Install uv package managerCOPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/+# Create working directoryWORKDIR ${APP_HOME}-# Copy only requirements files first to leverage Docker cache-COPY ${BUILD_CONTEXT_PATH}/pyproject.toml .-COPY ${BUILD_CONTEXT_PATH}/uv.lock .-# Copy local dependency packages+# -----------------------------------------------+# EXTERNAL DEPENDENCIES STAGE - This layer gets cached if external dependencies don't change+# -----------------------------------------------+FROM base AS ext-dependencies++# Copy dependency-related files+COPY ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ${BUILD_CONTEXT_PATH}/README.md ./++# Copy local package dependenciesCOPY ${BUILD_PACKAGES_PATH}/core /unstract/coreCOPY ${BUILD_PACKAGES_PATH}/flags /unstract/flags-COPY ${BUILD_CONTEXT_PATH} /app/--# Create virtual environment and install dependencies in one layer-RUN uv sync --frozen \- && uv sync --group deploy \- && .venv/bin/python3 -m ensurepip --upgrade \- && uv run opentelemetry-bootstrap -a install--RUN \- uv pip install --system; \- \- if [ -f cloud_requirements.txt ]; then \- uv pip install --no-cache-dir -r cloud_requirements.txt; \++# Install external dependencies from pyproject.toml+RUN uv sync --group deploy --locked --no-install-project --no-dev && \+ uv run opentelemetry-bootstrap -a install++# -----------------------------------------------+# FINAL STAGE - Minimal image for production+# -----------------------------------------------+FROM ext-dependencies AS production++# Copy application code (this layer changes most frequently)+COPY ${BUILD_CONTEXT_PATH} ./++# Install the application+RUN uv sync --group deploy --no-dev --locked++# Install cloud requirements if they exist+RUN if [ -f cloud_requirements.txt ]; then \+ uv pip install --system -r cloud_requirements.txt; \else \echo "cloud_requirements.txt does not exist"; \fiEXPOSE 5002-# During debugging, this entry point will be overridden. For more information, please refer to https://aka.ms/vscode-docker-python-debug-# The suggested maximum concurrent requests when using workers and threads is (2*CPU)+1CMD [ "./entrypoint.sh" ]diff --git a/docker/dockerfiles/runner.Dockerfile.dockerignore b/docker/dockerfiles/runner.Dockerfile.dockerignoreindex 5e1595cc4..b868173e8 100644--- a/docker/dockerfiles/runner.Dockerfile.dockerignore+++ b/docker/dockerfiles/runner.Dockerfile.dockerignore@@ -1,4 +1,5 @@**/__pycache__+**/.mypy_cache**/.pytest_cache**/.python-version**/.pycdiff --git a/docker/dockerfiles/tool-sidecar.Dockerfile b/docker/dockerfiles/tool-sidecar.Dockerfileindex ae0662a5f..b77ab56f5 100644--- a/docker/dockerfiles/tool-sidecar.Dockerfile+++ b/docker/dockerfiles/tool-sidecar.Dockerfile@@ -1,15 +1,19 @@# Use Python 3.12.9-slim for minimal size-FROM python:3.12.9-slim-LABEL maintainer="Zipstack Inc."+FROM python:3.12.9-slim AS base-ENV \- PYTHONDONTWRITEBYTECODE=1 \+ARG VERSION=dev+LABEL maintainer="Zipstack Inc." \+ description="Tool Sidecar Container" \+ version="${VERSION}"++ENV PYTHONDONTWRITEBYTECODE=1 \PYTHONUNBUFFERED=1 \LOG_PATH=/shared/logs/logs.txt \LOG_LEVEL=INFO \- PYTHONPATH=/unstract \- BUILD_CONTEXT_PATH=tool-sidecar \BUILD_PACKAGES_PATH=unstract \+ BUILD_CONTEXT_PATH=tool-sidecar \+ APP_USER=unstract \+ APP_HOME=/app \# OpenTelemetry configuration (disabled by default, enable in docker-compose)OTEL_TRACES_EXPORTER=none \OTEL_METRICS_EXPORTER=none \@@ -17,32 +21,53 @@ ENV \OTEL_SERVICE_NAME=unstract_tool_sidecar \PATH="/app/.venv/bin:$PATH"+# Install system dependencies in a single layerRUN apt-get update \- && apt-get --no-install-recommends install -y docker \- && apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*+ && apt-get --no-install-recommends install -y \+ docker \+ build-essential \+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* \+ && adduser --uid 5678 --disabled-password --gecos "" ${APP_USER} \+ && mkdir -p ${APP_HOME} \+ && chown -R ${APP_USER}:${APP_USER} ${APP_HOME}# Install uv package managerCOPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/-WORKDIR /app+WORKDIR ${APP_HOME}++# -----------------------------------------------+# EXTERNAL DEPENDENCIES STAGE+# -----------------------------------------------+FROM base AS ext-dependencies++# Copy dependency-related files+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ${BUILD_CONTEXT_PATH}/README.md ./++# Copy local package dependencies+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_PACKAGES_PATH}/core /unstract/core++# Install external dependencies from pyproject.toml+RUN uv sync --group deploy --locked --no-install-project --no-dev && \+ uv run opentelemetry-bootstrap -a install-# Copy local dependency packages first-COPY ${BUILD_CONTEXT_PATH}/pyproject.toml .-COPY ${BUILD_CONTEXT_PATH}/uv.lock .+# -----------------------------------------------+# FINAL STAGE - Minimal image for production+# -----------------------------------------------+FROM ext-dependencies AS production-# Copy local dependency packages-COPY ${BUILD_PACKAGES_PATH}/core /unstract/core+# Copy application code (this layer changes most frequently)+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH} ./-# Copy application files-COPY ${BUILD_CONTEXT_PATH} /app/+# Install just the application+RUN uv sync --group deploy --locked && \+ chmod +x ./entrypoint.sh-# Install dependencies in a single layer-RUN uv sync --frozen \- && uv sync --group deploy \- && .venv/bin/python3 -m ensurepip --upgrade \- && uv run opentelemetry-bootstrap -a install+# # Make entrypoint executable+# RUN chmod +x ./entrypoint.sh-COPY ${BUILD_CONTEXT_PATH}/entrypoint.sh /app/-RUN chmod +x /app/entrypoint.sh+# Switch to non-root user+USER ${APP_USER}CMD ["./entrypoint.sh"]diff --git a/docker/dockerfiles/tool-sidecar.Dockerfile.dockerignore b/docker/dockerfiles/tool-sidecar.Dockerfile.dockerignoreindex 51dc2dd3a..caf9b619b 100644--- a/docker/dockerfiles/tool-sidecar.Dockerfile.dockerignore+++ b/docker/dockerfiles/tool-sidecar.Dockerfile.dockerignore@@ -1,4 +1,5 @@**/__pycache__+**/.mypy_cache**/.pytest_cache**/.python-version**/.pycdiff --git a/docker/dockerfiles/x2text.Dockerfile b/docker/dockerfiles/x2text.Dockerfileindex c72fbe0c8..c3de9b66f 100644--- a/docker/dockerfiles/x2text.Dockerfile+++ b/docker/dockerfiles/x2text.Dockerfile@@ -1,9 +1,10 @@# Use a specific version of Python slim image-FROM python:3.12.9-slim+FROM python:3.12.9-slim AS base+ARG VERSION=devLABEL maintainer="Zipstack Inc." \description="X2Text Service Container" \- version="1.0"+ version="${VERSION}"# Set environment variablesENV PYTHONDONTWRITEBYTECODE=1 \@@ -19,8 +20,13 @@ ENV PYTHONDONTWRITEBYTECODE=1 \# Install system dependencies and create user in a single layerRUN apt-get update \- && apt-get install -y --no-install-recommends curl \- && rm -rf /var/lib/apt/lists/* \+ && apt-get install -y --no-install-recommends \+ build-essential \+ libmagic-dev \+ pkg-config \+ git \+ && apt-get clean \+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/* \&& adduser --uid 5678 --disabled-password --gecos "" ${APP_USER} \&& mkdir -p ${APP_HOME} \&& chown -R ${APP_USER}:${APP_USER} ${APP_HOME}@@ -30,23 +36,33 @@ COPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/WORKDIR ${APP_HOME}-# Copy only requirements files first to leverage Docker cache-COPY --chmod=755 ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ./+# -----------------------------------------------+# EXTERNAL DEPENDENCIES STAGE+# -----------------------------------------------+FROM base AS ext-dependencies-# Switch to non-root user-USER ${APP_USER}+# Copy dependency-related files+COPY ${BUILD_CONTEXT_PATH}/pyproject.toml ${BUILD_CONTEXT_PATH}/uv.lock ${BUILD_CONTEXT_PATH}/README.md ./++# Install external dependencies from pyproject.toml+RUN uv sync --group deploy --locked --no-install-project --no-dev && \+ uv run opentelemetry-bootstrap -a install++# -----------------------------------------------+# FINAL STAGE - Minimal image for production+# -----------------------------------------------+FROM ext-dependencies AS production-# Install dependencies-RUN uv sync --frozen \- && uv sync --group deploy \- && .venv/bin/python3 -m ensurepip --upgrade \- && uv run opentelemetry-bootstrap -a install+# Copy application code (this layer changes most frequently)+COPY --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH} ./-# Copy application code-COPY --chmod=755 --chown=${APP_USER}:${APP_USER} ${BUILD_CONTEXT_PATH} .+# Install just the application+RUN uv sync --group deploy --locked++# Switch to non-root user+USER ${APP_USER}EXPOSE 3004# During debugging, this entry point will be overridden.-# For more information, please refer to https://aka.ms/vscode-docker-python-debugCMD [".venv/bin/gunicorn", "--bind", "0.0.0.0:3004", "--timeout", "300", "run:app"]diff --git a/docker/dockerfiles/x2text.Dockerfile.dockerignore b/docker/dockerfiles/x2text.Dockerfile.dockerignoreindex 8972dd8bb..255b90427 100644--- a/docker/dockerfiles/x2text.Dockerfile.dockerignore+++ b/docker/dockerfiles/x2text.Dockerfile.dockerignore@@ -1,4 +1,5 @@**/__pycache__+**/.mypy_cache**/.pytest_cache**/.python-version**/.pycdiff --git a/tools/structure/Dockerfile b/tools/structure/Dockerfileindex 36276af02..e4d832a7e 100644--- a/tools/structure/Dockerfile+++ b/tools/structure/Dockerfile@@ -1,12 +1,15 @@FROM python:3.12.9-slim-LABEL maintainer="Zipstack Inc."+LABEL maintainer="Zipstack Inc." \+ description="Structure Tool Container" \+ version="1.0"ENV \# Keeps Python from generating .pyc files in the containerPYTHONDONTWRITEBYTECODE=1 \# Set to immediately flush stdout and stderr streams without first bufferingPYTHONUNBUFFERED=1 \+ APP_HOME=/app \# OpenTelemetry configurationOTEL_SERVICE_NAME="structure-tool" \OTEL_TRACES_EXPORTER=none \@@ -15,26 +18,37 @@ ENV \# Enable context propagationOTEL_PROPAGATORS="tracecontext"-# Install dependencies for SDK and prompt service-RUN apt-get update \- && apt-get install -y --no-install-recommends \+# Install system dependencies in one layer+RUN apt-get update && \+ apt-get install -y --no-install-recommends \ffmpeg libsm6 libxext6 libmagic-dev poppler-utils \- libreoffice freetds-dev freetds-bin \- && apt-get clean \- && rm -rf /var/lib/apt/lists/*+ libreoffice freetds-dev freetds-bin && \+ apt-get clean && \+ rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*++# Install uv package manager+COPY --from=ghcr.io/astral-sh/uv:0.6.14 /uv /uvx /bin/-RUN pip install --no-cache-dir -U pip# Set the working directory in the container-WORKDIR /app-COPY requirements.txt /app/-RUN pip install --no-cache-dir -r requirements.txt && \- pip install --no-cache-dir \+WORKDIR ${APP_HOME}++# Copy only requirements file first for better caching+COPY requirements.txt ${APP_HOME}/++# Install OpenTelemetry packages first (less likely to change)+RUN uv pip install --system \opentelemetry-distro \opentelemetry-exporter-otlp \- platformdirs>=3.0.0 \- && opentelemetry-bootstrap -a install-# Copy the contents of your project directory into the container at /app-COPY src /app/src/-WORKDIR /app/src+ platformdirs>=3.0.0 && \+ opentelemetry-bootstrap -a install++# Install project dependencies separately from OpenTelemetry+# This allows for better caching when only project dependencies change+RUN uv pip install --system -r requirements.txt++# Copy the source code after installing all dependencies+# This ensures that changes to the source code don't invalidate the dependency layers+COPY src ${APP_HOME}/src/+WORKDIR ${APP_HOME}/srcENTRYPOINT ["opentelemetry-instrument", "python", "main.py"]