LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Python ai-agentsdata-engineeringdocument-aigenerative-aiidpjson-extractionllmmcp-serverocrpdf-extractionprompt-engineeringstructured-output
Docker compose improvements and cleanup (#54)
Jaseem Jas05ba7da04c19
diff --git a/README.md b/README.mdindex 509c369e2..565e84400 100644--- a/README.md+++ b/README.md@@ -118,9 +118,6 @@ POSTGRES_DB: unstract_dbIf you require a different config, make sure the necessary envs from [backend/sample.env](/backend/sample.env) are exported.-- Execute the script [backend/init.sql](/backend/init.sql) that adds roles and creates a DB and extension for ZS Document Indexer tool to work.-Make sure that [pgvector](https://github.com/pgvector/pgvector#installation) is installed.-### Pre-commit hooks- We use pre-commit to run some hooks whenever code is pushed to perform linting and static code analysis among other checks.diff --git a/backend/init.sql b/backend/init.sqlindex 1463fe23f..a4c659986 100644--- a/backend/init.sql+++ b/backend/init.sql@@ -3,5 +3,4 @@ ALTER ROLE unstract_dev SET default_transaction_isolation TO 'read committed';ALTER ROLE unstract_dev SET timezone TO 'UTC';ALTER USER unstract_dev CREATEDB;GRANT ALL PRIVILEGES ON DATABASE unstract_db TO unstract_dev;-CREATE DATABASE unstract;-CREATE EXTENSION vector;+CREATE DATABASE unstract;\ No newline at end of filediff --git a/docker/docker-compose-dev-essentials.yaml b/docker/docker-compose-dev-essentials.yamlindex ad6bc3ffd..7962681fa 100644--- a/docker/docker-compose-dev-essentials.yaml+++ b/docker/docker-compose-dev-essentials.yaml@@ -2,9 +2,10 @@ version: '3.7'services:db:- # Supports pgvector for doc indexer- image: 'ankane/pgvector:v0.5.1'+ image: 'postgres:15.6'container_name: unstract-db+ # set shared memory limit when using docker-compose+ shm_size: 128mbports:- "5432:5432"volumes:@@ -14,6 +15,7 @@ services:- ./essentials.envlabels:- traefik.enable=false+ restart: alwaysredis:image: 'bitnami/redis:7.0.14'@@ -103,7 +105,7 @@ services:- traefik.enable=falseqdrant:- # Supports pgvector for doc indexer+ # Vector DB for doc indexerimage: 'qdrant/qdrant'container_name: unstract-vector-dbports:diff --git a/worker/requirements.txt b/worker/requirements.txtdeleted file mode 100644index d49c0c909..000000000--- a/worker/requirements.txt+++ /dev/null@@ -1,5 +0,0 @@-docker==6.1.3-flask==3.0.0-python-dotenv==1.0.0-redis==5.0.1-gunicorn==20.1.0\ No newline at end of file