LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Python ai-agentsdata-engineeringdocument-aigenerative-aiidpjson-extractionllmmcp-serverocrpdf-extractionprompt-engineeringstructured-output
Added adaptor details to README. Moved assets folder to docs/assets (#162)
Shuveb Hussain0bcd73007a6b
diff --git a/README.md b/README.mdindex d65fabceb..12f473d64 100644--- a/README.md+++ b/README.md@@ -1,5 +1,5 @@<div align="center">-<img src="assets/unstract_u_logo.png" style="height: 120px">+<img src="docs/assets/unstract_u_logo.png" style="height: 120px"># Unstract@@ -11,7 +11,7 @@While co-pilots enable machine-to-human automation, with Unstract, you can go beyond co-pilots by enabling machine-to-machine automation. You can launch APIs that take in complex documents and return structured JSON all with a simple no-code approach. You can also launch unstructured data ETL Pipelines that can read complex documents from a variety of cloud file / object storage systems and write structured data into popular data warehouses and databases.-+## 🧘♀️ Three step nirvana@@ -21,11 +21,11 @@ Automate critical business processes that involve complex documents with a human🌟 **Step 2**: Configure Prompt Studio project as API deployment or configure input source and output destination for ETL Pipeline<br>🌟 **Step 3**: Deploy Workflows as unstructured data APIs or unstructured data ETL Pipelines!-+## 🚀 Getting started-The easiest way to get started is with Docker. Clone this repo and do the following:+The easiest way to get started is with Docker. Either download a release or clone this repo and do the following:✅ `./run-platform.sh`<br>✅ Now visit [http://frontend.unstract.localhost](http://frontend.unstract.localhost) in your browser <br>@@ -33,10 +33,76 @@ The easiest way to get started is with Docker. Clone this repo and do the followThat's all there is to it!+Another really quick way to experience Unstract is by signing up for our [hosted version](https://us-central-1.gcp.unstract.com/).+## ⏩ Quick Start GuideUnstract comes well documented. You can get introduced to the [basics of Unstract](https://docs.unstract.com/), and [learn how to connect](https://docs.unstract.com/unstract_platform/setup_accounts/whats_needed) various systems like LLMs, Vector Databases, Embedding Models and Text Extractors to it. The easiest way to wet your feet is to go through our [Quick Start Guide](https://docs.unstract.com/unstract_platform/quick_start) where you actually get to do some prompt engineering in Prompt Studio and launch an API to structure varied credit card statements!+## 🤝 Ecosystem support++### LLM Providers+|| Provider | Status |+|---|---|---|+|<img src="docs/assets/3rd_party/openai.png" width="32"/>| OpenAI | ✅ Working |+|<img src="docs/assets/3rd_party/vertex_ai.png" width="32"/>| Google VertexAI, Gemini Pro | ✅ Working |+|<img src="docs/assets/3rd_party/azure_openai.png" width="32"/>| Azure OpenAI | ✅ Working |+|<img src="docs/assets/3rd_party/palm.png" width="32"/>| Google PaLM | ✅ Working |+|<img src="docs/assets/3rd_party/anyscale.png" width="32"/>| Anyscale | ✅ Working |+|<img src="docs/assets/3rd_party/replicate.png" width="32"/>| Replicate | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/anthropic.png" width="32"/>| Anthropic | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/mistral_ai.png" width="32"/>| Mistral AI | 🗓️ Coming soon! |++### Vector Databases+|| Provider | Status |+|---|---|---|+|<img src="docs/assets/3rd_party/qdrant.png" width="32"/>| Qdrant | ✅ Working |+|<img src="docs/assets/3rd_party/postgres.png" width="32"/>| PostgreSQL | ✅ Working |+|<img src="docs/assets/3rd_party/supabase.png" width="32"/>| Supabase | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/milvus.png" width="32"/>| Milvus | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/weaviate.png" width="32"/>| Weaviate | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/pinecone.png" width="32"/>| Pinecone | 🗓️ Coming soon! |++### Embeddings+|| Provider | Status |+|---|---|---|+|<img src="docs/assets/3rd_party/openai.png" width="32"/>| OpenAI | ✅ Working |+|<img src="docs/assets/3rd_party/azure_openai.png" width="32"/>| Azure OpenAI | ✅ Working |+|<img src="docs/assets/3rd_party/palm.png" width="32"/>| Google PaLM | ✅ Working |+|<img src="docs/assets/3rd_party/qdrant.png" width="32"/>| Qdrant | ✅ Working |+|<img src="docs/assets/3rd_party/huggingface.png" width="32"/>| Huggingface | 🗓️ Coming soon! |++### Text Extractors+|| Provider | Status |+|---|---|---|+|<img src="docs/assets/unstract_u_logo.png" width="32"/>| Unstract LLMWhisperer | ✅ Working |+|<img src="docs/assets/3rd_party/unstructured_io.png" width="32"/>| Unstructured.io Community | ✅ Working |+|<img src="docs/assets/3rd_party/unstructured_io.png" width="32"/>| Unstructured.io Enterprise | ✅ Working |+|<img src="docs/assets/3rd_party/llamaindex.png" width="32"/>| LlamaIndex Parse | 🗓️ Coming soon! |++### ETL Sources+|| Provider | Status |+|---|---|---|+|<img src="docs/assets/3rd_party/s3.png" width="32"/>| AWS S3 | ✅ Working |+|<img src="docs/assets/3rd_party/minio.png" width="32"/>| Minio | ✅ Working |+|<img src="docs/assets/3rd_party/dropbox.png" width="32"/>| Dropbox | ✅ Working |+|<img src="docs/assets/3rd_party/google_drive.png" width="32"/>| Google Drive | ✅ Working |+|<img src="docs/assets/3rd_party/box.png" width="32"/>| Box | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/gcp.png" width="32"/>| Google Cloud Storage | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/azure_openai.png" width="32"/>| Azure Cloud Storage | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/http.png" width="32"/>| HTTP/HTTPS | 🗓️ Coming soon! |++### ETL Destinations+|| Provider | Status |+|---|---|---|+|<img src="docs/assets/3rd_party/snowflake.png" width="32"/>| Snowflake | ✅ Working |+|<img src="docs/assets/3rd_party/amazon_redshift.png" width="32"/>| Amazon Redshift | ✅ Working |+|<img src="docs/assets/3rd_party/google_bigquery.png" width="32"/>| Google Bigquery | ✅ Working |+|<img src="docs/assets/3rd_party/postgres.png" width="32"/>| PostgreSQL | ✅ Working |+|<img src="docs/assets/3rd_party/mysql.png" width="32"/>| MySQL | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/mariadb.png" width="32"/>| MariaDB | 🗓️ Coming soon! |+|<img src="docs/assets/3rd_party/ms_sql.png" width="32"/>| Microsoft SQL Server | 🗓️ Coming soon! |+## 🙌 ContributingContributions are welcome! Please read [CONTRIBUTE.md](CONTRIBUTE.md) for further details on setting up the development environment, etc. It also points you to other detailed documents as needed.diff --git a/docs/assets/3rd_party/amazon_redshift.png b/docs/assets/3rd_party/amazon_redshift.pngnew file mode 100644index 000000000..51ac003c7Binary files /dev/null and b/docs/assets/3rd_party/amazon_redshift.png differdiff --git a/docs/assets/3rd_party/anthropic.png b/docs/assets/3rd_party/anthropic.pngnew file mode 100644index 000000000..ebeda3877Binary files /dev/null and b/docs/assets/3rd_party/anthropic.png differdiff --git a/docs/assets/3rd_party/anyscale.png b/docs/assets/3rd_party/anyscale.pngnew file mode 100644index 000000000..8cba20322Binary files /dev/null and b/docs/assets/3rd_party/anyscale.png differdiff --git a/docs/assets/3rd_party/azure_openai.png b/docs/assets/3rd_party/azure_openai.pngnew file mode 100644index 000000000..7c750318dBinary files /dev/null and b/docs/assets/3rd_party/azure_openai.png differdiff --git a/docs/assets/3rd_party/box.png b/docs/assets/3rd_party/box.pngnew file mode 100644index 000000000..c94c3b38dBinary files /dev/null and b/docs/assets/3rd_party/box.png differdiff --git a/docs/assets/3rd_party/cohere.png b/docs/assets/3rd_party/cohere.pngnew file mode 100644index 000000000..4b7b29768Binary files /dev/null and b/docs/assets/3rd_party/cohere.png differdiff --git a/docs/assets/3rd_party/dropbox.png b/docs/assets/3rd_party/dropbox.pngnew file mode 100644index 000000000..1ab04552dBinary files /dev/null and b/docs/assets/3rd_party/dropbox.png differdiff --git a/docs/assets/3rd_party/gcp.png b/docs/assets/3rd_party/gcp.pngnew file mode 100644index 000000000..3f88a2a27Binary files /dev/null and b/docs/assets/3rd_party/gcp.png differdiff --git a/docs/assets/3rd_party/google.png b/docs/assets/3rd_party/google.pngnew file mode 100644index 000000000..2b85ade18Binary files /dev/null and b/docs/assets/3rd_party/google.png differdiff --git a/docs/assets/3rd_party/google_bigquery.png b/docs/assets/3rd_party/google_bigquery.pngnew file mode 100644index 000000000..037fae952Binary files /dev/null and b/docs/assets/3rd_party/google_bigquery.png differdiff --git a/docs/assets/3rd_party/google_drive.png b/docs/assets/3rd_party/google_drive.pngnew file mode 100644index 000000000..aa64a40c7Binary files /dev/null and b/docs/assets/3rd_party/google_drive.png differdiff --git a/docs/assets/3rd_party/http.png b/docs/assets/3rd_party/http.pngnew file mode 100644index 000000000..83b931834Binary files /dev/null and b/docs/assets/3rd_party/http.png differdiff --git a/docs/assets/3rd_party/huggingface.png b/docs/assets/3rd_party/huggingface.pngnew file mode 100644index 000000000..6740fec74Binary files /dev/null and b/docs/assets/3rd_party/huggingface.png differdiff --git a/docs/assets/3rd_party/llamaindex.png b/docs/assets/3rd_party/llamaindex.pngnew file mode 100644index 000000000..54cd1cd94Binary files /dev/null and b/docs/assets/3rd_party/llamaindex.png differdiff --git a/docs/assets/3rd_party/llm_whisperer.png b/docs/assets/3rd_party/llm_whisperer.pngnew file mode 100644index 000000000..aed4e7797Binary files /dev/null and b/docs/assets/3rd_party/llm_whisperer.png differdiff --git a/docs/assets/3rd_party/mariadb.png b/docs/assets/3rd_party/mariadb.pngnew file mode 100644index 000000000..b94538defBinary files /dev/null and b/docs/assets/3rd_party/mariadb.png differdiff --git a/docs/assets/3rd_party/milvus.png b/docs/assets/3rd_party/milvus.pngnew file mode 100644index 000000000..4d3214c7dBinary files /dev/null and b/docs/assets/3rd_party/milvus.png differdiff --git a/docs/assets/3rd_party/minio.png b/docs/assets/3rd_party/minio.pngnew file mode 100644index 000000000..64c15a5deBinary files /dev/null and b/docs/assets/3rd_party/minio.png differdiff --git a/docs/assets/3rd_party/mistral_ai.png b/docs/assets/3rd_party/mistral_ai.pngnew file mode 100644index 000000000..65c8e7e36Binary files /dev/null and b/docs/assets/3rd_party/mistral_ai.png differdiff --git a/docs/assets/3rd_party/ms_sql.png b/docs/assets/3rd_party/ms_sql.pngnew file mode 100644index 000000000..64279bddbBinary files /dev/null and b/docs/assets/3rd_party/ms_sql.png differdiff --git a/docs/assets/3rd_party/mysql.png b/docs/assets/3rd_party/mysql.pngnew file mode 100644index 000000000..a501af1f3Binary files /dev/null and b/docs/assets/3rd_party/mysql.png differdiff --git a/docs/assets/3rd_party/openai.png b/docs/assets/3rd_party/openai.pngnew file mode 100644index 000000000..57c3aa7a5Binary files /dev/null and b/docs/assets/3rd_party/openai.png differdiff --git a/docs/assets/3rd_party/palm.png b/docs/assets/3rd_party/palm.pngnew file mode 100644index 000000000..5739ec3fbBinary files /dev/null and b/docs/assets/3rd_party/palm.png differdiff --git a/docs/assets/3rd_party/pinecone.png b/docs/assets/3rd_party/pinecone.pngnew file mode 100644index 000000000..ac397b4aaBinary files /dev/null and b/docs/assets/3rd_party/pinecone.png differdiff --git a/docs/assets/3rd_party/postgres.png b/docs/assets/3rd_party/postgres.pngnew file mode 100644index 000000000..cc7d07fffBinary files /dev/null and b/docs/assets/3rd_party/postgres.png differdiff --git a/docs/assets/3rd_party/qdrant.png b/docs/assets/3rd_party/qdrant.pngnew file mode 100644index 000000000..17903e262Binary files /dev/null and b/docs/assets/3rd_party/qdrant.png differdiff --git a/docs/assets/3rd_party/replicate.png b/docs/assets/3rd_party/replicate.pngnew file mode 100644index 000000000..525f9993bBinary files /dev/null and b/docs/assets/3rd_party/replicate.png differdiff --git a/docs/assets/3rd_party/s3.png b/docs/assets/3rd_party/s3.pngnew file mode 100644index 000000000..8bb301dccBinary files /dev/null and b/docs/assets/3rd_party/s3.png differdiff --git a/docs/assets/3rd_party/snowflake.png b/docs/assets/3rd_party/snowflake.pngnew file mode 100644index 000000000..74b99d8a5Binary files /dev/null and b/docs/assets/3rd_party/snowflake.png differdiff --git a/docs/assets/3rd_party/supabase.png b/docs/assets/3rd_party/supabase.pngnew file mode 100644index 000000000..843be59caBinary files /dev/null and b/docs/assets/3rd_party/supabase.png differdiff --git a/docs/assets/3rd_party/unstructured_io.png b/docs/assets/3rd_party/unstructured_io.pngnew file mode 100644index 000000000..a6ec855b6Binary files /dev/null and b/docs/assets/3rd_party/unstructured_io.png differdiff --git a/docs/assets/3rd_party/vertex_ai.png b/docs/assets/3rd_party/vertex_ai.pngnew file mode 100644index 000000000..3affe8931Binary files /dev/null and b/docs/assets/3rd_party/vertex_ai.png differdiff --git a/docs/assets/3rd_party/weaviate.png b/docs/assets/3rd_party/weaviate.pngnew file mode 100644index 000000000..9c4b489a9Binary files /dev/null and b/docs/assets/3rd_party/weaviate.png differdiff --git a/assets/Using_Unstract.png b/docs/assets/Using_Unstract.pngsimilarity index 100%rename from assets/Using_Unstract.pngrename to docs/assets/Using_Unstract.pngdiff --git a/assets/prompt_studio.png b/docs/assets/prompt_studio.pngsimilarity index 100%rename from assets/prompt_studio.pngrename to docs/assets/prompt_studio.pngdiff --git a/assets/unstract_u_logo.png b/docs/assets/unstract_u_logo.pngsimilarity index 100%rename from assets/unstract_u_logo.pngrename to docs/assets/unstract_u_logo.png