Skip to content

LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows

docs: Updated README.md on LLMWhisperer setup (#80)

Chandrasekharan M3041fa41a9df
diff --git a/README.md b/README.md
index 271aa3694..c1389f072 100644
--- a/README.md
+++ b/README.md
@@ -47,11 +47,27 @@ Now visit [http://frontend.unstract.localhost](http://frontend.unstract.localhos
That's all. Enjoy!
+## Configuring a Text Extractor
+
+Unstract predominantly works with PDF documents and it requires a `Text Extractor` to be configured in the application which helps retrieve text from the documents. Currently supported text extractors include
+
+- [LLMWhisperer](https://unstract-api-resource.developer.azure-api.net/) (works best)
+- Unstructured Community
+- Unstructured Enterprise
+
+### Steps to use LLMWhisperer Service
+
+[LLMWhisperer](https://unstract-api-resource.developer.azure-api.net/) is our text extraction service which provides best results with Unstract.
+- Create an account in the [developer portal](https://unstract-api-resource.developer.azure-api.net/signup)
+- Create a `Subscription` under [your profile](https://unstract-api-resource.developer.azure-api.net/profile) and copy the `Primary Key`
+- Try the APIs from the portal by passing the copied key in the request header `unstract-key`
+- This key needs to be passed in our application while creating an `LLM Whisperer Text Extractor`
+
## Running with docker compose
See [Docker README.md](docker/README.md).
-## Running locally
+## Setup Unstract for local development
### Installation
@@ -83,7 +99,6 @@ eval "$(pdm venv activate in-project)"
pdm venv remove in-project
```
-
### Install dependencies with PDM
[PDM](https://github.com/pdm-project/pdm) is used for dependency management.
@@ -205,15 +220,6 @@ We just need to override default Traefik proxy routing to allow this, that's all
Run the services.
-#### Generate Encryption key to be used in `backend` and `platform-service`
-
-An encryption key is used to securely encrypt and store data, for example credentials of connectors or adapters.
-We make use of [cryptography's](https://pypi.org/project/cryptography/) Fernet to perform this encryption. Use this snippet to generate a key that can be set in your respective `backend` and `platform-service` `.env` files.
-
-```bash
-ENCRYPTION_KEY=$(python -c "import secrets, base64; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())")
-```
-
#### Conflicting Host Names
When same host name environment variables are used by both the service running locally and a service
@@ -232,3 +238,12 @@ locally and the Tools are in containers, we could set the value to
```
<host_local_ip> prompt-service
```
+
+## Generate Encryption key to be used in `backend` and `platform-service`
+
+An encryption key is used to securely encrypt and store data, for example credentials of connectors or adapters.
+We make use of [cryptography's](https://pypi.org/project/cryptography/) Fernet to perform this encryption. Use this snippet to generate a key that can be set in your respective `backend` and `platform-service` `.env` files.
+
+```bash
+ENCRYPTION_KEY=$(python -c "import secrets, base64; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())")
+```