Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Update documentation

davidmezzetti76852202ce0e
diff --git a/docs/api/index.md b/docs/api/index.md
index 5445a62d..78c0e6bf 100644
--- a/docs/api/index.md
+++ b/docs/api/index.md
@@ -95,7 +95,7 @@ A local instance can be instantiated. In this case, a txtai application runs int
The configuration above can be run in Python with:
```python
-from txtai.app import Application
+from txtai import Application
# Load and run workflow
app = Application(config.yml)
diff --git a/docs/embeddings/index.md b/docs/embeddings/index.md
index c7fe5c4e..358af323 100644
--- a/docs/embeddings/index.md
+++ b/docs/embeddings/index.md
@@ -8,7 +8,7 @@ Embeddings databases are the engine that delivers semantic search. Data is trans
The following code snippet shows how to build and search an embeddings index.
```python
-from txtai.embeddings import Embeddings
+from txtai import Embeddings
# Create embeddings model, backed by sentence-transformers & transformers
embeddings = Embeddings(path="sentence-transformers/nli-mpnet-base-v2")
diff --git a/docs/pipeline/text/llm.md b/docs/pipeline/text/llm.md
index 75b3b7ca..32375568 100644
--- a/docs/pipeline/text/llm.md
+++ b/docs/pipeline/text/llm.md
@@ -10,10 +10,12 @@ The LLM pipeline runs prompts through a large language model (LLM). This pipelin
The following shows a simple example using this pipeline.
```python
-from txtai.pipeline import LLM
+from txtai import LLM
-# Create and run LLM pipeline
+# Create LLM pipeline
llm = LLM()
+
+# Run prompt
llm(
"""
Answer the following question using the provided context.
@@ -26,21 +28,38 @@ llm(
workflows powered by language models.
"""
)
+
+# Chat messages are also supported
+llm([
+ {"role": "system", "content": "You are a friendly assistant."},
+ {"role": "user", "content": "Answer the following question..."}
+])
+
```
The LLM pipeline automatically detects the underlying LLM framework. This can also be manually set.
-See the [LiteLLM documentation](https://litellm.vercel.app/docs/providers) for the options available with LiteLLM models. Models with llama.cpp support both local and remote GGUF paths on the HF Hub.
+See the [LiteLLM documentation](https://litellm.vercel.app/docs/providers) for the options available with LiteLLM models. llama.cpp models support both local and remote GGUF paths on the HF Hub.
```python
-from txtai.pipeline import LLM
+from txtai import LLM
-# Set method as litellm
-llm = LLM("vllm/Open-Orca/Mistral-7B-OpenOrca", method="litellm")
+# Transformers
+llm = LLM("google/gemma-2-9b")
+llm = LLM("google/gemma-2-9b", method="transformers")
-# Set method as llama.cpp
-llm = LLM("TheBloke/Mistral-7B-OpenOrca-GGUF/mistral-7b-openorca.Q4_K_M.gguf",
+# llama.cpp
+llm = LLM("microsoft/Phi-3-mini-4k-instruct-gguf/microsoft/Phi-3-mini-4k-instruct-gguf")
+llm = LLM("microsoft/Phi-3-mini-4k-instruct-gguf/microsoft/Phi-3-mini-4k-instruct-gguf",
method="llama.cpp")
+
+# LiteLLM
+llm = LLM("ollama/llama3")
+llm = LLM("ollama/llama3", method="litellm")
+
+# LLM APIs - must also set API key via environment variable
+llm = LLM("gpt-4o")
+llm = LLM("claude-3-5-sonnet-20240620")
```
Models can be externally loaded and passed to pipelines. This is useful for models that are not yet supported by Transformers and/or need special initialization.
@@ -49,7 +68,7 @@ Models can be externally loaded and passed to pipelines. This is useful for mode
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
-from txtai.pipeline import LLM
+from txtai import LLM
# Load Mistral-7B-OpenOrca
path = "Open-Orca/Mistral-7B-OpenOrca"
@@ -84,7 +103,6 @@ Pipelines are run with Python or configuration. Pipelines can be instantiated in
### config.yml
```yaml
# Create pipeline using lower case class name
-# Use `generator` or `sequences` to force model type
llm:
# Run pipeline with workflow
@@ -131,7 +149,7 @@ CONFIG=config.yml uvicorn "txtai.api:app" &
curl \
-X POST "http://localhost:8000/workflow" \
-H "Content-Type: application/json" \
- -d '{"name":"sequences", "elements": ["Answer the following question..."]}'
+ -d '{"name":"llm", "elements": ["Answer the following question..."]}'
```
## Methods
diff --git a/docs/pipeline/text/rag.md b/docs/pipeline/text/rag.md
index 5cded887..a645202c 100644
--- a/docs/pipeline/text/rag.md
+++ b/docs/pipeline/text/rag.md
@@ -12,19 +12,8 @@ The data store can be an embeddings database or a similarity instance with assoc
The following shows a simple example using this pipeline.
```python
-from txtai.embeddings import Embeddings
-from txtai.pipeline import RAG
-
-# LLM prompt
-def prompt(question):
- return f"""
- Answer the following question using the provided context.
-
- Question:
- {question}
-
- Context:
- """
+from txtai import Embeddings
+from txtai import RAG
# Input data
data = [
@@ -40,11 +29,20 @@ data = [
# Build embeddings index
embeddings = Embeddings({"content": True})
-embeddings.index([(uid, text, None) for uid, text in enumerate(data)])
+embeddings.index(data)
# Create and run pipeline
-extractor = RAG(embeddings, "google/flan-t5-base")
-extractor([{"query": "What was won?", "question": prompt("What was won?")}])
+rag = RAG(embeddings, "google/flan-t5-base", template="""
+ Answer the following question using the provided context.
+
+ Question:
+ {question}
+
+ Context:
+ {context}
+""")
+
+rag("What was won?")
```
See the links below for more detailed examples.
@@ -78,21 +76,21 @@ writable: True
embeddings:
content: True
-extractor:
+rag:
path: google/flan-t5-base
+ template: |
+ Answer the following question using the provided context.
+
+ Question:
+ {question}
+
+ Context:
+ {context}
workflow:
search:
tasks:
- - task: extractor
- template: |
- Answer the following question using the provided context.
-
- Question:
- {text}
-
- Context:
- action: extractor
+ - action: rag
```
### Run with Workflows