💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
Python agentsaiai-agentsembeddingsinformation-retrievallanguage-modellarge-language-modelsllmnlppythonragretrieval-augmented-generationsearchsearch-enginesemantic-searchsentence-embeddingstransformerstxtaivector-databasevector-search
Update documentation
davidmezzetti76852202ce0e
diff --git a/docs/api/index.md b/docs/api/index.mdindex 5445a62d..78c0e6bf 100644--- a/docs/api/index.md+++ b/docs/api/index.md@@ -95,7 +95,7 @@ A local instance can be instantiated. In this case, a txtai application runs intThe configuration above can be run in Python with:```python-from txtai.app import Application+from txtai import Application# Load and run workflowapp = Application(config.yml)diff --git a/docs/embeddings/index.md b/docs/embeddings/index.mdindex c7fe5c4e..358af323 100644--- a/docs/embeddings/index.md+++ b/docs/embeddings/index.md@@ -8,7 +8,7 @@ Embeddings databases are the engine that delivers semantic search. Data is transThe following code snippet shows how to build and search an embeddings index.```python-from txtai.embeddings import Embeddings+from txtai import Embeddings# Create embeddings model, backed by sentence-transformers & transformersembeddings = Embeddings(path="sentence-transformers/nli-mpnet-base-v2")diff --git a/docs/pipeline/text/llm.md b/docs/pipeline/text/llm.mdindex 75b3b7ca..32375568 100644--- a/docs/pipeline/text/llm.md+++ b/docs/pipeline/text/llm.md@@ -10,10 +10,12 @@ The LLM pipeline runs prompts through a large language model (LLM). This pipelinThe following shows a simple example using this pipeline.```python-from txtai.pipeline import LLM+from txtai import LLM-# Create and run LLM pipeline+# Create LLM pipelinellm = LLM()++# Run promptllm("""Answer the following question using the provided context.@@ -26,21 +28,38 @@ llm(workflows powered by language models.""")++# Chat messages are also supported+llm([+ {"role": "system", "content": "You are a friendly assistant."},+ {"role": "user", "content": "Answer the following question..."}+])+```The LLM pipeline automatically detects the underlying LLM framework. This can also be manually set.-See the [LiteLLM documentation](https://litellm.vercel.app/docs/providers) for the options available with LiteLLM models. Models with llama.cpp support both local and remote GGUF paths on the HF Hub.+See the [LiteLLM documentation](https://litellm.vercel.app/docs/providers) for the options available with LiteLLM models. llama.cpp models support both local and remote GGUF paths on the HF Hub.```python-from txtai.pipeline import LLM+from txtai import LLM-# Set method as litellm-llm = LLM("vllm/Open-Orca/Mistral-7B-OpenOrca", method="litellm")+# Transformers+llm = LLM("google/gemma-2-9b")+llm = LLM("google/gemma-2-9b", method="transformers")-# Set method as llama.cpp-llm = LLM("TheBloke/Mistral-7B-OpenOrca-GGUF/mistral-7b-openorca.Q4_K_M.gguf",+# llama.cpp+llm = LLM("microsoft/Phi-3-mini-4k-instruct-gguf/microsoft/Phi-3-mini-4k-instruct-gguf")+llm = LLM("microsoft/Phi-3-mini-4k-instruct-gguf/microsoft/Phi-3-mini-4k-instruct-gguf",method="llama.cpp")++# LiteLLM+llm = LLM("ollama/llama3")+llm = LLM("ollama/llama3", method="litellm")++# LLM APIs - must also set API key via environment variable+llm = LLM("gpt-4o")+llm = LLM("claude-3-5-sonnet-20240620")```Models can be externally loaded and passed to pipelines. This is useful for models that are not yet supported by Transformers and/or need special initialization.@@ -49,7 +68,7 @@ Models can be externally loaded and passed to pipelines. This is useful for modeimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer-from txtai.pipeline import LLM+from txtai import LLM# Load Mistral-7B-OpenOrcapath = "Open-Orca/Mistral-7B-OpenOrca"@@ -84,7 +103,6 @@ Pipelines are run with Python or configuration. Pipelines can be instantiated in### config.yml```yaml# Create pipeline using lower case class name-# Use `generator` or `sequences` to force model typellm:# Run pipeline with workflow@@ -131,7 +149,7 @@ CONFIG=config.yml uvicorn "txtai.api:app" &curl \-X POST "http://localhost:8000/workflow" \-H "Content-Type: application/json" \- -d '{"name":"sequences", "elements": ["Answer the following question..."]}'+ -d '{"name":"llm", "elements": ["Answer the following question..."]}'```## Methodsdiff --git a/docs/pipeline/text/rag.md b/docs/pipeline/text/rag.mdindex 5cded887..a645202c 100644--- a/docs/pipeline/text/rag.md+++ b/docs/pipeline/text/rag.md@@ -12,19 +12,8 @@ The data store can be an embeddings database or a similarity instance with assocThe following shows a simple example using this pipeline.```python-from txtai.embeddings import Embeddings-from txtai.pipeline import RAG--# LLM prompt-def prompt(question):- return f"""- Answer the following question using the provided context.-- Question:- {question}-- Context:- """+from txtai import Embeddings+from txtai import RAG# Input datadata = [@@ -40,11 +29,20 @@ data = [# Build embeddings indexembeddings = Embeddings({"content": True})-embeddings.index([(uid, text, None) for uid, text in enumerate(data)])+embeddings.index(data)# Create and run pipeline-extractor = RAG(embeddings, "google/flan-t5-base")-extractor([{"query": "What was won?", "question": prompt("What was won?")}])+rag = RAG(embeddings, "google/flan-t5-base", template="""+ Answer the following question using the provided context.++ Question:+ {question}++ Context:+ {context}+""")++rag("What was won?")```See the links below for more detailed examples.@@ -78,21 +76,21 @@ writable: Trueembeddings:content: True-extractor:+rag:path: google/flan-t5-base+ template: |+ Answer the following question using the provided context.++ Question:+ {question}++ Context:+ {context}workflow:search:tasks:- - task: extractor- template: |- Answer the following question using the provided context.-- Question:- {text}-- Context:- action: extractor+ - action: rag```### Run with Workflows