💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
Python agentsaiai-agentsembeddingsinformation-retrievallanguage-modellarge-language-modelsllmnlppythonragretrieval-augmented-generationsearchsearch-enginesemantic-searchsentence-embeddingstransformerstxtaivector-databasevector-search
Update documentation
davidmezzetti37dee5451bbb
diff --git a/docs/embeddings/configuration/index.md b/docs/embeddings/configuration/index.mdindex 9e373f19..f55c1605 100644--- a/docs/embeddings/configuration/index.md+++ b/docs/embeddings/configuration/index.md@@ -106,11 +106,12 @@ Instruction-based models use prefixes to modify how embeddings are computed. Thi## backend```yaml-backend: faiss|hnsw|annoy+backend: faiss|hnsw|annoy|custom```Approximate Nearest Neighbor (ANN) index backend for storing generated sentence embeddings. `Defaults to faiss`. Additional backends require the-[similarity](../../install/#similarity) extras package to be installed.+[similarity](../../install/#similarity) extras package to be installed. Add custom backends via setting this parameter to the fully resolvable+class string.Backend-specific settings are set with a corresponding configuration object having the same name as the backend (i.e. annoy, faiss, or hnsw). None of these are required and are set to defaults if omitted.@@ -159,10 +160,10 @@ See [Annoy documentation](https://github.com/spotify/annoy#full-python-api) for## content```yaml-content: string|boolean+content: boolean|sqlite|duckdb|custom```-Enables content storage. When true, the default content storage engine will be used. `Defaults to sqlite`. Otherwise, the string must specify the supported content storage engine to use.+Enables content storage. When true, the default storage engine, `sqlite` will be used. Also supports `duckdb`. Add custom storage engines via setting this parameter to the fully resolvable class string.## functions```yaml@@ -205,6 +206,8 @@ graph:Enables graph storage. When set, a graph network is built using the embeddings index. Graph nodes are synced with each embeddings index operation (index/upsert/delete). Graph edges are created using the embeddings index upon completion of each index/upsert/delete embeddings index call.+Add custom graph storage engines via setting the `graph.backend` parameter to the fully resolvable class string.+Defaults are tuned so that in most cases these values don't need to be changed.### topicsdiff --git a/src/python/txtai/ann/base.py b/src/python/txtai/ann/base.pyindex 18227865..b27e99b2 100644--- a/src/python/txtai/ann/base.py+++ b/src/python/txtai/ann/base.py@@ -10,7 +10,8 @@ from .. import __version__class ANN:"""- Base class for ANN instances.+ Base class for ANN instances. This class builds vector indexes to support similarity search.+ The built-in ANN backends store ids and vectors. Content storage is supported via database instances."""def __init__(self, config):diff --git a/src/python/txtai/database/base.py b/src/python/txtai/database/base.pyindex d619ce8d..2a4674e6 100644--- a/src/python/txtai/database/base.py+++ b/src/python/txtai/database/base.py@@ -14,8 +14,9 @@ logger = logging.getLogger(__name__)class Database:"""- Base class for database instances. This class encapsulates a document-oriented database- used for storing key-value content stored as dicts.+ Base class for database instances. This class encapsulates a content database used for+ storing field content as dicts and objects. The database instance works in conjuction+ with a vector index to execute SQL-driven similarity search."""def __init__(self, config):diff --git a/src/python/txtai/database/factory.py b/src/python/txtai/database/factory.pyindex 7ac92d82..ae680829 100644--- a/src/python/txtai/database/factory.py+++ b/src/python/txtai/database/factory.py@@ -36,10 +36,10 @@ class DatabaseFactory:content = "sqlite"# Create document database instance- if content == "sqlite":- database = SQLite(config)- elif content == "duckdb":+ if content == "duckdb":database = DuckDB(config)+ elif content == "sqlite":+ database = SQLite(config)elif content:database = DatabaseFactory.resolve(content, config)diff --git a/src/python/txtai/graph/base.py b/src/python/txtai/graph/base.pyindex 28a4ef93..d99b39c9 100644--- a/src/python/txtai/graph/base.py+++ b/src/python/txtai/graph/base.py@@ -18,7 +18,8 @@ from .topics import Topics# pylint: disable=R0904class Graph:"""- Base class for Graph instances.+ Base class for Graph instances. This class builds graph networks. Supports topic modeling+ and relationship traversal."""def __init__(self, config):diff --git a/src/python/txtai/vectors/base.py b/src/python/txtai/vectors/base.pyindex dbbf2f5e..0caf8c12 100644--- a/src/python/txtai/vectors/base.py+++ b/src/python/txtai/vectors/base.py@@ -12,7 +12,7 @@ from .. import __pickle__class Vectors:"""- Base class for sentence embeddings/vector models.+ Base class for sentence embeddings/vector models. Vector models transform input content into numeric vectors."""def __init__(self, config, scoring):