💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
Python agentsaiai-agentsembeddingsinformation-retrievallanguage-modellarge-language-modelsllmnlppythonragretrieval-augmented-generationsearchsearch-enginesemantic-searchsentence-embeddingstransformerstxtaivector-databasevector-search
Add way to load database connection URL via environment variable, closes #548
davidmezzettidbe0e8fe7f43
diff --git a/docs/embeddings/configuration/database.md b/docs/embeddings/configuration/database.mdindex 31820357..b54acf3c 100644--- a/docs/embeddings/configuration/database.md+++ b/docs/embeddings/configuration/database.md@@ -4,10 +4,14 @@ The following covers available content storage configuration options.## content```yaml-content: boolean|sqlite|duckdb|custom+content: boolean|sqlite|duckdb|client|url|custom```-Enables content storage. When true, the default storage engine, `sqlite` will be used to save metadata alongside embeddings vectors. Also supports `duckdb`. Add custom storage engines via setting this parameter to the fully resolvable class string.+Enables content storage. When true, the default storage engine, `sqlite` will be used to save metadata alongside embeddings vectors.++Client-server connections are supported with either `client` or a full connection URL. When set to `client`, the CLIENT_URL environment variable must be set to the full connection URL.++Add custom storage engines via setting this parameter to the fully resolvable class string.Content storage specific settings are set with a corresponding configuration object having the same name as the content storage engine (i.e. duckdb or sqlite). None of these are required and are set to defaults if omitted.diff --git a/docs/embeddings/configuration/scoring.md b/docs/embeddings/configuration/scoring.mdindex 50bf5c7a..67098fec 100644--- a/docs/embeddings/configuration/scoring.md+++ b/docs/embeddings/configuration/scoring.md@@ -10,10 +10,10 @@ The following covers the available options## method```yaml-method: bm25|tfidf|sif+method: bm25|tfidf|sif|custom```-Sets the scoring method.+Sets the scoring method. Add custom scoring via setting this parameter to the fully resolvable class string.## terms```yamldiff --git a/src/python/txtai/database/client.py b/src/python/txtai/database/client.pyindex 771f8aa4..e4ed8b46 100644--- a/src/python/txtai/database/client.py+++ b/src/python/txtai/database/client.py@@ -2,6 +2,7 @@Client module"""+import osimport time# Conditional import@@ -103,8 +104,14 @@ class Client(RDBMS):self.connection.execute(insert(Score), [{"indexid": i, "score": sum(s) / len(s)} for i, s in scores.items()])def connect(self, path=None):+ # Connection URL+ content = self.config["content"]++ # Read ENV variable, if necessary+ content = os.environ.get("CLIENT_URL") if content == "client" else content+# Create engine using database URL- engine = create_engine(self.config["content"], poolclass=StaticPool, echo=False)+ engine = create_engine(content, poolclass=StaticPool, echo=False)# Create database sessionreturn Session(engine)diff --git a/src/python/txtai/database/factory.py b/src/python/txtai/database/factory.pyindex 0b9732f0..d14b4edf 100644--- a/src/python/txtai/database/factory.py+++ b/src/python/txtai/database/factory.py@@ -46,7 +46,7 @@ class DatabaseFactory:elif content:# Check if content is a URLurl = urlparse(content)- if url.scheme:+ if content == "client" or url.scheme:# Connect to database server URLdatabase = Client(config)else: