Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Add way to load database connection URL via environment variable, closes #548

davidmezzettidbe0e8fe7f43
diff --git a/docs/embeddings/configuration/database.md b/docs/embeddings/configuration/database.md
index 31820357..b54acf3c 100644
--- a/docs/embeddings/configuration/database.md
+++ b/docs/embeddings/configuration/database.md
@@ -4,10 +4,14 @@ The following covers available content storage configuration options.
## content
```yaml
-content: boolean|sqlite|duckdb|custom
+content: boolean|sqlite|duckdb|client|url|custom
```
-Enables content storage. When true, the default storage engine, `sqlite` will be used to save metadata alongside embeddings vectors. Also supports `duckdb`. Add custom storage engines via setting this parameter to the fully resolvable class string.
+Enables content storage. When true, the default storage engine, `sqlite` will be used to save metadata alongside embeddings vectors.
+
+Client-server connections are supported with either `client` or a full connection URL. When set to `client`, the CLIENT_URL environment variable must be set to the full connection URL.
+
+Add custom storage engines via setting this parameter to the fully resolvable class string.
Content storage specific settings are set with a corresponding configuration object having the same name as the content storage engine (i.e. duckdb or sqlite). None of these are required and are set to defaults if omitted.
diff --git a/docs/embeddings/configuration/scoring.md b/docs/embeddings/configuration/scoring.md
index 50bf5c7a..67098fec 100644
--- a/docs/embeddings/configuration/scoring.md
+++ b/docs/embeddings/configuration/scoring.md
@@ -10,10 +10,10 @@ The following covers the available options
## method
```yaml
-method: bm25|tfidf|sif
+method: bm25|tfidf|sif|custom
```
-Sets the scoring method.
+Sets the scoring method. Add custom scoring via setting this parameter to the fully resolvable class string.
## terms
```yaml
diff --git a/src/python/txtai/database/client.py b/src/python/txtai/database/client.py
index 771f8aa4..e4ed8b46 100644
--- a/src/python/txtai/database/client.py
+++ b/src/python/txtai/database/client.py
@@ -2,6 +2,7 @@
Client module
"""
+import os
import time
# Conditional import
@@ -103,8 +104,14 @@ class Client(RDBMS):
self.connection.execute(insert(Score), [{"indexid": i, "score": sum(s) / len(s)} for i, s in scores.items()])
def connect(self, path=None):
+ # Connection URL
+ content = self.config["content"]
+
+ # Read ENV variable, if necessary
+ content = os.environ.get("CLIENT_URL") if content == "client" else content
+
# Create engine using database URL
- engine = create_engine(self.config["content"], poolclass=StaticPool, echo=False)
+ engine = create_engine(content, poolclass=StaticPool, echo=False)
# Create database session
return Session(engine)
diff --git a/src/python/txtai/database/factory.py b/src/python/txtai/database/factory.py
index 0b9732f0..d14b4edf 100644
--- a/src/python/txtai/database/factory.py
+++ b/src/python/txtai/database/factory.py
@@ -46,7 +46,7 @@ class DatabaseFactory:
elif content:
# Check if content is a URL
url = urlparse(content)
- if url.scheme:
+ if content == "client" or url.scheme:
# Connect to database server URL
database = Client(config)
else: