Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Add mmap parameter to Faiss, closes #308

davidmezzetti75dd514a14be
diff --git a/docs/embeddings/configuration.md b/docs/embeddings/configuration.md
index 8d3b92c0..40f764e7 100644
--- a/docs/embeddings/configuration.md
+++ b/docs/embeddings/configuration.md
@@ -78,11 +78,13 @@ Backend-specific settings are set with a corresponding configuration object havi
#### faiss
```yaml
faiss:
- components: Comma separated list of components - defaults to "Flat" for small
+ components: comma separated list of components - defaults to "Flat" for small
indices and "IVFx,Flat" for larger indexes where
x = 4 * sqrt(embeddings count)
nprobe: search probe setting (int) - defaults to x/16 (as defined above)
for larger indexes
+ mmap: load as on-disk index (boolean) - defaults to false, trade query response time
+ for a smaller RAM footprint
```
See the following Faiss documentation links for more information.
@@ -135,7 +137,7 @@ List of functions with user-defined SQL functions, only used when [content](#con
### query
```yaml
query:
- path: Sets the path for the query model. This can be any model on the
+ path: sets the path for the query model - this can be any model on the
Hugging Face Model Hub or a local file path.
prefix: text prefix to prepend to all inputs
maxlength: maximum generated sequence length
diff --git a/src/python/txtai/ann/faiss.py b/src/python/txtai/ann/faiss.py
index 1189fa83..6c871326 100644
--- a/src/python/txtai/ann/faiss.py
+++ b/src/python/txtai/ann/faiss.py
@@ -6,7 +6,7 @@ import math
import numpy as np
-from faiss import index_factory, METRIC_INNER_PRODUCT, read_index, write_index
+from faiss import index_factory, IO_FLAG_MMAP, METRIC_INNER_PRODUCT, read_index, write_index
from .base import ANN
@@ -18,7 +18,7 @@ class Faiss(ANN):
def load(self, path):
# Load index
- self.model = read_index(path)
+ self.model = read_index(path, IO_FLAG_MMAP if self.setting("mmap") is True else 0)
def index(self, embeddings):
# Configure embeddings index. Inner product is equal to cosine similarity on normalized vectors.
diff --git a/test/python/testann.py b/test/python/testann.py
index 71f42fd7..53244ee3 100644
--- a/test/python/testann.py
+++ b/test/python/testann.py
@@ -46,6 +46,14 @@ class TestANN(unittest.TestCase):
# Test with custom settings
self.runTests("faiss", {"faiss": {"nprobe": 2, "components": "PCA16,IDMap,SQ8"}}, False)
+ def testFaissMmap(self):
+ """
+ Test Faiss backend with mmap enabled
+ """
+
+ # Test to with mmap enabled
+ self.runTests("faiss", {"faiss": {"mmap": True}}, False)
+
def testHnsw(self):
"""
Test Hnswlib backend