[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.
Python aifaissgpt-osslangchainllama-indexllmlocalstorageoffline-firstollamaprivacypythonragretrieval-augmented-generationvector-databasevector-searchvectors
feat(core,diskann): robust embedding server (no-hang) + DiskANN fast mode (graph partition) (#29)
Andy Leefafdf8fcbef1
diff --git a/.github/workflows/build-and-publish.yml b/.github/workflows/build-and-publish.ymlindex bf076d3..9b60b2b 100644--- a/.github/workflows/build-and-publish.yml+++ b/.github/workflows/build-and-publish.yml@@ -5,6 +5,7 @@ on:branches: [ main ]pull_request:branches: [ main ]+ workflow_dispatch:jobs:build:diff --git a/.github/workflows/build-reusable.yml b/.github/workflows/build-reusable.ymlindex b95c281..b2323a3 100644--- a/.github/workflows/build-reusable.yml+++ b/.github/workflows/build-reusable.yml@@ -277,19 +277,16 @@ jobs:- name: Run tests with pytestenv:- CI: true # Mark as CI environment to skip memory-intensive tests+ CI: trueOPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}HF_HUB_DISABLE_SYMLINKS: 1TOKENIZERS_PARALLELISM: false- PYTORCH_ENABLE_MPS_FALLBACK: 0 # Disable MPS on macOS CI to avoid memory issues- OMP_NUM_THREADS: 1 # Disable OpenMP parallelism to avoid libomp crashes- MKL_NUM_THREADS: 1 # Single thread for MKL operations+ PYTORCH_ENABLE_MPS_FALLBACK: 0+ OMP_NUM_THREADS: 1+ MKL_NUM_THREADS: 1run: |- # Activate virtual environmentsource .venv/bin/activate || source .venv/Scripts/activate-- # Run tests- pytest -v tests/+ pytest tests/ -v --tb=short- name: Run sanity checks (optional)run: |diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yamlindex ad8fd0f..0b685cd 100644--- a/.pre-commit-config.yaml+++ b/.pre-commit-config.yaml@@ -1,6 +1,6 @@repos:- repo: https://github.com/pre-commit/pre-commit-hooks- rev: v4.5.0+ rev: v5.0.0hooks:- id: trailing-whitespace- id: end-of-file-fixer@@ -10,7 +10,7 @@ repos:- id: debug-statements- repo: https://github.com/astral-sh/ruff-pre-commit- rev: v0.2.1+ rev: v0.12.7 # Fixed version to match pyproject.tomlhooks:- id: ruff- id: ruff-formatdiff --git a/README.md b/README.mdindex 8eb4daa..702891a 100755--- a/README.md+++ b/README.md@@ -455,7 +455,7 @@ leann --help**To make it globally available:**```bash# Install the LEANN CLI globally using uv tool-uv tool install leann+uv tool install leann-core# Now you can use leann from anywhere without activating venvleann --help@@ -543,12 +543,16 @@ Options:- **Dynamic batching:** Efficiently batch embedding computations for GPU utilization- **Two-level search:** Smart graph traversal that prioritizes promising nodes-**Backends:** HNSW (default) for most use cases, with optional DiskANN support for billion-scale datasets.+**Backends:**+- **HNSW** (default): Ideal for most datasets with maximum storage savings through full recomputation+- **DiskANN**: Advanced option with superior search performance, using PQ-based graph traversal with real-time reranking for the best speed-accuracy trade-off## Benchmarks+**[DiskANN vs HNSW Performance Comparison →](benchmarks/diskann_vs_hnsw_speed_comparison.py)** - Compare search performance between both backends++**[Simple Example: Compare LEANN vs FAISS →](benchmarks/compare_faiss_vs_leann.py)** - See storage savings in action-**[Simple Example: Compare LEANN vs FAISS →](benchmarks/compare_faiss_vs_leann.py)**### 📊 Storage Comparison| System | DPR (2.1M) | Wiki (60M) | Chat (400K) | Email (780K) | Browser (38K) |diff --git a/apps/base_rag_example.py b/apps/base_rag_example.pyindex df88290..9fbb7d2 100644--- a/apps/base_rag_example.py+++ b/apps/base_rag_example.py@@ -178,6 +178,9 @@ class BaseRAGExample(ABC):config["host"] = args.llm_hostelif args.llm == "hf":config["model"] = args.llm_model or "Qwen/Qwen2.5-1.5B-Instruct"+ elif args.llm == "simulated":+ # Simulated LLM doesn't need additional configuration+ passreturn configdiff --git a/benchmarks/README.md b/benchmarks/README.mdindex 7456ce9..258f260 100644--- a/benchmarks/README.md+++ b/benchmarks/README.md@@ -1,9 +1,24 @@-# 🧪 Leann Sanity Checks+# 🧪 LEANN Benchmarks & Testing-This directory contains comprehensive sanity checks for the Leann system, ensuring all components work correctly across different configurations.+This directory contains performance benchmarks and comprehensive tests for the LEANN system, including backend comparisons and sanity checks across different configurations.## 📁 Test Files+### `diskann_vs_hnsw_speed_comparison.py`+Performance comparison between DiskANN and HNSW backends:+- ✅ **Search latency** comparison with both backends using recompute+- ✅ **Index size** and **build time** measurements+- ✅ **Score validity** testing (ensures no -inf scores)+- ✅ **Configurable dataset sizes** for different scales++```bash+# Quick comparison with 500 docs, 10 queries+python benchmarks/diskann_vs_hnsw_speed_comparison.py++# Large-scale comparison with 2000 docs, 20 queries+python benchmarks/diskann_vs_hnsw_speed_comparison.py 2000 20+```+### `test_distance_functions.py`Tests all supported distance functions across DiskANN backend:- ✅ **MIPS** (Maximum Inner Product Search)diff --git a/benchmarks/diskann_vs_hnsw_speed_comparison.py b/benchmarks/diskann_vs_hnsw_speed_comparison.pynew file mode 100644index 0000000..bb89692--- /dev/null+++ b/benchmarks/diskann_vs_hnsw_speed_comparison.py@@ -0,0 +1,268 @@+#!/usr/bin/env python3+"""+DiskANN vs HNSW Search Performance Comparison++This benchmark compares search performance between DiskANN and HNSW backends:+- DiskANN: With graph partitioning enabled (is_recompute=True)+- HNSW: With recompute enabled (is_recompute=True)+- Tests performance across different dataset sizes+- Measures search latency, recall, and index size+"""++import gc+import tempfile+import time+from pathlib import Path+from typing import Any++import numpy as np+++def create_test_texts(n_docs: int) -> list[str]:+ """Create synthetic test documents for benchmarking."""+ np.random.seed(42)+ topics = [+ "machine learning and artificial intelligence",+ "natural language processing and text analysis",+ "computer vision and image recognition",+ "data science and statistical analysis",+ "deep learning and neural networks",+ "information retrieval and search engines",+ "database systems and data management",+ "software engineering and programming",+ "cybersecurity and network protection",+ "cloud computing and distributed systems",+ ]++ texts = []+ for i in range(n_docs):+ topic = topics[i % len(topics)]+ variation = np.random.randint(1, 100)+ text = (+ f"This is document {i} about {topic}. Content variation {variation}. "+ f"Additional information about {topic} with details and examples. "+ f"Technical discussion of {topic} including implementation aspects."+ )+ texts.append(text)++ return texts+++def benchmark_backend(+ backend_name: str, texts: list[str], test_queries: list[str], backend_kwargs: dict[str, Any]+) -> dict[str, float]:+ """Benchmark a specific backend with the given configuration."""+ from leann.api import LeannBuilder, LeannSearcher++ print(f"\n🔧 Testing {backend_name.upper()} backend...")++ with tempfile.TemporaryDirectory() as temp_dir:+ index_path = str(Path(temp_dir) / f"benchmark_{backend_name}.leann")++ # Build index+ print(f"📦 Building {backend_name} index with {len(texts)} documents...")+ start_time = time.time()++ builder = LeannBuilder(+ backend_name=backend_name,+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ **backend_kwargs,+ )++ for text in texts:+ builder.add_text(text)++ builder.build_index(index_path)+ build_time = time.time() - start_time++ # Measure index size+ index_dir = Path(index_path).parent+ index_files = list(index_dir.glob(f"{Path(index_path).stem}.*"))+ total_size = sum(f.stat().st_size for f in index_files if f.is_file())+ size_mb = total_size / (1024 * 1024)++ print(f" ✅ Build completed in {build_time:.2f}s, index size: {size_mb:.1f}MB")++ # Search benchmark+ print("🔍 Running search benchmark...")+ searcher = LeannSearcher(index_path)++ search_times = []+ all_results = []++ for query in test_queries:+ start_time = time.time()+ results = searcher.search(query, top_k=5)+ search_time = time.time() - start_time+ search_times.append(search_time)+ all_results.append(results)++ avg_search_time = np.mean(search_times) * 1000 # Convert to ms+ print(f" ✅ Average search time: {avg_search_time:.1f}ms")++ # Check for valid scores (detect -inf issues)+ all_scores = [+ result.score+ for results in all_results+ for result in results+ if result.score is not None+ ]+ valid_scores = [+ score for score in all_scores if score != float("-inf") and score != float("inf")+ ]+ score_validity_rate = len(valid_scores) / len(all_scores) if all_scores else 0++ # Clean up+ try:+ if hasattr(searcher, "__del__"):+ searcher.__del__()+ del searcher+ del builder+ gc.collect()+ except Exception as e:+ print(f"⚠️ Warning: Resource cleanup error: {e}")++ return {+ "build_time": build_time,+ "avg_search_time_ms": avg_search_time,+ "index_size_mb": size_mb,+ "score_validity_rate": score_validity_rate,+ }+++def run_comparison(n_docs: int = 500, n_queries: int = 10):+ """Run performance comparison between DiskANN and HNSW."""+ print("🚀 Starting DiskANN vs HNSW Performance Comparison")+ print(f"📊 Dataset: {n_docs} documents, {n_queries} test queries")++ # Create test data+ texts = create_test_texts(n_docs)+ test_queries = [+ "machine learning algorithms",+ "natural language processing",+ "computer vision techniques",+ "data analysis methods",+ "neural network architectures",+ "database query optimization",+ "software development practices",+ "security vulnerabilities",+ "cloud infrastructure",+ "distributed computing",+ ][:n_queries]++ # HNSW benchmark+ hnsw_results = benchmark_backend(+ backend_name="hnsw",+ texts=texts,+ test_queries=test_queries,+ backend_kwargs={+ "is_recompute": True, # Enable recompute for fair comparison+ "M": 16,+ "efConstruction": 200,+ },+ )++ # DiskANN benchmark+ diskann_results = benchmark_backend(+ backend_name="diskann",+ texts=texts,+ test_queries=test_queries,+ backend_kwargs={+ "is_recompute": True, # Enable graph partitioning+ "num_neighbors": 32,+ "search_list_size": 50,+ },+ )++ # Performance comparison+ print("\n📈 Performance Comparison Results")+ print(f"{'=' * 60}")+ print(f"{'Metric':<25} {'HNSW':<15} {'DiskANN':<15} {'Speedup':<10}")+ print(f"{'-' * 60}")++ # Build time comparison+ build_speedup = hnsw_results["build_time"] / diskann_results["build_time"]+ print(+ f"{'Build Time (s)':<25} {hnsw_results['build_time']:<15.2f} {diskann_results['build_time']:<15.2f} {build_speedup:<10.2f}x"+ )++ # Search time comparison+ search_speedup = hnsw_results["avg_search_time_ms"] / diskann_results["avg_search_time_ms"]+ print(+ f"{'Search Time (ms)':<25} {hnsw_results['avg_search_time_ms']:<15.1f} {diskann_results['avg_search_time_ms']:<15.1f} {search_speedup:<10.2f}x"+ )++ # Index size comparison+ size_ratio = diskann_results["index_size_mb"] / hnsw_results["index_size_mb"]+ print(+ f"{'Index Size (MB)':<25} {hnsw_results['index_size_mb']:<15.1f} {diskann_results['index_size_mb']:<15.1f} {size_ratio:<10.2f}x"+ )++ # Score validity+ print(+ f"{'Score Validity (%)':<25} {hnsw_results['score_validity_rate'] * 100:<15.1f} {diskann_results['score_validity_rate'] * 100:<15.1f}"+ )++ print(f"{'=' * 60}")+ print("\n🎯 Summary:")+ if search_speedup > 1:+ print(f" DiskANN is {search_speedup:.2f}x faster than HNSW for search")+ else:+ print(f" HNSW is {1 / search_speedup:.2f}x faster than DiskANN for search")++ if size_ratio > 1:+ print(f" DiskANN uses {size_ratio:.2f}x more storage than HNSW")+ else:+ print(f" DiskANN uses {1 / size_ratio:.2f}x less storage than HNSW")++ print(+ f" Both backends achieved {min(hnsw_results['score_validity_rate'], diskann_results['score_validity_rate']) * 100:.1f}% score validity"+ )+++if __name__ == "__main__":+ import sys++ try:+ # Handle help request+ if len(sys.argv) > 1 and sys.argv[1] in ["-h", "--help", "help"]:+ print("DiskANN vs HNSW Performance Comparison")+ print("=" * 50)+ print(f"Usage: python {sys.argv[0]} [n_docs] [n_queries]")+ print()+ print("Arguments:")+ print(" n_docs Number of documents to index (default: 500)")+ print(" n_queries Number of test queries to run (default: 10)")+ print()+ print("Examples:")+ print(" python benchmarks/diskann_vs_hnsw_speed_comparison.py")+ print(" python benchmarks/diskann_vs_hnsw_speed_comparison.py 1000")+ print(" python benchmarks/diskann_vs_hnsw_speed_comparison.py 2000 20")+ sys.exit(0)++ # Parse command line arguments+ n_docs = int(sys.argv[1]) if len(sys.argv) > 1 else 500+ n_queries = int(sys.argv[2]) if len(sys.argv) > 2 else 10++ print("DiskANN vs HNSW Performance Comparison")+ print("=" * 50)+ print(f"Dataset: {n_docs} documents, {n_queries} queries")+ print()++ run_comparison(n_docs=n_docs, n_queries=n_queries)++ except KeyboardInterrupt:+ print("\n⚠️ Benchmark interrupted by user")+ sys.exit(130)+ except Exception as e:+ print(f"\n❌ Benchmark failed: {e}")+ sys.exit(1)+ finally:+ # Ensure clean exit+ try:+ gc.collect()+ print("\n🧹 Cleanup completed")+ except Exception:+ pass+ sys.exit(0)diff --git a/docs/configuration-guide.md b/docs/configuration-guide.mdindex 49cb516..badfc58 100644--- a/docs/configuration-guide.md+++ b/docs/configuration-guide.md@@ -97,16 +97,30 @@ ollama pull nomic-embed-text```### DiskANN-**Best for**: Large datasets (> 10M vectors, 10GB+ index size) - **⚠️ Beta version, still in active development**-- Uses Product Quantization (PQ) for coarse filtering during graph traversal-- Novel approach: stores only PQ codes, performs rerank with exact computation in final step-- Implements a corner case of double-queue: prunes all neighbors and recomputes at the end+**Best for**: Performance-critical applications and large datasets - **Production-ready with automatic graph partitioning**++**How it works:**+- **Product Quantization (PQ) + Real-time Reranking**: Uses compressed PQ codes for fast graph traversal, then recomputes exact embeddings for final candidates+- **Automatic Graph Partitioning**: When `is_recompute=True`, automatically partitions large indices and safely removes redundant files to save storage+- **Superior Speed-Accuracy Trade-off**: Faster search than HNSW while maintaining high accuracy++**Trade-offs compared to HNSW:**+- ✅ **Faster search latency** (typically 2-8x speedup)+- ✅ **Better scaling** for large datasets+- ✅ **Smart storage management** with automatic partitioning+- ✅ **Better graph locality** with `--ldg-times` parameter for SSD optimization+- ⚠️ **Slightly larger index size** due to PQ tables and graph metadata```bash-# For billion-scale deployments+# Recommended for most use cases+--backend-name diskann --graph-degree 32 --build-complexity 64++# For large-scale deployments--backend-name diskann --graph-degree 64 --build-complexity 128```+**Performance Benchmark**: Run `python benchmarks/diskann_vs_hnsw_speed_comparison.py` to compare DiskANN and HNSW on your system.+## LLM Selection: Engine and Model Comparison### LLM Engines@@ -283,3 +297,4 @@ LEANN's recomputation feature provides exact distance calculations but can be di- [Lessons Learned Developing LEANN](https://yichuan-w.github.io/blog/lessons_learned_in_dev_leann/)- [LEANN Technical Paper](https://arxiv.org/abs/2506.08276)- [DiskANN Original Paper](https://papers.nips.cc/paper/2019/file/09853c7fb1d3f8ee67a61b6bf4a7f8e6-Paper.pdf)+- [SSD-based Graph Partitioning](https://github.com/SonglinLife/SSD_BASED_PLAN)diff --git a/packages/leann-backend-diskann/leann_backend_diskann/__init__.py b/packages/leann-backend-diskann/leann_backend_diskann/__init__.pyindex cba4f62..f716d18 100644--- a/packages/leann-backend-diskann/leann_backend_diskann/__init__.py+++ b/packages/leann-backend-diskann/leann_backend_diskann/__init__.py@@ -1 +1,7 @@from . import diskann_backend as diskann_backend+from . import graph_partition++# Export main classes and functions+from .graph_partition import GraphPartitioner, partition_graph++__all__ = ["GraphPartitioner", "diskann_backend", "graph_partition", "partition_graph"]diff --git a/packages/leann-backend-diskann/leann_backend_diskann/diskann_backend.py b/packages/leann-backend-diskann/leann_backend_diskann/diskann_backend.pyindex ba5c0ad..6ef84cc 100644--- a/packages/leann-backend-diskann/leann_backend_diskann/diskann_backend.py+++ b/packages/leann-backend-diskann/leann_backend_diskann/diskann_backend.py@@ -22,6 +22,11 @@ logger = logging.getLogger(__name__)@contextlib.contextmanagerdef suppress_cpp_output_if_needed():"""Suppress C++ stdout/stderr based on LEANN_LOG_LEVEL"""+ # In CI we avoid fiddling with low-level file descriptors to prevent aborts+ if os.getenv("CI") == "true":+ yield+ return+log_level = os.getenv("LEANN_LOG_LEVEL", "WARNING").upper()# Only suppress if log level is WARNING or higher (ERROR, CRITICAL)@@ -137,6 +142,71 @@ class DiskannBuilder(LeannBackendBuilderInterface):def __init__(self, **kwargs):self.build_params = kwargs+ def _safe_cleanup_after_partition(self, index_dir: Path, index_prefix: str):+ """+ Safely cleanup files after partition.+ In partition mode, C++ doesn't read _disk.index content,+ so we can delete it if all derived files exist.+ """+ disk_index_file = index_dir / f"{index_prefix}_disk.index"+ beam_search_file = index_dir / f"{index_prefix}_disk_beam_search.index"++ # Required files that C++ partition mode needs+ # Note: C++ generates these with _disk.index suffix+ disk_suffix = "_disk.index"+ required_files = [+ f"{index_prefix}{disk_suffix}_medoids.bin", # Critical: assert fails if missing+ # Note: _centroids.bin is not created in single-shot build - C++ handles this automatically+ f"{index_prefix}_pq_pivots.bin", # PQ table+ f"{index_prefix}_pq_compressed.bin", # PQ compressed vectors+ ]++ # Check if all required files exist+ missing_files = []+ for filename in required_files:+ file_path = index_dir / filename+ if not file_path.exists():+ missing_files.append(filename)++ if missing_files:+ logger.warning(+ f"Cannot safely delete _disk.index - missing required files: {missing_files}"+ )+ logger.info("Keeping all original files for safety")+ return++ # Calculate space savings+ space_saved = 0+ files_to_delete = []++ if disk_index_file.exists():+ space_saved += disk_index_file.stat().st_size+ files_to_delete.append(disk_index_file)++ if beam_search_file.exists():+ space_saved += beam_search_file.stat().st_size+ files_to_delete.append(beam_search_file)++ # Safe to delete!+ for file_to_delete in files_to_delete:+ try:+ os.remove(file_to_delete)+ logger.info(f"✅ Safely deleted: {file_to_delete.name}")+ except Exception as e:+ logger.warning(f"Failed to delete {file_to_delete.name}: {e}")++ if space_saved > 0:+ space_saved_mb = space_saved / (1024 * 1024)+ logger.info(f"💾 Space saved: {space_saved_mb:.1f} MB")++ # Show what files are kept+ logger.info("📁 Kept essential files for partition mode:")+ for filename in required_files:+ file_path = index_dir / filename+ if file_path.exists():+ size_mb = file_path.stat().st_size / (1024 * 1024)+ logger.info(f" - {filename} ({size_mb:.1f} MB)")+def build(self, data: np.ndarray, ids: list[str], index_path: str, **kwargs):path = Path(index_path)index_dir = path.parent@@ -151,6 +221,17 @@ class DiskannBuilder(LeannBackendBuilderInterface):_write_vectors_to_bin(data, index_dir / data_filename)build_kwargs = {**self.build_params, **kwargs}++ # Extract is_recompute from nested backend_kwargs if needed+ is_recompute = build_kwargs.get("is_recompute", False)+ if not is_recompute and "backend_kwargs" in build_kwargs:+ is_recompute = build_kwargs["backend_kwargs"].get("is_recompute", False)++ # Flatten all backend_kwargs parameters to top level for compatibility+ if "backend_kwargs" in build_kwargs:+ nested_params = build_kwargs.pop("backend_kwargs")+ build_kwargs.update(nested_params)+metric_enum = _get_diskann_metrics().get(build_kwargs.get("distance_metric", "mips").lower())@@ -185,6 +266,30 @@ class DiskannBuilder(LeannBackendBuilderInterface):build_kwargs.get("pq_disk_bytes", 0),"",)++ # Auto-partition if is_recompute is enabled+ if build_kwargs.get("is_recompute", False):+ logger.info("is_recompute=True, starting automatic graph partitioning...")+ from .graph_partition import partition_graph++ # Partition the index using absolute paths+ # Convert to absolute paths to avoid issues with working directory changes+ absolute_index_dir = Path(index_dir).resolve()+ absolute_index_prefix_path = str(absolute_index_dir / index_prefix)+ disk_graph_path, partition_bin_path = partition_graph(+ index_prefix_path=absolute_index_prefix_path,+ output_dir=str(absolute_index_dir),+ partition_prefix=index_prefix,+ )++ # Safe cleanup: In partition mode, C++ doesn't read _disk.index content+ # but still needs the derived files (_medoids.bin, _centroids.bin, etc.)+ self._safe_cleanup_after_partition(index_dir, index_prefix)++ logger.info("✅ Graph partitioning completed successfully!")+ logger.info(f" - Disk graph: {disk_graph_path}")+ logger.info(f" - Partition file: {partition_bin_path}")+finally:temp_data_file = index_dir / data_filenameif temp_data_file.exists():@@ -213,7 +318,26 @@ class DiskannSearcher(BaseSearcher):# For DiskANN, we need to reinitialize the index when zmq_port changes# Store the initialization parameters for later use- full_index_prefix = str(self.index_dir / self.index_path.stem)+ # Note: C++ load method expects the BASE path (without _disk.index suffix)+ # C++ internally constructs: index_prefix + "_disk.index"+ index_name = self.index_path.stem # "simple_test.leann" -> "simple_test"+ diskann_index_prefix = str(self.index_dir / index_name) # /path/to/simple_test+ full_index_prefix = diskann_index_prefix # /path/to/simple_test (base path)++ # Auto-detect partition files and set partition_prefix+ partition_graph_file = self.index_dir / f"{index_name}_disk_graph.index"+ partition_bin_file = self.index_dir / f"{index_name}_partition.bin"++ partition_prefix = ""+ if partition_graph_file.exists() and partition_bin_file.exists():+ # C++ expects full path prefix, not just filename+ partition_prefix = str(self.index_dir / index_name) # /path/to/simple_test+ logger.info(+ f"✅ Detected partition files, using partition_prefix='{partition_prefix}'"+ )+ else:+ logger.debug("No partition files detected, using standard index files")+self._init_params = {"metric_enum": metric_enum,"full_index_prefix": full_index_prefix,@@ -221,8 +345,14 @@ class DiskannSearcher(BaseSearcher):"num_nodes_to_cache": kwargs.get("num_nodes_to_cache", 0),"cache_mechanism": 1,"pq_prefix": "",- "partition_prefix": "",+ "partition_prefix": partition_prefix,}++ # Log partition configuration for debugging+ if partition_prefix:+ logger.info(+ f"✅ Detected partition files, using partition_prefix='{partition_prefix}'"+ )self._diskannpy = diskannpyself._current_zmq_port = Noneself._index = Nonediff --git a/packages/leann-backend-diskann/leann_backend_diskann/diskann_embedding_server.py b/packages/leann-backend-diskann/leann_backend_diskann/diskann_embedding_server.pyindex 749d607..2b1e326 100644--- a/packages/leann-backend-diskann/leann_backend_diskann/diskann_embedding_server.py+++ b/packages/leann-backend-diskann/leann_backend_diskann/diskann_embedding_server.py@@ -81,7 +81,8 @@ def create_diskann_embedding_server(with open(passages_file) as f:meta = json.load(f)- passages = PassageManager(meta["passage_sources"])+ logger.info(f"Loading PassageManager with metadata_file_path: {passages_file}")+ passages = PassageManager(meta["passage_sources"], metadata_file_path=passages_file)logger.info(f"Loaded PassageManager with {len(passages.global_offset_map)} passages from metadata")@@ -102,8 +103,9 @@ def create_diskann_embedding_server(socket.bind(f"tcp://*:{zmq_port}")logger.info(f"DiskANN ZMQ REP server listening on port {zmq_port}")- socket.setsockopt(zmq.RCVTIMEO, 300000)- socket.setsockopt(zmq.SNDTIMEO, 300000)+ socket.setsockopt(zmq.RCVTIMEO, 1000)+ socket.setsockopt(zmq.SNDTIMEO, 1000)+ socket.setsockopt(zmq.LINGER, 0)while True:try:@@ -220,30 +222,217 @@ def create_diskann_embedding_server(traceback.print_exc()raise- zmq_thread = threading.Thread(target=zmq_server_thread, daemon=True)+ def zmq_server_thread_with_shutdown(shutdown_event):+ """ZMQ server thread that respects shutdown signal.++ This creates its own REP socket, binds to zmq_port, and periodically+ checks shutdown_event using recv timeouts to exit cleanly.+ """+ logger.info("DiskANN ZMQ server thread started with shutdown support")++ context = zmq.Context()+ rep_socket = context.socket(zmq.REP)+ rep_socket.bind(f"tcp://*:{zmq_port}")+ logger.info(f"DiskANN ZMQ REP server listening on port {zmq_port}")++ # Set receive timeout so we can check shutdown_event periodically+ rep_socket.setsockopt(zmq.RCVTIMEO, 1000) # 1 second timeout+ rep_socket.setsockopt(zmq.SNDTIMEO, 1000)+ rep_socket.setsockopt(zmq.LINGER, 0)++ try:+ while not shutdown_event.is_set():+ try:+ e2e_start = time.time()+ # REP socket receives single-part messages+ message = rep_socket.recv()++ # Check for empty messages - REP socket requires response to every request+ if not message:+ logger.warning("Received empty message, sending empty response")+ rep_socket.send(b"")+ continue++ # Try protobuf first (same logic as original)+ texts = []+ is_text_request = False++ try:+ req_proto = embedding_pb2.NodeEmbeddingRequest()+ req_proto.ParseFromString(message)+ node_ids = list(req_proto.node_ids)++ # Look up texts by node IDs+ for nid in node_ids:+ try:+ passage_data = passages.get_passage(str(nid))+ txt = passage_data["text"]+ if not txt:+ raise RuntimeError(f"FATAL: Empty text for passage ID {nid}")+ texts.append(txt)+ except KeyError:+ raise RuntimeError(f"FATAL: Passage with ID {nid} not found")++ logger.info(f"ZMQ received protobuf request for {len(node_ids)} node IDs")+ except Exception:+ # Fallback to msgpack for text requests+ try:+ import msgpack++ request = msgpack.unpackb(message)+ if isinstance(request, list) and all(+ isinstance(item, str) for item in request+ ):+ texts = request+ is_text_request = True+ logger.info(+ f"ZMQ received msgpack text request for {len(texts)} texts"+ )+ else:+ raise ValueError("Not a valid msgpack text request")+ except Exception:+ logger.error("Both protobuf and msgpack parsing failed!")+ # Send error response+ resp_proto = embedding_pb2.NodeEmbeddingResponse()+ rep_socket.send(resp_proto.SerializeToString())+ continue++ # Process the request+ embeddings = compute_embeddings(texts, model_name, mode=embedding_mode)+ logger.info(f"Computed embeddings shape: {embeddings.shape}")++ # Validation+ if np.isnan(embeddings).any() or np.isinf(embeddings).any():+ logger.error("NaN or Inf detected in embeddings!")+ # Send error response+ if is_text_request:+ import msgpack++ response_data = msgpack.packb([])+ else:+ resp_proto = embedding_pb2.NodeEmbeddingResponse()+ response_data = resp_proto.SerializeToString()+ rep_socket.send(response_data)+ continue++ # Prepare response based on request type+ if is_text_request:+ # For direct text requests, return msgpack+ import msgpack++ response_data = msgpack.packb(embeddings.tolist())+ else:+ # For protobuf requests, return protobuf+ resp_proto = embedding_pb2.NodeEmbeddingResponse()+ hidden_contiguous = np.ascontiguousarray(embeddings, dtype=np.float32)++ resp_proto.embeddings_data = hidden_contiguous.tobytes()+ resp_proto.dimensions.append(hidden_contiguous.shape[0])+ resp_proto.dimensions.append(hidden_contiguous.shape[1])++ response_data = resp_proto.SerializeToString()++ # Send response back to the client+ rep_socket.send(response_data)++ e2e_end = time.time()+ logger.info(f"⏱️ ZMQ E2E time: {e2e_end - e2e_start:.6f}s")++ except zmq.Again:+ # Timeout - check shutdown_event and continue+ continue+ except Exception as e:+ if not shutdown_event.is_set():+ logger.error(f"Error in ZMQ server loop: {e}")+ try:+ # Send error response for REP socket+ resp_proto = embedding_pb2.NodeEmbeddingResponse()+ rep_socket.send(resp_proto.SerializeToString())+ except Exception:+ pass+ else:+ logger.info("Shutdown in progress, ignoring ZMQ error")+ break+ finally:+ try:+ rep_socket.close(0)+ except Exception:+ pass+ try:+ context.term()+ except Exception:+ pass++ logger.info("DiskANN ZMQ server thread exiting gracefully")++ # Add shutdown coordination+ shutdown_event = threading.Event()++ def shutdown_zmq_server():+ """Gracefully shutdown ZMQ server."""+ logger.info("Initiating graceful shutdown...")+ shutdown_event.set()++ if zmq_thread.is_alive():+ logger.info("Waiting for ZMQ thread to finish...")+ zmq_thread.join(timeout=5)+ if zmq_thread.is_alive():+ logger.warning("ZMQ thread did not finish in time")++ # Clean up ZMQ resources+ try:+ # Note: socket and context are cleaned up by thread exit+ logger.info("ZMQ resources cleaned up")+ except Exception as e:+ logger.warning(f"Error cleaning ZMQ resources: {e}")++ # Clean up other resources+ try:+ import gc++ gc.collect()+ logger.info("Additional resources cleaned up")+ except Exception as e:+ logger.warning(f"Error cleaning additional resources: {e}")++ logger.info("Graceful shutdown completed")+ sys.exit(0)++ # Register signal handlers within this function scope+ import signal++ def signal_handler(sig, frame):+ logger.info(f"Received signal {sig}, shutting down gracefully...")+ shutdown_zmq_server()++ signal.signal(signal.SIGTERM, signal_handler)+ signal.signal(signal.SIGINT, signal_handler)++ # Start ZMQ thread (NOT daemon!)+ zmq_thread = threading.Thread(+ target=lambda: zmq_server_thread_with_shutdown(shutdown_event),+ daemon=False, # Not daemon - we want to wait for it+ )zmq_thread.start()logger.info(f"Started DiskANN ZMQ server thread on port {zmq_port}")# Keep the main thread alivetry:- while True:- time.sleep(1)+ while not shutdown_event.is_set():+ time.sleep(0.1) # Check shutdown more frequentlyexcept KeyboardInterrupt:logger.info("DiskANN Server shutting down...")+ shutdown_zmq_server()return+ # If we reach here, shutdown was triggered by signal+ logger.info("Main loop exited, process should be shutting down")+if __name__ == "__main__":- import signalimport sys- def signal_handler(sig, frame):- logger.info(f"Received signal {sig}, shutting down gracefully...")- sys.exit(0)-- # Register signal handlers for graceful shutdown- signal.signal(signal.SIGTERM, signal_handler)- signal.signal(signal.SIGINT, signal_handler)+ # Signal handlers are now registered within create_diskann_embedding_serverparser = argparse.ArgumentParser(description="DiskANN Embedding service")parser.add_argument("--zmq-port", type=int, default=5555, help="ZMQ port to run on")diff --git a/packages/leann-backend-diskann/leann_backend_diskann/graph_partition.py b/packages/leann-backend-diskann/leann_backend_diskann/graph_partition.pynew file mode 100644index 0000000..4c85e28--- /dev/null+++ b/packages/leann-backend-diskann/leann_backend_diskann/graph_partition.py@@ -0,0 +1,299 @@+#!/usr/bin/env python3+"""+Graph Partition Module for LEANN DiskANN Backend++This module provides Python bindings for the graph partition functionality+of DiskANN, allowing users to partition disk-based indices for better+performance.+"""++import os+import shutil+import subprocess+import tempfile+from pathlib import Path+from typing import Optional+++class GraphPartitioner:+ """+ A Python interface for DiskANN's graph partition functionality.++ This class provides methods to partition disk-based indices for improved+ search performance and memory efficiency.+ """++ def __init__(self, build_type: str = "release"):+ """+ Initialize the GraphPartitioner.++ Args:+ build_type: Build type for the executables ("debug" or "release")+ """+ self.build_type = build_type+ self._ensure_executables()++ def _get_executable_path(self, name: str) -> str:+ """Get the path to a graph partition executable."""+ # Get the directory where this Python module is located+ module_dir = Path(__file__).parent+ # Navigate to the graph_partition directory+ graph_partition_dir = module_dir.parent / "third_party" / "DiskANN" / "graph_partition"+ executable_path = graph_partition_dir / "build" / self.build_type / "graph_partition" / name++ if not executable_path.exists():+ raise FileNotFoundError(f"Executable {name} not found at {executable_path}")++ return str(executable_path)++ def _ensure_executables(self):+ """Ensure that the required executables are built."""+ try:+ self._get_executable_path("partitioner")+ self._get_executable_path("index_relayout")+ except FileNotFoundError:+ # Try to build the executables automatically+ print("Executables not found, attempting to build them...")+ self._build_executables()++ def _build_executables(self):+ """Build the required executables."""+ graph_partition_dir = (+ Path(__file__).parent.parent / "third_party" / "DiskANN" / "graph_partition"+ )+ original_dir = os.getcwd()++ try:+ os.chdir(graph_partition_dir)++ # Clean any existing build+ if (graph_partition_dir / "build").exists():+ shutil.rmtree(graph_partition_dir / "build")++ # Run the build script+ cmd = ["./build.sh", self.build_type, "split_graph", "/tmp/dummy"]+ subprocess.run(cmd, capture_output=True, text=True, cwd=graph_partition_dir)++ # Check if executables were created+ partitioner_path = self._get_executable_path("partitioner")+ relayout_path = self._get_executable_path("index_relayout")++ print(f"✅ Built partitioner: {partitioner_path}")+ print(f"✅ Built index_relayout: {relayout_path}")++ except Exception as e:+ raise RuntimeError(f"Failed to build executables: {e}")+ finally:+ os.chdir(original_dir)++ def partition_graph(+ self,+ index_prefix_path: str,+ output_dir: Optional[str] = None,+ partition_prefix: Optional[str] = None,+ **kwargs,+ ) -> tuple[str, str]:+ """+ Partition a disk-based index for improved performance.++ Args:+ index_prefix_path: Path to the index prefix (e.g., "/path/to/index")+ output_dir: Output directory for results (defaults to parent of index_prefix_path)+ partition_prefix: Prefix for output files (defaults to basename of index_prefix_path)+ **kwargs: Additional parameters for graph partitioning:+ - gp_times: Number of LDG partition iterations (default: 10)+ - lock_nums: Number of lock nodes (default: 10)+ - cut: Cut adjacency list degree (default: 100)+ - scale_factor: Scale factor (default: 1)+ - data_type: Data type (default: "float")+ - thread_nums: Number of threads (default: 10)++ Returns:+ Tuple of (disk_graph_index_path, partition_bin_path)++ Raises:+ RuntimeError: If the partitioning process fails+ """+ # Set default parameters+ params = {+ "gp_times": 10,+ "lock_nums": 10,+ "cut": 100,+ "scale_factor": 1,+ "data_type": "float",+ "thread_nums": 10,+ **kwargs,+ }++ # Determine output directory+ if output_dir is None:+ output_dir = str(Path(index_prefix_path).parent)++ # Create output directory if it doesn't exist+ Path(output_dir).mkdir(parents=True, exist_ok=True)++ # Determine partition prefix+ if partition_prefix is None:+ partition_prefix = Path(index_prefix_path).name++ # Get executable paths+ partitioner_path = self._get_executable_path("partitioner")+ relayout_path = self._get_executable_path("index_relayout")++ # Create temporary directory for processing+ with tempfile.TemporaryDirectory() as temp_dir:+ # Change to the graph_partition directory for temporary files+ graph_partition_dir = (+ Path(__file__).parent.parent / "third_party" / "DiskANN" / "graph_partition"+ )+ original_dir = os.getcwd()++ try:+ os.chdir(graph_partition_dir)++ # Create temporary data directory+ temp_data_dir = Path(temp_dir) / "data"+ temp_data_dir.mkdir(parents=True, exist_ok=True)++ # Set up paths for temporary files+ graph_path = temp_data_dir / "starling" / "_M_R_L_B" / "GRAPH"+ graph_gp_path = (+ graph_path+ / f"GP_TIMES_{params['gp_times']}_LOCK_{params['lock_nums']}_GP_USE_FREQ0_CUT{params['cut']}_SCALE{params['scale_factor']}"+ )+ graph_gp_path.mkdir(parents=True, exist_ok=True)++ # Find input index file+ old_index_file = f"{index_prefix_path}_disk_beam_search.index"+ if not os.path.exists(old_index_file):+ old_index_file = f"{index_prefix_path}_disk.index"++ if not os.path.exists(old_index_file):+ raise RuntimeError(f"Index file not found: {old_index_file}")++ # Run partitioner+ gp_file_path = graph_gp_path / "_part.bin"+ partitioner_cmd = [+ partitioner_path,+ "--index_file",+ old_index_file,+ "--data_type",+ params["data_type"],+ "--gp_file",+ str(gp_file_path),+ "-T",+ str(params["thread_nums"]),+ "--ldg_times",+ str(params["gp_times"]),+ "--scale",+ str(params["scale_factor"]),+ "--mode",+ "1",+ ]++ print(f"Running partitioner: {' '.join(partitioner_cmd)}")+ result = subprocess.run(+ partitioner_cmd, capture_output=True, text=True, cwd=graph_partition_dir+ )++ if result.returncode != 0:+ raise RuntimeError(+ f"Partitioner failed with return code {result.returncode}.\n"+ f"stdout: {result.stdout}\n"+ f"stderr: {result.stderr}"+ )++ # Run relayout+ part_tmp_index = graph_gp_path / "_part_tmp.index"+ relayout_cmd = [+ relayout_path,+ old_index_file,+ str(gp_file_path),+ params["data_type"],+ "1",+ ]++ print(f"Running relayout: {' '.join(relayout_cmd)}")+ result = subprocess.run(+ relayout_cmd, capture_output=True, text=True, cwd=graph_partition_dir+ )++ if result.returncode != 0:+ raise RuntimeError(+ f"Relayout failed with return code {result.returncode}.\n"+ f"stdout: {result.stdout}\n"+ f"stderr: {result.stderr}"+ )++ # Copy results to output directory+ disk_graph_path = Path(output_dir) / f"{partition_prefix}_disk_graph.index"+ partition_bin_path = Path(output_dir) / f"{partition_prefix}_partition.bin"++ shutil.copy2(part_tmp_index, disk_graph_path)+ shutil.copy2(gp_file_path, partition_bin_path)++ print(f"Results copied to: {output_dir}")+ return str(disk_graph_path), str(partition_bin_path)++ finally:+ os.chdir(original_dir)++ def get_partition_info(self, partition_bin_path: str) -> dict:+ """+ Get information about a partition file.++ Args:+ partition_bin_path: Path to the partition binary file++ Returns:+ Dictionary containing partition information+ """+ if not os.path.exists(partition_bin_path):+ raise FileNotFoundError(f"Partition file not found: {partition_bin_path}")++ # For now, return basic file information+ # In the future, this could parse the binary file for detailed info+ stat = os.stat(partition_bin_path)+ return {+ "file_size": stat.st_size,+ "file_path": partition_bin_path,+ "modified_time": stat.st_mtime,+ }+++def partition_graph(+ index_prefix_path: str,+ output_dir: Optional[str] = None,+ partition_prefix: Optional[str] = None,+ build_type: str = "release",+ **kwargs,+) -> tuple[str, str]:+ """+ Convenience function to partition a graph index.++ Args:+ index_prefix_path: Path to the index prefix+ output_dir: Output directory (defaults to parent of index_prefix_path)+ partition_prefix: Prefix for output files (defaults to basename of index_prefix_path)+ build_type: Build type for executables ("debug" or "release")+ **kwargs: Additional parameters for graph partitioning++ Returns:+ Tuple of (disk_graph_index_path, partition_bin_path)+ """+ partitioner = GraphPartitioner(build_type=build_type)+ return partitioner.partition_graph(index_prefix_path, output_dir, partition_prefix, **kwargs)+++# Example usage:+if __name__ == "__main__":+ # Example: partition an index+ try:+ disk_graph_path, partition_bin_path = partition_graph(+ "/path/to/your/index_prefix", gp_times=10, lock_nums=10, cut=100+ )+ print("Partitioning completed successfully!")+ print(f"Disk graph index: {disk_graph_path}")+ print(f"Partition binary: {partition_bin_path}")+ except Exception as e:+ print(f"Partitioning failed: {e}")diff --git a/packages/leann-backend-hnsw/leann_backend_hnsw/convert_to_csr.py b/packages/leann-backend-hnsw/leann_backend_hnsw/convert_to_csr.pyindex 1f9d4f1..f23bdd0 100644--- a/packages/leann-backend-hnsw/leann_backend_hnsw/convert_to_csr.py+++ b/packages/leann-backend-hnsw/leann_backend_hnsw/convert_to_csr.py@@ -1,5 +1,6 @@import argparseimport gc # Import garbage collector interface+import loggingimport osimport structimport sys@@ -7,6 +8,12 @@ import timeimport numpy as np+# Set up logging to avoid print buffer issues+logger = logging.getLogger(__name__)+LOG_LEVEL = os.getenv("LEANN_LOG_LEVEL", "WARNING").upper()+log_level = getattr(logging, LOG_LEVEL, logging.WARNING)+logger.setLevel(log_level)+# --- FourCCs (add more if needed) ---INDEX_HNSW_FLAT_FOURCC = int.from_bytes(b"IHNf", "little")# Add other HNSW fourccs if you expect different storage types inside HNSW@@ -243,6 +250,8 @@ def convert_hnsw_graph_to_csr(input_filename, output_filename, prune_embeddings=output_filename: Output CSR index fileprune_embeddings: Whether to prune embedding storage (write NULL storage marker)"""+ # Keep prints simple; rely on CI runner to flush output as needed+print(f"Starting conversion: {input_filename} -> {output_filename}")start_time = time.time()original_hnsw_data = {}diff --git a/packages/leann-backend-hnsw/leann_backend_hnsw/hnsw_embedding_server.py b/packages/leann-backend-hnsw/leann_backend_hnsw/hnsw_embedding_server.pyindex 013ae5a..7c472ad 100644--- a/packages/leann-backend-hnsw/leann_backend_hnsw/hnsw_embedding_server.py+++ b/packages/leann-backend-hnsw/leann_backend_hnsw/hnsw_embedding_server.py@@ -10,7 +10,7 @@ import sysimport threadingimport timefrom pathlib import Path-from typing import Union+from typing import Optionalimport msgpackimport numpy as np@@ -34,7 +34,7 @@ if not logger.handlers:def create_hnsw_embedding_server(- passages_file: Union[str, None] = None,+ passages_file: Optional[str] = None,zmq_port: int = 5555,model_name: str = "sentence-transformers/all-mpnet-base-v2",distance_metric: str = "mips",@@ -82,199 +82,317 @@ def create_hnsw_embedding_server(with open(passages_file) as f:meta = json.load(f)- # Convert relative paths to absolute paths based on metadata file location- metadata_dir = Path(passages_file).parent.parent # Go up one level from the metadata file- passage_sources = []- for source in meta["passage_sources"]:- source_copy = source.copy()- # Convert relative paths to absolute paths- if not Path(source_copy["path"]).is_absolute():- source_copy["path"] = str(metadata_dir / source_copy["path"])- if not Path(source_copy["index_path"]).is_absolute():- source_copy["index_path"] = str(metadata_dir / source_copy["index_path"])- passage_sources.append(source_copy)-- passages = PassageManager(passage_sources)+ # Let PassageManager handle path resolution uniformly. It supports fallback order:+ # 1) path/index_path; 2) *_relative; 3) standard siblings next to meta+ passages = PassageManager(meta["passage_sources"], metadata_file_path=passages_file)+ # Dimension from metadata for shaping responses+ try:+ embedding_dim: int = int(meta.get("dimensions", 0))+ except Exception:+ embedding_dim = 0logger.info(f"Loaded PassageManager with {len(passages.global_offset_map)} passages from metadata")- def zmq_server_thread():- """ZMQ server thread"""- context = zmq.Context()- socket = context.socket(zmq.REP)- socket.bind(f"tcp://*:{zmq_port}")- logger.info(f"HNSW ZMQ server listening on port {zmq_port}")+ # (legacy ZMQ thread removed; using shutdown-capable server only)- socket.setsockopt(zmq.RCVTIMEO, 300000)- socket.setsockopt(zmq.SNDTIMEO, 300000)+ def zmq_server_thread_with_shutdown(shutdown_event):+ """ZMQ server thread that respects shutdown signal.- while True:- try:- message_bytes = socket.recv()- logger.debug(f"Received ZMQ request of size {len(message_bytes)} bytes")-- e2e_start = time.time()- request_payload = msgpack.unpackb(message_bytes)-- # Handle direct text embedding request- if isinstance(request_payload, list) and len(request_payload) > 0:- # Check if this is a direct text request (list of strings)- if all(isinstance(item, str) for item in request_payload):- logger.info(- f"Processing direct text embedding request for {len(request_payload)} texts in {embedding_mode} mode"- )-- # Use unified embedding computation (now with model caching)- embeddings = compute_embeddings(- request_payload, model_name, mode=embedding_mode- )-- response = embeddings.tolist()- socket.send(msgpack.packb(response))+ Creates its own REP socket bound to zmq_port and polls with timeouts+ to allow graceful shutdown.+ """+ logger.info("ZMQ server thread started with shutdown support")++ context = zmq.Context()+ rep_socket = context.socket(zmq.REP)+ rep_socket.bind(f"tcp://*:{zmq_port}")+ logger.info(f"HNSW ZMQ REP server listening on port {zmq_port}")+ rep_socket.setsockopt(zmq.RCVTIMEO, 1000)+ # Keep sends from blocking during shutdown; fail fast and drop on close+ rep_socket.setsockopt(zmq.SNDTIMEO, 1000)+ rep_socket.setsockopt(zmq.LINGER, 0)++ # Track last request type/length for shape-correct fallbacks+ last_request_type = "unknown" # 'text' | 'distance' | 'embedding' | 'unknown'+ last_request_length = 0++ try:+ while not shutdown_event.is_set():+ try:+ e2e_start = time.time()+ logger.debug("🔍 Waiting for ZMQ message...")+ request_bytes = rep_socket.recv()++ # Rest of the processing logic (same as original)+ request = msgpack.unpackb(request_bytes)++ if len(request) == 1 and request[0] == "__QUERY_MODEL__":+ response_bytes = msgpack.packb([model_name])+ rep_socket.send(response_bytes)+ continue++ # Handle direct text embedding request+ if (+ isinstance(request, list)+ and request+ and all(isinstance(item, str) for item in request)+ ):+ last_request_type = "text"+ last_request_length = len(request)+ embeddings = compute_embeddings(request, model_name, mode=embedding_mode)+ rep_socket.send(msgpack.packb(embeddings.tolist()))e2e_end = time.time()logger.info(f"⏱️ Text embedding E2E time: {e2e_end - e2e_start:.6f}s")continue- # Handle distance calculation requests- if (- isinstance(request_payload, list)- and len(request_payload) == 2- and isinstance(request_payload[0], list)- and isinstance(request_payload[1], list)- ):- node_ids = request_payload[0]- query_vector = np.array(request_payload[1], dtype=np.float32)-- logger.debug("Distance calculation request received")- logger.debug(f" Node IDs: {node_ids}")- logger.debug(f" Query vector dim: {len(query_vector)}")-- # Get embeddings for node IDs- texts = []- for nid in node_ids:+ # Handle distance calculation request: [[ids], [query_vector]]+ if (+ isinstance(request, list)+ and len(request) == 2+ and isinstance(request[0], list)+ and isinstance(request[1], list)+ ):+ node_ids = request[0]+ # Handle nested [[ids]] shape defensively+ if len(node_ids) == 1 and isinstance(node_ids[0], list):+ node_ids = node_ids[0]+ query_vector = np.array(request[1], dtype=np.float32)+ last_request_type = "distance"+ last_request_length = len(node_ids)++ logger.debug("Distance calculation request received")+ logger.debug(f" Node IDs: {node_ids}")+ logger.debug(f" Query vector dim: {len(query_vector)}")++ # Gather texts for found ids+ texts: list[str] = []+ found_indices: list[int] = []+ for idx, nid in enumerate(node_ids):+ try:+ passage_data = passages.get_passage(str(nid))+ txt = passage_data.get("text", "")+ if isinstance(txt, str) and len(txt) > 0:+ texts.append(txt)+ found_indices.append(idx)+ else:+ logger.error(f"Empty text for passage ID {nid}")+ except KeyError:+ logger.error(f"Passage ID {nid} not found")+ except Exception as e:+ logger.error(f"Exception looking up passage ID {nid}: {e}")++ # Prepare full-length response with large sentinel values+ large_distance = 1e9+ response_distances = [large_distance] * len(node_ids)++ if texts:+ try:+ embeddings = compute_embeddings(+ texts, model_name, mode=embedding_mode+ )+ logger.info(+ f"Computed embeddings for {len(texts)} texts, shape: {embeddings.shape}"+ )+ if distance_metric == "l2":+ partial = np.sum(+ np.square(embeddings - query_vector.reshape(1, -1)), axis=1+ )+ else: # mips or cosine+ partial = -np.dot(embeddings, query_vector)++ for pos, dval in zip(found_indices, partial.flatten().tolist()):+ response_distances[pos] = float(dval)+ except Exception as e:+ logger.error(f"Distance computation error, using sentinels: {e}")++ # Send response in expected shape [[distances]]+ rep_socket.send(msgpack.packb([response_distances], use_single_float=True))+ e2e_end = time.time()+ logger.info(f"⏱️ Distance calculation E2E time: {e2e_end - e2e_start:.6f}s")+ continue++ # Fallback: treat as embedding-by-id request+ if (+ isinstance(request, list)+ and len(request) == 1+ and isinstance(request[0], list)+ ):+ node_ids = request[0]+ elif isinstance(request, list):+ node_ids = request+ else:+ node_ids = []+ last_request_type = "embedding"+ last_request_length = len(node_ids)+ logger.info(f"ZMQ received {len(node_ids)} node IDs for embedding fetch")++ # Preallocate zero-filled flat data for robustness+ if embedding_dim <= 0:+ dims = [0, 0]+ flat_data: list[float] = []+ else:+ dims = [len(node_ids), embedding_dim]+ flat_data = [0.0] * (dims[0] * dims[1])++ # Collect texts for found ids+ texts: list[str] = []+ found_indices: list[int] = []+ for idx, nid in enumerate(node_ids):try:passage_data = passages.get_passage(str(nid))- txt = passage_data["text"]- texts.append(txt)+ txt = passage_data.get("text", "")+ if isinstance(txt, str) and len(txt) > 0:+ texts.append(txt)+ found_indices.append(idx)+ else:+ logger.error(f"Empty text for passage ID {nid}")except KeyError:- logger.error(f"Passage ID {nid} not found")- raise RuntimeError(f"FATAL: Passage with ID {nid} not found")+ logger.error(f"Passage with ID {nid} not found")except Exception as e:logger.error(f"Exception looking up passage ID {nid}: {e}")- raise-- # Process embeddings- embeddings = compute_embeddings(texts, model_name, mode=embedding_mode)- logger.info(- f"Computed embeddings for {len(texts)} texts, shape: {embeddings.shape}"- )-- # Calculate distances- if distance_metric == "l2":- distances = np.sum(- np.square(embeddings - query_vector.reshape(1, -1)), axis=1- )- else: # mips or cosine- distances = -np.dot(embeddings, query_vector)-- response_payload = distances.flatten().tolist()- response_bytes = msgpack.packb([response_payload], use_single_float=True)- logger.debug(f"Sending distance response with {len(distances)} distances")-- socket.send(response_bytes)++ if texts:+ try:+ embeddings = compute_embeddings(texts, model_name, mode=embedding_mode)+ logger.info(+ f"Computed embeddings for {len(texts)} texts, shape: {embeddings.shape}"+ )++ if np.isnan(embeddings).any() or np.isinf(embeddings).any():+ logger.error(+ f"NaN or Inf detected in embeddings! Requested IDs: {node_ids[:5]}..."+ )+ dims = [0, embedding_dim]+ flat_data = []+ else:+ emb_f32 = np.ascontiguousarray(embeddings, dtype=np.float32)+ flat = emb_f32.flatten().tolist()+ for j, pos in enumerate(found_indices):+ start = pos * embedding_dim+ end = start + embedding_dim+ if end <= len(flat_data):+ flat_data[start:end] = flat[+ j * embedding_dim : (j + 1) * embedding_dim+ ]+ except Exception as e:+ logger.error(f"Embedding computation error, returning zeros: {e}")++ response_payload = [dims, flat_data]+ response_bytes = msgpack.packb(response_payload, use_single_float=True)++ rep_socket.send(response_bytes)e2e_end = time.time()- logger.info(f"⏱️ Distance calculation E2E time: {e2e_end - e2e_start:.6f}s")- continue+ logger.info(f"⏱️ ZMQ E2E time: {e2e_end - e2e_start:.6f}s")- # Standard embedding request (passage ID lookup)- if (- not isinstance(request_payload, list)- or len(request_payload) != 1- or not isinstance(request_payload[0], list)- ):- logger.error(- f"Invalid MessagePack request format. Expected [[ids...]] or [texts...], got: {type(request_payload)}"- )- socket.send(msgpack.packb([[], []]))+ except zmq.Again:+ # Timeout - check shutdown_event and continuecontinue+ except Exception as e:+ if not shutdown_event.is_set():+ logger.error(f"Error in ZMQ server loop: {e}")+ # Shape-correct fallback+ try:+ if last_request_type == "distance":+ large_distance = 1e9+ fallback_len = max(0, int(last_request_length))+ safe = [[large_distance] * fallback_len]+ elif last_request_type == "embedding":+ bsz = max(0, int(last_request_length))+ dim = max(0, int(embedding_dim))+ safe = (+ [[bsz, dim], [0.0] * (bsz * dim)] if dim > 0 else [[0, 0], []]+ )+ elif last_request_type == "text":+ safe = [] # direct text embeddings expectation is a flat list+ else:+ safe = [[0, int(embedding_dim) if embedding_dim > 0 else 0], []]+ rep_socket.send(msgpack.packb(safe, use_single_float=True))+ except Exception:+ pass+ else:+ logger.info("Shutdown in progress, ignoring ZMQ error")+ break+ finally:+ try:+ rep_socket.close(0)+ except Exception:+ pass+ try:+ context.term()+ except Exception:+ pass++ logger.info("ZMQ server thread exiting gracefully")++ # Add shutdown coordination+ shutdown_event = threading.Event()++ def shutdown_zmq_server():+ """Gracefully shutdown ZMQ server."""+ logger.info("Initiating graceful shutdown...")+ shutdown_event.set()++ if zmq_thread.is_alive():+ logger.info("Waiting for ZMQ thread to finish...")+ zmq_thread.join(timeout=5)+ if zmq_thread.is_alive():+ logger.warning("ZMQ thread did not finish in time")++ # Clean up ZMQ resources+ try:+ # Note: socket and context are cleaned up by thread exit+ logger.info("ZMQ resources cleaned up")+ except Exception as e:+ logger.warning(f"Error cleaning ZMQ resources: {e}")++ # Clean up other resources+ try:+ import gc++ gc.collect()+ logger.info("Additional resources cleaned up")+ except Exception as e:+ logger.warning(f"Error cleaning additional resources: {e}")++ logger.info("Graceful shutdown completed")+ sys.exit(0)++ # Register signal handlers within this function scope+ import signal++ def signal_handler(sig, frame):+ logger.info(f"Received signal {sig}, shutting down gracefully...")+ shutdown_zmq_server()++ signal.signal(signal.SIGTERM, signal_handler)+ signal.signal(signal.SIGINT, signal_handler)- node_ids = request_payload[0]- logger.debug(f"Request for {len(node_ids)} node embeddings")-- # Look up texts by node IDs- texts = []- for nid in node_ids:- try:- passage_data = passages.get_passage(str(nid))- txt = passage_data["text"]- if not txt:- raise RuntimeError(f"FATAL: Empty text for passage ID {nid}")- texts.append(txt)- except KeyError:- raise RuntimeError(f"FATAL: Passage with ID {nid} not found")- except Exception as e:- logger.error(f"Exception looking up passage ID {nid}: {e}")- raise-- # Process embeddings- embeddings = compute_embeddings(texts, model_name, mode=embedding_mode)- logger.info(- f"Computed embeddings for {len(texts)} texts, shape: {embeddings.shape}"- )-- # Serialization and response- if np.isnan(embeddings).any() or np.isinf(embeddings).any():- logger.error(- f"NaN or Inf detected in embeddings! Requested IDs: {node_ids[:5]}..."- )- raise AssertionError()-- hidden_contiguous_f32 = np.ascontiguousarray(embeddings, dtype=np.float32)- response_payload = [- list(hidden_contiguous_f32.shape),- hidden_contiguous_f32.flatten().tolist(),- ]- response_bytes = msgpack.packb(response_payload, use_single_float=True)-- socket.send(response_bytes)- e2e_end = time.time()- logger.info(f"⏱️ ZMQ E2E time: {e2e_end - e2e_start:.6f}s")-- except zmq.Again:- logger.debug("ZMQ socket timeout, continuing to listen")- continue- except Exception as e:- logger.error(f"Error in ZMQ server loop: {e}")- import traceback-- traceback.print_exc()- socket.send(msgpack.packb([[], []]))-- zmq_thread = threading.Thread(target=zmq_server_thread, daemon=True)+ # Pass shutdown_event to ZMQ thread+ zmq_thread = threading.Thread(+ target=lambda: zmq_server_thread_with_shutdown(shutdown_event),+ daemon=False, # Not daemon - we want to wait for it+ )zmq_thread.start()logger.info(f"Started HNSW ZMQ server thread on port {zmq_port}")# Keep the main thread alivetry:- while True:- time.sleep(1)+ while not shutdown_event.is_set():+ time.sleep(0.1) # Check shutdown more frequentlyexcept KeyboardInterrupt:logger.info("HNSW Server shutting down...")+ shutdown_zmq_server()return+ # If we reach here, shutdown was triggered by signal+ logger.info("Main loop exited, process should be shutting down")+if __name__ == "__main__":- import signalimport sys- def signal_handler(sig, frame):- logger.info(f"Received signal {sig}, shutting down gracefully...")- sys.exit(0)-- # Register signal handlers for graceful shutdown- signal.signal(signal.SIGTERM, signal_handler)- signal.signal(signal.SIGINT, signal_handler)+ # Signal handlers are now registered within create_hnsw_embedding_serverparser = argparse.ArgumentParser(description="HNSW Embedding service")parser.add_argument("--zmq-port", type=int, default=5555, help="ZMQ port to run on")diff --git a/packages/leann-core/src/leann/api.py b/packages/leann-core/src/leann/api.pyindex 65d75b9..bff3709 100644--- a/packages/leann-core/src/leann/api.py+++ b/packages/leann-core/src/leann/api.py@@ -115,20 +115,62 @@ class SearchResult:class PassageManager:- def __init__(self, passage_sources: list[dict[str, Any]]):+ def __init__(+ self, passage_sources: list[dict[str, Any]], metadata_file_path: Optional[str] = None+ ):self.offset_maps = {}self.passage_files = {}self.global_offset_map = {} # Combined map for fast lookup+ # Derive index base name for standard sibling fallbacks, e.g., <index_name>.passages.*+ index_name_base = None+ if metadata_file_path:+ meta_name = Path(metadata_file_path).name+ if meta_name.endswith(".meta.json"):+ index_name_base = meta_name[: -len(".meta.json")]+for source in passage_sources:assert source["type"] == "jsonl", "only jsonl is supported"- passage_file = source["path"]- index_file = source["index_path"] # .idx file+ passage_file = source.get("path", "")+ index_file = source.get("index_path", "") # .idx file++ # Fix path resolution - relative paths should be relative to metadata file directory+ def _resolve_candidates(+ primary: str,+ relative_key: str,+ default_name: Optional[str],+ source_dict: dict[str, Any],+ ) -> list[Path]:+ candidates: list[Path] = []+ # 1) Primary as-is (absolute or relative)+ if primary:+ p = Path(primary)+ candidates.append(p if p.is_absolute() else (Path.cwd() / p))+ # 2) metadata-relative explicit relative key+ if metadata_file_path and source_dict.get(relative_key):+ candidates.append(Path(metadata_file_path).parent / source_dict[relative_key])+ # 3) metadata-relative standard sibling filename+ if metadata_file_path and default_name:+ candidates.append(Path(metadata_file_path).parent / default_name)+ return candidates++ # Build candidate lists and pick first existing; otherwise keep last candidate for error message+ idx_default = f"{index_name_base}.passages.idx" if index_name_base else None+ idx_candidates = _resolve_candidates(+ index_file, "index_path_relative", idx_default, source+ )+ pas_default = f"{index_name_base}.passages.jsonl" if index_name_base else None+ pas_candidates = _resolve_candidates(passage_file, "path_relative", pas_default, source)- # Fix path resolution for Colab and other environments- if not Path(index_file).is_absolute():- # If relative path, try to resolve it properly- index_file = str(Path(index_file).resolve())+ def _pick_existing(cands: list[Path]) -> str:+ for c in cands:+ if c.exists():+ return str(c.resolve())+ # Fallback to last candidate (best guess) even if not exists; will error below+ return str(cands[-1].resolve()) if cands else ""++ index_file = _pick_existing(idx_candidates)+ passage_file = _pick_existing(pas_candidates)if not Path(index_file).exists():raise FileNotFoundError(f"Passage index file not found: {index_file}")@@ -314,8 +356,12 @@ class LeannBuilder:"passage_sources": [{"type": "jsonl",- "path": str(passages_file),- "index_path": str(offset_file),+ # Preserve existing relative file names (backward-compatible)+ "path": passages_file.name,+ "index_path": offset_file.name,+ # Add optional redundant relative keys for remote build portability (non-breaking)+ "path_relative": passages_file.name,+ "index_path_relative": offset_file.name,}],}@@ -430,8 +476,12 @@ class LeannBuilder:"passage_sources": [{"type": "jsonl",- "path": str(passages_file),- "index_path": str(offset_file),+ # Preserve existing relative file names (backward-compatible)+ "path": passages_file.name,+ "index_path": offset_file.name,+ # Add optional redundant relative keys for remote build portability (non-breaking)+ "path_relative": passages_file.name,+ "index_path_relative": offset_file.name,}],"built_from_precomputed_embeddings": True,@@ -473,7 +523,9 @@ class LeannSearcher:self.embedding_model = self.meta_data["embedding_model"]# Support both old and new formatself.embedding_mode = self.meta_data.get("embedding_mode", "sentence-transformers")- self.passage_manager = PassageManager(self.meta_data.get("passage_sources", []))+ self.passage_manager = PassageManager(+ self.meta_data.get("passage_sources", []), metadata_file_path=self.meta_path_str+ )backend_factory = BACKEND_REGISTRY.get(backend_name)if backend_factory is None:raise ValueError(f"Backend '{backend_name}' not found.")@@ -546,13 +598,13 @@ class LeannSearcher:zmq_port=zmq_port,**kwargs,)- time.time() - start_time# logger.info(f" Search time: {search_time} seconds")logger.info(f" Backend returned: labels={len(results.get('labels', [[]])[0])} results")enriched_results = []if "labels" in results and "distances" in results:logger.info(f" Processing {len(results['labels'][0])} passage IDs:")+ # Python 3.9 does not support zip(strict=...); lengths are expected to matchfor i, (string_id, dist) in enumerate(zip(results["labels"][0], results["distances"][0])):@@ -580,13 +632,26 @@ class LeannSearcher:)except KeyError:RED = "\033[91m"+ RESET = "\033[0m"logger.error(f" {RED}✗{RESET} [{i + 1:2d}] ID: '{string_id}' -> {RED}ERROR: Passage not found!{RESET}")+ # Define color codes outside the loop for final message+ GREEN = "\033[92m"+ RESET = "\033[0m"logger.info(f" {GREEN}✓ Final enriched results: {len(enriched_results)} passages{RESET}")return enriched_results+ def cleanup(self):+ """Explicitly cleanup embedding server resources.++ This method should be called after you're done using the searcher,+ especially in test environments or batch processing scenarios.+ """+ if hasattr(self.backend_impl, "embedding_server_manager"):+ self.backend_impl.embedding_server_manager.stop_server()+class LeannChat:def __init__(@@ -656,3 +721,12 @@ class LeannChat:except (KeyboardInterrupt, EOFError):print("\nGoodbye!")break++ def cleanup(self):+ """Explicitly cleanup embedding server resources.++ This method should be called after you're done using the chat interface,+ especially in test environments or batch processing scenarios.+ """+ if hasattr(self.searcher, "cleanup"):+ self.searcher.cleanup()diff --git a/packages/leann-core/src/leann/embedding_server_manager.py b/packages/leann-core/src/leann/embedding_server_manager.pyindex 74ecd69..3ed223f 100644--- a/packages/leann-core/src/leann/embedding_server_manager.py+++ b/packages/leann-core/src/leann/embedding_server_manager.py@@ -8,7 +8,7 @@ import timefrom pathlib import Pathfrom typing import Optional-import psutil+# Lightweight, self-contained server manager with no cross-process inspection# Set up logging based on environment variableLOG_LEVEL = os.getenv("LEANN_LOG_LEVEL", "WARNING").upper()@@ -43,130 +43,7 @@ def _check_port(port: int) -> bool:return s.connect_ex(("localhost", port)) == 0-def _check_process_matches_config(- port: int, expected_model: str, expected_passages_file: str-) -> bool:- """- Check if the process using the port matches our expected model and passages file.- Returns True if matches, False otherwise.- """- try:- for proc in psutil.process_iter(["pid", "cmdline"]):- if not _is_process_listening_on_port(proc, port):- continue-- cmdline = proc.info["cmdline"]- if not cmdline:- continue-- return _check_cmdline_matches_config(- cmdline, port, expected_model, expected_passages_file- )-- logger.debug(f"No process found listening on port {port}")- return False-- except Exception as e:- logger.warning(f"Could not check process on port {port}: {e}")- return False---def _is_process_listening_on_port(proc, port: int) -> bool:- """Check if a process is listening on the given port."""- try:- connections = proc.net_connections()- for conn in connections:- if conn.laddr.port == port and conn.status == psutil.CONN_LISTEN:- return True- return False- except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):- return False---def _check_cmdline_matches_config(- cmdline: list, port: int, expected_model: str, expected_passages_file: str-) -> bool:- """Check if command line matches our expected configuration."""- cmdline_str = " ".join(cmdline)- logger.debug(f"Found process on port {port}: {cmdline_str}")-- # Check if it's our embedding server- is_embedding_server = any(- server_type in cmdline_str- for server_type in [- "embedding_server",- "leann_backend_diskann.embedding_server",- "leann_backend_hnsw.hnsw_embedding_server",- ]- )-- if not is_embedding_server:- logger.debug(f"Process on port {port} is not our embedding server")- return False-- # Check model name- model_matches = _check_model_in_cmdline(cmdline, expected_model)-- # Check passages file if provided- passages_matches = _check_passages_in_cmdline(cmdline, expected_passages_file)-- result = model_matches and passages_matches- logger.debug(- f"model_matches: {model_matches}, passages_matches: {passages_matches}, overall: {result}"- )- return result---def _check_model_in_cmdline(cmdline: list, expected_model: str) -> bool:- """Check if the command line contains the expected model."""- if "--model-name" not in cmdline:- return False-- model_idx = cmdline.index("--model-name")- if model_idx + 1 >= len(cmdline):- return False-- actual_model = cmdline[model_idx + 1]- return actual_model == expected_model---def _check_passages_in_cmdline(cmdline: list, expected_passages_file: str) -> bool:- """Check if the command line contains the expected passages file."""- if "--passages-file" not in cmdline:- return False # Expected but not found-- passages_idx = cmdline.index("--passages-file")- if passages_idx + 1 >= len(cmdline):- return False-- actual_passages = cmdline[passages_idx + 1]- expected_path = Path(expected_passages_file).resolve()- actual_path = Path(actual_passages).resolve()- return actual_path == expected_path---def _find_compatible_port_or_next_available(- start_port: int, model_name: str, passages_file: str, max_attempts: int = 100-) -> tuple[int, bool]:- """- Find a port that either has a compatible server or is available.- Returns (port, is_compatible) where is_compatible indicates if we found a matching server.- """- for port in range(start_port, start_port + max_attempts):- if not _check_port(port):- # Port is available- return port, False-- # Port is in use, check if it's compatible- if _check_process_matches_config(port, model_name, passages_file):- logger.info(f"Found compatible server on port {port}")- return port, True- else:- logger.info(f"Port {port} has incompatible server, trying next port...")-- raise RuntimeError(- f"Could not find compatible or available port in range {start_port}-{start_port + max_attempts}"- )+# Note: All cross-process scanning helpers removed for simplicityclass EmbeddingServerManager:@@ -185,7 +62,16 @@ class EmbeddingServerManager:self.backend_module_name = backend_module_nameself.server_process: Optional[subprocess.Popen] = Noneself.server_port: Optional[int] = None+ # Track last-started config for in-process reuse only+ self._server_config: Optional[dict] = Noneself._atexit_registered = False+ # Also register a weakref finalizer to ensure cleanup when manager is GC'ed+ try:+ import weakref++ self._finalizer = weakref.finalize(self, self._finalize_process)+ except Exception:+ self._finalizer = Nonedef start_server(self,@@ -195,26 +81,24 @@ class EmbeddingServerManager:**kwargs,) -> tuple[bool, int]:"""Start the embedding server."""- passages_file = kwargs.get("passages_file")+ # passages_file may be present in kwargs for server CLI, but we don't need it here- # Check if we have a compatible server already running- if self._has_compatible_running_server(model_name, passages_file):- logger.info("Found compatible running server!")- return True, port+ # If this manager already has a live server, just reuse it+ if self.server_process and self.server_process.poll() is None and self.server_port:+ logger.info("Reusing in-process server")+ return True, self.server_port# For Colab environment, use a different strategyif _is_colab_environment():logger.info("Detected Colab environment, using alternative startup strategy")return self._start_server_colab(port, model_name, embedding_mode, **kwargs)- # Find a compatible port or next available- actual_port, is_compatible = _find_compatible_port_or_next_available(- port, model_name, passages_file- )-- if is_compatible:- logger.info(f"Found compatible server on port {actual_port}")- return True, actual_port+ # Always pick a fresh available port+ try:+ actual_port = _get_available_port(port)+ except RuntimeError:+ logger.error("No available ports found")+ return False, port# Start a new serverreturn self._start_new_server(actual_port, model_name, embedding_mode, **kwargs)@@ -247,17 +131,7 @@ class EmbeddingServerManager:logger.error(f"Failed to start embedding server in Colab: {e}")return False, actual_port- def _has_compatible_running_server(self, model_name: str, passages_file: str) -> bool:- """Check if we have a compatible running server."""- if not (self.server_process and self.server_process.poll() is None and self.server_port):- return False-- if _check_process_matches_config(self.server_port, model_name, passages_file):- logger.info(f"Existing server process (PID {self.server_process.pid}) is compatible")- return True-- logger.info("Existing server process is incompatible. Should start a new server.")- return False+ # Note: No compatibility check needed; manager is per-searcher and configs are stable per instancedef _start_new_server(self, port: int, model_name: str, embedding_mode: str, **kwargs@@ -304,22 +178,61 @@ class EmbeddingServerManager:project_root = Path(__file__).parent.parent.parent.parent.parentlogger.info(f"Command: {' '.join(command)}")- # Let server output go directly to console- # The server will respect LEANN_LOG_LEVEL environment variable+ # In CI environment, redirect stdout to avoid buffer deadlock but keep stderr for debugging+ # Embedding servers use many print statements that can fill stdout buffers+ is_ci = os.environ.get("CI") == "true"+ if is_ci:+ stdout_target = subprocess.DEVNULL+ stderr_target = None # Keep stderr for error debugging in CI+ logger.info(+ "CI environment detected, redirecting embedding server stdout to DEVNULL, keeping stderr"+ )+ else:+ stdout_target = None # Direct to console for visible logs+ stderr_target = None # Direct to console for visible logs++ # Start embedding server subprocessself.server_process = subprocess.Popen(command,cwd=project_root,- stdout=None, # Direct to console- stderr=None, # Direct to console+ stdout=stdout_target,+ stderr=stderr_target,)self.server_port = port+ # Record config for in-process reuse+ try:+ self._server_config = {+ "model_name": command[command.index("--model-name") + 1]+ if "--model-name" in command+ else "",+ "passages_file": command[command.index("--passages-file") + 1]+ if "--passages-file" in command+ else "",+ "embedding_mode": command[command.index("--embedding-mode") + 1]+ if "--embedding-mode" in command+ else "sentence-transformers",+ }+ except Exception:+ self._server_config = {+ "model_name": "",+ "passages_file": "",+ "embedding_mode": "sentence-transformers",+ }logger.info(f"Server process started with PID: {self.server_process.pid}")# Register atexit callback only when we actually start a processif not self._atexit_registered:- # Use a lambda to avoid issues with bound methods- atexit.register(lambda: self.stop_server() if self.server_process else None)+ # Always attempt best-effort finalize at interpreter exit+ atexit.register(self._finalize_process)self._atexit_registered = True+ # Touch finalizer so it knows there is a live process+ if getattr(self, "_finalizer", None) is not None and not self._finalizer.alive:+ try:+ import weakref++ self._finalizer = weakref.finalize(self, self._finalize_process)+ except Exception:+ passdef _wait_for_server_ready(self, port: int) -> tuple[bool, int]:"""Wait for the server to be ready."""@@ -344,22 +257,26 @@ class EmbeddingServerManager:if not self.server_process:return- if self.server_process.poll() is not None:+ if self.server_process and self.server_process.poll() is not None:# Process already terminatedself.server_process = None+ self.server_port = None+ self._server_config = Nonereturnlogger.info(f"Terminating server process (PID: {self.server_process.pid}) for backend {self.backend_module_name}...")++ # Use simple termination - our improved server shutdown should handle this properlyself.server_process.terminate()try:- self.server_process.wait(timeout=3)- logger.info(f"Server process {self.server_process.pid} terminated.")+ self.server_process.wait(timeout=5) # Give more time for graceful shutdown+ logger.info(f"Server process {self.server_process.pid} terminated gracefully.")except subprocess.TimeoutExpired:logger.warning(- f"Server process {self.server_process.pid} did not terminate gracefully within 3 seconds, killing it."+ f"Server process {self.server_process.pid} did not terminate within 5 seconds, force killing...")self.server_process.kill()try:@@ -369,15 +286,33 @@ class EmbeddingServerManager:logger.error(f"Failed to kill server process {self.server_process.pid} - it may be hung")- # Don't hang indefinitely- # Clean up process resources to prevent resource tracker warnings+ # Clean up process resources with timeout to avoid CI hang+ try:+ # Use shorter timeout in CI environments+ is_ci = os.environ.get("CI") == "true"+ timeout = 3 if is_ci else 10+ self.server_process.wait(timeout=timeout)+ logger.info(f"Server process {self.server_process.pid} cleanup completed")+ except subprocess.TimeoutExpired:+ logger.warning(f"Process cleanup timeout after {timeout}s, proceeding anyway")+ except Exception as e:+ logger.warning(f"Error during process cleanup: {e}")+ finally:+ self.server_process = None+ self.server_port = None+ self._server_config = None++ def _finalize_process(self) -> None:+ """Best-effort cleanup used by weakref.finalize/atexit."""try:- self.server_process.wait() # Ensure process is fully cleaned up+ self.stop_server()except Exception:pass- self.server_process = None+ def _adopt_existing_server(self, *args, **kwargs) -> None:+ # Removed: cross-process adoption no longer supported+ returndef _launch_server_process_colab(self, command: list, port: int) -> None:"""Launch the server process with Colab-specific settings."""@@ -393,10 +328,16 @@ class EmbeddingServerManager:self.server_port = portlogger.info(f"Colab server process started with PID: {self.server_process.pid}")- # Register atexit callback+ # Register atexit callback (unified)if not self._atexit_registered:- atexit.register(lambda: self.stop_server() if self.server_process else None)+ atexit.register(self._finalize_process)self._atexit_registered = True+ # Record config for in-process reuse is best-effort in Colab mode+ self._server_config = {+ "model_name": "",+ "passages_file": "",+ "embedding_mode": "sentence-transformers",+ }def _wait_for_server_ready_colab(self, port: int) -> tuple[bool, int]:"""Wait for the server to be ready with Colab-specific timeout."""diff --git a/packages/leann-core/src/leann/interface.py b/packages/leann-core/src/leann/interface.pyindex b98c28d..83803a0 100644--- a/packages/leann-core/src/leann/interface.py+++ b/packages/leann-core/src/leann/interface.py@@ -1,5 +1,5 @@from abc import ABC, abstractmethod-from typing import Any, Literal, Union+from typing import Any, Literal, Optionalimport numpy as np@@ -35,7 +35,7 @@ class LeannBackendSearcherInterface(ABC):@abstractmethoddef _ensure_server_running(- self, passages_source_file: str, port: Union[int, None], **kwargs+ self, passages_source_file: str, port: Optional[int], **kwargs) -> int:"""Ensure server is running"""pass@@ -50,7 +50,7 @@ class LeannBackendSearcherInterface(ABC):prune_ratio: float = 0.0,recompute_embeddings: bool = False,pruning_strategy: Literal["global", "local", "proportional"] = "global",- zmq_port: Union[int, None] = None,+ zmq_port: Optional[int] = None,**kwargs,) -> dict[str, Any]:"""Search for nearest neighbors@@ -76,7 +76,7 @@ class LeannBackendSearcherInterface(ABC):self,query: str,use_server_if_available: bool = True,- zmq_port: Union[int, None] = None,+ zmq_port: Optional[int] = None,) -> np.ndarray:"""Compute embedding for a query stringdiff --git a/packages/leann-core/src/leann/mcp.py b/packages/leann-core/src/leann/mcp.pyindex 92f79e3..f370f7b 100755--- a/packages/leann-core/src/leann/mcp.py+++ b/packages/leann-core/src/leann/mcp.py@@ -116,7 +116,6 @@ def handle_request(request):f"--top-k={args.get('top_k', 5)}",f"--complexity={args.get('complexity', 32)}",]-result = subprocess.run(cmd, capture_output=True, text=True)elif tool_name == "leann_status":diff --git a/pyproject.toml b/pyproject.tomlindex 7992b42..73dd7a7 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -43,6 +43,7 @@ dependencies = ["mlx>=0.26.3; sys_platform == 'darwin' and platform_machine == 'arm64'","mlx-lm>=0.26.0; sys_platform == 'darwin' and platform_machine == 'arm64'","psutil>=5.8.0",+ "pybind11>=3.0.0","pathspec>=0.12.1","nbconvert>=7.16.6","gitignore-parser>=0.1.12",@@ -54,7 +55,7 @@ dev = ["pytest-cov>=4.0","pytest-xdist>=3.0", # For parallel test execution"black>=23.0",- "ruff>=0.1.0",+ "ruff==0.12.7", # Fixed version to ensure consistent formatting across all environments"matplotlib","huggingface-hub>=0.20.0","pre-commit>=3.5.0",@@ -154,7 +155,7 @@ markers = ["slow: marks tests as slow (deselect with '-m \"not slow\"')","openai: marks tests that require OpenAI API key",]-timeout = 600+timeout = 300 # Reduced from 600s (10min) to 300s (5min) for CI safetyaddopts = ["-v","--tb=short",diff --git a/tests/README.md b/tests/README.mdindex 22822bd..4d56d10 100644--- a/tests/README.md+++ b/tests/README.md@@ -6,10 +6,11 @@ This directory contains automated tests for the LEANN project using pytest.### `test_readme_examples.py`Tests the examples shown in README.md:-- The basic example code that users see first+- The basic example code that users see first (parametrized for both HNSW and DiskANN backends)- Import statements work correctly- Different backend options (HNSW, DiskANN)-- Different LLM configuration options+- Different LLM configuration options (parametrized for both backends)+- **All main README examples are tested with both HNSW and DiskANN backends using pytest parametrization**### `test_basic.py`Basic functionality tests that verify:@@ -25,6 +26,16 @@ Tests the document RAG example functionality:- Tests error handling with invalid parameters- Verifies that normalized embeddings are detected and cosine distance is used+### `test_diskann_partition.py`+Tests DiskANN graph partitioning functionality:+- Tests DiskANN index building without partitioning (baseline)+- Tests automatic graph partitioning with `is_recompute=True`+- Verifies that partition files are created and large files are cleaned up for storage saving+- Tests search functionality with partitioned indices+- Validates medoid and max_base_norm file generation and usage+- Includes performance comparison between DiskANN (with partition) and HNSW+- **Note**: These tests are skipped in CI due to hardware requirements and computation time+## Running Tests### Install test dependencies:@@ -54,15 +65,23 @@ pytest tests/ -m "not openai"# Skip slow testspytest tests/ -m "not slow"++# Run DiskANN partition tests (requires local machine, not CI)+pytest tests/test_diskann_partition.py```### Run with specific backend:```bash# Test only HNSW backendpytest tests/test_basic.py::test_backend_basic[hnsw]+pytest tests/test_readme_examples.py::test_readme_basic_example[hnsw]# Test only DiskANN backendpytest tests/test_basic.py::test_backend_basic[diskann]+pytest tests/test_readme_examples.py::test_readme_basic_example[diskann]++# All DiskANN tests (parametrized + specialized partition tests)+pytest tests/ -k diskann```## CI/CD Integrationdiff --git a/tests/test_basic.py b/tests/test_basic.pyindex 800b0ac..651111f 100644--- a/tests/test_basic.py+++ b/tests/test_basic.py@@ -64,6 +64,9 @@ def test_backend_basic(backend_name):assert isinstance(results[0], SearchResult)assert "topic 2" in results[0].text or "document" in results[0].text+ # Ensure cleanup to avoid hanging background servers+ searcher.cleanup()+@pytest.mark.skipif(os.environ.get("CI") == "true", reason="Skip model tests in CI to avoid MPS memory issues"@@ -90,3 +93,5 @@ def test_large_index():searcher = LeannSearcher(index_path)results = searcher.search(["word10 word20"], top_k=10)assert len(results[0]) == 10+ # Cleanup+ searcher.cleanup()diff --git a/tests/test_diskann_partition.py b/tests/test_diskann_partition.pynew file mode 100644index 0000000..5b3d5f0--- /dev/null+++ b/tests/test_diskann_partition.py@@ -0,0 +1,369 @@+"""+Test DiskANN graph partitioning functionality.++Tests the automatic graph partitioning feature that was implemented to save+storage space by partitioning large DiskANN indices and safely deleting+redundant files while maintaining search functionality.+"""++import os+import tempfile+from pathlib import Path++import pytest+++ os.environ.get("CI") == "true",+ reason="Skip DiskANN partition tests in CI - requires specific hardware and large memory",+)+def test_diskann_without_partition():+ """Test DiskANN index building without partition (baseline)."""+ from leann.api import LeannBuilder, LeannSearcher++ with tempfile.TemporaryDirectory() as temp_dir:+ index_path = str(Path(temp_dir) / "test_no_partition.leann")++ # Test data - enough to trigger index building+ texts = [+ f"Document {i} discusses topic {i % 10} with detailed analysis of subject {i // 10}."+ for i in range(500)+ ]++ # Build without partition (is_recompute=False)+ builder = LeannBuilder(+ backend_name="diskann",+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ num_neighbors=32,+ search_list_size=50,+ is_recompute=False, # No partition+ )++ for text in texts:+ builder.add_text(text)++ builder.build_index(index_path)++ # Verify index was created+ index_dir = Path(index_path).parent+ assert index_dir.exists()++ # Check that traditional DiskANN files exist+ index_prefix = Path(index_path).stem+ # Core DiskANN files (beam search index may not be created for small datasets)+ required_files = [+ f"{index_prefix}_disk.index",+ f"{index_prefix}_pq_compressed.bin",+ f"{index_prefix}_pq_pivots.bin",+ ]++ # Check all generated files first for debugging+ generated_files = [f.name for f in index_dir.glob(f"{index_prefix}*")]+ print(f"Generated files: {generated_files}")++ for required_file in required_files:+ file_path = index_dir / required_file+ assert file_path.exists(), f"Required file {required_file} not found"++ # Ensure no partition files exist in non-partition mode+ partition_files = [f"{index_prefix}_disk_graph.index", f"{index_prefix}_partition.bin"]++ for partition_file in partition_files:+ file_path = index_dir / partition_file+ assert not file_path.exists(), (+ f"Partition file {partition_file} should not exist in non-partition mode"+ )++ # Test search functionality+ searcher = LeannSearcher(index_path)+ results = searcher.search("topic 3 analysis", top_k=3)++ assert len(results) > 0+ assert all(result.score is not None and result.score != float("-inf") for result in results)+++ os.environ.get("CI") == "true",+ reason="Skip DiskANN partition tests in CI - requires specific hardware and large memory",+)+def test_diskann_with_partition():+ """Test DiskANN index building with automatic graph partitioning."""+ from leann.api import LeannBuilder++ with tempfile.TemporaryDirectory() as temp_dir:+ index_path = str(Path(temp_dir) / "test_with_partition.leann")++ # Test data - enough to trigger partitioning+ texts = [+ f"Document {i} explores subject {i % 15} with comprehensive coverage of area {i // 15}."+ for i in range(500)+ ]++ # Build with partition (is_recompute=True)+ builder = LeannBuilder(+ backend_name="diskann",+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ num_neighbors=32,+ search_list_size=50,+ is_recompute=True, # Enable automatic partitioning+ )++ for text in texts:+ builder.add_text(text)++ builder.build_index(index_path)++ # Verify index was created+ index_dir = Path(index_path).parent+ assert index_dir.exists()++ # Check that partition files exist+ index_prefix = Path(index_path).stem+ partition_files = [+ f"{index_prefix}_disk_graph.index", # Partitioned graph+ f"{index_prefix}_partition.bin", # Partition metadata+ f"{index_prefix}_pq_compressed.bin",+ f"{index_prefix}_pq_pivots.bin",+ ]++ for partition_file in partition_files:+ file_path = index_dir / partition_file+ assert file_path.exists(), f"Expected partition file {partition_file} not found"++ # Check that large files were cleaned up (storage saving goal)+ large_files = [f"{index_prefix}_disk.index", f"{index_prefix}_disk_beam_search.index"]++ for large_file in large_files:+ file_path = index_dir / large_file+ assert not file_path.exists(), (+ f"Large file {large_file} should have been deleted for storage saving"+ )++ # Verify required auxiliary files for partition mode exist+ required_files = [+ f"{index_prefix}_disk.index_medoids.bin",+ f"{index_prefix}_disk.index_max_base_norm.bin",+ ]++ for req_file in required_files:+ file_path = index_dir / req_file+ assert file_path.exists(), (+ f"Required auxiliary file {req_file} missing for partition mode"+ )+++ os.environ.get("CI") == "true",+ reason="Skip DiskANN partition tests in CI - requires specific hardware and large memory",+)+def test_diskann_partition_search_functionality():+ """Test that search works correctly with partitioned indices."""+ from leann.api import LeannBuilder, LeannSearcher++ with tempfile.TemporaryDirectory() as temp_dir:+ index_path = str(Path(temp_dir) / "test_partition_search.leann")++ # Create diverse test data+ texts = [+ "LEANN is a storage-efficient approximate nearest neighbor search system.",+ "Graph partitioning helps reduce memory usage in large scale vector search.",+ "DiskANN provides high-performance disk-based approximate nearest neighbor search.",+ "Vector embeddings enable semantic search over unstructured text data.",+ "Approximate nearest neighbor algorithms trade accuracy for speed and storage.",+ ] * 100 # Repeat to get enough data++ # Build with partitioning+ builder = LeannBuilder(+ backend_name="diskann",+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ is_recompute=True, # Enable partitioning+ )++ for text in texts:+ builder.add_text(text)++ builder.build_index(index_path)++ # Test search with partitioned index+ searcher = LeannSearcher(index_path)++ # Test various queries+ test_queries = [+ ("vector search algorithms", 5),+ ("LEANN storage efficiency", 3),+ ("graph partitioning memory", 4),+ ("approximate nearest neighbor", 7),+ ]++ for query, top_k in test_queries:+ results = searcher.search(query, top_k=top_k)++ # Verify search results+ assert len(results) == top_k, f"Expected {top_k} results for query '{query}'"+ assert all(result.score is not None for result in results), (+ "All results should have scores"+ )+ assert all(result.score != float("-inf") for result in results), (+ "No result should have -inf score"+ )+ assert all(result.text is not None for result in results), (+ "All results should have text"+ )++ # Scores should be in descending order (higher similarity first)+ scores = [result.score for result in results]+ assert scores == sorted(scores, reverse=True), (+ "Results should be sorted by score descending"+ )+++ os.environ.get("CI") == "true",+ reason="Skip DiskANN partition tests in CI - requires specific hardware and large memory",+)+def test_diskann_medoid_and_norm_files():+ """Test that medoid and max_base_norm files are correctly generated and used."""+ import struct++ from leann.api import LeannBuilder, LeannSearcher++ with tempfile.TemporaryDirectory() as temp_dir:+ index_path = str(Path(temp_dir) / "test_medoid_norm.leann")++ # Small but sufficient dataset+ texts = [f"Test document {i} with content about subject {i % 10}." for i in range(200)]++ builder = LeannBuilder(+ backend_name="diskann",+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ is_recompute=True,+ )++ for text in texts:+ builder.add_text(text)++ builder.build_index(index_path)++ index_dir = Path(index_path).parent+ index_prefix = Path(index_path).stem++ # Test medoids file+ medoids_file = index_dir / f"{index_prefix}_disk.index_medoids.bin"+ assert medoids_file.exists(), "Medoids file should be generated"++ # Read and validate medoids file format+ with open(medoids_file, "rb") as f:+ nshards = struct.unpack("<I", f.read(4))[0]+ one_val = struct.unpack("<I", f.read(4))[0]+ medoid_id = struct.unpack("<I", f.read(4))[0]++ assert nshards == 1, "Single-shot build should have 1 shard"+ assert one_val == 1, "Expected value should be 1"+ assert medoid_id >= 0, "Medoid ID should be valid (not hardcoded 0)"++ # Test max_base_norm file+ norm_file = index_dir / f"{index_prefix}_disk.index_max_base_norm.bin"+ assert norm_file.exists(), "Max base norm file should be generated"++ # Read and validate norm file+ with open(norm_file, "rb") as f:+ npts = struct.unpack("<I", f.read(4))[0]+ ndims = struct.unpack("<I", f.read(4))[0]+ norm_val = struct.unpack("<f", f.read(4))[0]++ assert npts == 1, "Should have 1 norm point"+ assert ndims == 1, "Should have 1 dimension"+ assert norm_val > 0, "Norm value should be positive"+ assert norm_val != float("inf"), "Norm value should be finite"++ # Test that search works with these files+ searcher = LeannSearcher(index_path)+ results = searcher.search("test subject", top_k=3)++ # Verify that scores are not -inf (which indicates norm file was loaded correctly)+ assert len(results) > 0+ assert all(result.score != float("-inf") for result in results), (+ "Scores should not be -inf when norm file is correct"+ )+++ os.environ.get("CI") == "true",+ reason="Skip performance comparison in CI - requires significant compute time",+)+def test_diskann_vs_hnsw_performance():+ """Compare DiskANN (with partition) vs HNSW performance."""+ import time++ from leann.api import LeannBuilder, LeannSearcher++ with tempfile.TemporaryDirectory() as temp_dir:+ # Test data+ texts = [+ f"Performance test document {i} covering topic {i % 20} in detail." for i in range(1000)+ ]+ query = "performance topic test"++ # Test DiskANN with partitioning+ diskann_path = str(Path(temp_dir) / "perf_diskann.leann")+ diskann_builder = LeannBuilder(+ backend_name="diskann",+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ is_recompute=True,+ )++ for text in texts:+ diskann_builder.add_text(text)++ start_time = time.time()+ diskann_builder.build_index(diskann_path)++ # Test HNSW+ hnsw_path = str(Path(temp_dir) / "perf_hnsw.leann")+ hnsw_builder = LeannBuilder(+ backend_name="hnsw",+ embedding_model="facebook/contriever",+ embedding_mode="sentence-transformers",+ is_recompute=True,+ )++ for text in texts:+ hnsw_builder.add_text(text)++ start_time = time.time()+ hnsw_builder.build_index(hnsw_path)++ # Compare search performance+ diskann_searcher = LeannSearcher(diskann_path)+ hnsw_searcher = LeannSearcher(hnsw_path)++ # Warm up searches+ diskann_searcher.search(query, top_k=5)+ hnsw_searcher.search(query, top_k=5)++ # Timed searches+ start_time = time.time()+ diskann_results = diskann_searcher.search(query, top_k=10)+ diskann_search_time = time.time() - start_time++ start_time = time.time()+ hnsw_results = hnsw_searcher.search(query, top_k=10)+ hnsw_search_time = time.time() - start_time++ # Basic assertions+ assert len(diskann_results) == 10+ assert len(hnsw_results) == 10+ assert all(r.score != float("-inf") for r in diskann_results)+ assert all(r.score != float("-inf") for r in hnsw_results)++ # Performance ratio (informational)+ if hnsw_search_time > 0:+ speed_ratio = hnsw_search_time / diskann_search_time+ print(f"DiskANN search time: {diskann_search_time:.4f}s")+ print(f"HNSW search time: {hnsw_search_time:.4f}s")+ print(f"DiskANN is {speed_ratio:.2f}x faster than HNSW")diff --git a/tests/test_document_rag.py b/tests/test_document_rag.pyindex 97c5700..ddd0bfd 100644--- a/tests/test_document_rag.py+++ b/tests/test_document_rag.py@@ -58,6 +58,9 @@ def test_document_rag_simulated(test_data_dir):@pytest.mark.skipif(not os.environ.get("OPENAI_API_KEY"), reason="OpenAI API key not available")+ os.environ.get("CI") == "true", reason="Skip OpenAI tests in CI to avoid API costs"+)def test_document_rag_openai(test_data_dir):"""Test document_rag with OpenAI embeddings."""with tempfile.TemporaryDirectory() as temp_dir:diff --git a/tests/test_readme_examples.py b/tests/test_readme_examples.pyindex a05e98a..a562a02 100644--- a/tests/test_readme_examples.py+++ b/tests/test_readme_examples.py@@ -10,29 +10,33 @@ from pathlib import Pathimport pytest-def test_readme_basic_example():- """Test the basic example from README.md."""[email protected]("backend_name", ["hnsw", "diskann"])+def test_readme_basic_example(backend_name):+ """Test the basic example from README.md with both backends."""# Skip on macOS CI due to MPS environment issues with all-MiniLM-L6-v2if os.environ.get("CI") == "true" and platform.system() == "Darwin":pytest.skip("Skipping on macOS CI due to MPS environment issues with all-MiniLM-L6-v2")+ # Skip DiskANN on CI (Linux runners) due to C++ extension memory/hardware constraints+ if os.environ.get("CI") == "true" and backend_name == "diskann":+ pytest.skip("Skip DiskANN tests in CI due to resource constraints and instability")# This is the exact code from README (with smaller model for CI)from leann import LeannBuilder, LeannChat, LeannSearcherfrom leann.api import SearchResultwith tempfile.TemporaryDirectory() as temp_dir:- INDEX_PATH = str(Path(temp_dir) / "demo.leann")+ INDEX_PATH = str(Path(temp_dir) / f"demo_{backend_name}.leann")# Build an index# In CI, use a smaller model to avoid memory issuesif os.environ.get("CI") == "true":builder = LeannBuilder(- backend_name="hnsw",+ backend_name=backend_name,embedding_model="sentence-transformers/all-MiniLM-L6-v2", # Smaller modeldimensions=384, # Smaller dimensions)else:- builder = LeannBuilder(backend_name="hnsw")+ builder = LeannBuilder(backend_name=backend_name)builder.add_text("LEANN saves 97% storage compared to traditional vector databases.")builder.add_text("Tung Tung Tung Sahur called—they need their banana-crocodile hybrid back")builder.build_index(INDEX_PATH)@@ -52,9 +56,15 @@ def test_readme_basic_example():# Verify search resultsassert len(results) > 0assert isinstance(results[0], SearchResult)+ assert results[0].score != float("-inf"), (+ f"should return valid scores, got {results[0].score}"+ )# The second text about banana-crocodile should be more relevantassert "banana" in results[0].text or "crocodile" in results[0].text+ # Ensure we cleanup background embedding server+ searcher.cleanup()+# Chat with your data (using simulated LLM to avoid external dependencies)chat = LeannChat(INDEX_PATH, llm_config={"type": "simulated"})response = chat.ask("How much storage does LEANN save?", top_k=1)@@ -62,6 +72,8 @@ def test_readme_basic_example():# Verify chat worksassert isinstance(response, str)assert len(response) > 0+ # Cleanup chat resources+ chat.cleanup()def test_readme_imports():@@ -110,26 +122,31 @@ def test_backend_options():assert len(list(Path(diskann_path).parent.glob(f"{Path(diskann_path).stem}.*"))) > 0-def test_llm_config_simulated():- """Test simulated LLM configuration option."""[email protected]("backend_name", ["hnsw", "diskann"])+def test_llm_config_simulated(backend_name):+ """Test simulated LLM configuration option with both backends."""# Skip on macOS CI due to MPS environment issues with all-MiniLM-L6-v2if os.environ.get("CI") == "true" and platform.system() == "Darwin":pytest.skip("Skipping on macOS CI due to MPS environment issues with all-MiniLM-L6-v2")+ # Skip DiskANN tests in CI due to hardware requirements+ if os.environ.get("CI") == "true" and backend_name == "diskann":+ pytest.skip("Skip DiskANN tests in CI - requires specific hardware and large memory")+from leann import LeannBuilder, LeannChatwith tempfile.TemporaryDirectory() as temp_dir:# Build a simple index- index_path = str(Path(temp_dir) / "test.leann")+ index_path = str(Path(temp_dir) / f"test_{backend_name}.leann")# Use smaller model in CI to avoid memory issuesif os.environ.get("CI") == "true":builder = LeannBuilder(- backend_name="hnsw",+ backend_name=backend_name,embedding_model="sentence-transformers/all-MiniLM-L6-v2",dimensions=384,)else:- builder = LeannBuilder(backend_name="hnsw")+ builder = LeannBuilder(backend_name=backend_name)builder.add_text("Test document for LLM testing")builder.build_index(index_path)diff --git a/uv.lock b/uv.lockindex fcc82fa..5b990e7 100644--- a/uv.lock+++ b/uv.lock@@ -2223,7 +2223,7 @@ wheels = [[[package]]name = "leann-backend-diskann"-version = "0.2.6"+version = "0.2.8"source = { editable = "packages/leann-backend-diskann" }dependencies = [{ name = "leann-core" },@@ -2235,14 +2235,14 @@ dependencies = [[package.metadata]requires-dist = [- { name = "leann-core", specifier = "==0.2.6" },+ { name = "leann-core", specifier = "==0.2.8" },{ name = "numpy" },{ name = "protobuf", specifier = ">=3.19.0" },][[package]]name = "leann-backend-hnsw"-version = "0.2.6"+version = "0.2.8"source = { editable = "packages/leann-backend-hnsw" }dependencies = [{ name = "leann-core" },@@ -2255,7 +2255,7 @@ dependencies = [[package.metadata]requires-dist = [- { name = "leann-core", specifier = "==0.2.6" },+ { name = "leann-core", specifier = "==0.2.8" },{ name = "msgpack", specifier = ">=1.0.0" },{ name = "numpy" },{ name = "pyzmq", specifier = ">=23.0.0" },@@ -2263,7 +2263,7 @@ requires-dist = [[[package]]name = "leann-core"-version = "0.2.6"+version = "0.2.8"source = { editable = "packages/leann-core" }dependencies = [{ name = "accelerate" },@@ -2272,8 +2272,8 @@ dependencies = [{ name = "llama-index-core" },{ name = "llama-index-embeddings-huggingface" },{ name = "llama-index-readers-file" },- { name = "mlx", marker = "sys_platform == 'darwin'" },- { name = "mlx-lm", marker = "sys_platform == 'darwin'" },+ { name = "mlx", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'" },+ { name = "mlx-lm", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'" },{ name = "msgpack" },{ name = "nbconvert" },{ name = "numpy", version = "2.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },@@ -2302,8 +2302,8 @@ requires-dist = [{ name = "llama-index-core", specifier = ">=0.12.0" },{ name = "llama-index-embeddings-huggingface", specifier = ">=0.5.5" },{ name = "llama-index-readers-file", specifier = ">=0.4.0" },- { name = "mlx", marker = "sys_platform == 'darwin'", specifier = ">=0.26.3" },- { name = "mlx-lm", marker = "sys_platform == 'darwin'", specifier = ">=0.26.0" },+ { name = "mlx", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'", specifier = ">=0.26.3" },+ { name = "mlx-lm", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'", specifier = ">=0.26.0" },{ name = "msgpack", specifier = ">=1.0.0" },{ name = "nbconvert", specifier = ">=7.0.0" },{ name = "numpy", specifier = ">=1.20.0" },@@ -2343,8 +2343,8 @@ dependencies = [{ name = "llama-index-embeddings-huggingface" },{ name = "llama-index-readers-file" },{ name = "llama-index-vector-stores-faiss" },- { name = "mlx", marker = "sys_platform == 'darwin'" },- { name = "mlx-lm", marker = "sys_platform == 'darwin'" },+ { name = "mlx", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'" },+ { name = "mlx-lm", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'" },{ name = "msgpack" },{ name = "nbconvert" },{ name = "numpy", version = "2.0.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" },@@ -2356,6 +2356,7 @@ dependencies = [{ name = "pdfplumber" },{ name = "protobuf" },{ name = "psutil" },+ { name = "pybind11" },{ name = "pymupdf" },{ name = "pypdf2" },{ name = "pypdfium2" },@@ -2424,8 +2425,8 @@ requires-dist = [{ name = "llama-index-readers-file", marker = "extra == 'test'", specifier = ">=0.4.0" },{ name = "llama-index-vector-stores-faiss", specifier = ">=0.4.0" },{ name = "matplotlib", marker = "extra == 'dev'" },- { name = "mlx", marker = "sys_platform == 'darwin'", specifier = ">=0.26.3" },- { name = "mlx-lm", marker = "sys_platform == 'darwin'", specifier = ">=0.26.0" },+ { name = "mlx", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'", specifier = ">=0.26.3" },+ { name = "mlx-lm", marker = "platform_machine == 'arm64' and sys_platform == 'darwin'", specifier = ">=0.26.0" },{ name = "msgpack", specifier = ">=1.1.1" },{ name = "nbconvert", specifier = ">=7.16.6" },{ name = "numpy", specifier = ">=1.26.0" },@@ -2438,6 +2439,7 @@ requires-dist = [{ name = "pre-commit", marker = "extra == 'dev'", specifier = ">=3.5.0" },{ name = "protobuf", specifier = "==4.25.3" },{ name = "psutil", specifier = ">=5.8.0" },+ { name = "pybind11", specifier = ">=3.0.0" },{ name = "pymupdf", specifier = ">=1.26.0" },{ name = "pypdf2", specifier = ">=3.0.0" },{ name = "pypdfium2", specifier = ">=4.30.0" },@@ -2449,7 +2451,7 @@ requires-dist = [{ name = "python-docx", marker = "extra == 'documents'", specifier = ">=0.8.11" },{ name = "python-dotenv", marker = "extra == 'test'", specifier = ">=1.0.0" },{ name = "requests", specifier = ">=2.25.0" },- { name = "ruff", marker = "extra == 'dev'", specifier = ">=0.1.0" },+ { name = "ruff", marker = "extra == 'dev'", specifier = "==0.12.7" },{ name = "sentence-transformers", specifier = ">=2.2.0" },{ name = "sentence-transformers", marker = "extra == 'test'", specifier = ">=2.2.0" },{ name = "sglang" },@@ -4358,6 +4360,15 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/10/15/6b30e77872012bbfe8265d42a01d5b3c17ef0ac0f2fae531ad91b6a6c02e/pyarrow-21.0.0-cp39-cp39-win_amd64.whl", hash = "sha256:cdc4c17afda4dab2a9c0b79148a43a7f4e1094916b3e18d8975bfd6d6d52241f", size = 26227521 },]+[[package]]+name = "pybind11"+version = "3.0.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/ef/83/698d120e257a116f2472c710932023ad779409adf2734d2e940f34eea2c5/pybind11-3.0.0.tar.gz", hash = "sha256:c3f07bce3ada51c3e4b76badfa85df11688d12c46111f9d242bc5c9415af7862", size = 544819 }+wheels = [+ { url = "https://files.pythonhosted.org/packages/41/9c/85f50a5476832c3efc67b6d7997808388236ae4754bf53e1749b3bc27577/pybind11-3.0.0-py3-none-any.whl", hash = "sha256:7c5cac504da5a701b5163f0e6a7ba736c713a096a5378383c5b4b064b753f607", size = 292118 },+]+[[package]]name = "pycparser"version = "2.22"@@ -5204,27 +5215,27 @@ wheels = [[[package]]name = "ruff"-version = "0.12.5"-source = { registry = "https://pypi.org/simple" }-sdist = { url = "https://files.pythonhosted.org/packages/30/cd/01015eb5034605fd98d829c5839ec2c6b4582b479707f7c1c2af861e8258/ruff-0.12.5.tar.gz", hash = "sha256:b209db6102b66f13625940b7f8c7d0f18e20039bb7f6101fbdac935c9612057e", size = 5170722 }-wheels = [- { url = "https://files.pythonhosted.org/packages/d4/de/ad2f68f0798ff15dd8c0bcc2889558970d9a685b3249565a937cd820ad34/ruff-0.12.5-py3-none-linux_armv6l.whl", hash = "sha256:1de2c887e9dec6cb31fcb9948299de5b2db38144e66403b9660c9548a67abd92", size = 11819133 },- { url = "https://files.pythonhosted.org/packages/f8/fc/c6b65cd0e7fbe60f17e7ad619dca796aa49fbca34bb9bea5f8faf1ec2643/ruff-0.12.5-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:d1ab65e7d8152f519e7dea4de892317c9da7a108da1c56b6a3c1d5e7cf4c5e9a", size = 12501114 },- { url = "https://files.pythonhosted.org/packages/c5/de/c6bec1dce5ead9f9e6a946ea15e8d698c35f19edc508289d70a577921b30/ruff-0.12.5-py3-none-macosx_11_0_arm64.whl", hash = "sha256:962775ed5b27c7aa3fdc0d8f4d4433deae7659ef99ea20f783d666e77338b8cf", size = 11716873 },- { url = "https://files.pythonhosted.org/packages/a1/16/cf372d2ebe91e4eb5b82a2275c3acfa879e0566a7ac94d331ea37b765ac8/ruff-0.12.5-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:73b4cae449597e7195a49eb1cdca89fd9fbb16140c7579899e87f4c85bf82f73", size = 11958829 },- { url = "https://files.pythonhosted.org/packages/25/bf/cd07e8f6a3a6ec746c62556b4c4b79eeb9b0328b362bb8431b7b8afd3856/ruff-0.12.5-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8b13489c3dc50de5e2d40110c0cce371e00186b880842e245186ca862bf9a1ac", size = 11626619 },- { url = "https://files.pythonhosted.org/packages/d8/c9/c2ccb3b8cbb5661ffda6925f81a13edbb786e623876141b04919d1128370/ruff-0.12.5-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f1504fea81461cf4841778b3ef0a078757602a3b3ea4b008feb1308cb3f23e08", size = 13221894 },- { url = "https://files.pythonhosted.org/packages/6b/58/68a5be2c8e5590ecdad922b2bcd5583af19ba648f7648f95c51c3c1eca81/ruff-0.12.5-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:c7da4129016ae26c32dfcbd5b671fe652b5ab7fc40095d80dcff78175e7eddd4", size = 14163909 },- { url = "https://files.pythonhosted.org/packages/bd/d1/ef6b19622009ba8386fdb792c0743f709cf917b0b2f1400589cbe4739a33/ruff-0.12.5-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ca972c80f7ebcfd8af75a0f18b17c42d9f1ef203d163669150453f50ca98ab7b", size = 13583652 },- { url = "https://files.pythonhosted.org/packages/62/e3/1c98c566fe6809a0c83751d825a03727f242cdbe0d142c9e292725585521/ruff-0.12.5-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8dbbf9f25dfb501f4237ae7501d6364b76a01341c6f1b2cd6764fe449124bb2a", size = 12700451 },- { url = "https://files.pythonhosted.org/packages/24/ff/96058f6506aac0fbc0d0fc0d60b0d0bd746240a0594657a2d94ad28033ba/ruff-0.12.5-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2c47dea6ae39421851685141ba9734767f960113d51e83fd7bb9958d5be8763a", size = 12937465 },- { url = "https://files.pythonhosted.org/packages/eb/d3/68bc5e7ab96c94b3589d1789f2dd6dd4b27b263310019529ac9be1e8f31b/ruff-0.12.5-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:c5076aa0e61e30f848846f0265c873c249d4b558105b221be1828f9f79903dc5", size = 11771136 },- { url = "https://files.pythonhosted.org/packages/52/75/7356af30a14584981cabfefcf6106dea98cec9a7af4acb5daaf4b114845f/ruff-0.12.5-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:a5a4c7830dadd3d8c39b1cc85386e2c1e62344f20766be6f173c22fb5f72f293", size = 11601644 },- { url = "https://files.pythonhosted.org/packages/c2/67/91c71d27205871737cae11025ee2b098f512104e26ffd8656fd93d0ada0a/ruff-0.12.5-py3-none-musllinux_1_2_i686.whl", hash = "sha256:46699f73c2b5b137b9dc0fc1a190b43e35b008b398c6066ea1350cce6326adcb", size = 12478068 },- { url = "https://files.pythonhosted.org/packages/34/04/b6b00383cf2f48e8e78e14eb258942fdf2a9bf0287fbf5cdd398b749193a/ruff-0.12.5-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:5a655a0a0d396f0f072faafc18ebd59adde8ca85fb848dc1b0d9f024b9c4d3bb", size = 12991537 },- { url = "https://files.pythonhosted.org/packages/3e/b9/053d6445dc7544fb6594785056d8ece61daae7214859ada4a152ad56b6e0/ruff-0.12.5-py3-none-win32.whl", hash = "sha256:dfeb2627c459b0b78ca2bbdc38dd11cc9a0a88bf91db982058b26ce41714ffa9", size = 11751575 },- { url = "https://files.pythonhosted.org/packages/bc/0f/ab16e8259493137598b9149734fec2e06fdeda9837e6f634f5c4e35916da/ruff-0.12.5-py3-none-win_amd64.whl", hash = "sha256:ae0d90cf5f49466c954991b9d8b953bd093c32c27608e409ae3564c63c5306a5", size = 12882273 },- { url = "https://files.pythonhosted.org/packages/00/db/c376b0661c24cf770cb8815268190668ec1330eba8374a126ceef8c72d55/ruff-0.12.5-py3-none-win_arm64.whl", hash = "sha256:48cdbfc633de2c5c37d9f090ba3b352d1576b0015bfc3bc98eaf230275b7e805", size = 11951564 },+version = "0.12.7"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/a1/81/0bd3594fa0f690466e41bd033bdcdf86cba8288345ac77ad4afbe5ec743a/ruff-0.12.7.tar.gz", hash = "sha256:1fc3193f238bc2d7968772c82831a4ff69252f673be371fb49663f0068b7ec71", size = 5197814 }+wheels = [+ { url = "https://files.pythonhosted.org/packages/e1/d2/6cb35e9c85e7a91e8d22ab32ae07ac39cc34a71f1009a6f9e4a2a019e602/ruff-0.12.7-py3-none-linux_armv6l.whl", hash = "sha256:76e4f31529899b8c434c3c1dede98c4483b89590e15fb49f2d46183801565303", size = 11852189 },+ { url = "https://files.pythonhosted.org/packages/63/5b/a4136b9921aa84638f1a6be7fb086f8cad0fde538ba76bda3682f2599a2f/ruff-0.12.7-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:789b7a03e72507c54fb3ba6209e4bb36517b90f1a3569ea17084e3fd295500fb", size = 12519389 },+ { url = "https://files.pythonhosted.org/packages/a8/c9/3e24a8472484269b6b1821794141f879c54645a111ded4b6f58f9ab0705f/ruff-0.12.7-py3-none-macosx_11_0_arm64.whl", hash = "sha256:2e1c2a3b8626339bb6369116e7030a4cf194ea48f49b64bb505732a7fce4f4e3", size = 11743384 },+ { url = "https://files.pythonhosted.org/packages/26/7c/458dd25deeb3452c43eaee853c0b17a1e84169f8021a26d500ead77964fd/ruff-0.12.7-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:32dec41817623d388e645612ec70d5757a6d9c035f3744a52c7b195a57e03860", size = 11943759 },+ { url = "https://files.pythonhosted.org/packages/7f/8b/658798472ef260ca050e400ab96ef7e85c366c39cf3dfbef4d0a46a528b6/ruff-0.12.7-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:47ef751f722053a5df5fa48d412dbb54d41ab9b17875c6840a58ec63ff0c247c", size = 11654028 },+ { url = "https://files.pythonhosted.org/packages/a8/86/9c2336f13b2a3326d06d39178fd3448dcc7025f82514d1b15816fe42bfe8/ruff-0.12.7-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a828a5fc25a3efd3e1ff7b241fd392686c9386f20e5ac90aa9234a5faa12c423", size = 13225209 },+ { url = "https://files.pythonhosted.org/packages/76/69/df73f65f53d6c463b19b6b312fd2391dc36425d926ec237a7ed028a90fc1/ruff-0.12.7-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:5726f59b171111fa6a69d82aef48f00b56598b03a22f0f4170664ff4d8298efb", size = 14182353 },+ { url = "https://files.pythonhosted.org/packages/58/1e/de6cda406d99fea84b66811c189b5ea139814b98125b052424b55d28a41c/ruff-0.12.7-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:74e6f5c04c4dd4aba223f4fe6e7104f79e0eebf7d307e4f9b18c18362124bccd", size = 13631555 },+ { url = "https://files.pythonhosted.org/packages/6f/ae/625d46d5164a6cc9261945a5e89df24457dc8262539ace3ac36c40f0b51e/ruff-0.12.7-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5d0bfe4e77fba61bf2ccadf8cf005d6133e3ce08793bbe870dd1c734f2699a3e", size = 12667556 },+ { url = "https://files.pythonhosted.org/packages/55/bf/9cb1ea5e3066779e42ade8d0cd3d3b0582a5720a814ae1586f85014656b6/ruff-0.12.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:06bfb01e1623bf7f59ea749a841da56f8f653d641bfd046edee32ede7ff6c606", size = 12939784 },+ { url = "https://files.pythonhosted.org/packages/55/7f/7ead2663be5627c04be83754c4f3096603bf5e99ed856c7cd29618c691bd/ruff-0.12.7-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:e41df94a957d50083fd09b916d6e89e497246698c3f3d5c681c8b3e7b9bb4ac8", size = 11771356 },+ { url = "https://files.pythonhosted.org/packages/17/40/a95352ea16edf78cd3a938085dccc55df692a4d8ba1b3af7accbe2c806b0/ruff-0.12.7-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:4000623300563c709458d0ce170c3d0d788c23a058912f28bbadc6f905d67afa", size = 11612124 },+ { url = "https://files.pythonhosted.org/packages/4d/74/633b04871c669e23b8917877e812376827c06df866e1677f15abfadc95cb/ruff-0.12.7-py3-none-musllinux_1_2_i686.whl", hash = "sha256:69ffe0e5f9b2cf2b8e289a3f8945b402a1b19eff24ec389f45f23c42a3dd6fb5", size = 12479945 },+ { url = "https://files.pythonhosted.org/packages/be/34/c3ef2d7799c9778b835a76189c6f53c179d3bdebc8c65288c29032e03613/ruff-0.12.7-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:a07a5c8ffa2611a52732bdc67bf88e243abd84fe2d7f6daef3826b59abbfeda4", size = 12998677 },+ { url = "https://files.pythonhosted.org/packages/77/ab/aca2e756ad7b09b3d662a41773f3edcbd262872a4fc81f920dc1ffa44541/ruff-0.12.7-py3-none-win32.whl", hash = "sha256:c928f1b2ec59fb77dfdf70e0419408898b63998789cc98197e15f560b9e77f77", size = 11756687 },+ { url = "https://files.pythonhosted.org/packages/b4/71/26d45a5042bc71db22ddd8252ca9d01e9ca454f230e2996bb04f16d72799/ruff-0.12.7-py3-none-win_amd64.whl", hash = "sha256:9c18f3d707ee9edf89da76131956aba1270c6348bfee8f6c647de841eac7194f", size = 12912365 },+ { url = "https://files.pythonhosted.org/packages/4c/9b/0b8aa09817b63e78d94b4977f18b1fcaead3165a5ee49251c5d5c245bb2d/ruff-0.12.7-py3-none-win_arm64.whl", hash = "sha256:dfce05101dbd11833a0776716d5d1578641b7fddb537fe7fa956ab85d1769b69", size = 11982083 },][[package]]