[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.
Python aifaissgpt-osslangchainllama-indexllmlocalstorageoffline-firstollamaprivacypythonragretrieval-augmented-generationvector-databasevector-searchvectors
fix: force fp32 embeddings when the resolved device is CPU (#405)
Christoph Stadlbauer9cbe7c2a3b2b
diff --git a/packages/leann-core/src/leann/embedding_compute.py b/packages/leann-core/src/leann/embedding_compute.pyindex 6ff5d8b..c7125d7 100644--- a/packages/leann-core/src/leann/embedding_compute.py+++ b/packages/leann-core/src/leann/embedding_compute.py@@ -546,6 +546,12 @@ def compute_embeddings_sentence_transformers(else:device = "cpu"+ # CPU torch ships no fp16 kernels for several ops (LayerNorm raises+ # "LayerNormKernelImpl" not implemented for 'Half') - force fp32 on CPU.+ if device == "cpu" and use_fp16:+ use_fp16 = False+ logger.info("CPU device detected: disabling fp16 (no fp16 CPU kernels)")+# Apply optimizations based on benchmark resultsif adaptive_optimization:batch_size = _resolve_adaptive_batch_size(device, model_name)