[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.
Python aifaissgpt-osslangchainllama-indexllmlocalstorageoffline-firstollamaprivacypythonragretrieval-augmented-generationvector-databasevector-searchvectors
fix(cli): run uvicorn on existing loop for leann serve (#296)
Aakash Suresh05899ea42574
diff --git a/packages/leann-backend-hnsw/CMakeLists.txt b/packages/leann-backend-hnsw/CMakeLists.txtindex b46596a..4e55890 100644--- a/packages/leann-backend-hnsw/CMakeLists.txt+++ b/packages/leann-backend-hnsw/CMakeLists.txt@@ -73,9 +73,17 @@ if(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64|arm64")message(STATUS "Configuring Faiss for ARM64 architecture")if(CMAKE_SYSTEM_NAME STREQUAL "Linux")- # Use SVE optimization level for ARM64 Linux (as seen in Faiss conda build)- set(FAISS_OPT_LEVEL "sve" CACHE STRING "" FORCE)- message(STATUS "Setting FAISS_OPT_LEVEL to 'sve' for ARM64 Linux")+ # Default to generic: many ARM64 Linux CI runners (and some consumer+ # devices) don't implement SVE, and Faiss built with SVE can SIGILL at+ # runtime. Opt in with `-DLEANN_FAISS_ARM64_SVE=ON`.+ option(LEANN_FAISS_ARM64_SVE "Enable SVE in Faiss on ARM64 Linux (requires SVE at runtime)" OFF)+ if(LEANN_FAISS_ARM64_SVE)+ set(FAISS_OPT_LEVEL "sve" CACHE STRING "" FORCE)+ message(STATUS "Setting FAISS_OPT_LEVEL to 'sve' for ARM64 Linux (LEANN_FAISS_ARM64_SVE=ON)")+ else()+ set(FAISS_OPT_LEVEL "generic" CACHE STRING "" FORCE)+ message(STATUS "Setting FAISS_OPT_LEVEL to 'generic' for ARM64 Linux")+ endif()else()# Use generic optimization for other ARM64 platforms (like macOS)set(FAISS_OPT_LEVEL "generic" CACHE STRING "" FORCE)diff --git a/packages/leann-core/src/leann/cli.py b/packages/leann-core/src/leann/cli.pyindex 919fcd3..29852b3 100644--- a/packages/leann-core/src/leann/cli.py+++ b/packages/leann-core/src/leann/cli.py@@ -3116,7 +3116,7 @@ Examples:import ostry:- from .server import main as server_main+ from .server import serve_async# Override host/port if provided via CLI argsif args.host:@@ -3124,9 +3124,9 @@ Examples:if args.port:os.environ["LEANN_SERVER_PORT"] = str(args.port)- # Run the server (this is blocking, so we don't await it)- # The server_main function handles uvicorn.run which blocks- server_main()+ # Must await uvicorn on this loop: sync uvicorn.run() starts a nested loop+ # and fails with "Cannot run the event loop while another loop is running".+ await serve_async()except ImportError as e:print("❌ HTTP server dependencies not installed.\n"diff --git a/packages/leann-core/src/leann/server.py b/packages/leann-core/src/leann/server.pyindex 6a76209..ecc07a3 100644--- a/packages/leann-core/src/leann/server.py+++ b/packages/leann-core/src/leann/server.py@@ -7,13 +7,14 @@ This exposes LEANN indexes over HTTP so clients can:The design intentionally keeps dependencies optional:- FastAPI + pydantic are imported lazily inside `create_app()`-- uvicorn is imported lazily inside `main()`+- uvicorn is imported lazily inside `serve_async()` (and `main()` calls that)This way, core LEANN usage is unaffected unless you actually run the server."""from __future__ import annotations+import asyncioimport osfrom pathlib import Pathfrom typing import Any@@ -167,14 +168,13 @@ def create_app():return app-def main() -> None:+async def serve_async() -> None:"""- Entrypoint to run the HTTP server with uvicorn.+ Run the HTTP server on the current asyncio event loop.- Example:- uv run python -m leann.server- # or, after wiring a CLI command:- leann serve+ Use this from async contexts (e.g. ``leann serve``, which runs under+ ``asyncio.run()``). Do not use ``uvicorn.run()`` there: it starts its own+ loop and raises "Cannot run the event loop while another loop is running"."""try:import uvicorn@@ -188,8 +188,21 @@ def main() -> None:app = create_app()host = os.getenv("LEANN_SERVER_HOST", "0.0.0.0")port = int(os.getenv("LEANN_SERVER_PORT", "8000"))+ config = uvicorn.Config(app, host=host, port=port)+ server = uvicorn.Server(config)+ await server.serve()+++def main() -> None:+ """+ Entrypoint to run the HTTP server with uvicorn.- uvicorn.run(app, host=host, port=port)+ Example:+ uv run python -m leann.server+ # or:+ leann serve+ """+ asyncio.run(serve_async())if __name__ == "__main__": # pragma: no cover