Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

2.7 KB
Raw Blame

Choosing embeddings for your hardware

If you have never changed your embedding settings and indexing makes your computer slow or keeps CPU usage high, try a smaller embedding model. With a supported GPU and enough free VRAM, Ollama can also be a useful option.

Use the × button on a performance tip to hide these tips for your account across collection and embedding settings pages. To show them again, turn off Dismiss Embedding Performance Tips in Settings. This preference only hides the tips; it does not change your embedding configuration.

Use a smaller model on CPU

Open Library → Embedding Settings (/library/embedding-settings). Select Sentence Transformers and try all-MiniLM-L6-v2, a lightweight model available in the model dropdown. Use Test Embedding Model before indexing; settings save automatically.

Smaller models can trade retrieval quality and context length for lower resource use. This model accepts up to 256 tokens per chunk, so review your chunk size: longer chunks may be truncated. Try a small collection and check search results before rebuilding a large one.

Consider Ollama with GPU acceleration

Ollama supports local embedding models and can use supported GPUs. Check Ollama hardware support for your GPU and drivers. Leave enough free VRAM for the embedding model, its working memory, and any LLM running at the same time. Switching providers alone does not guarantee lower CPU usage or faster indexing.

  1. Start Ollama and download an embedding model, for example with ollama pull nomic-embed-text.
  2. In Embedding Settings, select Ollama, configure its server URL, and select the downloaded embedding model. The URL must be reachable from LDR; in Docker, localhost refers to the LDR container itself.
  3. Use Test Embedding Model, then try indexing a small collection.
  4. Check GPU usage while indexing to confirm that acceleration is active.

Use an embedding model rather than a chat model. See Ollama embeddings for background.

Apply changes to existing collections

Saved defaults apply to new collections. Existing collections retain their embedding settings until you choose Re-Index All Documents on the collection page to rebuild with the current settings. A full rebuild processes every document and can take time; keep it running until it completes. Cancelling before publication preserves the previous index and its settings, if any.

If indexing is already making your computer unresponsive, use Cancel Indexing on the collection page before changing settings and starting a rebuild.

See Library and RAG for more about indexing.