Personal AI, On Personal Devices
Python
docs: update engine recommendation docs to reflect implementation (#989)
Johnny C Watt50a08083e3c0
diff --git a/docs/architecture/design-principles.md b/docs/architecture/design-principles.mdindex aa310a6e..54b09712 100644--- a/docs/architecture/design-principles.md+++ b/docs/architecture/design-principles.md@@ -128,10 +128,11 @@ The `recommend_engine()` function maps hardware to engines:| Hardware | Recommended Engine ||----------|-------------------|| No GPU | `llamacpp` (CPU-optimized) |-| Apple Silicon | `ollama` (Metal acceleration) |+| Apple Silicon | `mlx` (native MLX inference) || NVIDIA datacenter (A100, H100, etc.) | `vllm` (high throughput) || NVIDIA consumer | `ollama` (easy setup) |-| AMD GPU | `vllm` (ROCm support) |+| AMD datacenter (MI300, MI325, MI350, MI355) | `vllm` (high throughput) |+| AMD consumer | `lemonade` (AMD GPU/NPU support) |This recommendation is written to `config.toml` during `jarvis init` and used as the default engine:diff --git a/docs/architecture/engine.md b/docs/architecture/engine.mdindex ba4be3c1..fc9098b9 100644--- a/docs/architecture/engine.md+++ b/docs/architecture/engine.md@@ -125,7 +125,7 @@ All providers produce the same output format consumed by agents:### Ollama-The Ollama backend communicates via Ollama's native HTTP API at `/api/chat` and `/api/tags`. It is the default engine on Apple Silicon and consumer NVIDIA GPUs.+The Ollama backend communicates via Ollama's native HTTP API at `/api/chat` and `/api/tags`. It is the recommended engine for consumer NVIDIA GPUs.- **Default host:** `http://localhost:11434`- **Health check:** `GET /api/tags`@@ -286,7 +286,7 @@ graph TDA["detect_hardware()"] --> B{"GPU detected?"}B -->|No| C["llamacpp"]B -->|Yes| D{"GPU vendor?"}- D -->|Apple| E["ollama"]+ D -->|Apple| E["mlx"]D -->|NVIDIA| F{"Datacenter card?<br/>(A100, H100, H200,<br/>L40, A10, A30)"}F -->|Yes| G["vllm"]F -->|No| H["ollama"]diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.mdindex 4c5f2b38..6655b436 100644--- a/docs/getting-started/configuration.md+++ b/docs/getting-started/configuration.md@@ -114,7 +114,7 @@ host = "http://localhost:30000"| Field | Type | Default | Description ||-------|------|---------|-------------|-| `default` | string | Auto-detected | Default engine backend. One of: `ollama`, `vllm`, `llamacpp`, `sglang`, `cloud`. Set automatically by `jarvis init` based on hardware detection. |+| `default` | string | Auto-detected | Default engine backend. Registered keys include `ollama`, `vllm`, `sglang`, `llamacpp`, `mlx`, `lmstudio`, `exo`, `nexa`, `uzu`, `apple_fm`, `afm`, `lemonade`, `nim`, `cloud`, `litellm`, and `gemma_cpp`. Optional engines are available only when their dependencies are installed. |**`[engine.ollama]`:**@@ -704,20 +704,23 @@ graph TDA[detect_hardware] --> B{GPU detected?}B -->|No| C[llamacpp]B -->|Yes| D{GPU vendor?}- D -->|Apple| E[ollama]+ D -->|Apple| E[mlx]D -->|NVIDIA| F{Datacenter GPU?}- D -->|AMD| G[vllm]+ D -->|AMD| G{Datacenter GPU?}F -->|Yes: A100, H100, H200, L40, A10, A30| H[vllm]F -->|No: consumer GPU| I[ollama]+ G -->|Yes: MI300, MI325, MI350, MI355| J[vllm]+ G -->|No: consumer GPU| K[lemonade]```| Hardware | Recommended Engine | Reason ||----------|--------------------|--------|| No GPU | `llamacpp` | Efficient CPU inference with GGUF quantized models |-| Apple Silicon | `ollama` | Native Metal acceleration, easy model management |+| Apple Silicon | `mlx` | Native inference through the MLX framework || NVIDIA consumer GPU (RTX 3090, 4090, etc.) | `ollama` | Simple setup, good performance for single-user || NVIDIA datacenter GPU (A100, H100, H200, L40, A10, A30) | `vllm` | High-throughput batched serving, continuous batching |-| AMD GPU | `vllm` | ROCm support via vLLM |+| AMD consumer GPU | `lemonade` | Optimized support for AMD GPUs and Ryzen AI NPUs |+| AMD datacenter GPU (MI300, MI325, MI350, MI355) | `vllm` | High-throughput serving on supported datacenter accelerators |---@@ -725,19 +728,25 @@ graph TD### Apple Silicon Mac+Start an MLX server with the same model ID used in the configuration:++```bash+jarvis host mlx-community/Qwen2.5-7B-4bit --backend mlx --port 8080+```+```toml# ~/.openjarvis/config.toml# Apple Silicon MacBook Pro (M3 Max, 128 GB unified memory)[engine]-default = "ollama"+default = "mlx"-[engine.ollama]-host = "http://localhost:11434"+[engine.mlx]+host = "http://localhost:8080"[intelligence]-default_model = "qwen3:8b"-fallback_model = "llama3.2:3b"+default_model = "mlx-community/Qwen2.5-7B-4bit"+fallback_model = ""temperature = 0.7max_tokens = 1024