Skip to content

Personal AI, On Personal Devices

Python

docs: update engine recommendation docs to reflect implementation (#989)

Johnny C Watt50a08083e3c0
diff --git a/docs/architecture/design-principles.md b/docs/architecture/design-principles.md
index aa310a6e..54b09712 100644
--- a/docs/architecture/design-principles.md
+++ b/docs/architecture/design-principles.md
@@ -128,10 +128,11 @@ The `recommend_engine()` function maps hardware to engines:
| Hardware | Recommended Engine |
|----------|-------------------|
| No GPU | `llamacpp` (CPU-optimized) |
-| Apple Silicon | `ollama` (Metal acceleration) |
+| Apple Silicon | `mlx` (native MLX inference) |
| NVIDIA datacenter (A100, H100, etc.) | `vllm` (high throughput) |
| NVIDIA consumer | `ollama` (easy setup) |
-| AMD GPU | `vllm` (ROCm support) |
+| AMD datacenter (MI300, MI325, MI350, MI355) | `vllm` (high throughput) |
+| AMD consumer | `lemonade` (AMD GPU/NPU support) |
This recommendation is written to `config.toml` during `jarvis init` and used as the default engine:
diff --git a/docs/architecture/engine.md b/docs/architecture/engine.md
index ba4be3c1..fc9098b9 100644
--- a/docs/architecture/engine.md
+++ b/docs/architecture/engine.md
@@ -125,7 +125,7 @@ All providers produce the same output format consumed by agents:
### Ollama
-The Ollama backend communicates via Ollama's native HTTP API at `/api/chat` and `/api/tags`. It is the default engine on Apple Silicon and consumer NVIDIA GPUs.
+The Ollama backend communicates via Ollama's native HTTP API at `/api/chat` and `/api/tags`. It is the recommended engine for consumer NVIDIA GPUs.
- **Default host:** `http://localhost:11434`
- **Health check:** `GET /api/tags`
@@ -286,7 +286,7 @@ graph TD
A["detect_hardware()"] --> B{"GPU detected?"}
B -->|No| C["llamacpp"]
B -->|Yes| D{"GPU vendor?"}
- D -->|Apple| E["ollama"]
+ D -->|Apple| E["mlx"]
D -->|NVIDIA| F{"Datacenter card?<br/>(A100, H100, H200,<br/>L40, A10, A30)"}
F -->|Yes| G["vllm"]
F -->|No| H["ollama"]
diff --git a/docs/getting-started/configuration.md b/docs/getting-started/configuration.md
index 4c5f2b38..6655b436 100644
--- a/docs/getting-started/configuration.md
+++ b/docs/getting-started/configuration.md
@@ -114,7 +114,7 @@ host = "http://localhost:30000"
| Field | Type | Default | Description |
|-------|------|---------|-------------|
-| `default` | string | Auto-detected | Default engine backend. One of: `ollama`, `vllm`, `llamacpp`, `sglang`, `cloud`. Set automatically by `jarvis init` based on hardware detection. |
+| `default` | string | Auto-detected | Default engine backend. Registered keys include `ollama`, `vllm`, `sglang`, `llamacpp`, `mlx`, `lmstudio`, `exo`, `nexa`, `uzu`, `apple_fm`, `afm`, `lemonade`, `nim`, `cloud`, `litellm`, and `gemma_cpp`. Optional engines are available only when their dependencies are installed. |
**`[engine.ollama]`:**
@@ -704,20 +704,23 @@ graph TD
A[detect_hardware] --> B{GPU detected?}
B -->|No| C[llamacpp]
B -->|Yes| D{GPU vendor?}
- D -->|Apple| E[ollama]
+ D -->|Apple| E[mlx]
D -->|NVIDIA| F{Datacenter GPU?}
- D -->|AMD| G[vllm]
+ D -->|AMD| G{Datacenter GPU?}
F -->|Yes: A100, H100, H200, L40, A10, A30| H[vllm]
F -->|No: consumer GPU| I[ollama]
+ G -->|Yes: MI300, MI325, MI350, MI355| J[vllm]
+ G -->|No: consumer GPU| K[lemonade]
```
| Hardware | Recommended Engine | Reason |
|----------|--------------------|--------|
| No GPU | `llamacpp` | Efficient CPU inference with GGUF quantized models |
-| Apple Silicon | `ollama` | Native Metal acceleration, easy model management |
+| Apple Silicon | `mlx` | Native inference through the MLX framework |
| NVIDIA consumer GPU (RTX 3090, 4090, etc.) | `ollama` | Simple setup, good performance for single-user |
| NVIDIA datacenter GPU (A100, H100, H200, L40, A10, A30) | `vllm` | High-throughput batched serving, continuous batching |
-| AMD GPU | `vllm` | ROCm support via vLLM |
+| AMD consumer GPU | `lemonade` | Optimized support for AMD GPUs and Ryzen AI NPUs |
+| AMD datacenter GPU (MI300, MI325, MI350, MI355) | `vllm` | High-throughput serving on supported datacenter accelerators |
---
@@ -725,19 +728,25 @@ graph TD
### Apple Silicon Mac
+Start an MLX server with the same model ID used in the configuration:
+
+```bash
+jarvis host mlx-community/Qwen2.5-7B-4bit --backend mlx --port 8080
+```
+
```toml
# ~/.openjarvis/config.toml
# Apple Silicon MacBook Pro (M3 Max, 128 GB unified memory)
[engine]
-default = "ollama"
+default = "mlx"
-[engine.ollama]
-host = "http://localhost:11434"
+[engine.mlx]
+host = "http://localhost:8080"
[intelligence]
-default_model = "qwen3:8b"
-fallback_model = "llama3.2:3b"
+default_model = "mlx-community/Qwen2.5-7B-4bit"
+fallback_model = ""
temperature = 0.7
max_tokens = 1024