ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.
Python ai-agentsamdcomfyuidockerllama-cppllmlocal-ain8nnvidiaopen-webuiragself-hostedspeech-to-textstrix-halotext-to-speechworkflow-automation
# ODS - Cloud / external LLM overlay
#
# Cloud mode keeps the ODS services running but does not launch a local
# llama-server container. Open WebUI, Hermes, Perplexica, and other clients
# route through LLM_API_URL / HERMES_LLM_BASE_URL instead.
services:
llama-server:
profiles:
- local-inference
restart: "no"
# Model Router only routes to a local runtime; cloud-only mode has no local
# model to switch, so gate it behind the same local-inference profile.
model-router:
profiles:
- local-inference
restart: "no"