Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
Offer Strands Decider in local compose on an NVIDIA GPU
agent-runner353a175d66ea6d8
diff --git a/deploy/local/docker-compose.yml b/deploy/local/docker-compose.ymlindex 98f693d5a..fab947943 100644--- a/deploy/local/docker-compose.yml+++ b/deploy/local/docker-compose.yml@@ -135,12 +135,20 @@ services:- "${SWITCH_BIND_ADDR:-127.0.0.1}:8025:8025"# ── Decision service (auto-route) ───────────────────────────────────────────- # `just decision-up`, or `--profile decision` (CPU, any machine) or- # `--profile decision-gpu` (Linux with an NVIDIA GPU and the NVIDIA Container- # Toolkit). Either starts the decision service on :8230 and a llama.cpp- # server for its model, with the weights downloaded once into the- # `decision-models` volume. switch-core runs on the host, so point it at the- # service in .env (see .env.example, "Decision routing"):+ # `just decision-up [cpu|gpu|strands]`, or one of these profiles. Each starts+ # the decision service on :8230 and a model server beside it:+ # decision Qwen3.5-4B on llama.cpp, on CPU (any machine). The+ # local default.+ # decision-gpu the same on an NVIDIA GPU.+ # decision-strands-gpu Strands Decider 4B, the production default (the+ # Helm chart's): more accurate, but it needs an NVIDIA+ # GPU with 24 GB, about 24 GiB of free RAM to load,+ # and the first start downloads about 17 GB.+ # The GPU profiles need Linux, an NVIDIA GPU and the NVIDIA Container+ # Toolkit; there is no GPU passthrough on macOS. Weights download once into+ # a volume (`decision-models`, or `decision-hf` for Strands). switch-core+ # runs on the host, so point it at the service in .env (see .env.example,+ # "Decision routing"):# DECISION_ENDPOINT_URL=http://localhost:8230# DECISION_TIMEOUT_SECONDS=30# On CPU each decision takes several seconds (far over the one-second target@@ -238,7 +246,35 @@ services:decision-model-download:condition: service_completed_successfully- decision:+ # Strands Decider 4B's model server (decision-strands-gpu). The checkpoint+ # and its Gemma 4 base download from Hugging Face into `decision-hf` on the+ # first start, then load; the port opens only once the model is loaded, so+ # allow several minutes (much longer on the first start). HF_TOKEN is+ # optional: for a gated checkpoint or higher Hub rate limits.+ decision-model-strands:+ build:+ context: decision/model-servers/strands+ image: switch-decision-strands:local+ profiles: ["decision-strands-gpu"]+ environment:+ HF_TOKEN: ${HF_TOKEN:-}+ volumes:+ - decision-hf:/models+ deploy:+ resources:+ reservations:+ devices:+ - driver: nvidia+ count: 1+ capabilities: [gpu]+ healthcheck:+ test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8400/health', timeout=3)"]+ interval: 10s+ timeout: 5s+ retries: 3+ start_period: 30m++ decision: &decisionbuild:context: decisiondockerfile: Dockerfile@@ -265,7 +301,21 @@ services:condition: service_healthyrequired: false+ # The decision service in front of Strands Decider: the same image, speaking+ # SystemOne to its model server.+ decision-strands:+ <<: *decision+ profiles: ["decision-strands-gpu"]+ environment:+ DECISION_BACKEND: systemone+ DECISION_MODEL: strands-decider-E4B+ SYSTEMONE_URL: http://decision-model-strands:8400/v1/systemone+ depends_on:+ decision-model-strands:+ condition: service_healthy+volumes:pgdata:mmdata:decision-models:+ decision-hf:diff --git a/justfile b/justfileindex 7f50e80bd..4048ab13d 100644--- a/justfile+++ b/justfile@@ -56,18 +56,26 @@ down:reset:docker compose -f deploy/local/docker-compose.yml --project-directory . down -v-# Start the decision service auto-route asks (with its model server), on CPU or,-# with `just decision-up gpu`, an NVIDIA GPU on Linux. Weights download once-# into a volume. docs/old/LOCAL_DEVELOPMENT.md, "Auto-route locally".+# Start the decision service auto-route asks (with its model server): Qwen on+# CPU by default, `gpu` for Qwen on an NVIDIA GPU, or `strands` for Strands+# Decider 4B (the production default; an NVIDIA GPU with 24 GB). GPU modes are+# Linux only. Weights download once into a volume.+# docs/old/LOCAL_DEVELOPMENT.md, "Auto-route locally".decision-up mode="cpu":#!/usr/bin/env bashset -euo pipefail+ service=decisioncase "{{mode}}" incpu) profile=decision ;;gpu) profile=decision-gpu ;;- *) echo "usage: just decision-up [cpu|gpu]" >&2; exit 2 ;;+ strands) profile=decision-strands-gpu service=decision-strands ;;+ *) echo "usage: just decision-up [cpu|gpu|strands]" >&2; exit 2 ;;esac- docker compose -f deploy/local/docker-compose.yml --project-directory . --profile "$profile" up -d --build --wait decision+ if [ "$profile" = decision-strands-gpu ]; then+ echo "Strands Decider: the first start downloads about 17 GB, then loads it;"+ echo "this waits until the model server is up (minutes)."+ fi+ docker compose -f deploy/local/docker-compose.yml --project-directory . --profile "$profile" up -d --build --wait "$service"port="${DECISION_PORT:-8230}"echo "Decision service: http://localhost:$port (curl http://localhost:$port/health)"if [ "$profile" = decision ]; then@@ -80,7 +88,7 @@ decision-up mode="cpu":echo " FEATURE_FLAGS_ENABLED=follow_the_conversation,decision_routing (plus any you had)"decision-down:- docker compose -f deploy/local/docker-compose.yml --project-directory . --profile decision --profile decision-gpu stop decision decision-model decision-model-gpu+ docker compose -f deploy/local/docker-compose.yml --project-directory . --profile decision --profile decision-gpu --profile decision-strands-gpu stop decision decision-model decision-model-gpu decision-strands decision-model-strands# ── Run switch-core locally ────────────────────────────────────────────────────# The Python project lives in core/; `--project core` selects that environment