Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

Give each llama.cpp slot room for a full decision prompt

agent-runner3536583463afc13
diff --git a/deploy/local/docker-compose.yml b/deploy/local/docker-compose.yml
index 42b007b4b..98f693d5a 100644
--- a/deploy/local/docker-compose.yml
+++ b/deploy/local/docker-compose.yml
@@ -182,7 +182,9 @@ services:
- --port
- "8300"
- -c
- - "8192"
+ - "16384"
+ - -np
+ - "4"
- --jinja
volumes:
- decision-models:/models:ro
@@ -209,7 +211,9 @@ services:
- -ngl
- "99"
- -c
- - "8192"
+ - "16384"
+ - -np
+ - "4"
- --jinja
# The decision service reaches whichever model server runs at one name.
networks:
diff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yaml
index 1c6bcac24..5eae1c238 100644
--- a/deploy/remote/helm/switch/values.yaml
+++ b/deploy/remote/helm/switch/values.yaml
@@ -930,7 +930,9 @@ decision:
- -ngl
- "99"
- -c
- - "8192"
+ - "16384"
+ - -np
+ - "4"
- --jinja
# The weights, downloaded into the PVC by an init container when the file
# is not there yet. Changing them downloads the new file on the next start.