Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

agent: dsh lists a vendor's reply limit at most a quarter of the window, so it has room to compact (#2823, LoKi on Discord)

yetone822f2b30427a
diff --git a/docs/subsystems/agent-wiring.md b/docs/subsystems/agent-wiring.md
index e1fd5f48e..ce7f4e420 100644
--- a/docs/subsystems/agent-wiring.md
+++ b/docs/subsystems/agent-wiring.md
@@ -58,6 +58,7 @@ something else shows up as drift.
- The YAML ones go through `yamlKept` and `mergeKept` in [`sync.go`](../../internal/agent/sync.go). Each is tested by a `Test<Agent>KeepsTheUsersSettings`.
- Copilot in JetBrains IDEs keeps its Bring Your Own Key providers in the Copilot language server's `byok.json`, shared with Xcode's and Eclipse's Copilot (`copilotJetBrains` in [`copilotjetbrains.go`](../../internal/agent/copilotjetbrains.go)). magpie writes its own three keys there, `magpie-provider-config`, `magpie-api-key` and `magpie-models-config`, in the bytes the server itself writes, and only when they differ; the user's providers and keys stay. A magpie model the user hid in Manage Models (`isRegistered: false`) stays hidden through a sync. A new file is made 0600 in a 0700 folder, as the server makes it.
- dsh (DeepSeek Harness) sets one API per llm-pi-ai route, so magpie writes a route per protocol (`dshKinds` in [`dsh.go`](../../internal/agent/dsh.go); 01huadalang on Discord): `magpie` (Chat Completions) for most models, `magpie-responses` (`openai-responses`) for a model served on Responses, `magpie-anthropic` (`anthropic-messages`, the gateway root, `forceAdaptiveThinking` where `AdaptiveThinking`) for one served on Messages, by the model's `APIs` (`dshRouteOf`). A model's `APIs` is the one its provider serves it on best (`Provider.Native`, in `magpieModels`, [`route.go`](../../internal/agent/route.go)), and Pi, omp and droid read it the same way. A routing group has `responses` when every member is best on Responses (`allResponses`, #1594), so its tool-call arguments are relayed as they stream; any other group has none and stays on Chat, a Messages-only group too, as a Claude member's thinking is known by its own model. A route with no models is taken out. An api the user picked for a route in dsh's Models page is kept, and so are keys they added there. The start (`agent-default-model`) names its model's route, and Sync moves a start on one of magpie's routes to the route its model is now on (`dshStartOnItsRoute`), its effort and marks kept. Check asks the route the start names to list its model. A dsh session resumed on a model that has since moved routes asks for the model to be picked again, since dsh finds a model by route and id.
+- dsh takes a model's listed `maxTokens` as every request's output and reserves it, with 65536 of headroom, out of `contextWindow` before deciding when to compact (compaction-basic's `resolveCompactSpec`: the smaller of 80% of the window and window − maxTokens − 65536). A vendor's reply limit near its window had dsh compact at 41248 of a 500000 group (#2823) or fail with no budget (524288/524288, LoKi on Discord; 131072/65536), so magpie lists it held to a quarter of the window on every route and in the older `llm-deepseek` entry (`dshMaxTokens`), as VS Code and Copilot in JetBrains do. A Max output the user set stays as they set it (`outputOr`), even one that leaves dsh no budget. `TestDshOutputLeavesCompactionRoom`.
- DeepSeek Harness Desktop (macOS arm64 and Windows x64; no Linux build) reads the same `$DSH_HOME` or `~/.dsh` as the CLI, as its own profile `profiles/desktop` (`resolveDshHome` and `resolveDesktopPaths` in the app's `lib/main.js`), so it takes magpie's routes with no wiring of its own (#2279, liangjinyuan-sihan). It is found by `/Applications/DeepSeek Harness.app`, by `%LOCALAPPDATA%\Programs\DeepSeek Harness`, or by the folder its current-user installer records under `HKCU\Software\1bf39983-50d0-5fe0-9ef4-cece76f67c5e` (`InstallLocation`), whose Uninstall key gives the version (`registered` in [`desktopapp.go`](../../internal/agent/desktopapp.go)), so a user who picked another folder is found. With the app installed but never opened there is no `~/.dsh`: the row is still listed, and connecting writes `profiles/desktop/cordis.patch.yml` from dsh's own template (`dshProfileTemplate`) with magpie's routes, which the app keeps on its first start, since it writes only the files that are missing. The notice says the app starts on it the first time it opens (`noticeDsh5`). Disconnect before that first start removes the profile magpie made when nothing else is in it (`dshTidyUnopened`); once the app has opened it, the profile stays and only magpie's entries go (`TestDshDesktopNeverOpened`, `TestDshDesktopRegisteredFolder`).
- Ante's `catalog.json` keeps user-added fields in `providers.magpie` the same way (`ante` in [`ante.go`](../../internal/agent/ante.go)); its `auth`, `http_headers`, `extra_body` and `stream_idle_timeout_secs` are what a gateway shared on the local network or reached from WSL needs to send a key. Ante starts a session on the `provider` and `model` of `settings.json`, not on what its catalog holds (#1210), so magpie sets that pair and puts back what it found there; a pick the user makes in Ante while magpie is wired is theirs and stands.
- AstrBot keeps its providers in `data/cmd_config.json` under its root (`$ASTRBOT_ROOT`, else `~/.astrbot`, the desktop app's, else `~/astrbot`), written with a UTF-8 BOM, which magpie keeps (`astrbot` in [`astrbot.go`](../../internal/agent/astrbot.go)). magpie adds a `magpie` entry to `provider_sources` and one `magpie/<model>` entry per model to `provider`, keeps the user's own entries byte for byte and keeps what the user changes in magpie's (timeout, proxy, enable), and sets the default chat model (`agent_runner.config.model.provider_id`, or `provider_settings.default_provider_id` before the agent runner), putting back what it found there. A pick the user makes while magpie is wired is theirs and stands. The first connect copies the file to `cmd_config.json.before-magpie`. AstrBot reads the file only at start-up and its WebUI writes it back whole on save, so the agent's notice says to restart AstrBot before saving in the WebUI. A Docker install's root is inside its container, which magpie doesn't reach.
diff --git a/internal/agent/dsh.go b/internal/agent/dsh.go
index be8b7223b..3a6bce4f3 100644
--- a/internal/agent/dsh.go
+++ b/internal/agent/dsh.go
@@ -1500,6 +1500,23 @@ func dshLevels(ref string) []string {
return nil
}
+// dshMaxTokens is the maxTokens magpie lists m with. dsh takes a listed
+// maxTokens as every request's output and reserves it, with 65536 of
+// headroom, out of contextWindow before it decides when to compact
+// (compaction-basic's resolveCompactSpec: the threshold is the smaller of
+// 80% of the window and window - maxTokens - 65536). A vendor's reply limit
+// near its window leaves dsh compacting at a sliver of it, or failing to
+// (#2823): 500000 with 393216 compacted at 41248; 524288 with 524288, and
+// 131072 with 65536, left it no budget at all. So the vendor's figure is
+// held to a quarter of the window, as VS Code's and Copilot's are, while a
+// Max output the user set stays as they set it (outputOr).
+func dshMaxTokens(m catalog.Model) int {
+ if m.Context <= 0 {
+ return maxTokens(m)
+ }
+ return outputOr(m, min(maxTokens(m), max(1, m.Context/4)))
+}
+
// dshRouteConfig is magpie's first route with every one of models on it.
func dshRouteConfig(models []catalog.Model, api, gw string) dshPiRoute {
return dshKindConfig(dshKinds[0], models, api, gw)
@@ -1518,7 +1535,7 @@ func dshKindConfig(k dshKind, models []catalog.Model, api, gw string) dshPiRoute
for _, m := range models {
e := dshPiModel{ID: m.ID, Name: m.Name, ContextWindow: m.Context}
if m.Output > 0 {
- e.MaxTokens = maxTokens(m)
+ e.MaxTokens = dshMaxTokens(m)
}
switch {
case m.Images:
@@ -1604,7 +1621,7 @@ func dshProviderLines(effort string, models []catalog.Model, gw string) []string
lines = append(lines, fmt.Sprintf(" contextWindow: %d", m.Context))
}
if m.Output > 0 {
- lines = append(lines, fmt.Sprintf(" maxTokens: %d", maxTokens(m)))
+ lines = append(lines, fmt.Sprintf(" maxTokens: %d", dshMaxTokens(m)))
}
if m.Images {
lines = append(lines, " inputModalities: [text, image]")
diff --git a/internal/agent/dsh_compaction_budget_test.go b/internal/agent/dsh_compaction_budget_test.go
new file mode 100644
index 000000000..0531722bc
--- /dev/null
+++ b/internal/agent/dsh_compaction_budget_test.go
@@ -0,0 +1,66 @@
+package agent
+
+import (
+ "fmt"
+ "strings"
+ "testing"
+
+ "github.com/yetone/magpie/internal/catalog"
+)
+
+// dshCompactsAt is where dsh's compaction-basic starts compacting a model
+// listed with window and maxTokens out, as its resolveCompactSpec works it
+// out with the default policy (thresholdRatio 0.8, headroomTokens 65536):
+// 0 when it has no budget and fails instead (#2823).
+func dshCompactsAt(window, out int) int {
+ pressure := window - out - 65536
+ if window-out <= 0 || pressure <= 0 {
+ return 0
+ }
+ return min(window*8/10, pressure)
+}
+
+// TestDshOutputLeavesCompactionRoom: dsh reserves the maxTokens magpie lists
+// a model with, and 65536 more, out of its window before it compacts. A
+// vendor's reply limit near the window had dsh compact at 41248 of a 500000
+// group (#2823) or fail outright on 524288/524288 (LoKi on Discord) and
+// 131072/65536; it is held to a quarter of the window on every route and in
+// the older llm-deepseek entry, while a Max output the user set stays.
+func TestDshOutputLeavesCompactionRoom(t *testing.T) {
+ for _, tc := range []struct {
+ name string
+ model catalog.Model
+ want int
+ }{
+ {"full window", catalog.Model{ID: "space-bunny-free", Context: 524288, Output: 524288}, 131072},
+ {"above the window", catalog.Model{ID: "above", Context: 524288, Output: 1048576}, 131072},
+ {"group window under its members' output", catalog.Model{ID: "group/smart-split", Context: 500000, Output: 393216}, 125000},
+ {"half the window", catalog.Model{ID: "k131", Context: 131072, Output: 65536}, 32768},
+ {"within a quarter", catalog.Model{ID: "flash", Context: 1000000, Output: 128000}, 128000},
+ {"user output", catalog.Model{ID: "user", Context: 524288, Output: 393216, OutputSet: true}, 393216},
+ {"user full window", catalog.Model{ID: "user-full", Context: 524288, Output: 524288, OutputSet: true}, 524288},
+ {"unknown window", catalog.Model{ID: "unknown", Output: 128000}, 128000},
+ {"unknown output", catalog.Model{ID: "no-output", Context: 524288}, 0},
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ for _, k := range dshKinds {
+ got := dshKindConfig(k, []catalog.Model{tc.model}, "", "http://127.0.0.1:3425").Models[0]
+ if got.MaxTokens != tc.want || got.ContextWindow != tc.model.Context {
+ t.Fatalf("%s: contextWindow %d, maxTokens %d; want %d, %d", k.key, got.ContextWindow, got.MaxTokens, tc.model.Context, tc.want)
+ }
+ }
+ lines := strings.Join(dshProviderLines("high", []catalog.Model{tc.model}, "http://127.0.0.1:3425"), "\n")
+ if tc.want > 0 && !strings.Contains(lines+"\n", fmt.Sprintf("maxTokens: %d\n", tc.want)) {
+ t.Fatalf("llm-deepseek entry, want maxTokens: %d:\n%s", tc.want, lines)
+ }
+ if tc.want == 0 && strings.Contains(lines, "maxTokens") {
+ t.Fatalf("llm-deepseek entry lists a maxTokens for a model of no known output:\n%s", lines)
+ }
+ if tc.model.Context > 0 && !tc.model.OutputSet {
+ if at := dshCompactsAt(tc.model.Context, tc.want); at < tc.model.Context/4 {
+ t.Fatalf("dsh compacts %d/%d at %d tokens", tc.model.Context, tc.want, at)
+ }
+ }
+ })
+ }
+}
diff --git a/internal/agent/dsh_test.go b/internal/agent/dsh_test.go
index ee63e397b..1e309f473 100644
--- a/internal/agent/dsh_test.go
+++ b/internal/agent/dsh_test.go
@@ -197,7 +197,7 @@ func TestDshModelLimits(t *testing.T) {
want := ` - id: v/see
name: see · V
contextWindow: 400000
- maxTokens: 128000
+ maxTokens: 100000
input: [text, image]
- id: v/plain
name: plain · V
diff --git a/internal/agent/sync_test.go b/internal/agent/sync_test.go
index 5cbfe8d24..435eebcce 100644
--- a/internal/agent/sync_test.go
+++ b/internal/agent/sync_test.go
@@ -139,10 +139,11 @@ func TestCrushModelsCarryMaxTokens(t *testing.T) {
// output. ZCode and WorkBuddy cap it at zcodeMaxOutput besides, Crush falls
// back to 16384 without a known output, OpenCode is handed no limit without
// a window, and AtomCode and Snow CLI, each told a window of its own when
-// the model's isn't known, are cut to that.
+// the model's isn't known, are cut to that. dsh is held to a quarter of
+// a known window (dshMaxTokens, #2823).
func TestMaxTokensWithinContextWindow(t *testing.T) {
home := syncHome(t)
- check := func(limit string, want int) {
+ check := func(limit string, want, window int) {
t.Helper()
os.WriteFile(catalog.CachePath(), []byte(`{"zai":{"models":{"glm-4.6":{"id":"glm-4.6","name":"GLM-4.6","limit":{`+limit+`}}}}}`), 0o644)
catalog.Reset()
@@ -179,6 +180,12 @@ func TestMaxTokensWithinContextWindow(t *testing.T) {
if !strings.Contains(limit, "context") {
told = strconv.Itoa(min(want, atomcodeContext))
}
+ // dsh reserves maxTokens out of its window before compacting, so a
+ // known window holds it to a quarter (dshMaxTokens, #2823).
+ dshTold := n
+ if strings.Contains(limit, "context") {
+ dshTold = strconv.Itoa(min(want, window/4))
+ }
snowTold := n
if !strings.Contains(limit, "context") {
snowTold = strconv.Itoa(min(want, snowContext))
@@ -187,7 +194,7 @@ func TestMaxTokensWithinContextWindow(t *testing.T) {
"pi": {string(pi), `"maxTokens":` + n},
"cline": {string(cline), `"maxTokens":` + n},
"omp": {string(omp), "maxTokens: " + n},
- "dsh": {string(dshRoute), "maxTokens: " + n},
+ "dsh": {string(dshRoute), "maxTokens: " + dshTold},
"droid": {string(droid), `"maxOutputTokens":` + n},
"qoder": {string(qoder), `"maxOutputTokens":` + n},
"hanako": {string(hanako), `"maxOutput":` + n},
@@ -207,10 +214,10 @@ func TestMaxTokensWithinContextWindow(t *testing.T) {
}
}
}
- check(`"context":128000,"output":384000`, 128000)
- check(`"output":384000`, 384000)
- check(`"context":204800,"output":131072`, 131072)
- check(`"context":64000,"output":384000`, 64000)
+ check(`"context":128000,"output":384000`, 128000, 128000)
+ check(`"output":384000`, 384000, 0)
+ check(`"context":204800,"output":131072`, 131072, 204800)
+ check(`"context":64000,"output":384000`, 64000, 64000)
}
// A provider added after a magpie model was picked reaches the lists agents