Skip to content

Every agent's model. One place. Codex on DeepSeek, Claude Code on Kimi, from the menu bar.

dsh: maxTokens leaves dsh half its window, or a quarter for output where that is more (#3298 item 3)

yetoneca4a1cde0dca
diff --git a/docs/subsystems/agent-wiring.md b/docs/subsystems/agent-wiring.md
index aeff59d64..327c30af5 100644
--- a/docs/subsystems/agent-wiring.md
+++ b/docs/subsystems/agent-wiring.md
@@ -58,7 +58,7 @@ something else shows up as drift.
- The YAML ones go through `yamlKept` and `mergeKept` in [`sync.go`](../../internal/agent/sync.go). Each is tested by a `Test<Agent>KeepsTheUsersSettings`.
- Copilot in JetBrains IDEs keeps its Bring Your Own Key providers in the Copilot language server's `byok.json`, shared with Xcode's and Eclipse's Copilot (`copilotJetBrains` in [`copilotjetbrains.go`](../../internal/agent/copilotjetbrains.go)). magpie writes its own three keys there, `magpie-provider-config`, `magpie-api-key` and `magpie-models-config`, in the bytes the server itself writes, and only when they differ; the user's providers and keys stay. A magpie model the user hid in Manage Models (`isRegistered: false`) stays hidden through a sync. A new file is made 0600 in a 0700 folder, as the server makes it.
- dsh (DeepSeek Harness) sets one API per llm-pi-ai route, so magpie writes a route per protocol (`dshKinds` in [`dsh.go`](../../internal/agent/dsh.go); 01huadalang on Discord): `magpie` (Chat Completions) for most models, `magpie-responses` (`openai-responses`) for a model served on Responses, `magpie-anthropic` (`anthropic-messages`, the gateway root, `forceAdaptiveThinking` where `AdaptiveThinking`) for one served on Messages, by the model's `APIs` (`dshRouteOf`). A model's `APIs` is the one its provider serves it on best (`Provider.Native`, in `magpieModels`, [`route.go`](../../internal/agent/route.go)), and Pi, omp and droid read it the same way. A routing group has `responses` when every member is best on Responses (`allResponses`, #1594), so its tool-call arguments are relayed as they stream; any other group has none and stays on Chat, a Messages-only group too, as a Claude member's thinking is known by its own model. A route with no models is taken out. An api the user picked for a route in dsh's Models page is kept, and so are keys they added there. The start (`agent-default-model`) names its model's route, and Sync moves a start on one of magpie's routes to the route its model is now on (`dshStartOnItsRoute`), its effort and marks kept. Check asks the route the start names to list its model. A dsh session resumed on a model that has since moved routes asks for the model to be picked again, since dsh finds a model by route and id.
-- dsh takes a model's listed `maxTokens` as every request's output and reserves it, with 65536 of headroom, out of `contextWindow` before deciding when to compact (compaction-basic's `resolveCompactSpec`: the smaller of 80% of the window and window − maxTokens − 65536). A vendor's reply limit near its window had dsh compact at 41248 of a 500000 group (#2823) or fail with no budget (524288/524288, LoKi on Discord; 131072/65536), so magpie lists it held to a quarter of the window on every route and in the older `llm-deepseek` entry (`dshMaxTokens`), as VS Code and Copilot in JetBrains do. A Max output the user set stays as they set it (`outputOr`), even one that leaves dsh no budget. `TestDshOutputLeavesCompactionRoom`.
+- dsh takes a model's listed `maxTokens` as every request's output and reserves it, with 65536 of headroom, out of `contextWindow` before deciding when to compact (compaction-basic's `resolveCompactSpec`: the smaller of 80% of the window and window − maxTokens − 65536). A vendor's reply limit near its window had dsh compact at 41248 of a 500000 group (#2823) or fail with no budget (524288/524288, LoKi on Discord; 131072/65536), so magpie lists it held to what leaves dsh half the window to compact in (window/2 − 65536), or to a quarter of the window where that is more (every window under 262144, as VS Code and Copilot in JetBrains hold it), on every route and in the older `llm-deepseek` entry (`dshMaxTokens`). A quarter alone cut DeepSeek's 393216 on its 1000000 window to 250000 (#3298); now it keeps all of it and dsh compacts at 541248, while 500000/393216 is held to 184464 and compacts at 250000. A Max output the user set stays as they set it (`outputOr`), even one that leaves dsh no budget. `TestDshOutputLeavesCompactionRoom`.
- While magpie serves, `KeepDshWired` writes the routes again every 30 seconds, since dsh writes over its patch list. A round that writes nothing is kept (`dshRounds`), and the next round does nothing when nothing it reads has changed since. It checks the bytes of dsh's files (every profile's `cordis.patch.yml`, the home layer, `settings.yaml`, `.env`, `.credentials.yaml`, `config.yaml`) and the gateway's address. It also checks `provider.StampNow`: no Changed, plugin change or `filememo.Forget`, the same environment, and the providers', settings' and catalog's files unchanged. The models are built again when that stamp moves, and at least every `dshModelsFor` (5 minutes) for what no stamp sees, such as another app's account file or a date-retired model. Building the whole catalog and every route each round allocated 12 GB of a user's 35.7 GB in 48 minutes (`TestDshRoundWithNothingChangedBuildsNothing`, `TestDshRoundAfterAChangeWritesAgain`).
- DeepSeek Harness Desktop (macOS arm64 and Windows x64; no Linux build) reads the same `$DSH_HOME` or `~/.dsh` as the CLI, as its own profile `profiles/desktop` (`resolveDshHome` and `resolveDesktopPaths` in the app's `lib/main.js`), so it takes magpie's routes with no wiring of its own (#2279, liangjinyuan-sihan). It is found by `/Applications/DeepSeek Harness.app`, by `%LOCALAPPDATA%\Programs\DeepSeek Harness`, or by the folder its current-user installer records under `HKCU\Software\1bf39983-50d0-5fe0-9ef4-cece76f67c5e` (`InstallLocation`), whose Uninstall key gives the version (`registered` in [`desktopapp.go`](../../internal/agent/desktopapp.go)), so a user who picked another folder is found. With the app installed but never opened there is no `~/.dsh`: the row is still listed, and connecting writes `profiles/desktop/cordis.patch.yml` from dsh's own template (`dshProfileTemplate`) with magpie's routes, which the app keeps on its first start, since it writes only the files that are missing. The notice says the app starts on it the first time it opens (`noticeDsh5`). Disconnect before that first start removes the profile magpie made when nothing else is in it (`dshTidyUnopened`); once the app has opened it, the profile stays and only magpie's entries go (`TestDshDesktopNeverOpened`, `TestDshDesktopRegisteredFolder`).
- Ante's `catalog.json` keeps user-added fields in `providers.magpie` the same way (`ante` in [`ante.go`](../../internal/agent/ante.go)); its `auth`, `http_headers`, `extra_body` and `stream_idle_timeout_secs` are what a gateway shared on the local network or reached from WSL needs to send a key. Ante starts a session on the `provider` and `model` of `settings.json`, not on what its catalog holds (#1210), so magpie sets that pair and puts back what it found there; a pick the user makes in Ante while magpie is wired is theirs and stands.
diff --git a/internal/agent/dsh.go b/internal/agent/dsh.go
index a81ad2818..d1907bb82 100644
--- a/internal/agent/dsh.go
+++ b/internal/agent/dsh.go
@@ -1509,15 +1509,25 @@ func dshLevels(ref string) []string {
// near its window leaves dsh compacting at a sliver of it, or failing to
// (#2823): 500000 with 393216 compacted at 41248; 524288 with 524288, and
// 131072 with 65536, left it no budget at all. So the vendor's figure is
-// held to a quarter of the window, as VS Code's and Copilot's are, while a
-// Max output the user set stays as they set it (outputOr).
+// held to what leaves dsh half the window to compact in (window/2 - 65536),
+// or to a quarter of the window, as VS Code's and Copilot's are, where that
+// is more: a window under 262144 has no half to spare. A quarter alone cut
+// DeepSeek's 393216 on its 1000000 window to 250000, where dsh compacts at
+// 541248 with all of it (#3298); 500000 with 393216 is held to 184464 and
+// compacts at 250000. A Max output the user set stays as they set it
+// (outputOr).
func dshMaxTokens(m catalog.Model) int {
if m.Context <= 0 {
return maxTokens(m)
}
- return outputOr(m, min(maxTokens(m), max(1, m.Context/4)))
+ room := max(m.Context/4, m.Context/2-dshHeadroom)
+ return outputOr(m, min(maxTokens(m), max(1, room)))
}
+// dshHeadroom is compaction-basic's default headroomTokens: what dsh keeps
+// free beside a request's maxTokens before it compacts.
+const dshHeadroom = 65536
+
// dshRouteConfig is magpie's first route with every one of models on it.
func dshRouteConfig(models []catalog.Model, api, gw string) dshPiRoute {
return dshKindConfig(dshKinds[0], models, api, gw)
diff --git a/internal/agent/dsh_compaction_budget_test.go b/internal/agent/dsh_compaction_budget_test.go
index 0531722bc..625d2ef8c 100644
--- a/internal/agent/dsh_compaction_budget_test.go
+++ b/internal/agent/dsh_compaction_budget_test.go
@@ -24,19 +24,24 @@ func dshCompactsAt(window, out int) int {
// a model with, and 65536 more, out of its window before it compacts. A
// vendor's reply limit near the window had dsh compact at 41248 of a 500000
// group (#2823) or fail outright on 524288/524288 (LoKi on Discord) and
-// 131072/65536; it is held to a quarter of the window on every route and in
-// the older llm-deepseek entry, while a Max output the user set stays.
+// 131072/65536; it is held to what leaves dsh half the window
+// (window/2 - 65536), or a quarter of the window where that is more, on every
+// route and in the older llm-deepseek entry, while a Max output the user set
+// stays. DeepSeek's own 1000000/393216 keeps all of its output (#3298).
func TestDshOutputLeavesCompactionRoom(t *testing.T) {
for _, tc := range []struct {
name string
model catalog.Model
want int
}{
- {"full window", catalog.Model{ID: "space-bunny-free", Context: 524288, Output: 524288}, 131072},
- {"above the window", catalog.Model{ID: "above", Context: 524288, Output: 1048576}, 131072},
- {"group window under its members' output", catalog.Model{ID: "group/smart-split", Context: 500000, Output: 393216}, 125000},
+ {"full window", catalog.Model{ID: "space-bunny-free", Context: 524288, Output: 524288}, 196608},
+ {"above the window", catalog.Model{ID: "above", Context: 524288, Output: 1048576}, 196608},
+ {"group window under its members' output", catalog.Model{ID: "group/smart-split", Context: 500000, Output: 393216}, 184464},
{"half the window", catalog.Model{ID: "k131", Context: 131072, Output: 65536}, 32768},
{"within a quarter", catalog.Model{ID: "flash", Context: 1000000, Output: 128000}, 128000},
+ {"DeepSeek's window and output", catalog.Model{ID: "deepseek/deepseek-flash", Context: 1000000, Output: 393216}, 393216},
+ {"a 200K window", catalog.Model{ID: "ds200", Context: 200000, Output: 393216}, 50000},
+ {"a 128K window", catalog.Model{ID: "ds128", Context: 128000, Output: 393216}, 32000},
{"user output", catalog.Model{ID: "user", Context: 524288, Output: 393216, OutputSet: true}, 393216},
{"user full window", catalog.Model{ID: "user-full", Context: 524288, Output: 524288, OutputSet: true}, 524288},
{"unknown window", catalog.Model{ID: "unknown", Output: 128000}, 128000},
@@ -57,8 +62,12 @@ func TestDshOutputLeavesCompactionRoom(t *testing.T) {
t.Fatalf("llm-deepseek entry lists a maxTokens for a model of no known output:\n%s", lines)
}
if tc.model.Context > 0 && !tc.model.OutputSet {
- if at := dshCompactsAt(tc.model.Context, tc.want); at < tc.model.Context/4 {
- t.Fatalf("dsh compacts %d/%d at %d tokens", tc.model.Context, tc.want, at)
+ // half the window for messages where the window has a
+ // half to spare beside the headroom, and never under
+ // what a quarter for output leaves
+ floor := min(tc.model.Context/2, dshCompactsAt(tc.model.Context, min(tc.model.Output, tc.model.Context/4)))
+ if at := dshCompactsAt(tc.model.Context, tc.want); at == 0 || at < floor {
+ t.Fatalf("dsh compacts %d/%d at %d tokens, under %d", tc.model.Context, tc.want, at, floor)
}
}
})
diff --git a/internal/agent/dsh_test.go b/internal/agent/dsh_test.go
index 1e309f473..ee63e397b 100644
--- a/internal/agent/dsh_test.go
+++ b/internal/agent/dsh_test.go
@@ -197,7 +197,7 @@ func TestDshModelLimits(t *testing.T) {
want := ` - id: v/see
name: see · V
contextWindow: 400000
- maxTokens: 100000
+ maxTokens: 128000
input: [text, image]
- id: v/plain
name: plain · V
diff --git a/internal/agent/sync_test.go b/internal/agent/sync_test.go
index 435eebcce..8b3d56739 100644
--- a/internal/agent/sync_test.go
+++ b/internal/agent/sync_test.go
@@ -139,8 +139,8 @@ func TestCrushModelsCarryMaxTokens(t *testing.T) {
// output. ZCode and WorkBuddy cap it at zcodeMaxOutput besides, Crush falls
// back to 16384 without a known output, OpenCode is handed no limit without
// a window, and AtomCode and Snow CLI, each told a window of its own when
-// the model's isn't known, are cut to that. dsh is held to a quarter of
-// a known window (dshMaxTokens, #2823).
+// the model's isn't known, are cut to that. dsh is held to what leaves it
+// half of a known window, or a quarter of it (dshMaxTokens, #2823).
func TestMaxTokensWithinContextWindow(t *testing.T) {
home := syncHome(t)
check := func(limit string, want, window int) {
@@ -181,10 +181,11 @@ func TestMaxTokensWithinContextWindow(t *testing.T) {
told = strconv.Itoa(min(want, atomcodeContext))
}
// dsh reserves maxTokens out of its window before compacting, so a
- // known window holds it to a quarter (dshMaxTokens, #2823).
+ // known window holds it to what leaves dsh half of it, or a quarter
+ // where that is more (dshMaxTokens, #2823, #3298).
dshTold := n
if strings.Contains(limit, "context") {
- dshTold = strconv.Itoa(min(want, window/4))
+ dshTold = strconv.Itoa(min(want, max(window/4, window/2-65536)))
}
snowTold := n
if !strings.Contains(limit, "context") {