Skip to content

Run frontier MoE models on hardware you already own β€” pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

C

deepseek_v4: honor CTX in the generate CLI

Blake71ef303c4cd7
diff --git a/c/deepseek_v4.c b/c/deepseek_v4.c
index 9fe5d651..0dfeb43b 100644
--- a/c/deepseek_v4.c
+++ b/c/deepseek_v4.c
@@ -7387,7 +7387,8 @@ static void v4_cli_usage(FILE *stream, const char *program) {
" --oracle FILE validate against an oracle JSON fixture\n"
" --teacher-forcing N oracle: compare top-1 on N prompt positions\n"
" --greedy N oracle: compare N greedy continuation tokens\n"
- " --record-oracle FILE write greedy tokens + tf_pred to JSON\n",
+ " --record-oracle FILE write greedy tokens + tf_pred to JSON\n"
+ " CTX=N (environment) context window in tokens (default: 4096)\n",
program, program, program);
}
@@ -8617,6 +8618,9 @@ int main(int argc, char **argv) {
.target_model_dir = cli.model_dir,
.no_dspark = cli.no_dspark,
.pin_slots_per_layer = -1,
+ /* Same contract as v4_serve_main: CTX sets the session plan;
+ * unset or invalid falls back to the engine default (4096). */
+ .context_tokens = getenv("CTX") ? atoi(getenv("CTX")) : 0,
};
if (cli.memory_gib > 0.0)
open_opts.memory_limit_bytes =
diff --git a/docs/ENVIRONMENT.md b/docs/ENVIRONMENT.md
index e1ed32d1..ae0223ef 100644
--- a/docs/ENVIRONMENT.md
+++ b/docs/ENVIRONMENT.md
@@ -16,6 +16,7 @@ what follows, but the sister engines read their own:
| `kimi_k3` | `c/kimi_k3.c` | the `K3_*` family β€” see [Kimi K3 engine](#kimi-k3-engine-kimi_k3) |
| `inkling` | `c/inkling.c` | `INK_*`, plus `CTX_MAX`, `PIN_N`, `REP_PEN`, `GPU_DEV`, `NOGPU` β€” see [Inkling engine](#inkling-engine-inkling) |
| `olmoe` | `c/olmoe.c` | `HOT`, `WIDE`, `SMOOTH`, `CONF_LIMIT`, `MAX_NEW`, `CHAT`, `EXPERT_DROP`, `WARMUP` β€” see [OLMoE engine](#olmoe-engine-olmoe) |
+| `deepseek_v4` | `c/deepseek_v4.c` | `CTX` β€” context window in tokens (default 4096), honored by both the CLI and `SERVE` mode |
Setting an `INK_*` variable while running `colibri` does nothing, and vice
versa; nothing warns you about it. A few variables are genuinely shared because