Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

fix(eve): default mockModel context window so the docs example compiles (#3994)

roboeve[bot]dd4374e50c58
diff --git a/.changeset/mock-model-context-window.md b/.changeset/mock-model-context-window.md
new file mode 100644
index 000000000..7b8f2f1a3
--- /dev/null
+++ b/.changeset/mock-model-context-window.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Agents that use `mockModel` from `eve/evals` now compile without `modelContextWindowTokens`. Before, the documented one-line `mockModel` example failed with a missing AI Gateway context window error; mock models now default to a 1,000,000-token window.
diff --git a/docs/evals/overview.mdx b/docs/evals/overview.mdx
index 088b143cb..e3a8227d8 100644
--- a/docs/evals/overview.mdx
+++ b/docs/evals/overview.mdx
@@ -139,7 +139,7 @@ model: mockModel({
});
```
-`mockModel()` uses `"Mock response"` when no response is supplied. It handles both generated and streamed responses, derives deterministic response metadata, and estimates token usage. Because the model is part of the agent definition, use it for a dedicated fixture agent; it remains mocked whether that fixture runs locally or as a deployed eval target.
+`mockModel()` uses `"Mock response"` when no response is supplied. It handles both generated and streamed responses, derives deterministic response metadata, and estimates token usage. It compiles with a 1,000,000-token context window, so compaction stays out of the way; set `modelContextWindowTokens` on the agent to exercise compaction with a smaller window. Because the model is part of the agent definition, use it for a dedicated fixture agent; it remains mocked whether that fixture runs locally or as a deployed eval target.
## The `t` context
diff --git a/packages/eve/src/compiler/normalize-agent-config.ts b/packages/eve/src/compiler/normalize-agent-config.ts
index f6d304867..931cedb2b 100644
--- a/packages/eve/src/compiler/normalize-agent-config.ts
+++ b/packages/eve/src/compiler/normalize-agent-config.ts
@@ -5,6 +5,7 @@ import { normalizeLogicalPath } from "#discover/filesystem.js";
import { normalizeAgentDefinition } from "#internal/authored-definition/core.js";
import { formatLanguageModelGatewayId } from "#internal/runtime-model.js";
import { classifyModelRouting } from "#internal/classify-model-routing.js";
+import { isMockModel } from "#internal/mock-model-identity.js";
import { isChatGptModelRouting } from "#shared/chatgpt-model.js";
import { toErrorMessage } from "#shared/errors.js";
import { parseJsonObject, type JsonObject } from "#shared/json.js";
@@ -277,6 +278,14 @@ async function normalizeAuthoredModelReference(input: {
};
}
+ // Mock models have no catalog entry or real limit; a large window keeps compaction opt-in.
+ if (isMockModel(languageModel)) {
+ return {
+ ...sourceBackedModel,
+ contextWindowTokens: 1_000_000,
+ };
+ }
+
try {
const providerResult = await input.modelCatalog.getByProviderModelId(
languageModel.provider,
diff --git a/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts b/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts
index ef7bc32b2..c3e034b1e 100644
--- a/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts
+++ b/packages/eve/test/scenarios/compiler-model-catalog.scenario.test.ts
@@ -623,4 +623,33 @@ describe("compiler model catalog", () => {
});
expect(fetchSpy).not.toHaveBeenCalled();
});
+
+ it("compiles the one-line mockModel fixture without an authored context window", async () => {
+ const { agentRoot, appRoot } = await createAppRoot(
+ "eve-model-catalog-mock-model-",
+ APP_ROOT_OPTIONS,
+ );
+
+ await writeFile(
+ join(agentRoot, "agent.ts"),
+ [
+ 'import { defineAgent } from "eve";',
+ 'import { mockModel } from "eve/evals";',
+ "",
+ "export default defineAgent({",
+ ' model: mockModel("A deterministic reply"),',
+ "});",
+ "",
+ ].join("\n"),
+ );
+
+ vi.spyOn(globalThis, "fetch").mockImplementation(async () => mockCatalogResponse([]));
+
+ const result = await compileAgent({ startPath: appRoot });
+
+ expect(result.manifest.config.model).toMatchObject({
+ contextWindowTokens: expect.any(Number),
+ id: "eve-mock/model",
+ });
+ });
});