Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

Constrain optional effort selection

Rach Pradhana65193db6120
diff --git a/docs/adr/0193-jev-selects-effort-only.md b/docs/adr/0193-jev-selects-effort-only.md
new file mode 100644
index 00000000..ee1329ca
--- /dev/null
+++ b/docs/adr/0193-jev-selects-effort-only.md
@@ -0,0 +1,29 @@
+# 0193. Jev selects effort only
+
+Status: accepted 2026-09-23
+
+## Context
+
+An optional general judgment tool could assess arbitrary actions but did not
+change the harness's effort. A small decision evaluation saw correct first
+actions with and without that tool; it did not establish a benefit from
+generic judging. Effort is already a session setting shared by CLI, TUI, and
+ACP (ADR 0191).
+
+## Decision
+
+Expose only `jev_effort` to eligible root sessions with a separate Codegraff
+login. Accept a short task summary, then build a fixed choice from the active
+model's supported effort levels. Reject arbitrary questions or judgments.
+Queue a validated selection per agent and apply it on the owning thread at
+the next request boundary, only if the provider, model, and effort allowlist
+still match. A failed or invalid response leaves the setting unchanged.
+Explicit user `/effort` and ACP settings remain available. No automatic Jev
+call is made.
+
+## Consequences
+
+Changing effort may miss cached prefixes or reset a WebSocket reasoning chain;
+the explicit tool call is the opt-in boundary for that cost. ACP publishes the
+same thought-level update as a manual change. Billing and independent login
+rules from ADR 0187 remain in force.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 3ea46f0f..e4b4127e 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -218,6 +218,7 @@ record only when you need the evidence or the edge cases.
| [0190](0190-catalog-gets-share-bounded-http2-transport.md) | Catalog GETs use bounded HTTP/2 pages with joined deadlines, shared redirect limits, and credentials scoped to the original origin. |
| [0191](0191-acp-effort-is-session-configuration.md) | ACP thought-level options follow the active model and share `/effort` state; changes received during a turn apply after that turn. |
| [0192](0192-mimo-thinking-is-a-binary-setting.md) | MiMo exposes Off/On, preserves legacy positive settings as On, and uses the documented thinking control on each wire. |
+| [0193](0193-jev-selects-effort-only.md) | Optional Jev selects only supported session effort at the next request boundary; arbitrary judgments are not exposed. |
## When to write one
diff --git a/docs/experiments/jev-use-harness.md b/docs/experiments/jev-use-harness.md
index d78cbf19..1ce2f460 100644
--- a/docs/experiments/jev-use-harness.md
+++ b/docs/experiments/jev-use-harness.md
@@ -1,24 +1,25 @@
-# Native Jev judgment tool (experiment)
+# Optional Jev effort selection
-Graff offers `jev_judge` as a **native tool**, not an MCP server, when the
-active model is a GPT-6-series model on Codex/OpenAI or a Xiaomi MiMo model
-(including matching Codegraff gateway aliases), **and** the user has a
-persisted Codegraff login from `graff login`. A Codex login or
-`CODEGRAFF_API_KEY` alone does not satisfy that gate. Graff checks the
-recognized local login store; it does not probe the account online. Direct
-dispatch enforces the same rules. Model switches and an in-session
-`/login codegraff` refresh the root tool catalog.
+Graff offers `jev_effort` as a native, optional tool for a root session using
+GPT-6 on Codex/OpenAI or MiMo v2.6 on Xiaomi, including those eligible models
+through the Codegraff gateway. A persisted Codegraff login from `graff login`
+is required independently of the active model login. Model switches and an
+in-session login refresh the catalog. Subagents cannot invoke the tool.
-The tool sends only its explicitly supplied short `state` and one typed
-`question` to Codegraff's `/v1/systemone` endpoint using the persisted login;
-it never automatically uploads repository context. No separate upstream key is
-needed.
-Do not put code, paths, secrets, or customer data in a tool call. Use
-`JEV_BACKEND=mock` for a no-network wiring check.
+The tool accepts only `task`, a short, non-sensitive summary of the next work.
+It constructs a fixed choice from the active model's `/effort` options. GPT-6
+uses its existing effort ladder; MiMo v2.6 offers Off and On. It does not
+accept a caller-written question, candidate options, generic judgment, or
+source code. Do not send paths, secrets, or customer data. `JEV_BACKEND=mock`
+checks the path without a network request.
-Jev answers `noul` (yes/no probability), `choice`, or ordered `score`
-questions. Low-confidence answers tell the main model to decide. Any failed
-Jev request or malformed Jev response opens a one-strike, process-session
-circuit: later calls skip the network and return control to the main model.
-The tool disappears from the root catalog after that first failure. There is
-no retry, automatic routing, MCP permission, or claim of a measured speedup.
+A validated, allowlisted selection changes the session's effort
+at the next model request boundary; ACP receives the normal thought-level
+configuration update. It may change the cached request prefix. A failed,
+invalid, canceled, or stale-route selection leaves effort unchanged. One
+failed upstream request opens a session-long circuit with no retry. The tool
+never evaluates whether an answer or action was correct and never runs
+automatically on every turn.
+
+Usage and charges follow [ADR 0187](../adr/0187-optional-judgments-preserve-usage-uncertainty.md):
+only a confirmed gateway settlement receipt supplies a known charge.
diff --git a/evals/harness_behavior.jsonl b/evals/harness_behavior.jsonl
index d5b1b47a..92d0d68e 100644
--- a/evals/harness_behavior.jsonl
+++ b/evals/harness_behavior.jsonl
@@ -140,3 +140,4 @@
{"id":"large-log-preview-early-omitted-error","why":"A spilled tool result must surface useful omitted error lines in its first model request without an extra pager call.","args":["--old"],"env":{"GRAFF_NO_CODEDB_GUARD":"1"},"prompt":"Run the diagnostic report and summarize its errors.","script":[{"tool":"shell","arguments":{"command":"python3 -c '[print('\"'\"'ok %04d '\"'\"'%i+'\"'\"'x'\"'\"'*32) for i in range(24)]; print('\"'\"'ERROR '\"'\"'+ '\"'\"'EARLY_'\"'\"'+'\"'\"'GAP_VISIBLE'\"'\"'); [print('\"'\"'ok %04d '\"'\"'%i+'\"'\"'x'\"'\"'*32) for i in range(24,625)]'"}},{"text":"Report reviewed."}],"assert":[{"request_count":2},{"request_contains":{"index":1,"text":"ERROR EARLY_GAP_VISIBLE"}},{"request_contains":{"index":1,"text":"tool result handle"}},{"final_text_contains":"Report reviewed."}]}
{"id":"large-log-preview-short-error-after-long-line","why":"A spilled tool result must surface useful omitted error lines in its first model request without an extra pager call.","args":["--old"],"env":{"GRAFF_NO_CODEDB_GUARD":"1"},"prompt":"Run the diagnostic report and summarize its errors.","script":[{"tool":"shell","arguments":{"command":"python3 -c '[print('\"'\"'ok %04d '\"'\"'%i+'\"'\"'x'\"'\"'*32) for i in range(100)]; print('\"'\"'ERROR '\"'\"'+ '\"'\"'z'\"'\"'*2500); print('\"'\"'ERROR '\"'\"'+ '\"'\"'SHORT_'\"'\"'+'\"'\"'DETAIL_VISIBLE'\"'\"'); [print('\"'\"'ok %04d '\"'\"'%i+'\"'\"'x'\"'\"'*32) for i in range(100,700)]'"}},{"text":"Report reviewed."}],"assert":[{"request_count":2},{"request_contains":{"index":1,"text":"ERROR SHORT_DETAIL_VISIBLE"}},{"request_contains":{"index":1,"text":"tool result handle"}},{"final_text_contains":"Report reviewed."}]}
{"id":"stream-reused-index-distinct-tool-ids","why":"A streamed tool index can restart with a fresh call ID. Guard the incomplete first call, execute the separate complete call once, and preserve both outcomes in model history.","args":["--old"],"env":{"GRAFF_NO_CODEDB_GUARD":"1"},"prompt":"Run the scripted file operation and report the observed result.","script":[{"stream_deltas":[{"tool_calls":[{"index":0,"id":"old","type":"function","function":{"name":"write_file","arguments":"{\"path\":\"guard.txt\""}}]},{"tool_calls":[{"index":0,"id":"new","type":"function","function":{"name":"write_file","arguments":"{\"path\":\"result.txt\",\"content\":\"done\\n\"}"}}]}]},{"text":"The valid file operation completed."}],"verify_command":["python3","-c","from pathlib import Path; assert not Path('guard.txt').exists(); assert Path('result.txt').read_text() == 'done\\n'"],"assert":[{"request_count":2},{"event":{"type":"tool_call_finished","name":"write_file","is_error":true}},{"event":{"type":"tool_call_finished","name":"write_file","is_error":false}},{"events_at_most":{"match":{"type":"tool_call_finished","name":"write_file","is_error":false},"count":1}},{"request_contains":{"index":1,"text":"tool call arguments were truncated or not a JSON object"}},{"file_equals":{"path":"result.txt","content":"done\n"}},{"final_text_contains":"The valid file operation completed."}]}
+{"id":"jev-effort-only-rejects-judgment","why":"Jev is only an optional effort selector: the removed generic judgment call and an effort request on an ineligible route must both fail locally without changing session effort.","env":{"JEV_BACKEND":"mock"},"prompt":"Try the old judgment tool, then try effort selection here.","script":[{"tool":"jev_judge","arguments":{"state":"tests passed","question":"Did tests pass?","type":"noul"}},{"tool":"jev_effort","arguments":{"task":"a small read-only lookup"}},{"text":"Neither call changed effort."}],"assert":[{"event":{"type":"tool_result","name":"jev_judge","is_error":true,"text":"unknown tool: jev_judge"}},{"event":{"type":"tool_result","name":"jev_effort","is_error":true}},{"final_text_contains":"Neither call changed effort."}]}
diff --git a/src/acp_config.zig b/src/acp_config.zig
index 0d91301a..cf34a4bb 100644
--- a/src/acp_config.zig
+++ b/src/acp_config.zig
@@ -35,6 +35,7 @@ pub fn set(ctx: *anyopaque, value: []const u8) anyerror!bool {
const canonical = er.normalize(root.provider.id, root.provider.model, value);
if (!root.effortApplies() or !er.allows(root.provider.id, root.provider.model, canonical)) return false;
root.reasoning = std.meta.stringToEnum(ReasoningEffort, canonical) orelse return false;
+ root.jev_effort_pending.invalidate(root.io);
_ = @import("repl_glue.zig").saveThinkingSettings(root.io, root.gpa, root.reasoning, root.fast, root.ultracode_mode, root.show_thinking, root.ai_title);
return true;
}
@@ -71,6 +72,11 @@ test "MiMo thought-level discovery names Off and On and normalizes saved positiv
var keys: @import("provider.zig").Keys = .{ .values = @splat(null) };
var live: LiveTurn = .{ .root = &root, .keys = &keys, .out = undefined };
root.reasoning = .low;
+ const pending = root.jev_effort_pending.begin(root.io, root.provider).?;
+ try std.testing.expect(root.jev_effort_pending.commit(root.io, pending, .none));
+ try std.testing.expect(@import("jev_effort_state.zig").applyToState(&root));
+ try std.testing.expectEqualStrings("none", (try option(&live, arena)).?.currentValue);
+ root.reasoning = .low;
const on = (try option(&live, arena)).?;
try std.testing.expectEqualStrings("high", on.currentValue);
try std.testing.expectEqual(@as(usize, 2), on.options.len);
diff --git a/src/agent.zig b/src/agent.zig
index fc27488a..ba7527f4 100644
--- a/src/agent.zig
+++ b/src/agent.zig
@@ -149,6 +149,7 @@ pub const Agent = struct {
stored_keys_loaded: bool = true, // false after an explicit-provider launch; model surfaces fill the remaining Keychain slots
keep_context: bool = true, // carry the conversation across wire-format model switches (/keepcontext)
reasoning: ReasoningEffort = .medium, // reasoning/thinking depth — xai, codex, deepseek, codegraff (/effort, /reasoning)
+ jev_effort_pending: @import("jev_effort_state.zig").Pending = .{},
fast: bool = false, // codex "fast" mode → priority service_tier (/fast)
fallback_allow: []const []const u8 = &.{}, // explicit cross-provider allowlist from .harness/settings.json
fallback_active: bool = false, // current provider/model is a temporary fallback, not the saved preference
@@ -288,13 +289,10 @@ pub const Agent = struct {
pub fn toolsJson(self: *const Agent) []const u8 {
return @import("agent_catalog.zig").toolsJson(self);
}
-
pub fn ensureRootTools(self: *Agent, kind: Provider.Kind) !void {
return @import("agent_catalog.zig").ensureRootTools(self, kind);
}
-
pub const invalidateRootTools = @import("agent_catalog.zig").invalidateRootTools;
-
pub noinline fn ensureModelCatalog(self: *Agent, keys: provider_mod.Keys) void {
if (self.model_catalog) |*catalog|
catalog.ensure(self.io, self.gpa, self.arena, self.home, keys.get("codex") orelse "", keys.codex_account);
@@ -328,6 +326,8 @@ pub const Agent = struct {
}
pub fn runTurn(self: *Agent) anyerror![]const u8 {
+ defer @import("jev_effort_state.zig").finishTurn(self);
+ errdefer self.jev_effort_pending.invalidate(self.io);
self.async_tools_armed = !self.sub and self.eval_cmd == null;
defer self.async_tools_armed = false;
defer @import("agent_async_tools.zig").reset(self);
diff --git a/src/agent_argstream.zig b/src/agent_argstream.zig
index 8bdfa00f..9864bc5d 100644
--- a/src/agent_argstream.zig
+++ b/src/agent_argstream.zig
@@ -44,6 +44,7 @@ fn toolCtx(self: *Agent) tools_mod.ToolCtx {
.io = self.io,
.client = self.client,
.provider = self.provider,
+ .jev_effort_pending = &self.jev_effort_pending,
.subagent_provider = self.subagent_provider,
.subagent_cross_provider = self.subagent_cross_provider,
.mcp_context = self.mcp_context.value,
diff --git a/src/agent_request.zig b/src/agent_request.zig
index a4ae4e05..b02cb3fa 100644
--- a/src/agent_request.zig
+++ b/src/agent_request.zig
@@ -85,6 +85,11 @@ fn sayTypedApiError(self: *Agent, etype: []const u8, ecode: ?[]const u8, emsg: [
}
pub fn request(self: *Agent, tools_in: ?[]const u8) !std.json.ObjectMap {
+ // Pool-thread Jev results become session state only on this owner thread.
+ if (Agent.esc_cancel.load(.acquire) or @import("acp_engine.zig").cancel_flag.load(.acquire))
+ self.jev_effort_pending.invalidate(self.io)
+ else
+ @import("jev_effort_state.zig").apply(self);
var usage_attempts: @import("request_usage_attempts.zig").Ledger = .{};
defer usage_attempts.finish(self.io, &@import("pricing.zig").g_cost);
errdefer {
diff --git a/src/agent_tool_batch.zig b/src/agent_tool_batch.zig
index 581237d6..2bf637e5 100644
--- a/src/agent_tool_batch.zig
+++ b/src/agent_tool_batch.zig
@@ -30,12 +30,18 @@ pub const skipped_text = "tool execution skipped because the operation was abort
/// serializes, because write/edit do. Shell-only batches stay parallel
/// (#266); the parent joins all results before converting/freeing them (#1166).
pub fn isSequential(name: []const u8) bool {
+ if (std.mem.eql(u8, name, @import("jev_tool.zig").name)) return true;
if (std.mem.eql(u8, name, "write_file")) return true;
if (std.mem.eql(u8, name, "edit_file")) return true;
if (std.mem.eql(u8, name, imagegen.tool_name)) return true;
return false;
}
+test "optional effort selections serialize in model call order" {
+ try std.testing.expect(isSequential(@import("jev_tool.zig").name));
+ try std.testing.expect(!isSequential("shell"));
+}
+
fn isShellName(name: []const u8) bool {
return std.mem.eql(u8, name, "shell") or std.mem.eql(u8, name, "bash");
}
@@ -63,6 +69,7 @@ pub fn context(self: *Agent) ToolCtx {
.io = self.io,
.client = self.client,
.provider = self.provider,
+ .jev_effort_pending = &self.jev_effort_pending,
.subagent_provider = self.subagent_provider,
.subagent_cross_provider = self.subagent_cross_provider,
.mcp_context = self.mcp_context.value,
diff --git a/src/commands_effort.zig b/src/commands_effort.zig
index f14b3bea..45afd506 100644
--- a/src/commands_effort.zig
+++ b/src/commands_effort.zig
@@ -72,6 +72,7 @@ pub fn handle(root: *Agent, arena: std.mem.Allocator, line: []const u8, out: *st
try out.flush();
return true;
}
+ root.jev_effort_pending.invalidate(root.io);
_ = @import("repl_glue.zig").saveThinkingSettings(root.io, root.gpa, root.reasoning, root.fast, root.ultracode_mode, root.show_thinking, root.ai_title);
try out.print("reasoning effort: {s}{s}\n", .{
display(root, root.reasoning),
diff --git a/src/effort_route.zig b/src/effort_route.zig
index dcc4dd6e..ddaf512d 100644
--- a/src/effort_route.zig
+++ b/src/effort_route.zig
@@ -1,9 +1,8 @@
//! Lookup-shape detector. Used to decide a prompt is Q&A vs mutation.
-//! The harness does NOT auto-flip `reasoning_effort` — that field is part
-//! of the cached prefix on Codex, OpenAI, xAI, DeepSeek, and the codegraff
-//! gateway. Changing it mid-conversation misses the cache and drops the
-//! WS chain (codex_chain.propsFp). An explicit /effort or a worker
-//! `effort:` pin is the only way effort moves.
+//! The harness does NOT infer low effort from prompt shape: effort is part of
+//! the cached prefix and a change may miss the cache and reset the WS chain.
+//! Explicit /effort, a worker effort pin, or an optional jev_effort call may
+//! change it; Jev applies only at the next request boundary.
const std = @import("std");
diff --git a/src/exec.zig b/src/exec.zig
index 1481edaf..25e4b15b 100644
--- a/src/exec.zig
+++ b/src/exec.zig
@@ -422,10 +422,15 @@ test "native Jev dispatch rejects other models and latches off after one failed
}
}{});
const base: @import("provider.zig").Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
- var ctx: ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = undefined, .provider = base, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
- const parsed = try std.json.parseFromSlice(Value, std.testing.allocator, "{\"state\":\"10 tests passed\",\"question\":\"Did CI pass?\",\"type\":\"noul\"}", .{});
+ var pending: @import("jev_effort_state.zig").Pending = .{};
+ var ctx: ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = undefined, .provider = base, .jev_effort_pending = &pending, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
+ const parsed = try std.json.parseFromSlice(Value, std.testing.allocator, "{\"task\":\"choose effort for a test fix\"}", .{});
defer parsed.deinit();
const call: ToolCall = .{ .id = "1", .name = jev.name, .input = parsed.value };
+ const old_call: ToolCall = .{ .id = "old", .name = "jev_judge", .input = parsed.value };
+ const old = try execToolInner(ctx, old_call);
+ defer std.testing.allocator.free(old.text);
+ try std.testing.expect(old.is_error);
const no_login = try execToolInner(ctx, call);
defer std.testing.allocator.free(no_login.text);
try std.testing.expect(no_login.is_error);
diff --git a/src/jev_effort_state.zig b/src/jev_effort_state.zig
new file mode 100644
index 00000000..89f698e6
--- /dev/null
+++ b/src/jev_effort_state.zig
@@ -0,0 +1,105 @@
+//! Per-agent, one-at-a-time effort changes selected by the optional Jev tool.
+//! Pool threads only queue a selection; the owning Agent applies it at a
+//! request boundary. An admission token prevents canceled work from reviving
+//! a cleared selection or racing a later tool call.
+const std = @import("std");
+const Io = std.Io;
+const Provider = @import("provider.zig").Provider;
+const ReasoningEffort = @import("main.zig").ReasoningEffort;
+
+pub const Pending = struct {
+ mu: Io.Mutex = .init,
+ phase: enum { idle, in_flight, selected } = .idle,
+ token: u64 = 0,
+ provider_len: usize = 0,
+ model_len: usize = 0,
+ provider: [64]u8 = undefined,
+ model: [256]u8 = undefined,
+ effort: ReasoningEffort = .medium,
+
+ pub fn begin(self: *Pending, io: Io, route: Provider) ?u64 {
+ self.mu.lockUncancelable(io);
+ defer self.mu.unlock(io);
+ if (self.phase != .idle or route.id.len > self.provider.len or route.model.len > self.model.len) return null;
+ self.token +%= 1;
+ @memcpy(self.provider[0..route.id.len], route.id);
+ @memcpy(self.model[0..route.model.len], route.model);
+ self.provider_len = route.id.len;
+ self.model_len = route.model.len;
+ self.phase = .in_flight;
+ return self.token;
+ }
+
+ pub fn commit(self: *Pending, io: Io, token: u64, effort: ReasoningEffort) bool {
+ self.mu.lockUncancelable(io);
+ defer self.mu.unlock(io);
+ if (self.phase != .in_flight or self.token != token) return false;
+ self.effort = effort;
+ self.phase = .selected;
+ return true;
+ }
+
+ pub fn abort(self: *Pending, io: Io, token: u64) void {
+ self.mu.lockUncancelable(io);
+ defer self.mu.unlock(io);
+ if (self.phase == .in_flight and self.token == token) self.phase = .idle;
+ }
+
+ pub fn take(self: *Pending, io: Io, route: Provider) ?ReasoningEffort {
+ self.mu.lockUncancelable(io);
+ defer self.mu.unlock(io);
+ if (self.phase != .selected) return null;
+ self.phase = .idle;
+ if (!std.mem.eql(u8, route.id, self.provider[0..self.provider_len]) or
+ !std.mem.eql(u8, route.model, self.model[0..self.model_len])) return null;
+ return self.effort;
+ }
+
+ pub fn invalidate(self: *Pending, io: Io) void {
+ self.mu.lockUncancelable(io);
+ defer self.mu.unlock(io);
+ self.token +%= 1;
+ self.phase = .idle;
+ }
+};
+
+pub fn applyToState(agent: anytype) bool {
+ const effort = agent.jev_effort_pending.take(agent.io, agent.provider) orelse return false;
+ if (!@import("effort_route.zig").allows(agent.provider.id, agent.provider.model, @tagName(effort))) return false;
+ agent.reasoning = effort;
+ return true;
+}
+
+pub fn apply(agent: anytype) void {
+ if (!applyToState(agent)) return;
+ _ = @import("repl_glue.zig").saveThinkingSettings(agent.io, agent.gpa, agent.reasoning, agent.fast, agent.ultracode_mode, agent.show_thinking, agent.ai_title);
+}
+
+pub fn finishTurn(agent: anytype) void {
+ if (@TypeOf(agent.*).esc_cancel.load(.acquire) or @import("acp_engine.zig").cancel_flag.load(.acquire))
+ agent.jev_effort_pending.invalidate(agent.io)
+ else
+ apply(agent);
+}
+
+test "pending effort is per agent, first admission wins, and stale routes or tokens do not apply" {
+ const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
+ var a: Pending = .{};
+ var b: Pending = .{};
+ const first = a.begin(std.testing.io, p).?;
+ try std.testing.expect(a.begin(std.testing.io, p) == null);
+ const other = b.begin(std.testing.io, p).?;
+ a.abort(std.testing.io, first);
+ const second = a.begin(std.testing.io, p).?;
+ try std.testing.expect(!a.commit(std.testing.io, first, .high));
+ try std.testing.expect(a.commit(std.testing.io, second, .high));
+ try std.testing.expect(b.commit(std.testing.io, other, .low));
+ try std.testing.expectEqual(ReasoningEffort.low, b.take(std.testing.io, p).?);
+ var changed = p;
+ changed.model = "gpt-6-luna";
+ try std.testing.expect(a.take(std.testing.io, changed) == null);
+ try std.testing.expect(a.take(std.testing.io, p) == null);
+ const canceled = a.begin(std.testing.io, p).?;
+ a.invalidate(std.testing.io);
+ try std.testing.expect(!a.commit(std.testing.io, canceled, .none));
+}
diff --git a/src/jev_model_scope.zig b/src/jev_model_scope.zig
index 469c1bb1..71f5efd6 100644
--- a/src/jev_model_scope.zig
+++ b/src/jev_model_scope.zig
@@ -1,4 +1,4 @@
-//! Native Jev judgments are limited to GPT-6 on Codex/OpenAI and Xiaomi
+//! Native Jev effort selection is limited to GPT-6 on Codex/OpenAI and Xiaomi
//! MiMo v2.6, including those model ids served through the Codegraff gateway.
const std = @import("std");
const Provider = @import("provider.zig").Provider;
diff --git a/src/jev_provider_switch_tests.zig b/src/jev_provider_switch_tests.zig
index 34e867de..67bc981c 100644
--- a/src/jev_provider_switch_tests.zig
+++ b/src/jev_provider_switch_tests.zig
@@ -13,6 +13,8 @@ test "native Jev catalog refreshes on eligible and ineligible model switches" {
var root: Agent = undefined;
root.provider = p;
+ root.io = std.testing.io;
+ root.jev_effort_pending = .{};
root.subagent_provider = null;
root.subagent_provider_explicit = true;
root.arena = a;
@@ -52,7 +54,9 @@ test "native Jev catalog refreshes on eligible and ineligible model switches" {
gpt6.model = "gpt-6-sol";
_ = try providers.applyProviderInner(&root, a, gpt6, false);
try std.testing.expect(std.mem.indexOf(u8, root.tools_responses, jev_tool.name) != null);
+ const old_selection = root.jev_effort_pending.begin(std.testing.io, gpt6).?;
_ = try providers.applyProviderInner(&root, a, p, false);
+ try std.testing.expect(!root.jev_effort_pending.commit(std.testing.io, old_selection, .high));
try std.testing.expect(std.mem.indexOf(u8, root.tools_responses, jev_tool.name) == null);
var mimo = p;
@@ -81,9 +85,10 @@ test "native Jev catalog refreshes on eligible and ineligible model switches" {
}{});
_ = jev_tool.setCodegraffLoginKey(std.testing.io, "synthetic-login");
const old_catalog = root.toolsJson();
- const input = try std.json.parseFromSliceLeaky(std.json.Value, a, "{\"state\":\"10 tests passed\",\"question\":\"Did CI pass?\",\"type\":\"noul\"}", .{});
+ const input = try std.json.parseFromSliceLeaky(std.json.Value, a, "{\"task\":\"choose effort for a test fix\"}", .{});
var client: std.http.Client = undefined;
- const ctx: @import("tools.zig").ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = &client, .provider = eligible_openai, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
+ var pending: @import("jev_effort_state.zig").Pending = .{};
+ const ctx: @import("tools.zig").ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = &client, .provider = eligible_openai, .jev_effort_pending = &pending, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
const failed = try jev_tool.execute(ctx, input);
defer std.testing.allocator.free(failed.text);
const next_catalog = (try jev_tool.refreshCatalogForRequest(&root, old_catalog)).?;
diff --git a/src/jev_tool.zig b/src/jev_tool.zig
index 57cfe89d..26a4936d 100644
--- a/src/jev_tool.zig
+++ b/src/jev_tool.zig
@@ -1,4 +1,4 @@
-//! Native, opt-in Jev judgment tool. One failed upstream attempt opens a
+//! Native, opt-in Jev effort selector. One failed upstream attempt opens a
//! session-long circuit: later calls skip the network and return control to
//! the main model. No MCP process and no automatic context transfer.
const std = @import("std");
@@ -10,20 +10,22 @@ const ToolCtx = @import("tools.zig").ToolCtx;
const ToolOutput = @import("tools.zig").ToolOutput;
const ToolSpec = @import("schema.zig").ToolSpec;
const scope = @import("jev_model_scope.zig");
+const effort_route = @import("effort_route.zig");
+const ReasoningEffort = @import("main.zig").ReasoningEffort;
const pricing = @import("pricing.zig");
const buffered_https = @import("http2_buffered.zig");
-pub const name = "jev_judge";
-pub const description = "Ask Jev one closed-form judgment about a SHORT, NON-SENSITIVE state. Requires a Codegraff login and an eligible GPT-6 or MiMo v2.6 model. Use for yes/no (noul), a choice, or an ordered score, not writing or open-ended reasoning. Never send source code, secrets, customer data, paths, or unrelated context. A low-confidence verdict escalates to you. If Jev fails once, this tool skips all later Jev calls for this session; decide yourself instead.";
+pub const name = "jev_effort";
+pub const description = "Optionally ask Jev to choose the reasoning effort for the next request on an eligible GPT-6 or MiMo v2.6 model. Give only a short non-sensitive task summary. This changes the session effort; it does not judge an answer or action. Never send code, secrets, customer data, or paths. A failed or invalid selection leaves effort unchanged.";
pub const input_schema =
- \\{"type":"object","properties":{"state":{"type":"string","description":"Short non-sensitive facts needed for this judgment only; no code, paths or secrets"},"question":{"type":"string","description":"One closed-form question about state"},"type":{"type":"string","enum":["noul","choice","score"],"description":"noul=yes/no probability; choice=one option; score=ordered level"},"options":{"type":"array","items":{"type":"string"},"description":"Required for choice: 2-16 distinct labels"},"levels":{"type":"array","items":{"type":"string"},"description":"Required for score: 2-10 ordered descriptions"}},"required":["state","question","type"]}
+ \\{"type":"object","properties":{"task":{"type":"string","description":"Short non-sensitive summary of the next task; no code, paths or secrets"}},"required":["task"],"additionalProperties":false}
;
const spec = ToolSpec{ .name = name, .desc = description, .schema = input_schema };
const endpoint = "https://gateway.codegraff.com/v1/systemone";
-const skip_text = "Jev unavailable: skipped. No more Jev requests will be sent this session; judge this step with the main model instead.";
-const auth_skip_text = "Jev unavailable (Codegraff authorization failed): skipped. No more Jev requests will be sent this session; judge this step with the main model instead.";
-const credit_skip_text = "Jev unavailable (Codegraff credits or key budget): skipped. No more Jev requests will be sent this session; judge this step with the main model instead.";
-const rate_skip_text = "Jev unavailable (Codegraff rate limit): skipped. No more Jev requests will be sent this session; judge this step with the main model instead.";
+const skip_text = "Jev effort selection unavailable; session effort is unchanged. No more Jev requests will be sent this session.";
+const auth_skip_text = "Jev effort selection unavailable (Codegraff authorization failed); session effort is unchanged.";
+const credit_skip_text = "Jev effort selection unavailable (Codegraff credits or key budget); session effort is unchanged.";
+const rate_skip_text = "Jev effort selection unavailable (Codegraff rate limit); session effort is unchanged.";
const Backend = enum { gateway, mock, mock_fail };
const State = struct {
mu: Io.Mutex = .init,
@@ -66,6 +68,8 @@ pub fn available(provider: Provider) bool {
/// A model switch can change Jev visibility even when the wire format stays the same.
pub fn updateProvider(root: anytype, p: Provider) void {
+ // A switch away and back must not revive a selection from the old route.
+ root.jev_effort_pending.invalidate(root.io);
// Each wire format has its own cached catalog. An older catalog for the
// destination format may have been built before Jev became available.
if (available(root.provider) != available(p) or
@@ -111,42 +115,32 @@ fn number(v: Value) ?f64 {
};
}
-fn listValid(v: Value, max: usize) bool {
- if (v != .array or v.array.items.len < 2 or v.array.items.len > max) return false;
- for (v.array.items, 0..) |item, i| {
- const label = string(item) orelse return false;
- if (label.len == 0 or label.len > 120) return false;
- for (v.array.items[0..i]) |old| if (std.mem.eql(u8, old.string, label)) return false;
- }
- return true;
+fn effortDescription(tag: []const u8) []const u8 {
+ if (std.mem.eql(u8, tag, "none")) return "Off: no model reasoning";
+ if (std.mem.eql(u8, tag, "low")) return "Low: simple work";
+ if (std.mem.eql(u8, tag, "medium")) return "Medium: ordinary work";
+ if (std.mem.eql(u8, tag, "high")) return "High: complex work or MiMo thinking On";
+ if (std.mem.eql(u8, tag, "xhigh")) return "Extra high: demanding work";
+ return "Ultra: hardest work with delegation";
}
-fn makeBody(arena: Allocator, input: Value) ![]const u8 {
+fn makeBody(arena: Allocator, input: Value, provider: Provider) ![]const u8 {
const obj = input.object;
- const state_text = (obj.get("state") orelse return error.InvalidInput);
- const facts = string(state_text) orelse return error.InvalidInput;
- const question = string(obj.get("question") orelse return error.InvalidInput) orelse return error.InvalidInput;
- const kind = string(obj.get("type") orelse return error.InvalidInput) orelse return error.InvalidInput;
- if (facts.len == 0 or facts.len > 8192 or question.len == 0 or question.len > 512) return error.InvalidInput;
+ if (obj.count() != 1) return error.InvalidInput;
+ const task = string(obj.get("task") orelse return error.InvalidInput) orelse return error.InvalidInput;
+ if (task.len == 0 or task.len > 512) return error.InvalidInput;
var q = std.json.ObjectMap.empty;
- try q.put(arena, "type", .{ .string = kind });
- try q.put(arena, "instructions", .{ .string = question });
- if (std.mem.eql(u8, kind, "choice")) {
- const options = obj.get("options") orelse return error.InvalidInput;
- if (!listValid(options, 16)) return error.InvalidInput;
- var labels = std.json.ObjectMap.empty;
- for (options.array.items) |item| try labels.put(arena, item.string, item);
- try q.put(arena, "criteria", .{ .object = labels });
- } else if (std.mem.eql(u8, kind, "score")) {
- const levels = obj.get("levels") orelse return error.InvalidInput;
- if (!listValid(levels, 10)) return error.InvalidInput;
- try q.put(arena, "criteria", levels);
- } else if (!std.mem.eql(u8, kind, "noul")) return error.InvalidInput;
+ try q.put(arena, "type", .{ .string = "choice" });
+ try q.put(arena, "instructions", .{ .string = "Choose one reasoning effort for the next model request. Use only the supplied task summary and the fixed available levels. Prefer the least effort sufficient for the task." });
+ var labels = std.json.ObjectMap.empty;
+ for (effort_route.levels(provider.id, provider.model)) |tag|
+ try labels.put(arena, tag, .{ .string = effortDescription(tag) });
+ try q.put(arena, "criteria", .{ .object = labels });
var questions = std.json.ObjectMap.empty;
try questions.put(arena, "q1", .{ .object = q });
var root = std.json.ObjectMap.empty;
try root.put(arena, "model", .{ .string = "jev-latest" });
- try root.put(arena, "state", state_text);
+ try root.put(arena, "state", .{ .string = task });
try root.put(arena, "questions", .{ .object = questions });
var aw: Io.Writer.Allocating = .init(arena);
var serializer: std.json.Stringify = .{ .writer = &aw.writer };
@@ -154,14 +148,10 @@ fn makeBody(arena: Allocator, input: Value) ![]const u8 {
return aw.writer.buffered();
}
-fn mockResponse(arena: Allocator, input: Value) ![]const u8 {
- const kind = input.object.get("type").?.string;
- if (std.mem.eql(u8, kind, "noul")) return "{\"answers\":{\"q1\":{\"type\":\"noul\",\"noul\":0.98}}}";
- if (std.mem.eql(u8, kind, "score")) return "{\"answers\":{\"q1\":{\"type\":\"score\",\"score\":0.9,\"confidence\":0.95}}}";
- const label = input.object.get("options").?.array.items[0].string;
+fn mockResponse(arena: Allocator) ![]const u8 {
var aw: Io.Writer.Allocating = .init(arena);
var s: std.json.Stringify = .{ .writer = &aw.writer };
- try s.write(.{ .answers = .{ .q1 = .{ .type = "choice", .choice = label, .confidence = 0.95 } } });
+ try s.write(.{ .answers = .{ .q1 = .{ .type = "choice", .choice = "high", .confidence = 0.95 } } });
return aw.writer.buffered();
}
@@ -235,63 +225,47 @@ fn fetch(ctx: ToolCtx, arena: Allocator, body: []const u8) ![]const u8 {
return res.body;
}
-fn verdict(arena: Allocator, input: Value, raw: []const u8) ![]const u8 {
+fn selectedEffort(arena: Allocator, provider: Provider, raw: []const u8) !ReasoningEffort {
const parsed = try std.json.parseFromSliceLeaky(Value, arena, raw, .{ .allocate = .alloc_always });
if (parsed != .object) return error.InvalidResponse;
const answers = parsed.object.get("answers") orelse return error.InvalidResponse;
if (answers != .object) return error.InvalidResponse;
const answer = answers.object.get("q1") orelse return error.InvalidResponse;
if (answer != .object) return error.InvalidResponse;
- const kind = input.object.get("type").?.string;
const answer_type = string(answer.object.get("type") orelse return error.InvalidResponse) orelse return error.InvalidResponse;
- if (!std.mem.eql(u8, kind, answer_type)) return error.InvalidResponse;
- const value: Value = if (std.mem.eql(u8, kind, "noul")) answer.object.get("noul") orelse return error.InvalidResponse else if (std.mem.eql(u8, kind, "choice")) answer.object.get("choice") orelse return error.InvalidResponse else answer.object.get("score") orelse return error.InvalidResponse;
- if (std.mem.eql(u8, kind, "choice")) {
- const label = string(value) orelse return error.InvalidResponse;
- const options = input.object.get("options").?.array.items;
- var found = false;
- for (options) |opt| if (std.mem.eql(u8, opt.string, label)) {
- found = true;
- break;
- };
- if (!found) return error.InvalidResponse;
- } else {
- const n = number(value) orelse return error.InvalidResponse;
- const max: f64 = if (std.mem.eql(u8, kind, "score")) @floatFromInt(input.object.get("levels").?.array.items.len - 1) else 1;
- if (n < 0 or n > max) return error.InvalidResponse;
+ if (!std.mem.eql(u8, answer_type, "choice")) return error.InvalidResponse;
+ const label = string(answer.object.get("choice") orelse return error.InvalidResponse) orelse return error.InvalidResponse;
+ if (!effort_route.allows(provider.id, provider.model, label)) return error.InvalidResponse;
+ if (answer.object.get("confidence")) |reported| {
+ const confidence = number(reported) orelse return error.InvalidResponse;
+ if (confidence < 0 or confidence > 1) return error.InvalidResponse;
}
- const reported = if (answer.object.get("confidence")) |v| number(v) orelse return error.InvalidResponse else null;
- if (!std.mem.eql(u8, kind, "noul") and reported == null) return error.InvalidResponse;
- if (reported) |c| if (c < 0 or c > 1) return error.InvalidResponse;
- const estimated: f64 = if (std.mem.eql(u8, kind, "noul")) 2 * @abs(number(value).? - 0.5) else 0;
- const confidence = reported orelse estimated;
- const threshold: f64 = if (reported != null) 0.5 else 0.4;
- var aw: Io.Writer.Allocating = .init(arena);
- var s: std.json.Stringify = .{ .writer = &aw.writer };
- try s.write(.{ .answer = value, .confidence = confidence, .confidence_source = if (reported != null) "reported" else "estimated", .escalate = confidence < threshold });
- return aw.writer.buffered();
+ return std.meta.stringToEnum(ReasoningEffort, label) orelse error.InvalidResponse;
}
pub fn execute(ctx: ToolCtx, input: Value) !ToolOutput {
- if (ctx.from_sub) return invalid(ctx.gpa, "jev_judge is available only to the root agent");
- if (!state.codegraff_login.load(.acquire)) return invalid(ctx.gpa, "jev_judge requires a Codegraff login (`graff login`)");
- if (!scope.eligible(ctx.provider)) return invalid(ctx.gpa, "jev_judge is available only with Codex/OpenAI GPT-6 or Xiaomi MiMo v2.6 models");
+ if (ctx.from_sub) return invalid(ctx.gpa, "jev_effort is available only to the root agent");
+ if (!state.codegraff_login.load(.acquire)) return invalid(ctx.gpa, "jev_effort requires a Codegraff login (`graff login`)");
+ if (!scope.eligible(ctx.provider)) return invalid(ctx.gpa, "jev_effort is available only with Codex/OpenAI GPT-6 or Xiaomi MiMo v2.6 models");
if (state.down.load(.acquire)) return skipped(ctx.gpa);
- if (input != .object) return invalid(ctx.gpa, "jev_judge needs state, question and type");
+ if (input != .object) return invalid(ctx.gpa, "jev_effort needs one short task summary");
+ const pending = ctx.jev_effort_pending orelse return invalid(ctx.gpa, "jev_effort needs an active session");
var temp = std.heap.ArenaAllocator.init(ctx.gpa);
defer temp.deinit();
const arena = temp.allocator();
- const body = makeBody(arena, input) catch |err| switch (err) {
- error.InvalidInput => return invalid(ctx.gpa, "jev_judge needs a short state, one question, and valid noul/choice/score options"),
+ const body = makeBody(arena, input, ctx.provider) catch |err| switch (err) {
+ error.InvalidInput => return invalid(ctx.gpa, "jev_effort accepts only a short task summary"),
else => return err,
};
+ const token = pending.begin(ctx.io, ctx.provider) orelse return invalid(ctx.gpa, "jev_effort already has a selection in progress or awaiting application");
+ defer pending.abort(ctx.io, token);
state.mu.lockUncancelable(ctx.io);
defer state.mu.unlock(ctx.io);
- if (state.key.len == 0) return invalid(ctx.gpa, "jev_judge requires a Codegraff login (`graff login`)");
+ if (state.key.len == 0) return invalid(ctx.gpa, "jev_effort requires a Codegraff login (`graff login`)");
if (state.down.load(.acquire)) return skipped(ctx.gpa);
_ = state.attempts.fetchAdd(1, .acq_rel);
const raw = switch (state.backend) {
- .mock => try mockResponse(arena, input),
+ .mock => try mockResponse(arena),
.mock_fail => error.JevUnavailable,
.gateway => fetch(ctx, arena, body),
} catch |err| {
@@ -299,12 +273,13 @@ pub fn execute(ctx: ToolCtx, input: Value) !ToolOutput {
state.refresh.store(true, .release);
return .{ .text = try ctx.gpa.dupe(u8, skipForError(err)) };
};
- const result = verdict(arena, input, raw) catch {
+ const selected = selectedEffort(arena, ctx.provider, raw) catch {
state.down.store(true, .release);
state.refresh.store(true, .release);
return skipped(ctx.gpa);
};
- return .{ .text = try ctx.gpa.dupe(u8, result) };
+ if (!pending.commit(ctx.io, token, selected)) return invalid(ctx.gpa, "jev_effort selection was canceled");
+ return .{ .text = try std.fmt.allocPrint(ctx.gpa, "reasoning effort selected: {s}; applies at the next request boundary", .{@tagName(selected)}) };
}
test "native Jev failure makes exactly one attempt then skips without retry" {
@@ -322,8 +297,9 @@ test "native Jev failure makes exactly one attempt then skips without retry" {
_ = setCodegraffLoginKey(std.testing.io, "synthetic-login");
const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
var client: std.http.Client = undefined;
- const ctx: ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = &client, .provider = p, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
- const parsed = try std.json.parseFromSlice(Value, std.testing.allocator, "{\"state\":\"10 tests passed\",\"question\":\"Did CI pass?\",\"type\":\"noul\"}", .{});
+ var pending: @import("jev_effort_state.zig").Pending = .{};
+ const ctx: ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = &client, .provider = p, .jev_effort_pending = &pending, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
+ const parsed = try std.json.parseFromSlice(Value, std.testing.allocator, "{\"task\":\"choose effort for a small test fix\"}", .{});
defer parsed.deinit();
const first = try execute(ctx, parsed.value);
defer std.testing.allocator.free(first.text);
@@ -338,7 +314,7 @@ test "native Jev failure makes exactly one attempt then skips without retry" {
try std.testing.expect(!takeCatalogRefresh());
}
-test "native Jev mock uses SystemOne wire and returns a typed verdict" {
+test "native Jev mock sends fixed effort choices and queues the next effort" {
configure(struct {
fn get(_: @This(), key: []const u8) ?[]const u8 {
return if (std.mem.eql(u8, key, "JEV_BACKEND")) "mock" else null;
@@ -350,27 +326,29 @@ test "native Jev mock uses SystemOne wire and returns a typed verdict" {
}
}{});
_ = setCodegraffLoginKey(std.testing.io, "synthetic-login");
- const parsed = try std.json.parseFromSlice(Value, std.testing.allocator, "{\"state\":\"10 passed, 0 failed\",\"question\":\"Did CI pass?\",\"type\":\"noul\"}", .{});
+ const parsed = try std.json.parseFromSlice(Value, std.testing.allocator, "{\"task\":\"fix a complex failing test\"}", .{});
defer parsed.deinit();
var temp = std.heap.ArenaAllocator.init(std.testing.allocator);
defer temp.deinit();
const arena = temp.allocator();
- const body = try makeBody(arena, parsed.value);
+ const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
+ const body = try makeBody(arena, parsed.value, p);
const wire = try std.json.parseFromSliceLeaky(Value, arena, body, .{});
try std.testing.expectEqualStrings("jev-latest", wire.object.get("model").?.string);
- try std.testing.expectEqualStrings("10 passed, 0 failed", wire.object.get("state").?.string);
+ try std.testing.expectEqualStrings("fix a complex failing test", wire.object.get("state").?.string);
const q1 = wire.object.get("questions").?.object.get("q1").?.object;
- try std.testing.expectEqualStrings("Did CI pass?", q1.get("instructions").?.string);
- const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
+ try std.testing.expectEqualStrings("choice", q1.get("type").?.string);
+ try std.testing.expectEqual(@as(usize, 5), q1.get("criteria").?.object.count());
+ try std.testing.expect(q1.get("criteria").?.object.get("high") != null);
var client: std.http.Client = undefined;
- const ctx: ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = &client, .provider = p, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
+ var pending: @import("jev_effort_state.zig").Pending = .{};
+ const ctx: ToolCtx = .{ .gpa = std.testing.allocator, .io = std.testing.io, .client = &client, .provider = p, .jev_effort_pending = &pending, .registry = null, .from_sub = false, .approvals = null, .tracer = null };
const before = pricing.g_cost.snap(std.testing.io);
const out = try execute(ctx, parsed.value);
defer std.testing.allocator.free(out.text);
try std.testing.expect(!out.is_error);
- const result = try std.json.parseFromSliceLeaky(Value, arena, out.text, .{});
- try std.testing.expect(@abs(number(result.object.get("answer").?).? - 0.98) < 0.0001);
- try std.testing.expect(!result.object.get("escalate").?.bool);
+ try std.testing.expect(std.mem.indexOf(u8, out.text, "high") != null);
+ try std.testing.expectEqual(ReasoningEffort.high, pending.take(std.testing.io, p).?);
try std.testing.expectEqual(@as(usize, 1), state.attempts.load(.acquire));
const after = pricing.g_cost.snap(std.testing.io);
try std.testing.expectEqual(before.api_calls, after.api_calls);
@@ -378,16 +356,27 @@ test "native Jev mock uses SystemOne wire and returns a typed verdict" {
try std.testing.expectEqual(before.unreported_failed_attempts, after.unreported_failed_attempts);
}
-test "native Jev rejects out-of-range scores and malformed confidence" {
+test "native Jev rejects arbitrary judgments and unsupported or malformed effort" {
var temp = std.heap.ArenaAllocator.init(std.testing.allocator);
defer temp.deinit();
const a = temp.allocator();
- const input = try std.json.parseFromSliceLeaky(Value, a, "{\"state\":\"build passed\",\"question\":\"How complete?\",\"type\":\"score\",\"levels\":[\"none\",\"some\",\"all\"]}", .{});
- try std.testing.expectError(error.InvalidResponse, verdict(a, input, "{\"answers\":{\"q1\":{\"type\":\"score\",\"score\":2.5,\"confidence\":0.9}}}"));
- try std.testing.expectError(error.InvalidResponse, verdict(a, input, "{\"answers\":{\"q1\":{\"type\":\"score\",\"score\":1.5,\"confidence\":\"certain\"}}}"));
- try std.testing.expectError(error.InvalidResponse, verdict(a, input, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"score\":1.5,\"confidence\":0.9}}}"));
- const good = try verdict(a, input, "{\"answers\":{\"q1\":{\"type\":\"score\",\"score\":1.5,\"confidence\":0.9}}}");
- try std.testing.expect(std.mem.indexOf(u8, good, "\"answer\":1.5") != null);
+ const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
+ const arbitrary = try std.json.parseFromSliceLeaky(Value, a, "{\"state\":\"build passed\",\"question\":\"Did CI pass?\",\"type\":\"noul\"}", .{});
+ try std.testing.expectError(error.InvalidInput, makeBody(a, arbitrary, p));
+ try std.testing.expectError(error.InvalidResponse, selectedEffort(a, p, "{\"answers\":{\"q1\":{\"type\":\"noul\",\"noul\":0.9}}}"));
+ try std.testing.expectError(error.InvalidResponse, selectedEffort(a, p, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"none\",\"confidence\":0.9}}}"));
+ try std.testing.expectError(error.InvalidResponse, selectedEffort(a, p, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"high\",\"confidence\":\"certain\"}}}"));
+ try std.testing.expectEqual(ReasoningEffort.high, try selectedEffort(a, p, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"high\"}}}"));
+ try std.testing.expectEqual(ReasoningEffort.high, try selectedEffort(a, p, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"high\",\"confidence\":0.9}}}"));
+ const mimo: Provider = .{ .id = "xiaomi", .kind = .openai, .auth = .bearer, .url = "", .api_key = "", .model = "mimo-v2.6-flash", .context = 100_000 };
+ const task = try std.json.parseFromSliceLeaky(Value, a, "{\"task\":\"answer a trivial question\"}", .{});
+ const body = try makeBody(a, task, mimo);
+ const wire = try std.json.parseFromSliceLeaky(Value, a, body, .{});
+ const criteria = wire.object.get("questions").?.object.get("q1").?.object.get("criteria").?.object;
+ try std.testing.expectEqual(@as(usize, 2), criteria.count());
+ try std.testing.expect(criteria.get("none") != null and criteria.get("high") != null);
+ try std.testing.expectEqual(ReasoningEffort.none, try selectedEffort(a, mimo, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"none\"}}}"));
+ try std.testing.expectError(error.InvalidResponse, selectedEffort(a, mimo, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"low\"}}}"));
}
test "native Jev gateway uses persisted login independent of chat provider and upstream key" {
@@ -423,11 +412,8 @@ test "native Jev reports safe gateway failures and does not invent usage or cost
var temp = std.heap.ArenaAllocator.init(std.testing.allocator);
defer temp.deinit();
const a = temp.allocator();
- const input = try std.json.parseFromSliceLeaky(Value, a, "{\"state\":\"build passed\",\"question\":\"Did CI pass?\",\"type\":\"noul\"}", .{});
- const out = try verdict(a, input, "{\"answers\":{\"q1\":{\"type\":\"noul\",\"noul\":0.98}},\"usage\":{\"input_tokens\":20,\"output_tokens\":5}}");
- const parsed = try std.json.parseFromSliceLeaky(Value, a, out, .{});
- try std.testing.expect(parsed.object.get("usage") == null);
- try std.testing.expect(parsed.object.get("cost") == null);
+ const p: Provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6-sol", .context = 100_000 };
+ try std.testing.expectEqual(ReasoningEffort.high, try selectedEffort(a, p, "{\"answers\":{\"q1\":{\"type\":\"choice\",\"choice\":\"high\",\"confidence\":0.9}},\"usage\":{\"input_tokens\":20,\"output_tokens\":5}}"));
}
test "native Jev gateway usage preserves tokens but marks unsettled cost unknown" {
diff --git a/src/main.zig b/src/main.zig
index b6c304d8..2c6a6ac1 100644
--- a/src/main.zig
+++ b/src/main.zig
@@ -559,6 +559,7 @@ const workflow = @import("workflow_test.zig"); // the engine's tests live here (
const exec = @import("exec.zig");
test { // ── Unit tests (`zig build test`): pull in tests from imported modules (mcp.zig)
_ = .{ @import("jev_tool.zig"), @import("jev_model_scope.zig"), @import("jev_provider_switch_tests.zig") };
+ _ = @import("jev_effort_state.zig");
_ = .{ @import("mcp_apps.zig"), @import("html_view.zig"), @import("providers_confirmation_tests.zig"), @import("repl_model_confirmation_tests.zig"), @import("repl_model_pick_tests.zig"), @import("engine_sink_citation_tests.zig"), @import("cite_markup_stream_tests.zig"), @import("oneshot_citation_tests.zig"), @import("tui_acp.zig"), @import("acp_permission.zig"), @import("acp_usage.zig") };
_ = .{ @import("mcp_server_tests.zig"), @import("main_test.zig"), @import("artifact_claim_command.zig"), @import("cli_path_hint.zig"), @import("run_budget.zig"), @import("pr_review_input.zig"), @import("pr_claim_review.zig"), @import("tui_paste_transfer.zig") };
_ = .{ @import("session_catalog.zig"), @import("session_prompt.zig"), @import("session_discovery.zig") };
diff --git a/src/mainloop.zig b/src/mainloop.zig
index 6be5a599..9ef77dab 100644
--- a/src/mainloop.zig
+++ b/src/mainloop.zig
@@ -1,5 +1,4 @@
//! Root interactive/JSON event loop. `main` owns setup; `Ctx` is borrowed until `run` returns.
-
const std = @import("std");
const Io = std.Io;
const Value = std.json.Value;
@@ -242,6 +241,7 @@ pub fn run(ctx: *Ctx) !void {
ctx.root.emit(.{ .type = "error", .message = "set_effort needs a level supported by the current model" });
continue;
};
+ ctx.root.jev_effort_pending.invalidate(ctx.root.io);
_ = repl_glue.saveThinkingSettings(ctx.root.io, ctx.root.gpa, ctx.root.reasoning, ctx.root.fast, ctx.root.ultracode_mode, ctx.root.show_thinking, ctx.root.ai_title);
ctx.root.emit(.{ .type = "effort", .ok = true, .level = @tagName(ctx.root.reasoning), .applies = ctx.root.effortApplies() });
continue;
diff --git a/src/tools.zig b/src/tools.zig
index e6d4e203..5d1b977d 100644
--- a/src/tools.zig
+++ b/src/tools.zig
@@ -102,6 +102,7 @@ pub const ToolCtx = struct {
io: Io,
client: *std.http.Client,
provider: Provider,
+ jev_effort_pending: ?*@import("jev_effort_state.zig").Pending = null,
subagent_provider: ?Provider = null,
subagent_cross_provider: bool = false,
subagent_feedback: ?*@import("subagent_feedback.zig").Inbox = null,