Skip to content

CubeLoop — a Pythonic, async-native agent harness core designed for high performance, readability, and production-grade persistence.

docs: add GoalMiddleware, structured output, and tool_choice guides

xf gong0bcd09b6c91e
diff --git a/website/docs/guides/middleware/goal.md b/website/docs/guides/middleware/goal.md
new file mode 100644
index 0000000..6595805
--- /dev/null
+++ b/website/docs/guides/middleware/goal.md
@@ -0,0 +1,138 @@
+---
+title: Goal
+description: "Use GoalMiddleware to keep an agent working until an independent evaluator model confirms a completion condition is met."
+---
+
+# Goal
+
+`GoalMiddleware` keeps an agent running autonomously until a completion
+condition is met. A separate evaluator model (e.g. Haiku) judges
+whether the condition has been achieved — the worker model is not grading
+its own homework.
+
+Inspired by Claude Code's `/goal` command: set a condition, let the agent
+work, stop when a second model says the condition is satisfied.
+
+## Basic setup
+
+```python
+from cubepi import Agent
+from cubepi.providers.anthropic import AnthropicProvider
+from cubepi.middleware.goal import GoalMiddleware
+
+provider = AnthropicProvider(api_key="...")
+
+goal = GoalMiddleware(
+ evaluator=provider.model("claude-haiku-4-5-20251001"),
+ max_evaluations=10,
+)
+
+agent = Agent(
+ model=provider.model("claude-sonnet-4-6"),
+ middleware=[goal],
+ tools=[...],
+)
+
+# /goal prefix activates goal mode
+await agent.prompt("/goal all tests in tests/auth pass and ruff check is clean")
+
+# Check outcome
+print(agent.state.extra["goal"])
+# {"status": "achieved", "condition": "all tests in ...", "evaluations": 2, ...}
+```
+
+## How it works
+
+### Activation
+
+GoalMiddleware activates when the user message starts with `/goal `.
+Everything after the prefix is the completion condition.
+
+```python
+# Goal mode — evaluator will judge this condition
+await agent.prompt("/goal make the homepage load in under 2 seconds")
+
+# Normal mode — middleware is fully transparent
+await agent.prompt("fix the bug in auth.py")
+```
+
+The `/goal` prefix is stripped before the worker sees the message. The
+worker receives only the condition text as its work directive.
+
+### Evaluation loop
+
+After the worker finishes a complete run (all tool calls exhausted):
+
+1. The evaluator reads the condition + last 20 messages from the conversation.
+2. It returns `{achieved: bool, reason: str}` via structured output.
+3. If **achieved** — the loop ends, status is `"achieved"`.
+4. If **not achieved** and evaluations remain — feedback is injected
+ (`"Goal not yet met: {reason}. Continue working."`) and the worker
+ resumes.
+5. If **max_evaluations** reached — the loop ends, status is `"exhausted"`.
+
+```
+Worker run → on_run_end → evaluator judges
+ ├─ achieved=True → stop (status: "achieved")
+ ├─ achieved=False → inject feedback, continue
+ └─ max evals hit → stop (status: "exhausted")
+```
+
+## Parameters
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `evaluator` | `BoundModel` | required | Model that judges the condition |
+| `max_evaluations` | `int` | `10` | Safety cap on evaluator calls |
+
+Use a fast, cheap model for the evaluator (e.g. Haiku). It only reads
+the conversation transcript — it cannot call tools.
+
+## Reading the outcome
+
+After `agent.prompt()` returns, check `agent.state.extra["goal"]`:
+
+```python
+goal_state = agent.state.extra["goal"]
+
+match goal_state["status"]:
+ case "achieved":
+ print(f"Done in {goal_state['evaluations']} evaluations")
+ print(f"Reason: {goal_state['last_reason']}")
+ case "exhausted":
+ print(f"Gave up after {goal_state['max_evaluations']} evaluations")
+ print(f"Last reason: {goal_state['last_reason']}")
+```
+
+Full state shape:
+
+```python
+{
+ "status": "achieved" | "active" | "exhausted",
+ "condition": "all tests pass...",
+ "evaluations": 3,
+ "max_evaluations": 10,
+ "last_reason": "2 tests still failing in test_auth.py",
+}
+```
+
+## Tracing
+
+GoalMiddleware declares its evaluator via `extra_llm_calls()`, so the
+tracing Recorder can subscribe to the evaluator's provider and attribute
+evaluation spans correctly — they show up as evaluator calls, not worker
+calls.
+
+## Tips
+
+- **Keep conditions specific and verifiable.** "All tests pass" is
+ better than "code works well." The evaluator judges from transcript
+ text, not by running tools.
+- **Use a cheap evaluator.** Haiku is usually sufficient — the judgment
+ is binary (achieved/not) with a short reason.
+- **Set `max_evaluations` conservatively.** The default of 10 prevents
+ runaway loops. Increase only when you expect the agent to need many
+ iterations.
+- **Combine with other middleware.** GoalMiddleware composes with
+ `TodoListMiddleware`, compaction, etc. via standard middleware
+ composition.
diff --git a/website/docs/guides/middleware/hooks.md b/website/docs/guides/middleware/hooks.md
index 125840b..cf49486 100644
--- a/website/docs/guides/middleware/hooks.md
+++ b/website/docs/guides/middleware/hooks.md
@@ -207,14 +207,16 @@ async def on_run_end(
...
```
-Fires **once per `prompt()` call**, after all turns and tool calls
-complete, before `AgentEndEvent` is emitted. Return a non-empty
-`list[Message]` to inject those messages into context and run **one
-extra model turn** (the run-end pass). Return `None` or `[]` to do
-nothing.
+Fires **after each outer-loop iteration** — i.e. after all turns and
+tool calls complete before the loop would normally exit. Return a
+non-empty `list[Message]` to inject those messages into context and
+continue the loop (the agent runs again). Return `None` or `[]` to do
+nothing (the loop exits).
-The extra turn fires exactly once — a `_reflection_fired` guard in the
-loop prevents the injected turn from triggering another `on_run_end`.
+This hook can fire **multiple times** per `prompt()` call. Each time
+the middleware returns messages, the worker gets another run. Return
+`None` to stop the cycle. This makes `on_run_end` suitable for
+evaluation loops — see [GoalMiddleware](./goal) for an example.
**When it fires:**
- Normal completion (loop breaks naturally after all turns).
@@ -226,11 +228,12 @@ loop prevents the injected turn from triggering another `on_run_end`.
- HITL interruption (`HitlDetached` / `HitlAborted`) — the run is
paused, not finished.
-Use for: post-run memory consolidation, conversation summarisation,
-audit logging that needs the full turn context.
+Use for: goal evaluation loops, post-run memory consolidation,
+conversation summarisation, audit logging that needs the full turn
+context.
Composition: messages from all middleware **concatenate** into a single
-list; all are injected together before the extra model turn.
+list; all are injected together before the next model turn.
## Anatomy of a middleware
diff --git a/website/docs/guides/providers/structured-output.md b/website/docs/guides/providers/structured-output.md
new file mode 100644
index 0000000..a5d71d2
--- /dev/null
+++ b/website/docs/guides/providers/structured-output.md
@@ -0,0 +1,112 @@
+---
+title: Structured Output
+description: "Get validated Pydantic models from LLM calls with generate_structured() and tool_choice."
+---
+
+# Structured Output
+
+`BoundModel.generate_structured()` extracts typed, validated data from
+an LLM call. Pass a Pydantic model class and get a validated instance
+back — no JSON parsing, no schema wrangling.
+
+```python
+from pydantic import BaseModel
+from cubepi.providers.base import TextContent, UserMessage
+
+class Sentiment(BaseModel):
+ label: str
+ confidence: float
+
+result = await model.generate_structured(
+ Sentiment,
+ messages=[UserMessage(content=[TextContent(text="I love this product!")])],
+)
+print(result) # label='positive' confidence=0.95
+```
+
+## How it works
+
+Under the hood, `generate_structured()`:
+
+1. Converts the Pydantic model's JSON schema into a synthetic tool definition.
+2. Calls `generate()` with `tools=[synthetic_tool]` and
+ `tool_choice=tool_name` to force the model to call that tool.
+3. Extracts the `ToolCall.arguments` from the response.
+4. Validates through `output_type.model_validate()`.
+
+This is the same approach pydantic-ai uses by default (`ToolOutput` mode).
+It works across all providers because every LLM API supports tool calling.
+
+## Parameters
+
+```python
+await model.generate_structured(
+ output_type, # Pydantic model class (required)
+ messages, # list[Message] (required)
+ *,
+ system_prompt="", # Optional system prompt (a tool-use hint is always appended)
+ tool_name="structured_output",
+ tool_description="Return the structured output",
+ max_output_tokens=None,
+ temperature=None,
+ max_retries=1, # Retries on validation failure (feeds error back to model)
+)
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `output_type` | required | Pydantic `BaseModel` subclass |
+| `messages` | required | Conversation messages |
+| `system_prompt` | `""` | Custom system prompt (tool-use hint always appended) |
+| `tool_name` | `"structured_output"` | Name of the synthetic tool |
+| `tool_description` | `"Return the structured output"` | Tool description sent to model |
+| `max_retries` | `1` | Retries on Pydantic validation failure |
+| `max_output_tokens` | `None` | Override model's default max tokens |
+| `temperature` | `None` | Override model's default temperature |
+
+## Error handling
+
+`generate_structured()` raises `StructuredOutputError` in two cases:
+
+```python
+from cubepi.providers.base import StructuredOutputError
+
+try:
+ result = await model.generate_structured(MySchema, messages=[...])
+except StructuredOutputError as e:
+ print(e) # "no tool call" or "validation failed after retries"
+```
+
+- **No tool call**: the model returned text instead of calling the tool.
+- **Validation failed**: Pydantic validation failed on all attempts
+ (initial + retries).
+
+On validation failure, the error is fed back to the model as a
+`UserMessage` and it gets another chance (up to `max_retries` times).
+
+## `tool_choice`
+
+`generate_structured()` uses `tool_choice` internally to force the model
+to call the synthetic tool. You can also use `tool_choice` directly on
+`stream()` and `generate()`:
+
+```python
+reply = await model.generate(
+ messages=[...],
+ tools=[my_tool_def],
+ tool_choice="my_tool", # Force this specific tool
+)
+```
+
+Values:
+
+| Value | Behavior |
+|-------|----------|
+| `None` | Provider default (model decides) |
+| `"auto"` | Model decides whether to call a tool |
+| `"required"` | Must call some tool |
+| `"none"` | No tool calls allowed |
+| `"tool_name"` | Force a specific tool by name |
+
+`tool_choice` works on all built-in providers (Anthropic, OpenAI, OpenAI
+Responses). Each provider maps the value to its native wire format.
diff --git a/website/sidebars.ts b/website/sidebars.ts
index 8a89df5..aeb8020 100644
--- a/website/sidebars.ts
+++ b/website/sidebars.ts
@@ -29,6 +29,7 @@ const sidebars: SidebarsConfig = {
'guides/providers/overview',
'guides/providers/anthropic',
'guides/providers/openai',
+ 'guides/providers/structured-output',
'guides/providers/custom',
'guides/providers/image-generation',
]},
@@ -46,6 +47,7 @@ const sidebars: SidebarsConfig = {
'guides/middleware/subagents',
'guides/middleware/todo',
'guides/middleware/deferred-tools',
+ 'guides/middleware/goal',
]},
{ type: 'category', label: 'Human-in-the-Loop', items: [
'guides/hitl/overview',