CubeLoop — a Pythonic, async-native agent harness core designed for high performance, readability, and production-grade persistence.
Python agent-frameworkagent-harnessagent-runtimeagentic-aiai-agentsai-frameworkanthropicharnessharness-engineeringharness-frameworklanggraphlanggraph-alternativellmmcpopenaipi-agentpi-agent-corepythonpython-agent
docs: add GoalMiddleware, structured output, and tool_choice guides
xf gong0bcd09b6c91e
diff --git a/website/docs/guides/middleware/goal.md b/website/docs/guides/middleware/goal.mdnew file mode 100644index 0000000..6595805--- /dev/null+++ b/website/docs/guides/middleware/goal.md@@ -0,0 +1,138 @@+---+title: Goal+description: "Use GoalMiddleware to keep an agent working until an independent evaluator model confirms a completion condition is met."+---++# Goal++`GoalMiddleware` keeps an agent running autonomously until a completion+condition is met. A separate evaluator model (e.g. Haiku) judges+whether the condition has been achieved — the worker model is not grading+its own homework.++Inspired by Claude Code's `/goal` command: set a condition, let the agent+work, stop when a second model says the condition is satisfied.++## Basic setup++```python+from cubepi import Agent+from cubepi.providers.anthropic import AnthropicProvider+from cubepi.middleware.goal import GoalMiddleware++provider = AnthropicProvider(api_key="...")++goal = GoalMiddleware(+ evaluator=provider.model("claude-haiku-4-5-20251001"),+ max_evaluations=10,+)++agent = Agent(+ model=provider.model("claude-sonnet-4-6"),+ middleware=[goal],+ tools=[...],+)++# /goal prefix activates goal mode+await agent.prompt("/goal all tests in tests/auth pass and ruff check is clean")++# Check outcome+print(agent.state.extra["goal"])+# {"status": "achieved", "condition": "all tests in ...", "evaluations": 2, ...}+```++## How it works++### Activation++GoalMiddleware activates when the user message starts with `/goal `.+Everything after the prefix is the completion condition.++```python+# Goal mode — evaluator will judge this condition+await agent.prompt("/goal make the homepage load in under 2 seconds")++# Normal mode — middleware is fully transparent+await agent.prompt("fix the bug in auth.py")+```++The `/goal` prefix is stripped before the worker sees the message. The+worker receives only the condition text as its work directive.++### Evaluation loop++After the worker finishes a complete run (all tool calls exhausted):++1. The evaluator reads the condition + last 20 messages from the conversation.+2. It returns `{achieved: bool, reason: str}` via structured output.+3. If **achieved** — the loop ends, status is `"achieved"`.+4. If **not achieved** and evaluations remain — feedback is injected+ (`"Goal not yet met: {reason}. Continue working."`) and the worker+ resumes.+5. If **max_evaluations** reached — the loop ends, status is `"exhausted"`.++```+Worker run → on_run_end → evaluator judges+ ├─ achieved=True → stop (status: "achieved")+ ├─ achieved=False → inject feedback, continue+ └─ max evals hit → stop (status: "exhausted")+```++## Parameters++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `evaluator` | `BoundModel` | required | Model that judges the condition |+| `max_evaluations` | `int` | `10` | Safety cap on evaluator calls |++Use a fast, cheap model for the evaluator (e.g. Haiku). It only reads+the conversation transcript — it cannot call tools.++## Reading the outcome++After `agent.prompt()` returns, check `agent.state.extra["goal"]`:++```python+goal_state = agent.state.extra["goal"]++match goal_state["status"]:+ case "achieved":+ print(f"Done in {goal_state['evaluations']} evaluations")+ print(f"Reason: {goal_state['last_reason']}")+ case "exhausted":+ print(f"Gave up after {goal_state['max_evaluations']} evaluations")+ print(f"Last reason: {goal_state['last_reason']}")+```++Full state shape:++```python+{+ "status": "achieved" | "active" | "exhausted",+ "condition": "all tests pass...",+ "evaluations": 3,+ "max_evaluations": 10,+ "last_reason": "2 tests still failing in test_auth.py",+}+```++## Tracing++GoalMiddleware declares its evaluator via `extra_llm_calls()`, so the+tracing Recorder can subscribe to the evaluator's provider and attribute+evaluation spans correctly — they show up as evaluator calls, not worker+calls.++## Tips++- **Keep conditions specific and verifiable.** "All tests pass" is+ better than "code works well." The evaluator judges from transcript+ text, not by running tools.+- **Use a cheap evaluator.** Haiku is usually sufficient — the judgment+ is binary (achieved/not) with a short reason.+- **Set `max_evaluations` conservatively.** The default of 10 prevents+ runaway loops. Increase only when you expect the agent to need many+ iterations.+- **Combine with other middleware.** GoalMiddleware composes with+ `TodoListMiddleware`, compaction, etc. via standard middleware+ composition.diff --git a/website/docs/guides/middleware/hooks.md b/website/docs/guides/middleware/hooks.mdindex 125840b..cf49486 100644--- a/website/docs/guides/middleware/hooks.md+++ b/website/docs/guides/middleware/hooks.md@@ -207,14 +207,16 @@ async def on_run_end(...```-Fires **once per `prompt()` call**, after all turns and tool calls-complete, before `AgentEndEvent` is emitted. Return a non-empty-`list[Message]` to inject those messages into context and run **one-extra model turn** (the run-end pass). Return `None` or `[]` to do-nothing.+Fires **after each outer-loop iteration** — i.e. after all turns and+tool calls complete before the loop would normally exit. Return a+non-empty `list[Message]` to inject those messages into context and+continue the loop (the agent runs again). Return `None` or `[]` to do+nothing (the loop exits).-The extra turn fires exactly once — a `_reflection_fired` guard in the-loop prevents the injected turn from triggering another `on_run_end`.+This hook can fire **multiple times** per `prompt()` call. Each time+the middleware returns messages, the worker gets another run. Return+`None` to stop the cycle. This makes `on_run_end` suitable for+evaluation loops — see [GoalMiddleware](./goal) for an example.**When it fires:**- Normal completion (loop breaks naturally after all turns).@@ -226,11 +228,12 @@ loop prevents the injected turn from triggering another `on_run_end`.- HITL interruption (`HitlDetached` / `HitlAborted`) — the run ispaused, not finished.-Use for: post-run memory consolidation, conversation summarisation,-audit logging that needs the full turn context.+Use for: goal evaluation loops, post-run memory consolidation,+conversation summarisation, audit logging that needs the full turn+context.Composition: messages from all middleware **concatenate** into a single-list; all are injected together before the extra model turn.+list; all are injected together before the next model turn.## Anatomy of a middlewarediff --git a/website/docs/guides/providers/structured-output.md b/website/docs/guides/providers/structured-output.mdnew file mode 100644index 0000000..a5d71d2--- /dev/null+++ b/website/docs/guides/providers/structured-output.md@@ -0,0 +1,112 @@+---+title: Structured Output+description: "Get validated Pydantic models from LLM calls with generate_structured() and tool_choice."+---++# Structured Output++`BoundModel.generate_structured()` extracts typed, validated data from+an LLM call. Pass a Pydantic model class and get a validated instance+back — no JSON parsing, no schema wrangling.++```python+from pydantic import BaseModel+from cubepi.providers.base import TextContent, UserMessage++class Sentiment(BaseModel):+ label: str+ confidence: float++result = await model.generate_structured(+ Sentiment,+ messages=[UserMessage(content=[TextContent(text="I love this product!")])],+)+print(result) # label='positive' confidence=0.95+```++## How it works++Under the hood, `generate_structured()`:++1. Converts the Pydantic model's JSON schema into a synthetic tool definition.+2. Calls `generate()` with `tools=[synthetic_tool]` and+ `tool_choice=tool_name` to force the model to call that tool.+3. Extracts the `ToolCall.arguments` from the response.+4. Validates through `output_type.model_validate()`.++This is the same approach pydantic-ai uses by default (`ToolOutput` mode).+It works across all providers because every LLM API supports tool calling.++## Parameters++```python+await model.generate_structured(+ output_type, # Pydantic model class (required)+ messages, # list[Message] (required)+ *,+ system_prompt="", # Optional system prompt (a tool-use hint is always appended)+ tool_name="structured_output",+ tool_description="Return the structured output",+ max_output_tokens=None,+ temperature=None,+ max_retries=1, # Retries on validation failure (feeds error back to model)+)+```++| Parameter | Default | Description |+|-----------|---------|-------------|+| `output_type` | required | Pydantic `BaseModel` subclass |+| `messages` | required | Conversation messages |+| `system_prompt` | `""` | Custom system prompt (tool-use hint always appended) |+| `tool_name` | `"structured_output"` | Name of the synthetic tool |+| `tool_description` | `"Return the structured output"` | Tool description sent to model |+| `max_retries` | `1` | Retries on Pydantic validation failure |+| `max_output_tokens` | `None` | Override model's default max tokens |+| `temperature` | `None` | Override model's default temperature |++## Error handling++`generate_structured()` raises `StructuredOutputError` in two cases:++```python+from cubepi.providers.base import StructuredOutputError++try:+ result = await model.generate_structured(MySchema, messages=[...])+except StructuredOutputError as e:+ print(e) # "no tool call" or "validation failed after retries"+```++- **No tool call**: the model returned text instead of calling the tool.+- **Validation failed**: Pydantic validation failed on all attempts+ (initial + retries).++On validation failure, the error is fed back to the model as a+`UserMessage` and it gets another chance (up to `max_retries` times).++## `tool_choice`++`generate_structured()` uses `tool_choice` internally to force the model+to call the synthetic tool. You can also use `tool_choice` directly on+`stream()` and `generate()`:++```python+reply = await model.generate(+ messages=[...],+ tools=[my_tool_def],+ tool_choice="my_tool", # Force this specific tool+)+```++Values:++| Value | Behavior |+|-------|----------|+| `None` | Provider default (model decides) |+| `"auto"` | Model decides whether to call a tool |+| `"required"` | Must call some tool |+| `"none"` | No tool calls allowed |+| `"tool_name"` | Force a specific tool by name |++`tool_choice` works on all built-in providers (Anthropic, OpenAI, OpenAI+Responses). Each provider maps the value to its native wire format.diff --git a/website/sidebars.ts b/website/sidebars.tsindex 8a89df5..aeb8020 100644--- a/website/sidebars.ts+++ b/website/sidebars.ts@@ -29,6 +29,7 @@ const sidebars: SidebarsConfig = {'guides/providers/overview','guides/providers/anthropic','guides/providers/openai',+ 'guides/providers/structured-output','guides/providers/custom','guides/providers/image-generation',]},@@ -46,6 +47,7 @@ const sidebars: SidebarsConfig = {'guides/middleware/subagents','guides/middleware/todo','guides/middleware/deferred-tools',+ 'guides/middleware/goal',]},{ type: 'category', label: 'Human-in-the-Loop', items: ['guides/hitl/overview',