CubeLoop — a Pythonic, async-native agent harness core designed for high performance, readability, and production-grade persistence.
Python agent-frameworkagent-harnessagent-runtimeagentic-aiai-agentsai-frameworkanthropicharnessharness-engineeringharness-frameworklanggraphlanggraph-alternativellmmcpopenaipi-agentpi-agent-corepythonpython-agent
plan: merge Task 3+4 into one atomic compaction migration
xf gongfbab36017184
diff --git a/dev/plans/2026-06-08-boundmodel-convenience-methods.md b/dev/plans/2026-06-08-boundmodel-convenience-methods.mdindex 5d3a67c..ebbc0eb 100644--- a/dev/plans/2026-06-08-boundmodel-convenience-methods.md+++ b/dev/plans/2026-06-08-boundmodel-convenience-methods.md@@ -26,7 +26,7 @@- **Modify** `tests/tracing/test_recorder.py` — three middleware mocks (`extra_llm_calls` at lines ~666, ~706, ~735) return `BoundModel` instead of tuples.- **Modify** `website/docs/agents/providers.md` (or whichever current providers page documents `provider.model(...)`) — add a one-paragraph note that the returned `BoundModel` is itself callable via `.stream()` / `.generate()`.-`tests/middleware/test_compaction.py` already constructs `CompactionMiddleware(summary_model=BoundModel(...), ...)` (line 75), so the public-API surface needs no test changes there. Recorder tests at `tests/tracing/test_recorder.py:515` / `:591` likewise construct `BoundModel(...)` directly and stay green through both Task 3 (internal call-site change) and Task 4 (field collapse + extra_llm_calls shape change).+`tests/middleware/test_compaction.py` already constructs `CompactionMiddleware(summary_model=BoundModel(...), ...)` (line 75), so the public-API surface needs no test changes there. Recorder tests at `tests/tracing/test_recorder.py:515` / `:591` likewise already construct `BoundModel(...)` directly and stay green after the Task 3 atomic flip.---@@ -285,26 +285,37 @@ git commit -m "feat(providers): add BoundModel.stream convenience method"----## Task 3: Migrate `summarize()` to take a `BoundModel` (signature-only)+## Task 3: Migrate compaction + `extra_llm_calls()` to `BoundModel` (atomic)-**Files:**-- Modify: `cubepi/middleware/compaction/summarizer.py:55-80`-- Modify: `cubepi/middleware/compaction/__init__.py` (call site inside `transform_context` only — `__init__` and `extra_llm_calls` untouched)+**Files (all flip in one commit):**+- Modify: `cubepi/middleware/compaction/summarizer.py` (function signature + body, imports)+- Modify: `cubepi/middleware/compaction/__init__.py` (`__init__`, `transform_context` call site, `extra_llm_calls`, imports)+- Modify: `cubepi/middleware/base.py:96-116` (`extra_llm_calls` signature + docstring, imports)+- Modify: `cubepi/tracing/recorder.py:286-299` (the unpacking loop, imports)- Modify: `tests/middleware/compaction/test_summarizer.py` (three call sites at lines ~60, ~85, ~119)+- Modify: `tests/tracing/test_recorder.py` (three mocks at ~666, ~706, ~735 — per-test replacements, not one template)-**Scope discipline:** This task only changes the `summarize` boundary. `CompactionMiddleware` keeps its `_summary_provider` + `_summary_model: Model` split fields and its existing `extra_llm_calls()` *unchanged* — Task 4 collapses them. This keeps the intermediate commit fully working (compaction tests + recorder tests both pass after Task 3).+**Why atomic:** The producer side (`Middleware.extra_llm_calls` return shape) and the consumer side (`cubepi/tracing/recorder.py` + the test mocks) are coupled. Splitting this into two commits leaves an intermediate state where either the recorder unpacks the wrong shape or `CompactionMiddleware` re-pairs split fields that have already been collapsed. One commit, one atomic flip, larger diff — accepted trade-off for skipping the awkward `BoundModel(provider=self._summary_provider, spec=self._summary_model)` transition.-- [ ] **Step 1: Run the existing compaction + summarizer tests as a baseline**+- [ ] **Step 0: Confirm no third-party `extra_llm_calls` overrides exist**```bash-uv run pytest tests/middleware/test_compaction.py tests/middleware/compaction/test_summarizer.py tests/tracing/test_recorder.py -v+grep -rn "extra_llm_calls" cubepi/ tests/ --include="*.py"```-Expected: PASS. Record the totals — we need them all to still pass after Task 3.+Expected hits: the base definition (`cubepi/middleware/base.py`), the compaction override (`cubepi/middleware/compaction/__init__.py`), the recorder consumer (`cubepi/tracing/recorder.py`), and four test references (`tests/tracing/test_recorder.py`). If anything else shows up, fold it into the migration before continuing.-- [ ] **Step 2: Update `summarize()` signature**+- [ ] **Step 1: Baseline run**++```bash+uv run pytest tests/middleware/ tests/tracing/test_recorder.py tests/middleware/compaction/test_summarizer.py -v+```-In `cubepi/middleware/compaction/summarizer.py`, replace the function header and the provider call site. Replace the import block + the function body (lines 6-14 imports, lines 55-80 function):+Expected: PASS. Record the totals — these must all still pass after the refactor.++- [ ] **Step 2: Update `summarize()` to take a `BoundModel`**++In `cubepi/middleware/compaction/summarizer.py`, replace the import block (lines 6-14) and the function (lines 55-80):```pythonfrom cubepi.middleware.compaction.state import CompactionState, message_refs@@ -344,21 +355,37 @@ async def summarize()```-Remove `Model` and `Provider` from the imports (now unused inside this file).+Drop `Model` and `Provider` from the imports (now unused).-- [ ] **Step 3: Update the `summarize` call site in `CompactionMiddleware`**+- [ ] **Step 3: Collapse `CompactionMiddleware` to one `BoundModel` field**-In `cubepi/middleware/compaction/__init__.py`, **only** the call inside `transform_context` changes. Do NOT touch `__init__` or `extra_llm_calls()` in this task.+In `cubepi/middleware/compaction/__init__.py`, three changes in one pass.-Replace the existing `summarize(...)` call (the block around `try: new_state = await summarize(...)`) with:+**(a)** Replace `__init__`:++```python+ def __init__(+ self,+ *,+ summary_model: BoundModel,+ max_tokens_before_compact: int,+ keep_recent_messages: int = 8,+ max_summary_tokens: int = 1024,+ min_compact_messages: int = 4,+ ) -> None:+ self._summary_model = summary_model+ self._max_tokens_before = max_tokens_before_compact+ self._keep_recent = keep_recent_messages+ self._max_summary_tokens = max_summary_tokens+ self._min_compact = min_compact_messages+```++**(b)** Simplify the `summarize` call inside `transform_context`:```pythontry:new_state = await summarize(- model=BoundModel(- provider=self._summary_provider,- spec=self._summary_model,- ),+ model=self._summary_model,messages_to_summarize=messages[boundary:new_boundary],existing=state,max_summary_tokens=self._max_summary_tokens,@@ -366,84 +393,30 @@ Replace the existing `summarize(...)` call (the block around `try: new_state = a)```-`BoundModel` is already imported at the top of this file. The `extra_llm_calls()` method below still returns `((self._summary_provider, self._summary_model),)` — leave it; Task 4 handles it.--- [ ] **Step 4: Migrate `tests/middleware/compaction/test_summarizer.py` to the new signature**--This file has three `summarize(provider=..., model=...)` call sites (around lines 60, 85, 119). All three need to wrap the provider + model in a `BoundModel`. Example for the first call site:+**(c)** Replace `extra_llm_calls`:```python-# At the top, extend the imports:-from cubepi.providers.base import (- AssistantMessage,- BoundModel,- Message,- Model,- StreamOptions,- TextContent,- ToolCall,- ToolDefinition,- UserMessage,-)--# Then replace each `summarize(provider=provider, model=model, ...)` with:-result = await summarize(- model=BoundModel(provider=provider, spec=model),- messages_to_summarize=[...],- existing=None,- max_summary_tokens=512,- abort_signal=signal,-)-```--`_FakeProvider` (defined in the same file) only implements `generate` — that's fine because `BoundModel.generate` delegates to `self.provider.generate(model=self.spec, ...)`, so `_FakeProvider.generate` receives the same kwargs it does today and the `provider.calls[0]["max_output_tokens"]` style assertions stay green.--Do all three call sites in one edit. Don't change the `_FakeProvider` definition or the assertions.--- [ ] **Step 5: Run the touched test suites**--```bash-uv run pytest tests/middleware/compaction/test_summarizer.py tests/middleware/test_compaction.py tests/tracing/test_recorder.py -v-```--Expected: same PASS totals as Step 1 (no behavior change). If `test_summarizer.py` fails, the migration in Step 4 missed a call site.--- [ ] **Step 6: Commit**--```bash-git add cubepi/middleware/compaction/summarizer.py cubepi/middleware/compaction/__init__.py tests/middleware/compaction/test_summarizer.py-git commit -m "refactor(compaction): summarize() takes BoundModel"-```-------## Task 4: Collapse compaction fields + `extra_llm_calls()` returns `Iterable[BoundModel]`--**Files:**-- Modify: `cubepi/middleware/base.py:96-116`-- Modify: `cubepi/middleware/compaction/__init__.py` (`__init__`, the `summarize` call site, and `extra_llm_calls`)-- Modify: `cubepi/tracing/recorder.py:286-299`-- Test: `tests/tracing/test_recorder.py` (three mocks at ~666, ~706, ~735)--This task does the atomic switchover: one `BoundModel` field instead of split provider+spec, and a `BoundModel`-shaped `extra_llm_calls()` API. All four pieces (base, compaction impl, recorder consumer, recorder test mocks) flip together to keep tests green.--- [ ] **Step 0: Confirm no third-party `extra_llm_calls` overrides exist**--```bash-grep -rn "extra_llm_calls" cubepi/ tests/ --include="*.py"+ def extra_llm_calls(self) -> tuple[BoundModel, ...]:+ # Surface the bound summary model so ``cubepi.tracing.Recorder`` can+ # both subscribe its listeners (the summarizer's chat span lands in+ # the trace) AND identify the summary call by spec — important when+ # the summary model's provider is the same instance as the agent's+ # main provider, the common "reuse the client, swap the model"+ # pattern.+ return (self._summary_model,)```-Expected hits: the base definition, the compaction override, the recorder consumer, and four test references (`tests/tracing/test_recorder.py`). If anything else shows up, fold it into the migration before continuing.+Drop `Model` and `Provider` from the imports at the top of this file.-- [ ] **Step 1: Update the base interface and docstring**+- [ ] **Step 4: Update the `Middleware.extra_llm_calls` base signature + docstring**-In `cubepi/middleware/base.py`, replace lines 96–116 (the `extra_llm_calls` method) and adjust imports:+In `cubepi/middleware/base.py`, adjust imports and replace lines 96–116:```pythonfrom cubepi.providers.base import AssistantMessage, BoundModel, Message # drop Model, Provider```-(Keep `Provider` only if it's still referenced elsewhere in the file — currently it isn't outside `extra_llm_calls`'s annotation, so drop it. Same for `Model`. Verify with `grep -n "Provider\b\|Model\b" cubepi/middleware/base.py` after editing.)+(Verify with `grep -n "Provider\b\|Model\b" cubepi/middleware/base.py` after editing — neither should remain in the file.)```pythondef extra_llm_calls(self) -> Iterable[BoundModel]:@@ -469,60 +442,9 @@ from cubepi.providers.base import AssistantMessage, BoundModel, Message # dropreturn ()```-- [ ] **Step 2: Collapse `CompactionMiddleware` fields and update `extra_llm_calls`**--In `cubepi/middleware/compaction/__init__.py`, three changes in one pass:--**(a)** Replace `__init__`:--```python- def __init__(- self,- *,- summary_model: BoundModel,- max_tokens_before_compact: int,- keep_recent_messages: int = 8,- max_summary_tokens: int = 1024,- min_compact_messages: int = 4,- ) -> None:- self._summary_model = summary_model- self._max_tokens_before = max_tokens_before_compact- self._keep_recent = keep_recent_messages- self._max_summary_tokens = max_summary_tokens- self._min_compact = min_compact_messages-```--**(b)** Simplify the `summarize` call inside `transform_context` — Task 3 set it to `BoundModel(provider=self._summary_provider, spec=self._summary_model)`; now it's just:--```python- try:- new_state = await summarize(- model=self._summary_model,- messages_to_summarize=messages[boundary:new_boundary],- existing=state,- max_summary_tokens=self._max_summary_tokens,- abort_signal=signal,- )-```--**(c)** Replace `extra_llm_calls`:--```python- def extra_llm_calls(self) -> tuple[BoundModel, ...]:- # Surface the bound summary model so ``cubepi.tracing.Recorder`` can- # both subscribe its listeners (the summarizer's chat span lands in- # the trace) AND identify the summary call by spec — important when- # the summary model's provider is the same instance as the agent's- # main provider, the common "reuse the client, swap the model"- # pattern.- return (self._summary_model,)-```--Drop `Model` and `Provider` from the imports at the top of this file (no longer referenced).--- [ ] **Step 3: Update the recorder consumer**+- [ ] **Step 5: Update the recorder consumer**-In `cubepi/tracing/recorder.py`, around lines 286–299, replace the unpacking loop. The current code is:+In `cubepi/tracing/recorder.py`, around lines 286–299, the current unpacking loop is:```pythonfor p, m in extra:@@ -550,20 +472,54 @@ for bound in extra:_subscribe(provider)```-Add `BoundModel` to the file's imports if not already imported (the surrounding code uses `Model`, `Provider`, etc., from `cubepi.providers.base`).+**Do not** add `BoundModel` to this file's imports. The loop body only uses attribute access (`bound.spec`, `bound.provider`) and no other code in `recorder.py` references the `BoundModel` type, so an import would be flagged by ruff as `F401`. (Verify with `grep -n "from cubepi.providers.base" cubepi/tracing/recorder.py` — only `StreamEvent`, `ToolCall`, `ToolResultMessage`, `UserMessage` are imported today; leave that list unchanged.)++- [ ] **Step 6: Migrate `tests/middleware/compaction/test_summarizer.py` to the new signature**++This file has three `summarize(provider=..., model=...)` call sites (around lines 60, 85, 119). All three wrap the provider + model in a `BoundModel`. Extend the imports:++```python+from cubepi.providers.base import (+ AssistantMessage,+ BoundModel,+ Message,+ Model,+ StreamOptions,+ TextContent,+ ToolCall,+ ToolDefinition,+ UserMessage,+)+```++Replace each `summarize(provider=provider, model=model, ...)` with:++```python+result = await summarize(+ model=BoundModel(provider=provider, spec=model),+ messages_to_summarize=[...],+ existing=None,+ max_summary_tokens=512,+ abort_signal=signal,+)+```++`_FakeProvider` (defined in the same file) only implements `generate` — that's fine. `BoundModel.generate` delegates to `self.provider.generate(model=self.spec, ...)`, so `_FakeProvider.generate` receives the same kwargs it does today, and the `provider.calls[0]["max_output_tokens"]` style assertions stay green. Don't change `_FakeProvider` or the assertions.++Do all three call sites in one edit.-- [ ] **Step 4: Update the three mocks in the recorder tests**+- [ ] **Step 7: Update the three recorder test mocks**-In `tests/tracing/test_recorder.py`, three middleware classes return from `extra_llm_calls`. They are NOT interchangeable — each exercises a different recorder branch, so spell out the per-test replacement instead of applying one template. `BoundModel` is already imported at line 22 of the test file.+In `tests/tracing/test_recorder.py`, three middleware classes return from `extra_llm_calls`. They are NOT interchangeable — each exercises a different recorder branch, so spell out the per-test replacement. `BoundModel` is already imported at line 22 of the test file.-**(a)** `_SameModelMiddleware` (around line 666): exercises the "same `(provider_id, id)` as the agent's bound model — must be excluded from `_extra_call_models`" branch. The mock takes `provider` and `model` via its constructor and stores them on `self._p` / `self._m`.+**(a)** `_SameModelMiddleware` (around line 666): exercises the "same `(provider_id, id)` as the agent's bound model — must be excluded from `_extra_call_models`" branch. The mock stores `provider` and `model` on `self._p` / `self._m`.```pythondef extra_llm_calls(self):return [BoundModel(provider=self._p, spec=self._m)]```-**(b)** `_DuckMiddleware` (around line 706): exercises the "provider not derived from `BaseProvider` is skipped in `_subscribe`" branch. The provider here MUST stay `_DuckProvider()` — using the agent's BaseProvider would silently kill the branch this test exists to cover. The model field is `self._m` from the middleware's constructor.+**(b)** `_DuckMiddleware` (around line 706): exercises the "provider not derived from `BaseProvider` is skipped in `_subscribe`" branch. The provider MUST stay `_DuckProvider()` — using the agent's BaseProvider would silently kill the branch this test exists to cover. The model field is `self._m` from the middleware's constructor.```pythondef extra_llm_calls(self):@@ -574,23 +530,21 @@ In `tests/tracing/test_recorder.py`, three middleware classes return from `extra**(c)** `_BoomMiddleware` (around line 735): raises before returning, so the return shape doesn't matter. **Do not change this class.**-After editing, verify the test method names that contain the changed classes still pass intact:+- [ ] **Step 8: Run the touched suites, then the full sweep**```bash-uv run pytest tests/tracing/test_recorder.py -k "extra_llm_calls or extra_provider_not_baseprovider" -v+uv run pytest tests/middleware/ tests/middleware/compaction/test_summarizer.py tests/tracing/test_recorder.py -v```-Expected: all PASS — in particular the duck-provider test must still hit the "skipped non-BaseProvider" branch (you can sanity-check by reading the recorder's `_subscribe` and confirming the same code path runs).+Expected: same PASS totals as Step 1. If anything fails, the producer/consumer didn't flip together — re-check Steps 2-7.-- [ ] **Step 5: Run the recorder + middleware test suites**+Specifically verify the branch-coverage tests still hit their intended branches:```bash-uv run pytest tests/tracing/test_recorder.py tests/middleware/ -v+uv run pytest tests/tracing/test_recorder.py -k "extra_llm_calls or extra_provider_not_baseprovider" -v```-Expected: PASS. Any new failure here means the consumer or a mock wasn't fully migrated.--- [ ] **Step 6: Full test sweep + lint + types**+Then the full sweep + lint + types:```bashuv run pytest tests/ -x@@ -599,18 +553,18 @@ uv run ruff format --check cubepi/ tests/uv run mypy cubepi```-Expected: all green. (`pytest -x` stops at the first failure so issues surface fast.)+Expected: all green.-- [ ] **Step 7: Commit**+- [ ] **Step 9: Commit**```bash-git add cubepi/middleware/base.py cubepi/middleware/compaction/__init__.py cubepi/tracing/recorder.py tests/tracing/test_recorder.py-git commit -m "refactor(middleware): extra_llm_calls returns BoundModel"+git add cubepi/middleware/compaction/summarizer.py cubepi/middleware/compaction/__init__.py cubepi/middleware/base.py cubepi/tracing/recorder.py tests/middleware/compaction/test_summarizer.py tests/tracing/test_recorder.py+git commit -m "refactor(middleware): take and surface BoundModel through compaction + extra_llm_calls"```----## Task 5: Document the new ergonomics+## Task 4: Document the new ergonomics**Files:**- Modify: `website/docs/agents/providers.md` (or the closest existing page; verify with `ls website/docs/agents/ website/docs/getting-started/`)@@ -672,7 +626,7 @@ git commit -m "docs(providers): document BoundModel.generate/stream"----## Task 6 (gated): Local codex review loop+## Task 5 (gated): Local codex review loop**Per CLAUDE.md, ask the user before starting this loop.** Do not enter it autonomously.@@ -714,12 +668,14 @@ Poll `gh api repos/cubeplexai/cubepi/pulls/<#>/comments` every ~2 minutes, resol1. **Spec coverage** — every item from the conversation is mapped to a task:- "Add `BoundModel.generate/stream`" → Task 1, Task 2.- - "`summarize()` takes `BoundModel`" → Task 3.- - "`extra_llm_calls()` returns `Iterable[BoundModel]`" → Task 4.- - Consumer updates (recorder + test mocks) → Task 4 Steps 3–4.- - Docs requirement (CLAUDE.md) → Task 5.+ - "`summarize()` takes `BoundModel`" → Task 3 Step 2.+ - "`extra_llm_calls()` returns `Iterable[BoundModel]`" → Task 3 Step 4.+ - Recorder consumer + test mocks → Task 3 Steps 5, 7.+ - Compaction field collapse + summarizer test migration → Task 3 Steps 3, 6.+ - Docs requirement (CLAUDE.md) → Task 4.2. **Placeholders** — none. Every code step has the actual code; commands are concrete.-3. **Type consistency** — `BoundModel.generate` / `BoundModel.stream` signatures mirror `Provider.generate` / `Provider.stream` exactly (verified against `cubepi/providers/base.py:579-603`). Field name `self._summary_model: BoundModel` is used consistently in Tasks 3 and 4.+3. **Type consistency** — `BoundModel.generate` / `BoundModel.stream` signatures mirror `Provider.generate` / `Provider.stream` exactly (verified against `cubepi/providers/base.py:579-603`). `self._summary_model: BoundModel` is the single field name used throughout Task 3.+4. **Atomicity** — Task 3 producer-side (compaction `__init__` / `extra_llm_calls` return shape) and consumer-side (recorder loop + test mocks) flip in one commit. No intermediate broken state.## Follow-ups (out of scope)