Skip to content

armgit / avibe

Public Imported from GitHub
0

Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.

fix(model-hub): preserve capability intent and explicit limit authority (#1959)

Call Whited50e1985c18c
diff --git a/docs/plans/model-hub-capability-authority.md b/docs/plans/model-hub-capability-authority.md
new file mode 100644
index 000000000..896af4501
--- /dev/null
+++ b/docs/plans/model-hub-capability-authority.md
@@ -0,0 +1,149 @@
+# Backend capability and limit authority audit
+
+Lane B, Session `seshneu2u9cbw`, base
+`76e269c9a9dfd3a6286ce0da81b77316ca4be38a`.
+
+## Contract
+
+Unknown custom capability metadata must not suppress supplied images or
+reasoning. Explicit modality lists, `supports_reasoning: false`, and effort
+lists retain their meanings. Native rows keep their own metadata. Custom rows
+must not acquire an unrelated model's context, tools, service tier, or default
+effort. Persisted BackendModel shapes and Avibe-owned model selection remain
+unchanged.
+
+## Authority and ambiguity inventory at the shared base
+
+- BackendModel stores optional context/output numbers, optional capability
+ booleans, and empty-by-default modality/effort lists. `origin` identifies the
+ creation path for the whole row; it cannot distinguish an imported field
+ from a later user edit. No existing per-field limit provenance exists.
+- The UI's candidate path leaves limits and capabilities unstated. Its manual
+ editor visibly starts with text input and reasoning disabled. A models.dev
+ selection fills the editor; saving commits that full row. The backend saves
+ those values literally and built-in reconciliation only adds missing rows.
+ Therefore a saved non-null limit should remain the selected model's
+ description, without trying to reconstruct how it was authored.
+- Route resolution reads metadata by the requested BackendModel ID, before
+ mapping it to the upstream target. This is intentional: an alias owns its
+ planning metadata. Codex's authenticated turn gateway retains that alias as
+ the runtime model, so the projected catalog row matches. Claude launches the
+ target but carries the selected row's limits. OpenCode overlays use the
+ public alias and selected row's limits.
+- Claude overwrites both `CLAUDE_CODE_MAX_*_TOKENS` environment
+ variables, in both Hub and native CLI launches. Hub also masks them as if
+ they were credentials. Separately, `claude_settings_for_launch` copies the
+ catalog limits into the highest-priority launch settings, so changing only
+ the subprocess environment would still lose explicit choices. Project/local
+ settings are enabled, user settings are disabled for Hub transport ownership.
+ Native CLI retains all setting sources. Parent environment, settings env, and
+ the saved model row can conflict; there is no provenance for ranking two
+ user-authored descriptions. Credential/connection masking is independent.
+- Codex projects the saved context into `context_window` and
+ `max_context_window`, removing the old catalog compaction limit so it can be
+ derived from the new context. Custom rows without a limit inherit no context.
+ Launch arguments do not inject a context override. Native CLI launches do
+ not consume this projected catalog. The real consumer checks below cover
+ `model_context_window` and `model_auto_compact_token_limit`. BackendModel output limits have no
+ Codex projection; silently claiming support or inventing a new output cap
+ would be incorrect.
+- OpenCode projects saved limits into `limit.context` / `limit.output` when
+ present and otherwise omits them. The managed plugin replaces complete Hub
+ provider definitions to protect authentication, so a same-name native
+ provider cannot remain a second owner. Ordinary native provider definitions
+ are separate. No new merge of arbitrary native provider data is proposed.
+- Unknown input metadata is converted into `["text"]` only for
+ custom Codex rows. Codex 0.153.2's protocol schema itself defaults omitted
+ input modalities to text and image. Explicit nonempty lists can narrow that
+ default without adding persisted state. Original-resolution image detail is
+ a separate provider feature and should not be enabled from absence.
+- Unknown reasoning is converted into a `none`-only list and default.
+ Explicit false and explicit lists must remain distinct from absence.
+- The UI model picker currently returns an empty effort menu for an unknown
+ row, and the manual editor starts with explicit reasoning false. These are
+ separate visible-product policies; this lane will not change them without
+ coordination.
+
+## Approved repair
+
+The orchestrator approved the limit rule in integration commit `dc2d5f7bc` and
+the Codex consumer follow-up in `a3065a7d1`. The shared plan remains owned by
+the orchestrator.
+
+- Preserve the persisted schema and requested alias's metadata authority.
+- Unknown custom Codex input omits `input_modalities`, invoking the native
+ text/image compatibility default, while original-resolution detail remains
+ disabled. A nonempty modality list still narrows the accepted inputs.
+- Unknown custom reasoning has the schema-required empty supported list and
+ no default effort. Actual turns can still supply an effort. Explicit false
+ retains `none`, explicit lists retain their selected/default efforts, and
+ known native rows retain their own defaults.
+- Claude catalog limits fill only absent subprocess variables, including
+ preservation of explicitly empty strings. Only the two limit keys leave the
+ credential tombstone set. Hub connection settings no longer promote
+ catalog limits above native project/local settings. Credential masking and
+ enabled settings sources are unchanged.
+- Applying a saved Codex context now removes `max_context_window` instead of
+ synthesizing an equal hard ceiling. The planning context remains and the
+ obsolete native compaction threshold is removed as before. Without a saved
+ context override, native model metadata remains unchanged.
+- OpenCode needs no production change: the saved BackendModel remains the
+ managed provider's authority. Alias/overlay tests confirm supplied limits
+ and omitted metadata do not inherit an unrelated target row's values.
+
+## Consumer evidence and limitations
+
+- Codex 0.153.2 loaded a native context of 512,000 but originally reported only
+ 121,600 usable tokens with the projected 128,000 hard ceiling. The pinned
+ source's `models-manager/src/model_info.rs::with_config_overrides` confirms
+ the minimum operation. Removing the synthesized ceiling makes its actual
+ token-usage consumer report 486,400 usable tokens, preserving 95% headroom.
+ Smaller explicit windows and the catalog fallback also work.
+- Two-turn tests feed real CLI responses with synthetic token usage. With the
+ 128,000 planning window, 150,000 tokens trigger compaction. With an explicit
+ 512,000 window and 300,000 threshold, 150,000 do not trigger compaction and
+ 350,000 do. The mock sees the extra compaction request and the app-server
+ completes its compaction item and following turn.
+- Actual Codex requests retain a valid inline PNG and non-ASCII text. Unstated
+ effort adds no default; explicit `none`, `minimal`, `low`, `medium`, `high`,
+ `xhigh`, and `max` reach the local endpoint. Negative image/reasoning choices
+ and nonempty effort lists are exercised through the same consumer.
+- Codex `ultra` is a native multi-agent mode: the pinned client's
+ `reasoning_effort_for_request` maps it to a non-Ultra tier, falling back to
+ medium for an empty ladder. This repair does not change that native
+ interpretation or claim literal Ultra wire preservation. Native CLI tools
+ may also have their own defaults independent of catalog projection.
+- Claude SDK 0.2.93's bundled CLI preserves parent/project/local output limits
+ in real Messages requests, for both Hub and native CLI paths. Its normal
+ context planner does not consume `CLAUDE_CODE_MAX_CONTEXT_TOKENS` unless
+ compaction is disabled. The repair preserves that value through SessionHandler
+ into SDK options but does not claim to change native Claude context planning.
+ Compaction is never disabled as a workaround.
+- BackendModel output metadata has no Codex output-cap projection. No output
+ cap, UI policy, schema, engine, gateway, credential, or routing-policy change
+ is introduced. Engine-to-upstream preservation remains lane D's boundary.
+
+All CLI tests use test-owned homes, allowlisted environments, synthetic tokens,
+and loopback endpoints. External HTTP(S) is directed at a test-owned rejecting
+proxy. On macOS, a separate OS sandbox denies non-loopback outbound sockets;
+a representative forbidden connection was verified to return a permission
+error. Claude nonessential traffic is explicitly disabled; Codex launches use
+Avibe's existing analytics/update/host-feature disable settings.
+
+## Validation results
+
+- Baseline: 70 catalog/injection tests passed before the repair.
+- Focused catalog, injection, and SessionHandler coverage: 201 passed.
+- Related scenario/catalog/route/projection selection: 219 passed.
+- Actual Codex/Claude consumers: 30 cases, including the three compaction
+ boundaries above.
+- Pinned Ruff 0.4.9 and Model Hub authority closure pass.
+- These are implementation-head results, not evidence for a later PR head.
+ The owner's 2026-09-09 independent-delivery decision assigns this lane its
+ own PR, canonical CLI fixture registration, and durable review/CI Watch.
+ MH-PROTOCOL-004 remains partial CLI-only evidence; MH-CLAUDE-LAUNCH-001 gains
+ related launch-limit evidence without claiming managed-engine coverage.
+ After normal landed-master integration, rerun focused tests, real isolated
+ CLI consumers, authority/scenario validation, and pinned Ruff at the PR head.
+ The orchestrator independently verifies all gates and executes any merge;
+ no runtime update or publication is authorized.
diff --git a/docs/plans/model-hub-permissive-boundaries.md b/docs/plans/model-hub-permissive-boundaries.md
new file mode 100644
index 000000000..14fba7dd0
--- /dev/null
+++ b/docs/plans/model-hub-permissive-boundaries.md
@@ -0,0 +1,409 @@
+# Model Hub permissive request boundaries
+
+## Ownership and authorization
+
+Owner approval on 2026-09-09 authorizes implementation and parallel Codex
+delegation following the preceding request-path audit. The user-started Session
+remains the orchestrator and final reviewer. On 2026-09-09 at 04:11 +08
+(2026-09-08 at 20:11 UTC), the owner superseded the initial combined-PR plan:
+each lane must submit its own PR and merge after its delivery gates pass.
+The orchestrator owns final gate verification and merge execution. This does
+not authorize publishing an engine or application release, deploying,
+updating/restarting the running local Avibe, mutating user configuration, or
+advancing the primary checkout.
+
+Implementation starts from default-branch commit
+`e4924db41bcbf37581cda08cf1376a899e70906c`. Independent lanes share this committed
+contract before they branch. Each lane now delivers one independent Avibe PR
+against the current default branch; no lane stacks a PR on an unmerged peer.
+The combined task branch is retained only as integration-test evidence.
+
+## Evidence and goal
+
+A Codex Blender Session failed at the authenticated Avibe gateway's 16 MiB
+request boundary. Reconstructing its latest persisted context checkpoint and
+subsequent model items found 20 inline PNG images totaling approximately 42 MiB.
+This was not a capture of the rejected wire request. The new user text was
+39 UTF-8 bytes. A hermetic 18 MiB aiohttp request passed with the boundary
+disabled, but the read/parse/reserialize fixture recorded about 76.5 MiB of peak
+Python allocations, excluding the engine. Removing every resource bound is
+therefore not the immediate remedy.
+
+The product should accept legitimate multi-image requests, preserve explicit
+model intent, and never equate absent capability knowledge with demonstrated
+non-support. Resource ownership, authentication, cancellation, routing, and
+persisted identity remain real constraints.
+
+## Shared invariants
+
+1. **Request envelope.** The common authenticated gateway accepts valid request
+ bodies through 128 MiB for Messages, Responses, and Chat Completions, with
+ consistent byte semantics for Content-Length and chunked bodies. Oversize
+ input returns a structured, non-secret 413 describing the local boundary,
+ before upstream admission. It does not mark a provider unhealthy, consume an
+ alternative provider, or silently truncate history. Do not add image-count,
+ turn-count, output-length, or normal-inference-duration caps.
+2. **Capabilities.** An absent override remains distinct from an explicit
+ negative or an explicit user selection. Custom model projection must not
+ silently strip supplied images or force reasoning to `none` solely because
+ catalog metadata is absent. Do not inherit a different model's provider-only
+ features, context size, tools, or expensive defaults. Preserve native known
+ rows, explicit negatives, persisted shapes, and model-selection ownership.
+3. **Limit authority.** Catalog context/output metadata assists planning; it
+ must not silently replace explicit user configuration with a stale default.
+ Preserve useful automatic compaction and backend schema requirements.
+ Determine authority from existing fields/owners where possible. A new
+ persisted provenance model requires an orchestrator decision before editing.
+4. **Model identity.** Reassess the 256-character admission bound across manual
+ input, discovery, routing, UI/API, metering, and load. Do not replace it with
+ another arbitrary number or remove it without checking downstream identity
+ and usage-key behavior. No truncation, identity merging, secret admission,
+ startup breakage, or silent loss of historical usage is permitted. The lane
+ reports its smallest complete proposal before implementation.
+5. **Engine intent.** Inspect the exact pinned CLIProxyAPI v7.2.149 source
+ `2a6b87aca083a5bf498ac1f68a1b636c500d7aaa`. Same-protocol explicit reasoning
+ intent must not be removed or downgraded solely by incomplete capability
+ metadata. Preserve required cross-protocol translation, OAuth lifetimes,
+ credential replacement, Source/model identity, and all routing policy.
+ Do not remove registration metadata as a shortcut: it can worsen stripping.
+ Report the source/build/release plan before implementing engine policy.
+ Never point a shipped manifest at missing assets or claim wire preservation
+ from a fake-adapter-only test.
+6. **Non-goals.** Keep bounded observation copies, temporary-file spill
+ thresholds, chunk sizes, authentication, cancellation cleanup, real upstream
+ failures, and existing retry ownership. Do not weaken CI, migrate unrelated
+ state, add a detection save gate, replay live credentials, or change unrelated
+ pending PRs.
+
+## Boundary ownership
+
+| Boundary | Producer | Consumer | Contract owner |
+| --- | --- | --- | --- |
+| Native model request | Claude/Codex/OpenCode | authenticated turn gateway | gateway lane |
+| Request model/protocol/header envelope | gateway | resolver and managed adapter | existing shared request owner, unchanged |
+| Backend model capability/limit projection | selected backend model catalog | CLI launch/catalog consumers | capability lane |
+| Model ID admission and durable usage identity | setup/discovery | config, routes, ledger, UI | identity lane, orchestrator approval |
+| Reasoning wire conversion | managed engine | exact selected mock upstream | engine lane, orchestrator approval |
+| Integration, authority/scenario inventory, PR gates | all lanes | owner | orchestrator |
+
+No new signature or credential field is introduced. Existing local bearer
+authentication and exact Source/model binding must remain unchanged.
+
+## Parallel delivery
+
+- **Gateway lane:** owns gateway request-size/error handling and focused tests
+ plus only the locale/catalog entries required for its new error surface.
+- **Capability lane:** owns backend catalog projection, CLI limit precedence,
+ and their consuming tests. Report cross-lane changes rather than editing
+ gateway/engine/identity code.
+- **Identity lane:** owns the complete identifier/ledger boundary diagnosis
+ first; implementation is gated on the orchestrator's recorded decision.
+- **Engine lane:** owns exact-pinned-source diagnosis and a reproducible
+ hermetic wire/build proposal first; policy edits, publication, and a manifest
+ change are separate decisions.
+- **Orchestrator:** owns this plan, cross-lane validation, independent PR/CI
+ observation, final review, guarded merge execution, and user-facing close-out.
+
+Each lane may push its assigned branch, open its own non-draft PR, and follow
+review/CI with one durable combined Watch. It must immediately return the PR,
+exact head, Watch/state, review inventory, tests, and known gaps to the
+orchestrator, who arms a separate gate Watch and independently inspects evidence.
+Lanes must not merge, create release tags/assets, deploy, or edit peers. The
+orchestrator executes the owner's gated merge authorization after all checks
+hold together. A callback is evidence to inspect, not proof of readiness.
+
+## Acceptance
+
+- Exercise real hermetic HTTP ingress with a multi-image-size payload exceeding
+ 16 MiB; compare exact input reaching the fake upstream/adapter, including
+ non-ASCII text and image data. Check both body framing shapes, threshold
+ boundaries, bad credentials, malformed JSON, cancellation, and no provider
+ health mutation for a local 413.
+- Cover unknown custom models, known built-ins, explicit supported/unsupported
+ choices, aliases, user-selected reasoning, and context/output precedence in
+ consuming launch/catalog paths.
+- For any identifier change, cover every admission surface and legacy persisted
+ IDs, non-ASCII names, collision-shaped names, reload and usage stability.
+- For engine policy, require actual pinned engine-to-mock-upstream wire evidence
+ for applicable same- and cross-protocol paths; distinguish source proof,
+ built artifacts, published availability, and installed behavior.
+- Run focused Python checks and pinned Ruff, then relevant Model Hub suites,
+ authority closure and scenario/catalog checks. UI changes require UI tests,
+ TypeScript and build. Use only isolated fixtures/local regression targets.
+- Require current-head Codex pass, all expected CI jobs, and zero unresolved
+ paginated threads. Record findings-bearing heads/root causes before edits;
+ enforce the review circuit breaker. Before merging, the orchestrator must
+ also verify an open non-draft PR, CLEAN merge state, no running/queued lane
+ edits, and every merge gate in one fail-closed conditional, using the exact
+ validated head. Authorization does not permit skipping a gate.
+
+## Decisions and status
+
+- Initial decision: implement the 128 MiB compatibility boundary, not unlimited
+ buffering; investigate the identifier and engine seams before choosing edits.
+- Initial findings-bearing review heads: zero. No PR has been submitted yet.
+
+### Independent delivery allocation
+
+The four PRs own disjoint product concerns: A owns the gateway and request-budget
+scenario registration; B owns capability/limit projection and its CLI scenario
+registration; C owns identifier admission/discovery/UI plus the integration
+guard correction and qualified usage scenario descriptions; D owns the inactive
+exact-base engine patch, recipe, and consuming evidence. Each carries the same
+orchestrator-owned shared contract. Lane-specific assessments must describe
+independent delivery rather than the superseded combined-PR plan.
+
+Merge landed default-branch changes normally into each task branch before
+submission. Do not cherry-pick another lane's product changes or count the
+combined branch's tests as exact-head PR evidence. Recheck shared scenario IDs
+and catalog entries after each earlier PR lands. Subsequent gate evaluations
+remain exact-head and must inspect all paginated reviews/threads and every lint
+run, including old unresolved findings. Repeated root-cause classes are escalated
+to this same orchestrator before another edit/push; they do not automatically
+require another owner decision.
+
+Keep separate original durable lane and orchestrator gate Watch states across
+all rounds. Do not merge or remove a lane Watch merely because a callback reports
+green tests. After GitHub confirms a guarded merge, report that PR's result to
+the owner before removing its observation. Completed historical PR authority
+and Watches are not reused.
+
+### Capability and launch-limit decision (2026-09-09)
+
+The orchestrator independently inspected catalog projection, Claude environment
+construction, its SessionHandler call site, and injection/CLI-path consuming
+tests at the shared contract base. The current Hub path both tombstones limit
+variables as though they were credentials and promotes catalog limits into
+highest-priority launch settings. Native CLI launches also overwrite explicit
+environment limits. These are competing owners, not evidence that a persisted
+provenance system is needed.
+
+Approved scope:
+
+- Keep existing BackendModel storage and the requested alias's metadata
+ authority. A saved non-null limit remains that selected model's planning
+ description; do not infer field provenance from row-level `origin`.
+- For Claude, leave credential/connection tombstones intact but remove the
+ two context/output limit variables from that boundary. Catalog limits fill
+ only absent subprocess variables. Empty explicit values are not permission
+ to invent another override. Do not inject model limits in the Hub connection
+ settings override; preserve native project/local settings precedence.
+- Verify the real launch consumer as well as pure environment helpers: parent
+ environment, explicit settings, absent limits, selected aliases, and both Hub
+ and native CLI paths. Do not broaden the enabled native setting sources.
+- Codex keeps its native configuration precedence if consumer evidence
+ confirms it. Custom capability absence must allow supplied images/reasoning
+ without importing a different model's private features or expensive default
+ effort. Explicit negatives and nonempty lists retain their meanings.
+- OpenCode retains the selected BackendModel as owner of the managed Hub
+ provider's planning metadata. Preserve separate native providers; do not merge
+ arbitrary native provider objects into a managed authentication definition.
+- No new persisted schema, UI policy, engine policy, credential flow, routing
+ policy, or output cap is approved by this decision.
+
+The remaining uncertainty is the native CLI consumer's treatment of absent
+capabilities and conflicting settings. Hermetic consumer evidence must confirm
+the representation and precedence before integration; a schema requirement or
+unexpected consumer override calls for another bounded decision, not a fallback
+to silent filtering.
+
+### Identifier decision (2026-09-09, local +08)
+
+Lane C's diagnosis is committed as
+`e3f3628230333b141449684ba2f9af90ad0e3d1a`. The orchestrator independently
+inspected live and persisted ledger key owners, their consuming closure test,
+the selective parser and inventory projection, new-ID admission, OpenCode
+aggregate loading, and the typeahead refusal. A separate read-only synthetic
+run confirmed 12 valid long/Unicode/folded-literal identities remain distinct
+with stable read-back keys. It also reproduced the pre-existing whitespace
+normalization exception and admission of unencodable surrogate text.
+
+Approved smallest complete scope:
+
+- Remove the 256-character admission ceiling and its editor, schema, locale,
+ and typeahead-query mirrors. Keep canonical outer-whitespace spelling,
+ nonblank text, complete credential scanning, exact Source/model pairing, and
+ backend/source eligibility policy.
+- Require newly admitted IDs and unsaved observation IDs to be representable
+ as UTF-8, without replacement, truncation, Unicode normalization, or case
+ folding. This is a transport/storage validity check, not a new length cap.
+ Do not apply new admission rules to legacy persisted identities.
+- Separate OpenCode aggregate loading and whole-list editing from new-ID
+ admission while retaining existing spelling, nonblank, native-protocol,
+ menu, and route-membership invariants.
+- Extend the existing selective parser with a default-empty lossless-string
+ path option. Opt only inventory string-row IDs, object-row IDs, and
+ supported-parameter string values into it. Decide at value-token start;
+ object keys and unrelated values keep the existing bounded behavior.
+ Preserve malformed-JSON rejection, duplicate-member/scope semantics,
+ document completion, spooling, and the discovery deadline.
+- Keep the 200-character ledger head, digest calculation, persisted key
+ reader, stored rows, label joins, and retention unchanged. New admission can
+ safely include folded-looking literals because live derivation folds those
+ literals again; persisted key recognition is intentionally a different owner.
+- Limit shared service/config changes to the diagnosed admission, discovery,
+ typeahead, and load seams. Lane B's projection and lane A's gateway remain
+ independently owned. No new schema revision, dependency, ledger namespace,
+ engine setting, or arbitrary replacement size is approved.
+
+Selected identity/parameter facts necessarily occupy memory proportional to
+their full values. This decision does not remove generic HTTP/resource limits
+or promise arbitrary URI-size support. Consuming tests must cover lexical
+JSON expansion, long credential-shaped tails, exact runtime IDs, key-literal
+closure, reload, and the UI's whole-ID submission.
+
+Known-by-design legacy exceptions: an already loadable long all-whitespace
+identity can be misattributed by the existing persisted-key normalization;
+unencodable historical identities can also fail later encoding. Neither is
+introduced by removing the limit, and blank/unencodable new IDs are refused.
+Historical merged counters cannot identify their original owner, so this PR
+must not rekey or split them speculatively. Preserve the diagnostic evidence
+and record that attribution-policy follow-up separately; do not claim a proof
+over every malformed legacy string.
+
+### Native consumer follow-up
+
+The orchestrator inspected Codex 0.153.2's
+`models-manager/src/model_info.rs::with_config_overrides` and lane B's actual
+app-server consumer test. Codex takes the minimum of explicit
+`model_context_window` and catalog `max_context_window`; Avibe currently
+synthesizes both catalog fields from one BackendModel planning value.
+
+Approve removing `max_context_window` only when applying an explicit saved
+BackendModel `context_window` override. Retain `context_window` as the planning
+fallback and remove its obsolete catalog compaction threshold as before.
+Native rows without that override and native-CLI launches remain unchanged.
+The consumer test must prove both the fallback and an explicit larger context,
+alongside the native compaction setting. No Codex output cap is introduced.
+
+Claude's bundled CLI consumer honors explicit output limits, but its normal
+context planner does not necessarily consume `CLAUDE_CODE_MAX_CONTEXT_TOKENS`;
+the diagnosed context branch checks it only with compaction disabled. Retain
+the approved environment/settings preservation, prove actual output and
+Avibe-to-SDK context delivery, and document this native planner limitation.
+Do not disable compaction or claim end-to-end context control from environment
+injection alone.
+
+### Engine source-only decision
+
+Lane D's complete assessment is at
+`2a756b4b19a1478d06e1cc2a382aad0a9268acb5`. The orchestrator independently
+verified the clean exact upstream SHA, read the shared thinking entry point,
+validation and configured-model consuming tests, inspected the HTTP fixture,
+and verified the baseline artifact digest and all 380 result records. The
+recorded native disable-to-positive outcomes agree with the inspected
+capability clamp. These are baseline defects, not patched acceptance results.
+
+Approve a maintained, inactive candidate source/test patch series under
+`patches/cliproxyapi/`, against upstream
+`2a6b87aca083a5bf498ac1f68a1b636c500d7aaa`, with a small apply/test recipe
+and frozen input receipt. This is a local source-maintenance decision only,
+not authorization for a public fork, upstream contribution, release assets,
+manifest changes, installation, or runtime replacement.
+
+Source scope:
+
+- Distinguish identical reasoning wire representations from broad provider
+ families at the shared thinking entry point. Without an explicit suffix,
+ preserve the already prepared native target payload instead of performing
+ catalog-driven strip/map/validate/reapply. Recognize verified Responses
+ aliases; do not conflate Chat/Responses or Kimi's different dialect.
+- Extract explicit disable before capability gating, retain suffix priority,
+ reuse native disable writers before their support guards, and never turn
+ disable into a positive level or reactivate it through summary handling.
+- For genuine conversion with missing thinking metadata, use the existing
+ capability-free conversion responsibility, supplying original source
+ payload/format instead of extracting source fields from translated data.
+ Preserve positive conversion with known metadata and real protocol
+ constraints, normalizer/payload-rule ownership, and forced tool choice.
+- Preserve all registration metadata, including the existing Chat conversion
+ hint, exact selected credential snapshot, aliases/prefixes, routing, OAuth,
+ replacement, and refresh lifetimes. No fabricated UserDefined/capability
+ claim, broad validator deletion, new user setting, or runtime config change.
+
+Acceptance must include the maintained policy tests, patched HTTP matrix
+through real registration/execution, source/auth identity and replacement,
+stream/cancellation, and fake-auth subscription HTTP/WebSocket/dialect tests
+with external egress rejected. Keep source execution isolated and concurrency
+bounded. Use the committed engine catalogs, locked Go modules and exact
+toolchain; no catalog refresh from a moving branch.
+
+Release/packaging is deliberately separate. The existing manifest guard only
+accepts a truthful upstream identity and verified available four-platform
+assets. Do not weaken it or label locally patched bytes as upstream. Local
+diagnostic builds are not four-platform production reproducibility evidence.
+An upstream acceptance/release or a separately authorized Avibe source/
+provenance release contract is required before shipping this engine change.
+The Avibe PR may carry the reviewed inactive patch, but must explicitly state
+that its current engine pin and installed behavior remain unchanged.
+
+### xAI post-translation capability gate
+
+The orchestrator independently inspected the exact pinned source's
+`prepareResponsesRequestTo` chain, `sanitizeXAIResponsesBody`,
+`xaiSupportsReasoningEffort`, both existing catalog-strip tests, and the new
+fake-OAuth executor consumer and failing wire record. A whole-source search
+found one production caller of the capability helper: the sanitizer removes
+`reasoning.effort` after shared thinking and payload rules when registry
+thinking levels are absent. This is another catalog policy owner, not a
+Responses representation constraint. Shared passthrough alone cannot satisfy
+the accepted intent contract.
+
+Approve this additional source-only scope:
+
+- Remove only the catalog-conditioned reasoning deletion from
+ `sanitizeXAIResponsesBody`. Keep its `stop` removal and every other xAI
+ schema, tool-choice, replay, image, normalizer, and payload-rule constraint.
+ Preserve the current prepared target fields; do not restore the whole
+ original request or override configured payload rules.
+- Remove the now-unconsumed private `xaiSupportsReasoningEffort` helper,
+ its obsolete assertion-only test, unused imports, and the sanitizer's
+ now-unused model argument. No registry or registration policy change.
+- Replace catalog-strip assertions with actual executor evidence for unknown,
+ known-nil, empty/narrow, and supported metadata populations; native future
+ effort and explicit disable remain present, missing native intent remains
+ absent, and `stop` still does not reach Responses. Cover cross-protocol
+ conversion and stream/non-stream paths without claiming upstream support.
+- Reconcile the complete manager/credential-replacement and Kimi fixture
+ failures from the same test run independently of this xAI defect. A passing
+ xAI case alone is not source acceptance. Check the final egress preparation
+ of the other supported executor paths for an equivalent downstream
+ catalog-only gate; report another boundary before expanding changes.
+
+This narrowly supersedes the earlier blanket preservation of normalizers
+only for the identified catalog deletion. All changes stay in the inactive
+exact-base source/test patch. Avibe runtime/config/manifest, release guards,
+publication authority, and the installed engine remain unchanged.
+
+### Kimi native representation clarification
+
+The orchestrator inspected the pinned Kimi executor's original/prepared payload
+flow, the original Kimi applier and extraction precedence, the current shared
+native-format gate, and the failing actual OAuth executor test. An OpenAI Chat
+transport can already carry Kimi's native `thinking` object. Treating every
+`openai` to `kimi` pair as conversion rejects a native future effort using
+catalog levels, although extraction already gives the native object priority.
+This is the same representation-identity defect, not evidence to remove
+genuine cross-protocol validation.
+
+Approve this bounded source-only clarification:
+
+- With no model suffix, recognize `openai` to `kimi` as native only when the
+ original source contains `thinking.type` or `thinking.effort`. A `keep`-only
+ object and a legacy `reasoning_effort` input do not establish native intent.
+- Preserve the prepared target payload and its native object without restoring
+ source fields or interpreting unknown effort/type values. Remove only the
+ legacy `reasoning_effort` alias, consistent with the existing Kimi wire writer.
+ Native disabled input must not be re-enabled or replaced by the legacy alias.
+- Preserve suffix priority and the existing conversion owner for legacy-only,
+ keep-only, and genuine cross-protocol inputs. No new interface, metadata,
+ configuration, registration, routing, or authentication policy is needed.
+- Verify native enabled/future/disabled/type-only input, conflicting legacy
+ input, keep-only conversion, suffix priority, and prepared-target authority
+ through policy tests and actual streaming/non-streaming executor captures.
+
+The expected result is exact prepared native intent reaching the loopback
+upstream, with the legacy alias absent; it is not a claim that a real upstream
+supports an arbitrary future value. Any target normalization or another
+protocol constraint that makes this preservation unsafe requires evidence and
+a new scope decision. The patch remains inactive and the engine pin unchanged.
diff --git a/modules/agents/model_hub.py b/modules/agents/model_hub.py
index 3c82f8a8f..e28bfb690 100644
--- a/modules/agents/model_hub.py
+++ b/modules/agents/model_hub.py
@@ -220,7 +220,12 @@ def claude_settings_for_launch(base_settings: str, launch: ModelHubLaunch | None
if launch is None or launch.channel != "hub":
return base_settings
settings = json.loads(base_settings)
- settings["env"] = {**settings.get("env", {}), **build_claude_hub_env({}, launch)}
+ connection_env = build_claude_hub_env({}, launch)
+ # Catalog limits assist planning through the subprocess environment. They
+ # must not become launch-settings overrides of native project/local choices.
+ connection_env.pop("CLAUDE_CODE_MAX_CONTEXT_TOKENS", None)
+ connection_env.pop("CLAUDE_CODE_MAX_OUTPUT_TOKENS", None)
+ settings["env"] = {**settings.get("env", {}), **connection_env}
settings["apiKeyHelper"] = ""
return json.dumps(settings)
@@ -329,8 +334,6 @@ def build_claude_hub_env(
"CLAUDE_CODE_USE_BEDROCK",
"CLAUDE_CODE_USE_VERTEX",
"CLAUDE_CODE_USE_FOUNDRY",
- "CLAUDE_CODE_MAX_CONTEXT_TOKENS",
- "CLAUDE_CODE_MAX_OUTPUT_TOKENS",
}
result = {
key: "" if key.startswith("ANTHROPIC_") or key in masked_keys else value
@@ -341,13 +344,12 @@ def build_claude_hub_env(
result["ANTHROPIC_AUTH_TOKEN"] = launch.gateway_token
result["CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY"] = "1"
else:
- # A native_cli hop keeps the user's official CLI authentication while
- # applying metadata owned by the selected Hub catalog row.
+ # A native_cli hop keeps the user's official CLI authentication.
result = dict(base_env)
if launch.context_window is not None:
- result["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] = str(launch.context_window)
+ result.setdefault("CLAUDE_CODE_MAX_CONTEXT_TOKENS", str(launch.context_window))
if launch.max_output_tokens is not None:
- result["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] = str(launch.max_output_tokens)
+ result.setdefault("CLAUDE_CODE_MAX_OUTPUT_TOKENS", str(launch.max_output_tokens))
return result
diff --git a/tests/e2e/test_model_hub_catalog_consumer.py b/tests/e2e/test_model_hub_catalog_consumer.py
new file mode 100644
index 000000000..a09b21fa7
--- /dev/null
+++ b/tests/e2e/test_model_hub_catalog_consumer.py
@@ -0,0 +1,351 @@
+"""Exercise projected metadata in the actual CLI, without a managed engine."""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+import shlex
+import shutil
+import socket
+import subprocess
+from pathlib import Path
+
+import pytest
+
+from config.v2_config import ModelHubBackendModelConfig
+from modules.agents.codex.transport import CodexTransport
+from modules.agents.model_hub import (
+ ModelHubLaunch,
+ build_claude_hub_env,
+ build_codex_hub_launch,
+ claude_setting_sources_for_launch,
+ claude_settings_for_launch,
+)
+from tests.e2e.drivers.model_hub_app import ModelHubTestApp
+from tests.e2e.drivers.mock_llm_upstream import MockLLMUpstream
+from vibe.backend_model_catalog import _codex_hub_catalog_bytes
+
+
+pytestmark = pytest.mark.e2e_model_hub
+
+# A complete one-pixel PNG, supplied directly to the turn rather than downloaded.
+IMAGE_URL = (
+ "data:image/png;base64,"
+ "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
+)
+
+
+def rejected_external_proxy():
+ # Reserve a loopback port without listening: external HTTP(S) attempts fail
+ # at this test-owned transport boundary instead of reaching a real service.
+ with socket.socket() as reserved:
+ reserved.bind(("127.0.0.1", 0))
+ yield f"http://127.0.0.1:{reserved.getsockname()[1]}"
+
+
+def _isolated_runtime(tmp_path, rejected_external_proxy):
+ runtime = ModelHubTestApp(Path(__file__).resolve().parents[2], tmp_path / "runtime")
+ runtime.home.mkdir(parents=True)
+ Path(runtime.env["TMPDIR"]).mkdir()
+ for key in ("HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "http_proxy", "https_proxy", "all_proxy"):
+ runtime.env[key] = rejected_external_proxy
+ return runtime
+
+
+def _loopback_cli(binary, runtime):
+ sandbox = shutil.which("sandbox-exec")
+ if sandbox is None:
+ return str(binary)
+ # On macOS also enforce loopback-only egress below the CLI's HTTP stack,
+ # including background clients that might not honor standard proxy settings.
+ profile = '(version 1) (allow default) (deny network-outbound) (allow network-outbound (remote ip "localhost:*"))'
+ wrapper = runtime.home / "loopback-cli"
+ wrapper.write_text(
+ f"#!/bin/sh\nexec {shlex.quote(sandbox)} -p {shlex.quote(profile)} {shlex.quote(str(binary))} \"$@\"\n",
+ )
+ wrapper.chmod(0o700)
+ return str(wrapper)
+
+
+def codex_catalog_runtime(tmp_path, rejected_external_proxy):
+ binary = shutil.which("codex")
+ if binary is None:
+ pytest.skip("Codex executable is unavailable")
+ runtime = _isolated_runtime(tmp_path, rejected_external_proxy)
+ binary = _loopback_cli(binary, runtime)
+ Path(runtime.env["CODEX_HOME"]).mkdir()
+ node = shutil.which("node")
+ if node:
+ runtime.env["PATH"] += os.pathsep + str(Path(node).parent)
+ # This fixture uses the harness's allowlisted environment, never the live
+ # credential stores. The native binary can only discover test-owned config.
+ exported = subprocess.run(
+ [binary, "debug", "models", "--bundled", "-c", "model_catalog_json=null"],
+ cwd=runtime.home,
+ env=runtime.env,
+ capture_output=True,
+ timeout=30,
+ check=True,
+ )
+ return binary, runtime, exported.stdout
+
+
+async def _codex_turn(binary, runtime, gateway, catalog, *, effort, native_config="", turns=1):
+ codex_home = Path(runtime.env["CODEX_HOME"])
+ config_path = codex_home / "config.toml"
+ config_path.write_text('cli_auth_credentials_store = "file"\n' + native_config)
+ catalog_path = runtime.home / "models.json"
+ catalog_path.write_bytes(catalog)
+ model_id = json.loads(catalog)["models"][0]["slug"]
+ launch = ModelHubLaunch(
+ backend="codex",
+ channel="hub",
+ requested_model=model_id,
+ target_model="unrelated-upstream-alias",
+ runtime_model=model_id,
+ gateway_base_url=gateway.url,
+ gateway_token="isolated-catalog-fixture",
+ )
+ args, env = build_codex_hub_launch([], runtime.env, launch, model_catalog_path=catalog_path)
+ transport = CodexTransport(
+ binary=binary, cwd=str(runtime.home), runtime_args=args, runtime_env=env,
+ )
+ notifications = []
+ completed = asyncio.Event()
+
+ async def notify(method, params):
+ notifications.append((method, params))
+ if method == "turn/completed":
+ completed.set()
+
+ transport.on_notification(notify)
+ try:
+ await transport.start()
+ models = await transport.send_request("model/list", {})
+ config = await transport.send_request("config/read", {})
+ thread = await transport.send_request(
+ "thread/start",
+ {"model": model_id, "cwd": str(runtime.home), "approvalPolicy": "never"},
+ )
+ for _turn in range(turns):
+ completed.clear()
+ await transport.send_request(
+ "turn/start",
+ {
+ "threadId": thread["thread"]["id"],
+ "input": [
+ {"type": "text", "text": "Describe this image: 中文"},
+ {"type": "image", "url": IMAGE_URL},
+ ],
+ **({"effort": effort} if effort is not None else {}),
+ },
+ )
+ await asyncio.wait_for(completed.wait(), timeout=30)
+ finally:
+ await transport.stop()
+ assert config_path.read_text().startswith('cli_auth_credentials_store = "file"\n' + native_config)
+ bodies = [request["body"] for request in gateway.requests() if request["path"] == "/v1/responses"]
+ assert bodies, notifications
+ assert [
+ params["turn"]["status"] for method, params in notifications if method == "turn/completed"
+ ] == ["completed"] * turns
+ return bodies, models, notifications, config["config"]
+
+
[email protected]("effort", [None, "none", "minimal", "low", "medium", "high", "xhigh", "max"])
+def test_codex_unknown_catalog_preserves_supplied_image_and_effort(codex_catalog_runtime, effort):
+ """MH-PROTOCOL-004: absence must survive the real catalog/turn consumer."""
+ binary, runtime, raw_catalog = codex_catalog_runtime
+ model = ModelHubBackendModelConfig.from_payload({"id": "custom-unknown"})
+ # Round-trip the released shape so empty lists are covered, not just missing
+ # JSON keys. The model's alias deliberately differs from its route target.
+ catalog = _codex_hub_catalog_bytes(raw_catalog, [model.to_payload()])
+ with MockLLMUpstream() as gateway:
+ gateway.configure(protocol="openai_responses")
+ bodies, _models, _notifications, _config = asyncio.run(
+ _codex_turn(binary, runtime, gateway, catalog, effort=effort),
+ )
+ body = bodies[0]
+ assert body["model"] == model.id
+ content = [part for item in body["input"] for part in item.get("content", [])]
+ images = [part for part in content if part.get("type") == "input_image"]
+ assert [part["image_url"] for part in images] == [IMAGE_URL]
+ assert all(part.get("detail") != "original" for part in images)
+ assert any("中文" in part.get("text", "") for part in content)
+ assert (body.get("reasoning") or {}).get("effort") == effort
+ assert not body.get("service_tier")
+
+
+ "metadata,effort,expected_effort,image_expected",
+ [
+ ({"input_modalities": ["text"], "supports_reasoning": False}, None, "none", False),
+ ({"input_modalities": ["text"], "supports_reasoning": False, "reasoning_efforts": ["high"]}, None, "none", False),
+ ({"input_modalities": ["text", "image"], "reasoning_efforts": ["low", "high"]}, None, "low", True),
+ ({"input_modalities": ["text", "image"], "reasoning_efforts": ["low", "high"]}, "high", "high", True),
+ ({"supports_reasoning": True}, "high", "high", True),
+ ],
+)
+def test_codex_explicit_catalog_choices_reach_the_consumer(
+ codex_catalog_runtime, metadata, effort, expected_effort, image_expected,
+):
+ """MH-PROTOCOL-004: explicit negatives and effort lists remain meaningful."""
+ binary, runtime, raw_catalog = codex_catalog_runtime
+ model = ModelHubBackendModelConfig.from_payload({"id": "explicit-custom", **metadata})
+ catalog = _codex_hub_catalog_bytes(raw_catalog, [model.to_payload()])
+ with MockLLMUpstream() as gateway:
+ gateway.configure(protocol="openai_responses")
+ bodies, _models, _notifications, _config = asyncio.run(
+ _codex_turn(binary, runtime, gateway, catalog, effort=effort),
+ )
+ content = [part for item in bodies[0]["input"] for part in item.get("content", [])]
+ images = [part for part in content if part.get("type") == "input_image"]
+ assert [part["image_url"] for part in images] == ([IMAGE_URL] if image_expected else [])
+ assert bodies[0]["reasoning"]["effort"] == expected_effort
+
+
[email protected]("explicit_context", [None, 64_000, 512_000])
[email protected]("model_kind", ["custom", "native"])
+def test_codex_native_context_override_wins_over_catalog_planning(codex_catalog_runtime, explicit_context, model_kind):
+ """MH-PROTOCOL-004: the actual context consumer retains native overrides."""
+ binary, runtime, raw_catalog = codex_catalog_runtime
+ model_id = "context-alias" if model_kind == "custom" else json.loads(raw_catalog)["models"][0]["slug"]
+ model = ModelHubBackendModelConfig(id=model_id, context_window=128_000, max_output_tokens=32_000)
+ catalog = _codex_hub_catalog_bytes(raw_catalog, [model.to_payload()])
+ native_config = 'model = "unrelated-native-default"\n'
+ if explicit_context is not None:
+ native_config += f"model_context_window = {explicit_context}\nmodel_auto_compact_token_limit = 300000\n"
+ with MockLLMUpstream() as gateway:
+ gateway.configure(protocol="openai_responses")
+ bodies, _models, notifications, config = asyncio.run(
+ _codex_turn(binary, runtime, gateway, catalog, effort=None, native_config=native_config),
+ )
+ usage = [params["tokenUsage"] for method, params in notifications if method == "thread/tokenUsage/updated"]
+ percent = json.loads(catalog)["models"][0].get("effective_context_window_percent", 95)
+ assert usage, [method for method, _params in notifications]
+ if explicit_context is not None:
+ assert config["model_context_window"] == explicit_context
+ assert config["model_auto_compact_token_limit"] == 300_000
+ assert usage[-1]["modelContextWindow"] == (explicit_context or model.context_window) * percent // 100
+ assert bodies[0]["model"] == model.id
+ # BackendModel output metadata has no Codex output-cap projection.
+ assert "max_output_tokens" not in bodies[0]
+
+
+ "native_config,input_tokens,compacts",
+ [
+ ("", 150_000, True),
+ ("model_context_window = 512000\nmodel_auto_compact_token_limit = 300000\n", 150_000, False),
+ ("model_context_window = 512000\nmodel_auto_compact_token_limit = 300000\n", 350_000, True),
+ ],
+)
+def test_codex_auto_compaction_consumes_resolved_limits(
+ codex_catalog_runtime, monkeypatch, native_config, input_tokens, compacts,
+):
+ """MH-PROTOCOL-004: compaction follows the active window and explicit threshold."""
+ from tests.e2e.drivers import mock_llm_upstream
+
+ binary, runtime, raw_catalog = codex_catalog_runtime
+ model = ModelHubBackendModelConfig(id="compact-alias", context_window=128_000)
+ catalog = _codex_hub_catalog_bytes(raw_catalog, [model.to_payload()])
+ original_frames = mock_llm_upstream._responses_stream_frames
+ calls = 0
+
+ def frames_with_large_first_usage(model_id):
+ nonlocal calls
+ frames = original_frames(model_id)
+ calls += 1
+ if calls == 1:
+ completed = json.loads(frames[-1].split(b"data: ", 1)[1])
+ completed["response"]["usage"]["input_tokens"] = input_tokens
+ completed["response"]["usage"]["total_tokens"] = input_tokens + 5
+ frames[-1] = mock_llm_upstream._sse("response.completed", completed)
+ return frames
+
+ monkeypatch.setattr(mock_llm_upstream, "_responses_stream_frames", frames_with_large_first_usage)
+ with MockLLMUpstream() as gateway:
+ gateway.configure(protocol="openai_responses")
+ bodies, _models, notifications, _config = asyncio.run(
+ _codex_turn(binary, runtime, gateway, catalog, effort=None, native_config=native_config, turns=2),
+ )
+ compacted = [
+ params for method, params in notifications
+ if method == "item/completed" and params.get("item", {}).get("type") == "contextCompaction"
+ ]
+ assert bool(compacted) is compacts
+ assert len(bodies) == (3 if compacts else 2)
+
+
[email protected]("channel", ["hub", "native_cli"])
[email protected]("settings_scope", ["parent", "project", "local", "catalog"])
+def test_claude_consumer_preserves_explicit_limit_settings(tmp_path, rejected_external_proxy, channel, settings_scope):
+ """MH-CLAUDE-LAUNCH-001: limits remain separate from transport ownership."""
+ import claude_agent_sdk
+
+ binary = Path(claude_agent_sdk.__file__).parent / "_bundled" / "claude"
+ if not binary.is_file():
+ pytest.skip("The installed Claude SDK has no bundled CLI")
+ runtime = _isolated_runtime(tmp_path, rejected_external_proxy)
+ binary = _loopback_cli(binary, runtime)
+ project = runtime.home / "project"
+ project.mkdir(parents=True)
+ selected_limits = {
+ "CLAUDE_CODE_MAX_CONTEXT_TOKENS": "234567",
+ "CLAUDE_CODE_MAX_OUTPUT_TOKENS": "1234",
+ }
+ fixture_settings = None
+ if settings_scope in {"project", "local"}:
+ filename = "settings.local.json" if settings_scope == "local" else "settings.json"
+ fixture_settings = project / ".claude" / filename
+ fixture_settings.parent.mkdir()
+ fixture_settings.write_text(json.dumps({"env": selected_limits}))
+ elif settings_scope == "parent":
+ runtime.env.update(selected_limits)
+ runtime.env["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"] = "1"
+ runtime.env["CLAUDE_CODE_ENTRYPOINT"] = "sdk-py"
+ with MockLLMUpstream() as gateway:
+ gateway.configure(protocol="anthropic")
+ launch = ModelHubLaunch(
+ backend="claude",
+ channel=channel,
+ requested_model="planning-alias",
+ target_model="custom-limit-model",
+ runtime_model="custom-limit-model",
+ gateway_base_url=gateway.url,
+ gateway_token="isolated-claude-fixture",
+ context_window=128_000,
+ max_output_tokens=4096,
+ )
+ # Native CLI authority is tested with synthetic local credentials, never
+ # OAuth/keychain discovery. All filesystem paths belong to the fixture.
+ runtime.env["ANTHROPIC_BASE_URL"] = gateway.url
+ runtime.env["ANTHROPIC_AUTH_TOKEN"] = "isolated-native-fixture"
+ env = build_claude_hub_env(runtime.env, launch)
+ result = subprocess.run(
+ [
+ str(binary), "-p", "Reply with hello 中文.", "--output-format", "json",
+ "--model", launch.runtime_model, "--tools", "", "--max-turns", "1",
+ "--setting-sources", ",".join(claude_setting_sources_for_launch(launch)),
+ "--settings", claude_settings_for_launch("{}", launch),
+ ],
+ cwd=project,
+ env=env,
+ capture_output=True,
+ text=True,
+ timeout=30,
+ )
+ assert result.returncode == 0, (result.stdout[-2000:], result.stderr[-2000:])
+ bodies = [request["body"] for request in gateway.requests() if request["path"] == "/v1/messages"]
+ assert bodies
+ expected_output = 4096 if settings_scope == "catalog" else 1234
+ assert bodies[-1]["model"] == launch.runtime_model
+ assert bodies[-1]["max_tokens"] == expected_output
+ # Normal Claude compaction currently ignores MAX_CONTEXT_TOKENS; do not
+ # certify its modelUsage display as a consumer of this environment key.
+ # SessionHandler coverage separately checks delivery of that planning value.
+ if fixture_settings is not None:
+ assert json.loads(fixture_settings.read_text()) == {"env": selected_limits}
diff --git a/tests/scenarios/INDEX.yaml b/tests/scenarios/INDEX.yaml
index 5463d5b02..3c805ad79 100644
--- a/tests/scenarios/INDEX.yaml
+++ b/tests/scenarios/INDEX.yaml
@@ -133,6 +133,7 @@ capabilities:
- tests/e2e/test_model_hub_mock_upstream.py
- tests/e2e/test_model_hub_routing.py
- tests/e2e/test_model_hub_backend_settings.py
+ - tests/e2e/test_model_hub_catalog_consumer.py
- tests/e2e/test_model_hub_runtime.py
- tests/e2e/test_model_hub_sources.py
- tests/e2e/test_model_hub_usage_events.py
diff --git a/tests/scenarios/model_hub/catalog.yaml b/tests/scenarios/model_hub/catalog.yaml
index 3d58c8467..29322627f 100644
--- a/tests/scenarios/model_hub/catalog.yaml
+++ b/tests/scenarios/model_hub/catalog.yaml
@@ -35,6 +35,7 @@ capability:
- tests/e2e/test_model_hub_mock_upstream.py
- tests/e2e/test_model_hub_routing.py
- tests/e2e/test_model_hub_backend_settings.py
+ - tests/e2e/test_model_hub_catalog_consumer.py
- tests/e2e/test_model_hub_runtime.py
- tests/e2e/test_model_hub_sources.py
- tests/e2e/test_model_hub_usage_events.py
@@ -254,12 +255,21 @@ scenarios:
layer: contract
missing_layer: scenario
test: tests/test_model_hub_injection.py::test_codex_hub_launch_uses_responses_wire_api_and_ephemeral_token
+ partial_evidence:
+ covers: Real Codex catalog and turn consumption preserves unknown image and explicit effort intent; this does not exercise the managed engine
+ test: tests/e2e/test_model_hub_catalog_consumer.py::test_codex_unknown_catalog_preserves_supplied_image_and_effort
+ related_tests:
+ - tests/e2e/test_model_hub_catalog_consumer.py::test_codex_explicit_catalog_choices_reach_the_consumer
+ - tests/e2e/test_model_hub_catalog_consumer.py::test_codex_native_context_override_wins_over_catalog_planning
+ - tests/e2e/test_model_hub_catalog_consumer.py::test_codex_auto_compaction_consumes_resolved_limits
- id: MH-CLAUDE-LAUNCH-001
name: Claude Hub turns preserve the configured Source and model despite native connection settings
status: covered
kind: injection
layer: e2e
test: tests/e2e/test_model_hub_routing.py::test_mh_claude_launch_001_settings_cannot_bypass_the_configured_route
+ related_tests:
+ - tests/e2e/test_model_hub_catalog_consumer.py::test_claude_consumer_preserves_explicit_limit_settings
- id: MH-CODEX-LAUNCH-001
name: Native Codex provider settings preserve the configured Hub Source and upstream model
status: covered
diff --git a/tests/test_backend_model_catalog.py b/tests/test_backend_model_catalog.py
index 89c0f6e0e..88a8d52c6 100644
--- a/tests/test_backend_model_catalog.py
+++ b/tests/test_backend_model_catalog.py
@@ -242,7 +242,6 @@ def test_codex_hub_catalog_projects_custom_backend_models_from_native_shape():
"visibility": "list",
"supported_in_api": True,
"context_window": 1_048_576,
- "max_context_window": 1_048_576,
"input_modalities": ["text"],
"supports_image_detail_original": False,
"default_reasoning_level": "low",
@@ -313,8 +312,10 @@ def test_codex_hub_catalog_does_not_inherit_native_model_metadata_for_custom_row
assert custom["model_messages"] == {"instructions_template": "preserved"}
assert custom["shell_type"] == "unified_exec"
assert custom["truncation_policy"] == {"mode": "tokens", "limit": 10_000}
- assert custom["input_modalities"] == ["text"]
- assert custom["supported_reasoning_levels"] == [{"effort": "none", "description": "None"}]
+ assert "input_modalities" not in custom
+ assert custom["supports_image_detail_original"] is False
+ assert custom["supported_reasoning_levels"] == []
+ assert "default_reasoning_level" not in custom
assert custom["support_verbosity"] is False
assert custom["experimental_supported_tools"] == []
assert custom["supports_parallel_tool_calls"] is False
@@ -367,6 +368,8 @@ def test_codex_hub_catalog_preserves_native_modalities_without_an_override():
assert payload["models"][0]["input_modalities"] == ["text", "image"]
assert payload["models"][0]["supports_image_detail_original"] is True
+ assert payload["models"][0]["context_window"] == 200_000
+ assert payload["models"][0]["max_context_window"] == 200_000
assert payload["models"][0]["auto_compact_token_limit"] == 180_000
@@ -447,18 +450,19 @@ def test_codex_hub_catalog_retires_native_compaction_limit_after_context_overrid
projected = payload["models"][0]
assert projected["context_window"] == 400_000
- assert projected["max_context_window"] == 400_000
+ assert "max_context_window" not in projected
assert projected["effective_context_window_percent"] == 90
assert "auto_compact_token_limit" not in projected
-def test_codex_hub_catalog_does_not_give_custom_models_template_modalities():
[email protected]("template_modalities", [["text"], ["text", "image"]])
+def test_codex_hub_catalog_leaves_unknown_custom_modalities_to_the_consumer(template_modalities):
raw = json.dumps(
{
"models": [
{
"slug": "gpt-native",
- "input_modalities": ["text", "image"],
+ "input_modalities": template_modalities,
"supports_image_detail_original": True,
}
]
@@ -472,10 +476,38 @@ def test_codex_hub_catalog_does_not_give_custom_models_template_modalities():
)
)
- assert payload["models"][0]["input_modalities"] == ["text"]
+ assert "input_modalities" not in payload["models"][0]
assert payload["models"][0]["supports_image_detail_original"] is False
[email protected]("supports_reasoning", [None, True, False])
[email protected]("efforts", [[], ["low", "high"]])
[email protected]("modalities", [[], ["text"], ["text", "image"], ["audio"]])
+def test_codex_custom_catalog_preserves_explicit_capability_choices(supports_reasoning, efforts, modalities):
+ from config.v2_config import ModelHubBackendModelConfig
+
+ raw = b'{"models":[{"slug":"unrelated","default_reasoning_level":"xhigh","context_window":999999}]}'
+ configured = ModelHubBackendModelConfig.from_payload({
+ "id": "custom",
+ "supports_reasoning": supports_reasoning,
+ "reasoning_efforts": efforts,
+ "input_modalities": modalities,
+ }).to_payload()
+ row = json.loads(backend_model_catalog._codex_hub_catalog_bytes(raw, [configured]))["models"][0]
+
+ if modalities:
+ assert row["input_modalities"] == ([item for item in modalities if item in {"text", "image"}] or ["text"])
+ else:
+ assert "input_modalities" not in row
+ settled = ["none"] if supports_reasoning is False else efforts
+ assert [item["effort"] for item in row["supported_reasoning_levels"]] == settled
+ if settled:
+ assert row["default_reasoning_level"] == settled[0]
+ else:
+ assert "default_reasoning_level" not in row
+ assert "context_window" not in row
+
+
def test_codex_hub_catalog_can_disable_reasoning_and_preserve_native_default():
raw = json.dumps(
{
diff --git a/tests/test_claude_cli_path.py b/tests/test_claude_cli_path.py
index 8b4652102..b83381168 100644
--- a/tests/test_claude_cli_path.py
+++ b/tests/test_claude_cli_path.py
@@ -236,6 +236,58 @@ def test_session_handler_uses_native_cli_launch_reasoning_catalog(
assert captured["options"].env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] == "32000"
[email protected]("channel", ["hub", "native_cli"])
[email protected]("explicit", [None, "", "333333"])
[email protected]("has_metadata", [False, True])
+def test_session_handler_preserves_explicit_limits_and_passes_alias_planning_metadata(
+ monkeypatch, tmp_path: Path, channel, explicit, has_metadata,
+) -> None:
+ import json
+ from modules.agents.model_hub import ModelHubLaunch
+
+ captured = {}
+ keys = ("CLAUDE_CODE_MAX_CONTEXT_TOKENS", "CLAUDE_CODE_MAX_OUTPUT_TOKENS")
+ for key in keys:
+ if explicit is None:
+ monkeypatch.delenv(key, raising=False)
+ else:
+ monkeypatch.setenv(key, explicit)
+
+ class Client:
+ def __init__(self, options):
+ captured["options"] = options
+
+ async def connect(self):
+ pass
+
+ class Runtime:
+ async def resolve(self, backend, requested_model, **_kwargs):
+ return ModelHubLaunch(
+ backend=backend, channel=channel,
+ requested_model=requested_model, target_model="custom-upstream",
+ runtime_model="custom-upstream", source_id="src_limitfixture",
+ gateway_base_url="http://127.0.0.1:18443/claude",
+ gateway_token="limit-fixture-token",
+ context_window=128_000 if has_metadata else None,
+ max_output_tokens=32_000 if has_metadata else None,
+ )
+
+ monkeypatch.setattr(session_handler_module, "ClaudeAgentOptions", _StubClaudeAgentOptions)
+ monkeypatch.setattr(session_handler_module, "ClaudeSDKClient", Client)
+ controller = _Controller(tmp_path)
+ controller.model_hub_runtime = Runtime()
+ controller.settings_manager.get_channel_routing = lambda _: RoutingSettings(model="planning-alias")
+ _run_session(SessionHandler(controller), MessageContext(user_id="U123", channel_id="C123"))
+
+ options = captured["options"]
+ assert options.extra_args["model"] == "custom-upstream"
+ for key, fallback in zip(keys, ("128000", "32000")):
+ expected = explicit if explicit is not None else fallback if has_metadata else None
+ assert options.env.get(key) == expected
+ assert key not in json.loads(options.settings).get("env", {})
+ assert options.setting_sources == (["project", "local"] if channel == "hub" else ["user", "project", "local"])
+
+
def test_session_handler_pins_hub_connection_in_launch_settings(monkeypatch, tmp_path: Path) -> None:
import json
from modules.agents.model_hub import ModelHubLaunch
diff --git a/tests/test_model_hub_injection.py b/tests/test_model_hub_injection.py
index 24114da13..54dd0284e 100644
--- a/tests/test_model_hub_injection.py
+++ b/tests/test_model_hub_injection.py
@@ -137,8 +137,8 @@ def test_hub_launch_masks_inherited_claude_auth_and_injects_gateway():
assert env["ANTHROPIC_BASE_URL"] == launch.gateway_base_url
assert env["ANTHROPIC_AUTH_TOKEN"] == launch.gateway_token
assert env["CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY"] == "1"
- assert env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] == "128000"
- assert env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] == "32000"
+ assert env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] == "999999"
+ assert env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] == "999999"
assert env["CLAUDE_CODE_OAUTH_TOKEN"] == ""
assert claude_setting_sources_for_launch(launch) == ["project", "local"]
@@ -187,7 +187,7 @@ def test_claude_hub_launch_cannot_fall_back_to_native_auth(missing):
claude_settings_for_launch("{}", launch)
-def test_native_cli_launch_keeps_auth_and_applies_catalog_limits():
+def test_native_cli_launch_keeps_auth_and_explicit_limits():
launch = hub_launch(
channel="native_cli",
gateway_base_url=None,
@@ -209,11 +209,89 @@ def test_native_cli_launch_keeps_auth_and_applies_catalog_limits():
assert env["ANTHROPIC_AUTH_TOKEN"] == "user-token"
assert env["ANTHROPIC_BASE_URL"] == "https://user.example"
assert env["CLAUDE_CODE_OAUTH_TOKEN"] == "oauth-token"
- assert env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] == "128000"
- assert env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] == "32000"
+ assert env["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] == "999999"
+ assert env["CLAUDE_CODE_MAX_OUTPUT_TOKENS"] == "999999"
assert "CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY" not in env
[email protected]("channel", ["hub", "native_cli"])
[email protected]("explicit", [{}, {"CLAUDE_CODE_MAX_CONTEXT_TOKENS": ""}, {"CLAUDE_CODE_MAX_OUTPUT_TOKENS": "8192"}])
[email protected]("limits", [{}, {"context_window": 128_000, "max_output_tokens": 32_000}])
+def test_claude_catalog_limits_fill_only_absent_environment_values(channel, explicit, limits):
+ launch = hub_launch(channel=channel, **limits)
+ original = dict(explicit)
+ env = build_claude_hub_env(explicit, launch)
+ expected = {
+ env_key: explicit.get(env_key, str(limits[field]) if field in limits else None)
+ for env_key, field in (
+ ("CLAUDE_CODE_MAX_CONTEXT_TOKENS", "context_window"),
+ ("CLAUDE_CODE_MAX_OUTPUT_TOKENS", "max_output_tokens"),
+ )
+ }
+ assert {key: env.get(key) for key in expected} == expected
+ assert explicit == original
+
+
+def test_hub_connection_settings_preserve_explicit_limits_without_promoting_catalog_defaults():
+ launch = hub_launch(context_window=128_000, max_output_tokens=32_000)
+ settings = json.loads(claude_settings_for_launch(
+ json.dumps({"env": {"CLAUDE_CODE_MAX_CONTEXT_TOKENS": ""}}), launch,
+ ))
+ assert settings["env"]["CLAUDE_CODE_MAX_CONTEXT_TOKENS"] == ""
+ assert "CLAUDE_CODE_MAX_OUTPUT_TOKENS" not in settings["env"]
+ assert settings["env"]["ANTHROPIC_AUTH_TOKEN"] == launch.gateway_token
+
+
[email protected]("backend", ["claude", "codex", "opencode"])
[email protected]("has_metadata", [False, True])
+def test_route_alias_keeps_its_own_planning_metadata(tmp_path, monkeypatch, backend, has_metadata):
+ from unittest.mock import AsyncMock
+ from config.v2_config import ModelHubBackendModelConfig
+ from modules.agents.model_hub import ModelHubRuntimeRouter
+ from tests.scenario_harness.model_hub import (
+ MemoryModelHubStore, ModelHubScenarioAdapter, config_with_sources, service_for, source,
+ )
+
+ monkeypatch.setenv("VIBE_MODEL_HUB_ENABLED", "1")
+ supplied = source("src_planning", ["upstream-target"])
+ config = config_with_sources(
+ [supplied], backend=backend, menu_model="planning-alias",
+ hops=[(supplied.id, "upstream-target")],
+ )
+ config.agents[backend].models = [
+ ModelHubBackendModelConfig(
+ id="planning-alias", native_protocol="openai_responses" if backend == "opencode" else None,
+ context_window=128_000 if has_metadata else None,
+ max_output_tokens=32_000 if has_metadata else None,
+ ),
+ ModelHubBackendModelConfig(
+ id="upstream-target", native_protocol="openai_responses" if backend == "opencode" else None,
+ context_window=999_999, max_output_tokens=99_999,
+ ),
+ ]
+ if backend == "opencode":
+ config.agents[backend].menu.checked = ["planning-alias"]
+ before = config.to_payload()
+ store = MemoryModelHubStore(config)
+ router = ModelHubRuntimeRouter(
+ service=service_for(tmp_path, store, ModelHubScenarioAdapter()),
+ turn_gateway=SimpleNamespace(endpoint=AsyncMock(return_value=("http://127.0.0.1:19000", "fixture-token"))),
+ overlay_path=tmp_path / "overlay.json",
+ )
+
+ launch = asyncio.run(router.resolve(backend, "planning-alias"))
+ assert launch.requested_model == "planning-alias"
+ assert launch.target_model == "upstream-target"
+ assert launch.context_window == (128_000 if has_metadata else None)
+ assert launch.max_output_tokens == (32_000 if has_metadata else None)
+ if backend == "opencode":
+ overlay = asyncio.run(router.prepare_opencode_overlay())
+ row = json.loads(overlay.content)["provider"]["avibe-openai"]["models"]["planning-alias"]
+ assert row.get("limit") == ({"context": 128_000, "output": 32_000} if has_metadata else None)
+ assert store.load().to_payload() == before
+ assert store.saved_payloads == []
+
+
def test_codex_hub_launch_uses_responses_wire_api_and_ephemeral_token(tmp_path):
"""MH-PROTOCOL-004: Hub launches consume a prepared standard Responses catalog."""
diff --git a/vibe/backend_model_catalog.py b/vibe/backend_model_catalog.py
index c992b46ac..07cbb7654 100644
--- a/vibe/backend_model_catalog.py
+++ b/vibe/backend_model_catalog.py
@@ -202,22 +202,25 @@ def _codex_hub_catalog_bytes(
and context_window > 0
):
row["context_window"] = context_window
- row["max_context_window"] = context_window
+ # This saved planning value is not an independently known hard
+ # ceiling. Codex clamps explicit model_context_window overrides
+ # to max_context_window, so duplicating it here silently defeats
+ # a user's larger native setting.
+ row.pop("max_context_window", None)
# Codex derives this from the active window when omitted; a
# bundled value belongs to the native window we just replaced.
row.pop("auto_compact_token_limit", None)
input_modalities = configured.get("input_modalities")
- # Persisted built-ins use an empty list to mean "not overridden".
- # Custom rows still need a conservative text-only default instead
- # of inheriting the native template model's image capabilities.
- if native_row is None or (
- isinstance(input_modalities, list) and input_modalities
- ):
+ # Empty lists in released rows mean "not overridden". Codex's
+ # omitted-modality default accepts text and images; narrowing that
+ # to text would discard supplied images merely for lack of metadata.
+ # Original-resolution detail is a separate provider capability.
+ if native_row is None:
+ row["supports_image_detail_original"] = False
+ if isinstance(input_modalities, list) and input_modalities:
supported_modalities = [
modality
- for modality in (
- input_modalities if isinstance(input_modalities, list) else []
- )
+ for modality in input_modalities
if modality in {"text", "image"}
]
row["input_modalities"] = supported_modalities or ["text"]
@@ -233,12 +236,17 @@ def _codex_hub_catalog_bytes(
if isinstance(effort, str) and effort
]
elif native_row is None:
- settled_efforts = ["none"]
+ # The list is required by Codex's schema, but an unknown model
+ # has neither an advertised ladder nor a default effort. Codex
+ # can still carry the effort explicitly supplied with a turn.
+ settled_efforts = []
else:
settled_efforts = None
if settled_efforts is not None:
native_default = row.get("default_reasoning_level")
- if not (
+ if not settled_efforts:
+ row.pop("default_reasoning_level", None)
+ elif not (
isinstance(native_default, str)
and native_default in settled_efforts
):
@@ -247,7 +255,8 @@ def _codex_hub_catalog_bytes(
if "medium" in settled_efforts
else settled_efforts[0]
)
- row["default_reasoning_level"] = native_default
+ if settled_efforts:
+ row["default_reasoning_level"] = native_default
row["supported_reasoning_levels"] = [
{
"effort": effort,