The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.
Rust agent-orchestrationai-agentsai-assistantdesktopllmlocal-firstmcppersonal-aiprivacyrustsecond-braintauri
docs(inference): update context window explanation
Steven Enamakel77d504a520c6
diff --git a/crates/openhuman-core/src/inference/README.md b/crates/openhuman-core/src/inference/README.mdindex e747c97c42..7bd18b1c43 100644--- a/crates/openhuman-core/src/inference/README.md+++ b/crates/openhuman-core/src/inference/README.md@@ -128,23 +128,25 @@ do not each instantiate its state machine.### Context windows-The context window drives pre-dispatch trimming and the compaction trigger, so [`context_window.rs`](./context_window.rs)-resolves it per turn in this order:+The context window drives pre-dispatch trimming and the compaction trigger. [`context_window.rs`](./context_window.rs)+supplies the selected route's discovery request, config override, local profile,+catalog hint, managed-provider default, and configured CA/proxy client to+TinyInference's route resolver. TinyInference owns discovery URL normalization,+caching, remembered route limits, and the shared fallback order:```text1. config override model_registry[].context_window (non-zero)- 2. provider-reported tinyinference_llm::model::discover, cached per- endpoint+model, lowered by any limit learned from- a context-overflow error on that endpoint- 3. local profile for local runtimes- 4. static guess model_context::static_context_window_for_model,- logged at warn once per model+ 2. provider-reported discovered limits and learned overflow limits for+ the exact endpoint/model/cache identity+ 3. remembered provider limits for the route/model identity+ 4. static/local managed default, workload tier, pricing catalog,+ model-id hint, then local profile fallback```The discovery request comes from [`provider/factory/discovery.rs`](./provider/factory/discovery.rs) (`model_limits_request`), built-from the same endpoint and credential resolution the chat factory uses. The resolved value is-remembered, so the synchronous `context_window_for_model` (usage meters, the context breakdown)-reports the same number the turn used. [`model_context.rs`](./model_context.rs) also answers vision capability through+from the same endpoint and credential resolution the chat factory uses. Route-aware synchronous+projection uses the exact discovery identity when available; model-only displays use a conservative+route projection and cannot borrow a learned limit from an unrelated provider. [`model_context.rs`](./model_context.rs) also answers vision capability through`model_supports_vision` and the user's `model_registry` `vision` flag.### Failures