Skip to content

Agent Mesh

docs: the node rotates among remote inference providers

Tomas Tormo956ea516cebc
diff --git a/site/content/docs/concepts/networking.md b/site/content/docs/concepts/networking.md
index 80e413f8..8d5e50df 100644
--- a/site/content/docs/concepts/networking.md
+++ b/site/content/docs/concepts/networking.md
@@ -142,9 +142,9 @@ deliver every event to them, and run the gossip mesh among themselves.
LiteLLM, or a hosted API behind a local proxy. The node's `/v1` endpoint
hides which node serves a model. It keeps a short-lived view of the model
list of every reachable provider. It prefers a local provider when one
-serves the model, and otherwise picks among remote providers, ranked by the
-labels it knows about and by load. Backends are registered by their root URL
-and the node adds the `/v1` prefix. The same backend is therefore reachable
+serves the model. Otherwise it rotates among the remote providers that serve
+the model, and skips a provider that failed recently. Backends are
+registered by their root URL and the node adds the `/v1` prefix. The same backend is therefore reachable
through the `/v1` endpoint and, with `/v1/...` appended, through the proxy
path.
diff --git a/site/content/docs/reference/node-api.md b/site/content/docs/reference/node-api.md
index e44f57d6..e6fffc48 100644
--- a/site/content/docs/reference/node-api.md
+++ b/site/content/docs/reference/node-api.md
@@ -301,9 +301,11 @@ a2a send -a $CARD "hello"
`/v1/models` collects the model list of every reachable `inference` provider
(cached for 30 seconds) and returns the union. `owned_by` on each entry is
the peer that serves the model. A completion request names a model, and the
-node picks a provider that serves it. It prefers a local provider, and ranks
-remote providers by the labels the caller required, the operator's floor,
-and load. A model that no provider lists answers `404`.
+node picks a provider that serves it. It prefers a local provider.
+Otherwise it rotates among the remote providers that serve the model, and
+skips one that failed recently. A remote provider must attest the labels the
+caller required and the operator's floor before the node sends it the
+request. A model that no provider lists answers `404`.
```bash
curl -s --unix-socket $SOCK http://localhost/v1/models