Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(serve): demonstrate local and deployed routing snapshots

wowcf2963bc010b
diff --git a/docs/http-api.md b/docs/http-api.md
index 4e8a453a..d49071e7 100644
--- a/docs/http-api.md
+++ b/docs/http-api.md
@@ -171,6 +171,24 @@ returns `422` with its message, including one raised by an operator hook. Other
are logged server-side and return `500` with `routing failed`.
There is no batch routing endpoint or SDK routing method in this change.
+#### Try a configuration mismatch locally
+
+From a checkout, run the complete example without downloading checkpoints or starting a server:
+
+```bash
+python -m pip install -e ".[serve]" httpx
+python examples/http_route.py
+```
+
+The example uses FastAPI's in-process `TestClient` against the real handler. Its local Router
+defaults to `multilingual`, while its injected server Router defaults to `english`. The same short
+state produces identical language diagnostics but different model selections, demonstrating why
+local diagnostics cannot establish a deployment's decision. It also checks the default three-field
+response and an explicit model pin, whose diagnostic is `null`, and verifies that neither Router
+loaded checkpoints. Each response is a snapshot of that request, not a reservation for later
+inference. Injecting the Router keeps the example independent of normal startup preload;
+`httpx` is needed for the test client, not by the routing endpoint.
+
### `POST /v1/systemone`
One request carries a `state` and any number of questions over it:
diff --git a/examples/http_route.py b/examples/http_route.py
new file mode 100644
index 00000000..ba77ddc8
--- /dev/null
+++ b/examples/http_route.py
@@ -0,0 +1,59 @@
+"""Compare local routing with the deployed Router, without loading checkpoints.
+
+From a checkout with the serve extra and its HTTP test client installed:
+ python -m pip install -e ".[serve]" httpx
+ python examples/http_route.py
+
+TestClient exercises the real HTTP handler in process; it opens no listening socket.
+"""
+import json
+import os
+
+from _common import Router
+from fastapi.testclient import TestClient
+
+from laya.serve import create_app
+
+
+def main():
+ state = {"message": "Turn lights off"}
+ body = {"state": state, "questions": {}}
+ local = Router(default="multilingual")
+ deployed = Router(default="english")
+ local_decision = local.route(state, explain=True)
+
+ # Injecting a Router avoids the normal server's startup preload. Honour a key
+ # from the environment without printing it or sending it over a network.
+ api_key = os.environ.get("LAYA_API_KEY")
+ headers = {"Authorization": "Bearer " + api_key} if api_key else {}
+ with TestClient(create_app(deployed), headers=headers) as client:
+ plain = client.post("/v1/route", json=body)
+ plain.raise_for_status()
+ assert set(plain.json()) == {"model", "repo", "reason"}
+
+ response = client.post("/v1/route", json={**body, "explain": True})
+ response.raise_for_status()
+ snapshot = response.json()
+ assert local_decision["model"] == "multilingual"
+ assert snapshot["model"] == "english"
+ assert snapshot["diagnostic"] == local_decision["detection"]["diagnostic"]
+ assert {key: snapshot[key] for key in plain.json()} == plain.json()
+
+ # A new request with a model pin has a new decision. The earlier snapshot
+ # neither pins a later prediction nor manufactures language evidence.
+ response = client.post("/v1/route", json={**body, "model": "multilingual", "explain": True})
+ response.raise_for_status()
+ pinned = response.json()
+ assert pinned["model"] == "multilingual" and pinned["diagnostic"] is None
+
+ assert local.loaded == [] and deployed.loaded == []
+ print(json.dumps({
+ "local_model": local_decision["model"],
+ "server_snapshot": snapshot,
+ "server_snapshot_with_pin": pinned,
+ "resident_checkpoints": deployed.loaded,
+ }, indent=2))
+
+
+if __name__ == "__main__":
+ main()