Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(serve): expose server-side routing diagnostics over HTTP

wow1329fed11e2c
diff --git a/docs/http-api.md b/docs/http-api.md
index f5d5e854..4e8a453a 100644
--- a/docs/http-api.md
+++ b/docs/http-api.md
@@ -3,7 +3,8 @@
`laya-serve` exposes Laya over the TypeSafe Jev `/v1/systemone` wire protocol. A client written
against Jev -- `hs-jev`, `typesafe-sdk`, or your own -- can point its base URL at this server and
keep working: Laya's `predict()` output is already schema-compatible, and the server adds only the
-HTTP surface: one decision route, a health probe, an optional bearer check and request limits.
+HTTP surface: single and batch inference, a routing snapshot, a health probe, an optional bearer
+check and request limits.
The [Python HTTP client](https://github.com/NandhaKishorM/laya/blob/main/sdk/python/README.md)
provides synchronous `predict` and `health` calls with no runtime dependencies. Install it
@@ -53,7 +54,7 @@ Everything is environment variables, so one image serves a laptop dev run and a
For a deployment published under a prefix such as `/laya`, set `LAYA_ROOT_PATH=/laya`.
FastAPI uses it when generating OpenAPI and Swagger UI URLs. Configure the reverse proxy to
strip `/laya` before forwarding requests to Laya; the app's routes remain `/health` and
-`/v1/systemone` internally.
+`/v1/systemone`, `/v1/systemone/batch` and `/v1/route` internally.
For containers, including CUDA and ARM64 images, see [Docker quickstart](docker.md).
@@ -88,7 +89,8 @@ sample to the handler that produces it, field by field.
With idle unloading enabled, authenticated health responses also include `idle_unload_seconds`
(the configured window) and `idle_seconds` (time since the last inference request or completion).
-Health probes do not reset that clock. An empty `loaded` list is normal after an idle unload.
+Health probes and routing snapshots do not reset that clock. An empty `loaded` list is normal
+after an idle unload.
- `status` is `ok` whenever the process answers at all. It says nothing about the checkpoints.
- `loaded` lists the checkpoints resident in memory. It is empty until a request builds one, which is
@@ -109,6 +111,66 @@ Health probes do not reset that clock. An empty `loaded` list is normal after an
of the latest one. The demotion is scoped to the request that failed, so a checkpoint built on CPU
because the GPU was never available is not a fallback and counts `0` here -- that shows in `device`.
+### `POST /v1/route`
+
+Inspect the deployed Router's checkpoint selection without requesting inference. This is useful
+when an application's local routing configuration differs from the service's. MCP remote routing
+still runs locally; it does not call this endpoint.
+
+```bash
+curl -s localhost:8000/v1/route -H 'content-type: application/json' -d '{
+ "state": {"message": "Turn lights off"},
+ "questions": {},
+ "explain": true
+}'
+```
+
+With the default fallback model, this returns:
+
+```json
+{"model": "multilingual", "repo": "convaiinnovations/laya/multilingual",
+ "reason": "Latin script, language not identified and no non-English letters; using default (multilingual)",
+ "diagnostic": {"stage": "whole_state", "reason": "insufficient_words", "source": null}}
+```
+
+`state` and `questions` are required, with the same shape and size limits as prediction. An empty
+questions object is allowed. This is a routing snapshot, not a complete inference-request validator:
+a successful route does not certify that every question will pass inference-time validation.
+
+The optional routing controls are `model`, `task`, `lang` and `lang_guess`, with the same
+[model aliases, registry and override semantics](#post-v1systemone) as prediction. Omitted controls
+(and `null` routing overrides) leave deployment settings in charge. `explain` defaults to `false`;
+only JSON booleans are accepted. Strings, numbers and `null` return `422` naming `explain` before
+routing. Unknown fields are ignored, as on the prediction endpoints, but prediction-only controls
+`max_len`, `head_max_len`, `min_confidence`, `batch_size`, `sort_by_length`, `diagnostics` and `states`
+are refused with `422`, including when their value is `null`. The same applies to `hooks`,
+`on_route`, `on_predict_start`, `on_predict_end`, `hooks_raise` and `hooks_timeout`: hook policy
+belongs to the operator.
+
+The default response contains only `model`, `repo` and `reason`. With `explain: true`, it also
+contains `diagnostic`, read from the final decision **after** `on_route`; no detector is run a
+second time. It is `null` when that decision carries no diagnostic. Built-in diagnostics contain
+`stage`, `reason` and `source`, preserving `source: null` when no source is available. The full
+`detection` mapping, including `mixed_segment` containing verbatim input, is never exported.
+Built-in diagnostic metadata does not include caller values, but `source.path` may contain
+caller-chosen field names. Hook-provided diagnostics remain operator-controlled. The routing
+`reason` can include input text, so the response as a whole is **not text-free**.
+
+Authentication, admission, body-read timeout and request limits are shared with prediction.
+Routing runs behind the same gate on the same worker, so it can wait behind a forward pass;
+use `/health` for liveness. The endpoint does not call prediction or load a checkpoint. Startup
+preload is unchanged, and operator-installed hooks still run and may have side effects.
+Cancellation of an HTTP request or a hook timeout does not forcibly stop code already running.
+Routing does not reset the inference idle deadline on success, failure or cancellation, though
+queued/running work can delay unloading. The result describes routing **now**, not a guarantee
+about a later prediction if configuration, hooks or input change.
+
+An injected legacy router without the `explain` parameter still supports omitted/false requests;
+`explain: true` returns a clear `422`, without silently retrying. As with prediction, a `ValueError`
+returns `422` with its message, including one raised by an operator hook. Other unexpected errors
+are logged server-side and return `500` with `routing failed`.
+There is no batch routing endpoint or SDK routing method in this change.
+
### `POST /v1/systemone`
One request carries a `state` and any number of questions over it:
@@ -461,8 +523,8 @@ as long as it keeps the socket open.
## Not (yet) here
This server speaks one protocol on purpose. There is no OpenAI-compatible endpoint; run several
-questions in one request instead, since they share a single forward pass per question set. The one
-other route is `POST /v1/systemone/batch`, which answers one `questions` set over an array of
+questions in one request instead, since they share a single forward pass per question set.
+`POST /v1/systemone/batch` answers one `questions` set over an array of
`states`. It has no section on this page yet -- its request shape is in the README's self-hosting
section -- and every check above applies to it as it does to `POST /v1/systemone`: the same shape
`400`s, the same unpaired-surrogate refusal, the same auth, admission, size limits, body-control
diff --git a/docs/typescript-sdk.md b/docs/typescript-sdk.md
index 9a8adb49..b922366d 100644
--- a/docs/typescript-sdk.md
+++ b/docs/typescript-sdk.md
@@ -48,8 +48,10 @@ Optional extensions are validated when present.
`predict` posts `/v1/systemone` and `health` gets `/health` -- those two calls are the client's whole
network surface, and `/v1/systemone` is the only inference endpoint among them. There is no standalone
-routing method: `laya-client` exposes `predict` and `health` and nothing else, and the live integration
-test asserts the server answers `404` for `/v1/route`. The controls the endpoint does honour are
+routing method: `laya-client` exposes `predict` and `health` and nothing else. The server separately
+offers [`POST /v1/route`](http-api.md#post-v1route) for a routing snapshot without inference; call it
+with ordinary HTTP when needed. The live integration test covers this server contract without adding
+an SDK method. The controls the prediction endpoint does honour are
per-request, and each is sent only when the caller supplied the option -- an absent option leaves the
deployment's own `Router(...)` settings in charge instead of overriding them with a client-side default:
diff --git a/laya/serve.py b/laya/serve.py
index c96bb0a2..0e848293 100644
--- a/laya/serve.py
+++ b/laya/serve.py
@@ -6,7 +6,7 @@ output_tokens}`` usage block -- so a client written against Jev (for example the
`hs-jev` Haskell client) can point its ``baseUrl`` at this server and keep
working unchanged. All this module adds is the HTTP surface Laya itself does not
ship: a ``POST /v1/systemone`` route and its ``POST /v1/systemone/batch`` sibling,
-an optional bearer check, and a health probe.
+``POST /v1/route`` for routing without inference, an optional bearer check, and a health probe.
Each ``choice`` and ``score`` answer also carries ``x_jev_confidence``: the
``confidence`` Jev's own formula gives for the same probabilities. Laya's
@@ -199,6 +199,14 @@ BATCH_BODY_ITEM_CONTROLS = ("max_len", "head_max_len", "task", "lang", "lang_gue
# because that tuple is pinned against ``Router.predict``, which has no ``diagnostics``.
BATCH_BODY_OUT_PARAMETERS = ("diagnostics",)
+# Route-only requests cannot apply prediction budgets, abstention or batch controls.
+# Hook policy belongs to the operator, including the on_route callback itself.
+ROUTE_BODY_CONTROLS = ("model", "task", "lang", "lang_guess", "explain")
+ROUTE_BODY_REFUSALS = BODY_REFUSALS + (
+ "on_route", "max_len", "head_max_len", "min_confidence", "batch_size",
+ "sort_by_length", "diagnostics", "states",
+)
+
def _env_bool(name: str, default: bool) -> bool:
"""A `LAYA_*` switch. An unset or blank value means `default`; only the four spellings turn it on.
@@ -1408,6 +1416,76 @@ def create_app(router: Optional[Any] = None):
finally:
admission.release()
+ @app.post("/v1/route")
+ async def route(request: Request, authorization: Optional[str] = Header(default=None)):
+ nonlocal gate
+ _check_auth(authorization)
+ async with _admit():
+ body = await _read_json_body(request, body_read_timeout)
+ if not isinstance(body, dict) or "questions" not in body:
+ raise HTTPException(status_code=400, detail="request body must be an object with a 'questions' field")
+ state, questions = body.get("state"), body["questions"]
+ _check_request_limits(state, questions, MAX_TOURNAMENT_CHOICE_OPTIONS if tournament_group
+ else MAX_CHOICE_OPTIONS)
+ if _has_lone_surrogate(body):
+ raise HTTPException(status_code=400, detail=_LONE_SURROGATE_DETAIL)
+ refused = sorted(key for key in ROUTE_BODY_REFUSALS if key in body)
+ if refused:
+ raise HTTPException(status_code=422,
+ detail="%s cannot be sent to the routing endpoint" % ", ".join(refused))
+ explain = body.get("explain", False)
+ if not isinstance(explain, bool):
+ raise HTTPException(status_code=422, detail="explain must be a boolean")
+ model = _resolve_model(body.get("model"), getattr(router, "resolve", None))
+ route_kwargs = {}
+ if body.get("task") is not None:
+ route_kwargs["task"] = body["task"]
+ for key in ("lang", "lang_guess"):
+ value = _validate_language_param(body, key)
+ if value is not None:
+ route_kwargs[key] = value
+
+ def run_route():
+ if explain:
+ try:
+ decision = router.route(state, questions, model=model, explain=True, **route_kwargs)
+ except TypeError as error:
+ # A legacy injected router may lack the keyword. Do not retry (hooks
+ # may have side effects), or mislabel an error raised inside route.
+ if (error.__traceback__.tb_next is None
+ and "unexpected keyword argument 'explain'" in str(error)):
+ raise HTTPException(status_code=422,
+ detail="this router does not support explain=True") from None
+ raise
+ else:
+ decision = router.route(state, questions, model=model, **route_kwargs)
+ # Read the final on_route result, without rerunning language detection or
+ # exporting detection.mixed_segment (which can contain verbatim input).
+ result = {key: decision[key] for key in ("model", "repo", "reason")}
+ if explain:
+ detection = decision.get("detection")
+ result["diagnostic"] = detection.get("diagnostic") if isinstance(detection, dict) else None
+ return result
+
+ if gate is None:
+ gate = asyncio.Lock()
+ try:
+ async with gate:
+ loop = asyncio.get_running_loop()
+ # Share the inference worker, but not _run_inference: diagnostics must
+ # not refresh the idle deadline, even on failure or HTTP cancellation.
+ result = await loop.run_in_executor(pool, run_route)
+ from fastapi.responses import JSONResponse
+ return JSONResponse(content=result)
+ except HTTPException:
+ raise
+ except ValueError as error:
+ raise HTTPException(status_code=422, detail=str(error)) from None
+ except Exception: # noqa: BLE001 -- never expose internal paths or hook errors
+ _log.exception("routing failed for model=%s",
+ str(model).replace("\n", "\\n").replace("\r", "\\r"))
+ raise HTTPException(status_code=500, detail="routing failed")
+
@app.post("/v1/systemone")
async def systemone(request: Request, authorization: Optional[str] = Header(default=None)):
_check_auth(authorization)
diff --git a/sdk/typescript/test/integration.mjs b/sdk/typescript/test/integration.mjs
index 2e5a1534..513b9f30 100644
--- a/sdk/typescript/test/integration.mjs
+++ b/sdk/typescript/test/integration.mjs
@@ -127,5 +127,20 @@ test('JavaScript → HTTP → Python Router → real offline Agent inference', {
await assert.rejects(client.predict('x'.repeat(50_001), fixture.questions), error =>
error instanceof LayaAPIError && error.status === 413 && error.message.includes('state too large'));
assert.equal((await fetch(`${fixture.baseURL}/v1/predict`, { method: 'POST' })).status, 404);
- assert.equal((await fetch(`${fixture.baseURL}/v1/route`, { method: 'POST' })).status, 404);
+ assert.equal((await fetch(`${fixture.baseURL}/v1/route`, { method: 'POST' })).status, 401);
+ for (const explain of [false, true]) {
+ const response = await fetch(`${fixture.baseURL}/v1/route`, {
+ method: 'POST',
+ headers: { authorization: 'Bearer integration-test', 'content-type': 'application/json' },
+ body: JSON.stringify({ state: 'hello', questions: {}, explain }),
+ });
+ assert.equal(response.status, 200);
+ const route = await response.json();
+ assert.deepEqual(Object.keys(route).sort(),
+ (explain ? ['model', 'repo', 'reason', 'diagnostic'] : ['model', 'repo', 'reason']).sort());
+ assert.equal(route.model, 'multilingual');
+ if (explain) {
+ assert.deepEqual(route.diagnostic, { stage: 'whole_state', reason: 'insufficient_words', source: null });
+ }
+ }
});
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index a6e24ae6..0a4bf045 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -1152,6 +1152,15 @@ check("serve/BODY_REFUSALS names nothing but hooks",
[key for key in _http_refusals if "hook" not in key and "predict" not in key], [])
check("serve forwards and refuses disjoint sets", sorted(set(_http_controls) & set(_http_refusals)), [])
+from laya.serve import ROUTE_BODY_CONTROLS, ROUTE_BODY_REFUSALS # noqa: E402
+
+check("HTTP route controls track Router.route", sorted(ROUTE_BODY_CONTROLS),
+ sorted(set(sig(Router.route)) - {"self", "state", "questions"} - set(ROUTE_BODY_REFUSALS)))
+check("HTTP route accepts and refuses disjoint controls",
+ sorted(set(ROUTE_BODY_CONTROLS) & set(ROUTE_BODY_REFUSALS)), [])
+check("HTTP route refuses prediction-only controls and operator hook policy",
+ sorted(set(_http_controls + _http_refusals) - set(ROUTE_BODY_CONTROLS) - set(ROUTE_BODY_REFUSALS)), [])
+
# The opt-in shortlist evaluator is a public Python entry point. Pin its required
# provenance arguments without adding an eager import to the package root.
from laya.evals_shortlist import evaluate_shortlist # noqa: E402
diff --git a/tests/test_serve.py b/tests/test_serve.py
index 28df5497..2f7cf7a5 100644
--- a/tests/test_serve.py
+++ b/tests/test_serve.py
@@ -4390,3 +4390,225 @@ def test_a_body_delivered_in_time_still_answers(monkeypatch):
status, payload = asyncio.run(_asgi_post(app, "/v1/systemone", _ASGI_BODY))
assert status == 200
assert json.loads(payload)["usage"]["input_tokens"] == 42
+
+
+# --------------------------------------------------------------------------- HTTP routing diagnostics
+
+_ROUTE_REQ = {"state": {"message": "Turn lights off"}, "questions": REQ["questions"]}
+_ROUTE_DECISION = {"model": "english", "repo": "example/english", "reason": "test routing"}
+_ROUTE_DIAGNOSTIC = {"stage": "whole_state", "reason": "insufficient_words", "source": None}
+
+
+class DiagnosticRouter(FakeRouter):
+ """Records routing separately from inference; its omitted kwargs stay observable."""
+
+ def __init__(self):
+ super().__init__()
+ self.route_calls = []
+
+ def route(self, state, questions, model=None, **kwargs):
+ self.route_calls.append({"state": state, "questions": questions, "model": model, **kwargs})
+ return dict(_ROUTE_DECISION, detection={"diagnostic": dict(_ROUTE_DIAGNOSTIC),
+ "mixed_segment": "private caller text"})
+
+ def load(self, *args, **kwargs):
+ raise AssertionError("routing must not load a checkpoint")
+
+
[email protected]("flag", [{}, {"explain": False}, {"explain": True}])
+def test_http_route_opt_in_projection_and_no_inference(monkeypatch, flag):
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = DiagnosticRouter()
+ with TestClient(create_app(fake)) as client:
+ response = client.post("/v1/route", json={**_ROUTE_REQ, **flag})
+ assert response.status_code == 200, response.text
+ expected = dict(_ROUTE_DECISION)
+ if flag.get("explain"):
+ expected["diagnostic"] = _ROUTE_DIAGNOSTIC
+ assert response.json() == expected
+ assert fake.route_calls == [{**_ROUTE_REQ, "model": None,
+ **({"explain": True} if flag.get("explain") else {})}]
+ assert fake.calls == []
+ assert "private caller text" not in response.text
+ assert "X-Inference-Time-Ms" not in response.headers
+
+
[email protected]("bad", [None, 0, 1, 0.0, 1.0, "false", "true", "", [], {}])
+def test_http_route_refuses_non_boolean_explain_before_dispatch(monkeypatch, bad):
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = DiagnosticRouter()
+ with TestClient(create_app(fake)) as client:
+ response = client.post("/v1/route", json={**_ROUTE_REQ, "explain": bad})
+ assert response.status_code == 422, response.text
+ assert "explain" in response.json()["detail"]
+ assert fake.route_calls == fake.calls == []
+
+
[email protected]("diagnostic", [None, {"stage": "operator", "reason": "hook replacement", "source": None}])
+def test_http_route_observes_the_final_hook_decision_once(monkeypatch, diagnostic):
+ from laya import BaseHook, Router
+
+ class ReplaceRoute(BaseHook):
+ def on_route(self, ctx):
+ ctx.decision = dict(_ROUTE_DECISION)
+ if diagnostic is not None:
+ ctx.decision["detection"] = {"diagnostic": diagnostic, "mixed_segment": "private value"}
+
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ router = Router(hooks=[ReplaceRoute()])
+ calls = []
+ original_route = router._route
+
+ def counted_route(*args, **kwargs):
+ calls.append(kwargs)
+ return original_route(*args, **kwargs)
+
+ def forbidden(*args, **kwargs):
+ raise AssertionError("routing must not load or predict")
+
+ monkeypatch.setattr(router, "_route", counted_route)
+ monkeypatch.setattr(router, "predict", forbidden)
+ monkeypatch.setattr(router, "load", forbidden)
+ with TestClient(create_app(router)) as client:
+ response = client.post("/v1/route", json={**_ROUTE_REQ, "explain": True})
+ assert response.status_code == 200, response.text
+ assert response.json() == dict(_ROUTE_DECISION, diagnostic=diagnostic)
+ assert len(calls) == 1
+ assert router.loaded == []
+
+
[email protected]("control", [{"model": "english"}, {"task": "typed_decisions"},
+ {"lang": "de"}, {"lang_guess": "de"}])
+def test_http_route_explicit_controls_match_core_and_have_null_diagnostic(monkeypatch, control):
+ from laya import Router
+
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ router = Router()
+ expected = router.route(**_ROUTE_REQ, **control, explain=True)
+ with TestClient(create_app(router)) as client:
+ response = client.post("/v1/route", json={**_ROUTE_REQ, **control, "explain": True})
+ assert response.status_code == 200, response.text
+ assert response.json() == {**{key: expected[key] for key in _ROUTE_DECISION}, "diagnostic": None}
+ assert router.loaded == []
+
+
+def test_http_route_uses_server_defaults_and_registered_models(monkeypatch):
+ from laya import Router
+
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ router = Router(default="multilingual", models={"private": "/tmp/operator-checkpoint"})
+ with TestClient(create_app(router)) as client:
+ inherited = client.post("/v1/route", json={**_ROUTE_REQ, "explain": True})
+ pinned = client.post("/v1/route", json={**_ROUTE_REQ, "model": "private", "explain": True})
+ jev = client.post("/v1/route", json={**_ROUTE_REQ, "model": "jev-1", "explain": True})
+ assert inherited.status_code == pinned.status_code == jev.status_code == 200
+ assert inherited.json()["model"] == "multilingual"
+ assert inherited.json()["diagnostic"] == _ROUTE_DIAGNOSTIC
+ assert jev.json() == inherited.json()
+ assert pinned.json()["model"] == "private" and pinned.json()["diagnostic"] is None
+ assert router.loaded == []
+
+
[email protected]("control", [{}, {"task": None, "lang": None, "lang_guess": None}])
+def test_http_route_omitted_or_null_controls_inherit_router_hints(monkeypatch, control):
+ from laya import Router
+
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ router = Router(default="english", lang_guess="de")
+ with TestClient(create_app(router)) as client:
+ response = client.post("/v1/route", json={**_ROUTE_REQ, **control, "explain": True})
+ assert response.status_code == 200, response.text
+ assert response.json()["model"] == "multilingual"
+ assert response.json()["diagnostic"] is None
+
+
[email protected]("name", ["max_len", "head_max_len", "min_confidence", "batch_size",
+ "sort_by_length", "diagnostics", "states", "hooks", "on_route",
+ "on_predict_start", "on_predict_end", "hooks_raise", "hooks_timeout"])
[email protected]("value", [None, 1])
+def test_http_route_refuses_prediction_and_hook_controls(monkeypatch, name, value):
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = DiagnosticRouter()
+ with TestClient(create_app(fake)) as client:
+ response = client.post("/v1/route", json={**_ROUTE_REQ, name: value})
+ assert response.status_code == 422, response.text
+ assert name in response.json()["detail"]
+ assert fake.route_calls == []
+
+
+def test_http_route_auth_precedes_body_validation(monkeypatch):
+ monkeypatch.setenv("LAYA_API_KEY", "key")
+ fake = DiagnosticRouter()
+ with TestClient(create_app(fake)) as client:
+ assert client.post("/v1/route", content=b"not JSON").status_code == 401
+ assert client.post("/v1/route", json=_ROUTE_REQ,
+ headers={"Authorization": "Bearer wrong"}).status_code == 401
+ response = client.post("/v1/route", json=_ROUTE_REQ, headers={"Authorization": "Bearer key"})
+ assert response.status_code == 200, response.text
+ assert len(fake.route_calls) == 1
+
+
[email protected]("body, status", [
+ ([], 400), ({"state": "hello"}, 400), ({"questions": {}}, 400),
+ ({"state": None, "questions": {}}, 400), ({"state": "hello", "questions": []}, 400),
+ ({"state": "x" * (MAX_STATE_CHARS + 1), "questions": {}}, 413),
+ ({"state": "\ud800", "questions": {}}, 400),
+])
+def test_http_route_body_contract_and_limits(monkeypatch, body, status):
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = DiagnosticRouter()
+ with TestClient(create_app(fake)) as client:
+ response = client.post("/v1/route", content=json.dumps(body),
+ headers={"Content-Type": "application/json"})
+ assert response.status_code == status, response.text
+ assert fake.route_calls == []
+
+
[email protected]("body", [b"not JSON", b'{"state":'])
+def test_http_route_malformed_json_does_not_dispatch(monkeypatch, body):
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = DiagnosticRouter()
+ with TestClient(create_app(fake)) as client:
+ response = client.post("/v1/route", content=body, headers={"Content-Type": "application/json"})
+ assert response.status_code == 400, response.text
+ assert fake.route_calls == []
+
+
+def test_http_route_legacy_router_only_refuses_requested_explanation(monkeypatch):
+ class LegacyRouter(FakeRouter):
+ def route(self, state, questions, model=None):
+ self.calls.append((state, questions, model))
+ return dict(_ROUTE_DECISION)
+
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = LegacyRouter()
+ with TestClient(create_app(fake)) as client:
+ for flag in ({}, {"explain": False}):
+ response = client.post("/v1/route", json={**_ROUTE_REQ, **flag})
+ assert response.status_code == 200, response.text
+ assert response.json() == _ROUTE_DECISION
+ fake.calls.clear()
+ response = client.post("/v1/route", json={**_ROUTE_REQ, "explain": True})
+ assert response.status_code == 422, response.text
+ assert "explain" in response.json()["detail"]
+ assert fake.calls == [], "the failed explained route must not retry without explain"
+
+
[email protected]("error", [RuntimeError("secret /opt/checkpoint location"),
+ TypeError("wrapped.route() got an unexpected keyword argument 'explain'")])
+def test_http_route_internal_failure_is_sanitized_not_retried(monkeypatch, caplog, error):
+ class BrokenRouter(DiagnosticRouter):
+ def route(self, *args, **kwargs):
+ self.route_calls.append(kwargs)
+ raise error
+
+ monkeypatch.delenv("LAYA_API_KEY", raising=False)
+ fake = BrokenRouter()
+ with TestClient(create_app(fake), raise_server_exceptions=False) as client:
+ with caplog.at_level(logging.ERROR, logger="laya.serve"):
+ response = client.post("/v1/route", json={**_ROUTE_REQ, "explain": True})
+ assert response.status_code == 500, response.text
+ assert response.json() == {"detail": "routing failed"}
+ assert str(error) not in response.text
+ assert len(fake.route_calls) == 1
+ assert any(record.exc_info for record in caplog.records)