Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(router)!: default to the multilingual checkpoint for undecided text

Nandakishor3cf26cbcb187
diff --git a/README.md b/README.md
index 0b157ce9..f748de58 100644
--- a/README.md
+++ b/README.md
@@ -94,137 +94,20 @@ The shipped checkpoints work zero-shot, but fine-tuning on decisions from your o
**[nandhakishorm.github.io/laya](https://nandhakishorm.github.io/laya/)**: guides for [prediction hooks](https://nandhakishorm.github.io/laya/hooks/), [schema-driven decisions](https://nandhakishorm.github.io/laya/structured/), [Docker](https://nandhakishorm.github.io/laya/docker/) and [LangChain and LangGraph](https://nandhakishorm.github.io/laya/langchain/), plus a full [API reference](https://nandhakishorm.github.io/laya/reference/).
-## What's new in 0.3.29
-
-* **`predict_long` no longer splits a state that fits.** The one-pass shortcut compared the state against the default window instead of the room the questions actually leave, so any state in the band between the two was scanned in two windows and answered with the max or the most confident window rather than the plain answer (#966). On `laya-multilingual` that band is roughly 761 to 989 state tokens, an ordinary email: an 878-token note returned noul 0.998 against `predict`'s 0.752, at twice the forward passes. It now matches `predict` exactly, and an explicit `window=` still scans.
-* **A line of English acronyms no longer routes to the multilingual checkpoint.** `MON`, `DES`, `EST` and `LA` collide with French stopwords, so one line of ordinary abbreviations inside an English state made `analyse` name the whole state French (#1013). The all-caps guard is kept, because a customer shouting in Portuguese is still Portuguese, and the fix separates that case from a line of bare acronyms. A second cause is fixed with it: acronyms were voting in the whole-state verdict too, which no helper was filtering.
-* **A `min_confidence` map key that cannot name a bucket is refused.** The validator checked only that a key was a string, so `{"choice:2-5": 0.9}` passed and then silently gated nothing: the lookup misses, falls back to `default`, then to 0.0 (#1002). A caller who mis-spells one key believed that bucket was gated at 0.9 while every answer in it abstained never.
-* **Fine-tuning reports what it learned, on held-out data.** `laya-train --eval eval.csv` evaluates the base checkpoint before training and the fine-tune after, and writes `train_report.json` with accuracy, loss, ECE, Brier and signed deltas (#967, #887). Evaluation items are checked for overlap against both the training and calibration splits, and a set that overlaps is labelled `overlapping_eval` rather than reported as generalization. With no `--eval` and no calibration items, evaluation is skipped rather than reporting training fit.
-* **A fine-tune that collapses to the class prior now says so.** On a few hundred rows the default epoch budget can finish at chance with a near-constant logit, and the run still looks successful (#968, #963).
-* **`laya-evals evidence --checkpoint DIR`** reads the calibration evidence a fine-tune persists and reports it as `PRESENT`, `MISSING`, `INSUFFICIENT` or `UNKNOWN`, so zero calibration items stops reading as healthy (#964). It loads no weights and does not need torch.
-* **The fine-tuning scripts now share one loop.** `notebooks/laya_finetune_typed_decisions_mps.py` and `research/scripts/finetune_single_device.py` call `laya.train` instead of carrying their own copies, which is what #851 and #885 were: the same temperature-clamp bug fixed twice in parallel (#965, #887).
-* **The default loss stays `rlcd`, now on evidence.** A three-seed panel on typed-decisions says the single-seed soft-CE win does not replicate: paired `soft-ce − rlcd` accuracy is +0.0002 on average and changes sign between seeds, with soft-CE far noisier (std 0.0155 against 0.0016) (#1012, #887, #741).
-* **Java.** `laya-java` gains language routing, the preset question sets, the embedding shortlist and the email cleaner, gated against fixtures recorded from the Python package, plus a JDK matrix lane because `\p{L}` follows the JDK's own Unicode version (#938). Three defects on `main` are fixed with it, including a release workflow that could publish a jar named after a branch, and a release test floor that counted skipped tests.
-* **Thirty-one documentation corrections**, each landing a gate that fails if the page drifts again (#972 to #1011, from @aashish254). The largest class: `confidence` was documented as one entropy formula across every question type, when it is `1 - H/log(k)` for `choice` and `score` and equals `answer_confidence` for `noul`. Others name parameters a tool actually takes, correct key counts, and stop calling the shipped `answer_confidence` calibrated when the loader itself warns that it is not.
-* **Portability.** Six repo reads in `tests/` left the codec to the runner, which is cp1252 on the Windows lane, so a UTF-8 page killed the whole suite at the read rather than failing one assertion. A gate now refuses an unpinned repo read (#1001).
-* **`laya[onnx]` no longer caps NumPy for everyone.** The cap that torch below 2.3 needs is scoped by marker to macOS on Intel, instead of making the extra unresolvable next to anything requiring `numpy>=2` (#947).
-
-44 pull requests from 8 contributors.
+## What's new in 0.4.0
----
-
-## What's new in 0.3.28
-
-* **`laya.backends` was missing from every published wheel.** The package was never added to setuptools' explicit list, so the wheels for 0.3.25, 0.3.26 and 0.3.27 shipped without it (#940). The documented backend selection API was therefore unavailable to anyone who installed with pip: `Agent(backend="eager")` raised `ModuleNotFoundError` and `agent.set_backend("eager")` raised `ImportError`, while the compile guide described both. The default path was unaffected, which is why this went unnoticed; it also only ever worked from a source checkout, which every contributor here uses. A packaging check now keeps the declared list aligned with every importable `laya` source package, so the next one fails at build time instead of at install time.
-* **Fine-tuning from the command line.** `laya-train --data tickets.csv --out ./ft` fine-tunes a checkpoint from a plain labelled CSV, discovering the labels and synthesising the question (#931, #887). It also reads the `{state, questions, expected}` rows `laya-evals` already takes, so an evaluation set trains without conversion, and `--dry-run` prints the item and skip counts before anything loads. The generated question schema is saved beside the checkpoint, so the labels and wording can be reused exactly at inference.
-* **A fine-tune now says how much its calibration can be trusted.** A short run could fit temperatures of `[1.0, 1.0, 1.0]` from a handful of items and save a checkpoint whose confidences were never calibrated, indistinguishable from one that was. `finetune` warns per question type and records the same report in the checkpoint config (#933). Questions whose options run past `max_len` are skipped and counted rather than crashing the batch mid-run (#934), and a final partial accumulation window is normalised by the micro-batches it actually holds (#941).
-* **Many-option choices.** `laya.predict_tournament(agent, state, questions)` answers a choice question whose labels do not fit the option budget by splitting them into groups of 16, answering every group in one forward pass and running the winners in a final call (#950). On full test splits: BANKING77 0.430 to 0.610, CLINC150 0.625 to 0.876, with ECE falling from 0.350 to 0.082. No embedder and no new dependency.
-* **Order-invariant checkpoints.** An opt-in parallel option layout gives every option the same position ids and blocks attention between options, so reordering can only permute the logits (#951). A checkpoint trained with `TrainConfig(option_layout="parallel")` gives the same answer under reordering for 100% of decisions, against 91.7% for the current layout. Every published checkpoint stays on the default sequential layout and nothing changes for them.
-* **Serve your own checkpoints beside the built-ins.** `Router(models=...)` and `Router.register(name, source)` accept any name, with a Hub repo id, a `(repo, subfolder)` pair or a local directory as the source, and the router loads, evicts and unloads it exactly like a built-in (#937, #919). `Router.registered` lists them and `Router.unregister` removes one.
-* **pydantic `Enum` fields work.** `questions_from_pydantic`, `decide(schema=Model)` and `answer_to_pydantic` raised `SchemaError` on every enum field, because pydantic v2 renders them as a `$ref` and the planner never resolved one. Local `$defs` references now resolve, including inside `Optional[Enum]` and pydantic v1's one-item `allOf` wrapper (#960).
-* **Abstention cuts are fitted on the precision the gate reads.** `fit_abstention_thresholds` chose a cut from full-precision confidences while the gate compares the 4-decimal `answer_confidence`. With a binning map installed the whole cut bin was lost: the fitter judged 532 accepted answers and the gate delivered 382 (#957).
-* **`laya-evals` refuses a NaN threshold.** `--min-accuracy nan` and `--max-ece nan` were accepted and every comparison against NaN is false, so a run at 0% accuracy and ECE 0.99 exited 0 (#954).
-* **MCP keeps `option_order`.** `laya_predict`, `laya_route`, `laya_shortlist` and every `laya_predict_batch` item dropped the key, so the README's rotation-averaging recipe returned k identical answers (#958).
-* **The ONNX agent catches an in-place question rewrite.** A start hook that widens a question in place mutated the before-image `predict_long` compared against, so the scan-budget guard returned early and up to 59% of a document reached no model while `usage` reported every window as read. The torch agent already raised on the identical input (#959).
-* **torch 2.0 to 2.3 can run the package again.** The action-head dtype check called a form of `torch.is_autocast_enabled` that only exists from torch 2.4, so every forward pass raised `TypeError` across the bottom of the declared `torch>=2.0.0` range (#945). Two test suites that read `torch.fx.experimental._config` at module level no longer fail collection on a torch without it (#946), and the Windows lane's bf16 legs are skipped on a Windows CPU, where they hit an uncatchable SIGILL (#949).
-* **macOS and Intel.** `setup_laya.sh` and `LOCAL_SETUP.md` pin the stack that actually runs there, since a plain `git clone` resolves torch 2.2.2 against a transformers that wants 2.4 and nothing in the error message says so (#948).
-* **Docker.** The default base image moves to Debian 13 (trixie), with bookworm one build arg away (#953, #739). Verified with a decision diff across both bases: 291 answers per checkpoint on torch CPU, torch CUDA and ONNX Runtime CPU, zero differing decisions.
-* **TypeScript.** `Router.predictBatch` and `predictMany` forward per-call `onPredictStart`, `onPredictEnd` and `hooksRaise`, which were accepted and silently dropped (#936, #935). French mail is cleaned the way Python cleans it, over 4,000 generated messages with zero mismatches (#939). An English word that doubles as a foreign function word is counted once, so plain English like `Come one, come all` is no longer routed to the multilingual checkpoint (#955).
-* **.NET and Java.** The French device footer is matched with the spacing people actually write (#956), and an empty action-logit row gives `[0.5, 0.5]` instead of `[NaN, NaN]` (#942).
-* **Security.** The Gradle wrapper jar is validated against Gradle's published checksums before anything executes it, and the distribution it downloads is pinned by sha256. The email examples no longer use a real bank's name with two unregistered lookalike domains as the phishing lure; the preset still flags the generic version at 0.926. A full audit of the repository, the published wheel, both npm packages and all ten workflows found no malicious content.
-
-28 pull requests from 12 contributors.
-
----
-
-## What's new in 0.3.27
+* **The routing default is now `multilingual`.** Text whose language the detector cannot place
+ used to fall back to the English checkpoint. On the 51-language sweep the multilingual
+ checkpoint leads on 50 of the 51, the exception being English itself (0.820 against 0.710),
+ and by 0.180 macro accuracy excluding English. Identified English still routes to the English
+ checkpoint, so only undecided text moves. Set `Router(default="english")` or
+ `LAYA_DEFAULT_MODEL=english` to restore the old behaviour; the break-even is around 62%
+ English traffic.
-* **A plain `import laya` no longer crashes when TensorFlow is installed.** transformers 4.x imports TensorFlow while laya builds the model, and a broken TF build turns that into `Fatal Python error: Bus error` from a library laya never uses (#915). The package now sets `USE_TF=0` the way CI, the Dockerfile and the examples already did.
-* **Per-call hooks reach the batch path.** `Router.predict_batch` takes `hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise` and `hooks_timeout`, matching `predict`; `Router.route_batch` takes `hooks`, `hooks_raise` and `hooks_timeout`, since it fires only `on_route` and has no predict events for the convenience callables to bind to (#909). The batched dispatch also composes installed hooks properly, so a `set_default_hooks` default no longer fires on `predict` and silently vanishes on `predict_batch`.
-* **A stale abstention flag is cleared on re-evaluation.** Re-running the gate on a reused result dict with a more permissive threshold used to leave `low_confidence: True` behind, so downstream callers treated the answer as permanently abstained (#910).
-* **`predict_long` runs its scan after the hook chain** rather than as a start hook, so laya's own forward pass no longer executes inside `dispatch()` and inherits the caller's hook machinery.
-* **Abstention thresholds are fitted on the scale the gate reads.** With a histogram-binning map installed, the runtime reports a binned confidence; `fit_abstention_thresholds` now fits against that instead of the temperature-scaled value, so a fitted cut and the gate agree.
-* **Selective-classification metrics no longer split tied confidences.** `selective_accuracy` and `aurc` cut at a row index, so with ties the result depended on dataset order. Ties are the normal case for `answer_confidence`, not a corner.
-* **Java.** `laya-java/` is a JVM inference package with the same live parity gate the .NET port uses: fixtures generated from the Python package by a committed script, a drift check, and a skip guard so a missing artifact fails rather than passes. The lane is advisory for the same reason the .NET one is.
-* **French email.** Device footers with ordinary spacing are matched, not only the tight spacing the original samples happened to have.
-* **MPS autocast, measured on an M1 Pro.** `mps_amp_min_rows` has defaulted to 5 since #109 on the strength of one M5 measurement, hedged for M1 to M5. `BENCHMARKS.md` now carries paired fp32-against-fp16 numbers on an M1 Pro, with the script that produces them.
-
-9 pull requests from 7 contributors.
+Earlier releases are in the [GitHub releases](https://github.com/NandhaKishorM/laya/releases).
---
-## What's new in 0.3.26
-
-A small release: one new SDK, one TypeScript port, and the dependency bumps.
-
-* **.NET SDK.** `laya-dotnet/` is a C# port of the inference path running the official checkpoints exported to ONNX on ONNX Runtime, so calling Laya from .NET no longer needs a Python sidecar (#673). It ships with a parity lane that re-records its golden fixtures from the Python package at every commit, so drift surfaces on the pull request that causes it; the lane is advisory, because a Python improvement should not wait on a C# port. Long prediction, per-language temperatures, hooks, batching, option ordering, digest controls and usage diagnostics are not ported yet, and there is no NuGet package.
-* **Histogram binning in laya-ts.** `applyBinningMap` / `checkBinningMap` port the non-parametric recalibration from core, including the language-override bypass, and the recalibrated `answer_confidence` is what the abstention gate reads.
-* **Dependencies.** `@types/node` 24 to 26, zensical 0.0.65 to 0.0.67, ruff 0.16.8 to 0.16.9, and the CodeQL actions to 4.38.2. The three that could have broken a lane were each verified against that lane before merging.
-
-7 pull requests: 2 from contributors and 5 dependency bumps.
-
----
-
-## What's new in 0.3.25
-
-* **Memory on demand.** `LAYA_IDLE_UNLOAD_SECONDS` lets `laya-serve` give its checkpoints back after an idle window, and `LAYA_BASE_URL` lets `laya-mcp-server` answer from a running server over HTTP instead of loading its own copy, so several editor sessions share one resident model (#888). Both default to off.
-* **Backends.** `Agent(backend=...)` and `agent.set_backend(...)` select `auto` / `eager` / `compile` / `tilelang` / `onnx` through one class layer, with ONNX routed to the existing `ONNXAgent` rather than a second copy of that path.
-* **Compile defaults.** `compile=True` warms during load instead of paying 50 s on the first request, with `compile_warmup=False` to restore the lazy shape. `compile_cache` gives Inductor a Laya-specific directory and `compile_mode` reaches `reduce-overhead`.
-* **AOTInductor packaging works.** `DecisionModel.forward` cast to fp32 before the action head while the encoder ran under AMP, which packaged a graph that died with `mat1 and mat2 must have the same dtype`. The cast now matches the head's weight dtype outside autocast only, so eager and AMP numerics are untouched.
-* **Oversized batches split instead of collating whole.** `/v1/systemone/batch` chooses a state chunk size when the token product would exceed the cap, rather than building one 4,096-row forward pass. No request that worked before is refused and none changes its answer.
-* **Calibration.** Histogram binning is wired into answers and the calibration payload, so a fitted `binning_map` ships and loads like the temperatures.
-* **TileLang.** All five kernels now lower for the CPU target as an explicit fp32 specialization, and GEGLU uses two pipeline stages.
-* **Stricter device handling.** A CUDA ordinal past `device_count`, and `auto` in any casing, are resolved at load with the same warn-and-CPU shape as the existing fallbacks instead of dying later in `.to()`.
-* **Encoding.** `laya` writes redirected stdout as utf-8, the output half of the stdin fix in #799.
-* **TypeScript.** Both clients accept the per-bucket `minConfidence` map, and `laya-ts` ports it to the on-device engine.
-
-13 pull requests from 9 contributors.
-
----
-
-## What's new in 0.3.24
-
-* **Concurrency.** `Router.load` builds a checkpoint outside the lifecycle lock, so a cold build no longer blocks calls for a checkpoint that is already resident, and `unload` is synchronized per checkpoint rather than globally (#848). `predict_long` holds the tokenizer lock like every other encode, so scanning a document next to ordinary predictions no longer raises `Already borrowed` (#825).
-* **Calibration.** Per-option-count abstention thresholds, because one `min_confidence` does not transfer across option counts (#394): `fit_abstention_thresholds` fits one cut per bucket, keyed like `temperature_by_options`, and fails closed on a bucket too unreliable to accept anything. Histogram binning (`fit_binning_map`, `apply_binning_map`) recalibrates a bucket whose reliability curve temperature scaling cannot reach. `save_calibration` writes atomically, and `records_from_labeled` works on a loaded agent (#826).
-* **Evals.** Selective-classification metrics (Brier, AURC, selective accuracy) from the same pairs ECE already uses, so the abstention gate has a yardstick. A NaN metric, limit or tolerance can no longer pass a gate (#830).
-* **Stricter inputs.** A non-finite `hooks_timeout` (#828), a non-scalar `choice` label, a batch item's untyped `lang_guess`, and an unvalidated `min_confidence` or `hooks_timeout` on `laya_predict_batch` are all refused where they are read. `hooks_installed` accepts a sequence, as documented (#829). A `null` score level is a 422 instead of a response no Jev client can parse.
-* **laya-serve.** `LAYA_JEV_STRICT` projects the response onto the strict Jev wire contract for clients that reject unknown fields. `/v1/systemone/batch` sums `output_tokens` instead of reporting zero, and refuses an unpaired surrogate like the single endpoint.
-* **Fallback notes are warnings, not prints.** Eight device- and inference-fallback notes went to stdout, which corrupted `laya --json` output; they are `warnings.warn` now, so they are filterable.
-* **One test list.** CI and the release gate run the same shared suite list (`scripts/test_suites.py`). They had drifted to 80 suites against 47, so a regression in any of the 33 CI-only suites could reach PyPI.
-* **TypeScript.** The SDK types and validates the whole `usage` report, the four confidence fields, and the routing detection's `mixed_segment`. `laya-ts` reports collapsed options like `laya.common` and keeps bytecode out of the npm tarball.
-* **Docs and examples.** Nine example and page corrections where the prose contradicted the code, each with a test that holds the page to the source. The PyPI README links absolutely, so its 31 relative links no longer 404 on the project page.
-
-56 pull requests from 16 contributors.
-
----
-
-## What's new in 0.3.23
-
-* **Security.** `GET /health` no longer answers deployment internals to an unauthenticated caller on a server that set `LAYA_API_KEY` (#812): liveness stays open so every shipped probe keeps working, while the resident checkpoint names, revision SHAs, device state and fallback reasons need the bearer. A deployment with no key set is unchanged. `SECURITY.md` now documents private vulnerability reporting.
-* **Concurrency.** A GPU OOM fallback no longer moves the shared model under another in-flight request (#649). Normal forwards stay concurrent through a reader-writer gate; only the device demotion is exclusive.
-* **Per-call controls reach every surface.** `lang`, `lang_guess`, `min_confidence`, `task` and the token budgets now forward through the CLI (`--lang-guess`, `--min-confidence`), `/v1/systemone/batch`, the MCP single-request and batch tools, the TypeScript SDK and the LangChain, CrewAI and LlamaIndex wrappers. The three framework wrappers also gate `confidence_threshold` on `answer_confidence` rather than the entropy confidence, matching core's own gate.
-* **Routing.** Swedish is detected with MASSIVE evidence across all 51 locales and no locale regressing. `LAYA_DEFAULT_MODEL` makes the routing fallback settable from the environment, and a per-checkpoint pin no longer silently disables the caller's digest or revision.
-* **ONNX.** `--quantize` defaults to per-tensor, because per-channel collapsed the decision model to 32 percent agreement with eager. Export declares its dynamic dims in a spelling every torch in the supported range accepts, and verifies the symbolic axes it wrote.
-* **Long documents.** `predict_long` sizes its windows from the room a question actually leaves for the state, so a scan no longer skips part of the document and reports that it read it.
-* **Evals.** Opt-in per-slice quality gates catch a slice regressing while the overall metric improves, shortlist runs attribute errors to retrieval or decision, `--min-confidence` reaches the abstention gate, and `--calibration` works on the ONNX path.
-* **Abstention reporting.** With `min_confidence` set, every answer reports `abstention` and `abstention_threshold`. With no threshold the payload is byte for byte what it was.
-* **Research.** A Spanish phone-turn benchmark (217 frozen sentences, label policy fixed before any model ran) and a Chinese reliability evaluation with source-group isolation and paired bootstrap intervals.
-* **Email.** French mail clients are cleaned the way English, Portuguese and Spanish already were.
-
-64 pull requests from 21 contributors. #742 landed inside #684, which had been rebased onto it. Full list in the [0.3.23 release](https://github.com/NandhaKishorM/laya/releases/tag/v0.3.23).
-
----
-
-## What's new in 0.3.21
-
-* **ONNX catches up with PyTorch.** `ONNXAgent` gains `predict_batch` (with `sort_by_length`), `predict_long` and `decide_batch`, `scripts/export_onnx.py --quantize` writes a per-channel INT8 copy for CPU, and `laya-evals run --onnx` scores an export with the same gates as the torch path.
-* **Opt-in abstention.** `min_confidence=` on `predict`, `predict_batch`, `decide` and `decide_batch` flags answers below a threshold on `answer_confidence` with `low_confidence: True`, and `decide` returns `None` for them. When a threshold is set, every answer also reports `abstention` — `passed`, `abstained` or `unevaluated` — so a caller can tell a gate that cleared from a gate that never ran; with no threshold, nothing is added at all.
-* **Batch everywhere.** `decide_batch`, `Router.predict_long`, `laya --batch FILE`, the MCP `laya_predict_batch` / `laya_route_batch` / `laya_decide` tools, and LangChain `batch()` / `abatch()` all run on shared forward passes. New `LayaDecision` (LangChain), LlamaIndex selectors (`laya[llamaindex]`) and CrewAI routing (`laya[crewai]`).
-* **Per-request token budget.** `max_len` / `head_max_len` now reach every surface: `laya-serve` (capped by `LAYA_MAX_TOKEN_BUDGET`), `Router.predict_batch` requests, the CLI (`--questions`, `--max-len`, `--head-max-len`), MCP tools and LangChain nodes.
-* **Operations.** `LAYA_MAX_LOADED`, `LAYA_REVISION` and per-checkpoint SHA-256 maps; `/health` reports the device a checkpoint really runs on and its CPU-fallback count; the 503 busy answer carries `Retry-After`; `compile=True` no longer recompiles for every request shape.
-* **Stricter inputs.** A null or duplicate `choice` label, a short temperature list, a `None` state and non-dict questions are refused with a message that names them, and `usage["options"]` says when the head budget left two options with the same tokens.
-
----
<p align="center">
<img src="https://raw.githubusercontent.com/NandhaKishorM/laya/main/assets/laya_vs_jev_full.png" alt="Laya versus TypeSafe Jev: accuracy on shared public datasets, every application workflow, all 51 languages, speed, calibration, and the cost of not preloading" width="100%" />
@@ -670,14 +553,20 @@ router.route({"body": "Der Kunde wurde zweimal belastet"}, questions).reason
# "Latin script but language looks like 'de', not English"
```
-Very short Latin-script text often carries nothing that identifies its language (`"Quero cancelar"`, `"Esqueci minha senha"`). Such text goes to `default`, which is `"english"` unless you change it. If most of your traffic is not English, set:
+Very short Latin-script text often carries nothing that identifies its language (`"Quero cancelar"`, `"Esqueci minha senha"`). Such text goes to `default`, which is `"multilingual"` since 0.4.0. Text the detector *can* place is unaffected: identified English still goes to the English checkpoint.
```python
-router = Router(default="multilingual")
+router = Router() # default="multilingual"
router.route({"body": "Esqueci minha senha"}).model # -> multilingual
router.route({"body": "Please refund the duplicate charge"}).model # -> english
```
+If most of your traffic is English, set it back. The English checkpoint scores 0.820 on English against the multilingual checkpoint's 0.710, and the break-even is around 62% English traffic:
+
+```python
+router = Router(default="english")
+```
+
Running one of the shipped servers rather than your own `Router`, the same setting is
`LAYA_DEFAULT_MODEL=multilingual` — in the environment of `laya-serve`, the MCP server or the
`laya-serve` container ([docs/docker.md](https://github.com/NandhaKishorM/laya/blob/main/docs/docker.md)). It is the routing fallback only: text
@@ -703,6 +592,8 @@ Each item can independently set `model`, `task`, `lang`, `lang_guess`, or the to
Requests are validated before model loading. Different requests may use different question schemas; requests sharing a checkpoint, question schema and token budget are passed together to `Agent.predict_batch()`.
+* **Per-call hooks reach the batch path.** `Router.predict_batch` takes `hooks`, `on_predict_start`, `on_predict_end`, `hooks_raise` and `hooks_timeout`, matching `predict`; `Router.route_batch` takes `hooks`, `hooks_raise` and `hooks_timeout`, since it fires only `on_route` and has no predict events for the convenience callables to bind to (#909). The batched dispatch also composes installed hooks properly, so a `set_default_hooks` default no longer fires on `predict` and silently vanishes on `predict_batch`.
+
[Prediction hooks](#prediction-hooks) installed on the `Router` run once per request, as they do for `predict()`, so a redaction hook rewrites every state before the model sees it. Requests that share a checkpoint run all their start hooks before their shared forward pass; see [`docs/hooks/lifecycle.md`](https://github.com/NandhaKishorM/laya/blob/main/docs/hooks/lifecycle.md#routerpredict_batch).
You can also bound the Agent-level forward-pass batch size, and forward the length grouping knob
@@ -832,7 +723,7 @@ Configuration is by environment variable: `LAYA_HOST`, `LAYA_PORT`,
`LAYA_DEVICE`, `LAYA_PRELOAD`, `LAYA_MODELS` (comma list to preload),
`LAYA_THREADS` (cap torch intra-op threads for CPU inference — keep at or below
physical cores), `LAYA_AUTO_TASK`, `LAYA_DEFAULT_MODEL` (the checkpoint a state with no
-language evidence falls back to, `english` by default; set it to `multilingual` when most of
+language evidence falls back to, `multilingual` by default; set it to `english` when most of
your traffic is not English), `LAYA_MAX_LOADED` (checkpoints resident at
once, 2 by default; raise it to 3 when `LAYA_AUTO_TASK` makes a third one
reachable on demand, or the server rebuilds one every time routing switches),
diff --git a/docs/cli-mcp.md b/docs/cli-mcp.md
index 9433cbef..6a9ad724 100644
--- a/docs/cli-mcp.md
+++ b/docs/cli-mcp.md
@@ -239,7 +239,7 @@ the server is ready.
| `LAYA_MODELS` | `english,multilingual` | Comma-separated checkpoints to preload. An empty value keeps the MCP default rather than preloading every checkpoint. |
| `LAYA_THREADS` | PyTorch default | Caps Torch intra-op threads for CPU inference; keep it at or below the physical core count. |
| `LAYA_AUTO_TASK` | `0` | Set to `1` to let a request auto-route to the `typed-decisions` checkpoint. Same meaning as in `laya.serve`; it does not preload that checkpoint, so `LAYA_MODELS` still decides what is built at startup. |
-| `LAYA_DEFAULT_MODEL` | `english` | The checkpoint a state with no language evidence falls back to, same meaning as in `laya.serve`. Unlike `laya.serve`, an unresolvable name does not stop the server: it comes back as a `router construction failed` tool error on the next call, because a stdio server has no startup to refuse. |
+| `LAYA_DEFAULT_MODEL` | `multilingual` | The checkpoint a state with no language evidence falls back to, same meaning as in `laya.serve`. Unlike `laya.serve`, an unresolvable name does not stop the server: it comes back as a `router construction failed` tool error on the next call, because a stdio server has no startup to refuse. |
| `LAYA_BASE_URL` | unset | Send predictions to a `laya-serve` on your own hardware instead of loading checkpoints in each MCP process. A bare `host:port` is read as HTTP. |
| `LAYA_REMOTE_TIMEOUT` | `300` | HTTP timeout in seconds when `LAYA_BASE_URL` is set, including the server's cold load. Invalid or non-positive values use the default. |
diff --git a/docs/docker.md b/docs/docker.md
index e89f2321..99a43e67 100644
--- a/docs/docker.md
+++ b/docs/docker.md
@@ -338,7 +338,7 @@ These apply to the `laya-serve` service only.
| `LAYA_MODELS` | (all) | comma list to preload: `english,multilingual,typed-decisions` |
| `LAYA_THREADS` | `OMP_NUM_THREADS` | caps torch intra-op threads; keep at or below physical cores |
| `LAYA_AUTO_TASK` | `0` | `1` lets the router reach `typed-decisions` automatically |
-| `LAYA_DEFAULT_MODEL` | `english` | Checkpoint a state with no language evidence falls back to (no letters, or Latin text too short to identify). Set `multilingual` for mostly non-English traffic; an unresolvable name stops the container at startup instead of serving a configuration nobody asked for |
+| `LAYA_DEFAULT_MODEL` | `multilingual` | Checkpoint a state with no language evidence falls back to (no letters, or Latin text too short to identify). Set `english` for mostly English traffic; an unresolvable name stops the container at startup instead of serving a configuration nobody asked for |
| `LAYA_MAX_LOADED` | `2` | Checkpoints kept resident; `LAYA_AUTO_TASK` makes a third reachable on demand, and a cap below what routing chooses rebuilds one per switch |
| `LAYA_MAX_CONCURRENT` | `16` | requests admitted at once; later ones get `503` (a value that does not parse, or is not positive, falls back to `16`) |
| `LAYA_LOG_LEVEL` | `info` | uvicorn log level |
diff --git a/docs/routing.md b/docs/routing.md
index 3233acb7..c5e527a8 100644
--- a/docs/routing.md
+++ b/docs/routing.md
@@ -2,8 +2,9 @@
`Router` chooses a checkpoint for each request and loads its `Agent` when prediction needs it.
The default router sends English text to the English checkpoint and other supported languages to
-the multilingual checkpoint. You can override that choice, supply your own language hint, or select
-the typed-decisions checkpoint explicitly.
+the multilingual checkpoint. Text whose language it cannot place follows `default`, which is the
+multilingual checkpoint since 0.4.0. You can override that choice, supply your own language hint, or
+select the typed-decisions checkpoint explicitly.
This guide covers model selection and lifecycle. For the question types accepted by prediction,
see [Structured decisions](structured.md); for lifecycle callbacks, see [Prediction hooks](hooks/index.md).
@@ -51,7 +52,7 @@ Routing checks inputs in this order:
4. A recognized `lang=` value selects English or multilingual.
5. A per-call `lang_guess=` or the router's configured `lang_guess` is consulted.
6. Built-in script and language analysis selects a checkpoint. If there is no reliable language
- signal, the router uses its configured `default` (English by default).
+ signal, the router uses its configured `default` (multilingual since 0.4.0).
The first matching rule wins. For example, `model="multilingual"` overrides `lang="en"`. Invalid
model names raise `ValueError` instead of falling through to detection.
diff --git a/laya/__init__.py b/laya/__init__.py
index a391b85d..f42f5bd0 100644
--- a/laya/__init__.py
+++ b/laya/__init__.py
@@ -35,7 +35,7 @@ from .presets import (
from .router import DEFAULT_MODELS, RouteDecision, Router
from .structured import DecisionResult, decide, decide_batch
-__version__ = "0.3.29"
+__version__ = "0.4.0"
# Routing, language detection and email cleaning are pure Python. The torch-backed names are
# resolved lazily so that `import laya` -- and therefore `from laya import Router` or
diff --git a/laya/router.py b/laya/router.py
index bbcdc3e2..00bf193a 100644
--- a/laya/router.py
+++ b/laya/router.py
@@ -639,7 +639,17 @@ class Router(HookRegistry):
revision: Optional[str] = None,
revisions: Optional[Dict[str, Optional[str]]] = None,
max_loaded: int = 2,
- default: str = "english",
+ # The fallback for text whose language detection abstains, not for all traffic:
+ # detected non-Latin script already routes to `multilingual` whatever this says.
+ # `multilingual` since 0.4.0, on the refreshed 51-language sweep: it leads on 50 of
+ # the 51, the one exception being English itself (0.820 against 0.710), and by 0.180
+ # macro accuracy excluding English. The break-even is about 62% English traffic, so a
+ # mostly-English deployment should set `default="english"` back. The asymmetry that
+ # settles it for undecided text is the failure mode rather than the mean: off English
+ # the English checkpoint collapses while staying confident (Khmer 0.000 accuracy at
+ # 0.952 mean confidence), so no `min_confidence` gate downstream can catch it, while
+ # the multilingual checkpoint gives up 0.110 on English and stays gateable.
+ default: str = "multilingual",
auto_task_detection: bool = False,
standalone_repos: bool = False,
preload: bool = False,
@@ -1211,8 +1221,11 @@ class Router(HookRegistry):
elif det["language_undecided"]:
# Nothing identifies the language: too short, or only content words ("Quero cancelar",
# "Esqueci minha senha"). That is no evidence of English either, so it takes the same
- # `default` as a state with no letters. A deployment that serves mostly non-English
- # traffic sets `Router(default="multilingual")`; the stock default keeps it English.
+ # `default` as a state with no letters, which is `multilingual` since 0.4.0.
+ # #54 measured the cost of the old English default here: 128 of 200 German
+ # utterances reached this branch and lost 20 accuracy points against
+ # `model="multilingual"` on the same rows. A mostly-English deployment sets
+ # `Router(default="english")`.
key = self.default
reason = ("Latin script, language not identified and no non-English letters; "
"using default (%s)" % key)
diff --git a/nix/package.nix b/nix/package.nix
index 2ca76a97..f18512fc 100644
--- a/nix/package.nix
+++ b/nix/package.nix
@@ -12,7 +12,7 @@ let
# side, where tests/test_packaging.py compares all three declarations (this one,
# pyproject.toml and laya.__version__) and fails CI when they diverge. Nothing
# did, which is how this sat at 0.3.4 while the package reached 0.3.20.
- version = "0.3.29";
+ version = "0.4.0";
src = ../.;
format = "setuptools";
propagatedBuildInputs = with python3Packages; [
diff --git a/pyproject.toml b/pyproject.toml
index 01441018..71ac3df7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "laya"
-version = "0.3.29"
+version = "0.4.0"
description = "Fast, non-autoregressive System 1 decision engine with calibrated probabilities"
readme = "README.md"
requires-python = ">=3.10"
diff --git a/tests/test_hooks.py b/tests/test_hooks.py
index 813622f0..d079be82 100644
--- a/tests/test_hooks.py
+++ b/tests/test_hooks.py
@@ -522,7 +522,7 @@ check_true("skip/refusal under hooks_raise=False is warned",
# Router.predict used to turn an empty answer into `ctx.results[0]` -- IndexError, a 500
# over serve. The same refusal, raised where the contract is written.
-_skip_router = Router()
+_skip_router = Router(default="english")
_skip_router.attach("english", FakeAgent())
check_raises("router/skip empty is a ValueError, not an IndexError",
ValueError,
@@ -530,7 +530,7 @@ check_raises("router/skip empty is a ValueError, not an IndexError",
on_predict_start=lambda ctx: ctx.skip([])))
log = []
-r = Router()
+r = Router(default="english")
r.add_hook(Tag(log, "router"))
r.attach("english", FakeAgent())
r.predict("hello", QUESTIONS)
@@ -573,13 +573,13 @@ class LenFake:
lf = LenFake()
-r = Router()
+r = Router(default="english")
r.attach("english", lf)
r.predict("hello", QUESTIONS, max_len=256, head_max_len=128)
check("budget/router per-call reaches the agent", lf.seen, [(256, 128)])
lf = LenFake()
-r = Router()
+r = Router(default="english")
r.attach("english", lf)
r.predict("hello", QUESTIONS, on_predict_start=lambda c: setattr(c, "head_max_len", 96))
check("budget/router hook-set reaches the agent", lf.seen, [(None, 96)])
@@ -592,7 +592,7 @@ class StrictFake:
return {"model": "x", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}
-r = Router()
+r = Router(default="english")
r.attach("english", StrictFake())
r.predict("hello", QUESTIONS)
check("budget/default does not pass override kwargs", True, True)
@@ -639,7 +639,7 @@ class RouteHook:
rh = RouteHook()
-r = Router(hooks=[rh])
+r = Router(default="english", hooks=[rh])
decision = r.route("hello", QUESTIONS)
check("router/on_route fired", len(rh.decisions), 1)
check("router/on_route saw the original", rh.decisions[0]["model"], "english")
@@ -667,7 +667,7 @@ lh = LoadHook()
real_agent = _agent_mod.Agent
_agent_mod.Agent = BuiltAgent
try:
- r = Router(max_loaded=1, hooks=[lh])
+ r = Router(max_loaded=1, default="english", hooks=[lh])
r.load("english")
r.load("multilingual") # evicts english
finally:
@@ -677,7 +677,8 @@ check("router/on_evict fired on eviction", lh.evicts, ["english"])
predict_seen = {}
-r = Router(on_predict_start=lambda ctx: predict_seen.update(decision=dict(ctx.decision)),
+r = Router(default="english",
+ on_predict_start=lambda ctx: predict_seen.update(decision=dict(ctx.decision)),
on_predict_end=lambda ctx: predict_seen.update(results=ctx.results))
r.attach("english", FakeAgent())
out = r.predict("hello", QUESTIONS)
@@ -701,7 +702,7 @@ class PredictSuccessTrace:
success_trace = PredictSuccessTrace()
-r = Router(hooks=[success_trace])
+r = Router(default="english", hooks=[success_trace])
r.attach("english", FakeAgent())
success_out = r.predict("hello", QUESTIONS)
check("router/predict success lifecycle", success_trace.events, ["start", "end"])
@@ -724,14 +725,14 @@ class TimedRouter(Router):
timed_trace = PredictSuccessTrace()
-timed_router = TimedRouter(hooks=[timed_trace])
+timed_router = TimedRouter(default="english", hooks=[timed_trace])
timed_router.predict("hello", QUESTIONS)
timed_ctx = timed_trace.contexts[0]
check_true("router/success elapsed starts after load", timed_ctx.started_at >= timed_router.load_finished)
cached = [{"model": "cached", "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}}]
-r = Router()
+r = Router(default="english")
r.attach("english", FakeAgent())
skipped = r.predict("hello", QUESTIONS, on_predict_start=lambda c: c.skip(cached))
check("router/skip returns the cached payload", skipped, cached[0])
@@ -747,7 +748,7 @@ class PerCallRoute:
pcr = PerCallRoute()
-r = Router()
+r = Router(default="english")
r.attach("english", FakeAgent())
r.predict("hello", QUESTIONS, hooks=[pcr])
check("router/per-call hooks apply to on_route", len(pcr.decisions), 1)
@@ -800,7 +801,7 @@ class LoadFailRouter(Router):
load_trace = PredictFailureTrace()
load_error = None
try:
- LoadFailRouter(hooks=[load_trace]).predict("hello", QUESTIONS)
+ LoadFailRouter(default="english", hooks=[load_trace]).predict("hello", QUESTIONS)
except RuntimeError as exc:
load_error = exc
check_true("router/load failure propagates", isinstance(load_error, RuntimeError))
@@ -1159,7 +1160,7 @@ check_true("router_batch/hooks_raise=False warns", any("start hook failed" in st
# --------------------------------------------------------------- hooks_concurrent storage
-r = Router()
+r = Router(default="english")
check("router/hooks_concurrent default True", r.hooks_concurrent, True)
check_true("router/hooks_concurrent default has no lock", r._hooks_lock is None)
r = Router(hooks_concurrent=False)
@@ -1380,7 +1381,7 @@ check("defaults/cover router lifecycle", ld.events,
events = []
_hooks.set_default_hooks([LevelTag("default")])
-r = Router()
+r = Router(default="english")
r.attach("english", make_fake())
r.predict("a", QUESTIONS, model="english")
r.predict_batch([req("b")])
diff --git a/tests/test_lang_guess.py b/tests/test_lang_guess.py
index ce23facf..864d647c 100644
--- a/tests/test_lang_guess.py
+++ b/tests/test_lang_guess.py
@@ -171,8 +171,9 @@ check("standalone/hint still uses the standalone repo",
BEFORE = [("plain english", "I was charged twice and want a refund", "english"),
("German with umlauts", "Mein Konto wurde zweimal belastet, bitte erstatten Sie", "multilingual"),
("Hindi", "यह एक हिंदी वाक्य है", "multilingual"),
- ("empty", "", "english"),
- ("digits", "12345", "english")]
+ # letterless, so these follow `default`, which is multilingual since 0.4.0
+ ("empty", "", "multilingual"),
+ ("digits", "12345", "multilingual")]
for label, s, want in BEFORE:
check("unchanged/" + label, r0.route(s, GENERIC)["model"], want)
@@ -459,7 +460,11 @@ _COMPLAINT = ("I ordered a blender on the 3rd of March and it arrived broken.\n"
"I asked for a refund the same week and nobody has replied to me since.\n"
"%s\n"
"Please tell me when the money will be back on my card.")
-_route = Router(preload=False)
+_route = Router(preload=False, default="english") # see the note below
+# `default="english"` deliberately: a bare acronym line is language-undecided, so under the
+# stock 0.4.0 default it routes multilingual for want of evidence rather than because it was
+# named French. Pinning the default keeps these rows testing #1013 (the acronym must not be
+# named as foreign prose) instead of re-testing the default, which test_router.py covers.
check("shouted/bare acronym state routes english",
_route.route("MON DES EST LA")["model"], "english")
for _n in (1, 2, 3):
diff --git a/tests/test_mcp.py b/tests/test_mcp.py
index df59301e..a0611990 100644
--- a/tests/test_mcp.py
+++ b/tests/test_mcp.py
@@ -10,6 +10,7 @@ Device and preload-list tests follow the laya.serve environment contract
(LAYA_DEVICE / LAYA_PRELOAD / LAYA_MODELS / LAYA_THREADS / LAYA_AUTO_TASK /
LAYA_DEFAULT_MODEL).
"""
+import inspect
import asyncio
import os
import sys
@@ -2482,10 +2483,14 @@ def test_default_model_env():
mcp_mod._ROUTER = None # the server caches the Router it built
return mcp_mod._ensure_router(), build_router()
+ # Unset / empty / blank mean "whatever the Router's own default is", read off the
+ # signature rather than restated, so this cannot drift the next time it moves.
+ from laya.router import Router as _Router
+ _stock = inspect.signature(_Router.__init__).parameters["default"].default
for label, value, want in (
- ("unset", None, "english"),
- ("empty", "", "english"),
- ("blank", " ", "english"),
+ ("unset", None, _stock),
+ ("empty", "", _stock),
+ ("blank", " ", _stock),
("canonical", "multilingual", "multilingual"),
("alias", "ml", "multilingual"),
("padded_upper", " MULTI ", "multilingual"),
@@ -2510,8 +2515,8 @@ def test_default_model_env():
stock_mcp, stock_serve = build(None)
nonenglish_mcp, nonenglish_serve = build("multilingual")
for index, state in enumerate(("12345 !!!", "Quero cancelar")):
- ok("default_model/stock_english_%d" % index,
- stock_mcp.route(state).model == "english" == stock_serve.route(state).model,
+ ok("default_model/stock_%d" % index,
+ stock_mcp.route(state).model == _stock == stock_serve.route(state).model,
"mcp=%r serve=%r" % (stock_mcp.route(state).model, stock_serve.route(state).model))
ok("default_model/fallback_multilingual_%d" % index,
nonenglish_mcp.route(state).model == "multilingual"
diff --git a/tests/test_predict_batch.py b/tests/test_predict_batch.py
index 776a1677..bdcce554 100644
--- a/tests/test_predict_batch.py
+++ b/tests/test_predict_batch.py
@@ -138,9 +138,12 @@ class RouterPredictBatchTests(unittest.TestCase):
return [{"state": s, "questions": questions} for s in states]
def test_mixed_language_batch_routes_and_preserves_order(self):
- states = ["hello", # latin -> english
- "नमस्ते", # devanagari -> multilingual
- "goodbye"] # latin -> english
+ # Sentences rather than single words: a bare "hello" names no language, so since 0.4.0 it
+ # follows `default` and all three rows would group together, which would stop this
+ # testing that a mixed batch routes per item at all.
+ states = ["Please refund the duplicate charge on my invoice", # identified en -> english
+ "नमस्ते", # devanagari -> multilingual
+ "I would like to close my account today please"] # identified en -> english
results = self.router.predict_batch(self._requests(states))
self.assertEqual(len(results), 3)
self.assertEqual([r["routing"]["model"] for r in results],
diff --git a/tests/test_router.py b/tests/test_router.py
index a79a843a..acca020e 100644
--- a/tests/test_router.py
+++ b/tests/test_router.py
@@ -134,7 +134,9 @@ check("latin_lang/long english stays en",
# `github` + `com`, so every domain in a state scored a Portuguese hit: two of them crossed the
# margin and sent an English state with links in it to the multilingual checkpoint, reported as
# Portuguese. A token whose dot or @ joins word characters is an identifier, not prose.
-_r_dotted = Router()
+# `default="english"` because this block is about identifiers not reading as foreign prose, not
+# about what the stock default is: a state of bare identifiers is language-undecided either way.
+_r_dotted = Router(default="english")
for label, state in [
("url and email fields", {"url": "github.com", "email": "[email protected]"}),
("two bare domains", "github.com acme.com"),
@@ -394,8 +396,9 @@ cases = [
("explicit lang en", {"body": "मुझसे दो बार"}, Q_GENERIC, {"lang": "en"}, "english"),
("explicit lang de", {"body": "hello there"}, Q_GENERIC, {"lang": "de"}, "multilingual"),
("td workflow, auto OFF", {"body": "I was charged twice"}, Q_TD, {}, "english"),
- ("empty state", {}, Q_GENERIC, {}, "english"),
- ("none state", None, Q_GENERIC, {}, "english"),
+ # letterless, so these follow `default`, which is multilingual since 0.4.0
+ ("empty state", {}, Q_GENERIC, {}, "multilingual"),
+ ("none state", None, Q_GENERIC, {}, "multilingual"),
]
for label, state, qs, kw, want in cases:
check("route/" + label, r.route(state, qs, **kw)["model"], want)
@@ -447,17 +450,23 @@ check("route/undecided reason mentions letters",
"not identified" in _r_lat.route("Müşteriden iki kez ücret alındı ve para iadesi istiyor").reason, True)
check("route/english still english",
_r_lat.route("Please refund the duplicate charge on invoice 4411 today.").model, "english")
-check("route/short english still english", _r_lat.route("refund me").model, "english")
+# "refund me" names no language, so it follows `default` like any other undecided text rather
+# than being hard-coded English. With an English default it still reaches the English checkpoint.
+check("route/short english follows default",
+ Router(default="english").route("refund me").model, "english")
# Undecided Latin text follows `default`, as a state with no letters already did. Short messages made
# only of content words carry nothing that names their language, and hard-coding English for them
# sent every short Portuguese message to the checkpoint that is 0.97 confident at 0.47 accuracy on
-# `pt`, whatever the router was configured with.
+# `pt`, whatever the router was configured with. Since 0.4.0 the stock default is `multilingual`,
+# so both arms below are exercised: the stock one and an explicit English override.
_r_ml = Router(default="multilingual")
+_r_en = Router(default="english")
for text in ["Quero cancelar", "Esqueci minha senha", "Fui cobrado duas vezes",
"Produto veio quebrado, quero trocar", "refund me"]:
check("route/undecided follows default " + text[:24], _r_ml.route(text).model, "multilingual")
- check("route/undecided stock default " + text[:24], _r_lat.route(text).model, "english")
+ check("route/undecided stock default " + text[:24], _r_lat.route(text).model, "multilingual")
+ check("route/undecided english override " + text[:24], _r_en.route(text).model, "english")
check("route/undecided reason names the default",
"using default (multilingual)" in _r_ml.route("Esqueci minha senha").reason, True)
# identified English is not undecided, so a non-English default leaves it alone
@@ -606,7 +615,8 @@ for text in [
"The son of the director filed a complaint about the duplicate invoice",
]:
check("is_english/romance control " + text[:32], is_english(text), True)
- check("route/romance control " + text[:32], _r_lat.route(text).model, "english")
+ check("route/romance control " + text[:32],
+ Router(default="english").route(text).model, "english")
# A word several lists claim (`la`, `e`, `o`) says "not English" without saying *which* language, so
# it may not name one on its own -- the same rule as the 0-0 tie above, which is why the sample
@@ -1547,12 +1557,15 @@ for label, text in (
# Fullwidth Latin is Latin, not an unlisted script.
check("unlisted/fullwidth latin is latin", detect_script("HELLO"), "latin")
-# A state with no letters at all must keep behaving exactly as before.
+# A state with no letters at all takes `default`, whatever that is, which is the behaviour that
+# must not change. Both arms are checked so this cannot pass by agreeing with one constant.
for label, text in (("empty", ""), ("digits only", "12345 67890"), ("emoji only", "😀😀😀")):
check("unlisted/letterless " + label + " is still unknown",
analyse(text)["script"], "unknown")
- check("unlisted/letterless " + label + " keeps the default",
- Router().route(text)["model"], "english")
+ check("unlisted/letterless " + label + " keeps the stock default",
+ Router().route(text)["model"], "multilingual")
+ check("unlisted/letterless " + label + " keeps an english default",
+ Router(default="english").route(text)["model"], "english")
# The scripts the table does name must be untouched.
for label, text, script in (
diff --git a/tests/test_router_batch.py b/tests/test_router_batch.py
index 65354349..3cd9e208 100644
--- a/tests/test_router_batch.py
+++ b/tests/test_router_batch.py
@@ -52,7 +52,7 @@ def fake_agent(monkeypatch):
])
def test_mixed_groups_keep_order_and_lru(fake_agent, capacity, expected):
built, calls = fake_agent
- router = Router(max_loaded=capacity, auto_task_detection=True)
+ router = Router(max_loaded=capacity, default="english", auto_task_detection=True)
typed = {key: {"type": "noul", "instructions": "?"} for key in
("action", "needs_review", "outcome", "risk", "urgency")}
items = [request("English text one"), request("مرحبا"),
@@ -97,7 +97,7 @@ def test_invalid_batch_fails_before_loading(fake_agent, items, error, fragment):
def test_inference_exception_propagates_and_cache_remains_consistent(fake_agent):
built, calls = fake_agent
- router = Router(max_loaded=1)
+ router = Router(max_loaded=1, default="english")
with pytest.raises(RuntimeError, match="inference failed"):
router.predict_batch([request("first"), request("raise", lang="ar"),
request("unreached", lang="ar")])
@@ -137,14 +137,14 @@ def test_evicted_checkpoint_is_unreferenced_when_it_is_evicted(monkeypatch):
evicted_alive.append((ctx.model, refs[ctx.model]() is not None))
monkeypatch.setattr(laya.agent, "Agent", Agent)
- router = Router(max_loaded=1, hooks=[CheckFreed()])
+ router = Router(max_loaded=1, default="english", hooks=[CheckFreed()])
router.predict_batch([request("english text"), request("مرحبا")])
assert evicted_alive == [("english", False)]
def test_warm_cache_and_repeated_batches(fake_agent):
built, _ = fake_agent
- router = Router(max_loaded=2)
+ router = Router(max_loaded=2, default="english")
router.load("multilingual")
result = router.predict_batch([request("en", model="english"),
request("ar", model="multilingual"),
@@ -159,7 +159,7 @@ def test_warm_cache_and_repeated_batches(fake_agent):
def test_same_checkpoint_same_questions_uses_one_agent_batch(fake_agent):
_, calls = fake_agent
- router = Router(max_loaded=2)
+ router = Router(max_loaded=2, default="english")
items = [
request("one", model="english"),
request("two", model="english"),
@@ -178,7 +178,7 @@ def test_same_checkpoint_same_questions_uses_one_agent_batch(fake_agent):
def test_same_checkpoint_different_questions_split_agent_batches(fake_agent):
_, calls = fake_agent
- router = Router(max_loaded=2)
+ router = Router(max_loaded=2, default="english")
q2 = {"risk": {"type": "noul", "instructions": "Risky?"}}
items = [
{"state": "one", "questions": Q, "model": "english"},
@@ -246,7 +246,7 @@ def test_concurrent_batch_load_deduplicates(monkeypatch):
return self.predict_batch([state], questions)[0]
monkeypatch.setattr(laya.agent, "Agent", SlowAgent)
- router = Router(max_loaded=2)
+ router = Router(max_loaded=2, default="english")
start = Barrier(8)
def worker(_):
diff --git a/tests/test_serve.py b/tests/test_serve.py
index eb5df2dc..26f7a152 100644
--- a/tests/test_serve.py
+++ b/tests/test_serve.py
@@ -792,7 +792,7 @@ def test_default_model_reaches_the_router_the_server_builds(monkeypatch):
# docs/docker.md quote, so moving Router's default has to move those too.
router, _ = _server_router(monkeypatch)
assert router.default == Router().default
- assert router.default == "english"
+ assert router.default == "multilingual"
for raw, want in (("multilingual", "multilingual"), ("ml", "multilingual"),
(" MULTI ", "multilingual"), ("typed-decisions", "typed-decisions")):
router, _ = _server_router(monkeypatch, LAYA_DEFAULT_MODEL=raw)
@@ -803,10 +803,14 @@ def test_default_model_reaches_the_router_the_server_builds(monkeypatch):
# this costs no weights.
ambiguous = ("12345 !!!", "Quero cancelar", "Esqueci minha senha")
stock, _ = _server_router(monkeypatch)
+ english, _ = _server_router(monkeypatch, LAYA_DEFAULT_MODEL="english")
portuguese, _ = _server_router(monkeypatch, LAYA_DEFAULT_MODEL="multilingual")
for state in ambiguous:
- assert stock.route(state).model == "english", state
- assert "using default (english)" in stock.route(state).reason, state
+ # the stock default since 0.4.0, and an explicit override in each direction
+ assert stock.route(state).model == "multilingual", state
+ assert "using default (multilingual)" in stock.route(state).reason, state
+ assert english.route(state).model == "english", state
+ assert "using default (english)" in english.route(state).reason, state
assert portuguese.route(state).model == "multilingual", state
assert "using default (multilingual)" in portuguese.route(state).reason, state
# A fallback, not a pin: text the detector can place routes on what it detects.