Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
laya-routing
The combined routing sample. It exercises everything the base Rust SDK does not cover on its
own: language/script detection, LayaRouter routing decisions and predictions, email cleaning,
and the embedding shortlist.
Output contract
This is the load-bearing part of this file. This sample's stdout format is a stable contract (apart from timings, which never go to stdout — see below), so anyone changing it should keep this section up to date.
- Flags:
--model-root <dir>(default: theLAYA_ONNX_ROOTenv var),--download,--preload,-h/--help. - Exit codes:
0ok,1a runtime or artifact error (bad model root, missing checkpoint file),2bad usage (unknown flag, a flag missing its value). - Section headers, printed in this order:
== 1. Language detection ==,== 2. Routing decisions ==,== 3. Router predict ==,== 4. Email cleaning ==,== 5. Shortlist ==. A single blank line separates each section's output from the next section's header (no blank line after the last section). - All numbers are printed with exactly 4 decimals (
0.1234), not fewer, not scientific notation. - Section 1 (language detection), one line per state:
{name:<22} script={script:<10} language={language or "-":<4} is_english={true|false} - Section 2 (routing decisions,
Routeonly, nothing loaded), one line per case:{name:<26} -> {model subdir} ({reason})— two spaces before the parenthesised reason. - Section 3 (router predict), per email: a routed-to header,
[{lang}] routed to {subdir}: {reason}, then one line per answered question:{id:<18} {value}, wherevalueis the choice label for a choice question, the score to 4 decimals for a score question, or theP(true)probability to 4 decimals for a noul question (notyes/no). - Section 4 (email cleaning): the cleaned body verbatim between a
--- cleaned ---line and an--- end ---line, then theLayaPresets.Email()answers in the same routed-header-plus-lines format as section 3. - Section 5 (shortlist):
kept {k} of {n}:, then one line per kept label in rank order —{label:<28} {score:.4}— thenanswer: {choice} ({probability:.4}). - Timings go to stderr only, never stdout, and are not part of the diff.
Sample inputs are copied verbatim from laya-rust/tools/routing_cases.py's SAMPLE_* constants into
src/main.rs — no runtime JSON is read, so both samples read literally the same
bytes, recorded once from Python as laya-rust/tools/dump_routing_golden.py's sample_inputs.json.
Verified output
Run against the three exported checkpoints, this sample's stdout is:
== 1. Language detection ==
english script=latin language=en is_english=true
spanish_accent_stripped script=latin language=es is_english=false
french script=latin language=fr is_english=false
romanian script=latin language=ro is_english=false
hindi script=devanagari language=- is_english=false
arabic script=arabic language=- is_english=false
japanese script=kana language=- is_english=false
dict_state script=devanagari language=- is_english=false
== 2. Routing decisions ==
auto_english -> english (English Latin text)
auto_hindi -> multilingual (non-Latin script (devanagari, 100% of letters); the English checkpoint cannot read it)
auto_unknown_latin_romanian -> multilingual (Latin script but language looks like 'ro', not English)
explicit_model_multilingual -> multilingual (explicit model='multilingual')
explicit_lang_de -> multilingual (explicit lang='de')
workflow_opt_in_customer_service -> typed-decisions (question ids match the 'customer_service' typed-decisions workflow)
workflow_opt_in_off_by_default -> english (English Latin text)
== 3. Router predict ==
[en] routed to english: English Latin text
department billing
urgency 1.4400
churn_risk 0.8248
refund_requested 0.8430
[hi] routed to multilingual: non-Latin script (devanagari, 74% of letters); the English checkpoint cannot read it
department billing
urgency 1.9072
churn_risk 0.1489
refund_requested 0.9966
== 4. Email cleaning ==
--- cleaned ---
Hi team,
I was charged twice for the Pro plan this month and the second charge still has not been refunded. This is the third time I have written about it -- please resolve this today or I will need to cancel the account.
--- end ---
[en] routed to english: English Latin text
category billing
is_spam 0.1318
is_phishing 0.1345
urgency 1.4932
needs_reply 0.1545
== 5. Shortlist ==
kept 20 of 40:
card_swallowed 0.5369
compromised_card 0.5229
cash_withdrawal_not_recognised 0.4926
card_payment_not_recognised 0.4672
automatic_top_up 0.4631
card_linking 0.4371
atm_support 0.3783
direct_debit_payment_not_recognised 0.3643
card_payment_wrong_exchange_rate 0.3472
balance_not_updated_after_bank_transfer 0.3320
card_not_working 0.3301
declined_card_payment 0.3205
card_payment_fee_charged 0.3154
card_acceptance 0.2983
exchange_rate 0.2915
card_delivery_estimate 0.2862
activate_my_card 0.2751
balance_not_updated_after_cheque_or_cash_deposit 0.2737
card_about_to_expire 0.2668
country_support 0.2631
answer: card_swallowed (0.7885)
Ambiguities resolved while implementing this contract
The plan's contract text left two things unspecified:
{lang}in section 3/4's routed-to header is a short, fixed, caller-chosen tag per input ("en","hi") — not derived fromRouteDecision.detection.language. A non-Latin-script decision (the Hindi email here) never gets alanguageguess at all: routing is decided by script alone, sodetection.languageisNone/nulland there would be nothing to print.- The shortlist question's id (used nowhere in
laya-rust/tools/routing_cases.py, which only records state/instructions/criteria, not a question id) is"banking_intent".
Run it
--model-roottakes the export root, not one checkpoint's folder. Point it at the directory that containsenglish/,multilingual/andtyped-decisions/(for exampleonnx/), not atonnx/multilingual. This is the opposite oflaya-sampleandlaya-benchmark, which take a single checkpoint's folder with--model-dirand reject--model-root.
Everything after the bare -- is passed to the app; without it, cargo run tries to read
--model-root itself. From the laya-rust directory:
cargo run --release -p laya-routing -- --model-root ../onnx
cargo run --release -p laya-routing -- --model-root ../onnx --preload
Or set the environment variable once and leave out the flag:
$env:LAYA_ONNX_ROOT = "C:\path\to\onnx"
cargo run --release -p laya-routing
The root must contain one subdirectory per checkpoint used by the sample (english/,
multilingual/, typed-decisions/), each holding model.onnx, model.onnx.data,
rl_agent_config.json and tokenizer/tokenizer.json — the same layout laya-sample expects for a
single checkpoint. LayaOptions has no "root" field of its own (only a single-checkpoint
model_directory), so --model-root works by overriding the LAYA_ONNX_ROOT process environment
variable that ModelArtifacts::resolve already reads per checkpoint.
Command-line options
| Option | Effect |
|---|---|
--model-root <dir> |
Root directory holding every checkpoint subdirectory. Overrides LAYA_ONNX_ROOT for this process. |
--download |
Fetch missing checkpoints from Hugging Face into the local cache, with per-file progress on stderr. |
--preload |
Build and load every checkpoint up front (LayaRouterOptions::preload) instead of on first use. |
-h, --help |
Print usage and exit 0. |
| Exit code | Meaning |
|---|---|
0 |
All five sections printed. |
1 |
A checkpoint's artifact could not be resolved, or another runtime error occurred. The resolver's message, which lists every location tried, is printed to stderr. |
2 |
Bad arguments: a flag missing its value, or an unrecognized argument. |
See laya-sample's README for the two path pitfalls that apply
here too (quoting Windows paths in a POSIX shell, and LAYA_ONNX_ROOT needing to be absolute).
--download has the same limitation as in laya-sample: it can't succeed today, because
convaiinnovations/laya publishes PyTorch weights, not an ONNX export.
What each section demonstrates
- Language detection —
LanguageDetection::analyseover ~8 states (English, accent-stripped Spanish, French, Romanian, Hindi, Arabic, Japanese, and aserde_json::json!dict state), showing detected script, best-effort Latin-language guess, and whether the English checkpoint can be expected to read the text. No model is loaded for this section. - Routing decisions —
LayaRouter::routealone (neverpredict), so nothing is loaded. Covers the full precedence ladder: script/language auto-detection, an explicitmodel, an explicitlang, and the opt-in typed-decisions workflow match (shown both withauto_task_detectionon, where it decides the route, and off, where the match is still detected but does not win). - Router predict — the same four support questions (
department,urgency,churn_risk,refund_requested) asked of an English and a Hindi support email through oneLayaRouterwithmax_loaded: 2, so both checkpoints stay resident and neither request evicts the other. - Email cleaning — a raw email with quoted history, a signature, and a confidentiality
disclaimer goes through
LayaEmail::clean_body(printed verbatim) andLayaEmail::state, thenLayaPresets::email(None)through the same router. - Shortlist — a 40-label banking-intent choice question is narrowed to the top 20 by cosine
similarity (
LayaShortlist::predict,DEFAULT_SHORTLIST_K), using the demo hashing embedder inhashing_embedder.rs— demo only, replace with a real embedding model. It is a deterministic, dependency-free, hashed-character-trigram bag-of-features vector, ported bit-for-bit fromlaya-rust/tools/hashing_embedder.py(the same port used by the SDK's own test suite attests/common/hashing_embedder.rs), chosen only because every language in this repo can reproduce it identically without a network call or a bi-encoder model.
Code walkthrough
src/main.rs follows the same shape as
laya-sample: hand-parsed arguments (no argument-parsing crate,
because an unrecognized flag should fail loudly rather than be silently dropped), a LayaRouter
built once up front, then one function per section. timed() wraps each of the three sections
that actually call predict, printing wall-clock time to stderr only — the sample's stdout is a
stable contract, and wall-clock time never could be part of it.
Troubleshooting
Same as laya-sample's table, with --model-root /
LAYA_ONNX_ROOT in place of --model-dir / LAYA_ONNX_DIR. In particular:
| Symptom | Cause |
|---|---|
Laya model artifact (~1.3 GB) not found under LAYA_ONNX_ROOT env var <root>\<checkpoint> |
--model-root/LAYA_ONNX_ROOT points at a directory that has no <checkpoint>/model.onnx subdirectory for the checkpoint being routed to. |
Devanagari, Arabic, or CJK text prints as ? or boxes |
The console code page or font can't display it. The answers aren't affected. Run chcp 65001 or use Windows Terminal. |
First predict call takes noticeably longer |
ORT session warm-up for that checkpoint. Pass --preload to pay this cost once, up front, for every checkpoint instead of on first use. |