Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(cli): make core's soft language hint reachable from the command line
Aashish5918228df5d7
diff --git a/README.md b/README.mdindex 328a6194..a6b52e8c 100644--- a/README.md+++ b/README.md@@ -219,6 +219,7 @@ Installing the package also installs a `laya` command for quick local testing, nlaya "I was charged twice, please refund" # routing decision only; works offline, no downloadlaya "Refactor this service" --predict # full answers (downloads the checkpoint on first use)laya "Mein Konto wurde zweimal belastet" --lang de # force a language instead of detecting it+laya "My payment failed twice" --lang-guess en # a soft hint: nudge routing, still fall through to detectionlaya "My payment failed twice" --model ml # pin a checkpoint: names, aliases and casing all resolve as the SDK resolves themlaya "My payment failed twice" --preset triage # answer a ready-made preset (triage, email, guard, moderation, router)laya --batch tickets.txt --predict # score a file of requests, one per line, in one batch@@ -227,6 +228,12 @@ laya "Where is my card" --questions intents.json # answer your own questionslaya # interactive mode```+`--lang` is decisive: a real code picks the checkpoint and skips detection. `--lang-guess` is the+soft sibling core checks in between, so a probable-but-uncertain language can nudge routing without+forcing it — and a hint that resolves to nothing falls through to the built-in detector, exactly the+`Router(lang_guess=...)` behaviour the library offers. Neither flag changes the routing decision's+precedence rules; `--lang-guess` just adds a hint those rules read before detection.+Routing alone never downloads a checkpoint, so it returns in milliseconds. `--predict` loads the routed checkpoint, which needs network access to the Hugging Face hub the first time; if a checkpoint cannot be downloaded, the CLI says so instead of crashing. `--batch` (with or without `--predict`) sends the whole file through `Router.predict_batch` in one process, so the requests share checkpoint loads and forward passes — measured 2.6x on 20 tickets vs looping `predict` one by one, with `--batch-size N` to bound the forward pass, `--sort-by-length` to group similarly sized requests inside it, and `--json` for JSONL output. Batch routing (`laya --batch FILE`, no `--predict`) likewise answers with `route_batch` in one pass, still without loading anything.`--sort-by-length` reaches the length grouping `Agent.predict_batch` has done since #294, whichdiff --git a/docs/cli-mcp.md b/docs/cli-mcp.mdindex c880bcf3..e9e1916b 100644--- a/docs/cli-mcp.md+++ b/docs/cli-mcp.md@@ -71,6 +71,9 @@ The main controls are:- `--model english|multilingual|typed-decisions` pins a checkpoint instead of auto-routing.- `--lang en|de|...` supplies an explicit language code instead of automatic detection.+- `--lang-guess en|de|...` supplies a soft hint that routing reads after `--lang` and before its+ own detector; a hint that resolves to nothing falls through, so it nudges the checkpoint without+ forcing it.- `--task NAME` forces the typed-decisions workflow instead of detecting it.- `--device cpu|cuda|...` passes a device choice to the Router.- `--json` emits machine-readable output.diff --git a/laya/cli.py b/laya/cli.pyindex 6b5a7955..8bed9271 100644--- a/laya/cli.py+++ b/laya/cli.py@@ -4,6 +4,7 @@laya "Refactor this service" --predict # full answers, loads the checkpointlaya # interactive modelaya "Mein Konto wurde zweimal belastet" --lang de # explicit language+ laya "My payment failed twice" --lang-guess en # a soft hint that still lets routing decidelaya "My payment failed twice" --model ml # pin a checkpoint, by name or aliaslaya "My payment failed twice" --preset triage # a ready-made question presetlaya --batch tickets.txt --predict # a file of requests, one per line@@ -81,6 +82,10 @@ def build_parser():help="force a checkpoint instead of auto-routing: a checkpoint name or any of ""core's aliases, in any casing, or 'auto' to route it (the default)")parser.add_argument("--lang", help="force a language, e.g. en or de, instead of detecting it")+ parser.add_argument("--lang-guess", dest="lang_guess", metavar="CODE",+ help="a soft language hint that participates in routing instead of skipping "+ "detection: checked after --lang and before the built-in detector, so a "+ "probable-but-uncertain code can nudge the checkpoint without forcing it")parser.add_argument("--task", help="force a typed-decisions workflow instead of detecting it")parser.add_argument("--preset", choices=sorted(PRESETS), metavar="NAME",help="answer a ready-made question preset (%s) instead of the router questions; implies --predict"@@ -216,6 +221,7 @@ def run(text, args, router=None):state = {state_key: text}result = router.predict(state, questions,model=args.model, task=args.task, lang=args.lang,+ lang_guess=args.lang_guess,**budget_overrides(args))if args.json:print(json.dumps(result, ensure_ascii=False, indent=2, default=str))@@ -223,7 +229,8 @@ def run(text, args, router=None):show_answers(result)else:state = {"text": text}- decision = router.route(state, model=args.model, task=args.task, lang=args.lang)+ decision = router.route(state, model=args.model, task=args.task, lang=args.lang,+ lang_guess=args.lang_guess)if args.json:print(json.dumps(dict(decision), ensure_ascii=False, indent=2, default=str))else:@@ -255,7 +262,9 @@ def run_batch(lines, args, router=None):forward passes; returns 0 on success, 2 on a handled error."""router = router or make_router(args)overrides = {key: value for key, value in (("model", args.model), ("task", args.task),- ("lang", args.lang)) if value is not None}+ ("lang", args.lang),+ ("lang_guess", args.lang_guess))+ if value is not None}try:if args.predict or args.preset or getattr(args, "questions", None):questions, key = resolve_questions(args)diff --git a/tests/test_cli_lang_guess.py b/tests/test_cli_lang_guess.pynew file mode 100644index 00000000..2c622699--- /dev/null+++ b/tests/test_cli_lang_guess.py@@ -0,0 +1,200 @@+"""The CLI's `--lang-guess` must reach core's soft language hint. No weights are loaded."""+import io+import os+import sys+import inspect+from contextlib import redirect_stderr, redirect_stdout++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))++from laya import cli # noqa: E402++PASS, FAIL = [], []+++def check(name, condition, detail=""):+ if condition:+ PASS.append(name)+ else:+ FAIL.append("%s: %s" % (name, detail))+++class StubDecision(dict):+ def __init__(self):+ super().__init__(model="multilingual", repo="convaiinnovations/laya",+ reason="detected non-English text", detection={"lang": "de"}, workflow=None)+++class StubRouter:+ """Records the kwargs of every route/predict call the CLI makes."""++ def __init__(self):+ self.route_calls = []+ self.predict_calls = []++ def route(self, state, **kwargs):+ self.route_calls.append((state, kwargs))+ return StubDecision()++ def predict(self, state, questions, **kwargs):+ self.predict_calls.append((state, kwargs))+ return {"answers": {"difficulty": {"score": 1.4}}, "routing": dict(StubDecision())}+++class BatchRouter:+ def __init__(self):+ self.predict_batch_calls = []+ self.route_batch_calls = []++ def predict_batch(self, requests, batch_size=None, **extra):+ self.predict_batch_calls.append((requests, batch_size, extra))+ return [{"model": "laya-rl", "answers": {"difficulty": {"score": float(i)}}, "usage": {}}+ for i in range(len(requests))]++ def route_batch(self, requests):+ self.route_batch_calls.append(requests)+ return [StubDecision() for _ in requests]+++def run_cli(argv, router=None):+ stub = router or StubRouter()+ original = cli.make_router+ cli.make_router = lambda args: stub+ out, err = io.StringIO(), io.StringIO()+ code = None+ try:+ with redirect_stdout(out), redirect_stderr(err):+ code = cli.main(argv)+ except SystemExit as exit_:+ code = exit_.code+ except Exception as error: # noqa: BLE001 - a forwarding bug should fail a named check, not+ code = "raised: %s" % error.__class__.__name__ # crash the suite before the report prints+ finally:+ cli.make_router = original+ return code, out.getvalue(), err.getvalue(), stub+++def first_kwargs(calls):+ """The kwargs dict of the first recorded call, or {} when none ran.++ Read defensively so a broken flag fails its own named check instead of raising IndexError+ and aborting the run before the report prints -- a red that names the check is the witness.+ """+ return calls[0][1] if calls else {}+++def parse_value(argv):+ """Parse argv through the real parser, or None if argparse rejects it (an unknown flag)."""+ try:+ with redirect_stderr(io.StringIO()):+ return cli.build_parser().parse_args(argv)+ except SystemExit:+ return None+++# ------------------------------------------------------------------ the parser knows the flag+ns = parse_value(["t", "--lang-guess", "de"])+check("parse: --lang-guess reaches the namespace",+ ns is not None and getattr(ns, "lang_guess", None) == "de", repr(ns))+check("parse: omitted means no hint (None), not '' or 'auto'",+ getattr(parse_value(["t"]), "lang_guess", "MISSING") is None,+ repr(getattr(parse_value(["t"]), "lang_guess", "MISSING")))++# `--lang` forces a language and skips detection; `--lang-guess` is the soft sibling that+# participates in routing. Two different parameters with two different contracts, so the flag+# must drive the one core names `lang_guess` -- and the names are core's, not a copy.+from laya.router import Router as _CoreRouter # noqa: E402++check("core: Router.route really takes lang_guess",+ "lang_guess" in inspect.signature(_CoreRouter.route).parameters,+ str(list(inspect.signature(_CoreRouter.route).parameters)))+check("core: Router.predict really takes lang_guess",+ "lang_guess" in inspect.signature(_CoreRouter.predict).parameters,+ str(list(inspect.signature(_CoreRouter.predict).parameters)))++# ------------------------------------------------------------------ single-request forwarding+code, out, err, stub = run_cli(["Ich wurde doppelt belastet", "--lang-guess", "de"])+check("route: exit code", code == 0, "got %r, err %r" % (code, err))+check("route: --lang-guess reaches Router.route",+ first_kwargs(stub.route_calls).get("lang_guess") == "de", str(first_kwargs(stub.route_calls)))++code, out, err, stub = run_cli(["My payment failed twice", "--predict", "--lang-guess", "en"])+check("predict: exit code", code == 0, "got %r, err %r" % (code, err))+check("predict: --lang-guess reaches Router.predict",+ first_kwargs(stub.predict_calls).get("lang_guess") == "en", str(first_kwargs(stub.predict_calls)))++# Unset is a real state, not a missing one: core reads `lang_guess=None` as "no per-call hint,+# fall through to Router(lang_guess=...) and detection", so the CLI must send None -- not omit the+# keyword and not send the empty string, which would name a language that is not there.+code, out, err, stub = run_cli(["hello", "--predict"])+sent = first_kwargs(stub.predict_calls)+check("predict: unset sends lang_guess=None (a fall-through, not a blank hint)",+ "lang_guess" in sent and sent["lang_guess"] is None, str(sent))+code, out, err, stub = run_cli(["hello"])+sent = first_kwargs(stub.route_calls)+check("route: unset sends lang_guess=None",+ "lang_guess" in sent and sent["lang_guess"] is None, str(sent))++# --lang (decisive) and --lang-guess (soft) are separate parameters; setting both must forward+# both under their own keys rather than one clobbering the other.+code, out, err, stub = run_cli(["hi", "--predict", "--lang", "de", "--lang-guess", "fr"])+sent = first_kwargs(stub.predict_calls)+check("both flags forwarded under distinct keys",+ sent.get("lang") == "de" and sent.get("lang_guess") == "fr", str(sent))++# ------------------------------------------------------------------ batch forwarding+import tempfile # noqa: E402++tmp = tempfile.mkdtemp()+path = os.path.join(tmp, "requests.txt")+with open(path, "w", encoding="utf-8") as handle:+ handle.write("first ticket\nsecond ticket\n")+++def first_batch_requests(calls):+ return calls[0][0] if calls else []+++code, out, err, stub = run_cli(["--batch", path, "--predict", "--lang-guess", "de"],+ router=BatchRouter())+requests = first_batch_requests(stub.predict_batch_calls)+check("batch predict: exit code", code == 0, "got %r, err %r" % (code, err))+check("batch predict: every request carries lang_guess",+ requests and all(r.get("lang_guess") == "de" for r in requests),+ str([sorted(r) for r in requests]))++requests = run_cli(["--batch", path, "--lang-guess", "de"], router=BatchRouter())[3].route_batch_calls+requests = requests[0] if requests else []+check("batch route: every request carries lang_guess",+ requests and all(r.get("lang_guess") == "de" for r in requests),+ str([sorted(r) for r in requests]))++# Batch shares the same `is not None` rule the model/task/lang overrides already use: an unset+# hint is absent from the request dict, so route()/predict() see their own default, not a forced+# None that could shadow a Router(lang_guess=...) the caller installed.+code, out, err, stub = run_cli(["--batch", path, "--predict"], router=BatchRouter())+requests = first_batch_requests(stub.predict_batch_calls)+check("batch predict: unset hint is absent, not sent as None",+ requests and all("lang_guess" not in r for r in requests),+ str([sorted(r) for r in requests]))++# ------------------------------------------------------------------ drift guard vs the serve surface+# `laya.serve` forwards lang_guess as a per-call body control. The CLI is the same wrapper class,+# so the routing hints it can send should cover the ones the HTTP surface can send. If core ever+# renames or drops lang_guess from that set, this fails rather than the CLI quietly going stale.+from laya import serve as _serve # noqa: E402++check("parity: lang_guess is a laya.serve body control",+ "lang_guess" in _serve.BODY_CONTROLS, str(_serve.BODY_CONTROLS))+parsed = parse_value(["t"])+namespace_keys = set(vars(parsed)) if parsed is not None else set()+check("parity: the CLI flag set covers the routing controls serve forwards",+ {"model", "task", "lang", "lang_guess"} <= namespace_keys, str(sorted(namespace_keys)))++# --------------------------------------------------------------------- report+print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))+for f in FAIL:+ print(" FAIL", f)+if not FAIL:+ print("all lang-guess CLI tests passed")+sys.exit(1 if FAIL else 0)