Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(lang): add opt-in detection reasons and source diagnostics

xpeb494474ff69
diff --git a/docs/routing.md b/docs/routing.md
index a9a452f9..ff63e8fd 100644
--- a/docs/routing.md
+++ b/docs/routing.md
@@ -70,6 +70,99 @@ print(decision.reason) # why that checkpoint was selected
`decision["model"]` and `decision["reason"]`. `Router.predict()` includes the same decision under
the result's `"routing"` key.
+## Explain a detection result
+
+Use `explain=True` to inspect why built-in detection reached its result, without loading a
+checkpoint. The existing detection fields and routing rules stay unchanged; only the optional
+`diagnostic` field is added.
+
+```python
+from laya import Router
+from laya.lang import analyse
+
+detection = analyse("Turn lights off", explain=True)
+print(detection["diagnostic"])
+# {"stage": "whole_state", "reason": "insufficient_words", "source": None}
+
+router = Router(default="english")
+decision = router.route("Turn lights off", explain=True)
+assert decision["model"] == "english"
+assert decision["detection"]["language_undecided"] is True
+```
+
+The detector's reason is separate from the router's policy: an unidentified language follows
+the configured `default`, which need not be multilingual. Reasons describe heuristic branches,
+not calibrated language confidence. Explicit model/task/language overrides, an accepted
+`lang_guess`, and automatic workflow selection still skip detection and return
+`detection=None`. As before, an `on_route` hook can replace the whole decision.
+
+For structured input, a later value can change the result even when a long earlier value has
+filled the joined-text window:
+
+```python
+state = {
+ "note": "This is an English note about the order. " * 140,
+ "payload": {"customer": "Ich kann mich nicht anmelden und brauche Hilfe"},
+}
+detection = analyse(state, explain=True)
+assert detection["language"] == "de"
+assert detection["mixed_segment"] is None # its existing meaning is unchanged
+assert detection["diagnostic"] == {
+ "stage": "value",
+ "reason": "named_value",
+ "source": {"leaf_index": 1, "path": ["payload", "customer"], "line": 1},
+}
+```
+
+`stage` is `whole_state` for the initial joined sample, `segment` for the first qualifying
+line in the shared scan window, or `value` for the subsequent per-value fallback. Only the
+selected result is reported; this is not a trace of every candidate.
+An override changes only the fields it changed before: for example, a late non-Latin value can
+make `is_english=False` while `script_profile` still describes the initial Latin sample. The
+diagnostic identifies the override's source; it does not relabel those older measurements.
+
+| Reason | Meaning |
+|---|---|
+| `no_letters` | The initial sample contains no alphabetic characters. |
+| `non_latin_script` | The script check selected a non-Latin script, including mixed-script promotion. |
+| `insufficient_words` | The Latin language guess returned before general scoring because it had fewer than four words. |
+| `no_language_evidence` | General word scoring did not identify a language. |
+| `language_evidence` | Word evidence identified a language; see the existing `language` field. |
+| `short_swedish` | The existing short Swedish phrase rule identified the language. |
+| `caps_veto` | The all-capitals evidence check rejected the language name. |
+| `caps_filtered` | The language name was reconsidered with acronym-shaped capitals removed. |
+| `non_english_markers` | No language was named, but diacritics or a shared Nordic marker still made the text non-English. This takes precedence over an inconclusive word/caps reason. |
+| `named_segment` | A named non-English prose segment overrode the joined result. |
+| `named_value` | The selected fallback line passed the named-language prose check. |
+| `non_latin_value` | The selected fallback line passed the non-Latin word/letter check. |
+| `diacritic_value` | The selected fallback line passed the diacritic and word-count check. |
+
+For `whole_state`, `source` is `None`: the joined sample need not correspond to a single field.
+Other stages report a zero-based `leaf_index` in the existing string-leaf traversal and a
+one-based `line` within that decoded leaf. `path` is an array of mapping keys and sequence
+indices, so keys containing dots or slashes remain unambiguous; a root string has path `[]`.
+The detector's existing depth and character limits still apply. Invalid UTF-8 byte leaves and
+non-text values do not contribute a leaf index.
+
+A path retains string keys of at most 128 characters and integer keys with at most 63 bits of
+magnitude, including negative integers. If a key anywhere on that path is unsupported or exceeds
+those bounds, the entire path is `None`; the leaf index and line remain available. No arbitrary
+key is stringified, and the diagnostic adds no input text. The existing `mixed_segment` can
+still contain text from the segment scan, as it did before.
+
+`Router.route_batch(requests, explain=True)` applies the same option to the whole routing batch,
+preserving order. The CLI exposes it in route-only mode:
+
+```bash
+laya "Turn lights off" --explain --json
+laya --batch tickets.txt --explain --json
+```
+
+The flag also works in interactive mode and ordinary text output. Combining it with
+`--predict`, `--preset` or `--questions` is rejected: diagnostics belong to the routing call,
+not a separate detector pass attached to an inference result.
+
+
## Override language routing
Use `lang=` when the application already knows the request language. Language tags such as
diff --git a/laya/cli.py b/laya/cli.py
index 4b31d5c7..24121be6 100644
--- a/laya/cli.py
+++ b/laya/cli.py
@@ -130,6 +130,8 @@ def build_parser():
" rather than a refusal. Needs --predict, --preset or --questions.")
parser.add_argument("--device", help="torch device, e.g. cpu or cuda")
parser.add_argument("--json", action="store_true", help="print the raw result as JSON")
+ parser.add_argument("--explain", action="store_true",
+ help="include language-detection reasons and sources in routing mode (no model loading)")
parser.add_argument("--batch", metavar="FILE",
help="score a file of requests, one per line (use '-' for stdin), instead "
"of a single text; implies neither --predict nor --preset, but the "
@@ -284,7 +286,8 @@ def run(text, args, router=None):
else:
state = {"text": text}
decision = router.route(state, model=args.model, task=args.task, lang=args.lang,
- lang_guess=args.lang_guess)
+ lang_guess=args.lang_guess,
+ **({"explain": True} if args.explain else {}))
if args.json:
print(json.dumps(dict(decision), ensure_ascii=False, indent=2, default=str))
else:
@@ -407,10 +410,13 @@ def run_batch(lines, args, router=None):
print()
else:
decisions = router.route_batch([{"state": {"text": line}, "questions": {},
- **overrides} for line in lines])
+ **overrides} for line in lines],
+ **({"explain": True} if args.explain else {}))
for line, decision in zip(lines, decisions):
if args.json:
print(json.dumps(dict(decision), ensure_ascii=False, default=str))
+ elif args.explain:
+ show_decision(decision)
else:
print("%-14s %s <- %s" % (decision["model"], decision["reason"],
line if len(line) <= 48 else line[:45] + "..."))
@@ -460,6 +466,10 @@ def main(argv=None):
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
text = " ".join(args.text).strip()
+ if args.explain and (args.predict or args.preset or args.questions):
+ print("laya: --explain is for routing only; omit --predict, --preset and --questions.",
+ file=sys.stderr)
+ return 2
# Routing answers nothing, so there is no confidence to gate: `route` does not take the
# threshold at all. Accepting the flag there would let `laya --min-confidence 0.9 "..."` exit 0
# having ignored it, which is the thing this flag exists to avoid.
diff --git a/laya/lang.py b/laya/lang.py
index 58ea55ea..28a76aa3 100644
--- a/laya/lang.py
+++ b/laya/lang.py
@@ -239,7 +239,8 @@ _WORD = re.compile(r"[^\W\d_]+", re.UNICODE)
_IDENTIFIER = re.compile(r"(?<![\w-])[\w-]*(?:[.@][\w-]+)+", re.UNICODE)
-def _iter_text(state: Union[str, bytes, Mapping, list, None], _depth: int = 0) -> List[str]:
+def _iter_text(state: Union[str, bytes, Mapping, list, None], _depth: int = 0,
+ _paths=None, _path=()) -> List[str]:
"""Collect the string leaves of a state, so detection sees real content.
Keys are ignored: they are usually English field names. Any mapping counts, not only
@@ -249,21 +250,35 @@ def _iter_text(state: Union[str, bytes, Mapping, list, None], _depth: int = 0) -
if _depth > 6 or state is None:
return []
if isinstance(state, str):
+ if _paths is not None:
+ _paths.append(_path)
return [state]
if isinstance(state, (bytes, bytearray)):
try:
- return [bytes(state).decode("utf-8")]
+ text = bytes(state).decode("utf-8")
except UnicodeDecodeError:
return []
+ if _paths is not None:
+ _paths.append(_path)
+ return [text]
if isinstance(state, Mapping):
out = []
- for v in state.values():
- out.extend(_iter_text(v, _depth + 1))
+ if _paths is None:
+ for v in state.values():
+ out.extend(_iter_text(v, _depth + 1))
+ else:
+ for key, v in state.items():
+ # Do not stringify arbitrary keys or copy an unbounded field name into a report.
+ bounded = ((type(key) is str and len(key) <= 128)
+ or (type(key) is int and key.bit_length() <= 63))
+ path = _path + (key,) if _path is not None and bounded else None
+ out.extend(_iter_text(v, _depth + 1, _paths, path))
return out
if isinstance(state, (list, tuple)):
out = []
- for v in state:
- out.extend(_iter_text(v, _depth + 1))
+ for i, v in enumerate(state):
+ path = _path + (i,) if _paths is not None and _path is not None else _path
+ out.extend(_iter_text(v, _depth + 1, _paths, path))
return out
return []
@@ -442,6 +457,10 @@ def latin_profile(text: str) -> Dict[str, object]:
A non-English language is only named when it matched at least one word that no other list
claims: shared function words alone (`la`, `e`, `o`) identify no particular language.
"""
+ return _latin_profile(text)
+
+
+def _latin_profile(text: str, diagnostic=None) -> Dict[str, object]:
# 'İ'.lower() is 'i' + a combining dot, which matches no word list
words = _WORD.findall(_IDENTIFIER.sub(" ", text).replace("İ", "i").lower())
lowered = text.lower()
@@ -450,9 +469,13 @@ def latin_profile(text: str) -> Dict[str, object]:
non_english = diac_rate >= NON_EN_DIACRITIC_RATE
nordic_overlap = bool(set(words) & _NORDIC_OVERLAP_WORDS) and not bool(set(words) & _EN_ONLY_WORDS)
if 1 < len(words) < 4 and set(words) & _SHORT_SWEDISH_WORDS:
+ if diagnostic is not None:
+ diagnostic["reason"] = "short_swedish"
return {"language": "sv", "english_hits": 0, "diacritic_rate": diac_rate,
"looks_non_english": non_english}
if len(words) < 4:
+ if diagnostic is not None:
+ diagnostic["reason"] = "insufficient_words"
return {"language": None, "english_hits": 0, "diacritic_rate": diac_rate,
"looks_non_english": non_english or nordic_overlap}
@@ -486,6 +509,8 @@ def latin_profile(text: str) -> Dict[str, object]:
lang = best_lg
elif en and (not non_english or _english_rescued_by_words(words, diac_rate)):
lang = "en"
+ if diagnostic is not None:
+ diagnostic["reason"] = "language_evidence" if lang else "no_language_evidence"
return {"language": lang, "english_hits": en, "diacritic_rate": diac_rate,
"looks_non_english": non_english or (lang is None and nordic_overlap)}
@@ -639,7 +664,8 @@ def _named_prose_language(segment: str):
return lang
-def _non_english_segment(state: Union[str, bytes, Mapping, list, None], max_chars: int = 4000):
+def _non_english_segment(state: Union[str, bytes, Mapping, list, None], max_chars: int = 4000,
+ diagnostic=None):
"""First line or field that, read on its own, is named a non-English language, else None.
Returns (language, segment). A segment needs the evidence a whole state needs -- at least four
@@ -650,19 +676,28 @@ def _non_english_segment(state: Union[str, bytes, Mapping, list, None], max_char
longer English part from outvoting a foreign one. Reads at most `max_chars` characters in all.
"""
seen = 0
- for leaf in _iter_text(state):
- for seg in leaf.split("\n"):
+ paths = [] if diagnostic is not None else None
+ for index, leaf in enumerate(_iter_text(state, _paths=paths)):
+ for line, seg in enumerate(leaf.split("\n"), 1):
if seen >= max_chars:
return None
seg = seg[:max_chars - seen]
seen += len(seg)
lang = _named_prose_language(seg)
if lang:
+ if diagnostic is not None:
+ diagnostic.update(stage="segment", reason="named_segment",
+ source=_diagnostic_source(paths, index, line))
return lang, seg.strip()
return None
-def _analyse_text(text: str) -> Dict[str, object]:
+def _diagnostic_source(paths, index, line):
+ path = paths[index]
+ return {"leaf_index": index, "path": list(path) if path is not None else None, "line": line}
+
+
+def _analyse_text(text: str, diagnostic=None) -> Dict[str, object]:
"""Detection result for one already-flattened string.
Does not look for a foreign line inside mostly-English text. `analyse` does that, because it
@@ -677,14 +712,18 @@ def _analyse_text(text: str) -> Dict[str, object]:
and _non_latin_carries(non_latin, n_non_latin)):
script = max((s for s in prof if s != "latin"), key=prof.get)
if script == "unknown":
+ if diagnostic is not None:
+ diagnostic["reason"] = "no_letters"
return {"script": "unknown", "script_profile": prof, "language": None,
"is_english": True, "language_undecided": True, "diacritic_rate": 0.0,
"non_latin_fraction": 0.0, "mixed_segment": None}
if script != "latin":
+ if diagnostic is not None:
+ diagnostic["reason"] = "non_latin_script"
return {"script": script, "script_profile": prof, "language": None,
"is_english": False, "language_undecided": True, "diacritic_rate": 0.0,
"non_latin_fraction": non_latin, "mixed_segment": None}
- prof_lat = latin_profile(text)
+ prof_lat = _latin_profile(text, diagnostic)
lang = prof_lat["language"]
# The acronym bar belongs here too, not only in the segment scan. The scan runs only once a
# state already reads English overall, so a state that is *nothing but* an acronym line --
@@ -707,6 +746,8 @@ def _analyse_text(text: str) -> Dict[str, object]:
if not _shouted_evidence(_WORD.findall(text), lang,
float(prof_lat["diacritic_rate"])):
lang = None
+ if diagnostic is not None:
+ diagnostic["reason"] = "caps_veto"
else:
# Mixed-case text: the segment scan has always held that an acronym is not a word,
# but the whole-state verdict never applied that rule, so the acronyms voted in it.
@@ -734,19 +775,23 @@ def _analyse_text(text: str) -> Dict[str, object]:
lambda m: " " if m.group().isupper() else m.group(), text)
if blanked != text:
lang = latin_profile(blanked)["language"]
+ if diagnostic is not None:
+ diagnostic["reason"] = "caps_filtered"
# Undecided is not English. Treating it as English sent every Latin-script language we hold no
# stopwords for to the checkpoint that cannot read it, silently. When nothing identifies the
# language, non-English letters or a shared Swedish-Danish marker can still prefer the
# multilingual checkpoint; text with neither signal still goes to the English one.
undecided = lang is None
english = lang == "en" or (undecided and not prof_lat["looks_non_english"])
+ if diagnostic is not None and undecided and not english:
+ diagnostic["reason"] = "non_english_markers"
return {"script": "latin", "script_profile": prof, "language": lang,
"is_english": english, "language_undecided": undecided,
"diacritic_rate": round(float(prof_lat["diacritic_rate"]), 4),
"non_latin_fraction": non_latin, "mixed_segment": None}
-def _leaf_non_english(leaf: str) -> Optional[Dict[str, object]]:
+def _leaf_non_english(leaf: str, diagnostic=None) -> Optional[Dict[str, object]]:
"""A string value that is itself not safe for the English checkpoint, else None.
A one-word name ("José") and a capitalised non-Latin name stay out: the same rules
@@ -758,7 +803,7 @@ def _leaf_non_english(leaf: str) -> Optional[Dict[str, object]]:
"""
best_n = -1
best: Optional[Dict[str, object]] = None
- for line in leaf.split("\n"):
+ for line_number, line in enumerate(leaf.split("\n"), 1):
# A line this short cannot be selected, so skip it before the slice, `strip`,
# `_CODE_LINE` and a full `_analyse_text` pass. Each branch below needs four `_WORD`
# tokens or NON_LATIN_MIN_LETTERS letters. `_WORD` matches maximal runs of letters and
@@ -802,10 +847,13 @@ def _leaf_non_english(leaf: str) -> Optional[Dict[str, object]]:
if n_alpha > best_n:
best_n = n_alpha
best = det
+ if diagnostic is not None:
+ diagnostic.update(line=line_number, reason=("named_value" if named else
+ "non_latin_value" if det["script"] not in ("latin", "unknown") else "diacritic_value"))
return best
-def analyse(state: Union[str, bytes, Mapping, list, None]) -> Dict[str, object]:
+def analyse(state: Union[str, bytes, Mapping, list, None], *, explain: bool = False) -> Dict[str, object]:
"""Full detection result for a state.
Returns `script`, `script_profile`, `language` (best effort, may be None),
@@ -819,8 +867,18 @@ def analyse(state: Union[str, bytes, Mapping, list, None]) -> Dict[str, object]:
or outvote a short German message, and that message was then sent to the English checkpoint
(#384). The segment scan still stops at 4000 characters, which is what keeps a huge field
cheap; a value it did not reach is read on its own afterwards.
+
+ With `explain=True`, adds `diagnostic`: the selected analysis stage, its reason and, for a
+ segment or value override, the source leaf index, path and 1-based line. Paths retain string
+ keys (up to 128 characters), integers (up to 63 bits) and sequence indices. Other keys make
+ the path None; the leaf index remains available. No input text is added to the diagnostic.
+ The original fields and detection rules are unchanged. These are heuristic reasons, not
+ calibrated confidence or the router's final model choice.
"""
- result = _analyse_text(state_text(state))
+ diagnostic = {"stage": "whole_state", "reason": None, "source": None} if explain else None
+ result = _analyse_text(state_text(state), diagnostic)
+ if diagnostic is not None:
+ result["diagnostic"] = diagnostic
if result["script"] == "latin" and result["is_english"]:
# A Portuguese ticket with an English stack trace, error payload or form template reads as
# English as a whole, because the English part is longer -- yet the part a question is about
@@ -831,7 +889,7 @@ def analyse(state: Union[str, bytes, Mapping, list, None]) -> Dict[str, object]:
leaves = _iter_text(state)
# a single line has no other part to be outvoted by, and was just read whole
if len(leaves) > 1 or any("\n" in leaf for leaf in leaves):
- found = _non_english_segment(state)
+ found = _non_english_segment(state, diagnostic=diagnostic)
if found:
lang, mixed = found
result = dict(result)
@@ -845,20 +903,29 @@ def analyse(state: Union[str, bytes, Mapping, list, None]) -> Dict[str, object]:
return result
best_n = -1
best: Optional[Dict[str, object]] = None
- for leaf in _iter_text(state):
- det = _leaf_non_english(leaf)
+ paths = [] if explain else None
+ best_source = None
+ best_reason = None
+ for index, leaf in enumerate(_iter_text(state, _paths=paths)):
+ leaf_diagnostic = {} if explain else None
+ det = _leaf_non_english(leaf, leaf_diagnostic)
if det is None:
continue
n_alpha = sum(ch.isalpha() for ch in leaf[:4000])
if n_alpha > best_n:
best_n = n_alpha
best = det
+ if explain:
+ best_source = _diagnostic_source(paths, index, leaf_diagnostic["line"])
+ best_reason = leaf_diagnostic["reason"]
if best is None:
return result
updated = dict(result)
updated["language"] = best["language"]
updated["is_english"] = False
updated["language_undecided"] = best["language_undecided"]
+ if diagnostic is not None:
+ diagnostic.update(stage="value", reason=best_reason, source=best_source)
return updated
diff --git a/laya/router.py b/laya/router.py
index 4113ba3f..f538d589 100644
--- a/laya/router.py
+++ b/laya/router.py
@@ -1183,16 +1183,22 @@ class Router(HookRegistry):
hooks=None,
hooks_raise: Optional[bool] = None,
hooks_timeout: Optional[float] = None,
+ *,
+ explain: bool = False,
) -> RouteDecision:
"""Decide which checkpoint to use, then let `on_route` hooks observe or replace it.
`ctx.decision` is the `RouteDecision`; a hook may replace it (for example to pin a
checkpoint) and the replacement is what gets returned and used. `hooks` are per-call
hooks, appended after any installed on the Router.
+
+ `explain=True` adds language diagnostics when built-in detection runs. Explicit routing
+ overrides still skip it. A hook can replace the decision, including its diagnostics.
"""
if questions is not None and not isinstance(questions, dict):
raise TypeError("questions must be a dict of question id -> definition, got %s" % type(questions).__name__)
- decision = self._route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess)
+ decision = self._route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess,
+ **({"explain": True} if explain else {}))
raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)
active = compose_hooks(self.hooks, hooks)
timeout = self.hooks_timeout if hooks_timeout is None else validate_timeout(hooks_timeout)
@@ -1208,6 +1214,8 @@ class Router(HookRegistry):
task: Optional[str] = None,
lang: Optional[str] = None,
lang_guess: Optional[Any] = None,
+ *,
+ explain: bool = False,
) -> RouteDecision:
"""Decide which checkpoint to use, without loading or running anything.
@@ -1261,7 +1269,7 @@ class Router(HookRegistry):
source, "English" if resolved else "non-English"),
detection=None, workflow=workflow)
- det = analyse(state)
+ det = analyse(state, explain=True) if explain else analyse(state)
if det["script"] == "unknown":
key = self.default
reason = "no letters detected in state; using default (%s)" % key
@@ -1534,6 +1542,7 @@ class Router(HookRegistry):
*,
hooks=None,
hooks_raise: Optional[bool] = None,
+ explain: bool = False,
) -> List[RouteDecision]:
"""Route a heterogeneous request batch without loading any checkpoints.
@@ -1551,6 +1560,7 @@ class Router(HookRegistry):
every request's ``on_route`` dispatch, as on :meth:`route`.
hooks (HookArg): Per-call hook or sequence of hooks for this call.
hooks_raise: Override the Router's ``hooks_raise`` policy for this call.
+ explain: Include built-in language diagnostics for each request that uses detection.
"""
if not isinstance(requests, SequenceABC) or isinstance(requests, (str, bytes)):
raise TypeError("requests must be a sequence of request dictionaries")
@@ -1584,6 +1594,7 @@ class Router(HookRegistry):
hooks=hooks,
hooks_raise=hooks_raise,
hooks_timeout=hooks_timeout,
+ **({"explain": True} if explain else {}),
)
)
diff --git a/scripts/test_suites.py b/scripts/test_suites.py
index a5a6859a..a720e352 100644
--- a/scripts/test_suites.py
+++ b/scripts/test_suites.py
@@ -114,6 +114,7 @@ SCRIPT_SUITES = [
]
PYTEST_SUITES = [
+ "tests/test_lang_diagnostics.py",
"tests/test_evals_agreement.py",
"tests/test_evals_tournament.py",
"tests/test_serve.py",
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index 1a5840d9..3f03f282 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -243,6 +243,19 @@ check("predict_tournament/positional prefix",
check_param("predict_tournament", laya.predict_tournament, "predict_kwargs", inspect.Parameter.empty,
inspect.Parameter.VAR_KEYWORD)
+# Detection diagnostics are opt-in and do not replace existing analysis fields.
+from laya.lang import analyse, latin_profile # noqa: E402
+
+for label, fn in (("analyse", analyse), ("Router.route", Router.route),
+ ("Router.route_batch", Router.route_batch)):
+ check_param(label, fn, "explain", False, inspect.Parameter.KEYWORD_ONLY)
+check("latin_profile keeps its signature", list(sig(latin_profile)), ["text"])
+_plain_analysis = analyse("Turn lights off")
+_explained_analysis = analyse("Turn lights off", explain=True)
+check("analyse opts into one extra key", set(_explained_analysis) - set(_plain_analysis), {"diagnostic"})
+check("analyse diagnostic fields", set(_explained_analysis.pop("diagnostic")), {"stage", "reason", "source"})
+check("analyse retains original fields", _explained_analysis, _plain_analysis)
+
# route() and route_batch() take per-call hooks so a hook can pin a checkpoint for one call
for label, fn in (("Router.route", Router.route),
("Router.route_batch", Router.route_batch)):
diff --git a/tests/test_lang_diagnostics.py b/tests/test_lang_diagnostics.py
new file mode 100644
index 00000000..d7a9b4b3
--- /dev/null
+++ b/tests/test_lang_diagnostics.py
@@ -0,0 +1,258 @@
+"""Opt-in diagnostics explain the executed detector, without changing its decisions."""
+import copy
+import json
+import os
+from pathlib import Path
+import subprocess
+import sys
+from collections import UserDict
+from concurrent.futures import ThreadPoolExecutor
+from types import MappingProxyType
+from unittest.mock import patch
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+from laya import Router, BaseHook # noqa: E402
+from laya import lang # noqa: E402
+
+EN = "This is an English note about the order. "
+DE = "Ich kann mich nicht anmelden und brauche Hilfe"
+PT = "Quero cancelar a minha conta porque não consigo entrar"
+
+
+def explained(state):
+ plain = lang.analyse(state)
+ result = lang.analyse(state, explain=True)
+ diagnostic = result.pop("diagnostic")
+ assert result == plain
+ assert set(diagnostic) == {"stage", "reason", "source"}
+ json.dumps(diagnostic, ensure_ascii=False, allow_nan=False)
+ return diagnostic
+
+
[email protected](("state", "reason"), [
+ ("Turn lights off", "insufficient_words"),
+ ("xyzzy frobnitz quux blorb", "no_language_evidence"),
+ ("I would like to book a flight to Berlin tomorrow", "language_evidence"),
+ (DE, "language_evidence"),
+ ("glömt lösenord", "short_swedish"),
+ ("MON DES EST LA", "caps_veto"),
+ ("This is my ticket. MON DES EST LA", "caps_filtered"),
+ ("åå øø äa", "non_english_markers"),
+ ("请帮我取消订单", "non_latin_script"),
+ ("1234", "no_letters"),
+ ("", "no_letters"),
+ (None, "no_letters"),
+ (b"\xff", "no_letters"),
+])
+def test_whole_state_reason(state, reason):
+ assert explained(state) == {"stage": "whole_state", "reason": reason, "source": None}
+
+
+def test_original_fields_stay_fixed():
+ assert lang.analyse("Turn lights off") == {
+ "script": "latin", "script_profile": {"latin": 1.0}, "language": None,
+ "is_english": True, "language_undecided": True, "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0, "mixed_segment": None,
+ }
+ assert set(lang.latin_profile("Turn lights off")) == {
+ "language", "english_hits", "diacritic_rate", "looks_non_english",
+ }
+
+
+def test_segment_keeps_its_original_text_and_identifies_the_first_matching_line():
+ state = {"note": EN * 20, "messages": [{"customer": "status\n" + DE + "\n" + PT}]}
+ d = explained(state)
+ assert d == {"stage": "segment", "reason": "named_segment",
+ "source": {"leaf_index": 1, "path": ["messages", 0, "customer"], "line": 2}}
+ assert lang.analyse(state)["mixed_segment"] == DE
+ # Equal text in another field must not replace the source by searching the state afterwards.
+ state["duplicate"] = DE
+ assert explained(state)["source"] == d["source"]
+
+
+def test_root_string_source_is_a_line_not_a_fabricated_field():
+ d = explained(EN * 20 + "\n" + DE)
+ assert d["stage"] == "segment"
+ assert d["source"] == {"leaf_index": 0, "path": [], "line": 2}
+
+
[email protected]("wrap", [dict, UserDict, MappingProxyType])
+def test_late_value_records_nested_path_without_changing_mixed_segment(wrap):
+ state = wrap({"note": EN * 140, "messages": ({"customer": "status\n" + DE},)})
+ d = explained(state)
+ assert d == {"stage": "value", "reason": "named_value",
+ "source": {"leaf_index": 1, "path": ["messages", 0, "customer"], "line": 2}}
+ assert lang.analyse(state)["mixed_segment"] is None
+ assert DE not in json.dumps(d)
+
+
[email protected](("message", "reason"), [
+ ("请帮我取消这个订单并且退款给我", "non_latin_value"),
+ ("WIE SPÄT IST ES IN KÖLN", "diacritic_value"),
+])
+def test_value_eligibility_reason(message, reason):
+ d = explained({"note": EN * 140, "customer": message})
+ assert d["stage"] == "value"
+ assert d["reason"] == reason
+ assert d["source"]["path"] == ["customer"]
+ assert lang.analyse({"note": EN * 140, "customer": message})["script_profile"] == {"latin": 1.0}
+
+
+def test_value_selection_keeps_both_existing_winner_rules():
+ # Inner scan chooses the longest qualifying line; outer scan weighs the entire leaf window.
+ state = {"note": EN * 140, "short": DE,
+ "long": "status\n" + DE + "\n" + PT * 3}
+ d = explained(state)
+ assert d["source"] == {"leaf_index": 2, "path": ["long"], "line": 3}
+ tie = {"note": EN * 140, "first": DE, "second": DE}
+ assert explained(tie)["source"]["path"] == ["first"]
+ lines_tie = {"note": EN * 140, "first": DE + "\n" + DE}
+ assert explained(lines_tie)["source"]["line"] == 1
+
+
+class UnprintableKey:
+ def __repr__(self):
+ raise AssertionError("diagnostics must not stringify arbitrary keys")
+
+
[email protected]("key", [UnprintableKey(), ("tuple",), "k" * 129, 2 ** 80, True])
+def test_unrepresentable_path_retains_leaf_index(key):
+ d = explained({"note": EN * 140, key: {"customer": DE}})
+ assert d["source"] == {"leaf_index": 1, "path": None, "line": 1}
+
+
[email protected]("key", ["k" * 128, -10, "a/b~c\nfield"])
+def test_bounded_keys_are_preserved_as_path_components(key):
+ d = explained({"note": EN * 140, key: DE})
+ assert d["source"]["path"] == [key]
+
+
+def test_decode_depth_and_window_limits_are_unchanged():
+ d = explained({"note": EN * 140, "bad": b"\xff", "good": DE.encode()})
+ assert d["source"] == {"leaf_index": 1, "path": ["good"], "line": 1}
+ at_limit = DE
+ for _ in range(5):
+ at_limit = [at_limit]
+ d = explained({"note": EN * 140, "deep": at_limit})
+ assert d["source"]["path"] == ["deep", 0, 0, 0, 0, 0]
+ beyond = {"note": EN * 140, "deep": [at_limit]}
+ assert explained(beyond)["source"] is None
+ # The fallback reads each line's first 4000 characters, not an unbounded field.
+ assert explained({"note": EN * 140, "hidden": EN * 140 + DE})["source"] is None
+
+
+def test_diagnostics_are_per_call_and_do_not_mutate_state():
+ states = [{"note": EN * 140, "customer": DE}, "Turn lights off", "MON DES EST LA"]
+ original = copy.deepcopy(states)
+ expected = [lang.analyse(s, explain=True) for s in states]
+ with ThreadPoolExecutor(max_workers=4) as executor:
+ actual = list(executor.map(lambda s: lang.analyse(s, explain=True), states * 8))
+ assert actual == expected * 8
+ assert states == original
+ actual[0]["diagnostic"]["source"]["path"].append("changed")
+ assert lang.analyse(states[0], explain=True) == expected[0]
+
+
+def test_explain_does_not_run_a_second_detection_pass():
+ state = {"note": EN * 140, "customer": DE}
+ with patch.object(lang, "_analyse_text", wraps=lang._analyse_text) as detector:
+ lang.analyse(state)
+ plain_calls = detector.call_args_list
+ with patch.object(lang, "_analyse_text", wraps=lang._analyse_text) as detector:
+ lang.analyse(state, explain=True)
+ assert [c.args[0] for c in detector.call_args_list] == [c.args[0] for c in plain_calls]
+
+
[email protected]("default", ["english", "multilingual"])
+def test_router_keeps_fallback_and_never_loads_models(default):
+ router = Router(default=default)
+ with patch.object(router, "load", side_effect=AssertionError("must not load")):
+ plain = router.route("Turn lights off")
+ result = router.route("Turn lights off", explain=True)
+ assert result["model"] == default
+ assert result["detection"].pop("diagnostic")["reason"] == "insufficient_words"
+ assert result == plain
+
+
[email protected]("override", [
+ {"model": "english"}, {"task": "typed-decisions"}, {"lang": "de"}, {"lang_guess": "en"},
+])
+def test_explicit_routes_do_not_invent_detector_evidence(override):
+ router = Router()
+ with patch("laya.router.analyse", side_effect=AssertionError("detector must be skipped")):
+ result = router.route(DE, explain=True, **override)
+ assert result["detection"] is None
+
+
+def test_abstaining_hint_runs_detection_once():
+ calls = []
+ def hint(state):
+ calls.append(state)
+ return None
+ result = Router(lang_guess=hint).route("Turn lights off", explain=True)
+ assert calls == ["Turn lights off"]
+ assert result["detection"]["diagnostic"]["reason"] == "insufficient_words"
+
+
+def test_automatic_workflow_selection_skips_detection():
+ questions = {key: {"type": "bool"} for key in
+ ("action", "category", "churn_risk", "needs_human", "urgency")}
+ with patch("laya.router.analyse", side_effect=AssertionError("detector must be skipped")):
+ result = Router(auto_task_detection=True).route(DE, questions, explain=True)
+ assert result["workflow"] == "customer_service"
+ assert result["detection"] is None
+
+
+def test_hooks_observe_and_may_replace_the_actual_decision():
+ seen = []
+ class Hook(BaseHook):
+ def on_route(self, ctx):
+ seen.append(ctx.decision["detection"]["diagnostic"]["reason"])
+ ctx.decision = {"model": "custom", "reason": "hook"}
+ assert Router().route("Turn lights off", explain=True, hooks=Hook()) == {
+ "model": "custom", "reason": "hook"}
+ assert seen == ["insufficient_words"]
+
+
+def test_batch_uses_same_explanation_and_order():
+ states = ["Turn lights off", {"note": EN * 140, "customer": DE}]
+ router = Router()
+ batch = router.route_batch([{"state": s, "questions": {}} for s in states], explain=True)
+ assert batch == [router.route(s, explain=True) for s in states]
+ assert all("diagnostic" not in r["detection"] for r in router.route_batch(
+ [{"state": s, "questions": {}} for s in states]))
+
+
+def cli(*args, stdin=None):
+ return subprocess.run([sys.executable, "-m", "laya.cli", *args],
+ cwd=Path(__file__).resolve().parents[1], input=stdin, text=True, encoding="utf-8",
+ capture_output=True, timeout=45,
+ env={**os.environ, "PYTHONUTF8": "1", "HF_HUB_OFFLINE": "1", "TRANSFORMERS_OFFLINE": "1"})
+
+
+def test_cli_real_process_json_text_batch_and_interactive():
+ result = cli("Turn lights off", "--explain", "--json")
+ assert result.returncode == 0, result.stderr
+ assert json.loads(result.stdout)["detection"]["diagnostic"]["reason"] == "insufficient_words"
+ result = cli("Turn lights off", "--explain")
+ assert result.returncode == 0, result.stderr
+ assert "Detected" in result.stdout and "insufficient_words" in result.stdout
+ result = cli("--batch", "-", "--explain", "--json", stdin="Turn lights off\n" + DE + "\n")
+ assert result.returncode == 0, result.stderr
+ assert len([json.loads(line)["detection"]["diagnostic"] for line in result.stdout.splitlines()]) == 2
+ result = cli("--batch", "-", "--explain", stdin="Turn lights off\n")
+ assert result.returncode == 0 and "insufficient_words" in result.stdout
+ result = cli("--explain", stdin="Turn lights off\nquit\n")
+ assert result.returncode == 0 and "insufficient_words" in result.stdout
+
+
[email protected]("args", [("--predict",), ("--preset", "email"), ("--questions", "absent.json")])
+def test_cli_rejects_prediction_modes_before_any_loading(args):
+ result = cli("Turn lights off", "--explain", *args)
+ assert result.returncode == 2
+ assert "--explain is for routing only" in result.stderr
+ assert result.stdout == ""
+