Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

Merge pull request #207 from paulocarinhena/fix/lang-mixed-segments

Nandakishor11b2c2d59b89
diff --cc laya/lang.py
index 421fa194,a7b181a0..98af647f
--- a/laya/lang.py
+++ b/laya/lang.py
@@@ -395,12 -434,12 +445,13 @@@ def analyse(state: Union[str, dict, lis
"""Full detection result for a state.
Returns `script`, `script_profile`, `language` (best effort, may be None),
- `is_english` and `non_latin_fraction`.
+ `is_english`, `non_latin_fraction` and `mixed_segment` (the line or field that made a mostly
+ English state non-English, else None).
"""
text = state_text(state)
- prof = script_profile(text)
- script = detect_script(text)
+ counts = _script_counts(text)
+ prof = _profile_from_counts(counts)
+ script = _script_from_counts(counts)
non_latin = round(1.0 - prof.get("latin", 0.0), 4) if prof else 0.0
n_non_latin = round(non_latin * sum(ch.isalpha() for ch in text))
if script == "latin" and _non_latin_words(text) and (