Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
Merge PR #1446
Nandakishorb939072879b4
diff --cc laya-ts/src/lang.tsindex 14576726,f4f5c53e..d60e5f9a--- a/laya-ts/src/lang.ts+++ b/laya-ts/src/lang.ts@@@ -213,19 -213,19 +213,32 @@@ function scriptOf(ch: string): stringreturn null;}+/** Python's `len()`: the number of code points, not of UTF-16 units. Python's `lang.py` divides+ * the diacritic count by `len(lowered)`, and every other port counts code points too (Go+ * `utf8.RuneCountInString`, Rust `chars().count()`, Java `codePointLength`, C++ `RuneCount`), so+ * an astral character must count as one here as well. `String.length` counts it as two, which+ * understated `diacriticRate` on any text carrying emoji -- e.g. `analyse()` on+ * `"é\uD83D\uDE00\uD83D\uDE00" + "-".repeat(46)` read `isEnglish: true` in this port where+ * Python answers `false`, routing the text to the English checkpoint. */+function width(text: string): number {+ let count = 0;+ for (const _ch of text) count++;+ return count;+}++ /** Python's `text[:count]`: a prefix cut at code points, so a surrogate pair is never split+ * in half. Python's `lang.py` slices the same budgets by code points (`state_text`,+ * `_non_english_segment`, the 4000-character leaf samples), and the email port already cuts this+ * way (`cutCodePoints` in `email.ts`, added when a budget cut an emoji in half). `String.slice`+ * counts UTF-16 units, so `stateText("A".repeat(3999) + "\uD83D\uDE00" + "...")` used to return a+ * string ending in the orphaned high half of the emoji -- text Python never produces, and a lone+ * surrogate the serve side already refuses as malformed (#454, #814). */+ function cutCodePoints(text: string, count: number): string {+ const chars = Array.from(text);+ const end = count < 0 ? Math.max(0, chars.length + count) : Math.min(count, chars.length);+ return chars.slice(0, end).join("");+ }+/** String leaves of a state. Keys are ignored: they are usually English field names. */function iterText(state: unknown, depth = 0): string[] {if (depth > 6 || state == null) return [];