Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(ts): match accented words in latinProfile like Python (WORD_RE was ASCII-only)
divit aggarwalf48080594131
diff --git a/laya-ts/src/lang.ts b/laya-ts/src/lang.tsindex 90f54b65..7d46ff58 100644--- a/laya-ts/src/lang.ts+++ b/laya-ts/src/lang.ts@@ -123,7 +123,8 @@ const SHARED_WORDS: Set<string> = (() => {return out;})();-const WORD_RE = /[^\W\d_]+/gu;+// JS `\w` is ASCII-only, so Python's `[^\W\d_]` needs the Unicode classes spelled out (Nl/No are in Python's `\w`).+const WORD_RE = /[\p{L}\p{Nl}\p{No}]+/gu;// Lookbehind for the same reason as the Python side (see laya/lang.py): without it the// greedy prefix is retried at every offset inside a run of word characters, which is// quadratic in the run's length -- 50 000 characters of one token took 1540 ms here.diff --git a/laya-ts/tests/lang.test.ts b/laya-ts/tests/lang.test.tsindex 120018fe..5a1f8653 100644--- a/laya-ts/tests/lang.test.ts+++ b/laya-ts/tests/lang.test.ts@@ -69,4 +69,15 @@ describe("lang", () => {expect(analyse("que hora es en australia").language).toBe("es");expect(analyse("baisse le volume du haut-parleur").language).toBe("fr");});+ it.each<[string, string | null, boolean]>([+ ["Preciso do contrato confidencial assinado até sexta.", "pt", false],+ ["Gătește-mi o rețetă de sarmale de post pentru mâine.", "ro", false],+ ["Müşteriden iki kez ücret alındı ve para iadesi istiyor", null, false],+ ["Khách hàng đã bị thu phí hai lần và muốn được hoàn tiền ngay", null, false],+ ])("counts accented words whole, like Python: %s", (text, language, english) => {+ const a = analyse(text);+ expect(a.language).toBe(language);+ expect(a.isEnglish).toBe(english);+ expect(a.languageUndecided).toBe(language === null);+ });});