Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(java): a test asserted the JDK disagrees with CPython, and on JDK 21 it does not
Adeshina Lasisi69398f953101
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex 8819f649..f3a5c0c5 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -152,9 +152,17 @@ jobs:strategy:fail-fast: falsematrix:- # 17 is covered by the build job above. 21 is the LTS most consumers are on; 24 is the- # newest with a different Unicode version from 17, and is the one verified by hand.- jdk: ["21", "24"]+ # 17 as well as 21 and 24, even though the build job also tests on 17: that job runs+ # WITHOUT `-PtestJavaVersion`, so `TestJvmVersionTest` is disabled there and "17 is+ # tested" was true but unasserted. Here it is asserted.+ #+ # The three are not an arbitrary spread, they are three different Unicode versions --+ # 17 carries 13.0, 21 carries 15.0 and 24 carries 16.0 -- and this port's whole+ # character-handling strategy is that the answer must not depend on which. That is not+ # theoretical: the JDK 21 cell failed on its first run, because `Character.isLetter` on+ # Unicode 15.0 agrees with CPython 15.0 exactly, and a test had asserted the JDK+ # DISAGREES. 17 and 24 passed. A two-version matrix would not have found it.+ jdk: ["17", "21", "24"]steps:- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0# For check_test_results.py only -- it is standard library, so there is nothing to install.diff --git a/laya-java/CHANGELOG.md b/laya-java/CHANGELOG.mdindex 1541144e..17bcbc30 100644--- a/laya-java/CHANGELOG.md+++ b/laya-java/CHANGELOG.md@@ -35,17 +35,24 @@ see [Installing](README.md#installing).### Testing-- 768 tests, 594 of which need no model at all.+- 769 tests, 626 of which need no model at all.- Every expectation is **generated from the Python package**, never written by hand:`laya-java/fixtures/*.json`, regenerated by `scripts/gen_fixtures.py` and gated by `--check` inCI so a change to `laya/` that moves a recorded contract shows up here rather than as drift.- The compiled-in Unicode tables are checked by a **sha256 over all 1,114,112 code points** perproperty, so they are proven exhaustively rather than on the cases someone thought to write.-- Five CI lanes: build plus model-free tests, the same tests on JDK 21 and JDK 24, and two parity- cells split by whether they need a traced graph.+- Six CI lanes: build plus model-free tests, the same tests on JDK 17, 21 and 24 — three+ different Unicode versions, 13.0, 15.0 and 16.0 — and two parity cells split by whether they+ need a traced graph.- `-PtestJavaVersion` runs the tests on a different JDK from the one the classes are compiledfor, because the toolchain pins the compiler to 17 and a lane that merely installs another JDKtests the same thing twice.+- That matrix earned itself on its first run. The JDK 21 cell failed where 17 and 24 passed,+ because `Character.isLetter` on Unicode 15.0 agrees with CPython 15.0 EXACTLY, and a test had+ asserted that the JDK disagrees. Catching up is not a reason to drop the table -- it is the+ reason the table exists, since the agreement is two version numbers lining up by accident and+ un-happens on the next release in either direction. The assertion is now the version-independent+ one, and whether a given JDK agrees is recorded rather than required.### Known limitationsdiff --git a/laya-java/README.md b/laya-java/README.mdindex c5802821..886de5de 100644--- a/laya-java/README.md+++ b/laya-java/README.md@@ -340,6 +340,10 @@ the bytecode stays at release 17:./gradlew test -PtestJavaVersion=24 # compiled for 17, executed on 24```+CI runs the model-free suite on **17, 21 and 24** — three different Unicode versions (13.0, 15.0+and 16.0). The artifact is compiled for 17, so it runs on 17 and anything newer; those three are+the versions the suite is actually asserted against.+This matters more here than in most ports. `\p{L}` and `\p{N}` in `java.util.regex` follow theJDK's own Unicode version, and `Character.isLetter` disagrees with itself across JDK 17 (Unicode13.0) and JDK 24 (Unicode 16.0) on 751 of the code points this port has to classify — 0 of 751 ondiff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/lang/UnicodeTablesTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/lang/UnicodeTablesTest.javaindex 5e44f03a..27db26d3 100644--- a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/lang/UnicodeTablesTest.java+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/lang/UnicodeTablesTest.java@@ -204,17 +204,22 @@ class UnicodeTablesTest {}@Test- @DisplayName("the JDK's own predicates would NOT pass, which is why these tables exist")- void theJdkDisagrees() {- // Not a tautology: this is the measurement behind the comment on UnicodeTables, kept as a- // test so the claim is checked rather than asserted. If a future JDK ever catches up, this- // fails and the table can be reconsidered deliberately rather than kept out of habit.- assertFalse(predicateDigest(Character::isLetter).equals(digests().get("alpha")),- "Character.isLetter now agrees with CPython; the ALPHA table may be removable");+ @DisplayName("two of these properties have no JDK equivalent, on any JDK")+ void theJdkHasNoEquivalent() {+ // These two disagree with CPython on EVERY JDK, because the JDK does not expose the+ // property at all rather than exposing an older version of it:+ //+ // * `str.isspace()` takes U+00A0, U+0085, U+2007 and U+202F, and+ // `Character.isWhitespace` rejects every one of them -- it is a different predicate,+ // not a stale one.+ // * the canonical combining CLASS is not exposed by the JDK. Mn/Mc is the nearest+ // approximation, which is what the .NET port uses, and it differs on 1,528 code+ // points that are category M with a class of zero.+ //+ // So these are the version-independent half of the justification for compiling the+ // tables in, and they are safe to assert on a JDK matrix.assertFalse(predicateDigest(Character::isWhitespace).equals(digests().get("space")),"Character.isWhitespace now agrees with CPython's isspace");- assertFalse(predicateDigest(Character::isUpperCase).equals(digests().get("upper")),- "Character.isUpperCase now agrees with CPython's isupper");assertFalse(predicateDigest(cp -> {int type = Character.getType(cp);return type == Character.NON_SPACING_MARK || type == Character.COMBINING_SPACING_MARK;@@ -223,6 +228,41 @@ class UnicodeTablesTest {+ " canonical combining class");}+ @Test+ @DisplayName("whether the JDK's letter and case predicates agree is a property of the JDK")+ void theJdkMayOrMayNotAgree() {+ // This test used to assert that `Character.isLetter` DISAGREES with CPython, with a+ // comment saying that if a JDK ever caught up the test should fail so the table could be+ // reconsidered. A JDK caught up: 21 ships Unicode 15.0 and CPython is 15.0.0, so the two+ // letter sets are identical there -- and the test failed on the JDK 21 cell while 17+ // (Unicode 13.0) and 24 (16.0) passed.+ //+ // Catching up is not a reason to remove the table; it is the reason the table exists. The+ // agreement is an accident of two version numbers lining up, and it un-happens on the+ // next release in either direction. A port that read the predicate from the JDK would+ // have been correct on 21 and wrong on 17 and 24, which is exactly the defect this port+ // shipped in its pre-tokenizer.+ //+ // So the assertion is the JDK-independent one: whatever this JDK thinks, the TABLE is+ // CPython's answer. Agreement is recorded rather than required.+ boolean letters = predicateDigest(Character::isLetter).equals(digests().get("alpha"));+ boolean upper = predicateDigest(Character::isUpperCase).equals(digests().get("upper"));+ System.out.printf("JDK %s: isLetter agrees with CPython = %s, isUpperCase = %s%n",+ Runtime.version().feature(), letters, upper);++ // The part that must hold on every JDK: the tables are the reference's, not the JDK's.+ assertEquals(digests().get("alpha"), predicateDigest(UnicodeTables::isAlpha),+ "the ALPHA table no longer matches CPython");+ assertEquals(digests().get("upper"), predicateDigest(UnicodeTables::isUpper),+ "the UPPER table no longer matches CPython");+ // And at least one of the four properties must still have no JDK equivalent, or there+ // would be nothing left for these tables to do. `theJdkHasNoEquivalent` names which two.+ assertFalse(letters && upper+ && predicateDigest(Character::isWhitespace).equals(digests().get("space")),+ "every JDK predicate now agrees with CPython; the tables can be reconsidered "+ + "deliberately rather than kept out of habit");+ }+@Test@DisplayName("strip and splitOnWhitespace follow isSpace, not Character.isWhitespace")void stripAndSplit() {