Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(java): run the router's forward pass without default hooks, and pin what review found
Adeshina Lasisi7a7916811ef9
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex d01263dc..c85eb5c2 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -154,17 +154,17 @@ jobs:# Without LAYA_CHECKPOINTS the checkpoint-backed factories abort by assumption and the rest# run. `--min-tests` is what stops an aborted suite from looking like a pass.#- # Measured on the merged tree: 1021 register here, 1003 run, 18 abort on a checkpoint this+ # Measured on the merged tree: 1025 register here, 1007 run, 18 abort on a checkpoint this# lane does not have. Exact, not padded -- the floor stood at 680 long enough for the real# figure to pass it by, which is how a floor stops being one. ONE below the JDK-matrix# lane's floor on purpose: that lane passes `-PtestJavaVersion`, which is what makes- # `TestJvmVersionTest` run instead of skip, so it executes 1004. Same suite, one more test,+ # `TestJvmVersionTest` run instead of skip, so it executes 1008. Same suite, one more test,# because the flag enables it. The step below is where the number is justified.- name: Tests that need no modelworking-directory: laya-javarun: ./gradlew --no-daemon test- # Measured on the merged tree, Corretto 17, no checkpoints: 1021 registered, 18 abort,- # 1003 RUN -- the floor counts what ran, so the aborted checkpoint-backed cases are not in+ # Measured on the merged tree, Corretto 17, no checkpoints: 1025 registered, 18 abort,+ # 1007 RUN -- the floor counts what ran, so the aborted checkpoint-backed cases are not in# it. The floor is EXACT, with no margin, which is a change from the margin-of-4 this# replaced. A margin only binds classes larger than itself, and three classes in this lane# run exactly ONE test (`DecoderTest`, `TypedDecisionsParityTest`, `WindowPlanTest`), so no@@ -179,7 +179,7 @@ jobs:if: ${{ !cancelled() }}run: >python laya-java/scripts/check_test_results.py- laya-java/laya-java/build/test-results/test --min-tests 1003 --allow-aborted+ laya-java/laya-java/build/test-results/test --min-tests 1007 --allow-aborted# The samples are single-file source programs, deliberately NOT a Gradle module: a# `samples` subproject would inherit the root build's MavenPublication and@@ -335,7 +335,7 @@ jobs:# dropped, Gradle exits 0 and the floor alone exits 0 on 488 tests, having run JDK 17 twice# and reported it as two JDKs. With `--require-passed` it exits 1.#- # 1004, not the build lane's 1003, and the one is the point: this lane runs one test MORE+ # 1008, not the build lane's 1007, and the one is the point: this lane runs one test MORE# than that one, because `TestJvmVersionTest` is `@EnabledIfSystemProperty` and only# enabled here. Both are EXACT, with no margin, for the reason the build lane gives --# three classes here run exactly one test, so no positive margin can bind them.@@ -356,7 +356,7 @@ jobs:if: ${{ !cancelled() }}run: >python laya-java/scripts/check_test_results.py- laya-java/laya-java/build/test-results/test --min-tests 1004 --allow-aborted+ laya-java/laya-java/build/test-results/test --min-tests 1008 --allow-aborted--require-passed TestJvmVersionTest- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2if: ${{ failure() }}diff --git a/.github/workflows/release-java.yml b/.github/workflows/release-java.ymlindex d41a89ee..239728e7 100644--- a/.github/workflows/release-java.yml+++ b/.github/workflows/release-java.yml@@ -67,7 +67,7 @@ jobs:# that lane already covered.- name: Model-free testsrun: ./gradlew --no-daemon test- # 1003 of the 1021 registered tests run without a checkpoint. This floor has been raised+ # 1007 of the 1025 registered tests run without a checkpoint. This floor has been raised# as the model-free suite grew -- 440 left 191 of slack, 625 left 60 -- because a floor# that does not bind is not a gate: under 625 the whole confidence-gate family of 26# tests could have vanished from this lane with the gate still green.@@ -83,7 +83,7 @@ jobs:- name: The suite was not vacuousrun: >python3 scripts/check_test_results.py laya-java/build/test-results/test- --min-tests 1003 --allow-aborted+ --min-tests 1007 --allow-aborted- name: Build the jarsrun: ./gradlew --no-daemon build -x testdiff --git a/laya-java/README.md b/laya-java/README.mdindex d8119f05..fa35a088 100644--- a/laya-java/README.md+++ b/laya-java/README.md@@ -205,7 +205,8 @@ List<Prediction> mixed = router.predictBatch(List.of(Requests are routed first, grouped by checkpoint in order of first appearance, and within acheckpoint those sharing a question schema and token budget share `Agent.predictBatch` calls.-Results come back in request order and equal per-request `predict`. `Router` is a+Results come back in request order and match per-request `predict` within 1e-4 (padding in a+shared forward pass moves the last bits). `Router` is a`BatchPredictor`, so `Decisions.decideBatch(router, states, schema)` works; to pin it, pass`(s, q) -> router.predictBatch(s, q, options)` as the runner.@@ -514,6 +515,9 @@ the pair, and on success `elapsedMs` does not include a cold load. A routing orreaches `onError` and `onPredictEnd`, with no start event, and its `elapsedMs` covers the routingand the failed load. The agent's own pair stillfires for the forward pass, so a router-level hook and an agent-level hook see different spans.+Process-wide defaults (`Hooks.setDefaultHooks`) see only the router's pair: the forward pass runs+without them, as the reference runs it under `_SKIP_DEFAULTS`, so they fire once per request on+`predict` and `predictBatch` alike.`Router.predictBatch` dispatches one pair **per request**, as `predict` does, so a start hook canrewrite or `skip` a request before it joins a shared forward pass. The order is the reference's:@@ -521,8 +525,8 @@ every `onRoute`, then per checkpoint its `onEvict`/`onLoad`, its requests' starttheir ends in reverse. If a checkpoint's inference fails, each of its started requests gets`onError` and its end before the failure propagates. A routing failure reaches that request, and aloading failure every request of that checkpoint, as `onError` and `onPredictEnd` with no start —-as on `predict`, where the reference's `predict_batch` reports neither. The agent's pair fires once per forward-pass, not per request. Unlike `predict`, the batch path honours `ctx.maxLen` / `ctx.headMaxLen`+as on `predict`, where the reference's `predict_batch` reports neither. The agent's installed hooks fire once+per forward pass, not per request. Unlike `predict`, the batch path honours `ctx.maxLen` / `ctx.headMaxLen`from a router-level start hook, since a request's own budget is seeded there.Hooks are dispatched outside the router's lock, so a hook may call back into the router —@@ -546,7 +550,6 @@ router.hooks().addHook(new Hook() {| | ||---|---|-| a process-wide default hook sees **two** predict pairs per `Router.predict` (per request plus per forward pass on `predictBatch`) | the router's and the agent's. A metric that sums `usage()` on `on_predict_end` double-counts. Install on `router.hooks()` or `agent.hooks()`, not `Hooks.setDefaultHooks`, unless you want both || a throwing `on_evict` hook fails `Router.close()` | and the remaining evictions are not dispatched. In try-with-resources that masks the body's own failure || calling back into the router from `on_evict` **during `close()`** | throws `this Router is closed`: `close()` marks the router closed before unloading || `concurrent(false)` with an `AsyncHook` whose callback dispatches a hook on the same registry | deadlocks with no deadline, and stalls until it with one. Serial dispatch holds a lock across the callback, and the callback is on another thread, so the inner dispatch cannot re-enter that lock. A callback that dispatches nothing — `loaded()`, a cached `load` — is fine |diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.javaindex 9302499f..d50abf2c 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.java@@ -1575,8 +1575,8 @@ public final class Router implements AutoCloseable, Predictor, BatchPredictor {PredictContext ctx = new PredictContext(Collections.singletonList(state), questions,decision.model().wireName(), this);answered = Hooks.around(composed, ctx, policy,- (states, asked, maxLen, headMaxLen) ->- List.of(held.agent().predict(states.get(0), asked, resolved)));+ (states, asked, maxLen, headMaxLen) -> withoutDefaultHooks(() ->+ List.of(held.agent().predict(states.get(0), asked, resolved))));}if (answered == null || answered.isEmpty()) {// Only a hook can cause this, by skipping with no result. Inventing one would report@@ -1587,6 +1587,17 @@ public final class Router implements AutoCloseable, Predictor, BatchPredictor {return answered.get(0);}+ /**+ * The agent's forward pass without the process-wide defaults, as the reference runs it under+ * {@code _SKIP_DEFAULTS}: a default hook already sees this request's router-level pair.+ */+ @SuppressWarnings("try")+ private static <T> T withoutDefaultHooks(Supplier<T> pass) {+ try (Hooks.DefaultsScope ignored = Hooks.withoutDefaultHooks()) {+ return pass.get();+ }+ }+/*** The language the checkpoint is told: the caller's if they gave one, else whatever routing* detected -- the reference's rule, and it matters, because the multilingual checkpoint takes@@ -1632,8 +1643,8 @@ public final class Router implements AutoCloseable, Predictor, BatchPredictor {* <p>Every request is routed first, then grouped by checkpoint in order of first appearance.* Within a checkpoint, requests that share a question schema, a token budget and (when the* checkpoint has per-language temperatures) a language share {@link Agent#predictBatch}- * calls. Each request is answered exactly as {@link #predict(Object, Map, RouteOptions)}- * would answer it.+ * calls. Each answer matches {@link #predict(Object, Map, RouteOptions)} within padding noise+ * (1e-4 measured on real checkpoints).** <p>Router hooks run per request, as {@link #predict} runs them, and in the reference's* order: every {@code on_route} first, then per checkpoint any {@code on_evict}/{@code on_load},@@ -1765,9 +1776,9 @@ public final class Router implements AutoCloseable, Predictor, BatchPredictor {states.add(ctx.states().get(0));}Key key = pass.getKey();- List<Prediction> answered = agent.predictBatch(states, members.get(0).questions(),- key.language(), batchSize, sortByLength, HookCall.none(), key.maxLen(),- key.headMaxLen());+ List<Prediction> answered = withoutDefaultHooks(() -> agent.predictBatch(states,+ members.get(0).questions(), key.language(), batchSize, sortByLength,+ HookCall.none(), key.maxLen(), key.headMaxLen()));if (answered.size() != members.size()) {throw new IllegalStateException("internal error: Agent.predictBatch returned "+ answered.size() + " results for " + members.size() + " states");diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterBatchTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterBatchTest.javaindex e28d9367..54db30b5 100644--- a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterBatchTest.java+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterBatchTest.java@@ -10,6 +10,7 @@ import com.convaiinnovations.laya.Router.Checkpoint;import com.convaiinnovations.laya.Router.Request;import com.convaiinnovations.laya.Router.RouteOptions;import com.convaiinnovations.laya.hooks.Hook;+import com.convaiinnovations.laya.hooks.Hooks;import com.convaiinnovations.laya.hooks.PredictContext;import com.convaiinnovations.laya.infer.InferenceSession;import com.convaiinnovations.laya.sequence.Collator;@@ -494,4 +495,100 @@ class RouterBatchTest {assertEquals(List.of("route:english", "error:bad@null", "end:bad"), trace.events);}}++ @Test+ @DisplayName("a start hook's replacement questions are the ones its request is asked")+ void hookQuestionsHonoured() throws IOException {+ try (Router router = router()) {+ router.hooks().addHook(new Hook() {+ @Override+ public void onPredictStart(PredictContext ctx) {+ if ("b".equals(ctx.states().get(0))) {+ ctx.questions(topic());+ }+ }+ });+ List<Prediction> out = router.predictBatch(List.of(+ Request.of("a", urgent()).options(ENGLISH),+ Request.of("b", urgent()).options(ENGLISH),+ Request.of("c", urgent()).options(ENGLISH)));+ assertEquals(List.of("urgent"), List.copyOf(out.get(0).answers().keySet()));+ assertEquals(List.of("topic"), List.copyOf(out.get(1).answers().keySet()));+ assertEquals(List.of("urgent"), List.copyOf(out.get(2).answers().keySet()));+ assertEquals(2, calls(Checkpoint.ENGLISH), "the rewritten request has its own pass");+ }+ }++ @Test+ @DisplayName("a throwing start hook fails every started request once, itself included")+ void startHookFailure() throws IOException {+ Lifecycle trace = new Lifecycle();+ try (Router router = router()) {+ router.hooks().addHook(trace);+ router.hooks().addHook(new Hook() {+ @Override+ public void onPredictStart(PredictContext ctx) {+ if ("b".equals(ctx.states().get(0))) {+ throw new IllegalStateException("start refused");+ }+ }+ });+ IllegalStateException failure = assertThrows(IllegalStateException.class,+ () -> router.predictBatch(List.of(+ Request.of("a", urgent()).options(ENGLISH),+ Request.of("b", urgent()).options(ENGLISH),+ Request.of("c", urgent()).options(ENGLISH))));+ assertEquals("start refused", failure.getMessage());+ assertEquals(List.of("route:english", "route:english", "route:english",+ "start:a", "start:b", "error:b@english", "end:b", "error:a@english", "end:a"),+ trace.events);+ assertEquals(0, calls(Checkpoint.ENGLISH), "nothing reached the model");+ }+ }++ /** Counts start events, for the process-wide defaults. */+ private static final class Starts implements Hook {++ int count;++ @Override+ public void onPredictStart(PredictContext ctx) {+ count++;+ }+ }++ @Test+ @DisplayName("a process-wide default hook starts once per request on predictBatch")+ void defaultsOncePerRequestBatch() throws IOException {+ Starts starts = new Starts();+ Hooks.setDefaultHooks(List.of(starts));+ try (Router router = router()) {+ router.predictBatch(List.of(+ Request.of("a", urgent()).options(ENGLISH),+ Request.of("b", urgent()).options(ENGLISH),+ Request.of("c", urgent()).options(ENGLISH)));+ assertEquals(3, starts.count, "the forward pass runs without the defaults");+ assertEquals(1, calls(Checkpoint.ENGLISH));+ } finally {+ Hooks.clearDefaultHooks();+ }+ }++ @Test+ @DisplayName("a process-wide default hook starts once on Router.predict")+ void defaultsOncePerPredict() throws IOException {+ Starts starts = new Starts();+ Starts installed = new Starts();+ Hooks.setDefaultHooks(List.of(starts));+ try (Router router = Router.builder().build()) {+ Agent agent = TinyCheckpoint.agent(root, new TinyCheckpoint.RecordingSession());+ agent.hooks().addHook(installed);+ router.attach("english", agent);+ router.predict("a", urgent(), ENGLISH);+ assertEquals(1, starts.count, "the forward pass runs without the defaults");+ assertEquals(1, installed.count, "the agent's own hooks still run");+ } finally {+ Hooks.clearDefaultHooks();+ }+ }}