Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(java): refuse a malformed predict before it loads, and what else reviewing 743fc47 found
Adeshina Lasisi1d1b6564fec9
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex b777fd0d..30ed56c2 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -154,17 +154,17 @@ jobs:# Without LAYA_CHECKPOINTS the checkpoint-backed factories abort by assumption and the rest# run. `--min-tests` is what stops an aborted suite from looking like a pass.#- # Measured on the merged tree: 991 register here, 974 run, 17 abort on a checkpoint this+ # Measured on the merged tree: 1000 register here, 983 run, 17 abort on a checkpoint this# lane does not have. Exact, not padded -- the floor stood at 680 long enough for the real# figure to pass it by, which is how a floor stops being one. ONE below the JDK-matrix# lane's floor on purpose: that lane passes `-PtestJavaVersion`, which is what makes- # `TestJvmVersionTest` run instead of skip, so it executes 975. Same suite, one more test,+ # `TestJvmVersionTest` run instead of skip, so it executes 984. Same suite, one more test,# because the flag enables it. The step below is where the number is justified.- name: Tests that need no modelworking-directory: laya-javarun: ./gradlew --no-daemon test- # Measured on the merged tree, Corretto 17, no checkpoints: 991 registered, 17 abort,- # 974 RUN -- the floor counts what ran, so the aborted checkpoint-backed cases are not in+ # Measured on the merged tree, Corretto 17, no checkpoints: 1000 registered, 17 abort,+ # 983 RUN -- the floor counts what ran, so the aborted checkpoint-backed cases are not in# it. The floor is EXACT, with no margin, which is a change from the margin-of-4 this# replaced. A margin only binds classes larger than itself, and three classes in this lane# run exactly ONE test (`DecoderTest`, `TypedDecisionsParityTest`, `WindowPlanTest`), so no@@ -179,7 +179,7 @@ jobs:if: ${{ !cancelled() }}run: >python laya-java/scripts/check_test_results.py- laya-java/laya-java/build/test-results/test --min-tests 974 --allow-aborted+ laya-java/laya-java/build/test-results/test --min-tests 983 --allow-aborted# The samples are single-file source programs, deliberately NOT a Gradle module: a# `samples` subproject would inherit the root build's MavenPublication and@@ -335,7 +335,7 @@ jobs:# dropped, Gradle exits 0 and the floor alone exits 0 on 488 tests, having run JDK 17 twice# and reported it as two JDKs. With `--require-passed` it exits 1.#- # 975, not the build lane's 974, and the one is the point: this lane runs one test MORE+ # 984, not the build lane's 983, and the one is the point: this lane runs one test MORE# than that one, because `TestJvmVersionTest` is `@EnabledIfSystemProperty` and only# enabled here. Both are EXACT, with no margin, for the reason the build lane gives --# three classes here run exactly one test, so no positive margin can bind them.@@ -356,7 +356,7 @@ jobs:if: ${{ !cancelled() }}run: >python laya-java/scripts/check_test_results.py- laya-java/laya-java/build/test-results/test --min-tests 975 --allow-aborted+ laya-java/laya-java/build/test-results/test --min-tests 984 --allow-aborted--require-passed TestJvmVersionTest- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2if: ${{ failure() }}diff --git a/.github/workflows/release-java.yml b/.github/workflows/release-java.ymlindex e90a85cc..9d0e571b 100644--- a/.github/workflows/release-java.yml+++ b/.github/workflows/release-java.yml@@ -67,7 +67,7 @@ jobs:# that lane already covered.- name: Model-free testsrun: ./gradlew --no-daemon test- # 974 of the 991 registered tests run without a checkpoint. This floor has been raised+ # 983 of the 1000 registered tests run without a checkpoint. This floor has been raised# as the model-free suite grew -- 440 left 191 of slack, 625 left 60 -- because a floor# that does not bind is not a gate: under 625 the whole confidence-gate family of 26# tests could have vanished from this lane with the gate still green.@@ -83,7 +83,7 @@ jobs:- name: The suite was not vacuousrun: >python3 scripts/check_test_results.py laya-java/build/test-results/test- --min-tests 974 --allow-aborted+ --min-tests 983 --allow-aborted- name: Build the jarsrun: ./gradlew --no-daemon build -x testdiff --git a/laya-java/README.md b/laya-java/README.mdindex e3645e9c..d5c3f01d 100644--- a/laya-java/README.md+++ b/laya-java/README.md@@ -493,8 +493,9 @@ events no agent can:`Router.predict` also dispatches one `onPredictStart`/`onPredictEnd` pair for the answer. As in thereference, routing and loading run first, so the order is `onRoute`, any `onEvict`/`onLoad`, then-the pair, and `elapsedMs` does not include a cold load. A routing or loading failure still reaches-`onError` and `onPredictEnd`, with no start event. The agent's own pair still+the pair, and on success `elapsedMs` does not include a cold load. A routing or loading failure+reaches `onError` and `onPredictEnd`, with no start event, and its `elapsedMs` covers the routing+and the failed load. The agent's own pair stillfires for the forward pass, so a router-level hook and an agent-level hook see different spans.Hooks are dispatched outside the router's lock, so a hook may call back into the router —diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.javaindex 6485ae55..7a9ef177 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Router.java@@ -962,12 +962,19 @@ public final class Router implements AutoCloseable, Predictor {}}+ /** {@code addSuppressed}, unless the hook rethrew {@code failure} itself, which cannot suppress itself. */+ private static void attachHookFailure(Throwable failure, Throwable hookFailure) {+ if (hookFailure != failure) {+ failure.addSuppressed(hookFailure);+ }+ }+/** {@link #dispatchOutcome} on a failing path: a hook failure is attached, not substituted. */private void dispatchOwed(Checkpoint checkpoint, Outcome outcome, Throwable failure) {try {dispatchOutcome(checkpoint, outcome);} catch (RuntimeException | Error hookFailure) {- failure.addSuppressed(hookFailure);+ attachHookFailure(failure, hookFailure);}}@@ -1364,7 +1371,7 @@ public final class Router implements AutoCloseable, Predictor {try {dispatchLifecycle(Hooks.Event.EVICT, evicted);} catch (RuntimeException | Error hookFailure) {- failure.addSuppressed(hookFailure);+ attachHookFailure(failure, hookFailure);}}throw failure;@@ -1408,7 +1415,7 @@ public final class Router implements AutoCloseable, Predictor {try {dispatchLifecycle(Hooks.Event.EVICT, evicted);} catch (RuntimeException | Error hookFailure) {- failure.addSuppressed(hookFailure);+ attachHookFailure(failure, hookFailure);}}throw failure;@@ -1484,6 +1491,14 @@ public final class Router implements AutoCloseable, Predictor {*/private Prediction predictRouted(Supplier<RouteDecision> decide, Object state,Map<String, Question> questions, String lang) {+ // Before routing, as the reference does: a malformed call must not cold-load a+ // checkpoint, fire on_load or evict a resident one before it is refused.+ if (state == null) {+ throw new IllegalArgumentException("state must not be null");+ }+ if (questions == null) {+ throw new IllegalArgumentException("questions must not be null");+ }List<Hook> composed = hooks.composeFor(HookCall.none());if (composed.isEmpty()) {RouteDecision decision = decide.get();@@ -1492,6 +1507,7 @@ public final class Router implements AutoCloseable, Predictor {}}Hooks.Policy policy = hooks.policyFor(HookCall.none());+ long startedAt = System.nanoTime(); // a failure's elapsedMs covers route + loadRouteDecision decision = null;Lease lease;try {@@ -1500,7 +1516,7 @@ public final class Router implements AutoCloseable, Predictor {} catch (RuntimeException | Error failure) {PredictContext failed = new PredictContext(Collections.singletonList(state), questions,decision == null ? null : decision.model().wireName(), this);- Hooks.failedBeforeStart(composed, failed, policy, failure);+ Hooks.failedBeforeStart(composed, failed, policy, failure, startedAt);throw failure;}// `Hooks.around` so ctx.skip, the error-before-end order, failure suppression anddiff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/AsyncHook.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/AsyncHook.javaindex 0e1c3305..4eb31433 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/AsyncHook.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/AsyncHook.java@@ -27,8 +27,9 @@ import java.util.function.Consumer;* {@code hooks_timeout}: anything it writes to the context afterwards is refused. Unlike that* case the executor's thread is reused, so the abandonment ends when the overrunning task* returns, and a later callback of the same call on that thread is accepted; an abandonment- * made by anything else on that thread stays in force. A callback that had not started by the- * deadline never runs. It still occupies the executor's+ * made by anything else on that thread stays in force. A callback that had not started when the+ * caller stopped waiting -- on the deadline, or on an interrupt while it waited for one -- never+ * runs. Without a deadline the wait ignores interrupts, so it always ends with the callback. It still occupies the executor's* thread until it returns, so a single-threaded executor has none left for the next callback --* size the executor for the deadline, or leave the deadline off.*/@@ -153,14 +154,7 @@ public final class AsyncHook implements Hook {} catch (TimeoutException expired) {// cancel(true) cannot interrupt a task already running on an executor, so the callback// is cut off from the call instead: whatever it writes from here on is refused.- synchronized (gate) {- gaveUp[0] = true;- Thread overrunning = runner.get();- if (overrunning != null && !finished[0]) {- ctx.abandon(overrunning);- abandonedHere[0] = true;- }- }+ giveUp(gate, gaveUp, finished, abandonedHere, runner, ctx);running.cancel(true);throw new IllegalStateException(delegate + " did not finish " + event + " within " + timeout, expired);@@ -168,11 +162,27 @@ public final class AsyncHook implements Hook {// The flag is restored before unwinding, so a caller using interruption to cancel is// not left thinking the interrupt was swallowed.Thread.currentThread().interrupt();+ // The call stops waiting here just as it does on a deadline, so the same cut-off.+ giveUp(gate, gaveUp, finished, abandonedHere, runner, ctx);running.cancel(true);throw new IllegalStateException(delegate + " was interrupted in " + event, interrupted);}}+ /** The call has stopped waiting: a callback not yet started never runs, a running one is cut off. */+ private static void giveUp(Object gate, boolean[] gaveUp, boolean[] finished,+ boolean[] abandonedHere, AtomicReference<Thread> runner,+ PredictContext ctx) {+ synchronized (gate) {+ gaveUp[0] = true;+ Thread overrunning = runner.get();+ if (overrunning != null && !finished[0]) {+ ctx.abandon(overrunning);+ abandonedHere[0] = true;+ }+ }+ }+@Overridepublic String toString() {return "AsyncHook(" + delegate + (timeout == null ? "" : ", timeout=" + timeout) + ")";diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/Hooks.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/Hooks.javaindex e0bde893..2f37a0dd 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/Hooks.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/Hooks.java@@ -1,5 +1,7 @@package com.convaiinnovations.laya.hooks;+import static java.util.Objects.requireNonNull;+import com.convaiinnovations.laya.Prediction;import com.convaiinnovations.laya.Question;import java.time.Duration;@@ -567,6 +569,8 @@ public final class Hooks {// flag: a caller up the stack that polls `Thread.interrupted()` to decide whether to// shut down would otherwise be told it was never asked to.Thread.currentThread().interrupt();+ // The call stops waiting here, so the hook is cut off exactly as on a deadline.+ ctx.abandon(runner);throw new IllegalStateException("interrupted while waiting for hook "+ hookName(hook) + "." + event.wireName(), interrupted);}@@ -741,7 +745,7 @@ public final class Hooks {} catch (RuntimeException | Error hookFailure) {// Attached, never substituted: the thing that broke the request is what the// caller has to catch, and a failing observer must not be able to hide it.- problem.addSuppressed(hookFailure);+ attach(problem, hookFailure);}}@@ -755,7 +759,7 @@ public final class Hooks {if (raised == null) {throw hookFailure;}- raised.addSuppressed(hookFailure);+ attach(raised, hookFailure);}if (raised instanceof RuntimeException problem) {@@ -770,20 +774,41 @@ public final class Hooks {/*** Reports a failure that happened before {@code on_predict_start} could fire -- routing or* loading, which the reference runs inside its predict {@code try} -- as {@code on_error} and- * then {@code on_predict_end}. A hook failure is attached to {@code problem}, never substituted.+ * then {@code on_predict_end}, with {@code elapsedMs} measured from {@code startedAtNanos}.+ * A hook failure is attached to {@code problem}, never substituted.+ *+ * <p>For a {@link com.convaiinnovations.laya.Predictor} that must do work before its start+ * event, as {@code Router} does. {@code ctx} must be fresh: one that already carries results+ * or an error belongs to a call that has been reported, and is refused.*/public static void failedBeforeStart(List<? extends Hook> hooks, PredictContext ctx,- Policy policy, Throwable problem) {+ Policy policy, Throwable problem, long startedAtNanos) {+ requireNonNull(problem, "problem");+ if (ctx.results() != null || ctx.error() != null) {+ throw new IllegalStateException(+ "failedBeforeStart needs a fresh context; this one already has an outcome");+ }ctx.error(problem);try {dispatch(hooks, Event.ERROR, ctx, policy);} catch (RuntimeException | Error hookFailure) {- problem.addSuppressed(hookFailure);+ attach(problem, hookFailure);}- ctx.elapsedMs((System.nanoTime() - ctx.startedAt()) / 1_000_000.0);+ ctx.elapsedMs((System.nanoTime() - startedAtNanos) / 1_000_000.0);try {dispatch(hooks, Event.PREDICT_END, ctx, policy);} catch (RuntimeException | Error hookFailure) {+ attach(problem, hookFailure);+ }+ }++ /**+ * {@code addSuppressed}, except when a hook rethrew the failure it was shown: a throwable+ * cannot suppress itself, and trying replaces the real failure with an+ * {@code IllegalArgumentException}.+ */+ private static void attach(Throwable problem, Throwable hookFailure) {+ if (hookFailure != problem) {problem.addSuppressed(hookFailure);}}diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/PredictContext.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/PredictContext.javaindex 3fa12769..e4e2c4d2 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/PredictContext.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/hooks/PredictContext.java@@ -7,6 +7,7 @@ import java.util.ArrayList;import java.util.LinkedHashMap;import java.util.List;import java.util.Map;+import java.util.Set;import java.util.UUID;import java.util.concurrent.ConcurrentHashMap;@@ -65,9 +66,7 @@ public final class PredictContext {* <p>Empty for every call that does not use {@link Hooks.Policy#timeout}, and at most one* entry per overrun. A set rather than a flag because a chain can overrun more than once.*/- // A count, not a set: a hook abandoned by Hooks can call an AsyncHook that abandons and- // releases the same thread, and that release must not cancel the outer abandonment.- private final Map<Thread, Integer> abandoned = new ConcurrentHashMap<>();+ private final Set<Thread> abandoned = ConcurrentHashMap.newKeySet();/*** A context for one call.@@ -310,20 +309,20 @@ public final class PredictContext {* call stops being able to wait for it. See {@link #states(List)}.*/void abandon(Thread runner) {- abandoned.merge(runner, 1, Integer::sum);+ abandoned.add(runner);}/*** Undoes one {@link #abandon} once the overrunning task has returned, so a pooled thread that* runs this call's next callback is not refused for the previous one's overrun. Only the- * caller that abandoned may release, and abandonments of one thread nest.+ * caller that abandoned a thread may release it.*/void release(Thread runner) {- abandoned.computeIfPresent(runner, (thread, count) -> count == 1 ? null : count - 1);+ abandoned.remove(runner);}private void refuseIfAbandoned(String what) {- if (!abandoned.isEmpty() && abandoned.containsKey(Thread.currentThread())) {+ if (!abandoned.isEmpty() && abandoned.contains(Thread.currentThread())) {throw new IllegalStateException(String.format("laya: hook thread %s exceeded its deadline and was abandoned; this call has "+ "moved on, so ctx.%s() is refused rather than applied to it",diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/AsyncHookTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/AsyncHookTest.javaindex 73ff48b1..030e94e6 100644--- a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/AsyncHookTest.java+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/AsyncHookTest.java@@ -236,6 +236,86 @@ class AsyncHookTest {assertEquals(List.of("state"), ctx.states());}+ @Test+ @DisplayName("an interrupted wait cuts the callback off, as a deadline does")+ void anInterruptedWaitAbandonsTheCallback() throws Exception {+ ExecutorService pool = Executors.newSingleThreadExecutor();+ try {+ CountDownLatch started = new CountDownLatch(1);+ CountDownLatch done = new CountDownLatch(1);+ Hook async = AsyncHook.of(new Hook() {+ @Override+ public void onPredictStart(PredictContext c) {+ started.countDown();+ try {+ Thread.sleep(300);+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ try {+ c.states(List.of("late"));+ } catch (RuntimeException refused) {+ // refused, which is the point+ } finally {+ done.countDown();+ }+ }+ }, pool, Duration.ofSeconds(5)); // timed: an untimed join() ignores interrupts+ assertLateWriteRefusedAfterInterrupt(ctx -> async.onPredictStart(ctx), started, done);+ } finally {+ pool.shutdownNow();+ }+ }++ @Test+ @DisplayName("an interrupted wait on a Hooks timeout thread cuts that hook off too")+ void anInterruptedHooksWaitAbandonsTheHook() throws Exception {+ CountDownLatch started = new CountDownLatch(1);+ CountDownLatch done = new CountDownLatch(1);+ Hook slow = new Hook() {+ @Override+ public void onPredictStart(PredictContext c) {+ started.countDown();+ try {+ Thread.sleep(300);+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ try {+ c.states(List.of("late"));+ } catch (RuntimeException refused) {+ // refused, which is the point+ } finally {+ done.countDown();+ }+ }+ };+ assertLateWriteRefusedAfterInterrupt(ctx -> Hooks.dispatch(List.of(slow),+ Hooks.Event.PREDICT_START, ctx, Hooks.Policy.raising().timeout(Duration.ofSeconds(5))),+ started, done);+ }++ private static void assertLateWriteRefusedAfterInterrupt(+ java.util.function.Consumer<PredictContext> waitOn, CountDownLatch started,+ CountDownLatch done) throws Exception {+ PredictContext ctx = ctx();+ AtomicReference<Throwable> caught = new AtomicReference<>();+ Thread caller = new Thread(() -> {+ try {+ waitOn.accept(ctx);+ } catch (RuntimeException e) {+ caught.set(e);+ }+ });+ caller.start();+ assertTrue(started.await(3, TimeUnit.SECONDS));+ caller.interrupt();+ caller.join(3_000);+ assertTrue(caught.get() instanceof IllegalStateException, String.valueOf(caught.get()));+ assertTrue(done.await(3, TimeUnit.SECONDS));+ assertEquals(List.of("state"), ctx.states(), "a write after the caller moved on landed");+ }+@Test@DisplayName("a non-positive timeout is refused rather than meaning 'never wait'")void refusesANonPositiveTimeout() {diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterHooksTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterHooksTest.javaindex 10f7603f..5d8fe542 100644--- a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterHooksTest.java+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/RouterHooksTest.java@@ -2,6 +2,7 @@ package com.convaiinnovations.laya;import static org.junit.jupiter.api.Assertions.assertEquals;import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertSame;import static org.junit.jupiter.api.Assertions.assertThrows;import static org.junit.jupiter.api.Assertions.assertTrue;@@ -300,8 +301,10 @@ class RouterHooksTest {IllegalStateException thrown = assertThrows(IllegalStateException.class,() -> router.predict("hello", questions()));assertEquals("no weights here", thrown.getMessage());- List<String> kinds = trace.events.stream().map(e -> e.split(":")[0]).toList();- assertEquals(List.of("route", "error", "end"), kinds, trace.events.toString());+ // The load was attempted on a decided checkpoint, so error and end name it, as the+ // reference's ctx.model does once route() has returned.+ String model = trace.events.get(0).split(":")[1];+ assertEquals(List.of("route:" + model, "error:" + model, "end:" + model), trace.events);}}@@ -380,6 +383,143 @@ class RouterHooksTest {}}+ /** A factory that keeps every session it hands out, so a test can see which were closed. */+ private static Router.AgentFactory recording(Path root, List<TinyCheckpoint.RecordingSession> out) {+ return checkpoint -> {+ TinyCheckpoint.RecordingSession session = new TinyCheckpoint.RecordingSession();+ out.add(session);+ return TinyCheckpoint.agent(root, session);+ };+ }++ @Test+ @DisplayName("a hooked predict releases its lease, so unloading closes the agent")+ void hookedPredictReleasesItsLease(@TempDir Path root) throws IOException {+ TinyCheckpoint.write(root, 64, 32);+ List<TinyCheckpoint.RecordingSession> sessions = new ArrayList<>();+ try (Router router = Router.builder().agents(recording(root, sessions)).build()) {+ router.hooks().addHook(new Trace());+ router.predict("hello", questions());+ router.unloadAll();+ assertEquals(1, sessions.size());+ assertTrue(sessions.get(0).closed, "a held lease keeps a retired agent open forever");+ }+ }++ @Test+ @DisplayName("a hooked predict whose start hook throws still releases its lease")+ void hookedPredictReleasesItsLeaseOnFailure(@TempDir Path root) throws IOException {+ TinyCheckpoint.write(root, 64, 32);+ List<TinyCheckpoint.RecordingSession> sessions = new ArrayList<>();+ try (Router router = Router.builder().agents(recording(root, sessions)).build()) {+ router.hooks().addHook(new Hook() {+ @Override+ public void onPredictStart(PredictContext ctx) {+ throw new IllegalStateException("start refused");+ }+ });+ assertThrows(IllegalStateException.class, () -> router.predict("hello", questions()));+ router.unloadAll();+ assertTrue(sessions.get(0).closed, "a held lease keeps a retired agent open forever");+ }+ }++ @Test+ @DisplayName("unloadAll: every eviction is dispatched and a hook failure is attached, not swapped")+ void unloadAllDoesNotMaskTheCloseFailure(@TempDir Path root) throws IOException {+ TinyCheckpoint.write(root, 64, 32);+ Trace trace = new Trace();+ try (Router router = Router.builder().agents(angryEnglish(root)).maxLoaded(2).build()) {+ router.load("english");+ router.load("multilingual");+ router.hooks().addHook(trace); // first, so it sees the event that throws+ router.hooks().addHook(throwsOnceOnEvict(new IllegalArgumentException("the sink refused it")));+ IllegalStateException thrown =+ assertThrows(IllegalStateException.class, router::unloadAll);+ assertEquals("a native close can fail", thrown.getMessage());+ assertEquals(1, thrown.getSuppressed().length);+ assertEquals(List.of("evict:english", "evict:multilingual"), trace.events,+ "a hook failure on the first eviction must not skip the second");+ }+ }++ @Test+ @DisplayName("null questions are refused before routing: nothing is built, loaded or evicted")+ void nullQuestionsAreRefusedBeforeAnyLoad(@TempDir Path root) throws IOException {+ TinyCheckpoint.write(root, 64, 32);+ StubAgents agents = new StubAgents(root);+ Trace trace = new Trace();+ try (Router router = Router.builder().agents(agents).build()) {+ router.hooks().addHook(trace);+ assertThrows(IllegalArgumentException.class, () -> router.predict("hello", null));+ assertEquals(0, agents.builds);+ assertEquals(List.of(), trace.events);+ }+ }++ @Test+ @DisplayName("a failure before start: elapsedMs covers the load, ctx.error is the failure")+ void aFailureBeforeStartIsTimedAndCarriesTheError(@TempDir Path root) throws IOException {+ IllegalStateException failure = new IllegalStateException("no weights here");+ Router.AgentFactory slow = checkpoint -> {+ try {+ Thread.sleep(60);+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ throw failure;+ };+ AtomicReference<Double> elapsed = new AtomicReference<>();+ AtomicReference<Throwable> seen = new AtomicReference<>();+ try (Router router = Router.builder().agents(slow).build()) {+ router.hooks().addHook(new Hook() {+ @Override+ public void onPredictEnd(PredictContext ctx) {+ elapsed.set(ctx.elapsedMs());+ seen.set(ctx.error());+ }+ });+ assertSame(failure, assertThrows(IllegalStateException.class,+ () -> router.predict("hello", questions())));+ assertTrue(elapsed.get() >= 50.0, "elapsedMs " + elapsed.get() + " missed the load");+ assertSame(failure, seen.get());+ }+ }++ @Test+ @DisplayName("an on_error hook that rethrows ctx.error() leaves the caller the real failure")+ void aRethrowingErrorHookDoesNotReplaceTheFailure(@TempDir Path root) throws IOException {+ Router.AgentFactory broken = checkpoint -> {+ throw new IllegalStateException("no weights here");+ };+ try (Router router = Router.builder().agents(broken).build()) {+ router.hooks().addHook(new Hook() {+ @Override+ public void onError(PredictContext ctx) {+ throw (RuntimeException) ctx.error(); // log-and-rethrow+ }+ });+ IllegalStateException thrown = assertThrows(IllegalStateException.class,+ () -> router.predict("hello", questions()));+ assertEquals("no weights here", thrown.getMessage());+ }+ }++ @Test+ @DisplayName("an Error while loading still reaches on_error and on_predict_end")+ void anErrorBeforeStartIsReported(@TempDir Path root) throws IOException {+ Trace trace = new Trace();+ Router.AgentFactory native_ = checkpoint -> {+ throw new UnsatisfiedLinkError("no onnxruntime here");+ };+ try (Router router = Router.builder().agents(native_).build()) {+ router.hooks().addHook(trace);+ assertThrows(UnsatisfiedLinkError.class, () -> router.predict("hello", questions()));+ List<String> kinds = trace.events.stream().map(e -> e.split(":")[0]).toList();+ assertEquals(List.of("route", "error", "end"), kinds, trace.events.toString());+ }+ }+/** A hook whose first on_evict throws {@code failure}; later ones (from close()) do not. */private static Hook throwsOnceOnEvict(Throwable failure) {AtomicBoolean once = new AtomicBoolean();