Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(java): MODELS.md, a CHANGELOG, and three samples that are run before they ship
Adeshina Lasisifb2b559f769b
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex eae3e241..2d55712f 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -72,6 +72,25 @@ jobs:python laya-java/scripts/check_test_results.pylaya-java/laya-java/build/test-results/test --min-tests 585 --allow-aborted+ # The samples are single-file source programs, deliberately NOT a Gradle module: a+ # `samples` subproject would inherit the root build's MavenPublication and+ # `publishAllPublicationsToCentralRepository` would push it, which is why the empty+ # `laya-java-client` module was taken out of settings.gradle.kts in the first place. A+ # version burned on Central cannot be replaced.+ #+ # Compiled here so they cannot rot: a sample that stops matching the API fails the build+ # rather than sitting in the tree misleading a reader. `-Werror` because the samples are the+ # first code a user copies.+ - name: The samples still compile against the built classes+ working-directory: laya-java+ run: |+ mkdir -p "$RUNNER_TEMP/samples"+ javac -Xlint:all -Werror \+ -cp "laya-java/build/classes/java/main:$(find ~/.gradle/caches/modules-2 \+ -name 'onnxruntime-*.jar' | head -1)" \+ -d "$RUNNER_TEMP/samples" samples/*.java+ echo "compiled:"; find samples -name '*.java' -print+# LAST, deliberately. This gate and the tests are independent signals -- the tests compare# against the COMMITTED fixture, this compares the committed fixture against a fresh# recording -- and running it first meant a stale fixture skipped the build and the testsdiff --git a/laya-java/CHANGELOG.md b/laya-java/CHANGELOG.mdnew file mode 100644index 00000000..1541144e--- /dev/null+++ b/laya-java/CHANGELOG.md@@ -0,0 +1,91 @@+# Changelog++All notable changes to the `com.convaiinnovations:laya-java` artifact are documented here. The+format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project uses+[Semantic Versioning](https://semver.org/). Releases are tagged `laya-java-v<version>`.++## [0.1.0] - unreleased++First release: a port of the Laya Python SDK to the JVM, with every expectation recorded by+running the Python package rather than written by hand. **Not yet published to Maven Central** —+see [Installing](README.md#installing).++### Added++- `Agent`: `predict` and `predictBatch` over an exported ONNX checkpoint, with every question for+ a state answered in one batched forward pass. Typed answers — `Answer.Choice`, `Answer.Score`,+ `Answer.Noul` — as a sealed interface, so a choice is not a score at the API boundary.+- Both ONNX layouts: the fused `laya.onnx` and the split `encoder.onnx` + `head.onnx`.+- `Usage` and truncation reporting, including which question is blamed for a truncation, and+ `confidence` alongside `answerConfidence`.+- Python-compatible tokenization (BPE and Metaspace, added tokens, normalizers), sequence+ building, collation, answer decoding and calibration.+- `lang.LanguageDetection`: script and language detection, the eight public fields and the+ fourteen intermediate steps the reference exposes.+- `Presets`: the five preset question sets, compared against the reference word for word.+- `Router`: checkpoint selection with its reason strings, plus a load-and-evict lifecycle with+ leases, so a checkpoint in use is never closed under a caller.+- `Shortlist`: cosine ranking over a caller's embedder, with an LRU cache and numpy's tie order.+- `LayaEmail`: `cleanEmailBody` and `emailState`, with the English, Portuguese, Spanish and French+ marker sets, and `emailQuestions` re-exported from `Presets`.+- `Predictor`: one interface satisfied unchanged by both `Agent` and `Router`.+- `json.PythonJson`: CPython's `repr` and `%.0f`, which the router's reason strings are built from.+- `scripts/prepare_checkpoint.py`: a pinned download and export, writing the two directories+ `Agent.open` takes. See [MODELS.md](MODELS.md).++### Testing++- 768 tests, 594 of which need no model at all.+- Every expectation is **generated from the Python package**, never written by hand:+ `laya-java/fixtures/*.json`, regenerated by `scripts/gen_fixtures.py` and gated by `--check` in+ CI so a change to `laya/` that moves a recorded contract shows up here rather than as drift.+- The compiled-in Unicode tables are checked by a **sha256 over all 1,114,112 code points** per+ property, so they are proven exhaustively rather than on the cases someone thought to write.+- Five CI lanes: build plus model-free tests, the same tests on JDK 21 and JDK 24, and two parity+ cells split by whether they need a traced graph.+- `-PtestJavaVersion` runs the tests on a different JDK from the one the classes are compiled+ for, because the toolchain pins the compiler to 17 and a lane that merely installs another JDK+ tests the same thing twice.++### Known limitations++- Not implemented: hooks, `predictLong`, structured `decide`, the `laya-java-client` HTTP module,+ Android.+- `typed-decisions` is routable — `Router` knows it and resolves its aliases — but has no recorded+ end-to-end fixtures. The routing decision is tested; a forward pass against that checkpoint is+ not.+- The softmax is computed in `double` where Python computes it in float32, as `laya-ts` and+ `laya-dotnet` also do. Over 200,000 random logit rows the four-decimal probabilities differ on+ 88 of them. Rounding, by contrast, is exact: `BigDecimal` at scale 4 with `HALF_EVEN` is+ Python's `round(v, 4)` on Python's operand.+- Float `repr` spelling diverges from CPython on 218 of 1.7M doubles, in the branch that chooses+ scientific notation. It reaches a reason string, never a number.+- Seven tokenizer behaviours are implemented but unexercised, because no shipped checkpoint+ declares them: declared added-token ids, duplicate merge ranks, `end_of_word_suffix`,+ `prepend_scheme: "first"`, the `single_word` fallback, `model.dropout` and `truncation`.++### Where Java is not Python, and what it cost++Recorded here because each one was a defect first and a lesson second.++- `\p{L}` and `\p{N}` in `java.util.regex` follow **the JDK's** Unicode version, so the+ pre-tokenizer returned different token ids from the same jar on different JDKs — Corretto 17+ carries Unicode 13.0 and Corretto 24 carries 16.0, and they disagree on 9,917 code points. The+ classes are recorded from the reference tokenizer and compiled in, and the pattern is a scanner+ over them.+- `truncate_left = isinstance(state, list)`. A conversation serialises newest-last, so passing a+ literal `false` showed the model the *opening* of an oversized conversation and discarded the+ current turn. `usage` was byte-identical either way.+- Python's `str.strip()` takes U+0085, U+00A0, U+2007 and U+202F; `String.trim()` does not. The+ gap routed `"en"` plus an ideographic space to the multilingual checkpoint while the reason+ string still said the caller asked for English.+- Python's `\s` is 29 code points and Java's is 6, or 25 under `UNICODE_CHARACTER_CLASS`; Python's+ `\d` is Nd and Java's follows the JDK. Both are built from the recorded tables.+- Python slices a `str` by **code point**, and a negative index drops the tail rather than+ returning nothing. `substring` does neither.+- `Double.compare` ranks NaN as the largest double; `np.argsort` puts it last. In a shortlist that+ does not merely misreport a score, it changes which labels survive the cut.+- `%.0f` and `round()` are half-to-**even** in CPython; `String.format` and `Math.round` are+ half-up.++[0.1.0]: https://github.com/NandhaKishorM/layadiff --git a/laya-java/MODELS.md b/laya-java/MODELS.mdnew file mode 100644index 00000000..06ec071a--- /dev/null+++ b/laya-java/MODELS.md@@ -0,0 +1,180 @@+# Laya models: getting the artifacts `laya-java` reads++`laya-java` runs ONNX. It does not download anything, does not call Hugging Face, and does not+start a Python process — `Agent.open` takes two directories that already exist on disk:++| directory | holds | made by |+|---|---|---|+| the **checkpoint** | `rl_agent_config.json`, `tokenizer/tokenizer.json`, `encoder/config.json` | a download |+| the **graph** | `laya.onnx` (fused), or `encoder.onnx` + `head.onnx` (split) | the exporter |++One script does both. From the repository root:++```bash+python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual+```++It writes, under `laya-java/.work/` so the repository root stays clean and one `.gitignore`+covers it:++```+laya-java/.work/checkpoints/multilingual/ the checkpoint, as Agent.open expects it+laya-java/.work/onnx/multilingual/laya.onnx the fused graph, as Agent.open expects it+```++Then point the SDK at them:++```bash+export LAYA_CHECKPOINTS=$PWD/laya-java/.work/checkpoints+export LAYA_ONNX_GRAPH=$PWD/laya-java/.work/onnx/multilingual+```++or pass the paths directly:++```java+Agent agent = Agent.open(+ Path.of("laya-java/.work/checkpoints/multilingual"),+ Path.of("laya-java/.work/onnx/multilingual"));+```++---++## The three checkpoints++Named as `laya.router` names them, and resolved through it — the script reads+`laya.router.DEFAULT_MODELS` rather than keeping its own copy of the repository names, so a+checkpoint the Python package knows about is one this can fetch.++| `--checkpoint` | model | tokens | notes |+|---|---|---|---|+| `english` | ModernBERT-large, 421M | 512 | English only, and it does not gently degrade off English |+| `multilingual` | mmBERT-base, 322M | 1024 | 100+ languages |+| `typed-decisions` | ModernBERT-large, 421M | 1024 | fine-tuned on four synthetic workflows |++`typed-decisions` is never selected automatically. `Router` knows it, and the aliases `typed`,+`typed_decisions`, `decisions` and `laya-typed-decisions` all resolve to it, but reaching it takes+an explicit `task` or `autoTaskDetection` — it is fine-tuned on four specific workflows and is the+wrong silent default. **It has no recorded end-to-end fixtures in this port yet**: the routing+decision is tested, the forward pass against that checkpoint is not.++A prepared checkpoint is about **680–810 MB** on disk, and the fused multilingual graph is+**1.29 GB** (`laya.onnx` is a 2.8 MB graph beside a `laya.onnx.data` of external weights). Budget+accordingly; the export is the slow part.++---++## Only the tokenizer, when that is all you need++Most of what this SDK does needs no weights. The tokenizer, the sequence builder, the collator,+language detection, the presets, the router's *decision* and the email cleaner are all exercised+by `tokenizer/tokenizer.json` and `rl_agent_config.json`, which are kilobytes:++```bash+python laya-java/scripts/prepare_checkpoint.py --checkpoint english --no-graph+```++That skips `model.safetensors` and the export entirely. It is what the `jvm parity (tokenizer)`+CI cell uses, and why that cell runs in seconds while the cell that needs a graph takes a cache+miss measured in tens of minutes.++---++## Pinning, and why a stamp sits next to the artifact++The download is pinned: `HF_REPO = "convaiinnovations/laya"` at revision+`55cf4c4ebb4ebe31b2550e8bdf3bd21b99753851`, the same pin `laya-dotnet/tools/regen_golden.py`+uses. Both lanes measure the same checkpoint, so they must not drift to different revisions — a+port that matched one and not the other would look like a port bug. The script refuses to run if+a checkpoint has moved to a different repository rather than silently fetching from the new one.++Beside each artifact it writes a `.laya-revision` stamp, and "already here" means "already here at+THIS revision". Without it, `--force` would be the only way to pick up a revision bump, and a+restored CI cache would make the whole lane measure the old model while reporting on the new one:+the fixture would be re-recorded from the stale graph, both sides of the comparison would agree,+and the cell would go green having tested nothing it claims to. The stamp also records whether+weights were fetched, because a `--no-graph` checkpoint cannot be traced and must not satisfy a+request that needs to.++To re-download and re-export anyway:++```bash+python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual --force+```++To fetch a different revision (and get a different stamp):++```bash+python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual --revision <sha>+```++---++## Both graph layouts++`LayaSession.open` takes a directory and accepts either layout:++* **fused** — `laya.onnx`, one graph from ids to logits. What the exporter writes, and what+ `prepare_checkpoint.py` produces.+* **split** — `encoder.onnx` + `head.onnx`. A documented, advertised layout, so it is supported:+ the encoder returns a pooled representation and the head turns it into logits.++If you export the split layout yourself, note that the head declares `qtype` as `[B, 1]`, not a+rank-1 tensor. This port passed rank-1 at first and ORT refused every graph the only exporter for+that layout produces, with an error that read like a bad model rather than a bad client.++A directory with neither layout is an error that names what it looked for, rather than a null+later on.++---++## Exporting by hand++`prepare_checkpoint.py` shells out to the repository's own exporter, so the graph the Java tests+run is the graph `laya` ships:++```bash+PYTHONPATH=. python scripts/export_onnx.py \+ --model laya-java/.work/checkpoints/multilingual \+ --output laya-java/.work/onnx/multilingual/laya.onnx+```++`PYTHONPATH`, not just the working directory: a script's `sys.path[0]` is the *script's*+directory, so `scripts/export_onnx.py` would get `<repo>/scripts` and `import laya` fails with+`ModuleNotFoundError` even when run from the repository root. Setting it rather than installing+the package keeps the export measuring the working tree.++---++## What the SDK needs at run time++| variable | read by | meaning |+|---|---|---|+| `LAYA_CHECKPOINTS` | the tests | a directory holding `english/`, `multilingual/`, … |+| `LAYA_ONNX_GRAPH` | the tests | the directory holding `laya.onnx`, or the file itself for the generator |+| `LAYA_FIXTURE_CHECKPOINTS` | `gen_fixtures.py` | the checkpoint root when re-recording |+| `LAYA_PREDICT_MODEL` | `gen_fixtures.py` | which checkpoint the end-to-end golden is recorded from |++A **blank** value counts as absent, so a CI cell that owns no graph can set+`LAYA_ONNX_GRAPH=''` without turning a test's deliberate abort into a failure.++None of these are read by the library itself. `Agent.open` and `Router` take paths; the+environment is the test harness's business.++---++## Without a graph, tests abort rather than pass++A parity test that needs a checkpoint says so and is skipped. That is deliberate, and the CI+floors exist because a skipped parity test is not a green one:++```+$ ./gradlew test+...+PredictParityTest > singleMatches() SKIPPED+```++With `LAYA_CHECKPOINTS` and `LAYA_ONNX_GRAPH` set, nothing in the suite may abort, and+`scripts/check_test_results.py` fails the run if anything does.++For the library's own API, see the [README](README.md). For what changed, see the+[CHANGELOG](CHANGELOG.md).diff --git a/laya-java/README.md b/laya-java/README.mdindex a51faabc..c5802821 100644--- a/laya-java/README.md+++ b/laya-java/README.md@@ -20,6 +20,14 @@ Not implemented yet: hooks, `predictLong`, structured `decide`, the`laya-java-client` HTTP module, Android. **Not published to Maven Central** — see[Installing](#installing).+## Where to go next++| | |+|---|---|+| [MODELS.md](MODELS.md) | how to get the checkpoint and the graph `Agent.open` takes |+| [samples/](samples/) | three runnable programs: quickstart, routing and shortlisting, benchmark |+| [CHANGELOG.md](CHANGELOG.md) | what is in 0.1.0, and where Java is not Python |+## Export a graph (once per checkpoint)```bashdiff --git a/laya-java/samples/Benchmark.java b/laya-java/samples/Benchmark.javanew file mode 100644index 00000000..22175a50--- /dev/null+++ b/laya-java/samples/Benchmark.java@@ -0,0 +1,150 @@+// What this SDK costs, measured the only way a JVM measurement means anything: warm, best of N,+// and with the thing you are timing separated from the thing you are not.+//+// A first measurement on a cold JVM is dominated by class loading, JIT and -- for this SDK -- the+// one-time walk of Unicode that builds the character classes. Quoting that number as throughput+// is how a port comes to look slower than it is, and quoting a single warm run is how noise comes+// to look like a regression. So every figure below is the best of N after a warmup, and the cold+// first call is reported separately rather than averaged into the rest.+//+// python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual+// java -cp "<jar>:<onnxruntime>" samples/Benchmark.java .work/checkpoints/multilingual .work/onnx/multilingual+//+// See samples/README.md for the classpath.++import com.convaiinnovations.laya.Agent;+import com.convaiinnovations.laya.LayaEmail;+import com.convaiinnovations.laya.Presets;+import com.convaiinnovations.laya.Question;+import com.convaiinnovations.laya.lang.LanguageDetection;+import java.nio.file.Path;+import java.util.ArrayList;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;+import java.util.function.Supplier;++public final class Benchmark {++ // Enough warmup to get past the interpreter, and no more. A model-free call is+ // microseconds, so 20 of them cost nothing; a forward pass is tens of milliseconds and a+ // batch of 16 is more, so the same count there would run for minutes and read as a hang. A+ // sample that looks hung is a sample nobody finishes running.+ private static final int WARMUP = 20;+ private static final int ROUNDS = 5;+ private static final int MODEL_WARMUP = 1;+ private static final int MODEL_ROUNDS = 2;+ /** Small, because on CPU this is seconds per round and a sample has to finish. */+ private static final int BATCH = 8;++ public static void main(String[] args) throws Exception {+ // ------------------------------------------------- the model-free half, which needs nothing+ String shortState = "I cannot log in and need a password reset.";+ String longState = shortState.repeat(90); // ~3.8 KB+ String email = "I was charged twice for order 8812.\n\nAtenciosamente,\nAna Souza\n"+ + "Enviado do meu iPhone\n\nEsta mensagem e confidencial.\n\n"+ + "Em ter., 3 de set. de 2025, Suporte <[email protected]> escreveu:\n> Podemos ajudar?";++ System.out.println("model-free (no checkpoint, no graph)");+ // The cold call is reported, not hidden: it is what a caller's first request pays.+ long coldDetect = time(() -> LanguageDetection.analyse(shortState));+ System.out.printf(" %-34s %8.3f ms <- cold, includes class init%n",+ "detect, first ever call", coldDetect / 1e6);+ bench("detect, 42-char state", () -> LanguageDetection.analyse(shortState));+ bench("detect, 3.8 KB state", () -> LanguageDetection.analyse(longState));++ long coldClean = time(() -> LayaEmail.cleanEmailBody(email));+ System.out.printf(" %-34s %8.3f ms <- cold, builds the character classes%n",+ "email clean, first ever call", coldClean / 1e6);+ bench("email clean", () -> LayaEmail.cleanEmailBody(email));++ if (args.length < 2) {+ System.out.println();+ System.out.println("pass <checkpoint-dir> <graph-dir> to also measure the forward pass");+ return;+ }++ // ------------------------------------------------- the forward pass+ Map<String, Question> questions = Presets.email();+ Map<String, Object> state = new LinkedHashMap<>();+ state.put("subject", "Charged twice for order 8812");+ state.put("body", "I was billed two times this morning. Please refund one charge.");++ List<Object> batch = new ArrayList<>();+ for (int i = 0; i < BATCH; i++) {+ batch.add(state);+ }++ System.out.println();+ System.out.println("forward pass (" + questions.size() + " questions per state)");+ try (Agent agent = Agent.open(Path.of(args[0]), Path.of(args[1]))) {+ long coldPredict = time(() -> agent.predict(state, questions));+ System.out.printf(" %-34s %8.3f ms <- cold, includes the ORT session%n",+ "predict, first ever call", coldPredict / 1e6);+ bench("predict, 1 state", () -> agent.predict(state, questions),+ MODEL_WARMUP, MODEL_ROUNDS);+ long batched = best(() -> agent.predictBatch(batch, questions),+ MODEL_WARMUP, MODEL_ROUNDS);+ System.out.printf(" %-34s %8.3f ms%n",+ "predictBatch, " + BATCH + " states", batched / 1e6);+ // Per state, which is the number that matters when you are sizing a queue.+ System.out.printf(" %-34s %8.3f ms%n",+ " ... per state", batched / 1e6 / batch.size());+ }++ System.out.println();+ System.out.printf("Model-free figures are the best of %d after %d warmup calls; the "+ + "model-backed%nones are the best of %d after %d, because a forward pass is "+ + "milliseconds and not%nmicroseconds. The two cold figures are single calls.%n",+ ROUNDS, WARMUP, MODEL_ROUNDS, MODEL_WARMUP);+ System.out.println("Best, not mean: the mean of a JVM measurement is a mean over how much");+ System.out.println("other work the machine happened to be doing.");+ System.out.println();+ System.out.println("If the per-state batch figure is no better than the single-state one,");+ System.out.println("that is the expected shape on CPU rather than a problem: batching");+ System.out.println("amortises tokenization, collation and the session call, and none of");+ System.out.println("those is the cost here -- the matmuls are, and they do not amortise.");+ System.out.println("Batching pays on a GPU, and on CPU it pays for small models.");+ }++ /** One untimed-warmup-free call, for the cold number. */+ private static long time(Supplier<?> work) {+ long started = System.nanoTime();+ Object result = work.get();+ long elapsed = System.nanoTime() - started;+ if (result == null) {+ throw new IllegalStateException("the work returned nothing");+ }+ return elapsed;+ }++ private static void bench(String label, Supplier<?> work) {+ bench(label, work, WARMUP, ROUNDS);+ }++ private static void bench(String label, Supplier<?> work, int warmup, int rounds) {+ System.out.printf(" %-34s %8.3f ms%n", label, best(work, warmup, rounds) / 1e6);+ }++ /** Best of {@code rounds}, after {@code warmup} calls, with every result consumed. */+ private static long best(Supplier<?> work, int warmup, int rounds) {+ for (int i = 0; i < warmup; i++) {+ consume(work.get());+ }+ long best = Long.MAX_VALUE;+ for (int round = 0; round < rounds; round++) {+ long started = System.nanoTime();+ Object result = work.get();+ best = Math.min(best, System.nanoTime() - started);+ consume(result);+ }+ return best;+ }++ /** Keep the JIT from deciding the work is dead and removing it. */+ private static void consume(Object result) {+ if (result != null && result.hashCode() == Integer.MIN_VALUE) {+ System.out.print("");+ }+ }+}diff --git a/laya-java/samples/Quickstart.java b/laya-java/samples/Quickstart.javanew file mode 100644index 00000000..df3ae499--- /dev/null+++ b/laya-java/samples/Quickstart.java@@ -0,0 +1,84 @@+// The SDK quickstart as a runnable program: four typed questions about one support email, in one+// forward pass, read out as typed answers.+//+// Run it with the single-file launcher -- no build, no module, no Gradle:+//+// python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual+// cd laya-java && ./gradlew --no-daemon jar+// java -cp "laya-java/build/libs/laya-java-0.1.0-SNAPSHOT.jar:$(cat samples/.onnx-classpath)" \+// samples/Quickstart.java .work/checkpoints/multilingual .work/onnx/multilingual+//+// See samples/README.md for the classpath, which is the only fiddly part.++import com.convaiinnovations.laya.Agent;+import com.convaiinnovations.laya.Answer;+import com.convaiinnovations.laya.Prediction;+import com.convaiinnovations.laya.Presets;+import com.convaiinnovations.laya.Question;+import java.nio.file.Path;+import java.util.LinkedHashMap;+import java.util.Map;++public final class Quickstart {++ public static void main(String[] args) throws Exception {+ if (args.length < 2) {+ System.err.println("usage: Quickstart <checkpoint-dir> <graph-dir>");+ System.exit(2);+ }+ Path checkpoint = Path.of(args[0]);+ Path graph = Path.of(args[1]);++ // The questions are plain data. The same objects answer an English state and a Hindi one;+ // nothing about them is language-specific, which is the point of the preset.+ Map<String, Question> questions = Presets.email();++ // A state is a map, and every field is shown to the model. `subject` and `body` here+ // because that is what the email preset is written against.+ Map<String, Object> english = new LinkedHashMap<>();+ english.put("subject", "Charged twice for order 8812");+ english.put("body", "I was billed two times for the same order this morning. "+ + "Please refund one of the charges. I have attached both receipts.");++ Map<String, Object> hindi = new LinkedHashMap<>();+ hindi.put("subject", "ऑर्डर 8812 का "+ + "दो बार बिल");+ hindi.put("body", "मुझे उसी ऑर्डर "+ + "के लिए दो बार "+ + "बिल किया गया। "+ + "कृपया रिफंड करें।");++ // try-with-resources: the agent owns a native ONNX session, and closing it is not optional.+ try (Agent agent = Agent.open(checkpoint, graph)) {+ report("English", agent.predict(english, questions));+ report("Hindi", agent.predict(hindi, questions));+ }+ }++ /** Print every answer by its actual type, which is what the sealed interface is for. */+ private static void report(String label, Prediction prediction) {+ System.out.println();+ System.out.println("== " + label + " ==");+ for (String id : prediction.answers().keySet()) {+ Answer answer = prediction.answer(id);+ // A choice is not a score. Pattern matching over the sealed interface means the+ // compiler checks that every shape is handled, rather than a map lookup that cannot.+ if (answer instanceof Answer.Choice choice) {+ System.out.printf(" %-12s %-22s confidence %.3f%n",+ id, choice.choice(), choice.confidence());+ } else if (answer instanceof Answer.Score score) {+ System.out.printf(" %-12s %-22.3f confidence %.3f%n",+ id, score.score(), score.confidence());+ } else if (answer instanceof Answer.Noul noul) {+ System.out.printf(" %-12s %-22.3f confidence %.3f%n",+ id, noul.noul(), noul.confidence());+ }+ }+ // Usage is not decoration: it says whether the model actually saw the state, and a+ // truncated request names the question that was cut.+ System.out.printf(" usage %d input tokens, %d state tokens dropped, truncated=%s%n",+ prediction.usage().inputTokens(),+ prediction.usage().stateTokensDropped(),+ prediction.usage().truncated());+ }+}diff --git a/laya-java/samples/README.md b/laya-java/samples/README.mdnew file mode 100644index 00000000..caee0003--- /dev/null+++ b/laya-java/samples/README.md@@ -0,0 +1,98 @@+# laya-java samples++Three runnable programs. They are **single-file source programs**, not a Gradle module — run them+with `java Foo.java` and no build step of their own.++| | needs | shows |+|---|---|---|+| [`Quickstart.java`](Quickstart.java) | a checkpoint **and** a graph | four typed questions in one forward pass, in English and Hindi |+| [`Routing.java`](Routing.java) | both checkpoints; a graph for the one it routes to | the routing decision and its reason, the load-and-evict lifecycle, and shortlisting 400 labels |+| [`Benchmark.java`](Benchmark.java) | nothing, or a checkpoint and a graph | what the SDK costs, warm and best-of-N |++`Benchmark.java` runs with no arguments and still measures something: detection and the email+cleaner need no model at all.++## Why single files and not a module++A `samples` subproject would inherit the root build's `maven-publish` and `signing`+configuration, which creates a `MavenPublication` for every subproject — and+`publishAllPublicationsToCentralRepository` would push it. That is not hypothetical: the empty+`laya-java-client` module was removed from `settings.gradle.kts` for exactly this reason, because+a version burned on Maven Central cannot be replaced. Samples are demonstration code and have no+business being publishable, so they stay out of the build graph.++They are still compiled. The `jvm build + model-free tests` CI lane compiles all three against the+built classes, so a sample that stops matching the API fails the build rather than rotting.++## Running them++You need two things on the classpath: the SDK and ONNX Runtime.++```bash+# 1. the artifacts the SDK reads -- see ../MODELS.md+python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual++# 2. the SDK jar, and ONNX Runtime from the Gradle cache+cd laya-java+./gradlew --no-daemon :laya-java:jar++JAR=$(ls laya-java/build/libs/laya-java-*.jar | head -1)+ORT=$(find ~/.gradle/caches/modules-2 -name 'onnxruntime-1.20.0.jar' | head -1)++# 3. run+java -cp "$JAR:$ORT" samples/Quickstart.java \+ .work/checkpoints/multilingual .work/onnx/multilingual+```++`java Foo.java` compiles in memory and runs `main`; nothing is written to disk. It needs JDK 11 or+newer, and the SDK targets 17.++If `find` turns up nothing for ONNX Runtime, resolve it once:++```bash+./gradlew --no-daemon :laya-java:dependencies --configuration runtimeClasspath | grep onnxruntime+```++## What each one is for++### `Quickstart.java`++The smallest program that covers the whole path: open a checkpoint, build a state, ask the preset+email questions, read typed answers back. It asks the **same question objects** about an English+state and a Hindi one, which is the point of a question set being plain data.++It reads answers by pattern-matching the sealed `Answer` interface rather than by looking up a+map, because a choice is not a score and the compiler can check that every shape is handled.++It also prints `usage`. That is not decoration: it says whether the model actually saw the state,+and a truncated request names the question that was cut.++### `Routing.java`++Two halves that fail in opposite directions.++`route` needs no model and loads nothing, so the first half just prints the decision and its+reason for four states in four scripts. The reason string is what you will read in a log when a+request went somewhere you did not expect.++The second half gives the router an `AgentFactory` and `maxLoaded(1)`, so the load-and-evict+lifecycle is visible: it loads on demand, keeps one checkpoint resident, and closes the least+recently used — but never one a caller still holds a lease on.++The third part shortlists 400 labels down to 5 with a deliberately terrible embedder: hashed+character trigrams, which have no notion of meaning and match on shared spelling. It is there to+make the shape clear without pulling in a model. **In a real program that is your bi-encoder, and+the quality of the shortlist is the quality of that encoder** — a shortlist is a filter in front+of the model, so a bad one removes the right answer before the model can be asked about it.++### `Benchmark.java`++Warm, best-of-N, with the cold first call reported separately instead of averaged in.++A first measurement on a cold JVM is dominated by class loading, JIT, and — for this SDK — the+one-time walk of Unicode that builds the email cleaner's character classes. Quoting that as+throughput is how a port comes to look slower than it is. Quoting a single warm run is how noise+comes to look like a regression. So the samples' own numbers are the best of 5 after 20 warmup+calls, and the two cold figures are printed as cold.++It also reports the batch cost **per state**, which is the number to use when sizing a queue.diff --git a/laya-java/samples/Routing.java b/laya-java/samples/Routing.javanew file mode 100644index 00000000..948d9728--- /dev/null+++ b/laya-java/samples/Routing.java@@ -0,0 +1,147 @@+// Routing and shortlisting: pick a checkpoint from the state's own script, then cut a large label+// set down before the model sees it.+//+// The two go together because they fail in opposite directions. The English checkpoint does not+// gently degrade off English -- on 20-option intent it scores 0.100 on Hindi against 0.050 for+// random guessing, and reports high confidence while doing it -- so the router's job is to never+// send it text it cannot read. The shortlist's job is the other end: a question with 400 options+// does not fit a context window, so something has to choose which 20 the model is asked about,+// and if that choice is wrong the right answer was never on the ballot.+//+// python laya-java/scripts/prepare_checkpoint.py --checkpoint multilingual+// java -cp "<jar>:<onnxruntime>" samples/Routing.java .work/checkpoints .work/onnx+//+// See samples/README.md for the classpath.++import com.convaiinnovations.laya.Agent;+import com.convaiinnovations.laya.Question;+import com.convaiinnovations.laya.Router;+import com.convaiinnovations.laya.Shortlist;+import java.nio.file.Path;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;++public final class Routing {++ public static void main(String[] args) throws Exception {+ if (args.length < 2) {+ System.err.println("usage: Routing <checkpoints-root> <onnx-root>");+ System.exit(2);+ }+ Path checkpoints = Path.of(args[0]);+ Path graphs = Path.of(args[1]);++ // ---------------------------------------------------------------- the decision alone+ //+ // `route` needs no model and loads nothing. It is worth looking at on its own, because+ // the reason string is the part you will read in a log when a request went to a+ // checkpoint you did not expect.+ try (Router router = Router.withDefaults()) {+ for (String state : List.of(+ "I cannot log in and need a password reset.",+ "मुझे अपना "+ + "पासवर्ड बदलना "+ + "है।",+ "Je n'arrive pas à me connecter.",+ "注文をキャンセルしたいです。")) {+ Router.RouteDecision decision = router.route(state);+ System.out.printf("%-10s %s%n", decision.model().wireName(), decision.reason());+ }+ }++ // ---------------------------------------------------------------- routing a real request+ //+ // `agents` is how the router opens a checkpoint. It is a factory rather than a path so+ // that the lifecycle stays the router's: it loads on demand, keeps at most `maxLoaded`+ // resident, and closes the least recently used one -- but never one a caller still holds.+ System.out.println();+ try (Router router = Router.builder()+ .maxLoaded(1) // deliberately 1, so the eviction is visible+ .agents(checkpoint -> Agent.open(+ checkpoints.resolve(checkpoint.wireName()),+ graphs.resolve(checkpoint.wireName())))+ .build()) {++ // A choice is a MAP of label to criterion, rendered in the map's iteration order.+ // A criterion of "" means "no description", and the option renders as its label.+ Map<String, String> intents = new LinkedHashMap<>();+ intents.put("reset a password", "the sender cannot sign in");+ intents.put("cancel an order", "the sender wants an order stopped");+ intents.put("request a refund", "the sender wants money back");+ intents.put("report a bug", "something is broken");+ intents.put("something else", "");+ Map<String, Question> questions = new LinkedHashMap<>();+ questions.put("intent", Question.choice("What does the sender want?", intents));++ // A Portuguese state, deliberately: it routes to multilingual, so this sample needs+ // only ONE graph. An English state would route to the english checkpoint and ask you+ // to export a second 1.3 GB artifact to see the same thing.+ Map<String, Object> state = new LinkedHashMap<>();+ state.put("subject", "Nao consigo entrar na conta");+ state.put("body", "Minha senha parou de funcionar depois da atualizacao de hoje.");++ // One call: route, load if needed, predict. The decision is reported alongside.+ Router.RouteDecision decision = router.route(state, questions);+ System.out.println("routed to " + decision.model().wireName()+ + " because: " + decision.reason());+ System.out.println("answer: "+ + router.predict(decision, state, questions).answer("intent"));+ }++ // ---------------------------------------------------------------- shortlisting+ //+ // A deliberately terrible embedder: hashed character trigrams. It is here to make the+ // shape clear without pulling in a model -- in a real program this is your bi-encoder,+ // and the quality of the shortlist is the quality of that encoder, not of laya.+ System.out.println();+ Map<String, String> departments = new LinkedHashMap<>();+ for (int i = 1; i <= 400; i++) {+ departments.put("department-" + i, "");+ }+ departments.put("billing disputes and duplicate charges", "refunds and double charges");++ Shortlist.Embedder embedder = Routing::hashedTrigrams;+ Question question = Question.choice("Which team should own this?", departments);+ Shortlist.Ranking ranking = Shortlist.rank(+ "I was charged twice for the same order and want one charge refunded.",+ question, embedder, 5);++ System.out.printf("shortlisted %d of %d labels (passthrough=%s)%n",+ ranking.labels().size(), ranking.total(), ranking.passthrough());+ for (int i = 0; i < ranking.labels().size(); i++) {+ System.out.printf(" %.4f %s%n", ranking.scores()[i], ranking.labels().get(i));+ }+ System.out.println();+ System.out.println("The real answer is on the shortlist only because the embedder put it");+ System.out.println("there. A shortlist is a filter in front of the model, so a bad one");+ System.out.println("removes the right answer before the model can be asked about it.");+ }++ /**+ * Hashed character trigrams, L2-normalised. A demo, not a recommendation: it has no notion of+ * meaning and matches on shared spelling.+ */+ private static double[][] hashedTrigrams(List<String> texts) {+ int width = 256;+ double[][] rows = new double[texts.size()][width];+ for (int r = 0; r < texts.size(); r++) {+ String text = texts.get(r).toLowerCase(java.util.Locale.ROOT);+ for (int i = 0; i + 3 <= text.length(); i++) {+ int bucket = Math.abs(text.substring(i, i + 3).hashCode()) % width;+ rows[r][bucket] += 1.0;+ }+ double norm = 0.0;+ for (double v : rows[r]) {+ norm += v * v;+ }+ norm = Math.sqrt(norm);+ if (norm > 0.0) {+ for (int c = 0; c < width; c++) {+ rows[r][c] /= norm;+ }+ }+ }+ return rows;+ }+}