Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(java): typed-decisions end to end, and the pairing that answers anyway
Adeshina Lasisiba028702ebac
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex b8a8ac0c..7b5a117d 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -333,15 +333,23 @@ jobs:# and is the whole runtime of this lane. Running them in one job made a 13-second check wait on# a 40-minute export.#- # NOT split per checkpoint, which is where the .NET lane splits. Two reasons, both measured.- # The fixtures are not per-checkpoint documents -- `tokenizer_ids.json`, `sequences.json` and- # `decode.json` each carry BOTH checkpoints' sections in one file, so a job holding one of them- # cannot regenerate any of the three and `--strict` fails by construction; that is what- # happened on the first run of this lane. And the Java tests assert that both checkpoints' cases- # are present, a floor that exists because a @TestFactory whose fixture key emptied used to- # generate zero assertions and still exit 0. Splitting per checkpoint would mean deleting that- # floor to buy nothing: the two tokenizers together are a few megabytes, so there is no export- # to parallelise away.+ # The GRAPHLESS cell is not split per checkpoint, which is where the .NET lane splits. Two+ # reasons, both measured. The fixtures it owns are not per-checkpoint documents --+ # `tokenizer_ids.json`, `sequences.json` and `decode.json` each carry BOTH checkpoints'+ # sections in one file, so a job holding one of them cannot regenerate any of the three and+ # `--strict` fails by construction; that is what happened on the first run of this lane. And+ # the Java tests assert that both checkpoints' cases are present, a floor that exists because a+ # @TestFactory whose fixture key emptied used to generate zero assertions and still exit 0.+ # Splitting it per checkpoint would mean deleting that floor to buy nothing: the two tokenizers+ # together are a few megabytes, so there is no export to parallelise away.+ #+ # The WEIGHT-BEARING cells are one per checkpoint, and have to be. A graph is traced from+ # exactly one checkpoint and carries its weights, so `predict.json` (multilingual) and+ # `typed_decisions.json` (typed-decisions) need two graphs; a cell holding both would pay both+ # exports serially, which is the cost this whole split exists to avoid. Each such cell declares+ # the checkpoint it traces in `graph-model`, which is also what keys its export cache and which+ # of the two graph variables gets a path -- so adding a third never silently reuses the second's+ # graph.parity:name: jvm parity (${{ matrix.name }})runs-on: ubuntu-latest@@ -359,7 +367,7 @@ jobs:# 40-minute export for nothing.- name: tokenizercheckpoints: english multilingual- graph: false+ graph-model: "" # nothing traced herefamilies: >---only tokenizer_ids.json --only sequences.json --only decode.json--only decode_text.json --only window_plan.json --only predict_long.json@@ -385,15 +393,44 @@ jobs:# assertion in TokenDecoderTest and the `cases.size() >= 23` corpus floor beside it.min-tests: 489timeout: 25- # The one cell that needs the traced graph, and the only checkpoint the end-to-end- # golden was recorded from.+ # Weight-bearing: the multilingual end-to-end golden, traced from that checkpoint.- name: predictcheckpoints: multilingual- graph: true+ graph-model: multilingualfamilies: --only predict.jsontests: --tests com.convaiinnovations.laya.PredictParityTestmin-tests: 9timeout: 75+ # Weight-bearing, and a SEPARATE cell rather than a second family in the one above.+ # Three reasons, in order of how much they cost:+ #+ # * A graph is traced from one checkpoint. `predict.json` is recorded against+ # multilingual and `typed_decisions.json` against typed-decisions, so one cell+ # holding both would export two graphs and run two exports serially -- the 40+ # minutes this matrix was split to avoid, paid twice in one cell instead of once+ # in each of two that run in parallel.+ # * The export cache is keyed on `graph-model`, so each checkpoint's graph is+ # invalidated by its own revision bump. A merged cell would key two graphs under+ # one entry and re-export both whenever either moved.+ # * These two goldens fail for unrelated reasons. A merged cell reports one status+ # for both, and the first thing a reader of a red parity lane needs to know is+ # which checkpoint moved.+ #+ # Cost, measured locally with the Hub blobs already cached: 846 MB of checkpoint, a+ # 39 s ONNX export producing 3.6 MB of graph beside 1.7 GB of external weights, 103 s+ # to record the family and 109 s for the 13 tests. On a cold runner the download is+ # the open variable and is why this carries the same 75-minute ceiling as `predict`.+ - name: typed-decisions+ checkpoints: typed-decisions+ graph-model: typed-decisions+ families: --only typed_decisions.json+ tests: --tests com.convaiinnovations.laya.TypedDecisionsParityTest+ # Measured locally: 13 -- 8 single-state cases, 3 batch configs, the config identity+ # check and the workflow-signature check. Exact rather than padded down, because the+ # two non-factory tests are what catch a wrong checkpoint/graph pairing and a margin+ # of 2 would let both be dropped with this floor still green.+ min-tests: 13+ timeout: 75steps:- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0# Validate the wrapper jar BEFORE anything can execute it. See this file's@@ -435,16 +472,20 @@ jobs:# the revision stamp now makes safe) and accepting a multi-gigabyte entry for the# weight-bearing cell; neither is this change's call to make.- name: Cache the exported graph- if: ${{ matrix.graph }}+ if: ${{ matrix.graph-model != '' }}uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0with:path: laya-java/.work/onnx- key: laya-java-onnx-${{ runner.os }}-${{ matrix.name }}-${{ hashFiles('laya-java/scripts/prepare_checkpoint.py', 'scripts/export_onnx.py', 'laya/common.py', 'laya-dotnet/tools/requirements-regen.txt') }}+ key: laya-java-onnx-${{ runner.os }}-${{ matrix.graph-model }}-${{ hashFiles('laya-java/scripts/prepare_checkpoint.py', 'scripts/export_onnx.py', 'laya/common.py', 'laya-dotnet/tools/requirements-regen.txt') }}+ # A graph is exported for the ONE checkpoint this cell's golden was recorded from, and+ # every other checkpoint it needs is fetched `--no-graph`. Keyed on the name rather than on+ # a boolean, because `graph: true` could not say WHICH checkpoint to trace, and a cell that+ # names two would otherwise have traced both.- name: Prepare the checkpoints this cell ownsrun: |for c in ${{ matrix.checkpoints }}; do- if [ "${{ matrix.graph }}" = "true" ]; then+ if [ "$c" = "${{ matrix.graph-model }}" ]; thenpython laya-java/scripts/prepare_checkpoint.py --checkpoint "$c"elsepython laya-java/scripts/prepare_checkpoint.py --checkpoint "$c" --no-graph@@ -458,13 +499,19 @@ jobs:- name: Regenerate this cell's fixtures from Pythonenv:LAYA_FIXTURE_CHECKPOINTS: ${{ github.workspace }}/laya-java/.work/checkpoints- LAYA_ONNX_GRAPH: ${{ matrix.graph && format('{0}/laya-java/.work/onnx/multilingual/laya.onnx', github.workspace) || '' }}+ # One variable per golden, and only this cell's is populated. They cannot share one:+ # a graph carries the weights and nothing in it says which checkpoint traced it, so+ # `predict.json` pointed at the typed-decisions graph would re-record itself from the+ # wrong weights and compare them against themselves. Measured: the english checkpoint+ # against the typed graph raises nothing and answers to the same last digit.+ LAYA_ONNX_GRAPH: ${{ matrix.graph-model == 'multilingual' && format('{0}/laya-java/.work/onnx/multilingual/laya.onnx', github.workspace) || '' }}LAYA_PREDICT_MODEL: multilingual+ LAYA_TYPED_ONNX_GRAPH: ${{ matrix.graph-model == 'typed-decisions' && format('{0}/laya-java/.work/onnx/typed-decisions/laya.onnx', github.workspace) || '' }}run: python laya-java/scripts/gen_fixtures.py --strict ${{ matrix.families }}# The pre-tokenizer's letter and number classes are probed out of the reference tokenizer,# so this is the one generator gate that needs a checkpoint. It belongs to the cheap cell.- name: Pre-tokenizer class tables match the reference- if: ${{ !matrix.graph }}+ if: ${{ matrix.graph-model == '' }}run: >python laya-java/scripts/gen_pretokenizer_tables.py --check--tokenizer laya-java/.work/checkpoints/english/tokenizer/tokenizer.json@@ -479,7 +526,8 @@ jobs:working-directory: laya-javaenv:LAYA_CHECKPOINTS: ${{ github.workspace }}/laya-java/.work/checkpoints- LAYA_ONNX_GRAPH: ${{ matrix.graph && format('{0}/laya-java/.work/onnx/multilingual/laya.onnx', github.workspace) || '' }}+ LAYA_ONNX_GRAPH: ${{ matrix.graph-model == 'multilingual' && format('{0}/laya-java/.work/onnx/multilingual/laya.onnx', github.workspace) || '' }}+ LAYA_TYPED_ONNX_GRAPH: ${{ matrix.graph-model == 'typed-decisions' && format('{0}/laya-java/.work/onnx/typed-decisions/laya.onnx', github.workspace) || '' }}run: ./gradlew --no-daemon test ${{ matrix.tests }}# A checkpoint-backed factory that aborts is not a pass. With this cell's checkpoints present# nothing it runs may abort, and the floor catches a suite that silently stopped registeringdiff --git a/laya-java/CHANGELOG.md b/laya-java/CHANGELOG.mdindex ef9dba65..13de5510 100644--- a/laya-java/CHANGELOG.md+++ b/laya-java/CHANGELOG.md@@ -67,14 +67,26 @@ see [Installing](README.md#installing).### Testing- 769 tests, 626 of which need no model at all.+- `typed-decisions` is covered end to end by `TypedDecisionsParityTest` against+ `fixtures/typed_decisions.json`: **13 tests** with the checkpoint present — eight single-state+ cases spanning all four fine-tuned workflow signatures, three batch configurations, and two+ checks on the weights themselves. Four of the 13 register without a checkpoint and one of those+ passes there, since the workflow-signature check reads only the fixture.+- That second pair of checks is not ceremony. `typed-decisions` is fine-tuned from `english` and+ ships its `tokenizer.json` byte for byte and the same encoder, so opening the english+ checkpoint against the typed graph raises nothing and **seven of the eight** single-state cases+ still pass. The eighth is `over-budget`, which only truncates differently because `max_len` is+ 512 there and 1024 here; the config check is what names the problem, and it runs before any+ probability is compared.- Every expectation is **generated from the Python package**, never written by hand:`laya-java/fixtures/*.json`, regenerated by `scripts/gen_fixtures.py` and gated by `--check` inCI so a change to `laya/` that moves a recorded contract shows up here rather than as drift.- The compiled-in Unicode tables are checked by a **sha256 over all 1,114,112 code points** perproperty, so they are proven exhaustively rather than on the cases someone thought to write.-- Six CI lanes: build plus model-free tests, the same tests on JDK 17, 21 and 24 — three- different Unicode versions, 13.0, 15.0 and 16.0 — and two parity cells split by whether they- need a traced graph.+- Seven CI lanes: build plus model-free tests, the same tests on JDK 17, 21 and 24 — three+ different Unicode versions, 13.0, 15.0 and 16.0 — and three parity cells split by whether they+ need a traced graph, and then by which checkpoint that graph was traced from. A graph is+ traced from exactly one checkpoint and carries its weights, so the cells cannot share one.- `-PtestJavaVersion` runs the tests on a different JDK from the one the classes are compiledfor, because the toolchain pins the compiler to 17 and a lane that merely installs another JDKtests the same thing twice.@@ -89,9 +101,10 @@ see [Installing](README.md#installing).- Not implemented: hooks, `predictLong`, structured `decide`, the `laya-java-client` HTTP module,Android.-- `typed-decisions` is routable — `Router` knows it and resolves its aliases — but has no recorded- end-to-end fixtures. The routing decision is tested; a forward pass against that checkpoint is- not.+- Only `invoice_processing`'s trained question schema is in this repository, so the other three+ typed-decisions workflows are recorded by their question-id signature with instruction text+ written for the fixture. The ids are what routing reads and the text is recorded verbatim, so+ the forward pass is real; the trained wording is not claimed to be.- The softmax is computed in `double` where Python computes it in float32, as `laya-ts` and`laya-dotnet` also do. Over 200,000 random logit rows the four-decimal probabilities differ on88 of them. Rounding, by contrast, is exact: `BigDecimal` at scale 4 with `HALF_EVEN` isdiff --git a/laya-java/MODELS.md b/laya-java/MODELS.mdindex 06ec071a..6905e492 100644--- a/laya-java/MODELS.md+++ b/laya-java/MODELS.md@@ -54,11 +54,44 @@ checkpoint the Python package knows about is one this can fetch.`typed-decisions` is never selected automatically. `Router` knows it, and the aliases `typed`,`typed_decisions`, `decisions` and `laya-typed-decisions` all resolve to it, but reaching it takesan explicit `task` or `autoTaskDetection` — it is fine-tuned on four specific workflows and is the-wrong silent default. **It has no recorded end-to-end fixtures in this port yet**: the routing-decision is tested, the forward pass against that checkpoint is not.+wrong silent default.-A prepared checkpoint is about **680–810 MB** on disk, and the fused multilingual graph is-**1.29 GB** (`laya.onnx` is a 2.8 MB graph beside a `laya.onnx.data` of external weights). Budget+Its forward pass **is** recorded, in `fixtures/typed_decisions.json`, and asserted by+`TypedDecisionsParityTest` in the `jvm parity (typed-decisions)` cell: eight single-state cases+covering all four workflow signatures, three batch configurations, and the two checks that the+weights under test are the right ones. Regenerate it with:++```bash+python laya-java/scripts/prepare_checkpoint.py --checkpoint typed-decisions+LAYA_TYPED_ONNX_GRAPH=laya-java/.work/onnx/typed-decisions/laya.onnx \+ python laya-java/scripts/gen_fixtures.py --only typed_decisions.json+```++`LAYA_TYPED_ONNX_GRAPH`, not `LAYA_ONNX_GRAPH`, and the two must never be crossed. A graph+carries the weights and nothing in it names the checkpoint it was traced from, so a wrong pairing+answers instead of failing — `typed-decisions` is fine-tuned **from** `english` and ships its+`tokenizer.json` byte for byte and its ModernBERT-large encoder. Measured: the english checkpoint+opened against the typed-decisions graph raises nothing and returns+`{refund: 0.9077, replace: 0.0923}`, the same digits as the correct pairing, and `model` reads+`laya-rl-agent-onnx` either way. Measured with the english checkpoint symlinked in as+`typed-decisions/`: **seven of the eight** single-state cases still pass, and the eighth,+`over-budget`, fails only because `max_len` is 512 there and 1024 here so the state truncates at+a different point. What names the problem is the `config` section of the golden — `max_len` 1024+against 512, `head_max_len` 256 against 192, base temperatures `[1.0148, 1.0374, 1.0575]` against+`[1.6369, 1.2514, 1.9834]` — which the test asserts before it compares a probability. Only the+multilingual checkpoint fails loudly against that graph, and for an unrelated reason: a 235292+token id against a 50368-row embedding, which ONNX Runtime rejects at the `node_embedding` Gather.++Of the four workflows, only `invoice_processing`'s trained question schema is in this repository+(`examples/32_typed_decisions_workflow.py`). The other three are known by their question-id+signatures alone, from `laya.router._TYPED_DECISION_WORKFLOWS`, so the golden carries those ids+with instruction text written for the fixture. The ids are what the routing decision reads and the+text is recorded verbatim, so both halves are real; the trained wording is not claimed.++A prepared checkpoint is about **680–850 MB** on disk, and the fused multilingual graph is+**1.29 GB** (`laya.onnx` is a 2.8 MB graph beside a `laya.onnx.data` of external weights). The+typed-decisions checkpoint is **846 MB** and its graph is **1.69 GB** (3.6 MB beside the external+weights); the export took **39 s** locally, and recording the family another **103 s**. Budgetaccordingly; the export is the slow part.---diff --git a/laya-java/fixtures/typed_decisions.json b/laya-java/fixtures/typed_decisions.jsonnew file mode 100644index 00000000..6e8a202d--- /dev/null+++ b/laya-java/fixtures/typed_decisions.json@@ -0,0 +1,2868 @@+{+ "checkpoint": "typed-decisions",+ "graph": "laya.onnx",+ "config": {+ "model_name": "laya-typed-decisions",+ "encoder": "answerdotai/ModernBERT-large",+ "max_len": 1024,+ "head_max_len": 256,+ "temperature": [+ 1.0148024559020996,+ 1.0374259948730469,+ 1.0575125217437744+ ],+ "temperature_by_options": {+ "choice:3-5": 1.7601518630981445,+ "choice:6-10": 1.0000158548355103,+ "score:3-5": 1.2514300346374512,+ "noul:2": 1.983399510383606,+ "choice:11+": 0.5,+ "choice:2": 1.9063563346862793+ },+ "temperature_raw": [+ 1.0148024559020996,+ 1.0374259948730469,+ 1.0575125217437744+ ],+ "temperature_by_options_raw": {+ "choice:3-5": 1.7601518630981445,+ "choice:6-10": 1.0000158548355103,+ "score:3-5": 1.2514300346374512,+ "noul:2": 1.983399510383606,+ "choice:11+": 0.10058280825614929,+ "choice:2": 1.9063563346862793+ }+ },+ "workflows": {+ "agent_trace_observability": [+ "action",+ "needs_review",+ "outcome",+ "risk",+ "urgency"+ ],+ "customer_service": [+ "action",+ "category",+ "churn_risk",+ "needs_human",+ "urgency"+ ],+ "invoice_processing": [+ "discrepancy_severity",+ "disposition",+ "duplicate",+ "matches_order",+ "urgency"+ ],+ "security_incidents": [+ "credential_compromise",+ "disposition",+ "severity",+ "true_positive",+ "urgency"+ ]+ },+ "single": {+ "invoice-variance": {+ "state": {+ "invoice": {+ "id": "INV-2026-8842",+ "vendor": "Northwind Castings",+ "currency": "USD",+ "total_usd": 300020.0,+ "lines": [+ {+ "sku": "SKU-940",+ "qty": 1000,+ "unit_usd": 300.02,+ "total_usd": 300020.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3392",+ "total_usd": 214300.0,+ "lines": [+ {+ "qty": 1000,+ "unit_usd": 214.3+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-18",+ "received_qty": 1000,+ "condition": "accepted with exceptions"+ },+ "payment": {+ "terms": "net 45",+ "status": "overdue",+ "days_until_due": -3+ },+ "vendor_history": {+ "invoices_12m": 40,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-6633",+ "INV-2026-9563",+ "INV-2026-7939"+ ]+ }+ },+ "lang": null,+ "workflow": "invoice_processing",+ "questions": {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."+ }+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile."+ }+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted."+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day."+ ]+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.0908,+ "hold": 0.3301,+ "manual_review": 0.4534,+ "reject": 0.1257+ },+ "confidence": 0.1322,+ "answer_confidence": 0.4534,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.2063,+ "confidence": 0.7937,+ "answer_confidence": 0.7937,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1399,+ "confidence": 0.8601,+ "answer_confidence": 0.8601,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 2.3799,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.0405,+ "1": 0.0602,+ "2": 0.378,+ "3": 0.5212+ },+ "confidence": 0.2739,+ "answer_confidence": 0.5212,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 2.1864,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0226,+ "1": 0.1418,+ "2": 0.4621,+ "3": 0.3735+ },+ "confidence": 0.2157,+ "answer_confidence": 0.4621,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1666,+ "output_tokens": 0,+ "state_tokens": 274,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ "invoice-clean": {+ "state": {+ "invoice": {+ "id": "INV-2026-9107",+ "vendor": "Cedar Freight",+ "currency": "USD",+ "total_usd": 18450.0,+ "lines": [+ {+ "sku": "SKU-221",+ "qty": 300,+ "unit_usd": 61.5,+ "total_usd": 18450.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3477",+ "total_usd": 18450.0,+ "lines": [+ {+ "qty": 300,+ "unit_usd": 61.5+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-20",+ "received_qty": 300,+ "condition": "accepted in full"+ },+ "payment": {+ "terms": "net 30",+ "status": "current",+ "days_until_due": 18+ },+ "vendor_history": {+ "invoices_12m": 12,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-7781"+ ]+ }+ },+ "lang": null,+ "workflow": "invoice_processing",+ "questions": {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."+ }+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile."+ }+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted."+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day."+ ]+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.4076,+ "hold": 0.2312,+ "manual_review": 0.2092,+ "reject": 0.152+ },+ "confidence": 0.0492,+ "answer_confidence": 0.4076,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6562,+ "confidence": 0.6562,+ "answer_confidence": 0.6562,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1452,+ "confidence": 0.8548,+ "answer_confidence": 0.8548,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.0056,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.4979,+ "1": 0.1355,+ "2": 0.2296,+ "3": 0.1369+ },+ "confidence": 0.1141,+ "answer_confidence": 0.4979,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5989,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0413,+ "1": 0.441,+ "2": 0.3952,+ "3": 0.1225+ },+ "confidence": 0.1945,+ "answer_confidence": 0.441,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1556,+ "output_tokens": 0,+ "state_tokens": 252,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ "customer-service": {+ "state": "We were billed twice for March. Third time this year -- refund it today or we are moving to a competitor.",+ "lang": null,+ "workflow": "customer_service",+ "questions": {+ "action": {+ "type": "choice",+ "instructions": "What should the support team do next?",+ "criteria": {+ "refund": "Return the money and close the ticket.",+ "replace": "Ship a replacement unit.",+ "explain": "Answer the question; nothing to send.",+ "escalate": "Hand to a senior agent."+ }+ },+ "category": {+ "type": "choice",+ "instructions": "Which queue does this ticket belong in?",+ "criteria": {+ "billing": "Charges, invoices and refunds.",+ "shipping": "Delivery, tracking and damage in transit.",+ "account": "Login, access and account settings."+ }+ },+ "churn_risk": {+ "type": "score",+ "instructions": "How likely is this customer to leave over this?",+ "criteria": [+ "Not at risk; routine contact.",+ "Mildly annoyed.",+ "Considering alternatives.",+ "Explicitly threatening to cancel."+ ]+ },+ "needs_human": {+ "type": "noul",+ "instructions": "This ticket needs a human agent rather than an automated reply."+ },+ "urgency": {+ "type": "score",+ "instructions": "How quickly must this be answered?",+ "criteria": [+ "No time pressure.",+ "Within the normal queue.",+ "Same week.",+ "Same day."+ ]+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "action": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.6664,+ "replace": 0.0648,+ "explain": 0.0534,+ "escalate": 0.2154+ },+ "confidence": 0.3256,+ "answer_confidence": 0.6664,+ "action": {+ "act_probability": 1.0+ }+ },+ "category": {+ "type": "choice",+ "choice": "billing",+ "probabilities": {+ "billing": 0.8789,+ "shipping": 0.0432,+ "account": 0.0779+ },+ "confidence": 0.5922,+ "answer_confidence": 0.8789,+ "action": {+ "act_probability": 1.0+ }+ },+ "churn_risk": {+ "type": "score",+ "score": 2.4863,+ "legend": {+ "0": "Not at risk; routine contact.",+ "1": "Mildly annoyed.",+ "2": "Considering alternatives.",+ "3": "Explicitly threatening to cancel."+ },+ "probabilities": {+ "0": 0.0101,+ "1": 0.0936,+ "2": 0.2962,+ "3": 0.6001+ },+ "confidence": 0.3256,+ "answer_confidence": 0.6001,+ "action": {+ "act_probability": 1.0+ }+ },+ "needs_human": {+ "type": "noul",+ "noul": 0.6657,+ "confidence": 0.6657,+ "answer_confidence": 0.6657,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 2.6585,+ "legend": {+ "0": "No time pressure.",+ "1": "Within the normal queue.",+ "2": "Same week.",+ "3": "Same day."+ },+ "probabilities": {+ "0": 0.0112,+ "1": 0.0364,+ "2": 0.2353,+ "3": 0.7172+ },+ "confidence": 0.4593,+ "answer_confidence": 0.7172,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 355,+ "output_tokens": 0,+ "state_tokens": 23,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ "security-incident": {+ "state": {+ "alert": "impossible travel",+ "user": "[email protected]",+ "signins": [+ {+ "city": "Lagos",+ "at": "2026-03-18T09:02Z"+ },+ {+ "city": "Vilnius",+ "at": "2026-03-18T09:41Z"+ }+ ],+ "mfa": "satisfied by push, approved on the second prompt",+ "prior_alerts_30d": 0+ },+ "lang": null,+ "workflow": "security_incidents",+ "questions": {+ "credential_compromise": {+ "type": "noul",+ "instructions": "Credentials belonging to this account were exposed or reused.",+ "criteria": {+ "false": "No sign that a secret left the organisation.",+ "true": "A password, token or key is in the wrong hands."+ }+ },+ "disposition": {+ "type": "choice",+ "instructions": "How should this alert be dispositioned?",+ "criteria": {+ "close": "Benign; close with no action.",+ "monitor": "Watch for a repeat before acting.",+ "contain": "Isolate the host or revoke the credential now.",+ "escalate": "Wake an incident responder."+ }+ },+ "severity": {+ "type": "score",+ "instructions": "How severe is the impact if this is real?",+ "criteria": [+ "Informational.",+ "Low: one account, no data.",+ "High: production data or lateral movement.",+ "Critical: active, spreading, or customer data."+ ]+ },+ "true_positive": {+ "type": "noul",+ "instructions": "This alert describes genuine malicious activity rather than noise."+ },+ "urgency": {+ "type": "score",+ "instructions": "How quickly must someone respond?",+ "criteria": [+ "No time pressure.",+ "Next business day.",+ "Within the hour.",+ "Immediately."+ ]+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "credential_compromise": {+ "type": "noul",+ "noul": 0.2801,+ "confidence": 0.7199,+ "answer_confidence": 0.7199,+ "action": {+ "act_probability": 1.0+ }+ },+ "disposition": {+ "type": "choice",+ "choice": "close",+ "probabilities": {+ "close": 0.3521,+ "monitor": 0.2603,+ "contain": 0.1785,+ "escalate": 0.2091+ },+ "confidence": 0.0242,+ "answer_confidence": 0.3521,+ "action": {+ "act_probability": 1.0+ }+ },+ "severity": {+ "type": "score",+ "score": 1.1706,+ "legend": {+ "0": "Informational.",+ "1": "Low: one account, no data.",+ "2": "High: production data or lateral movement.",+ "3": "Critical: active, spreading, or customer data."+ },+ "probabilities": {+ "0": 0.3117,+ "1": 0.3074,+ "2": 0.2794,+ "3": 0.1015+ },+ "confidence": 0.0518,+ "answer_confidence": 0.3117,+ "action": {+ "act_probability": 1.0+ }+ },+ "true_positive": {+ "type": "noul",+ "noul": 0.1774,+ "confidence": 0.8226,+ "answer_confidence": 0.8226,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.7277,+ "legend": {+ "0": "No time pressure.",+ "1": "Next business day.",+ "2": "Within the hour.",+ "3": "Immediately."+ },+ "probabilities": {+ "0": 0.1175,+ "1": 0.2869,+ "2": 0.3459,+ "3": 0.2496+ },+ "confidence": 0.0453,+ "answer_confidence": 0.3459,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 745,+ "output_tokens": 0,+ "state_tokens": 99,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ "agent-trace": {+ "state": {+ "run": "agt-7741",+ "tool_calls": 14,+ "wrote": [+ "s3://prod-reports/q1.csv"+ ],+ "errors": [+ "RateLimit on call 9, retried"+ ],+ "final": "uploaded the report, but the row count is 0"+ },+ "lang": null,+ "workflow": "agent_trace_observability",+ "questions": {+ "action": {+ "type": "choice",+ "instructions": "What should be done with this agent run?",+ "criteria": {+ "accept": "The run did what was asked; accept it.",+ "retry": "Transient failure; run it again.",+ "rollback": "Undo what it changed.",+ "escalate": "A person must look at this."+ }+ },+ "needs_review": {+ "type": "noul",+ "instructions": "A human should read this trace before the result is used."+ },+ "outcome": {+ "type": "choice",+ "instructions": "How did the run end?",+ "criteria": {+ "success": "Completed the task.",+ "partial": "Completed some of it.",+ "failure": "Did not complete the task."+ }+ },+ "risk": {+ "type": "score",+ "instructions": "How much damage could this run's actions have caused?",+ "criteria": [+ "Read-only; no effect.",+ "Reversible writes.",+ "Irreversible writes to internal systems.",+ "Irreversible, customer-visible, or spending money."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How quickly must this be looked at?",+ "criteria": [+ "No time pressure.",+ "Next business day.",+ "Within the hour.",+ "Immediately."+ ]+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "action": {+ "type": "choice",+ "choice": "retry",+ "probabilities": {+ "accept": 0.1787,+ "retry": 0.367,+ "rollback": 0.1785,+ "escalate": 0.2758+ },+ "confidence": 0.0345,+ "answer_confidence": 0.367,+ "action": {+ "act_probability": 1.0+ }+ },+ "needs_review": {+ "type": "noul",+ "noul": 0.3861,+ "confidence": 0.6139,+ "answer_confidence": 0.6139,+ "action": {+ "act_probability": 1.0+ }+ },+ "outcome": {+ "type": "choice",+ "choice": "failure",+ "probabilities": {+ "success": 0.1267,+ "partial": 0.3769,+ "failure": 0.4964+ },+ "confidence": 0.1106,+ "answer_confidence": 0.4964,+ "action": {+ "act_probability": 1.0+ }+ },+ "risk": {+ "type": "score",+ "score": 1.4808,+ "legend": {+ "0": "Read-only; no effect.",+ "1": "Reversible writes.",+ "2": "Irreversible writes to internal systems.",+ "3": "Irreversible, customer-visible, or spending money."+ },+ "probabilities": {+ "0": 0.1546,+ "1": 0.3994,+ "2": 0.2565,+ "3": 0.1894+ },+ "confidence": 0.0483,+ "answer_confidence": 0.3994,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.8262,+ "legend": {+ "0": "No time pressure.",+ "1": "Next business day.",+ "2": "Within the hour.",+ "3": "Immediately."+ },+ "probabilities": {+ "0": 0.0736,+ "1": 0.3121,+ "2": 0.3286,+ "3": 0.2856+ },+ "confidence": 0.0773,+ "answer_confidence": 0.3286,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 558,+ "output_tokens": 0,+ "state_tokens": 62,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ "many-options": {+ "state": "The invoice totals 300,020 USD against a 214,300 USD order.",+ "lang": null,+ "workflow": null,+ "questions": {+ "band": {+ "type": "choice",+ "instructions": "Which spend band does this invoice fall in?",+ "criteria": {+ "band00": "tier number 0",+ "band01": "tier number 1",+ "band02": "tier number 2",+ "band03": "tier number 3",+ "band04": "tier number 4",+ "band05": "tier number 5",+ "band06": "tier number 6",+ "band07": "tier number 7",+ "band08": "tier number 8",+ "band09": "tier number 9",+ "band10": "tier number 10",+ "band11": "tier number 11",+ "band12": "tier number 12",+ "band13": "tier number 13"+ }+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "band": {+ "type": "choice",+ "choice": "band01",+ "probabilities": {+ "band00": 0.1393,+ "band01": 0.3035,+ "band02": 0.1856,+ "band03": 0.0873,+ "band04": 0.0365,+ "band05": 0.0311,+ "band06": 0.0313,+ "band07": 0.0304,+ "band08": 0.0349,+ "band09": 0.043,+ "band10": 0.0163,+ "band11": 0.0132,+ "band12": 0.0218,+ "band13": 0.0257+ },+ "confidence": 0.1817,+ "answer_confidence": 0.3035,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 130,+ "output_tokens": 0,+ "state_tokens": 16,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ "over-budget": {+ "state": "line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. line item number one, unit price 300.02 against a purchase order at 214.30, delivery accepted with exceptions. ",+ "lang": null,+ "workflow": "invoice_processing",+ "questions": {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."+ }+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile."+ }+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted."+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day."+ ]+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.2397,+ "hold": 0.2893,+ "manual_review": 0.3046,+ "reject": 0.1664+ },+ "confidence": 0.0177,+ "answer_confidence": 0.3046,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.4998,+ "confidence": 0.5002,+ "answer_confidence": 0.5002,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.2023,+ "confidence": 0.7977,+ "answer_confidence": 0.7977,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.7091,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.2096,+ "1": 0.1382,+ "2": 0.3856,+ "3": 0.2666+ },+ "confidence": 0.0472,+ "answer_confidence": 0.3856,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.4485,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0834,+ "1": 0.4863,+ "2": 0.3288,+ "3": 0.1015+ },+ "confidence": 0.1663,+ "answer_confidence": 0.4863,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 5120,+ "output_tokens": 0,+ "state_tokens": 2161,+ "state_tokens_dropped": 1211,+ "truncated": true,+ "truncated_questions": [+ "disposition",+ "matches_order",+ "duplicate",+ "discrepancy_severity",+ "urgency"+ ]+ }+ },+ "off-workflow": {+ "state": "We were billed twice for March and want a refund today.",+ "lang": "en",+ "workflow": null,+ "questions": {+ "department": {+ "type": "choice",+ "instructions": "Which team owns this?",+ "criteria": {+ "billing": "charges and refunds",+ "other": "anything else"+ }+ }+ },+ "model": "laya-rl-agent-onnx",+ "answers": {+ "department": {+ "type": "choice",+ "choice": "billing",+ "probabilities": {+ "billing": 0.8967,+ "other": 0.1033+ },+ "confidence": 0.5207,+ "answer_confidence": 0.8967,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 36,+ "output_tokens": 0,+ "state_tokens": 12,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ }+ },+ "batch": [+ {+ "batch_size": null,+ "sort_by_length": false,+ "states": [+ {+ "invoice": {+ "id": "INV-2026-8842",+ "vendor": "Northwind Castings",+ "currency": "USD",+ "total_usd": 300020.0,+ "lines": [+ {+ "sku": "SKU-940",+ "qty": 1000,+ "unit_usd": 300.02,+ "total_usd": 300020.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3392",+ "total_usd": 214300.0,+ "lines": [+ {+ "qty": 1000,+ "unit_usd": 214.3+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-18",+ "received_qty": 1000,+ "condition": "accepted with exceptions"+ },+ "payment": {+ "terms": "net 45",+ "status": "overdue",+ "days_until_due": -3+ },+ "vendor_history": {+ "invoices_12m": 40,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-6633",+ "INV-2026-9563",+ "INV-2026-7939"+ ]+ }+ },+ {+ "invoice": {+ "id": "INV-2026-9107",+ "vendor": "Cedar Freight",+ "currency": "USD",+ "total_usd": 18450.0,+ "lines": [+ {+ "sku": "SKU-221",+ "qty": 300,+ "unit_usd": 61.5,+ "total_usd": 18450.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3477",+ "total_usd": 18450.0,+ "lines": [+ {+ "qty": 300,+ "unit_usd": 61.5+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-20",+ "received_qty": 300,+ "condition": "accepted in full"+ },+ "payment": {+ "terms": "net 30",+ "status": "current",+ "days_until_due": 18+ },+ "vendor_history": {+ "invoices_12m": 12,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-7781"+ ]+ }+ },+ "INV-2026-1004, 980 USD, matches PO-3100 and delivery.",+ "duplicate of INV-2026-1004",+ ""+ ],+ "questions": {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."+ }+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile."+ }+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted."+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day."+ ]+ }+ },+ "results": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.0908,+ "hold": 0.3301,+ "manual_review": 0.4534,+ "reject": 0.1257+ },+ "confidence": 0.1322,+ "answer_confidence": 0.4534,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.2063,+ "confidence": 0.7937,+ "answer_confidence": 0.7937,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1399,+ "confidence": 0.8601,+ "answer_confidence": 0.8601,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 2.3799,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.0405,+ "1": 0.0602,+ "2": 0.378,+ "3": 0.5212+ },+ "confidence": 0.2739,+ "answer_confidence": 0.5212,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 2.1864,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0226,+ "1": 0.1418,+ "2": 0.4621,+ "3": 0.3735+ },+ "confidence": 0.2157,+ "answer_confidence": 0.4621,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1666,+ "output_tokens": 0,+ "state_tokens": 274,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.4076,+ "hold": 0.2312,+ "manual_review": 0.2092,+ "reject": 0.152+ },+ "confidence": 0.0492,+ "answer_confidence": 0.4076,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6562,+ "confidence": 0.6562,+ "answer_confidence": 0.6562,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1452,+ "confidence": 0.8548,+ "answer_confidence": 0.8548,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.0056,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.4979,+ "1": 0.1355,+ "2": 0.2296,+ "3": 0.1369+ },+ "confidence": 0.1141,+ "answer_confidence": 0.4979,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5989,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0413,+ "1": 0.441,+ "2": 0.3952,+ "3": 0.1225+ },+ "confidence": 0.1945,+ "answer_confidence": 0.441,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1556,+ "output_tokens": 0,+ "state_tokens": 252,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.5173,+ "hold": 0.1756,+ "manual_review": 0.1803,+ "reject": 0.1267+ },+ "confidence": 0.122,+ "answer_confidence": 0.5173,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.7186,+ "confidence": 0.7186,+ "answer_confidence": 0.7186,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.2763,+ "confidence": 0.7237,+ "answer_confidence": 0.7237,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 0.6375,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.6903,+ "1": 0.0668,+ "2": 0.158,+ "3": 0.0849+ },+ "confidence": 0.3236,+ "answer_confidence": 0.6903,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5688,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0407,+ "1": 0.47,+ "2": 0.369,+ "3": 0.1203+ },+ "confidence": 0.2008,+ "answer_confidence": 0.47,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 396,+ "output_tokens": 0,+ "state_tokens": 20,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.1746,+ "hold": 0.2887,+ "manual_review": 0.3221,+ "reject": 0.2145+ },+ "confidence": 0.02,+ "answer_confidence": 0.3221,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6206,+ "confidence": 0.6206,+ "answer_confidence": 0.6206,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.4192,+ "confidence": 0.5808,+ "answer_confidence": 0.5808,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.823,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.1938,+ "1": 0.1166,+ "2": 0.3623,+ "3": 0.3273+ },+ "confidence": 0.0608,+ "answer_confidence": 0.3623,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.603,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0369,+ "1": 0.4401,+ "2": 0.4062,+ "3": 0.1169+ },+ "confidence": 0.2067,+ "answer_confidence": 0.4401,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 346,+ "output_tokens": 0,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.2066,+ "hold": 0.3032,+ "manual_review": 0.335,+ "reject": 0.1552+ },+ "confidence": 0.0311,+ "answer_confidence": 0.335,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.5773,+ "confidence": 0.5773,+ "answer_confidence": 0.5773,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.15,+ "confidence": 0.85,+ "answer_confidence": 0.85,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.5307,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.3219,+ "1": 0.1028,+ "2": 0.2982,+ "3": 0.2772+ },+ "confidence": 0.0513,+ "answer_confidence": 0.3219,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5328,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0409,+ "1": 0.4738,+ "2": 0.397,+ "3": 0.0883+ },+ "confidence": 0.2313,+ "answer_confidence": 0.4738,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 296,+ "output_tokens": 0,+ "state_tokens": 0,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ }+ ]+ },+ {+ "batch_size": 2,+ "sort_by_length": false,+ "states": [+ {+ "invoice": {+ "id": "INV-2026-8842",+ "vendor": "Northwind Castings",+ "currency": "USD",+ "total_usd": 300020.0,+ "lines": [+ {+ "sku": "SKU-940",+ "qty": 1000,+ "unit_usd": 300.02,+ "total_usd": 300020.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3392",+ "total_usd": 214300.0,+ "lines": [+ {+ "qty": 1000,+ "unit_usd": 214.3+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-18",+ "received_qty": 1000,+ "condition": "accepted with exceptions"+ },+ "payment": {+ "terms": "net 45",+ "status": "overdue",+ "days_until_due": -3+ },+ "vendor_history": {+ "invoices_12m": 40,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-6633",+ "INV-2026-9563",+ "INV-2026-7939"+ ]+ }+ },+ {+ "invoice": {+ "id": "INV-2026-9107",+ "vendor": "Cedar Freight",+ "currency": "USD",+ "total_usd": 18450.0,+ "lines": [+ {+ "sku": "SKU-221",+ "qty": 300,+ "unit_usd": 61.5,+ "total_usd": 18450.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3477",+ "total_usd": 18450.0,+ "lines": [+ {+ "qty": 300,+ "unit_usd": 61.5+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-20",+ "received_qty": 300,+ "condition": "accepted in full"+ },+ "payment": {+ "terms": "net 30",+ "status": "current",+ "days_until_due": 18+ },+ "vendor_history": {+ "invoices_12m": 12,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-7781"+ ]+ }+ },+ "INV-2026-1004, 980 USD, matches PO-3100 and delivery.",+ "duplicate of INV-2026-1004",+ ""+ ],+ "questions": {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."+ }+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile."+ }+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted."+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day."+ ]+ }+ },+ "results": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.0908,+ "hold": 0.3301,+ "manual_review": 0.4534,+ "reject": 0.1257+ },+ "confidence": 0.1322,+ "answer_confidence": 0.4534,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.2063,+ "confidence": 0.7937,+ "answer_confidence": 0.7937,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1399,+ "confidence": 0.8601,+ "answer_confidence": 0.8601,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 2.3799,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.0405,+ "1": 0.0602,+ "2": 0.378,+ "3": 0.5212+ },+ "confidence": 0.2739,+ "answer_confidence": 0.5212,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 2.1864,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0226,+ "1": 0.1418,+ "2": 0.4621,+ "3": 0.3735+ },+ "confidence": 0.2157,+ "answer_confidence": 0.4621,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1666,+ "output_tokens": 0,+ "state_tokens": 274,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.4076,+ "hold": 0.2312,+ "manual_review": 0.2092,+ "reject": 0.152+ },+ "confidence": 0.0492,+ "answer_confidence": 0.4076,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6562,+ "confidence": 0.6562,+ "answer_confidence": 0.6562,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1452,+ "confidence": 0.8548,+ "answer_confidence": 0.8548,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.0056,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.4979,+ "1": 0.1355,+ "2": 0.2296,+ "3": 0.1369+ },+ "confidence": 0.1141,+ "answer_confidence": 0.4979,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5989,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0413,+ "1": 0.441,+ "2": 0.3952,+ "3": 0.1225+ },+ "confidence": 0.1945,+ "answer_confidence": 0.441,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1556,+ "output_tokens": 0,+ "state_tokens": 252,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.5173,+ "hold": 0.1756,+ "manual_review": 0.1803,+ "reject": 0.1267+ },+ "confidence": 0.122,+ "answer_confidence": 0.5173,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.7186,+ "confidence": 0.7186,+ "answer_confidence": 0.7186,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.2763,+ "confidence": 0.7237,+ "answer_confidence": 0.7237,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 0.6375,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.6903,+ "1": 0.0668,+ "2": 0.158,+ "3": 0.0849+ },+ "confidence": 0.3236,+ "answer_confidence": 0.6903,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5688,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0407,+ "1": 0.47,+ "2": 0.369,+ "3": 0.1203+ },+ "confidence": 0.2008,+ "answer_confidence": 0.47,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 396,+ "output_tokens": 0,+ "state_tokens": 20,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.1746,+ "hold": 0.2887,+ "manual_review": 0.3221,+ "reject": 0.2145+ },+ "confidence": 0.02,+ "answer_confidence": 0.3221,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6206,+ "confidence": 0.6206,+ "answer_confidence": 0.6206,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.4192,+ "confidence": 0.5808,+ "answer_confidence": 0.5808,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.823,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.1938,+ "1": 0.1166,+ "2": 0.3623,+ "3": 0.3273+ },+ "confidence": 0.0608,+ "answer_confidence": 0.3623,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.603,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0369,+ "1": 0.4401,+ "2": 0.4062,+ "3": 0.1169+ },+ "confidence": 0.2067,+ "answer_confidence": 0.4401,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 346,+ "output_tokens": 0,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.2066,+ "hold": 0.3032,+ "manual_review": 0.335,+ "reject": 0.1552+ },+ "confidence": 0.0311,+ "answer_confidence": 0.335,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.5773,+ "confidence": 0.5773,+ "answer_confidence": 0.5773,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.15,+ "confidence": 0.85,+ "answer_confidence": 0.85,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.5307,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.3219,+ "1": 0.1028,+ "2": 0.2982,+ "3": 0.2772+ },+ "confidence": 0.0513,+ "answer_confidence": 0.3219,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5328,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0409,+ "1": 0.4738,+ "2": 0.397,+ "3": 0.0883+ },+ "confidence": 0.2313,+ "answer_confidence": 0.4738,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 296,+ "output_tokens": 0,+ "state_tokens": 0,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ }+ ]+ },+ {+ "batch_size": 3,+ "sort_by_length": true,+ "states": [+ {+ "invoice": {+ "id": "INV-2026-8842",+ "vendor": "Northwind Castings",+ "currency": "USD",+ "total_usd": 300020.0,+ "lines": [+ {+ "sku": "SKU-940",+ "qty": 1000,+ "unit_usd": 300.02,+ "total_usd": 300020.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3392",+ "total_usd": 214300.0,+ "lines": [+ {+ "qty": 1000,+ "unit_usd": 214.3+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-18",+ "received_qty": 1000,+ "condition": "accepted with exceptions"+ },+ "payment": {+ "terms": "net 45",+ "status": "overdue",+ "days_until_due": -3+ },+ "vendor_history": {+ "invoices_12m": 40,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-6633",+ "INV-2026-9563",+ "INV-2026-7939"+ ]+ }+ },+ {+ "invoice": {+ "id": "INV-2026-9107",+ "vendor": "Cedar Freight",+ "currency": "USD",+ "total_usd": 18450.0,+ "lines": [+ {+ "sku": "SKU-221",+ "qty": 300,+ "unit_usd": 61.5,+ "total_usd": 18450.0+ }+ ]+ },+ "purchase_order": {+ "id": "PO-3477",+ "total_usd": 18450.0,+ "lines": [+ {+ "qty": 300,+ "unit_usd": 61.5+ }+ ],+ "freight_terms": "freight prepaid by vendor, not separately billable"+ },+ "delivery": {+ "date": "2026-03-20",+ "received_qty": 300,+ "condition": "accepted in full"+ },+ "payment": {+ "terms": "net 30",+ "status": "current",+ "days_until_due": 18+ },+ "vendor_history": {+ "invoices_12m": 12,+ "disputes_12m": 0,+ "prior_invoice_ids": [+ "INV-2026-7781"+ ]+ }+ },+ "INV-2026-1004, 980 USD, matches PO-3100 and delivery.",+ "duplicate of INV-2026-1004",+ ""+ ],+ "questions": {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong."+ }+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile."+ }+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted."+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference."+ ]+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day."+ ]+ }+ },+ "results": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.0908,+ "hold": 0.3301,+ "manual_review": 0.4534,+ "reject": 0.1257+ },+ "confidence": 0.1322,+ "answer_confidence": 0.4534,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.2063,+ "confidence": 0.7937,+ "answer_confidence": 0.7937,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1399,+ "confidence": 0.8601,+ "answer_confidence": 0.8601,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 2.3799,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.0405,+ "1": 0.0602,+ "2": 0.378,+ "3": 0.5212+ },+ "confidence": 0.2739,+ "answer_confidence": 0.5212,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 2.1864,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0226,+ "1": 0.1418,+ "2": 0.4621,+ "3": 0.3735+ },+ "confidence": 0.2157,+ "answer_confidence": 0.4621,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1666,+ "output_tokens": 0,+ "state_tokens": 274,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.4076,+ "hold": 0.2312,+ "manual_review": 0.2092,+ "reject": 0.152+ },+ "confidence": 0.0492,+ "answer_confidence": 0.4076,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6562,+ "confidence": 0.6562,+ "answer_confidence": 0.6562,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.1452,+ "confidence": 0.8548,+ "answer_confidence": 0.8548,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.0056,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.4979,+ "1": 0.1355,+ "2": 0.2296,+ "3": 0.1369+ },+ "confidence": 0.1141,+ "answer_confidence": 0.4979,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5989,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0413,+ "1": 0.441,+ "2": 0.3952,+ "3": 0.1225+ },+ "confidence": 0.1945,+ "answer_confidence": 0.441,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1556,+ "output_tokens": 0,+ "state_tokens": 252,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "approve",+ "probabilities": {+ "approve": 0.5173,+ "hold": 0.1756,+ "manual_review": 0.1803,+ "reject": 0.1267+ },+ "confidence": 0.122,+ "answer_confidence": 0.5173,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.7186,+ "confidence": 0.7186,+ "answer_confidence": 0.7186,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.2763,+ "confidence": 0.7237,+ "answer_confidence": 0.7237,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 0.6375,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.6903,+ "1": 0.0668,+ "2": 0.158,+ "3": 0.0849+ },+ "confidence": 0.3236,+ "answer_confidence": 0.6903,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5688,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0407,+ "1": 0.47,+ "2": 0.369,+ "3": 0.1203+ },+ "confidence": 0.2008,+ "answer_confidence": 0.47,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 396,+ "output_tokens": 0,+ "state_tokens": 20,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.1746,+ "hold": 0.2887,+ "manual_review": 0.3221,+ "reject": 0.2145+ },+ "confidence": 0.02,+ "answer_confidence": 0.3221,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.6206,+ "confidence": 0.6206,+ "answer_confidence": 0.6206,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.4192,+ "confidence": 0.5808,+ "answer_confidence": 0.5808,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.823,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.1938,+ "1": 0.1166,+ "2": 0.3623,+ "3": 0.3273+ },+ "confidence": 0.0608,+ "answer_confidence": 0.3623,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.603,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0369,+ "1": 0.4401,+ "2": 0.4062,+ "3": 0.1169+ },+ "confidence": 0.2067,+ "answer_confidence": 0.4401,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 346,+ "output_tokens": 0,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "disposition": {+ "type": "choice",+ "choice": "manual_review",+ "probabilities": {+ "approve": 0.2066,+ "hold": 0.3032,+ "manual_review": 0.335,+ "reject": 0.1552+ },+ "confidence": 0.0311,+ "answer_confidence": 0.335,+ "action": {+ "act_probability": 1.0+ }+ },+ "matches_order": {+ "type": "noul",+ "noul": 0.5773,+ "confidence": 0.5773,+ "answer_confidence": 0.5773,+ "action": {+ "act_probability": 1.0+ }+ },+ "duplicate": {+ "type": "noul",+ "noul": 0.15,+ "confidence": 0.85,+ "answer_confidence": 0.85,+ "action": {+ "act_probability": 1.0+ }+ },+ "discrepancy_severity": {+ "type": "score",+ "score": 1.5307,+ "legend": {+ "0": "None: everything reconciles.",+ "1": "Trivial: rounding or a cosmetic difference.",+ "2": "Moderate: a real difference worth confirming.",+ "3": "Material: a large or unexplained difference."+ },+ "probabilities": {+ "0": 0.3219,+ "1": 0.1028,+ "2": 0.2982,+ "3": 0.2772+ },+ "confidence": 0.0513,+ "answer_confidence": 0.3219,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.5328,+ "legend": {+ "0": "No time pressure; can wait indefinitely.",+ "1": "Routine; handle within the normal queue.",+ "2": "Elevated; should be handled within the same week.",+ "3": "Critical; requires action within the same day."+ },+ "probabilities": {+ "0": 0.0409,+ "1": 0.4738,+ "2": 0.397,+ "3": 0.0883+ },+ "confidence": 0.2313,+ "answer_confidence": 0.4738,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 296,+ "output_tokens": 0,+ "state_tokens": 0,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": []+ }+ }+ ]+ }+ ]+}diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/config/AgentConfig.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/config/AgentConfig.javaindex cac69eb9..96e90291 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/config/AgentConfig.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/config/AgentConfig.java@@ -17,10 +17,11 @@ import java.util.Map;/*** A checkpoint's {@code rl_agent_config.json}: sequence budgets and the fitted temperatures.*- * <p>These must be read from the checkpoint and never defaulted. The two shipped checkpoints+ * <p>These must be read from the checkpoint and never defaulted. The three shipped checkpoints* disagree on every one of them: english is {@code max_len 512 / head_max_len 192} with fitted- * per-bucket temperatures, multilingual is {@code 1024/256} with none at all. A port that- * hard-coded the library's own defaults would halve a multilingual state's budget.+ * per-bucket temperatures, multilingual is {@code 1024/256} with none at all, and typed-decisions+ * is {@code 1024/256} with a fitted table of its own. A port that hard-coded the library's own+ * defaults would halve a multilingual state's budget.*/public final class AgentConfig {@@ -28,9 +29,12 @@ public final class AgentConfig {* A fitted temperature below 1 sharpens logits instead of softening them, and is refused.** <p>Not a hypothetical: the english checkpoint ships {@code choice:11+} at <b>0.1006</b>, a- * ~10x sharpener that publishes a 0.24 top probability as 0.99. A caller gating on confidence- * would be told a coin flip is a certainty, so the runtime confines every temperature to- * {@code [0.5, 5.0]}.+ * ~10x sharpener that publishes a 0.24 top probability as 0.99, and typed-decisions carries+ * the same value through from the checkpoint it was fine-tuned from. A caller gating on+ * confidence would be told a coin flip is a certainty, so the runtime confines every+ * temperature to {@code [0.5, 5.0]}. {@code TypedDecisionsParityTest} asserts the clamp+ * end to end on a 14-option question, where skipping it sharpens the recorded 0.3035 top+ * probability by roughly 5x without changing the argmax.*/public static final double TEMP_MIN = 0.5;public static final double TEMP_MAX = 5.0;diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/Fixtures.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/Fixtures.javaindex 2bb05820..c9334bdd 100644--- a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/Fixtures.java+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/Fixtures.java@@ -22,12 +22,28 @@ import java.util.Map;*/public final class Fixtures {- /** Set to the directory holding {@code english/} and {@code multilingual/} to enable those tests. */+ /**+ * Set to the directory holding {@code english/}, {@code multilingual/} and+ * {@code typed-decisions/} to enable those tests.+ */public static final String CHECKPOINTS_ENV = "LAYA_CHECKPOINTS";/** Set to the directory holding {@code laya.onnx} (or {@code encoder.onnx} + {@code head.onnx}). */public static final String GRAPH_ENV = "LAYA_ONNX_GRAPH";+ /**+ * As {@link #GRAPH_ENV}, for the graph traced from the {@code typed-decisions} checkpoint.+ *+ * <p>A second variable rather than reusing the first, because a graph carries the weights and+ * nothing in it says which checkpoint traced it -- so a wrong pairing answers rather than+ * failing. Measured: the english checkpoint opened against the typed-decisions graph raises+ * nothing and returns {@code {refund: 0.9077, replace: 0.0923}}, digit for digit what the+ * typed-decisions checkpoint returns against its own graph, and {@link Prediction#model()}+ * reads {@code laya-rl-agent-onnx} either way. One variable holding one directory would make+ * that pairing the default whenever both suites run in the same JVM.+ */+ public static final String TYPED_GRAPH_ENV = "LAYA_TYPED_ONNX_GRAPH";+/** Set to a {@code predict.json} recorded from Python with a graph present. */public static final String PREDICT_GOLDEN_ENV = "LAYA_PREDICT_GOLDEN";@@ -83,7 +99,7 @@ public final class Fixtures {/** The message a skipped checkpoint test prints, so the reason is actionable. */public static String missingCheckpoint(String name) {return "needs the " + name + " checkpoint: set " + CHECKPOINTS_ENV- + " to the directory holding english/ and multilingual/";+ + " to the directory holding " + name + "/";}/** Helper: a list of longs from a fixture array. */diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictParityTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictParityTest.javaindex 7abacf8a..e0a17c11 100644--- a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictParityTest.java+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictParityTest.java@@ -1,7 +1,9 @@package com.convaiinnovations.laya;+import static com.convaiinnovations.laya.PredictionGolden.assertAnswers;+import static com.convaiinnovations.laya.PredictionGolden.assertUsage;+import static com.convaiinnovations.laya.PredictionGolden.questionsOf;import static org.junit.jupiter.api.Assertions.assertEquals;-import static org.junit.jupiter.api.Assertions.assertIterableEquals;import static org.junit.jupiter.api.Assertions.assertTrue;import java.io.IOException;@@ -9,7 +11,6 @@ import java.nio.file.Files;import java.nio.file.Path;import java.nio.file.Paths;import java.util.ArrayList;-import java.util.LinkedHashMap;import java.util.List;import java.util.Map;import org.junit.jupiter.api.AfterAll;@@ -24,18 +25,11 @@ import org.junit.jupiter.api.TestFactory;* <p>The only check that covers the whole stack at once -- tokenizer, sequence budgets, collation,* graph I/O, decode, rounding and usage -- which the fixture families cannot do individually.*- * <p><b>Tolerance.</b> Structure is compared exactly: the answer type, the chosen label, the- * question ids and their order, the legend, and every integer in {@link Usage}. Probabilities are- * compared within {@value #PROBABILITY_TOLERANCE}, two units in the last reported decimal, because- * the encoder's own arithmetic is platform-sensitive and the golden is recorded on one machine. A- * chosen label is only asserted when the top two probabilities are clear of that tolerance; when- * they are not, the argmax is genuinely a coin toss and asserting it would be asserting noise.+ * <p>This one covers the multilingual checkpoint; {@link TypedDecisionsParityTest} covers+ * {@code typed-decisions}. The comparison rules they share live in {@link PredictionGolden}.*/final class PredictParityTest {- /** Two units in the last decimal an answer reports. */- private static final double PROBABILITY_TOLERANCE = 2e-4;-/*** One agent for the whole class, closed when the class is done.*@@ -108,121 +102,6 @@ final class PredictParityTest {return tests;}- @SuppressWarnings("unchecked")- private static void assertAnswers(Map<String, Object> want, Map<String, Answer> got) {- assertIterableEquals(want.keySet(), got.keySet(), "question ids and their order");- for (Map.Entry<String, Object> entry : want.entrySet()) {- Map<String, Object> w = (Map<String, Object>) entry.getValue();- Answer g = got.get(entry.getKey());- String at = entry.getKey();- assertEquals(w.get("type"), g.type(), at + ".type");- assertEquals(number(w.get("answer_confidence")), g.answerConfidence(),- PROBABILITY_TOLERANCE, at + ".answer_confidence");- assertEquals(number(w.get("confidence")), g.confidence(),- PROBABILITY_TOLERANCE, at + ".confidence");- assertEquals(number(((Map<String, Object>) w.get("action")).get("act_probability")),- g.actProbability(), PROBABILITY_TOLERANCE, at + ".act_probability");- if (g instanceof Answer.Choice choice) {- Map<String, Object> probabilities = (Map<String, Object>) w.get("probabilities");- assertProbabilities(at, probabilities, choice.probabilities());- if (decisive(probabilities)) {- assertEquals(w.get("choice"), choice.choice(), at + ".choice");- }- } else if (g instanceof Answer.Score score) {- assertEquals(number(w.get("score")), score.score(), PROBABILITY_TOLERANCE * 4,- at + ".score");- assertProbabilities(at, (Map<String, Object>) w.get("probabilities"),- score.probabilities());- Map<String, Object> legend = (Map<String, Object>) w.get("legend");- assertIterableEquals(legend.keySet(), score.legend().keySet(), at + ".legend order");- for (Map.Entry<String, Object> level : legend.entrySet()) {- assertEquals(level.getValue(), score.legend().get(level.getKey()));- }- } else if (g instanceof Answer.Noul noul) {- assertEquals(number(w.get("noul")), noul.noul(), PROBABILITY_TOLERANCE, at + ".noul");- }- }- }-- /** Whether the top two probabilities are far enough apart for the argmax to be meaningful. */- private static boolean decisive(Map<String, Object> probabilities) {- double best = -1.0;- double second = -1.0;- for (Object value : probabilities.values()) {- double p = number(value);- if (p > best) {- second = best;- best = p;- } else if (p > second) {- second = p;- }- }- return best - second > PROBABILITY_TOLERANCE * 5;- }-- private static void assertProbabilities(String at, Map<String, Object> want,- Map<String, Double> got) {- assertIterableEquals(want.keySet(), got.keySet(), at + ".probability key order");- for (Map.Entry<String, Object> entry : want.entrySet()) {- assertEquals(number(entry.getValue()), got.get(entry.getKey()),- PROBABILITY_TOLERANCE, at + ".p[" + entry.getKey() + "]");- }- }-- @SuppressWarnings("unchecked")- private static void assertUsage(Map<String, Object> want, Usage got) {- // Exact: these are counts, and a count that drifts is a bug, not float noise.- assertEquals(((Number) want.get("input_tokens")).intValue(), got.inputTokens(),- "usage.input_tokens");- assertEquals(((Number) want.get("output_tokens")).intValue(), got.outputTokens());- assertEquals(((Number) want.get("state_tokens")).intValue(), got.stateTokens(),- "usage.state_tokens");- assertEquals(((Number) want.get("state_tokens_dropped")).intValue(),- got.stateTokensDropped(), "usage.state_tokens_dropped");- assertEquals(want.get("truncated"), got.truncated(), "usage.truncated");- assertIterableEquals((List<Object>) want.get("truncated_questions"),- got.truncatedQuestions(), "usage.truncated_questions");- Map<String, Object> options = (Map<String, Object>) want.get("options");- if (options == null) {- assertTrue(got.collapsedOptions().isEmpty(),- "no collapse expected, got " + got.collapsedOptions().keySet());- return;- }- assertEquals(options.keySet(), got.collapsedOptions().keySet(), "usage.options");- for (Map.Entry<String, Object> entry : options.entrySet()) {- Map<String, Object> w = (Map<String, Object>) entry.getValue();- Usage.CollapsedOptions g = got.collapsedOptions().get(entry.getKey());- assertEquals(((Number) w.get("total")).intValue(), g.total());- assertEquals(((Number) w.get("distinct")).intValue(), g.distinct());- }- }-- private static double number(Object value) {- return ((Number) value).doubleValue();- }-- @SuppressWarnings("unchecked")- private static Map<String, Question> questionsOf(Map<String, Object> defs) {- Map<String, Question> out = new LinkedHashMap<>();- for (Map.Entry<String, Object> entry : defs.entrySet()) {- Map<String, Object> q = (Map<String, Object>) entry.getValue();- String type = (String) q.get("type");- String instructions = String.valueOf(q.get("instructions"));- Object criteria = q.get("criteria");- if ("choice".equals(type)) {- out.put(entry.getKey(), Question.choice(instructions, (Map<String, Object>) criteria));- } else if ("score".equals(type)) {- out.put(entry.getKey(), Question.score(instructions, (List<Object>) criteria));- } else {- Map<String, Object> sides = criteria instanceof Map- ? (Map<String, Object>) criteria : Map.of();- out.put(entry.getKey(), Question.noul(instructions, sides.get("false"),- sides.get("true"), null));- }- }- return out;- }-private static Map<String, Object> requireGolden() {Map<String, Object> golden = Fixtures.load("predict.json");Assumptions.assumeFalse(golden.containsKey("skipped"),diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictionGolden.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictionGolden.javanew file mode 100644index 00000000..7c07d248--- /dev/null+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictionGolden.java@@ -0,0 +1,148 @@+package com.convaiinnovations.laya;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertIterableEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;++import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;++/**+ * Comparing a recorded {@code predict} result against what this port answers.+ *+ * <p>Shared by {@link PredictParityTest} and {@link TypedDecisionsParityTest} rather than copied+ * into each. The copy is the thing worth avoiding: the tolerance rules below are the gate those+ * suites are, so two copies drifting apart means one checkpoint is held to a standard the other+ * is not, and nothing reports it.+ *+ * <p><b>Tolerance.</b> Structure is compared exactly: the answer type, the chosen label, the+ * question ids and their order, the legend, and every integer in {@link Usage}. Probabilities are+ * compared within {@value #PROBABILITY_TOLERANCE}, two units in the last reported decimal, because+ * the encoder's own arithmetic is platform-sensitive and the golden is recorded on one machine. A+ * chosen label is only asserted when the top two probabilities are clear of that tolerance; when+ * they are not, the argmax is genuinely a coin toss and asserting it would be asserting noise.+ */+final class PredictionGolden {++ /** Two units in the last decimal an answer reports. */+ static final double PROBABILITY_TOLERANCE = 2e-4;++ private PredictionGolden() {+ }++ @SuppressWarnings("unchecked")+ static void assertAnswers(Map<String, Object> want, Map<String, Answer> got) {+ assertIterableEquals(want.keySet(), got.keySet(), "question ids and their order");+ for (Map.Entry<String, Object> entry : want.entrySet()) {+ Map<String, Object> w = (Map<String, Object>) entry.getValue();+ Answer g = got.get(entry.getKey());+ String at = entry.getKey();+ assertEquals(w.get("type"), g.type(), at + ".type");+ assertEquals(number(w.get("answer_confidence")), g.answerConfidence(),+ PROBABILITY_TOLERANCE, at + ".answer_confidence");+ assertEquals(number(w.get("confidence")), g.confidence(),+ PROBABILITY_TOLERANCE, at + ".confidence");+ assertEquals(number(((Map<String, Object>) w.get("action")).get("act_probability")),+ g.actProbability(), PROBABILITY_TOLERANCE, at + ".act_probability");+ if (g instanceof Answer.Choice choice) {+ Map<String, Object> probabilities = (Map<String, Object>) w.get("probabilities");+ assertProbabilities(at, probabilities, choice.probabilities());+ if (decisive(probabilities)) {+ assertEquals(w.get("choice"), choice.choice(), at + ".choice");+ }+ } else if (g instanceof Answer.Score score) {+ assertEquals(number(w.get("score")), score.score(), PROBABILITY_TOLERANCE * 4,+ at + ".score");+ assertProbabilities(at, (Map<String, Object>) w.get("probabilities"),+ score.probabilities());+ Map<String, Object> legend = (Map<String, Object>) w.get("legend");+ assertIterableEquals(legend.keySet(), score.legend().keySet(), at + ".legend order");+ for (Map.Entry<String, Object> level : legend.entrySet()) {+ assertEquals(level.getValue(), score.legend().get(level.getKey()));+ }+ } else if (g instanceof Answer.Noul noul) {+ assertEquals(number(w.get("noul")), noul.noul(), PROBABILITY_TOLERANCE, at + ".noul");+ }+ }+ }++ /** Whether the top two probabilities are far enough apart for the argmax to be meaningful. */+ static boolean decisive(Map<String, Object> probabilities) {+ double best = -1.0;+ double second = -1.0;+ for (Object value : probabilities.values()) {+ double p = number(value);+ if (p > best) {+ second = best;+ best = p;+ } else if (p > second) {+ second = p;+ }+ }+ return best - second > PROBABILITY_TOLERANCE * 5;+ }++ static void assertProbabilities(String at, Map<String, Object> want, Map<String, Double> got) {+ assertIterableEquals(want.keySet(), got.keySet(), at + ".probability key order");+ for (Map.Entry<String, Object> entry : want.entrySet()) {+ assertEquals(number(entry.getValue()), got.get(entry.getKey()),+ PROBABILITY_TOLERANCE, at + ".p[" + entry.getKey() + "]");+ }+ }++ @SuppressWarnings("unchecked")+ static void assertUsage(Map<String, Object> want, Usage got) {+ // Exact: these are counts, and a count that drifts is a bug, not float noise.+ assertEquals(((Number) want.get("input_tokens")).intValue(), got.inputTokens(),+ "usage.input_tokens");+ assertEquals(((Number) want.get("output_tokens")).intValue(), got.outputTokens());+ assertEquals(((Number) want.get("state_tokens")).intValue(), got.stateTokens(),+ "usage.state_tokens");+ assertEquals(((Number) want.get("state_tokens_dropped")).intValue(),+ got.stateTokensDropped(), "usage.state_tokens_dropped");+ assertEquals(want.get("truncated"), got.truncated(), "usage.truncated");+ assertIterableEquals((List<Object>) want.get("truncated_questions"),+ got.truncatedQuestions(), "usage.truncated_questions");+ Map<String, Object> options = (Map<String, Object>) want.get("options");+ if (options == null) {+ assertTrue(got.collapsedOptions().isEmpty(),+ "no collapse expected, got " + got.collapsedOptions().keySet());+ return;+ }+ assertEquals(options.keySet(), got.collapsedOptions().keySet(), "usage.options");+ for (Map.Entry<String, Object> entry : options.entrySet()) {+ Map<String, Object> w = (Map<String, Object>) entry.getValue();+ Usage.CollapsedOptions g = got.collapsedOptions().get(entry.getKey());+ assertEquals(((Number) w.get("total")).intValue(), g.total());+ assertEquals(((Number) w.get("distinct")).intValue(), g.distinct());+ }+ }++ static double number(Object value) {+ return ((Number) value).doubleValue();+ }++ /** A recorded question mapping, rebuilt as the {@link Question} objects the port takes. */+ @SuppressWarnings("unchecked")+ static Map<String, Question> questionsOf(Map<String, Object> defs) {+ Map<String, Question> out = new LinkedHashMap<>();+ for (Map.Entry<String, Object> entry : defs.entrySet()) {+ Map<String, Object> q = (Map<String, Object>) entry.getValue();+ String type = (String) q.get("type");+ String instructions = String.valueOf(q.get("instructions"));+ Object criteria = q.get("criteria");+ if ("choice".equals(type)) {+ out.put(entry.getKey(), Question.choice(instructions, (Map<String, Object>) criteria));+ } else if ("score".equals(type)) {+ out.put(entry.getKey(), Question.score(instructions, (List<Object>) criteria));+ } else {+ Map<String, Object> sides = criteria instanceof Map+ ? (Map<String, Object>) criteria : Map.of();+ out.put(entry.getKey(), Question.noul(instructions, sides.get("false"),+ sides.get("true"), null));+ }+ }+ return out;+ }+}diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/TypedDecisionsParityTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/TypedDecisionsParityTest.javanew file mode 100644index 00000000..2b8c9e92--- /dev/null+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/TypedDecisionsParityTest.java@@ -0,0 +1,241 @@+package com.convaiinnovations.laya;++import static com.convaiinnovations.laya.PredictionGolden.assertAnswers;+import static com.convaiinnovations.laya.PredictionGolden.assertUsage;+import static com.convaiinnovations.laya.PredictionGolden.questionsOf;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;++import com.convaiinnovations.laya.config.AgentConfig;+import java.io.IOException;+import java.nio.file.Files;+import java.nio.file.Path;+import java.nio.file.Paths;+import java.util.ArrayList;+import java.util.List;+import java.util.Map;+import java.util.Set;+import java.util.TreeSet;+import org.junit.jupiter.api.AfterAll;+import org.junit.jupiter.api.Assumptions;+import org.junit.jupiter.api.DisplayName;+import org.junit.jupiter.api.DynamicTest;+import org.junit.jupiter.api.Test;+import org.junit.jupiter.api.TestFactory;++/**+ * End to end on the {@code typed-decisions} checkpoint, which nothing in this port used to run.+ *+ * <p>{@code Router} has always resolved the name and its four aliases, and {@link RouterTest}+ * asserts that it does. What no test did was take the checkpoint those decisions name and push a+ * state through it, so every number this port would produce from those weights was unmeasured --+ * the routing decision was gated and its destination was not. This class closes that: the same+ * comparison {@link PredictParityTest} applies to the multilingual checkpoint, applied to the+ * weights {@code typed}, {@code typed_decisions}, {@code decisions} and+ * {@code laya-typed-decisions} all resolve to.+ *+ * <p>What this checkpoint has that the multilingual one does not, and why each case is here:+ *+ * <ul>+ * <li>Four fine-tuned workflow signatures. Each recorded case names the workflow its question+ * ids are, and the workflow is re-derived here through+ * {@link Router#matchTypedDecisionsWorkflow} -- so the routing decision and the forward+ * pass are asserted on one object instead of in two suites that never meet.+ * <li>A populated {@code temperature_by_options}, where the multilingual checkpoint ships an+ * empty one. The {@code many-options} case has 14 options, which lands in the+ * {@code choice:11+} bucket this checkpoint ships at 0.1006 -- outside the accepted+ * [0.5, 5] band, so it is clamped to 0.5. Skipping the clamp sharpens the recorded top+ * probability of 0.3035 by roughly 5x and still returns the same argmax, so the+ * probabilities are the only thing that separates the two implementations.+ * <li>{@code max_len} 1024 against the english checkpoint's 512. The {@code over-budget} case+ * drops 1211 state tokens at that budget and marks all five questions truncated.+ * </ul>+ */+final class TypedDecisionsParityTest {++ /** The checkpoint this golden was recorded from; also the directory it lives in. */+ private static final String CHECKPOINT = "typed-decisions";++ /** Shared for the class: the graph is 1.7 GB of weights and opening it twice buys nothing. */+ private static Agent shared;++ @AfterAll+ static void closeAgent() {+ if (shared != null) {+ shared.close();+ shared = null;+ }+ }++ /**+ * The loaded config is the typed-decisions one, before any probability is compared.+ *+ * <p>Not ceremony. This checkpoint is fine-tuned FROM the english one and ships its+ * {@code tokenizer.json} byte for byte and its {@code answerdotai/ModernBERT-large} encoder,+ * so the english checkpoint opened against this graph raises nothing and answers+ * {@code {refund: 0.9077, replace: 0.0923}} -- the same digits as the right pairing, because+ * the weights are the graph's. {@link Prediction#model()} does not separate them either: the+ * ONNX path reports {@code laya-rl-agent-onnx} whichever checkpoint it opened. The budgets+ * and the temperatures do separate them, so they are what is asserted.+ */+ @Test+ @DisplayName("the agent under test is the typed-decisions checkpoint, not the english one")+ @SuppressWarnings("unchecked")+ void configIsTheTypedCheckpoint() throws IOException {+ Map<String, Object> golden = requireGolden();+ Map<String, Object> want = (Map<String, Object>) golden.get("config");+ AgentConfig config = agent(golden).config();+ assertEquals(want.get("encoder"), config.encoder(), "encoder");+ assertEquals(((Number) want.get("max_len")).intValue(), config.maxLen(), "max_len");+ assertEquals(((Number) want.get("head_max_len")).intValue(), config.headMaxLen(),+ "head_max_len");+ List<Object> temperature = (List<Object>) want.get("temperature");+ assertEquals(temperature.size(), config.temperature().length, "temperature length");+ for (int i = 0; i < temperature.size(); i++) {+ // Exact: these are read out of a JSON file, not computed, so any difference is a+ // parse bug rather than float noise.+ assertEquals(((Number) temperature.get(i)).doubleValue(), config.temperature()[i],+ "temperature[" + i + "]");+ }+ Map<String, Object> buckets = (Map<String, Object>) want.get("temperature_by_options");+ assertEquals(buckets.keySet(), config.temperatureByOptions().keySet(),+ "temperature_by_options buckets");+ for (Map.Entry<String, Object> entry : buckets.entrySet()) {+ assertEquals(((Number) entry.getValue()).doubleValue(),+ config.temperatureByOptions().get(entry.getKey()),+ "temperature_by_options[" + entry.getKey() + "]");+ }+ // The clamp, asserted on the one bucket that needs it rather than inferred from the map+ // above. 0.1006 in `rl_agent_config.json`, 0.5 in use: a port that read the file value+ // straight through would sharpen the many-options distribution by ~5x, and a lone 0.5 in+ // the golden could not be told from a bucket that was fitted at 0.5.+ Map<String, Object> raw = (Map<String, Object>) want.get("temperature_by_options_raw");+ assertEquals(0.10058280825614929, ((Number) raw.get("choice:11+")).doubleValue(),+ "the checkpoint still ships the sharpening bucket this case depends on");+ assertEquals(AgentConfig.TEMP_MIN, config.temperatureByOptions().get("choice:11+"),+ "choice:11+ must be clamped to TEMP_MIN, not applied as fitted");+ }++ /**+ * Every recorded workflow id set still routes to the workflow it was recorded under.+ *+ * <p>The link MODELS.md said was missing. A signature that stops matching would otherwise+ * leave the forward-pass cases below passing under a workflow name the router no longer+ * returns, which reads as coverage of a path that no longer exists.+ */+ @Test+ @DisplayName("each case's question ids still match the workflow it was recorded under")+ @SuppressWarnings("unchecked")+ void workflowsStillMatch() {+ Map<String, Object> golden = requireGolden();+ Map<String, Object> workflows = (Map<String, Object>) golden.get("workflows");+ assertEquals(4, workflows.size(), "the checkpoint is fine-tuned on four workflows");+ // DISTINCT, not a count of cases: two cases share the invoice_processing signature, so+ // counting cases made four workflows look like five and the floor failed on arithmetic+ // instead of on coverage.+ Set<String> covered = new TreeSet<>();+ for (Map.Entry<String, Object> entry+ : ((Map<String, Object>) golden.get("single")).entrySet()) {+ Map<String, Object> c = (Map<String, Object>) entry.getValue();+ Map<String, Object> questions = (Map<String, Object>) c.get("questions");+ String want = (String) c.get("workflow");+ assertEquals(want, Router.matchTypedDecisionsWorkflow(questions),+ entry.getKey() + ": workflow for ids " + questions.keySet());+ if (want != null) {+ assertTrue(workflows.containsKey(want), want + " is not one of the four");+ covered.add(want);+ }+ }+ assertEquals(workflows.keySet(), covered,+ "every workflow needs a recorded forward pass");+ }++ @TestFactory+ @DisplayName("single-state typed-decisions predictions match Python")+ @SuppressWarnings("unchecked")+ List<DynamicTest> singleMatches() throws IOException {+ Map<String, Object> golden = requireGolden();+ Map<String, Object> singles = (Map<String, Object>) golden.get("single");+ Agent agent = agent(golden);+ List<DynamicTest> tests = new ArrayList<>();+ for (Map.Entry<String, Object> entry : singles.entrySet()) {+ Map<String, Object> c = (Map<String, Object>) entry.getValue();+ tests.add(DynamicTest.dynamicTest(entry.getKey(), () -> {+ Prediction got = agent.predict(c.get("state"),+ questionsOf((Map<String, Object>) c.get("questions")),+ (String) c.get("lang"));+ assertEquals(c.get("model"), got.model());+ assertAnswers((Map<String, Object>) c.get("answers"), got.answers());+ assertUsage((Map<String, Object>) c.get("usage"), got.usage());+ }));+ }+ // Four workflows plus the three cases that are here for the checkpoint's own budgets and+ // temperature buckets. A factory whose fixture key emptied used to generate zero+ // assertions and still exit 0, which is what this floor is for.+ assertTrue(tests.size() >= 8, "expected the golden's cases, got " + tests.size());+ return tests;+ }++ @TestFactory+ @DisplayName("batched typed-decisions predictions match Python at every batch size")+ @SuppressWarnings("unchecked")+ List<DynamicTest> batchMatches() throws IOException {+ Map<String, Object> golden = requireGolden();+ Agent agent = agent(golden);+ List<DynamicTest> tests = new ArrayList<>();+ for (Object configObject : (List<Object>) golden.get("batch")) {+ Map<String, Object> config = (Map<String, Object>) configObject;+ Object size = config.get("batch_size");+ boolean sort = Boolean.TRUE.equals(config.get("sort_by_length"));+ tests.add(DynamicTest.dynamicTest(+ "batch_size=" + (size == null ? "all" : size) + " sort=" + sort, () -> {+ List<Prediction> got = agent.predictBatch(+ (List<Object>) config.get("states"),+ questionsOf((Map<String, Object>) config.get("questions")),+ null, size == null ? 0 : ((Number) size).intValue(), sort);+ List<Object> want = (List<Object>) config.get("results");+ assertEquals(want.size(), got.size(), "one prediction per state");+ for (int i = 0; i < want.size(); i++) {+ Map<String, Object> w = (Map<String, Object>) want.get(i);+ // The caller's order, whatever order the batcher grouped them in.+ assertAnswers((Map<String, Object>) w.get("answers"),+ got.get(i).answers());+ assertUsage((Map<String, Object>) w.get("usage"), got.get(i).usage());+ }+ }));+ }+ assertTrue(tests.size() >= 3, "expected the golden's batch configs, got " + tests.size());+ return tests;+ }++ private static Map<String, Object> requireGolden() {+ Map<String, Object> golden = Fixtures.load("typed_decisions.json");+ Assumptions.assumeFalse(golden.containsKey("skipped"),+ "typed_decisions.json was recorded without a graph: set " + Fixtures.TYPED_GRAPH_ENV+ + " and re-run scripts/gen_fixtures.py");+ return golden;+ }++ /** The shared agent, opened on first use so the assumptions still skip cleanly. */+ private static synchronized Agent agent(Map<String, Object> golden) throws IOException {+ if (shared == null) {+ shared = openAgent(golden);+ }+ return shared;+ }++ private static Agent openAgent(Map<String, Object> golden) throws IOException {+ String checkpoint = (String) golden.getOrDefault("checkpoint", CHECKPOINT);+ Path model = Fixtures.checkpoint(checkpoint);+ Assumptions.assumeTrue(model != null, Fixtures.missingCheckpoint(checkpoint));+ String graph = System.getenv(Fixtures.TYPED_GRAPH_ENV);+ Assumptions.assumeTrue(graph != null && !graph.isBlank(),+ "set " + Fixtures.TYPED_GRAPH_ENV + " to the laya.onnx exported from the "+ + checkpoint + " checkpoint");+ Path graphPath = Paths.get(graph);+ Path graphDirectory = Files.isDirectory(graphPath) ? graphPath : graphPath.getParent();+ Assumptions.assumeTrue(graphDirectory != null && Files.isDirectory(graphDirectory),+ Fixtures.TYPED_GRAPH_ENV + " does not name a graph: " + graph);+ return Agent.open(model, graphDirectory);+ }+}diff --git a/laya-java/scripts/gen_fixtures.py b/laya-java/scripts/gen_fixtures.pyindex 1b335cb9..26f66aa2 100644--- a/laya-java/scripts/gen_fixtures.py+++ b/laya-java/scripts/gen_fixtures.py@@ -1817,6 +1817,429 @@ def predict_golden():"single": single, "batch": batches}+# -- the typed-decisions checkpoint ---------------------------------------------------------+#+# `laya-typed-decisions` is fine-tuned from the ENGLISH checkpoint, which is what makes a+# fixture family for it worth its weight. It ships the same `tokenizer.json` byte for byte+# (diffed: identical) and the same `answerdotai/ModernBERT-large` encoder, and nothing in a+# `.onnx` file says which checkpoint traced it. Measured: opening the ENGLISH checkpoint against+# the typed graph raises nothing and answers {refund: 0.9077, replace: 0.0923} on a two-option+# question -- the same digits, to the last one, as the typed checkpoint against its own graph,+# because the weights are entirely the graph's and the two configs agree on the `choice:2`+# temperature. `model` does not separate them either: the ONNX path reports+# `laya-rl-agent-onnx` whichever checkpoint it opened. Only the multilingual checkpoint fails+# loudly, and for an unrelated reason -- a 235292 token id against a 50368-row embedding, which+# ONNX Runtime rejects at the `node_embedding` Gather.+#+# So the discriminator has to come from `rl_agent_config.json`, which is where the two do differ:+# `max_len` 1024 against english's 512, `head_max_len` 256 against 192, `model_name`+# `laya-typed-decisions` against `rl-agent`, and base temperatures [1.0148, 1.0374, 1.0575]+# against [1.6369, 1.2514, 1.9834]. Those are recorded below as `config`, and the Java test+# asserts them before it compares a single probability.+#+# The question ids are read from `laya.router` rather than retyped: an exact id-set match is the+# entire signature `match_typed_decisions_workflow` tests, so a signature change has to move this+# family rather than leave it asserting a workflow name the router no longer returns.++# `invoice_processing` is the one workflow whose TRAINED schema is in this repository, in+# `examples/32_typed_decisions_workflow.py` (dataset `LocalLLaMA/typed-decisions`, config+# `invoice_processing`). It is duplicated here rather than imported because that example has a+# module-level `agent = load("typed-decisions")`, so importing it would download 846 MB and run+# the model; `_typed_questions` below fails the generator if the ids ever stop matching the+# router's signature, which is the drift that would matter.+TYPED_INVOICE_QUESTIONS = {+ "disposition": {+ "type": "choice",+ "instructions": "How should this vendor invoice be dispositioned?",+ "criteria": {+ "approve": "Matches the order and delivery; approve for payment.",+ "hold": "Something needs confirming before payment; hold pending clarification.",+ "manual_review": "A human in finance must review the discrepancy.",+ "reject": "Should not be paid: duplicate, unauthorised or materially wrong.",+ },+ },+ "matches_order": {+ "type": "noul",+ "instructions": "The invoice reconciles with the purchase order and the recorded delivery.",+ "criteria": {+ "false": "There is a discrepancy against the order or the delivery.",+ "true": "Line items, quantities and amounts reconcile.",+ },+ },+ "duplicate": {+ "type": "noul",+ "instructions": "This invoice appears to duplicate an invoice already submitted.",+ },+ "discrepancy_severity": {+ "type": "score",+ "instructions": "How material is any discrepancy between the invoice, the order "+ "and the delivery?",+ "criteria": [+ "None: everything reconciles.",+ "Trivial: rounding or a cosmetic difference.",+ "Moderate: a real difference worth confirming.",+ "Material: a large or unexplained difference.",+ ],+ },+ "urgency": {+ "type": "score",+ "instructions": "How time-sensitive is processing this invoice?",+ "criteria": [+ "No time pressure; can wait indefinitely.",+ "Routine; handle within the normal queue.",+ "Elevated; should be handled within the same week.",+ "Critical; requires action within the same day.",+ ],+ },+}++# The other three workflows' TRAINED instruction text is not in this repository -- only their id+# signatures are, in `laya.router._TYPED_DECISION_WORKFLOWS`. These schemas therefore carry the+# real ids and instruction text written here, which is stated rather than glossed over: what the+# fixture gates is the forward pass and the routing decision that reaches it, and both are real+# for any schema whose ids match. The text is recorded verbatim in the fixture, so a port renders+# the identical prompt and must reproduce the identical answer.+TYPED_CUSTOMER_SERVICE_QUESTIONS = {+ "action": {+ "type": "choice",+ "instructions": "What should the support team do next?",+ "criteria": {+ "refund": "Return the money and close the ticket.",+ "replace": "Ship a replacement unit.",+ "explain": "Answer the question; nothing to send.",+ "escalate": "Hand to a senior agent.",+ },+ },+ "category": {+ "type": "choice",+ "instructions": "Which queue does this ticket belong in?",+ "criteria": {+ "billing": "Charges, invoices and refunds.",+ "shipping": "Delivery, tracking and damage in transit.",+ "account": "Login, access and account settings.",+ },+ },+ "churn_risk": {+ "type": "score",+ "instructions": "How likely is this customer to leave over this?",+ "criteria": [+ "Not at risk; routine contact.",+ "Mildly annoyed.",+ "Considering alternatives.",+ "Explicitly threatening to cancel.",+ ],+ },+ "needs_human": {+ "type": "noul",+ "instructions": "This ticket needs a human agent rather than an automated reply.",+ },+ "urgency": {+ "type": "score",+ "instructions": "How quickly must this be answered?",+ "criteria": [+ "No time pressure.",+ "Within the normal queue.",+ "Same week.",+ "Same day.",+ ],+ },+}++TYPED_SECURITY_QUESTIONS = {+ "credential_compromise": {+ "type": "noul",+ "instructions": "Credentials belonging to this account were exposed or reused.",+ "criteria": {+ "false": "No sign that a secret left the organisation.",+ "true": "A password, token or key is in the wrong hands.",+ },+ },+ "disposition": {+ "type": "choice",+ "instructions": "How should this alert be dispositioned?",+ "criteria": {+ "close": "Benign; close with no action.",+ "monitor": "Watch for a repeat before acting.",+ "contain": "Isolate the host or revoke the credential now.",+ "escalate": "Wake an incident responder.",+ },+ },+ "severity": {+ "type": "score",+ "instructions": "How severe is the impact if this is real?",+ "criteria": [+ "Informational.",+ "Low: one account, no data.",+ "High: production data or lateral movement.",+ "Critical: active, spreading, or customer data.",+ ],+ },+ "true_positive": {+ "type": "noul",+ "instructions": "This alert describes genuine malicious activity rather than noise.",+ },+ "urgency": {+ "type": "score",+ "instructions": "How quickly must someone respond?",+ "criteria": [+ "No time pressure.",+ "Next business day.",+ "Within the hour.",+ "Immediately.",+ ],+ },+}++TYPED_AGENT_TRACE_QUESTIONS = {+ "action": {+ "type": "choice",+ "instructions": "What should be done with this agent run?",+ "criteria": {+ "accept": "The run did what was asked; accept it.",+ "retry": "Transient failure; run it again.",+ "rollback": "Undo what it changed.",+ "escalate": "A person must look at this.",+ },+ },+ "needs_review": {+ "type": "noul",+ "instructions": "A human should read this trace before the result is used.",+ },+ "outcome": {+ "type": "choice",+ "instructions": "How did the run end?",+ "criteria": {+ "success": "Completed the task.",+ "partial": "Completed some of it.",+ "failure": "Did not complete the task.",+ },+ },+ "risk": {+ "type": "score",+ "instructions": "How much damage could this run's actions have caused?",+ "criteria": [+ "Read-only; no effect.",+ "Reversible writes.",+ "Irreversible writes to internal systems.",+ "Irreversible, customer-visible, or spending money.",+ ],+ },+ "urgency": {+ "type": "score",+ "instructions": "How quickly must this be looked at?",+ "criteria": [+ "No time pressure.",+ "Next business day.",+ "Within the hour.",+ "Immediately.",+ ],+ },+}++# The two states from `examples/32_typed_decisions_workflow.py`: one with a 40% unit-price+# variance against the purchase order, one that reconciles exactly. They are a pair on purpose --+# the same schema on two states that differ only in the numbers is what shows the forward pass is+# reading the state rather than the questions.+TYPED_INVOICE_VARIANCE = {+ "invoice": {+ "id": "INV-2026-8842", "vendor": "Northwind Castings", "currency": "USD",+ "total_usd": 300020.0,+ "lines": [{"sku": "SKU-940", "qty": 1000, "unit_usd": 300.02, "total_usd": 300020.0}],+ },+ "purchase_order": {+ "id": "PO-3392", "total_usd": 214300.0,+ "lines": [{"qty": 1000, "unit_usd": 214.3}],+ "freight_terms": "freight prepaid by vendor, not separately billable",+ },+ "delivery": {"date": "2026-03-18", "received_qty": 1000,+ "condition": "accepted with exceptions"},+ "payment": {"terms": "net 45", "status": "overdue", "days_until_due": -3},+ "vendor_history": {+ "invoices_12m": 40, "disputes_12m": 0,+ "prior_invoice_ids": ["INV-2026-6633", "INV-2026-9563", "INV-2026-7939"],+ },+}++TYPED_INVOICE_CLEAN = {+ "invoice": {+ "id": "INV-2026-9107", "vendor": "Cedar Freight", "currency": "USD",+ "total_usd": 18450.0,+ "lines": [{"sku": "SKU-221", "qty": 300, "unit_usd": 61.5, "total_usd": 18450.0}],+ },+ "purchase_order": {+ "id": "PO-3477", "total_usd": 18450.0,+ "lines": [{"qty": 300, "unit_usd": 61.5}],+ "freight_terms": "freight prepaid by vendor, not separately billable",+ },+ "delivery": {"date": "2026-03-20", "received_qty": 300, "condition": "accepted in full"},+ "payment": {"terms": "net 30", "status": "current", "days_until_due": 18},+ "vendor_history": {"invoices_12m": 12, "disputes_12m": 0,+ "prior_invoice_ids": ["INV-2026-7781"]},+}++# 14 options, which is what puts the `choice:11+` temperature bucket in play. This checkpoint+# ships that bucket at 0.1006 -- outside the [0.5, 5] band `laya.common` accepts, so it is clamped+# to 0.5 and the answer is published uncalibrated. A port that reads the bucket and skips the+# clamp sharpens this distribution by ~5x and still returns a plausible-looking argmax, so the+# probabilities here are the only thing that tells the two apart.+TYPED_MANY_OPTION_QUESTIONS = {+ "band": {"type": "choice", "instructions": "Which spend band does this invoice fall in?",+ "criteria": {("band%02d" % i): ("tier number %d" % i) for i in range(14)}},+}++# `max_len` is 1024 here against the english checkpoint's 512, so a state that truncates on one+# does not truncate on the other. This one is long enough to drop tokens at 1024, which is what+# makes `state_tokens_dropped` and `truncated` non-zero rather than decorative.+TYPED_OVER_BUDGET_STATE = ("line item number one, unit price 300.02 against a purchase order at "+ "214.30, delivery accepted with exceptions. ") * 90++# A question set whose ids are NOT a workflow signature. The checkpoint still answers it -- being+# fine-tuned on four schemas does not make it refuse a fifth -- and recording that is what stops+# a port from special-casing the signatures somewhere in the forward pass, where they have no+# business being. Routing is the only place the signature is read.+TYPED_OFF_WORKFLOW_QUESTIONS = {+ "department": {"type": "choice", "instructions": "Which team owns this?",+ "criteria": {"billing": "charges and refunds", "other": "anything else"}},+}+++def _typed_questions(workflow, questions):+ """`questions`, after checking its ids still are `workflow`'s signature.++ The schemas above are literals, and the signatures they have to match live in `laya.router`.+ Nothing else connects the two, so without this the router could gain a question id and this+ family would carry on recording a forward pass under a workflow name+ `match_typed_decisions_workflow` no longer returns -- a fixture that names a routing outcome+ it does not produce, which is worse than no fixture.+ """+ from laya.router import _TYPED_DECISION_WORKFLOWS, match_typed_decisions_workflow++ expected = _TYPED_DECISION_WORKFLOWS[workflow]+ got = match_typed_decisions_workflow(questions)+ if got != workflow:+ raise SystemExit(+ "gen_fixtures: the %s schema's ids are %s, but laya.router's signature is %s "+ "(match_typed_decisions_workflow returned %r)"+ % (workflow, sorted(questions), sorted(expected), got))+ return questions+++# (case id, state, lang, workflow or None, questions). `workflow` is recorded so the Java test can+# assert the routing decision and the forward pass on the SAME object: MODELS.md's gap was that+# `Router` resolving `typed-decisions` was tested while nothing ever ran the checkpoint, so a+# fixture that records one without the other leaves half of it open.+TYPED_CASES = [+ ("invoice-variance", TYPED_INVOICE_VARIANCE, None, "invoice_processing",+ TYPED_INVOICE_QUESTIONS),+ ("invoice-clean", TYPED_INVOICE_CLEAN, None, "invoice_processing", TYPED_INVOICE_QUESTIONS),+ ("customer-service", "We were billed twice for March. Third time this year -- refund it "+ "today or we are moving to a competitor.", None, "customer_service",+ TYPED_CUSTOMER_SERVICE_QUESTIONS),+ ("security-incident", {"alert": "impossible travel", "user": "[email protected]",+ "signins": [{"city": "Lagos", "at": "2026-03-18T09:02Z"},+ {"city": "Vilnius", "at": "2026-03-18T09:41Z"}],+ "mfa": "satisfied by push, approved on the second prompt",+ "prior_alerts_30d": 0}, None, "security_incidents",+ TYPED_SECURITY_QUESTIONS),+ ("agent-trace", {"run": "agt-7741", "tool_calls": 14, "wrote": ["s3://prod-reports/q1.csv"],+ "errors": ["RateLimit on call 9, retried"],+ "final": "uploaded the report, but the row count is 0"},+ None, "agent_trace_observability", TYPED_AGENT_TRACE_QUESTIONS),+ ("many-options", "The invoice totals 300,020 USD against a 214,300 USD order.", None, None,+ TYPED_MANY_OPTION_QUESTIONS),+ # Same ids as the two invoice cases, so the same workflow -- recording this as "no workflow"+ # because the state is different was the first thing the Java test caught.+ ("over-budget", TYPED_OVER_BUDGET_STATE, None, "invoice_processing",+ TYPED_INVOICE_QUESTIONS),+ ("off-workflow", "We were billed twice for March and want a refund today.", "en", None,+ TYPED_OFF_WORKFLOW_QUESTIONS),+]+++def typed_decisions_golden():+ """`ONNXAgent.predict` and `predict_batch` on the `typed-decisions` checkpoint.++ The sibling of `predict_golden`, against the third checkpoint. It is a separate family and a+ separate graph because a graph is traced from ONE checkpoint and `LAYA_ONNX_GRAPH` names one+ directory: pointing that at the typed graph while `predict.json` expects the multilingual one+ would re-record `predict.json` from the wrong weights, and both sides of that comparison come+ from the same mistake, so it passes. `LAYA_TYPED_ONNX_GRAPH` keeps the two unable to collide.++ Records `skipped` when the graph is absent, so `unverifiable` leaves the committed file alone+ rather than replacing it with a marker.++ Platform-sensitive in the last reported digit, as `predict_golden` is; the Java test's+ tolerance is the gate, not byte equality of this file.+ """+ import os++ graph = os.environ.get("LAYA_TYPED_ONNX_GRAPH")+ rig = os.environ.get("LAYA_FIXTURE_CHECKPOINTS", DEFAULT_CHECKPOINT_ROOT)+ model = "typed-decisions"+ if not graph or not os.path.exists(graph):+ return {"skipped": "set LAYA_TYPED_ONNX_GRAPH to a laya.onnx exported from the "+ "typed-decisions checkpoint to record this family"}+ from laya.onnx_agent import ONNXAgent+ from laya.router import _TYPED_DECISION_WORKFLOWS++ agent = ONNXAgent(os.path.join(rig, model), onnx_path=graph)+ cfg = agent.cfg+ single = {}+ for cid, state, lang, workflow, questions in TYPED_CASES:+ if workflow is not None:+ questions = _typed_questions(workflow, questions)+ result = agent.predict(state, questions, lang=lang)+ single[cid] = {"state": state, "lang": lang, "workflow": workflow,+ "questions": questions, "model": result["model"],+ "answers": result["answers"], "usage": result["usage"]}++ # Batched over the real invoice schema, including the empty state and two short ones, so+ # collation and padding are exercised here too. The empty state is the one that matters: it+ # contributes 0 state tokens and still costs 296 input tokens of rendered questions, so a+ # port that skips an empty member rather than padding it gets a different batch shape.+ batch_states = [TYPED_INVOICE_VARIANCE, TYPED_INVOICE_CLEAN,+ "INV-2026-1004, 980 USD, matches PO-3100 and delivery.",+ "duplicate of INV-2026-1004", ""]+ batch_questions = _typed_questions("invoice_processing", TYPED_INVOICE_QUESTIONS)+ batches = []+ for batch_size, sort_by_length in ((None, False), (2, False), (3, True)):+ kwargs = {"sort_by_length": sort_by_length}+ if batch_size is not None:+ kwargs["batch_size"] = batch_size+ results = agent.predict_batch(batch_states, batch_questions, **kwargs)+ batches.append({"batch_size": batch_size, "sort_by_length": sort_by_length,+ "states": batch_states, "questions": batch_questions,+ "results": [{"model": r["model"], "answers": r["answers"],+ "usage": r["usage"]} for r in results]})++ return {+ "checkpoint": model,+ "graph": os.path.basename(graph),+ # What tells this checkpoint from the english one it was fine-tuned from, which ships the+ # identical tokenizer and the identical encoder. Asserted by the Java test before any+ # probability is compared, because a mismatch here means the numbers below were produced+ # by the wrong weights and comparing them is worse than not running.+ "config": {+ "model_name": cfg["model_name"],+ "encoder": cfg["encoder"],+ "max_len": cfg["max_len"],+ "head_max_len": cfg["head_max_len"],+ # `agent.temperature*`, not `cfg[...]`: the agent clamps on load and the config+ # dict keeps the file's value, so the two disagree here and a port exposes only the+ # clamped one. Recording both is what makes the clamp visible -- a lone 0.5 cannot+ # be told from a temperature that was fitted at 0.5, and this checkpoint's+ # `choice:11+` is 0.1006 in the file and 0.5 in use.+ "temperature": list(agent.temperature),+ "temperature_by_options": dict(agent.temperature_by_options),+ "temperature_raw": list(agent.temperature_raw),+ "temperature_by_options_raw": dict(agent.temperature_by_options_raw),+ },+ "workflows": {name: sorted(ids) for name, ids in sorted(_TYPED_DECISION_WORKFLOWS.items())},+ "single": single,+ "batch": batches,+ }++def email_clean():"""`laya.email`'s cleaner and state builder, over a corpus built from its own reasoning.@@ -3105,6 +3528,7 @@ FAMILIES = {"python_json.json": python_json,"email.json": email_clean,"predict.json": predict_golden,+ "typed_decisions.json": typed_decisions_golden,}