Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(java): decide against a JSON schema, and the values a label is not

Adeshina Lasisi40a5260efb88
diff --git a/.github/workflows/java.yml b/.github/workflows/java.yml
index b8a8ac0c..c439af38 100644
--- a/.github/workflows/java.yml
+++ b/.github/workflows/java.yml
@@ -156,11 +156,16 @@ jobs:
- name: Tests that need no model
working-directory: laya-java
run: ./gradlew --no-daemon test
+ # Measured locally on JDK 17 with no checkpoints: 790 registered, 14 abort, 776 run. The
+ # floor keeps the 5-test margin the 685/680 pair had, which is what makes it bind the
+ # class this bump is for: DecisionsTest is 91 of those 776, so dropping it leaves 685 and
+ # the floor exits 1. Measured both ways -- 776 passes at 771, and with DecisionsTest.java
+ # moved out of the tree the suite is still green and the floor still exits 1.
- name: Model-free suite was not vacuous
if: ${{ !cancelled() }}
run: >
python laya-java/scripts/check_test_results.py
- laya-java/laya-java/build/test-results/test --min-tests 680 --allow-aborted
+ laya-java/laya-java/build/test-results/test --min-tests 771 --allow-aborted
# The samples are single-file source programs, deliberately NOT a Gradle module: a
# `samples` subproject would inherit the root build's MavenPublication and
@@ -319,7 +324,7 @@ jobs:
if: ${{ !cancelled() }}
run: >
python laya-java/scripts/check_test_results.py
- laya-java/laya-java/build/test-results/test --min-tests 680 --allow-aborted
+ laya-java/laya-java/build/test-results/test --min-tests 771 --allow-aborted
--require-passed TestJvmVersionTest
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: ${{ failure() }}
diff --git a/laya-java/CHANGELOG.md b/laya-java/CHANGELOG.md
index ef9dba65..bc4f81ee 100644
--- a/laya-java/CHANGELOG.md
+++ b/laya-java/CHANGELOG.md
@@ -59,6 +59,22 @@ see [Installing](README.md#installing).
- `Shortlist`: cosine ranking over a caller's embedder, with an LRU cache and numpy's tie order.
- `LayaEmail`: `cleanEmailBody` and `emailState`, with the English, Portuguese, Spanish and French
marker sets, and `emailQuestions` re-exported from `Presets`.
+- `Decisions`: schema-driven decisions — `decide` and `decideBatch` over a JSON schema. An
+ `enum` or `const` becomes a `choice`, a `boolean` a `noul`, a bounded `integer`/`number` a
+ `score`; `anyOf`/`oneOf` with one non-null branch is unwrapped (pydantic's `Optional`), a
+ one-item `allOf` is unwrapped with the outer keys on top, and a local `$ref` is inlined. The
+ answers come back as the schema's **values** — the choice's value, not the label it was shown
+ under — a score as `minimum + argmax(probabilities)`, and a `noul` as `>= 0.5`, so exactly
+ `0.5` is true. A field below `minConfidence` is `null` and a field that was never answered is
+ **absent**, because "not believed" and "not asked" are different outcomes. Anything that
+ cannot be answered from a fixed option set is refused with a `SchemaException` naming the
+ path. Unlike the reference, the gate is not re-implemented: `decide` reads
+ `ConfidenceGate`'s report, so the value a caller acts on and the state the gate reports cannot
+ drift apart. Only the JSON-schema path is ported — there is no pydantic on the JVM.
+- `BatchPredictor`: the batching half of `Predictor`, implemented by `Agent`. The reference
+ raises `TypeError` when a runner has no `predict_batch`; here a runner that cannot batch does
+ not compile, because looping `decide` N times behind the caller's back is the one thing
+ neither wants to do silently.
- `Predictor`: one interface satisfied unchanged by both `Agent` and `Router`.
- `json.PythonJson`: CPython's `repr` and `%.0f`, which the router's reason strings are built from.
- `scripts/prepare_checkpoint.py`: a pinned download and export, writing the two directories
@@ -66,7 +82,9 @@ see [Installing](README.md#installing).
### Testing
-- 769 tests, 626 of which need no model at all.
+- 1,190 tests, 776 of which need no model at all (measured on JDK 17 with both shipped
+ checkpoints present; the rest need one, and the end-to-end golden additionally needs a traced
+ graph).
- Every expectation is **generated from the Python package**, never written by hand:
`laya-java/fixtures/*.json`, regenerated by `scripts/gen_fixtures.py` and gated by `--check` in
CI so a change to `laya/` that moves a recorded contract shows up here rather than as drift.
diff --git a/laya-java/README.md b/laya-java/README.md
index 3337f767..00e5a1c2 100644
--- a/laya-java/README.md
+++ b/laya-java/README.md
@@ -13,12 +13,13 @@ four questions about a document cost one batched encode rather than four round t
Implemented: tokenizer, sequence builder, config, ONNX inference (fused or split graph), answer
decoding, `predict`, `predictBatch`, usage and truncation reporting, script and language detection
(`lang.LanguageDetection`), the question presets (`Presets`), the checkpoint `Router` with its
-load-and-evict lifecycle, the embedding `Shortlist` with its LRU cache, and the email cleaner
-and state builder (`LayaEmail`).
+load-and-evict lifecycle, the embedding `Shortlist` with its LRU cache, the email cleaner
+and state builder (`LayaEmail`), the abstention gate (`ConfidenceGate`), scanning a state longer
+than the context window (`predictLong`), and schema-driven decisions (`Decisions.decide` /
+`decideBatch`).
-Not implemented yet: hooks, `predictLong`, structured `decide`, the
-`laya-java-client` HTTP module, Android. **Not published to Maven Central** — see
-[Installing](#installing).
+Not implemented yet: hooks, the `laya-java-client` HTTP module, Android. **Not published to
+Maven Central** — see [Installing](#installing).
## Where to go next
@@ -321,6 +322,43 @@ c are correct" holds only after temperatures have been fitted and validated for
and question shape. Pass a `Map` instead of a scalar to gate each option-count bucket
(`choice:2`, `score:6-10`, …) at the level its calibration actually earns.
+## Deciding against a JSON schema
+
+```java
+import com.convaiinnovations.laya.json.Json;
+
+Map<String, Object> schema = (Map<String, Object>) Json.parse("""
+ {"type": "object", "properties": {
+ "department": {"enum": ["billing", "support", "sales"]},
+ "urgency": {"type": "integer", "minimum": 1, "maximum": 5},
+ "needs_human": {"type": "boolean"}}}""");
+
+Decisions.Decision d = Decisions.decide(agent, ticket, schema, null, 0.8);
+Map<String, Object> values = d.values(); // {"department": "billing", "urgency": 4, ...}
+```
+
+An `enum` or a `const` becomes a `choice`, a `boolean` becomes a `noul`, a bounded `integer` or
+`number` becomes a `score`, and the answers come back as **the schema's own values**: the choice's
+value and not the label it was shown under, so `enum: [10, 20, 30]` decides to the integer `20`.
+`anyOf` / `oneOf` with one non-null branch is unwrapped (pydantic's `Optional`), a one-item
+`allOf` is unwrapped with the outer keys on top, and a local `$ref` is inlined.
+
+Anything that cannot be answered from a fixed option set — a free string, an array, a nested
+object, a union of two real types — is refused with a `SchemaException` **naming the path**, which
+is the only part of the message a caller holding a 32-property schema can act on.
+
+`minConfidence` wires straight into `ConfidenceGate`: a field whose answer falls below the bar
+comes back as `null`, and a field that was never answered is **absent** rather than null, because
+"not believed" and "not asked" are different outcomes. The gate's full report is on
+`d.gate()`. `Decisions.decideBatch(agent, states, schema)` does the same for many states in one
+batched call, in input order.
+
+`Decisions.questions(schema)` and `Decisions.answersToJson(answers, schema)` are the two halves on
+their own, for a caller that runs the model itself.
+
+There is no pydantic on the JVM, so only the JSON-schema path is ported; mapping Java records
+would be a new design rather than a port.
+
## A state longer than the context window
```java
diff --git a/laya-java/fixtures/structured.json b/laya-java/fixtures/structured.json
new file mode 100644
index 00000000..806ed936
--- /dev/null
+++ b/laya-java/fixtures/structured.json
@@ -0,0 +1,3554 @@
+{
+ "limits": {
+ "max_properties": 32,
+ "max_options": 32,
+ "max_score_levels": 10
+ },
+ "schemas": [
+ {
+ "case": "enum-is-a-choice",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support",
+ "sales"
+ ]
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "What is `dept`?",
+ "criteria": {
+ "billing": null,
+ "support": null,
+ "sales": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ],
+ [
+ "sales",
+ "sales"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "enum-of-numbers-keeps-its-values",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "code": {
+ "enum": [
+ 10,
+ 20,
+ 30
+ ]
+ }
+ }
+ },
+ "questions": {
+ "code": {
+ "type": "choice",
+ "instructions": "What is `code`?",
+ "criteria": {
+ "10": null,
+ "20": null,
+ "30": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "code",
+ "kind": "choice",
+ "options": [
+ [
+ "10",
+ 10
+ ],
+ [
+ "20",
+ 20
+ ],
+ [
+ "30",
+ 30
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "enum-with-a-null-branch",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "tier": {
+ "enum": [
+ "gold",
+ null
+ ]
+ }
+ }
+ },
+ "questions": {
+ "tier": {
+ "type": "choice",
+ "instructions": "What is `tier`?",
+ "criteria": {
+ "gold": null,
+ "null": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "tier",
+ "kind": "choice",
+ "options": [
+ [
+ "gold",
+ "gold"
+ ],
+ [
+ "null",
+ null
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "const-is-a-one-option-choice",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "kind": {
+ "const": "refund"
+ }
+ }
+ },
+ "questions": {
+ "kind": {
+ "type": "choice",
+ "instructions": "What is `kind`?",
+ "criteria": {
+ "refund": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "kind",
+ "kind": "choice",
+ "options": [
+ [
+ "refund",
+ "refund"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "const-null-is-still-a-const",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "kind": {
+ "const": null
+ }
+ }
+ },
+ "questions": {
+ "kind": {
+ "type": "choice",
+ "instructions": "What is `kind`?",
+ "criteria": {
+ "null": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "kind",
+ "kind": "choice",
+ "options": [
+ [
+ "null",
+ null
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "boolean-is-a-noul",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "needs_human": {
+ "type": "boolean"
+ }
+ }
+ },
+ "questions": {
+ "needs_human": {
+ "type": "noul",
+ "instructions": "Is `needs_human` true?"
+ }
+ },
+ "plan": [
+ {
+ "name": "needs_human",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "enum-of-booleans-is-a-noul",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "flag": {
+ "enum": [
+ true,
+ false
+ ]
+ }
+ }
+ },
+ "questions": {
+ "flag": {
+ "type": "noul",
+ "instructions": "Is `flag` true?"
+ }
+ },
+ "plan": [
+ {
+ "name": "flag",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "integer-is-a-score",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "urgency": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 5
+ }
+ }
+ },
+ "questions": {
+ "urgency": {
+ "type": "score",
+ "instructions": "Score `urgency` from 1 to 5",
+ "criteria": [
+ "1",
+ "2",
+ "3",
+ "4",
+ "5"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "urgency",
+ "kind": "score",
+ "options": [],
+ "minimum": 1
+ }
+ ]
+ },
+ {
+ "case": "number-with-integer-bounds-is-a-score",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "number",
+ "minimum": 0,
+ "maximum": 2
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 0 to 2",
+ "criteria": [
+ "0",
+ "1",
+ "2"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 0
+ }
+ ]
+ },
+ {
+ "case": "score-from-zero",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 0,
+ "maximum": 3
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 0 to 3",
+ "criteria": [
+ "0",
+ "1",
+ "2",
+ "3"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 0
+ }
+ ]
+ },
+ {
+ "case": "score-with-a-negative-minimum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": -2,
+ "maximum": 2
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from -2 to 2",
+ "criteria": [
+ "-2",
+ "-1",
+ "0",
+ "1",
+ "2"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": -2
+ }
+ ]
+ },
+ {
+ "case": "score-of-one-level",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 7,
+ "maximum": 7
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 7 to 7",
+ "criteria": [
+ "7"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 7
+ }
+ ]
+ },
+ {
+ "case": "description-replaces-the-generated-instruction",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ],
+ "description": "Which desk owns this?"
+ },
+ "ok": {
+ "type": "boolean",
+ "description": "Is the customer satisfied?"
+ },
+ "n": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 3,
+ "description": "How loud is the complaint?"
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which desk owns this?",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ },
+ "ok": {
+ "type": "noul",
+ "instructions": "Is the customer satisfied?"
+ },
+ "n": {
+ "type": "score",
+ "instructions": "How loud is the complaint?",
+ "criteria": [
+ "1",
+ "2",
+ "3"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ },
+ {
+ "name": "ok",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 1
+ }
+ ]
+ },
+ {
+ "case": "anyof-optional-is-unwrapped",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "anyOf": [
+ {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "description": "pick a desk"
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "pick a desk",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "oneof-optional-is-unwrapped",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "oneOf": [
+ {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 4
+ },
+ {
+ "type": "null"
+ }
+ ]
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 1 to 4",
+ "criteria": [
+ "1",
+ "2",
+ "3",
+ "4"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 1
+ }
+ ]
+ },
+ {
+ "case": "anyof-branch-keeps-its-own-description",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "anyOf": [
+ {
+ "enum": [
+ "a",
+ "b"
+ ],
+ "description": "the branch's"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "description": "the outer one"
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "the branch's",
+ "criteria": {
+ "a": null,
+ "b": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "a",
+ "a"
+ ],
+ [
+ "b",
+ "b"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "nullable-type-list-is-unwrapped",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "type": [
+ "string",
+ "null"
+ ],
+ "enum": [
+ "billing",
+ "support"
+ ]
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "What is `dept`?",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "nullable-integer-type-list",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": [
+ "integer",
+ "null"
+ ],
+ "minimum": 0,
+ "maximum": 3
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 0 to 3",
+ "criteria": [
+ "0",
+ "1",
+ "2",
+ "3"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 0
+ }
+ ]
+ },
+ {
+ "case": "allof-of-one-is-unwrapped-outer-keys-on-top",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "allOf": [
+ {
+ "$ref": "#/$defs/Desk"
+ }
+ ],
+ "description": "the field's"
+ }
+ },
+ "$defs": {
+ "Desk": {
+ "enum": [
+ "billing",
+ "support"
+ ],
+ "description": "An enumeration."
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "the field's",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "allof-of-one-inline-outer-keys-win",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "allOf": [
+ {
+ "enum": [
+ "a",
+ "b"
+ ],
+ "description": "the branch's"
+ }
+ ],
+ "description": "the field's"
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "the field's",
+ "criteria": {
+ "a": null,
+ "b": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "a",
+ "a"
+ ],
+ [
+ "b",
+ "b"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "ref-sibling-keys-beat-the-definition",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "$ref": "#/$defs/Scale",
+ "maximum": 1
+ }
+ },
+ "$defs": {
+ "Scale": {
+ "type": "integer",
+ "minimum": 0,
+ "maximum": 3
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 0 to 1",
+ "criteria": [
+ "0",
+ "1"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 0
+ }
+ ]
+ },
+ {
+ "case": "ref-into-defs-drops-the-definition-description",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "$ref": "#/$defs/Desk"
+ }
+ },
+ "$defs": {
+ "Desk": {
+ "enum": [
+ "billing",
+ "support"
+ ],
+ "description": "An enumeration."
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "What is `dept`?",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "ref-into-definitions-draft-07",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "$ref": "#/definitions/Desk"
+ }
+ },
+ "definitions": {
+ "Desk": {
+ "enum": [
+ 1,
+ 2,
+ 3
+ ]
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "What is `dept`?",
+ "criteria": {
+ "1": null,
+ "2": null,
+ "3": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "1",
+ 1
+ ],
+ [
+ "2",
+ 2
+ ],
+ [
+ "3",
+ 3
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "ref-with-a-sibling-description",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "$ref": "#/$defs/Desk",
+ "description": "the field's"
+ }
+ },
+ "$defs": {
+ "Desk": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "the field's",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "ref-alias-chain",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "$ref": "#/$defs/Alias"
+ }
+ },
+ "$defs": {
+ "Alias": {
+ "$ref": "#/$defs/Desk"
+ },
+ "Desk": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ }
+ }
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "What is `dept`?",
+ "criteria": {
+ "billing": null,
+ "support": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "dept",
+ "kind": "choice",
+ "options": [
+ [
+ "billing",
+ "billing"
+ ],
+ [
+ "support",
+ "support"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "top-level-type-may-be-absent",
+ "schema": {
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "questions": {
+ "ok": {
+ "type": "noul",
+ "instructions": "Is `ok` true?"
+ }
+ },
+ "plan": [
+ {
+ "name": "ok",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "property-order-is-preserved",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "zebra": {
+ "type": "boolean"
+ },
+ "alpha": {
+ "enum": [
+ "z",
+ "a",
+ "m"
+ ]
+ },
+ "middle": {
+ "type": "integer",
+ "minimum": 0,
+ "maximum": 1
+ }
+ }
+ },
+ "questions": {
+ "zebra": {
+ "type": "noul",
+ "instructions": "Is `zebra` true?"
+ },
+ "alpha": {
+ "type": "choice",
+ "instructions": "What is `alpha`?",
+ "criteria": {
+ "z": null,
+ "a": null,
+ "m": null
+ }
+ },
+ "middle": {
+ "type": "score",
+ "instructions": "Score `middle` from 0 to 1",
+ "criteria": [
+ "0",
+ "1"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "zebra",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "alpha",
+ "kind": "choice",
+ "options": [
+ [
+ "z",
+ "z"
+ ],
+ [
+ "a",
+ "a"
+ ],
+ [
+ "m",
+ "m"
+ ]
+ ],
+ "minimum": null
+ },
+ {
+ "name": "middle",
+ "kind": "score",
+ "options": [],
+ "minimum": 0
+ }
+ ]
+ },
+ {
+ "case": "max-options-boundary",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "enum": [
+ "0",
+ "1",
+ "2",
+ "3",
+ "4",
+ "5",
+ "6",
+ "7",
+ "8",
+ "9",
+ "10",
+ "11",
+ "12",
+ "13",
+ "14",
+ "15",
+ "16",
+ "17",
+ "18",
+ "19",
+ "20",
+ "21",
+ "22",
+ "23",
+ "24",
+ "25",
+ "26",
+ "27",
+ "28",
+ "29",
+ "30",
+ "31"
+ ]
+ }
+ }
+ },
+ "questions": {
+ "c": {
+ "type": "choice",
+ "instructions": "What is `c`?",
+ "criteria": {
+ "0": null,
+ "1": null,
+ "2": null,
+ "3": null,
+ "4": null,
+ "5": null,
+ "6": null,
+ "7": null,
+ "8": null,
+ "9": null,
+ "10": null,
+ "11": null,
+ "12": null,
+ "13": null,
+ "14": null,
+ "15": null,
+ "16": null,
+ "17": null,
+ "18": null,
+ "19": null,
+ "20": null,
+ "21": null,
+ "22": null,
+ "23": null,
+ "24": null,
+ "25": null,
+ "26": null,
+ "27": null,
+ "28": null,
+ "29": null,
+ "30": null,
+ "31": null
+ }
+ }
+ },
+ "plan": [
+ {
+ "name": "c",
+ "kind": "choice",
+ "options": [
+ [
+ "0",
+ "0"
+ ],
+ [
+ "1",
+ "1"
+ ],
+ [
+ "2",
+ "2"
+ ],
+ [
+ "3",
+ "3"
+ ],
+ [
+ "4",
+ "4"
+ ],
+ [
+ "5",
+ "5"
+ ],
+ [
+ "6",
+ "6"
+ ],
+ [
+ "7",
+ "7"
+ ],
+ [
+ "8",
+ "8"
+ ],
+ [
+ "9",
+ "9"
+ ],
+ [
+ "10",
+ "10"
+ ],
+ [
+ "11",
+ "11"
+ ],
+ [
+ "12",
+ "12"
+ ],
+ [
+ "13",
+ "13"
+ ],
+ [
+ "14",
+ "14"
+ ],
+ [
+ "15",
+ "15"
+ ],
+ [
+ "16",
+ "16"
+ ],
+ [
+ "17",
+ "17"
+ ],
+ [
+ "18",
+ "18"
+ ],
+ [
+ "19",
+ "19"
+ ],
+ [
+ "20",
+ "20"
+ ],
+ [
+ "21",
+ "21"
+ ],
+ [
+ "22",
+ "22"
+ ],
+ [
+ "23",
+ "23"
+ ],
+ [
+ "24",
+ "24"
+ ],
+ [
+ "25",
+ "25"
+ ],
+ [
+ "26",
+ "26"
+ ],
+ [
+ "27",
+ "27"
+ ],
+ [
+ "28",
+ "28"
+ ],
+ [
+ "29",
+ "29"
+ ],
+ [
+ "30",
+ "30"
+ ],
+ [
+ "31",
+ "31"
+ ]
+ ],
+ "minimum": null
+ }
+ ]
+ },
+ {
+ "case": "max-score-levels-boundary",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 10
+ }
+ }
+ },
+ "questions": {
+ "n": {
+ "type": "score",
+ "instructions": "Score `n` from 1 to 10",
+ "criteria": [
+ "1",
+ "2",
+ "3",
+ "4",
+ "5",
+ "6",
+ "7",
+ "8",
+ "9",
+ "10"
+ ]
+ }
+ },
+ "plan": [
+ {
+ "name": "n",
+ "kind": "score",
+ "options": [],
+ "minimum": 1
+ }
+ ]
+ },
+ {
+ "case": "max-properties-boundary",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "p00": {
+ "type": "boolean"
+ },
+ "p01": {
+ "type": "boolean"
+ },
+ "p02": {
+ "type": "boolean"
+ },
+ "p03": {
+ "type": "boolean"
+ },
+ "p04": {
+ "type": "boolean"
+ },
+ "p05": {
+ "type": "boolean"
+ },
+ "p06": {
+ "type": "boolean"
+ },
+ "p07": {
+ "type": "boolean"
+ },
+ "p08": {
+ "type": "boolean"
+ },
+ "p09": {
+ "type": "boolean"
+ },
+ "p10": {
+ "type": "boolean"
+ },
+ "p11": {
+ "type": "boolean"
+ },
+ "p12": {
+ "type": "boolean"
+ },
+ "p13": {
+ "type": "boolean"
+ },
+ "p14": {
+ "type": "boolean"
+ },
+ "p15": {
+ "type": "boolean"
+ },
+ "p16": {
+ "type": "boolean"
+ },
+ "p17": {
+ "type": "boolean"
+ },
+ "p18": {
+ "type": "boolean"
+ },
+ "p19": {
+ "type": "boolean"
+ },
+ "p20": {
+ "type": "boolean"
+ },
+ "p21": {
+ "type": "boolean"
+ },
+ "p22": {
+ "type": "boolean"
+ },
+ "p23": {
+ "type": "boolean"
+ },
+ "p24": {
+ "type": "boolean"
+ },
+ "p25": {
+ "type": "boolean"
+ },
+ "p26": {
+ "type": "boolean"
+ },
+ "p27": {
+ "type": "boolean"
+ },
+ "p28": {
+ "type": "boolean"
+ },
+ "p29": {
+ "type": "boolean"
+ },
+ "p30": {
+ "type": "boolean"
+ },
+ "p31": {
+ "type": "boolean"
+ }
+ }
+ },
+ "questions": {
+ "p00": {
+ "type": "noul",
+ "instructions": "Is `p00` true?"
+ },
+ "p01": {
+ "type": "noul",
+ "instructions": "Is `p01` true?"
+ },
+ "p02": {
+ "type": "noul",
+ "instructions": "Is `p02` true?"
+ },
+ "p03": {
+ "type": "noul",
+ "instructions": "Is `p03` true?"
+ },
+ "p04": {
+ "type": "noul",
+ "instructions": "Is `p04` true?"
+ },
+ "p05": {
+ "type": "noul",
+ "instructions": "Is `p05` true?"
+ },
+ "p06": {
+ "type": "noul",
+ "instructions": "Is `p06` true?"
+ },
+ "p07": {
+ "type": "noul",
+ "instructions": "Is `p07` true?"
+ },
+ "p08": {
+ "type": "noul",
+ "instructions": "Is `p08` true?"
+ },
+ "p09": {
+ "type": "noul",
+ "instructions": "Is `p09` true?"
+ },
+ "p10": {
+ "type": "noul",
+ "instructions": "Is `p10` true?"
+ },
+ "p11": {
+ "type": "noul",
+ "instructions": "Is `p11` true?"
+ },
+ "p12": {
+ "type": "noul",
+ "instructions": "Is `p12` true?"
+ },
+ "p13": {
+ "type": "noul",
+ "instructions": "Is `p13` true?"
+ },
+ "p14": {
+ "type": "noul",
+ "instructions": "Is `p14` true?"
+ },
+ "p15": {
+ "type": "noul",
+ "instructions": "Is `p15` true?"
+ },
+ "p16": {
+ "type": "noul",
+ "instructions": "Is `p16` true?"
+ },
+ "p17": {
+ "type": "noul",
+ "instructions": "Is `p17` true?"
+ },
+ "p18": {
+ "type": "noul",
+ "instructions": "Is `p18` true?"
+ },
+ "p19": {
+ "type": "noul",
+ "instructions": "Is `p19` true?"
+ },
+ "p20": {
+ "type": "noul",
+ "instructions": "Is `p20` true?"
+ },
+ "p21": {
+ "type": "noul",
+ "instructions": "Is `p21` true?"
+ },
+ "p22": {
+ "type": "noul",
+ "instructions": "Is `p22` true?"
+ },
+ "p23": {
+ "type": "noul",
+ "instructions": "Is `p23` true?"
+ },
+ "p24": {
+ "type": "noul",
+ "instructions": "Is `p24` true?"
+ },
+ "p25": {
+ "type": "noul",
+ "instructions": "Is `p25` true?"
+ },
+ "p26": {
+ "type": "noul",
+ "instructions": "Is `p26` true?"
+ },
+ "p27": {
+ "type": "noul",
+ "instructions": "Is `p27` true?"
+ },
+ "p28": {
+ "type": "noul",
+ "instructions": "Is `p28` true?"
+ },
+ "p29": {
+ "type": "noul",
+ "instructions": "Is `p29` true?"
+ },
+ "p30": {
+ "type": "noul",
+ "instructions": "Is `p30` true?"
+ },
+ "p31": {
+ "type": "noul",
+ "instructions": "Is `p31` true?"
+ }
+ },
+ "plan": [
+ {
+ "name": "p00",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p01",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p02",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p03",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p04",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p05",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p06",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p07",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p08",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p09",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p10",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p11",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p12",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p13",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p14",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p15",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p16",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p17",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p18",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p19",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p20",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p21",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p22",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p23",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p24",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p25",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p26",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p27",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p28",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p29",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p30",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ },
+ {
+ "name": "p31",
+ "kind": "noul",
+ "options": [],
+ "minimum": null
+ }
+ ]
+ }
+ ],
+ "projections": [
+ {
+ "case": "choice-returns-the-value-not-the-label",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "code": {
+ "enum": [
+ 10,
+ 20,
+ 30
+ ]
+ }
+ }
+ },
+ "answers": {
+ "code": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "20",
+ "probabilities": {
+ "10": 0.1,
+ "20": 0.8,
+ "30": 0.1
+ }
+ }
+ },
+ "values": {
+ "code": 20
+ }
+ },
+ {
+ "case": "choice-value-and-label-coincide-for-a-string-enum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ }
+ }
+ },
+ "answers": {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "support",
+ "probabilities": {
+ "billing": 0.3,
+ "support": 0.7
+ }
+ }
+ },
+ "values": {
+ "dept": "support"
+ }
+ },
+ {
+ "case": "const-projects-to-the-const-value",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "kind": {
+ "const": 7
+ }
+ }
+ },
+ "answers": {
+ "kind": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "7",
+ "probabilities": {
+ "7": 1.0
+ }
+ }
+ },
+ "values": {
+ "kind": 7
+ }
+ },
+ {
+ "case": "choice-null-value",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "tier": {
+ "enum": [
+ "gold",
+ null
+ ]
+ }
+ }
+ },
+ "answers": {
+ "tier": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "null",
+ "probabilities": {
+ "gold": 0.2,
+ "null": 0.8
+ }
+ }
+ },
+ "values": {
+ "tier": null
+ }
+ },
+ {
+ "case": "choice-unknown-label-falls-back-to-the-label",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "code": {
+ "enum": [
+ 10,
+ 20
+ ]
+ }
+ }
+ },
+ "answers": {
+ "code": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "99",
+ "probabilities": {
+ "10": 0.5,
+ "20": 0.5
+ }
+ }
+ },
+ "values": {
+ "code": "99"
+ }
+ },
+ {
+ "case": "score-is-argmax-plus-minimum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "urgency": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 5
+ }
+ }
+ },
+ "answers": {
+ "urgency": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 0.9,
+ "probabilities": {
+ "0": 0.1,
+ "1": 0.1,
+ "2": 0.6,
+ "3": 0.1,
+ "4": 0.1
+ },
+ "legend": {}
+ }
+ },
+ "values": {
+ "urgency": 3
+ }
+ },
+ {
+ "case": "score-argmax-with-a-zero-minimum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 0,
+ "maximum": 3
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 0.0,
+ "probabilities": {
+ "0": 0.1,
+ "1": 0.2,
+ "2": 0.1,
+ "3": 0.6
+ },
+ "legend": {}
+ }
+ },
+ "values": {
+ "n": 3
+ }
+ },
+ {
+ "case": "score-argmax-with-a-negative-minimum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": -2,
+ "maximum": 2
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 4.0,
+ "probabilities": {
+ "0": 0.6,
+ "1": 0.1,
+ "2": 0.1,
+ "3": 0.1,
+ "4": 0.1
+ },
+ "legend": {}
+ }
+ },
+ "values": {
+ "n": -2
+ }
+ },
+ {
+ "case": "score-argmax-tie-takes-the-first-level",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 3
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 2.0,
+ "probabilities": {
+ "0": 0.4,
+ "1": 0.4,
+ "2": 0.2
+ },
+ "legend": {}
+ }
+ },
+ "values": {
+ "n": 1
+ }
+ },
+ {
+ "case": "score-without-probabilities-rounds",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 3,
+ "maximum": 7
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 2.4,
+ "probabilities": {},
+ "legend": {}
+ }
+ },
+ "values": {
+ "n": 5
+ }
+ },
+ {
+ "case": "score-without-probabilities-rounds-half-to-even",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": -2,
+ "maximum": 2
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 2.5,
+ "probabilities": {},
+ "legend": {}
+ }
+ },
+ "values": {
+ "n": 0
+ }
+ },
+ {
+ "case": "score-without-probabilities-rounds-half-to-even-odd-case",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": -2,
+ "maximum": 2
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 1.5,
+ "probabilities": {},
+ "legend": {}
+ }
+ },
+ "values": {
+ "n": 0
+ }
+ },
+ {
+ "case": "noul-exactly-half-is-true",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "answers": {
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.5
+ }
+ },
+ "values": {
+ "ok": true
+ }
+ },
+ {
+ "case": "noul-just-below-half-is-false",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "answers": {
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.49999999999999994
+ }
+ },
+ "values": {
+ "ok": false
+ }
+ },
+ {
+ "case": "noul-true",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "answers": {
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.93
+ }
+ },
+ "values": {
+ "ok": true
+ }
+ },
+ {
+ "case": "low-confidence-choice-is-null",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ }
+ }
+ },
+ "answers": {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "support",
+ "probabilities": {
+ "billing": 0.3,
+ "support": 0.7
+ },
+ "low_confidence": true
+ }
+ },
+ "values": {
+ "dept": null
+ }
+ },
+ {
+ "case": "low-confidence-score-is-null",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 3
+ }
+ }
+ },
+ "answers": {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 1.0,
+ "probabilities": {
+ "0": 0.2,
+ "1": 0.7,
+ "2": 0.1
+ },
+ "legend": {},
+ "low_confidence": true
+ }
+ },
+ "values": {
+ "n": null
+ }
+ },
+ {
+ "case": "low-confidence-noul-is-null",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "answers": {
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.93,
+ "low_confidence": true
+ }
+ },
+ "values": {
+ "ok": null
+ }
+ },
+ {
+ "case": "an-unanswered-field-is-omitted-not-null",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "answers": {
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.8
+ }
+ },
+ "values": {
+ "ok": true
+ }
+ },
+ {
+ "case": "every-kind-at-once",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ "urgency": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 5
+ },
+ "needs_human": {
+ "type": "boolean"
+ }
+ }
+ },
+ "answers": {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "billing",
+ "probabilities": {
+ "billing": 0.9,
+ "support": 0.1
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 3.0,
+ "probabilities": {
+ "0": 0.0,
+ "1": 0.1,
+ "2": 0.1,
+ "3": 0.7,
+ "4": 0.1
+ },
+ "legend": {}
+ },
+ "needs_human": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.2
+ }
+ },
+ "values": {
+ "dept": "billing",
+ "urgency": 4,
+ "needs_human": false
+ }
+ }
+ ],
+ "refusals": [
+ {
+ "case": "free-string",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "note": {
+ "type": "string"
+ }
+ }
+ },
+ "error": "properties.note: a free string cannot be a fixed option set; use 'enum' or a boolean"
+ },
+ {
+ "case": "array",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "tags": {
+ "type": "array",
+ "items": {
+ "type": "string"
+ }
+ }
+ }
+ },
+ "error": "properties.tags: arrays are not supported; ask one field per element"
+ },
+ {
+ "case": "nested-object",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "inner": {
+ "type": "object",
+ "properties": {
+ "a": {
+ "type": "boolean"
+ }
+ }
+ }
+ }
+ },
+ "error": "properties.inner: nested objects are not supported; flatten the schema"
+ },
+ {
+ "case": "union-of-two-real-types",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "x": {
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "integer"
+ }
+ ]
+ }
+ }
+ },
+ "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 2"
+ },
+ {
+ "case": "union-of-null-only",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "x": {
+ "anyOf": [
+ {
+ "type": "null"
+ }
+ ]
+ }
+ }
+ },
+ "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 0"
+ },
+ {
+ "case": "oneof-of-two-real-types",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "x": {
+ "oneOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "integer"
+ }
+ ]
+ }
+ }
+ },
+ "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 2"
+ },
+ {
+ "case": "type-list-with-two-non-null-types",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "x": {
+ "type": [
+ "string",
+ "integer"
+ ]
+ }
+ }
+ },
+ "error": "properties.x: 'type' has multiple non-null types; unions are not supported"
+ },
+ {
+ "case": "top-level-is-an-array",
+ "schema": {
+ "type": "array",
+ "items": {
+ "type": "string"
+ }
+ },
+ "error": "the top level must be an object with 'properties'"
+ },
+ {
+ "case": "top-level-has-no-properties",
+ "schema": {
+ "type": "object"
+ },
+ "error": "the top level must be an object with 'properties'"
+ },
+ {
+ "case": "top-level-type-is-wrong-but-properties-is-there",
+ "schema": {
+ "type": "array",
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "error": "the top level must be an object with 'properties'"
+ },
+ {
+ "case": "top-level-is-a-string-schema",
+ "schema": {
+ "type": "string"
+ },
+ "error": "the top level must be an object with 'properties'"
+ },
+ {
+ "case": "empty-properties",
+ "schema": {
+ "type": "object",
+ "properties": {}
+ },
+ "error": "'properties' must be a non-empty object"
+ },
+ {
+ "case": "properties-is-not-an-object",
+ "schema": {
+ "type": "object",
+ "properties": []
+ },
+ "error": "'properties' must be a non-empty object"
+ },
+ {
+ "case": "too-many-properties",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "p00": {
+ "type": "boolean"
+ },
+ "p01": {
+ "type": "boolean"
+ },
+ "p02": {
+ "type": "boolean"
+ },
+ "p03": {
+ "type": "boolean"
+ },
+ "p04": {
+ "type": "boolean"
+ },
+ "p05": {
+ "type": "boolean"
+ },
+ "p06": {
+ "type": "boolean"
+ },
+ "p07": {
+ "type": "boolean"
+ },
+ "p08": {
+ "type": "boolean"
+ },
+ "p09": {
+ "type": "boolean"
+ },
+ "p10": {
+ "type": "boolean"
+ },
+ "p11": {
+ "type": "boolean"
+ },
+ "p12": {
+ "type": "boolean"
+ },
+ "p13": {
+ "type": "boolean"
+ },
+ "p14": {
+ "type": "boolean"
+ },
+ "p15": {
+ "type": "boolean"
+ },
+ "p16": {
+ "type": "boolean"
+ },
+ "p17": {
+ "type": "boolean"
+ },
+ "p18": {
+ "type": "boolean"
+ },
+ "p19": {
+ "type": "boolean"
+ },
+ "p20": {
+ "type": "boolean"
+ },
+ "p21": {
+ "type": "boolean"
+ },
+ "p22": {
+ "type": "boolean"
+ },
+ "p23": {
+ "type": "boolean"
+ },
+ "p24": {
+ "type": "boolean"
+ },
+ "p25": {
+ "type": "boolean"
+ },
+ "p26": {
+ "type": "boolean"
+ },
+ "p27": {
+ "type": "boolean"
+ },
+ "p28": {
+ "type": "boolean"
+ },
+ "p29": {
+ "type": "boolean"
+ },
+ "p30": {
+ "type": "boolean"
+ },
+ "p31": {
+ "type": "boolean"
+ },
+ "p32": {
+ "type": "boolean"
+ }
+ }
+ },
+ "error": "33 properties exceeds MAX_PROPERTIES=32"
+ },
+ {
+ "case": "too-many-options",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "enum": [
+ "0",
+ "1",
+ "2",
+ "3",
+ "4",
+ "5",
+ "6",
+ "7",
+ "8",
+ "9",
+ "10",
+ "11",
+ "12",
+ "13",
+ "14",
+ "15",
+ "16",
+ "17",
+ "18",
+ "19",
+ "20",
+ "21",
+ "22",
+ "23",
+ "24",
+ "25",
+ "26",
+ "27",
+ "28",
+ "29",
+ "30",
+ "31",
+ "32"
+ ]
+ }
+ }
+ },
+ "error": "properties.c: 33 options exceeds MAX_OPTIONS=32"
+ },
+ {
+ "case": "empty-enum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "enum": []
+ }
+ }
+ },
+ "error": "properties.c: 'enum' must not be empty"
+ },
+ {
+ "case": "duplicate-choice-labels",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "enum": [
+ "1",
+ 1
+ ]
+ }
+ }
+ },
+ "error": "properties.c: enum values produce duplicate choice labels"
+ },
+ {
+ "case": "numeric-without-bounds",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer"
+ }
+ }
+ },
+ "error": "properties.n: a numeric field needs integer 'minimum' and 'maximum' to become a score"
+ },
+ {
+ "case": "numeric-with-float-bounds",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "number",
+ "minimum": 0.5,
+ "maximum": 1.5
+ }
+ }
+ },
+ "error": "properties.n: a numeric field needs integer 'minimum' and 'maximum' to become a score"
+ },
+ {
+ "case": "maximum-below-minimum",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 5,
+ "maximum": 1
+ }
+ }
+ },
+ "error": "properties.n: 'maximum' 1 is below 'minimum' 5"
+ },
+ {
+ "case": "too-many-score-levels",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 0,
+ "maximum": 10
+ }
+ }
+ },
+ "error": "properties.n: 11 levels exceeds MAX_SCORE_LEVELS=10; narrow the range or use an enum"
+ },
+ {
+ "case": "empty-property-schema",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {}
+ }
+ },
+ "error": "properties.c: unsupported schema {}"
+ },
+ {
+ "case": "property-is-not-an-object",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": "nope"
+ }
+ },
+ "error": "properties.c: property must be an object, got str"
+ },
+ {
+ "case": "unresolvable-ref",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "$ref": "#/$defs/Missing"
+ }
+ },
+ "$defs": {}
+ },
+ "error": "properties.c: $ref '#/$defs/Missing' does not resolve to an entry of this schema's '$defs' or 'definitions'"
+ },
+ {
+ "case": "remote-ref",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "$ref": "https://example.test/Desk"
+ }
+ }
+ },
+ "error": "properties.c: $ref 'https://example.test/Desk' does not resolve to an entry of this schema's '$defs' or 'definitions'"
+ },
+ {
+ "case": "recursive-ref",
+ "schema": {
+ "type": "object",
+ "properties": {
+ "c": {
+ "$ref": "#/$defs/Node"
+ }
+ },
+ "$defs": {
+ "Node": {
+ "$ref": "#/$defs/Node"
+ }
+ }
+ },
+ "error": "properties.c: $ref '#/$defs/Node' is recursive; flatten the schema"
+ }
+ ],
+ "arguments": [
+ {
+ "case": "decide-with-both",
+ "error": "pass exactly one of schema= or questions="
+ },
+ {
+ "case": "decide-with-neither",
+ "error": "pass exactly one of schema= or questions="
+ },
+ {
+ "case": "decide-batch-with-both",
+ "error": "pass exactly one of schema= or questions="
+ },
+ {
+ "case": "decide-batch-with-neither",
+ "error": "pass exactly one of schema= or questions="
+ },
+ {
+ "case": "decide-batch-over-a-string",
+ "error": "states must be a sequence of states, not str"
+ },
+ {
+ "case": "decide-batch-without-a-batching-runner",
+ "error": "object has no predict_batch; loop decide() over the states instead"
+ }
+ ],
+ "calls": [
+ {
+ "case": "decide-ungated",
+ "batch": false,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "min_confidence": null,
+ "predictions": [
+ {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.9,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "support",
+ "probabilities": {
+ "billing": 0.3,
+ "support": 0.7
+ }
+ },
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.55,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.9
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "dept": "support",
+ "ok": true
+ },
+ "confidence": {
+ "dept": 0.6,
+ "ok": 0.6
+ },
+ "answer_confidence": {
+ "dept": 0.9,
+ "ok": 0.55
+ },
+ "probabilities": {
+ "dept": {
+ "billing": 0.3,
+ "support": 0.7
+ },
+ "ok": {
+ "false": 0.1,
+ "true": 0.9
+ }
+ }
+ }
+ ],
+ "forwarded": []
+ },
+ {
+ "case": "decide-gated-nulls-only-the-field-below-the-bar",
+ "batch": false,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "min_confidence": 0.6,
+ "predictions": [
+ {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.9,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "support",
+ "probabilities": {
+ "billing": 0.3,
+ "support": 0.7
+ }
+ },
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.55,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.9
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "dept": "support",
+ "ok": null
+ },
+ "confidence": {
+ "dept": 0.6,
+ "ok": 0.6
+ },
+ "answer_confidence": {
+ "dept": 0.9,
+ "ok": 0.55
+ },
+ "probabilities": {
+ "dept": {
+ "billing": 0.3,
+ "support": 0.7
+ },
+ "ok": {
+ "false": 0.1,
+ "true": 0.9
+ }
+ }
+ }
+ ],
+ "forwarded": [
+ "min_confidence"
+ ]
+ },
+ {
+ "case": "decide-gated-nulls-everything",
+ "batch": false,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "min_confidence": 0.95,
+ "predictions": [
+ {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.9,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "support",
+ "probabilities": {
+ "billing": 0.3,
+ "support": 0.7
+ }
+ },
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.55,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.9
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "dept": null,
+ "ok": null
+ },
+ "confidence": {
+ "dept": 0.6,
+ "ok": 0.6
+ },
+ "answer_confidence": {
+ "dept": 0.9,
+ "ok": 0.55
+ },
+ "probabilities": {
+ "dept": {
+ "billing": 0.3,
+ "support": 0.7
+ },
+ "ok": {
+ "false": 0.1,
+ "true": 0.9
+ }
+ }
+ }
+ ],
+ "forwarded": [
+ "min_confidence"
+ ]
+ },
+ {
+ "case": "decide-gated-at-zero-nulls-nothing",
+ "batch": false,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ }
+ }
+ },
+ "min_confidence": 0.0,
+ "predictions": [
+ {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.1,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "billing",
+ "probabilities": {
+ "billing": 0.6,
+ "support": 0.4
+ }
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "dept": "billing"
+ },
+ "confidence": {
+ "dept": 0.6
+ },
+ "answer_confidence": {
+ "dept": 0.1
+ },
+ "probabilities": {
+ "dept": {
+ "billing": 0.6,
+ "support": 0.4
+ }
+ }
+ }
+ ],
+ "forwarded": [
+ "min_confidence"
+ ]
+ },
+ {
+ "case": "decide-rounds-the-nouls-two-sided-distribution",
+ "batch": false,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "min_confidence": null,
+ "predictions": [
+ {
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.88,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.123456
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "ok": false
+ },
+ "confidence": {
+ "ok": 0.6
+ },
+ "answer_confidence": {
+ "ok": 0.88
+ },
+ "probabilities": {
+ "ok": {
+ "false": 0.8765,
+ "true": 0.1235
+ }
+ }
+ }
+ ],
+ "forwarded": []
+ },
+ {
+ "case": "decide-batch-keeps-input-order",
+ "batch": true,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "n": {
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 3
+ }
+ }
+ },
+ "min_confidence": null,
+ "predictions": [
+ {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 0.0,
+ "probabilities": {
+ "0": 0.8,
+ "1": 0.1,
+ "2": 0.1
+ },
+ "legend": {}
+ }
+ },
+ {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 1.0,
+ "probabilities": {
+ "0": 0.1,
+ "1": 0.8,
+ "2": 0.1
+ },
+ "legend": {}
+ }
+ },
+ {
+ "n": {
+ "type": "score",
+ "confidence": 0.6,
+ "answer_confidence": 0.7,
+ "action": {
+ "act_probability": 0.2
+ },
+ "score": 2.0,
+ "probabilities": {
+ "0": 0.1,
+ "1": 0.1,
+ "2": 0.8
+ },
+ "legend": {}
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "n": 1
+ },
+ "confidence": {
+ "n": 0.6
+ },
+ "answer_confidence": {
+ "n": 0.7
+ },
+ "probabilities": {
+ "n": {
+ "0": 0.8,
+ "1": 0.1,
+ "2": 0.1
+ }
+ }
+ },
+ {
+ "values": {
+ "n": 2
+ },
+ "confidence": {
+ "n": 0.6
+ },
+ "answer_confidence": {
+ "n": 0.7
+ },
+ "probabilities": {
+ "n": {
+ "0": 0.1,
+ "1": 0.8,
+ "2": 0.1
+ }
+ }
+ },
+ {
+ "values": {
+ "n": 3
+ },
+ "confidence": {
+ "n": 0.6
+ },
+ "answer_confidence": {
+ "n": 0.7
+ },
+ "probabilities": {
+ "n": {
+ "0": 0.1,
+ "1": 0.1,
+ "2": 0.8
+ }
+ }
+ }
+ ],
+ "forwarded": []
+ },
+ {
+ "case": "decide-batch-gates-each-state-on-its-own",
+ "batch": true,
+ "schema": {
+ "type": "object",
+ "properties": {
+ "dept": {
+ "enum": [
+ "billing",
+ "support"
+ ]
+ },
+ "ok": {
+ "type": "boolean"
+ }
+ }
+ },
+ "min_confidence": 0.5,
+ "predictions": [
+ {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.9,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "support",
+ "probabilities": {
+ "billing": 0.3,
+ "support": 0.7
+ }
+ },
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.55,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.9
+ }
+ },
+ {
+ "dept": {
+ "type": "choice",
+ "confidence": 0.6,
+ "answer_confidence": 0.4,
+ "action": {
+ "act_probability": 0.2
+ },
+ "choice": "billing",
+ "probabilities": {
+ "billing": 0.6,
+ "support": 0.4
+ }
+ },
+ "ok": {
+ "type": "noul",
+ "confidence": 0.6,
+ "answer_confidence": 0.55,
+ "action": {
+ "act_probability": 0.2
+ },
+ "noul": 0.2
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "values": {
+ "dept": "support",
+ "ok": true
+ },
+ "confidence": {
+ "dept": 0.6,
+ "ok": 0.6
+ },
+ "answer_confidence": {
+ "dept": 0.9,
+ "ok": 0.55
+ },
+ "probabilities": {
+ "dept": {
+ "billing": 0.3,
+ "support": 0.7
+ },
+ "ok": {
+ "false": 0.1,
+ "true": 0.9
+ }
+ }
+ },
+ {
+ "values": {
+ "dept": null,
+ "ok": false
+ },
+ "confidence": {
+ "dept": 0.6,
+ "ok": 0.6
+ },
+ "answer_confidence": {
+ "dept": 0.4,
+ "ok": 0.55
+ },
+ "probabilities": {
+ "dept": {
+ "billing": 0.6,
+ "support": 0.4
+ },
+ "ok": {
+ "false": 0.8,
+ "true": 0.2
+ }
+ }
+ }
+ ],
+ "forwarded": []
+ }
+ ]
+}
diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java
index 13778d99..f2401e48 100644
--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java
+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java
@@ -41,7 +41,7 @@ import java.util.Map;
* <p>One session is not safe for concurrent {@code predict} calls unless ONNX Runtime is
* configured for it; hold one {@code Agent} per worker, or serialise access.
*/
-public final class Agent implements AutoCloseable, Predictor {
+public final class Agent implements AutoCloseable, Predictor, BatchPredictor {
private final Tokenizer tokenizer;
private final AgentConfig config;
@@ -140,6 +140,7 @@ public final class Agent implements AutoCloseable, Predictor {
* answer -- it only cuts padding -- but leaving it off keeps the batched path's grouping
* predictable, and a caller who wants the throughput can ask for it.
*/
+ @Override
public List<Prediction> predictBatch(List<?> states, Map<String, Question> questions) {
return predictBatch(states, questions, null, 0, false);
}
diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/BatchPredictor.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/BatchPredictor.java
new file mode 100644
index 00000000..2f4c00d2
--- /dev/null
+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/BatchPredictor.java
@@ -0,0 +1,28 @@
+package com.convaiinnovations.laya;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Anything that can answer one question set over many states in a shared forward pass.
+ *
+ * <p>Separate from {@link Predictor} rather than folded into it, because not every runtime
+ * batches. {@link Agent} does; {@link Router} does not — it selects a checkpoint per state, so
+ * there is no single graph call to share. The reference draws the same line at runtime, raising
+ * {@code TypeError} when a runner has no {@code predict_batch} rather than looping {@code decide}
+ * N times behind the caller's back; here the line is drawn by the type, so the same mistake does
+ * not compile.
+ *
+ * <p>{@link Agent#predictBatch(List, Map)} already had this signature, so this interface names an
+ * existing capability rather than adding one.
+ */
+public interface BatchPredictor {
+
+ /**
+ * Ask every question about every state.
+ *
+ * @return one prediction per state, <b>in the order the states were given</b>, which is the
+ * property a caller zipping the results back onto its own inputs depends on
+ */
+ List<Prediction> predictBatch(List<?> states, Map<String, Question> questions);
+}
diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Decisions.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Decisions.java
new file mode 100644
index 00000000..a8b7eac9
--- /dev/null
+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Decisions.java
@@ -0,0 +1,825 @@
+package com.convaiinnovations.laya;
+
+import com.convaiinnovations.laya.decode.Rounding;
+import com.convaiinnovations.laya.json.PythonJson;
+import java.math.BigInteger;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.OptionalDouble;
+import java.util.Set;
+
+/**
+ * Schema-driven decisions: a JSON schema becomes Laya questions, and the answers come back as
+ * the schema's own values.
+ *
+ * <p>The mapping is a documented subset, not a JSON-schema implementation: an object of
+ * properties, each one an enum choice, a boolean, or a bounded integer scale. Anything that
+ * cannot be answered from a fixed option set — a free string, an array, a nested object — is
+ * refused with a {@link SchemaException} that names the path, because a caller holding a
+ * 32-property schema cannot act on "this schema is not supported".
+ *
+ * <pre>{@code
+ * Map<String, Object> schema = (Map<String, Object>) Json.parse("""
+ * {"type": "object", "properties": {
+ * "department": {"enum": ["billing", "support", "sales"]},
+ * "urgency": {"type": "integer", "minimum": 1, "maximum": 5},
+ * "needs_human":{"type": "boolean"}}}""");
+ * Map<String, Object> values = Decisions.decide(agent, ticket, schema).values();
+ * }</pre>
+ *
+ * <p><b>Four rules a port gets wrong by default</b>, each pinned by
+ * {@code fixtures/structured.json}:
+ *
+ * <ul>
+ * <li>A {@code choice} projects to the schema <b>value</b>, not the label the model answered
+ * with. For {@code enum: ["billing", "support"]} the two coincide; for
+ * {@code enum: [10, 20, 30]} a port returning the label hands back the string
+ * {@code "20"} where the schema says the integer {@code 20}.</li>
+ * <li>A {@code score} is {@code minimum + argmax(probabilities)}. The answer's own
+ * {@code score} field is the probability-weighted <b>level index</b>, not a field value, so
+ * the no-probabilities fallback rounds it and adds {@code minimum} once. Subtracting
+ * {@code minimum} first and adding it back cancels out and silently drops it from every
+ * scale that does not start at zero. The rounding is Python's, half to <b>even</b>;
+ * {@code Math.round} is half up and disagrees at every {@code k + 0.5}.</li>
+ * <li>A {@code noul} is {@code >= 0.5}, so exactly {@code 0.5} is <b>true</b>.</li>
+ * <li>A low-confidence answer projects to <b>null</b>; an answer that is <b>absent</b> leaves
+ * the field out of the map entirely. "not believed" and "not asked" are different
+ * outcomes, and a caller that cannot tell them apart reports a decision nobody made.</li>
+ * </ul>
+ *
+ * <p><b>Where this differs from the reference, deliberately.</b>
+ *
+ * <ul>
+ * <li>Only the JSON-schema path is ported. There is no pydantic on the JVM, and the reference's
+ * {@code questions_from_pydantic} / {@code answer_to_pydantic} are a model-to-schema step in
+ * front of exactly this code. Mapping Java records would be a new design, not a port.</li>
+ * <li>{@code return_details} is gone: {@link #decide} always returns a {@link Decision}, whose
+ * {@link Decision#values()} is what the plain form returned. One return type instead of two
+ * selected by a boolean.</li>
+ * <li>The reference forwards {@code min_confidence} down to {@code runner.predict} and falls
+ * back to gating the result itself when the runner does not take the keyword. Here there is
+ * no such keyword on {@link Predictor}, so the second path is the only path — which is also
+ * the one the reference's own {@code decide_batch} always takes.</li>
+ * <li>{@code routing} is not reported. {@link Prediction} carries no routing block, so there is
+ * nothing to pass through; {@link Router} reports its decision at the point it makes it.</li>
+ * </ul>
+ */
+public final class Decisions {
+
+ /** The most properties one schema may have. Beyond this the states stop fitting a window. */
+ public static final int MAX_PROPERTIES = 32;
+
+ /** The most options one {@code enum} may have. */
+ public static final int MAX_OPTIONS = 32;
+
+ /** The most levels one numeric range may span before it should be an enum. */
+ public static final int MAX_SCORE_LEVELS = 10;
+
+ private Decisions() {
+ }
+
+ /**
+ * One option of a choice: the label the model is shown, and the value the schema means.
+ *
+ * <p>Two fields and not one, because they are not the same thing for anything but a string
+ * enum — {@code 20} is shown as {@code "20"} — and the projection has to go back the other
+ * way.
+ */
+ public record Option(String label, Object value) {
+ }
+
+ /**
+ * One planned property: the question to ask, and what is needed to read the answer back.
+ *
+ * @param kind which of the three question types this property became
+ * @param options the choice's options in schema order; empty for a score or a noul
+ * @param minimum the value of level 0 for a score, null otherwise. Without it a scale that
+ * does not start at zero projects to the wrong number and nothing about the answer says so
+ */
+ public record Field(String name, Question.Type kind, Question question, List<Option> options,
+ Long minimum) {
+
+ public Field {
+ options = List.copyOf(options);
+ }
+ }
+
+ /**
+ * A validated schema: one field per property, in the schema's own order.
+ *
+ * <p>Held as a value because both halves of a decision need it and they happen at different
+ * times — {@link #questions()} before the model runs, {@link #project} after. Re-deriving it
+ * from the schema for the projection is what lets the two disagree.
+ */
+ public record Plan(List<Field> fields) {
+
+ public Plan {
+ fields = List.copyOf(fields);
+ }
+
+ /** The questions to ask, keyed by property name, in the schema's property order. */
+ public Map<String, Question> questions() {
+ Map<String, Question> out = new LinkedHashMap<>();
+ for (Field field : fields) {
+ out.put(field.name(), field.question());
+ }
+ return Collections.unmodifiableMap(out);
+ }
+
+ /** The answers projected onto the schema's values, with no gate applied. */
+ public Map<String, Object> project(Map<String, Answer> answers) {
+ return project(answers, null);
+ }
+
+ /**
+ * The answers projected onto the schema's values.
+ *
+ * @param gate the {@link ConfidenceGate} report, or null when no gate ran. A field whose
+ * verdict is {@link ConfidenceGate.Gated#lowConfidence()} projects to <b>null</b>.
+ * The flag is read from the report rather than recomputed here, so the value a caller
+ * acts on and the state the gate reports cannot disagree
+ * @return the values, in the schema's property order. A field with no answer is absent
+ * rather than null; nulls are legitimate values here, which is why the map is an
+ * unmodifiable {@link LinkedHashMap} and not {@code Map.copyOf}
+ */
+ public Map<String, Object> project(Map<String, Answer> answers,
+ Map<String, ConfidenceGate.Gated> gate) {
+ Map<String, Object> values = new LinkedHashMap<>();
+ for (Field field : fields) {
+ Answer answer = answers.get(field.name());
+ if (answer == null) {
+ // Absent, not null. The reference skips the field entirely and the
+ // distinction is the point: a caller cannot tell "the model abstained" from
+ // "the question was never asked" if both arrive as null.
+ continue;
+ }
+ ConfidenceGate.Gated verdict = gate == null ? null : gate.get(field.name());
+ if (verdict != null && verdict.lowConfidence()) {
+ values.put(field.name(), null);
+ continue;
+ }
+ values.put(field.name(), value(field, answer));
+ }
+ return Collections.unmodifiableMap(values);
+ }
+
+ private static Object value(Field field, Answer answer) {
+ switch (field.kind()) {
+ case NOUL: {
+ // `>=`, so exactly 0.5 is true. Spelled `>` this flips on that one value and
+ // on no other, which is why the fixture carries 0.5 and the double just
+ // below it.
+ return expect(field, answer, Answer.Noul.class).noul() >= 0.5;
+ }
+ case SCORE: {
+ Answer.Score score = expect(field, answer, Answer.Score.class);
+ long level;
+ if (!score.probabilities().isEmpty()) {
+ level = argmax(score.probabilities());
+ } else {
+ // `score` is already the 0-based LEVEL INDEX, so it is rounded on its own
+ // with no `minimum` subtracted first. Math.rint is Python's round: half to
+ // even. Math.round is half up and gives 3 for 2.5.
+ level = (long) Math.rint(score.score());
+ }
+ return (field.minimum() == null ? 0L : field.minimum()) + level;
+ }
+ default: {
+ String label = expect(field, answer, Answer.Choice.class).choice();
+ for (Option option : field.options()) {
+ if (option.label().equals(label)) {
+ return option.value();
+ }
+ }
+ // A label no option carries comes back as the label. The model is constrained
+ // to the options, so this only reaches a hand-built answer -- and raising
+ // would turn a caller's own test double into a crash.
+ return label;
+ }
+ }
+ }
+
+ /**
+ * The index of the largest probability, keyed {@code "0"}, {@code "1"}, ...
+ *
+ * <p>Strictly greater, so a tie takes the <b>earliest</b> level, which is what Python's
+ * {@code max} over a range does. A missing index counts as 0.0 rather than being skipped,
+ * so the index space stays the level space.
+ */
+ private static long argmax(Map<String, Double> probabilities) {
+ long best = 0;
+ double highest = Double.NEGATIVE_INFINITY;
+ for (int i = 0; i < probabilities.size(); i++) {
+ Double probability = probabilities.get(Integer.toString(i));
+ double value = probability == null ? 0.0 : probability;
+ if (value > highest) {
+ highest = value;
+ best = i;
+ }
+ }
+ return best;
+ }
+
+ private static <T extends Answer> T expect(Field field, Answer answer, Class<T> wanted) {
+ if (!wanted.isInstance(answer)) {
+ // The reference reads a dict and would silently take the missing key's default,
+ // projecting `minimum + 0` or `false` for an answer of the wrong shape. A typed
+ // answer can say so instead, and this only reaches a caller who built the
+ // answers by hand.
+ throw new IllegalArgumentException(String.format(
+ "%s is a %s field, so its answer must be a %s and is a %s",
+ field.name(), field.kind().wireName(), wanted.getSimpleName(),
+ answer.getClass().getSimpleName()));
+ }
+ return wanted.cast(answer);
+ }
+ }
+
+ /**
+ * One decided state.
+ *
+ * <p>{@code values} is the schema-shaped output. The other maps are keyed by field, so a
+ * caller filtering on confidence does not have to re-derive which answer belonged to which
+ * property.
+ *
+ * @param answerConfidence {@code max(p)} per field — the probability mass on the answer being
+ * reported, which is the quantity {@code minConfidence} compares against and the one every
+ * calibration figure in laya is computed on. That is the reason to report it: a caller
+ * filtering this artifact to decide what to automate has to filter on the number the gate
+ * actually used. It is <b>not</b> a claim that the number is right — "about c of the
+ * answers returned at c are correct" holds only after temperatures have been fitted and
+ * validated on held-out data for that checkpoint and question shape, and
+ * {@code laya-multilingual} ships with no fitted temperatures at all. Empty for a field
+ * that reported no usable number, which is not the same as a reported 0.0
+ * @param confidence the normalised-entropy value, a <b>different</b> quantity on a scale that
+ * depends on the label count. Kept under its own name for that reason
+ * @param gate the {@link ConfidenceGate} report, or empty when no gate was configured. The
+ * presence of the report is how a caller tells "no gate ran" from "everything passed"
+ */
+ public record Decision(Map<String, Object> values, Map<String, Double> confidence,
+ Map<String, OptionalDouble> answerConfidence,
+ Map<String, Map<String, Double>> probabilities,
+ Map<String, Answer> answers, Usage usage,
+ Optional<Map<String, ConfidenceGate.Gated>> gate) {
+
+ public Decision {
+ // `values` legitimately holds nulls -- a gated field is null -- and `Map.copyOf`
+ // rejects them, so the unmodifiable wrapper is the only option that keeps both the
+ // nulls and the schema's property order.
+ values = Collections.unmodifiableMap(new LinkedHashMap<>(values));
+ confidence = Map.copyOf(confidence);
+ answerConfidence = Map.copyOf(answerConfidence);
+ probabilities = Map.copyOf(probabilities);
+ answers = Map.copyOf(answers);
+ }
+ }
+
+ // ---------------------------------------------------------------- schema -> questions
+
+ /** Validates a JSON schema and returns one planned field per property. */
+ public static Plan plan(Map<String, Object> schema) {
+ Object type = schema.get("type");
+ if (!(type == null || "object".equals(type)) || !schema.containsKey("properties")) {
+ throw new SchemaException("the top level must be an object with 'properties'");
+ }
+ Object raw = schema.get("properties");
+ if (!(raw instanceof Map) || ((Map<?, ?>) raw).isEmpty()) {
+ throw new SchemaException("'properties' must be a non-empty object");
+ }
+ @SuppressWarnings("unchecked")
+ Map<String, Object> properties = (Map<String, Object>) raw;
+ if (properties.size() > MAX_PROPERTIES) {
+ throw new SchemaException(properties.size() + " properties exceeds MAX_PROPERTIES="
+ + MAX_PROPERTIES);
+ }
+ List<Field> fields = new ArrayList<>(properties.size());
+ for (Map.Entry<String, Object> entry : properties.entrySet()) {
+ fields.add(field("properties." + entry.getKey(), entry.getKey(), entry.getValue(),
+ schema, List.of()));
+ }
+ return new Plan(fields);
+ }
+
+ /** Turns a JSON schema into Laya questions, in the schema's property order. */
+ public static Map<String, Question> questions(Map<String, Object> schema) {
+ return plan(schema).questions();
+ }
+
+ /** Projects answers onto a schema's values: the choice value, the integer level, the boolean. */
+ public static Map<String, Object> answersToJson(Map<String, Answer> answers,
+ Map<String, Object> schema) {
+ return plan(schema).project(answers);
+ }
+
+ @SuppressWarnings("unchecked")
+ private static Field field(String path, String name, Object raw, Map<String, Object> root,
+ List<String> seen) {
+ if (!(raw instanceof Map)) {
+ throw new SchemaException(path + ": property must be an object, got "
+ + pythonType(raw));
+ }
+ Map<String, Object> prop = (Map<String, Object>) raw;
+ while (prop.containsKey("$ref")) {
+ Object ref = prop.get("$ref");
+ String spelled = pythonRepr(ref);
+ if (seen.contains(spelled)) {
+ throw new SchemaException(path + ": $ref " + spelled
+ + " is recursive; flatten the schema");
+ }
+ Map<String, Object> target = definition(root, ref);
+ if (target == null) {
+ throw new SchemaException(path + ": $ref " + spelled + " does not resolve to an"
+ + " entry of this schema's '$defs' or 'definitions'");
+ }
+ // The definition's own `description` is DROPPED. pydantic fills it from the enum's
+ // docstring -- v1 writes "An enumeration." when there is none -- which describes the
+ // type rather than asking about this field, so the wording stays the field's and an
+ // `Enum` asks what the same `Literal` would.
+ Map<String, Object> merged = new LinkedHashMap<>();
+ for (Map.Entry<String, Object> entry : target.entrySet()) {
+ if (!"description".equals(entry.getKey())) {
+ merged.put(entry.getKey(), entry.getValue());
+ }
+ }
+ for (Map.Entry<String, Object> entry : prop.entrySet()) {
+ if (!"$ref".equals(entry.getKey())) {
+ merged.put(entry.getKey(), entry.getValue());
+ }
+ }
+ List<String> followed = new ArrayList<>(seen);
+ followed.add(spelled);
+ // A `$ref` the definition itself carries is an alias, followed by this same loop.
+ prop = merged;
+ seen = followed;
+ }
+
+ String description = describe(path, prop.get("description"));
+ if (!prop.containsKey("const") && !prop.containsKey("enum") && !prop.containsKey("type")) {
+ // pydantic v1 wraps a described `$ref` as {"allOf": [{"$ref": ...}], "description":
+ // ...}; a one-item `allOf` IS that schema, so it is unwrapped with the outer keys on
+ // top.
+ Object wrapped = prop.get("allOf");
+ if (wrapped instanceof List && ((List<?>) wrapped).size() == 1
+ && ((List<?>) wrapped).get(0) instanceof Map) {
+ Map<String, Object> branch =
+ new LinkedHashMap<>((Map<String, Object>) ((List<?>) wrapped).get(0));
+ for (Map.Entry<String, Object> entry : prop.entrySet()) {
+ if (!"allOf".equals(entry.getKey())) {
+ branch.put(entry.getKey(), entry.getValue());
+ }
+ }
+ return field(path, name, branch, root, seen);
+ }
+ // pydantic v2 renders Optional[X] as {"anyOf": [<X>, {"type": "null"}]} with no
+ // top-level type -- the same nullable shape the list form `type: ["string", "null"]`
+ // handles below. The single non-null branch is unwrapped so Optional[Literal[...]]
+ // and friends map; a union of two real types is genuinely ambiguous and refused.
+ //
+ // An EMPTY `anyOf` falls through to `oneOf`, because the reference spells this
+ // `prop.get("anyOf") or prop.get("oneOf")` and an empty list is falsy there.
+ Object union = prop.get("anyOf");
+ if (!(union instanceof List) || ((List<?>) union).isEmpty()) {
+ union = prop.get("oneOf");
+ }
+ if (union != null) {
+ List<Map<String, Object>> branches = new ArrayList<>();
+ if (union instanceof List) {
+ for (Object candidate : (List<?>) union) {
+ if (candidate instanceof Map
+ && !"null".equals(((Map<?, ?>) candidate).get("type"))) {
+ branches.add((Map<String, Object>) candidate);
+ }
+ }
+ }
+ if (branches.size() != 1) {
+ throw new SchemaException(path + ": only 'Optional[...]' unions (one non-null"
+ + " branch) are supported, got " + branches.size());
+ }
+ Map<String, Object> branch = new LinkedHashMap<>(branches.get(0));
+ // The outer description is carried onto the branch only when the branch has none.
+ // `containsKey`, not a null test: a branch that spells out `"description": null`
+ // has one, and the reference's `setdefault` leaves it alone.
+ if (!branch.containsKey("description")) {
+ branch.put("description", prop.get("description"));
+ }
+ return field(path, name, branch, root, seen);
+ }
+ }
+
+ // `containsKey`, because `{"const": null}` is a const of null and not an absent key. A
+ // port testing `get("const") != null` drops that field into "unsupported schema".
+ if (prop.containsKey("const")) {
+ List<Object> single = new ArrayList<>(1);
+ single.add(prop.get("const"));
+ return enumField(path, name, single, description);
+ }
+ if (prop.containsKey("enum")) {
+ Object values = prop.get("enum");
+ if (!(values instanceof List)) {
+ // Python would take `len()` of whatever this is and iterate it -- a string enum
+ // of "abc" becomes three one-character options. Refused here rather than
+ // reproduced: the reference's behaviour there is an artifact of duck typing, not
+ // a contract, and no JSON schema spells it.
+ throw new SchemaException(path + ": 'enum' must be an array, got "
+ + pythonType(values));
+ }
+ return enumField(path, name, (List<Object>) values, description);
+ }
+
+ Object jtype = prop.get("type");
+ if (jtype instanceof List) { // nullable: ["string", "null"]
+ List<Object> nonNull = new ArrayList<>();
+ for (Object candidate : (List<?>) jtype) {
+ if (!"null".equals(candidate)) {
+ nonNull.add(candidate);
+ }
+ }
+ if (nonNull.size() > 1) {
+ throw new SchemaException(path
+ + ": 'type' has multiple non-null types; unions are not supported");
+ }
+ jtype = nonNull.isEmpty() ? null : nonNull.get(0);
+ }
+ if ("boolean".equals(jtype)) {
+ return noulField(name, description);
+ }
+ if ("string".equals(jtype)) {
+ throw new SchemaException(path
+ + ": a free string cannot be a fixed option set; use 'enum' or a boolean");
+ }
+ if ("integer".equals(jtype) || "number".equals(jtype)) {
+ return scoreField(path, name, prop, description);
+ }
+ if ("array".equals(jtype)) {
+ throw new SchemaException(path
+ + ": arrays are not supported; ask one field per element");
+ }
+ if ("object".equals(jtype)) {
+ throw new SchemaException(path
+ + ": nested objects are not supported; flatten the schema");
+ }
+ throw new SchemaException(path + ": unsupported schema " + pythonRepr(prop));
+ }
+
+ private static Field enumField(String path, String name, List<Object> values,
+ String description) {
+ if (values.size() > MAX_OPTIONS) {
+ throw new SchemaException(path + ": " + values.size() + " options exceeds MAX_OPTIONS="
+ + MAX_OPTIONS);
+ }
+ if (values.isEmpty()) {
+ throw new SchemaException(path + ": 'enum' must not be empty");
+ }
+ boolean allBoolean = true;
+ for (Object value : values) {
+ allBoolean &= value instanceof Boolean;
+ }
+ if (allBoolean) {
+ // An enum of nothing but booleans is a noul, not a two-option choice: the labels
+ // would be "True"/"False" and the head would be asked to pick between them.
+ return noulField(name, description);
+ }
+ List<Option> options = new ArrayList<>(values.size());
+ Set<String> labels = new LinkedHashSet<>();
+ for (Object value : values) {
+ String label = value == null ? "null" : pythonStr(value);
+ options.add(new Option(label, value));
+ labels.add(label);
+ }
+ if (labels.size() != options.size()) {
+ throw new SchemaException(path + ": enum values produce duplicate choice labels");
+ }
+ Map<String, Object> criteria = new LinkedHashMap<>();
+ for (Option option : options) {
+ criteria.put(option.label(), null);
+ }
+ Question question = Question.choice(
+ description != null ? description : "What is `" + name + "`?", criteria);
+ return new Field(name, Question.Type.CHOICE, question, options, null);
+ }
+
+ private static Field noulField(String name, String description) {
+ Question question = Question.noul(
+ description != null ? description : "Is `" + name + "` true?");
+ return new Field(name, Question.Type.NOUL, question, List.of(), null);
+ }
+
+ private static Field scoreField(String path, String name, Map<String, Object> prop,
+ String description) {
+ Object lo = prop.get("minimum");
+ Object hi = prop.get("maximum");
+ if (!isInteger(lo) || !isInteger(hi)) {
+ throw new SchemaException(path + ": a numeric field needs integer 'minimum' and"
+ + " 'maximum' to become a score");
+ }
+ long low = ((Number) lo).longValue();
+ long high = ((Number) hi).longValue();
+ if (high < low) {
+ throw new SchemaException(String.format("%s: 'maximum' %d is below 'minimum' %d",
+ path, high, low));
+ }
+ long span = high - low + 1;
+ if (span > MAX_SCORE_LEVELS) {
+ throw new SchemaException(String.format(
+ "%s: %d levels exceeds MAX_SCORE_LEVELS=%d; narrow the range or use an enum",
+ path, span, MAX_SCORE_LEVELS));
+ }
+ List<String> levels = new ArrayList<>((int) span);
+ for (long level = low; level <= high; level++) {
+ levels.add(Long.toString(level));
+ }
+ Question question = Question.score(
+ description != null ? description
+ : String.format("Score `%s` from %d to %d", name, low, high),
+ levels);
+ return new Field(name, Question.Type.SCORE, question, List.of(), low);
+ }
+
+ /**
+ * Whether a bound counts as an integer.
+ *
+ * <p>A {@code Boolean} does not, where Python's {@code isinstance(x, int)} says it does
+ * because {@code bool} subclasses {@code int} there. The reference would accept
+ * {@code {"minimum": false, "maximum": true}} and ask "Score `x` from 0 to 1"; a JSON schema
+ * that spells a bound as a boolean is a mistake, and refusing it is the more useful answer.
+ */
+ private static boolean isInteger(Object value) {
+ return (value instanceof Long || value instanceof Integer || value instanceof BigInteger)
+ && !(value instanceof Boolean);
+ }
+
+ /**
+ * The instruction text a {@code description} supplies, or null to generate one.
+ *
+ * <p>Empty means "generate one", matching the reference's {@code description or ...}. A
+ * description that is not text is refused rather than stringified: the reference would put
+ * the value itself into the question's {@code instructions}, which is not text the model can
+ * be asked.
+ */
+ private static String describe(String path, Object raw) {
+ if (raw == null) {
+ return null;
+ }
+ if (raw instanceof String) {
+ return ((String) raw).isEmpty() ? null : (String) raw;
+ }
+ throw new SchemaException(path + ": 'description' must be a string, got "
+ + pythonType(raw));
+ }
+
+ /** The {@code $defs} or {@code definitions} entry a local {@code $ref} names, or null. */
+ @SuppressWarnings("unchecked")
+ private static Map<String, Object> definition(Map<String, Object> root, Object ref) {
+ if (!(ref instanceof String)) {
+ return null;
+ }
+ Map<String, Object> found = null;
+ // Only this schema's own definitions are looked up; anything else -- a remote ref, a
+ // pointer into another document -- is refused rather than fetched.
+ String[][] locals = {{"#/$defs/", "$defs"}, {"#/definitions/", "definitions"}};
+ for (String[] local : locals) {
+ Object defs = root.get(local[1]);
+ if (((String) ref).startsWith(local[0]) && defs instanceof Map) {
+ Object target = ((Map<String, Object>) defs)
+ .get(((String) ref).substring(local[0].length()));
+ if (target instanceof Map) {
+ found = (Map<String, Object>) target;
+ }
+ }
+ }
+ return found;
+ }
+
+ // ---------------------------------------------------------------- deciding
+
+ /** Decides one state against a schema, ungated. */
+ public static Decision decide(Predictor runner, Object state, Map<String, Object> schema) {
+ return decide(runner, state, schema, null, (Double) null);
+ }
+
+ /**
+ * Decides one state.
+ *
+ * <p>Pass exactly <b>one</b> of {@code schema} or {@code questions}. With a schema the values
+ * follow the schema; with explicit questions they are the raw answers, as the reference
+ * returns them.
+ *
+ * @param minConfidence the abstention threshold, or null for no gate. A field whose answer
+ * falls below it comes back as null, with the answer itself kept in the decision
+ * @throws IllegalArgumentException if both or neither of {@code schema} and {@code questions}
+ * is given
+ */
+ public static Decision decide(Predictor runner, Object state, Map<String, Object> schema,
+ Map<String, Question> questions, Double minConfidence) {
+ return decide(runner, state, schema, questions,
+ answers -> ConfidenceGate.apply(answers, minConfidence));
+ }
+
+ /** Decides one state, gating each answer at its own option-count bucket's threshold. */
+ public static Decision decide(Predictor runner, Object state, Map<String, Object> schema,
+ Map<String, Question> questions,
+ Map<String, ? extends Number> minConfidence) {
+ return decide(runner, state, schema, questions,
+ answers -> ConfidenceGate.apply(answers, minConfidence));
+ }
+
+ /** Decides many states against one schema in one batched call, ungated. */
+ public static List<Decision> decideBatch(BatchPredictor runner, List<?> states,
+ Map<String, Object> schema) {
+ return decideBatch(runner, states, schema, null, (Double) null);
+ }
+
+ /**
+ * Decides many states against one schema in one batched call, in input order.
+ *
+ * <p>The throughput form of {@link #decide}: the schema is planned once and its questions are
+ * evaluated over every state through {@link BatchPredictor#predictBatch} — the same shared
+ * forward pass {@link Agent#predictBatch} runs — then each state's answers are projected
+ * exactly as {@code decide} projects them.
+ *
+ * <p>A runner that cannot batch is a compile error here rather than the reference's
+ * {@code TypeError}, because looping {@code decide} N times behind the caller's back is the
+ * one thing neither wants to do silently.
+ */
+ public static List<Decision> decideBatch(BatchPredictor runner, List<?> states,
+ Map<String, Object> schema,
+ Map<String, Question> questions,
+ Double minConfidence) {
+ return decideBatch(runner, states, schema, questions,
+ answers -> ConfidenceGate.apply(answers, minConfidence));
+ }
+
+ /** Decides many states, gating each answer at its own option-count bucket's threshold. */
+ public static List<Decision> decideBatch(BatchPredictor runner, List<?> states,
+ Map<String, Object> schema,
+ Map<String, Question> questions,
+ Map<String, ? extends Number> minConfidence) {
+ return decideBatch(runner, states, schema, questions,
+ answers -> ConfidenceGate.apply(answers, minConfidence));
+ }
+
+ /** How a decision's answers are gated: one of {@link ConfidenceGate}'s two {@code apply}s. */
+ private interface Gating {
+ Optional<Map<String, ConfidenceGate.Gated>> of(Map<String, Answer> answers);
+ }
+
+ private static Decision decide(Predictor runner, Object state, Map<String, Object> schema,
+ Map<String, Question> questions, Gating gating) {
+ Plan plan = planFor(schema, questions);
+ Map<String, Question> asked = plan == null ? questions : plan.questions();
+ return decided(plan, runner.predict(state, asked), gating);
+ }
+
+ private static List<Decision> decideBatch(BatchPredictor runner, List<?> states,
+ Map<String, Object> schema,
+ Map<String, Question> questions, Gating gating) {
+ Plan plan = planFor(schema, questions);
+ Map<String, Question> asked = plan == null ? questions : plan.questions();
+ List<Prediction> predictions = runner.predictBatch(states, asked);
+ List<Decision> out = new ArrayList<>(predictions.size());
+ for (Prediction prediction : predictions) {
+ out.add(decided(plan, prediction, gating));
+ }
+ return List.copyOf(out);
+ }
+
+ private static Plan planFor(Map<String, Object> schema, Map<String, Question> questions) {
+ if ((schema == null) == (questions == null)) {
+ throw new IllegalArgumentException("pass exactly one of schema= or questions=");
+ }
+ return schema == null ? null : plan(schema);
+ }
+
+ private static Decision decided(Plan plan, Prediction prediction, Gating gating) {
+ Map<String, Answer> answers = prediction.answers();
+ Optional<Map<String, ConfidenceGate.Gated>> gate = gating.of(answers);
+ Map<String, Object> values;
+ if (plan == null) {
+ // The explicit-questions path returns the raw answers as the values, which is what
+ // the reference's `dict(answers)` does. There is no schema to project onto, so the
+ // gate's verdict is in `gate` and nothing is nulled.
+ values = new LinkedHashMap<>(answers);
+ } else {
+ values = plan.project(answers, gate.orElse(null));
+ }
+ Map<String, Double> confidence = new LinkedHashMap<>();
+ Map<String, OptionalDouble> answerConfidence = new LinkedHashMap<>();
+ Map<String, Map<String, Double>> probabilities = new LinkedHashMap<>();
+ for (Map.Entry<String, Answer> entry : answers.entrySet()) {
+ Answer answer = entry.getValue();
+ confidence.put(entry.getKey(), answer.confidence());
+ // Read through the gate's own accessor, so this and the threshold it was gated at
+ // cannot disagree about which quantity is being reported.
+ answerConfidence.put(entry.getKey(), ConfidenceGate.answerConfidence(answer));
+ if (answer instanceof Answer.Noul) {
+ // A noul has no probability map of its own; the reference reports the two-sided
+ // distribution it implies, rounded as every other number in an answer is.
+ double p = ((Answer.Noul) answer).noul();
+ Map<String, Double> sides = new LinkedHashMap<>();
+ sides.put("false", Rounding.round4(1.0 - p));
+ sides.put("true", Rounding.round4(p));
+ probabilities.put(entry.getKey(), Collections.unmodifiableMap(sides));
+ } else if (answer instanceof Answer.Choice) {
+ probabilities.put(entry.getKey(), ((Answer.Choice) answer).probabilities());
+ } else {
+ probabilities.put(entry.getKey(), ((Answer.Score) answer).probabilities());
+ }
+ }
+ return new Decision(values, confidence, answerConfidence, probabilities, answers,
+ prediction.usage(), gate);
+ }
+
+ // ---------------------------------------------------------------- Python spellings
+
+ /**
+ * {@code str(value)} for an option label.
+ *
+ * <p>The label is what the model is shown and what the answer comes back keyed by, so it has
+ * to be CPython's spelling and not Java's: {@code true} is {@code True} there, and a float is
+ * whatever {@link PythonJson} reproduces — {@code 1e+16}, not {@code 1.0E16}.
+ */
+ private static String pythonStr(Object value) {
+ if (value instanceof String) {
+ return (String) value;
+ }
+ return pythonRepr(value);
+ }
+
+ /**
+ * {@code repr(value)} for a parsed-JSON value, which is what the refusal messages interpolate.
+ *
+ * <p>The message is the contract — the fixtures assert it character for character — and
+ * {@code "unsupported schema %r"} puts a whole schema fragment into one.
+ */
+ private static String pythonRepr(Object value) {
+ if (value == null) {
+ return "None";
+ }
+ if (value instanceof Boolean) {
+ return ((Boolean) value) ? "True" : "False";
+ }
+ if (value instanceof String) {
+ return PythonJson.repr((String) value);
+ }
+ if (value instanceof Double || value instanceof Float) {
+ return PythonJson.repr(((Number) value).doubleValue());
+ }
+ if (value instanceof Number) {
+ return value.toString();
+ }
+ if (value instanceof List) {
+ StringBuilder out = new StringBuilder("[");
+ for (Object item : (List<?>) value) {
+ if (out.length() > 1) {
+ out.append(", ");
+ }
+ out.append(pythonRepr(item));
+ }
+ return out.append("]").toString();
+ }
+ if (value instanceof Map) {
+ StringBuilder out = new StringBuilder("{");
+ for (Map.Entry<?, ?> entry : ((Map<?, ?>) value).entrySet()) {
+ if (out.length() > 1) {
+ out.append(", ");
+ }
+ out.append(pythonRepr(entry.getKey())).append(": ")
+ .append(pythonRepr(entry.getValue()));
+ }
+ return out.append("}").toString();
+ }
+ throw new IllegalArgumentException(
+ "not a parsed-JSON value: " + value.getClass().getName());
+ }
+
+ /** {@code type(value).__name__}, which two of the refusal messages name. */
+ private static String pythonType(Object value) {
+ if (value == null) {
+ return "NoneType";
+ }
+ if (value instanceof String) {
+ return "str";
+ }
+ if (value instanceof Boolean) {
+ return "bool";
+ }
+ if (value instanceof Double || value instanceof Float) {
+ return "float";
+ }
+ if (value instanceof Number) {
+ return "int";
+ }
+ if (value instanceof List) {
+ return "list";
+ }
+ if (value instanceof Map) {
+ return "dict";
+ }
+ return value.getClass().getSimpleName();
+ }
+}
diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/SchemaException.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/SchemaException.java
new file mode 100644
index 00000000..de2a8f7a
--- /dev/null
+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/SchemaException.java
@@ -0,0 +1,22 @@
+package com.convaiinnovations.laya;
+
+/**
+ * A schema that cannot be expressed as Laya questions.
+ *
+ * <p>The message names the <b>path</b> — {@code properties.urgency: ...} — and that is the whole
+ * point of the type. A schema is refused field by field, and a caller holding a 32-property
+ * schema cannot act on "this schema is not supported".
+ *
+ * <p>An {@link IllegalArgumentException}, matching the reference's {@code SchemaError(ValueError)}
+ * and the rest of this module: a schema is an argument, and a caller catching
+ * {@code IllegalArgumentException} around a call that builds one should not have to know this
+ * subclass exists to catch it.
+ */
+public final class SchemaException extends IllegalArgumentException {
+
+ private static final long serialVersionUID = 1L;
+
+ public SchemaException(String message) {
+ super(message);
+ }
+}
diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/DecisionsTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/DecisionsTest.java
new file mode 100644
index 00000000..e2c47f7f
--- /dev/null
+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/DecisionsTest.java
@@ -0,0 +1,469 @@
+package com.convaiinnovations.laya;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.DynamicTest.dynamicTest;
+
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.OptionalDouble;
+import org.junit.jupiter.api.DisplayName;
+import org.junit.jupiter.api.DynamicTest;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestFactory;
+
+/**
+ * {@link Decisions} against {@code fixtures/structured.json}, recorded from
+ * {@code laya.structured}.
+ *
+ * <p>Four sections, because the module has four contracts and they fail independently: the
+ * schema-to-questions mapping, the projection back onto the schema's values, every refusal
+ * message, and the end-to-end wiring of {@code decide} / {@code decideBatch} to the confidence
+ * gate.
+ *
+ * <p>The refusals assert the <b>message</b>, not the exception type. The message names the path
+ * — {@code properties.urgency: ...} — and the path is the only part of it a caller can act on,
+ * so a port that refuses the right schemas with the wrong text has not ported the contract.
+ *
+ * <p>Two of the reference's argument refusals are not asserted, because they are compile errors
+ * here rather than runtime ones: {@code decide_batch} over a string, and {@code decide_batch}
+ * with a runner that cannot batch. {@link Decisions#decideBatch} takes a {@link List} and a
+ * {@link BatchPredictor}, which is a stronger guarantee than the reference's and is why those
+ * two are skipped rather than reproduced.
+ */
+final class DecisionsTest {
+
+ @SuppressWarnings("unchecked")
+ private static Map<String, Object> family() {
+ return Fixtures.load("structured.json");
+ }
+
+ @SuppressWarnings("unchecked")
+ private static List<Object> section(String name) {
+ return (List<Object>) family().get(name);
+ }
+
+ @SuppressWarnings("unchecked")
+ private static Map<String, Object> map(Object value) {
+ return (Map<String, Object>) value;
+ }
+
+ // ------------------------------------------------------------------ schema -> questions
+
+ @TestFactory
+ @DisplayName("every recorded schema maps to the reference's questions, in its own order")
+ List<DynamicTest> schemasMap() {
+ List<DynamicTest> tests = new ArrayList<>();
+ int choices = 0;
+ int scores = 0;
+ int nouls = 0;
+ for (Object raw : section("schemas")) {
+ Map<String, Object> one = map(raw);
+ for (Object planned : (List<?>) one.get("plan")) {
+ switch ((String) map(planned).get("kind")) {
+ case "choice": choices++; break;
+ case "score": scores++; break;
+ default: nouls++; break;
+ }
+ }
+ tests.add(dynamicTest((String) one.get("case"), () -> {
+ Map<String, Object> schema = map(one.get("schema"));
+ Map<String, Object> expected = map(one.get("questions"));
+ Decisions.Plan plan = Decisions.plan(schema);
+ Map<String, Question> questions = plan.questions();
+
+ // Property ORDER, not just the set: the questions are asked in this order and a
+ // state is re-encoded per question, so a reordering is a different request.
+ assertEquals(new ArrayList<>(expected.keySet()),
+ new ArrayList<>(questions.keySet()), "question order");
+ expected.forEach((name, value) -> {
+ Map<String, Object> want = map(value);
+ Map<String, Object> got = questions.get(name).spec();
+ assertEquals(want, got, name + " question");
+ // A choice's options are rendered in the criteria map's INSERTION order, so
+ // two orders are two different questions -- and Map.equals above cannot see
+ // the difference.
+ if (want.get("criteria") instanceof Map) {
+ assertEquals(new ArrayList<>(map(want.get("criteria")).keySet()),
+ new ArrayList<>(map(got.get("criteria")).keySet()),
+ name + " criteria order");
+ }
+ });
+
+ List<?> expectedPlan = (List<?>) one.get("plan");
+ assertEquals(expectedPlan.size(), plan.fields().size(), "one field per property");
+ for (int i = 0; i < expectedPlan.size(); i++) {
+ Map<String, Object> want = map(expectedPlan.get(i));
+ Decisions.Field got = plan.fields().get(i);
+ String id = one.get("case") + "/" + want.get("name");
+ assertEquals(want.get("name"), got.name(), id + " name");
+ assertEquals(want.get("kind"), got.kind().wireName(), id + " kind");
+ assertEquals(want.get("minimum"), got.minimum(), id + " minimum");
+ // The option VALUES never reach a question and are exactly what the
+ // projection needs, so they are asserted here rather than left to whichever
+ // projection case happens to cover them.
+ List<?> wantOptions = (List<?>) want.get("options");
+ assertEquals(wantOptions.size(), got.options().size(), id + " option count");
+ for (int o = 0; o < wantOptions.size(); o++) {
+ List<?> pair = (List<?>) wantOptions.get(o);
+ assertEquals(pair.get(0), got.options().get(o).label(), id + " label");
+ assertEquals(pair.get(1), got.options().get(o).value(), id + " value");
+ }
+ }
+ }));
+ }
+ final int c = choices;
+ final int s = scores;
+ final int n = nouls;
+ assertTrue(c >= 10 && s >= 8 && n >= 5,
+ () -> "the three question kinds are the whole mapping and this reaches "
+ + c + " choice, " + s + " score, " + n + " noul");
+ return tests;
+ }
+
+ // ------------------------------------------------------------------ projection
+
+ @TestFactory
+ @DisplayName("every recorded answer projects onto the schema's own values")
+ List<DynamicTest> projectionsMatch() {
+ List<DynamicTest> tests = new ArrayList<>();
+ int nulled = 0;
+ int omitted = 0;
+ for (Object raw : section("projections")) {
+ Map<String, Object> one = map(raw);
+ Map<String, Object> recorded = map(one.get("answers"));
+ Map<String, Object> expected = map(one.get("values"));
+ for (Map.Entry<String, Object> entry : expected.entrySet()) {
+ if (entry.getValue() == null
+ && Boolean.TRUE.equals(map(recorded.get(entry.getKey()))
+ .get("low_confidence"))) {
+ nulled++;
+ }
+ }
+ omitted += map(map(one.get("schema")).get("properties")).size() - expected.size();
+ tests.add(dynamicTest((String) one.get("case"), () -> {
+ Map<String, Answer> answers = answers(recorded);
+ Decisions.Plan plan = Decisions.plan(map(one.get("schema")));
+ // The reference reads `low_confidence` off the answer dict. On this side the
+ // flag lives in the gate's report -- a record cannot be written into -- so the
+ // recorded flags are handed over as a report. That IS the wiring under test:
+ // the projection must read the gate rather than re-deriving the rule.
+ Map<String, ConfidenceGate.Gated> gate = gateOf(recorded, answers);
+ Map<String, Object> values = plan.project(answers, gate);
+
+ assertEquals(expected, values, "projected values");
+ // A field with no answer is ABSENT, and a flagged one is present and null. Map
+ // equality above cannot tell those apart, so the key set is asserted too.
+ assertEquals(new ArrayList<>(expected.keySet()), new ArrayList<>(values.keySet()),
+ "which fields are present, and in which order");
+ if (gate == null) {
+ // No flags: the ungated spelling must agree, so `answersToJson` is not a
+ // second implementation of the same projection.
+ assertEquals(expected, Decisions.answersToJson(answers,
+ map(one.get("schema"))), "answersToJson agrees");
+ }
+ }));
+ }
+ final int flagged = nulled;
+ final int missing = omitted;
+ assertTrue(flagged >= 3, () -> "a low-confidence answer projects to null and only "
+ + flagged + " case(s) reach that");
+ assertTrue(missing >= 1, () -> "an unanswered field is omitted and no case reaches that");
+ return tests;
+ }
+
+ // ------------------------------------------------------------------ refusals
+
+ @TestFactory
+ @DisplayName("every schema the reference refuses is refused here, with the same message")
+ List<DynamicTest> refusalsMatch() {
+ List<DynamicTest> tests = new ArrayList<>();
+ for (Object raw : section("refusals")) {
+ Map<String, Object> one = map(raw);
+ tests.add(dynamicTest((String) one.get("case"), () -> {
+ SchemaException refused = assertThrows(SchemaException.class,
+ () -> Decisions.questions(map(one.get("schema"))),
+ one.get("case") + " must be refused");
+ assertEquals(one.get("error"), refused.getMessage(),
+ "the message names the path, which is the part a caller acts on");
+ }));
+ }
+ assertTrue(tests.size() >= 20,
+ () -> "only " + tests.size() + " refusal(s); this is the half of the mapping "
+ + "that tells a caller what to fix");
+ return tests;
+ }
+
+ @TestFactory
+ @DisplayName("decide takes exactly one of schema and questions")
+ List<DynamicTest> argumentRefusalsMatch() {
+ Map<String, Question> questions = Map.of("ok", Question.noul("Is it?"));
+ Map<String, Object> schema = Map.of("type", "object");
+ Predictor never = (state, asked) -> {
+ throw new AssertionError("the runner must not be reached");
+ };
+ BatchPredictor neverBatch = (states, asked) -> {
+ throw new AssertionError("the runner must not be reached");
+ };
+ Map<String, Runnable> calls = new LinkedHashMap<>();
+ calls.put("decide-with-both", () -> Decisions.decide(never, "s", schema, questions,
+ (Double) null));
+ calls.put("decide-with-neither", () -> Decisions.decide(never, "s", null, null,
+ (Double) null));
+ calls.put("decide-batch-with-both", () -> Decisions.decideBatch(neverBatch, List.of("s"),
+ schema, questions, (Double) null));
+ calls.put("decide-batch-with-neither", () -> Decisions.decideBatch(neverBatch,
+ List.of("s"), null, null, (Double) null));
+
+ List<DynamicTest> tests = new ArrayList<>();
+ int matched = 0;
+ for (Object raw : section("arguments")) {
+ Map<String, Object> one = map(raw);
+ Runnable call = calls.get((String) one.get("case"));
+ if (call == null) {
+ // `decide_batch` over a string and `decide_batch` without a batching runner.
+ // Both are compile errors here; see the class note.
+ continue;
+ }
+ matched++;
+ tests.add(dynamicTest((String) one.get("case"), () -> {
+ IllegalArgumentException refused =
+ assertThrows(IllegalArgumentException.class, call::run);
+ assertEquals(one.get("error"), refused.getMessage());
+ }));
+ }
+ final int reached = matched;
+ assertEquals(4, reached, "all four representable argument refusals must be reached");
+ return tests;
+ }
+
+ // ------------------------------------------------------------------ end to end
+
+ @TestFactory
+ @DisplayName("decide and decideBatch reproduce the recorded decisions")
+ List<DynamicTest> callsMatch() {
+ List<DynamicTest> tests = new ArrayList<>();
+ int gatedNulls = 0;
+ int batched = 0;
+ for (Object raw : section("calls")) {
+ Map<String, Object> one = map(raw);
+ boolean batch = Boolean.TRUE.equals(one.get("batch"));
+ if (batch) {
+ batched++;
+ }
+ for (Object decision : (List<?>) one.get("decisions")) {
+ for (Object value : map(map(decision).get("values")).values()) {
+ if (value == null) {
+ gatedNulls++;
+ }
+ }
+ }
+ tests.add(dynamicTest((String) one.get("case"), () -> {
+ Map<String, Object> schema = map(one.get("schema"));
+ Double threshold = one.get("min_confidence") == null ? null
+ : ((Number) one.get("min_confidence")).doubleValue();
+ List<Prediction> scripted = new ArrayList<>();
+ for (Object predicted : (List<?>) one.get("predictions")) {
+ scripted.add(new Prediction(Prediction.MODEL, answers(map(predicted)),
+ new Usage(12, 0, 9, 0, false, List.of(), Map.of())));
+ }
+ List<String> states = new ArrayList<>();
+ for (int i = 0; i < scripted.size(); i++) {
+ states.add("state " + i);
+ }
+
+ List<Decisions.Decision> got;
+ if (batch) {
+ BatchPredictor runner = (asked, questions) -> {
+ assertEquals(states, asked, "the states reach the runner in order");
+ return scripted;
+ };
+ got = Decisions.decideBatch(runner, states, schema, null, threshold);
+ } else {
+ Predictor runner = (state, questions) -> {
+ assertEquals(states.get(0), state);
+ return scripted.get(0);
+ };
+ got = List.of(Decisions.decide(runner, states.get(0), schema, null,
+ threshold));
+ }
+
+ List<?> expected = (List<?>) one.get("decisions");
+ assertEquals(expected.size(), got.size(), "one decision per state, in order");
+ for (int i = 0; i < expected.size(); i++) {
+ Map<String, Object> want = map(expected.get(i));
+ Decisions.Decision decision = got.get(i);
+ String id = one.get("case") + "[" + i + "]";
+
+ assertEquals(map(want.get("values")), decision.values(), id + " values");
+ assertEquals(new ArrayList<>(map(want.get("values")).keySet()),
+ new ArrayList<>(decision.values().keySet()), id + " value order");
+ map(want.get("confidence")).forEach((name, value) ->
+ assertEquals(((Number) value).doubleValue(),
+ decision.confidence().get(name), 1e-12, id + " confidence"));
+ map(want.get("answer_confidence")).forEach((name, value) ->
+ assertEquals(value == null ? OptionalDouble.empty()
+ : OptionalDouble.of(((Number) value).doubleValue()),
+ decision.answerConfidence().get(name),
+ id + " answer_confidence"));
+ map(want.get("probabilities")).forEach((name, value) -> {
+ Map<String, Object> sides = map(value);
+ Map<String, Double> reported = decision.probabilities().get(name);
+ assertEquals(sides.keySet(), reported.keySet(), id + " probability keys");
+ sides.forEach((label, p) -> assertEquals(((Number) p).doubleValue(),
+ reported.get(label), 1e-12, id + " p(" + label + ")"));
+ });
+
+ // The gate's own contract, carried through: no threshold means NO report,
+ // which is how a caller tells "no gate ran" from "everything passed".
+ if (threshold == null) {
+ assertTrue(decision.gate().isEmpty(), id + " ungated writes no report");
+ } else {
+ assertEquals(decision.answers().keySet(),
+ decision.gate().orElseThrow().keySet(),
+ id + " one verdict per answer");
+ }
+ }
+ }));
+ }
+ final int nulls = gatedNulls;
+ final int batches = batched;
+ assertTrue(nulls >= 3, () -> "the gate nulls a field and only " + nulls
+ + " recorded value(s) reach that");
+ assertTrue(batches >= 2, () -> "only " + batches + " batch case(s)");
+ return tests;
+ }
+
+ @Test
+ @DisplayName("the limits are the reference's")
+ void limitsMatch() {
+ Map<String, Object> limits = map(family().get("limits"));
+ assertEquals(((Number) limits.get("max_properties")).intValue(), Decisions.MAX_PROPERTIES);
+ assertEquals(((Number) limits.get("max_options")).intValue(), Decisions.MAX_OPTIONS);
+ assertEquals(((Number) limits.get("max_score_levels")).intValue(),
+ Decisions.MAX_SCORE_LEVELS);
+ }
+
+ @Test
+ @DisplayName("the explicit-questions path returns the raw answers")
+ void questionsPathReturnsRawAnswers() {
+ Map<String, Question> questions = new LinkedHashMap<>();
+ questions.put("ok", Question.noul("Is it?"));
+ Answer.Noul answer = new Answer.Noul(0.8, 0.8, 0.8, 0.2);
+ Predictor runner = (state, asked) -> {
+ assertEquals(questions, asked, "the questions reach the runner unchanged");
+ return new Prediction(Prediction.MODEL, Map.of("ok", answer),
+ new Usage(1, 0, 1, 0, false, List.of(), Map.of()));
+ };
+ Decisions.Decision decision = Decisions.decide(runner, "s", null, questions,
+ (Double) null);
+ // No schema, so nothing is projected: the reference returns `dict(answers)` and so does
+ // this. The values are the answers themselves, not a schema-shaped map.
+ assertEquals(Map.of("ok", answer), decision.values());
+ assertTrue(decision.gate().isEmpty());
+ }
+
+ @Test
+ @DisplayName("a per-bucket threshold gates each answer at its own bucket")
+ void perBucketThresholds() {
+ Map<String, Object> schema = Map.of("type", "object", "properties",
+ new LinkedHashMap<>(Map.of("ok", Map.of("type", "boolean"))));
+ Answer.Noul answer = new Answer.Noul(0.9, 0.6, 0.6, 0.2);
+ Predictor runner = (state, asked) -> new Prediction(Prediction.MODEL,
+ Map.of("ok", answer), new Usage(1, 0, 1, 0, false, List.of(), Map.of()));
+
+ Decisions.Decision abstained = Decisions.decide(runner, "s", schema, null,
+ Map.of("noul:2", 0.8));
+ assertTrue(abstained.values().containsKey("ok"), "the field is present...");
+ assertNull(abstained.values().get("ok"), "...and null, because its bucket abstained");
+
+ Decisions.Decision passed = Decisions.decide(runner, "s", schema, null,
+ Map.of("choice:2", 0.8));
+ // An unnamed bucket with no default gates at zero, so it never abstains by surprise.
+ assertEquals(Boolean.TRUE, passed.values().get("ok"));
+ }
+
+ @Test
+ @DisplayName("an answer of the wrong shape is refused rather than defaulted")
+ void answerShapeIsChecked() {
+ Map<String, Object> schema = Map.of("type", "object", "properties",
+ new LinkedHashMap<>(Map.of("n",
+ Map.of("type", "integer", "minimum", 1L, "maximum", 3L))));
+ Map<String, Answer> answers = Map.of("n", new Answer.Noul(0.9, 0.6, 0.6, 0.2));
+ IllegalArgumentException refused = assertThrows(IllegalArgumentException.class,
+ () -> Decisions.answersToJson(answers, schema));
+ assertTrue(refused.getMessage().contains("score"), refused.getMessage());
+ assertFalse(refused instanceof SchemaException, "the schema is fine; the answer is not");
+ }
+
+ // ------------------------------------------------------------------ fixture decoding
+
+ /** Every recorded answer as the typed record. */
+ private static Map<String, Answer> answers(Map<String, Object> recorded) {
+ Map<String, Answer> out = new LinkedHashMap<>();
+ recorded.forEach((name, value) -> out.put(name, answerOf(map(value))));
+ return out;
+ }
+
+ @SuppressWarnings("unchecked")
+ private static Answer answerOf(Map<String, Object> recorded) {
+ double confidence = number(recorded.get("confidence"));
+ double answerConfidence = number(recorded.get("answer_confidence"));
+ double act = number(map(recorded.get("action")).get("act_probability"));
+ Map<String, Double> probabilities = new LinkedHashMap<>();
+ if (recorded.get("probabilities") != null) {
+ map(recorded.get("probabilities"))
+ .forEach((k, v) -> probabilities.put(k, number(v)));
+ }
+ switch ((String) recorded.get("type")) {
+ case "noul":
+ return new Answer.Noul(number(recorded.get("noul")), confidence, answerConfidence,
+ act);
+ case "score": {
+ Map<String, String> legend = new LinkedHashMap<>();
+ map(recorded.get("legend")).forEach((k, v) -> legend.put(k, (String) v));
+ return new Answer.Score(number(recorded.get("score")), legend, probabilities,
+ confidence, answerConfidence, act);
+ }
+ default:
+ return new Answer.Choice((String) recorded.get("choice"), probabilities,
+ confidence, answerConfidence, act);
+ }
+ }
+
+ /** An absent number is NaN, never zero: "no usable confidence" is not "a confidence of 0". */
+ private static double number(Object value) {
+ return value == null ? Double.NaN : ((Number) value).doubleValue();
+ }
+
+ /**
+ * The gate report the recorded {@code low_confidence} flags imply, or null when none is set.
+ *
+ * <p>Null and not an empty map: an ungated call has no report at all, and handing the
+ * projection an empty one would make "no gate ran" indistinguishable from "a gate ran and
+ * flagged nothing".
+ */
+ private static Map<String, ConfidenceGate.Gated> gateOf(Map<String, Object> recorded,
+ Map<String, Answer> answers) {
+ boolean any = false;
+ for (Object value : recorded.values()) {
+ any |= Boolean.TRUE.equals(map(value).get("low_confidence"));
+ }
+ if (!any) {
+ return null;
+ }
+ Map<String, ConfidenceGate.Gated> gate = new LinkedHashMap<>();
+ recorded.forEach((name, value) -> {
+ boolean low = Boolean.TRUE.equals(map(value).get("low_confidence"));
+ gate.put(name, new ConfidenceGate.Gated(answers.get(name), low,
+ low ? ConfidenceGate.Abstention.ABSTAINED : ConfidenceGate.Abstention.PASSED,
+ 0.5));
+ });
+ return gate;
+ }
+}
diff --git a/laya-java/scripts/gen_fixtures.py b/laya-java/scripts/gen_fixtures.py
index 1b335cb9..9654e056 100644
--- a/laya-java/scripts/gen_fixtures.py
+++ b/laya-java/scripts/gen_fixtures.py
@@ -3069,6 +3069,605 @@ def confidence_gate():
" the three states and the no-op" % (gated, len(GATE_STATES)))
return {"states": list(GATE_STATES), "cases": cases, "refusals": refusals}
+#: Mirrored from `laya.structured` so the case lists below can be built at import time, which
+#: keeping this script's `laya` imports inside the builders requires. `structured()` asserts them
+#: against the reference, so a change there fails the generator rather than silently recording a
+#: boundary case that is no longer on the boundary.
+STRUCTURED_MAX_PROPERTIES = 32
+STRUCTURED_MAX_OPTIONS = 32
+STRUCTURED_MAX_SCORE_LEVELS = 10
+
+
+def _sd_answer(kind, **kw):
+ """One answer dict in the shape `predict` returns, with only the fields `_project` reads.
+
+ `action` and `legend` are carried because the Java side's typed `Answer` records require
+ them, not because the projection looks at either.
+ """
+ answer = {"type": kind,
+ "confidence": kw.get("confidence", 0.6),
+ "answer_confidence": kw.get("answer_confidence", 0.7),
+ "action": {"act_probability": kw.get("act", 0.2)}}
+ if kind == "choice":
+ answer["choice"] = kw["choice"]
+ answer["probabilities"] = kw.get("probabilities", {})
+ elif kind == "score":
+ answer["score"] = kw["score"]
+ answer["probabilities"] = kw.get("probabilities", {})
+ answer["legend"] = kw.get("legend", {})
+ else:
+ answer["noul"] = kw["noul"]
+ if kw.get("low_confidence"):
+ answer["low_confidence"] = True
+ return answer
+
+
+def _sd_enum(values, **extra):
+ prop = {"enum": list(values)}
+ prop.update(extra)
+ return prop
+
+
+#: `schema -> questions`. Each case is the schema and nothing else: what is recorded is what
+#: `questions_from_json_schema` makes of it, including the generated instruction wording and the
+#: criteria ORDER, which is positional in laya and therefore part of the question.
+STRUCTURED_SCHEMAS = [
+ ("enum-is-a-choice",
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support", "sales"])}}),
+ ("enum-of-numbers-keeps-its-values",
+ {"type": "object", "properties": {"code": _sd_enum([10, 20, 30])}}),
+ ("enum-with-a-null-branch",
+ {"type": "object", "properties": {"tier": _sd_enum(["gold", None])}}),
+ ("const-is-a-one-option-choice",
+ {"type": "object", "properties": {"kind": {"const": "refund"}}}),
+ # `{"const": None}` is a KEY THAT IS PRESENT WITH A NULL VALUE, not an absent key. A port
+ # testing `prop.get("const") is not None` drops this field into the "unsupported schema"
+ # branch instead of asking about it.
+ ("const-null-is-still-a-const",
+ {"type": "object", "properties": {"kind": {"const": None}}}),
+ ("boolean-is-a-noul",
+ {"type": "object", "properties": {"needs_human": {"type": "boolean"}}}),
+ # An enum whose values are ALL bools is a noul, not a two-option choice: the labels would be
+ # "True"/"False" and the head would be asked a choice question about a boolean.
+ ("enum-of-booleans-is-a-noul",
+ {"type": "object", "properties": {"flag": _sd_enum([True, False])}}),
+ ("integer-is-a-score",
+ {"type": "object", "properties": {"urgency": {"type": "integer",
+ "minimum": 1, "maximum": 5}}}),
+ ("number-with-integer-bounds-is-a-score",
+ {"type": "object", "properties": {"n": {"type": "number", "minimum": 0, "maximum": 2}}}),
+ ("score-from-zero",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 0, "maximum": 3}}}),
+ ("score-with-a-negative-minimum",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": -2, "maximum": 2}}}),
+ ("score-of-one-level",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 7, "maximum": 7}}}),
+ ("description-replaces-the-generated-instruction",
+ {"type": "object", "properties": {
+ "dept": _sd_enum(["billing", "support"], description="Which desk owns this?"),
+ "ok": {"type": "boolean", "description": "Is the customer satisfied?"},
+ "n": {"type": "integer", "minimum": 1, "maximum": 3,
+ "description": "How loud is the complaint?"}}}),
+ # pydantic v2 renders `Optional[X]` as a two-branch anyOf with no top-level type.
+ ("anyof-optional-is-unwrapped",
+ {"type": "object", "properties": {
+ "dept": {"anyOf": [_sd_enum(["billing", "support"]), {"type": "null"}],
+ "description": "pick a desk"}}}),
+ ("oneof-optional-is-unwrapped",
+ {"type": "object", "properties": {
+ "n": {"oneOf": [{"type": "integer", "minimum": 1, "maximum": 4}, {"type": "null"}]}}}),
+ # The outer description is carried onto the branch only when the branch has none.
+ ("anyof-branch-keeps-its-own-description",
+ {"type": "object", "properties": {
+ "dept": {"anyOf": [_sd_enum(["a", "b"], description="the branch's"), {"type": "null"}],
+ "description": "the outer one"}}}),
+ ("nullable-type-list-is-unwrapped",
+ {"type": "object", "properties": {"dept": {"type": ["string", "null"],
+ "enum": ["billing", "support"]}}}),
+ ("nullable-integer-type-list",
+ {"type": "object", "properties": {"n": {"type": ["integer", "null"],
+ "minimum": 0, "maximum": 3}}}),
+ # pydantic v1 wraps a described `$ref` in a one-item allOf; the outer keys win.
+ ("allof-of-one-is-unwrapped-outer-keys-on-top",
+ {"type": "object",
+ "properties": {"dept": {"allOf": [{"$ref": "#/$defs/Desk"}], "description": "the field's"}},
+ "$defs": {"Desk": _sd_enum(["billing", "support"], description="An enumeration.")}}),
+ # "Outer keys on top" is only observable when the one-item `allOf` member and the outer
+ # object SHARE a key. With a bare `$ref` member they never do, so a port that let the member
+ # win passed that case -- measured, by a mutant that swapped the two and survived.
+ ("allof-of-one-inline-outer-keys-win",
+ {"type": "object", "properties": {
+ "dept": {"allOf": [_sd_enum(["a", "b"], description="the branch's")],
+ "description": "the field's"}}}),
+ # The same rule for `$ref`: the property's own keys sit on top of the definition's, so a
+ # sibling `maximum` narrows the definition's range rather than being ignored.
+ ("ref-sibling-keys-beat-the-definition",
+ {"type": "object", "properties": {"n": {"$ref": "#/$defs/Scale", "maximum": 1}},
+ "$defs": {"Scale": {"type": "integer", "minimum": 0, "maximum": 3}}}),
+ # The definition's own description is DROPPED: pydantic fills it from the enum's docstring,
+ # which describes the type rather than asking about this field.
+ ("ref-into-defs-drops-the-definition-description",
+ {"type": "object", "properties": {"dept": {"$ref": "#/$defs/Desk"}},
+ "$defs": {"Desk": _sd_enum(["billing", "support"], description="An enumeration.")}}),
+ ("ref-into-definitions-draft-07",
+ {"type": "object", "properties": {"dept": {"$ref": "#/definitions/Desk"}},
+ "definitions": {"Desk": _sd_enum([1, 2, 3])}}),
+ ("ref-with-a-sibling-description",
+ {"type": "object",
+ "properties": {"dept": {"$ref": "#/$defs/Desk", "description": "the field's"}},
+ "$defs": {"Desk": _sd_enum(["billing", "support"])}}),
+ # A `$ref` the definition itself carries is an alias, followed by the caller's loop.
+ ("ref-alias-chain",
+ {"type": "object", "properties": {"dept": {"$ref": "#/$defs/Alias"}},
+ "$defs": {"Alias": {"$ref": "#/$defs/Desk"}, "Desk": _sd_enum(["billing", "support"])}}),
+ ("top-level-type-may-be-absent",
+ {"properties": {"ok": {"type": "boolean"}}}),
+ # Property order is the question order, and the criteria order is the option order.
+ ("property-order-is-preserved",
+ {"type": "object", "properties": {
+ "zebra": {"type": "boolean"}, "alpha": _sd_enum(["z", "a", "m"]),
+ "middle": {"type": "integer", "minimum": 0, "maximum": 1}}}),
+ ("max-options-boundary",
+ {"type": "object", "properties": {"c": _sd_enum([str(i) for i in range(STRUCTURED_MAX_OPTIONS)])}}),
+ ("max-score-levels-boundary",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 1,
+ "maximum": STRUCTURED_MAX_SCORE_LEVELS}}}),
+ ("max-properties-boundary",
+ {"type": "object",
+ "properties": {"p%02d" % i: {"type": "boolean"} for i in range(STRUCTURED_MAX_PROPERTIES)}}),
+]
+
+#: `answers -> schema values`. The schema is carried with each case because the projection is a
+#: function of BOTH: a choice maps back through its options and a score adds its `minimum`.
+STRUCTURED_PROJECTIONS = [
+ # The schema VALUE, not the label the model answered with. A port returning the label is
+ # indistinguishable here for a string enum and wrong for every other value type.
+ ("choice-returns-the-value-not-the-label",
+ {"type": "object", "properties": {"code": _sd_enum([10, 20, 30])}},
+ {"code": _sd_answer("choice", choice="20",
+ probabilities={"10": 0.1, "20": 0.8, "30": 0.1})}),
+ ("choice-value-and-label-coincide-for-a-string-enum",
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"])}},
+ {"dept": _sd_answer("choice", choice="support",
+ probabilities={"billing": 0.3, "support": 0.7})}),
+ ("const-projects-to-the-const-value",
+ {"type": "object", "properties": {"kind": {"const": 7}}},
+ {"kind": _sd_answer("choice", choice="7", probabilities={"7": 1.0})}),
+ ("choice-null-value",
+ {"type": "object", "properties": {"tier": _sd_enum(["gold", None])}},
+ {"tier": _sd_answer("choice", choice="null",
+ probabilities={"gold": 0.2, "null": 0.8})}),
+ # A label no option carries comes back as the label itself rather than raising -- the model
+ # is constrained to the options, so this is a contract for a hand-built answer.
+ ("choice-unknown-label-falls-back-to-the-label",
+ {"type": "object", "properties": {"code": _sd_enum([10, 20])}},
+ {"code": _sd_answer("choice", choice="99", probabilities={"10": 0.5, "20": 0.5})}),
+ # argmax over the probabilities, plus `minimum`. `score` itself is ignored when they exist,
+ # and it is set here to a level that disagrees so a port reading it is caught.
+ ("score-is-argmax-plus-minimum",
+ {"type": "object", "properties": {"urgency": {"type": "integer",
+ "minimum": 1, "maximum": 5}}},
+ {"urgency": _sd_answer("score", score=0.9,
+ probabilities={"0": 0.1, "1": 0.1, "2": 0.6, "3": 0.1, "4": 0.1})}),
+ ("score-argmax-with-a-zero-minimum",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 0, "maximum": 3}}},
+ {"n": _sd_answer("score", score=0.0,
+ probabilities={"0": 0.1, "1": 0.2, "2": 0.1, "3": 0.6})}),
+ ("score-argmax-with-a-negative-minimum",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": -2, "maximum": 2}}},
+ {"n": _sd_answer("score", score=4.0,
+ probabilities={"0": 0.6, "1": 0.1, "2": 0.1, "3": 0.1, "4": 0.1})}),
+ # A tie takes the FIRST level, because `max` over the index range keeps the earliest.
+ ("score-argmax-tie-takes-the-first-level",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 1, "maximum": 3}}},
+ {"n": _sd_answer("score", score=2.0, probabilities={"0": 0.4, "1": 0.4, "2": 0.2})}),
+ # With no probabilities the fallback rounds `score`, which is already a 0-based LEVEL INDEX
+ # -- so `minimum` is added once and never subtracted first.
+ ("score-without-probabilities-rounds",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 3, "maximum": 7}}},
+ {"n": _sd_answer("score", score=2.4, probabilities={})}),
+ # Python's `round` is half to EVEN. `Math.round` is half up and gives 3 here.
+ ("score-without-probabilities-rounds-half-to-even",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": -2, "maximum": 2}}},
+ {"n": _sd_answer("score", score=2.5, probabilities={})}),
+ ("score-without-probabilities-rounds-half-to-even-odd-case",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": -2, "maximum": 2}}},
+ {"n": _sd_answer("score", score=1.5, probabilities={})}),
+ # Exactly 0.5 is TRUE: the comparison is `>=`, so a port spelling it `>` flips this one case
+ # and no other.
+ ("noul-exactly-half-is-true",
+ {"type": "object", "properties": {"ok": {"type": "boolean"}}},
+ {"ok": _sd_answer("noul", noul=0.5)}),
+ ("noul-just-below-half-is-false",
+ {"type": "object", "properties": {"ok": {"type": "boolean"}}},
+ {"ok": _sd_answer("noul", noul=0.49999999999999994)}),
+ ("noul-true",
+ {"type": "object", "properties": {"ok": {"type": "boolean"}}},
+ {"ok": _sd_answer("noul", noul=0.93)}),
+ # A flagged answer is NULL, whatever it said. Null, not absent: the field was asked.
+ ("low-confidence-choice-is-null",
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"])}},
+ {"dept": _sd_answer("choice", choice="support", low_confidence=True,
+ probabilities={"billing": 0.3, "support": 0.7})}),
+ ("low-confidence-score-is-null",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 1, "maximum": 3}}},
+ {"n": _sd_answer("score", score=1.0, low_confidence=True,
+ probabilities={"0": 0.2, "1": 0.7, "2": 0.1})}),
+ ("low-confidence-noul-is-null",
+ {"type": "object", "properties": {"ok": {"type": "boolean"}}},
+ {"ok": _sd_answer("noul", noul=0.93, low_confidence=True)}),
+ # An ABSENT answer is absent from the values. Different from a flagged one, which is null:
+ # "the model was not asked" and "the model was not believed" are not the same outcome.
+ ("an-unanswered-field-is-omitted-not-null",
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"]),
+ "ok": {"type": "boolean"}}},
+ {"ok": _sd_answer("noul", noul=0.8)}),
+ ("every-kind-at-once",
+ {"type": "object", "properties": {
+ "dept": _sd_enum(["billing", "support"]),
+ "urgency": {"type": "integer", "minimum": 1, "maximum": 5},
+ "needs_human": {"type": "boolean"}}},
+ {"dept": _sd_answer("choice", choice="billing",
+ probabilities={"billing": 0.9, "support": 0.1}),
+ "urgency": _sd_answer("score", score=3.0,
+ probabilities={"0": 0.0, "1": 0.1, "2": 0.1, "3": 0.7, "4": 0.1}),
+ "needs_human": _sd_answer("noul", noul=0.2)}),
+]
+
+#: Every schema the mapping refuses, and the message a caller reads. The message is the contract
+#: here: it names the path, which is the only thing that makes a 32-field schema debuggable.
+STRUCTURED_REFUSALS = [
+ ("free-string", {"type": "object", "properties": {"note": {"type": "string"}}}),
+ ("array", {"type": "object",
+ "properties": {"tags": {"type": "array", "items": {"type": "string"}}}}),
+ ("nested-object", {"type": "object",
+ "properties": {"inner": {"type": "object",
+ "properties": {"a": {"type": "boolean"}}}}}),
+ ("union-of-two-real-types",
+ {"type": "object", "properties": {"x": {"anyOf": [{"type": "string"},
+ {"type": "integer"}]}}}),
+ ("union-of-null-only",
+ {"type": "object", "properties": {"x": {"anyOf": [{"type": "null"}]}}}),
+ ("oneof-of-two-real-types",
+ {"type": "object", "properties": {"x": {"oneOf": [{"type": "boolean"},
+ {"type": "integer"}]}}}),
+ ("type-list-with-two-non-null-types",
+ {"type": "object", "properties": {"x": {"type": ["string", "integer"]}}}),
+ ("top-level-is-an-array", {"type": "array", "items": {"type": "string"}}),
+ ("top-level-has-no-properties", {"type": "object"}),
+ # Both halves of the top-level test, separately. Every other case here is missing
+ # `properties` as well as having the wrong `type`, so a port that dropped the type check
+ # entirely still refused all of them -- measured, by a mutant that did exactly that.
+ ("top-level-type-is-wrong-but-properties-is-there",
+ {"type": "array", "properties": {"ok": {"type": "boolean"}}}),
+ ("top-level-is-a-string-schema", {"type": "string"}),
+ ("empty-properties", {"type": "object", "properties": {}}),
+ ("properties-is-not-an-object", {"type": "object", "properties": []}),
+ ("too-many-properties",
+ {"type": "object",
+ "properties": {"p%02d" % i: {"type": "boolean"} for i in range(STRUCTURED_MAX_PROPERTIES + 1)}}),
+ ("too-many-options",
+ {"type": "object",
+ "properties": {"c": _sd_enum([str(i) for i in range(STRUCTURED_MAX_OPTIONS + 1)])}}),
+ ("empty-enum", {"type": "object", "properties": {"c": _sd_enum([])}}),
+ ("duplicate-choice-labels", {"type": "object", "properties": {"c": _sd_enum(["1", 1])}}),
+ ("numeric-without-bounds", {"type": "object", "properties": {"n": {"type": "integer"}}}),
+ ("numeric-with-float-bounds",
+ {"type": "object", "properties": {"n": {"type": "number",
+ "minimum": 0.5, "maximum": 1.5}}}),
+ ("maximum-below-minimum",
+ {"type": "object", "properties": {"n": {"type": "integer",
+ "minimum": 5, "maximum": 1}}}),
+ ("too-many-score-levels",
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 0,
+ "maximum": STRUCTURED_MAX_SCORE_LEVELS}}}),
+ ("empty-property-schema", {"type": "object", "properties": {"c": {}}}),
+ ("property-is-not-an-object", {"type": "object", "properties": {"c": "nope"}}),
+ ("unresolvable-ref",
+ {"type": "object", "properties": {"c": {"$ref": "#/$defs/Missing"}}, "$defs": {}}),
+ ("remote-ref",
+ {"type": "object", "properties": {"c": {"$ref": "https://example.test/Desk"}}}),
+ ("recursive-ref",
+ {"type": "object", "properties": {"c": {"$ref": "#/$defs/Node"}},
+ "$defs": {"Node": {"$ref": "#/$defs/Node"}}}),
+]
+
+
+def _structured_runner(results):
+ """A runner that answers from a script, so `decide` is exercised without a checkpoint.
+
+ It accepts and records `**kwargs`, which is how the reference's `min_confidence` forwarding
+ stays visible: `decide` passes it down to `predict` and `decide_batch` does not.
+ """
+ import copy
+
+ class _Scripted:
+ def __init__(self):
+ self.predict_kwargs = []
+ self.batch_kwargs = []
+
+ def predict(self, state, questions, **kwargs):
+ self.predict_kwargs.append(sorted(kwargs))
+ return copy.deepcopy(results[0])
+
+ def predict_batch(self, states, questions, **kwargs):
+ self.batch_kwargs.append(sorted(kwargs))
+ return [copy.deepcopy(results[i]) for i in range(len(states))]
+
+ return _Scripted()
+
+
+def _structured_result(answers):
+ return {"model": "laya-rl-agent-onnx", "answers": answers,
+ "usage": {"input_tokens": 12, "output_tokens": 0, "state_tokens": 9,
+ "state_tokens_dropped": 0, "truncated": False,
+ "truncated_questions": []}}
+
+
+#: End-to-end `decide` / `decide_batch`: the answers a runner returns, and the decision that
+#: comes back. `min_confidence` is the wiring these pin -- the gate decides which fields are
+#: nulled, and the port must read the gate rather than re-implementing the rule.
+STRUCTURED_CALLS = [
+ ("decide-ungated", False,
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"]),
+ "ok": {"type": "boolean"}}}, None,
+ [{"dept": _sd_answer("choice", choice="support", answer_confidence=0.9,
+ probabilities={"billing": 0.3, "support": 0.7}),
+ "ok": _sd_answer("noul", noul=0.9, answer_confidence=0.55)}]),
+ # One field below the bar and one above it, in the same decision: the gate is per answer,
+ # so a port that nulls all or nothing passes neither half of this.
+ ("decide-gated-nulls-only-the-field-below-the-bar", False,
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"]),
+ "ok": {"type": "boolean"}}}, 0.6,
+ [{"dept": _sd_answer("choice", choice="support", answer_confidence=0.9,
+ probabilities={"billing": 0.3, "support": 0.7}),
+ "ok": _sd_answer("noul", noul=0.9, answer_confidence=0.55)}]),
+ ("decide-gated-nulls-everything", False,
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"]),
+ "ok": {"type": "boolean"}}}, 0.95,
+ [{"dept": _sd_answer("choice", choice="support", answer_confidence=0.9,
+ probabilities={"billing": 0.3, "support": 0.7}),
+ "ok": _sd_answer("noul", noul=0.9, answer_confidence=0.55)}]),
+ # A scalar 0.0 IS a gate: states are reported and nothing is nulled.
+ ("decide-gated-at-zero-nulls-nothing", False,
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"])}}, 0.0,
+ [{"dept": _sd_answer("choice", choice="billing", answer_confidence=0.1,
+ probabilities={"billing": 0.6, "support": 0.4})}]),
+ # A noul reports the two-sided distribution it implies, ROUNDED to four decimals as every
+ # other number in an answer is. 0.123456 is chosen so the rounding is visible: unrounded the
+ # sides are 0.876544 and 0.123456, which differ from the recorded 0.8765 and 0.1235 by 4.4e-5
+ # -- a gap a tolerance can see, where 0.9 would hide it at 1e-17.
+ ("decide-rounds-the-nouls-two-sided-distribution", False,
+ {"type": "object", "properties": {"ok": {"type": "boolean"}}}, None,
+ [{"ok": _sd_answer("noul", noul=0.123456, answer_confidence=0.88)}]),
+ ("decide-batch-keeps-input-order", True,
+ {"type": "object", "properties": {"n": {"type": "integer", "minimum": 1, "maximum": 3}}},
+ None,
+ [{"n": _sd_answer("score", score=0.0, probabilities={"0": 0.8, "1": 0.1, "2": 0.1})},
+ {"n": _sd_answer("score", score=1.0, probabilities={"0": 0.1, "1": 0.8, "2": 0.1})},
+ {"n": _sd_answer("score", score=2.0, probabilities={"0": 0.1, "1": 0.1, "2": 0.8})}]),
+ ("decide-batch-gates-each-state-on-its-own", True,
+ {"type": "object", "properties": {"dept": _sd_enum(["billing", "support"]),
+ "ok": {"type": "boolean"}}}, 0.5,
+ [{"dept": _sd_answer("choice", choice="support", answer_confidence=0.9,
+ probabilities={"billing": 0.3, "support": 0.7}),
+ "ok": _sd_answer("noul", noul=0.9, answer_confidence=0.55)},
+ {"dept": _sd_answer("choice", choice="billing", answer_confidence=0.4,
+ probabilities={"billing": 0.6, "support": 0.4}),
+ "ok": _sd_answer("noul", noul=0.2, answer_confidence=0.55)}]),
+]
+
+
+def structured():
+ """`laya.structured`: a JSON schema as questions, and the answers projected back onto it.
+
+ Four things a port gets wrong by default, and each has cases chosen so that getting it wrong
+ changes something here:
+
+ * a `choice` projects to the schema VALUE, not the label the model answered with. For a
+ string enum the two coincide, so `enum: [10, 20, 30]` is what tells them apart.
+ * a `score` is `minimum + argmax(probabilities)`. `score` itself is the probability-weighted
+ LEVEL INDEX, so the fallback for an answer with no probabilities rounds it and adds
+ `minimum` once -- subtracting `minimum` first and adding it back cancels out and silently
+ drops it. Python's `round` is half to EVEN, which `Math.round` is not.
+ * a `noul` is `>= 0.5`, so exactly 0.5 is TRUE and a port spelling it `>` differs on that
+ one value and no other.
+ * a low-confidence answer projects to NULL, and an UNANSWERED field is absent instead.
+ "not believed" and "not asked" are different outcomes and a port that conflates them
+ reports a decision nobody made.
+
+ The pydantic half of the module is deliberately NOT recorded: there is no pydantic on the
+ JVM, and `_schema_of` is the only thing between a model and the JSON-schema path this
+ records.
+ """
+ import copy
+
+ from laya import structured as reference
+ from laya.structured import (SchemaError, answers_to_json, decide, decide_batch,
+ plan_from_json_schema, questions_from_json_schema)
+
+ # The mirrored limits above are only safe if they are still the reference's.
+ limits = {"max_properties": reference.MAX_PROPERTIES,
+ "max_options": reference.MAX_OPTIONS,
+ "max_score_levels": reference.MAX_SCORE_LEVELS}
+ mirrored = {"max_properties": STRUCTURED_MAX_PROPERTIES,
+ "max_options": STRUCTURED_MAX_OPTIONS,
+ "max_score_levels": STRUCTURED_MAX_SCORE_LEVELS}
+ if limits != mirrored:
+ raise AssertionError(
+ "the mirrored limits %r are no longer laya.structured's %r, so every boundary case"
+ " below is recorded off the boundary" % (mirrored, limits))
+
+ schemas = []
+ for cid, schema in STRUCTURED_SCHEMAS:
+ plan = plan_from_json_schema(schema)
+ schemas.append({
+ "case": cid,
+ "schema": schema,
+ "questions": questions_from_json_schema(schema),
+ # The plan as well as the questions: the OPTION VALUES and the score `minimum` never
+ # reach a question, and they are exactly what the projection needs. A fixture that
+ # recorded only the questions would leave both unpinned until a projection case
+ # happened to cover them.
+ "plan": [{"name": f.name, "kind": f.kind,
+ "options": [[label, value] for label, value in f.options],
+ "minimum": f.minimum} for f in plan],
+ })
+
+ projections = []
+ for cid, schema, answers in STRUCTURED_PROJECTIONS:
+ projections.append({"case": cid, "schema": schema, "answers": answers,
+ "values": answers_to_json(answers, schema)})
+
+ refusals = []
+ for cid, schema in STRUCTURED_REFUSALS:
+ try:
+ questions_from_json_schema(schema)
+ except SchemaError as problem:
+ refusals.append({"case": cid, "schema": schema, "error": str(problem)})
+ else:
+ raise AssertionError("%s was supposed to be refused and was not" % cid)
+
+ # `decide` itself: exactly one of schema/questions, and a batch over a real sequence.
+ arguments = []
+ for cid, call in (
+ ("decide-with-both", lambda: decide(None, "s", schema={"type": "object"},
+ questions={})),
+ ("decide-with-neither", lambda: decide(None, "s")),
+ ("decide-batch-with-both",
+ lambda: decide_batch(None, ["s"], schema={"type": "object"}, questions={})),
+ ("decide-batch-with-neither", lambda: decide_batch(None, ["s"])),
+ ("decide-batch-over-a-string",
+ lambda: decide_batch(None, "not a list", schema={"type": "object"})),
+ ("decide-batch-without-a-batching-runner",
+ lambda: decide_batch(object(), ["s"],
+ schema={"type": "object",
+ "properties": {"ok": {"type": "boolean"}}})),
+ ):
+ try:
+ call()
+ except (ValueError, TypeError) as problem:
+ arguments.append({"case": cid, "error": str(problem)})
+ else:
+ raise AssertionError("%s was supposed to be refused and was not" % cid)
+
+ calls = []
+ for cid, batch, schema, threshold, scripted in STRUCTURED_CALLS:
+ runner = _structured_runner([_structured_result(a) for a in scripted])
+ states = ["state %d" % i for i in range(len(scripted))]
+ if batch:
+ decisions = decide_batch(runner, states, schema=schema, return_details=True,
+ min_confidence=threshold)
+ else:
+ decisions = [decide(runner, states[0], schema=schema, return_details=True,
+ min_confidence=threshold)]
+ calls.append({
+ "case": cid,
+ "batch": batch,
+ "schema": schema,
+ "min_confidence": threshold,
+ # The answers the runner HANDED BACK, before the gate wrote anything into them --
+ # which is what a port has to feed its own runner stub.
+ "predictions": copy.deepcopy(scripted),
+ "decisions": [{"values": d.values,
+ "confidence": d.confidence,
+ "answer_confidence": d.answer_confidence,
+ "probabilities": d.probabilities} for d in decisions],
+ # `decide` forwards `min_confidence` to `predict`; `decide_batch` applies the gate
+ # itself and forwards nothing. Recorded because a port whose runner has no such
+ # keyword must take the second path for both, and this is what says so.
+ "forwarded": sorted(set(sum(runner.predict_kwargs + runner.batch_kwargs, []))),
+ })
+
+ # Coverage. Each of these is a branch of the mapping, and a case list that stops reaching one
+ # leaves that branch asserted by nothing -- which is how a fixture family quietly stops being
+ # a gate.
+ kinds = {f["kind"] for case in schemas for f in case["plan"]}
+ missing_kinds = {"choice", "score", "noul"} - kinds
+ if missing_kinds:
+ raise AssertionError("no schema case produces a %s question"
+ % ", ".join(sorted(missing_kinds)))
+ required_messages = [
+ "a free string cannot be a fixed option set",
+ "arrays are not supported",
+ "nested objects are not supported",
+ "only 'Optional[...]' unions (one non-null branch) are supported",
+ "'type' has multiple non-null types",
+ "the top level must be an object with 'properties'",
+ "'properties' must be a non-empty object",
+ "exceeds MAX_PROPERTIES=%d" % STRUCTURED_MAX_PROPERTIES,
+ "exceeds MAX_OPTIONS=%d" % STRUCTURED_MAX_OPTIONS,
+ "exceeds MAX_SCORE_LEVELS=%d" % STRUCTURED_MAX_SCORE_LEVELS,
+ "'enum' must not be empty",
+ "duplicate choice labels",
+ "needs integer 'minimum' and 'maximum'",
+ "is below 'minimum'",
+ "unsupported schema",
+ "property must be an object",
+ "is recursive",
+ "does not resolve to an entry",
+ ]
+ recorded = [r["error"] for r in refusals]
+ unreached = [m for m in required_messages if not any(m in e for e in recorded)]
+ if unreached:
+ raise AssertionError("no refusal case reaches: %s" % "; ".join(unreached))
+
+ projected = [v for case in projections for v in case["values"].values()]
+ omitted = sum(1 for case in projections
+ if set(case["schema"]["properties"]) - set(case["values"]))
+ # A null from a FLAG and a null that is the schema's own value are different branches.
+ nulled = sum(1 for case in projections for name, value in case["values"].items()
+ if value is None and case["answers"][name].get("low_confidence"))
+ null_valued = sum(1 for case in projections for name, value in case["values"].items()
+ if value is None and not case["answers"][name].get("low_confidence"))
+ booleans = sum(1 for v in projected if isinstance(v, bool))
+ integers = sum(1 for v in projected if isinstance(v, int) and not isinstance(v, bool))
+ relabelled = sum(1 for case in projections
+ for name, value in case["values"].items()
+ if isinstance(value, int) and not isinstance(value, bool)
+ and case["answers"][name]["type"] == "choice")
+ no_probability_scores = sum(1 for case in projections
+ for answer in case["answers"].values()
+ if answer["type"] == "score" and not answer["probabilities"])
+ shifted = sum(1 for case in projections
+ for name in case["values"]
+ if case["answers"][name]["type"] == "score"
+ and case["schema"]["properties"][name].get("minimum") not in (0, None))
+ half = sum(1 for case in projections
+ for answer in case["answers"].values()
+ if answer.get("noul") == 0.5)
+ shortfalls = [name for name, count, floor in (
+ ("a null from a low-confidence answer", nulled, 3),
+ ("an omitted field", omitted, 1),
+ ("a schema whose own value is null", null_valued, 1),
+ ("a boolean", booleans, 4),
+ ("an integer level", integers, 7),
+ ("a choice whose VALUE is not its label", relabelled, 2),
+ ("a score with no probabilities", no_probability_scores, 3),
+ ("a score whose minimum is not zero", shifted, 4),
+ ("a noul of exactly 0.5", half, 1),
+ ) if count < floor]
+ if shortfalls:
+ raise AssertionError("the projection cases no longer reach: %s" % "; ".join(shortfalls))
+
+ gated_nulls = sum(1 for case in calls for d in case["decisions"]
+ for v in d["values"].values() if v is None)
+ batched = sum(1 for case in calls if case["batch"])
+ if gated_nulls < 3 or batched < 2 or len(arguments) < 4:
+ raise AssertionError(
+ "the end-to-end cases no longer pin the wiring: %d gated null(s), %d batch case(s),"
+ " %d argument refusal(s)" % (gated_nulls, batched, len(arguments)))
+
+ return {
+ "limits": limits,
+ "schemas": schemas,
+ "projections": projections,
+ "refusals": refusals,
+ "arguments": arguments,
+ "calls": calls,
+ }
+
+
def _decoder_shape(tokenizer_json):
"""The decoder section's type chain, flattened, so the fixture says what it was recorded on."""
import io
@@ -3090,6 +3689,7 @@ def _decoder_shape(tokenizer_json):
FAMILIES = {
+ "structured.json": structured,
"confidence_gate.json": confidence_gate,
"predict_long.json": predict_long_scan,
"window_plan.json": window_plan,