Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(rust): port predict_batch and router batch APIs

kevin-gatimu841ca1b1d2bf
diff --git a/laya-rust/CHANGELOG.md b/laya-rust/CHANGELOG.md
index fe9ddc57..2495da09 100644
--- a/laya-rust/CHANGELOG.md
+++ b/laya-rust/CHANGELOG.md
@@ -30,6 +30,26 @@ regenerated from laya (Python) 0.4.14 for the supported paths (see the known gap
`LayaError::{UnsupportedAggregate, InvalidWindow, NotSupported}` and a default
`LayaTokenizer::decode`. `LayaError` is not `#[non_exhaustive]`, so an exhaustive `match` on it
outside the crate needs arms for the three new variants.
+- `LayaEngine::predict_batch` (and `predict_batch_serialize`) and `LayaRouter::{predict_batch,
+ route_batch}`: Python's `ONNXAgent.predict_batch` and `Router.predict_batch` / `route_batch`. One
+ result per state in input order, `batch_size` states per forward pass (`None`, `0` and, in
+ Python, negative values send them all in one), and `sort_by_length` windows of eight batches
+ that apply only with a `batch_size` above one and below the state count. The router groups
+ requests by checkpoint and then by question schema, restores request order, attaches each
+ request's `routing`, and honours each request's `model`, `task`, `lang` and `lang_guess`;
+ `route_batch` loads nothing. New `BatchOptions`, `BatchRequest` (both `#[non_exhaustive]`, with
+ chained setters, so hooks, `min_confidence`, `lang` and per-request token budgets can be added
+ later), `LayaBatchPredictor` with the provided method `LayaPredictor::as_batch_predictor` (a
+ default of `None`, so existing implementors keep compiling) and `LayaError::InvalidState`, raised
+ for a `null` state as Python raises `TypeError`. Hooks, `min_confidence`, per-call `lang` and
+ `max_len` / `head_max_len` and the router's `diagnostics` report are not part of it. `LayaError`
+ is not `#[non_exhaustive]`, so an exhaustive `match` on it outside the crate needs an arm for the
+ new variant.
+- Behaviour change: `LayaEngine::predict` and `predict_serialize` now act as Python's `system_one`,
+ a one-state `predict_batch`. A `null` state returns `LayaError::InvalidState` (Python's
+ `TypeError`) instead of being scored as the text `"null"`, and an empty `QuestionSet` returns an
+ empty result with zero usage instead of `LayaError::InvalidQuestion`. `predict_long` refuses a
+ `null` state the same way, before it checks the aggregate mode, as Python does.
- `predict` now runs through a shared multi-state core that scores several states in one forward
pass; its results are unchanged.
- Advisory parity CI on every push and PR: regenerate fixtures from the current Python checkout,
@@ -65,9 +85,7 @@ regenerated from laya (Python) 0.4.14 for the supported paths (see the known gap
- The bare JSON constants `NaN`, `Infinity` and `-Infinity`, which Python's `json.loads` reads as floats, are accepted in `rl_agent_config.json` and `tokenizer_config.json` (a temperature of `NaN` clamps to 1.0 with Python's warning).
- Checkpoint JSON (`rl_agent_config.json`, `tokenizer_config.json`) is read by a reader that follows Python's `json.loads`: integers keep their literal (the temperature warning prints `0` for `-0` and big integers as Python does), `NaN`/`Infinity`/`-Infinity` are numbers only in value position, and nothing is encoded in-band. `serde_json` now uses `float_roundtrip` so JSON floats equal Python's `float()` bit for bit. An identical warning is emitted once per process, as Python's default filter does.
- Config loading matches Python's `ONNXAgent.__init__` and `json.loads` further: `option_layout` other than `sequential` is refused with Python's message; an integer literal of more than 4300 digits is refused with Python's `ValueError` text; `max_len`/`head_max_len` read `true`/`false` as 1/0 and `-0` as 0, and a float, string, `null`, list or object is an error (Python fails at the first prediction; this port refuses the config when it is loaded, with the same `TypeError` text, `head_max_len` first). A negative value is refused too, since Python slices from the end for it. The JSON reader no longer re-validates the rest of the document for every character of a string.
-- Config JSON errors read like Python's `JSONDecodeError` (`Expecting ',' delimiter: line 1 column 7 (char 6)`, checked on 1,500 generated malformed texts), a file is read in text mode (`
-` becomes `
-` first), and `temperature` / `temperature_by_options` of the wrong JSON type fail while loading with Python's `TypeError` / `AttributeError` text (a string iterates its characters, a dict its keys).
+- Config JSON errors read like Python's `JSONDecodeError` (`Expecting ',' delimiter: line 1 column 7 (char 6)`, checked on 1,500 generated malformed texts), a file is read in text mode (`\r\n` becomes `\n` first), and `temperature` / `temperature_by_options` of the wrong JSON type fail while loading with Python's `TypeError` / `AttributeError` text (a string iterates its characters, a dict its keys).
- A `temperature` list shorter than three loads, and `predict` fails with `IndexError: list index out of range` for a question whose type index it lacks (choice 0, score 1, noul 2), as Python does when it evaluates `self.temperature[qtype]` (even when a `temperature_by_options` bucket matches); entries past the third are ignored.
- `LangGuess` callbacks receive the state as Python passes it (`GuessInput::Value`, or the original
bytes as `GuessInput::Bytes` from `LayaRouter::route_bytes`).
diff --git a/laya-rust/README.md b/laya-rust/README.md
index d6e2ecef..4ccaa001 100644
--- a/laya-rust/README.md
+++ b/laya-rust/README.md
@@ -493,6 +493,43 @@ if let Some(window) = answer.window() { // None for a single-pass state
---
+## Batch prediction
+
+`LayaEngine::predict_batch` answers the same questions over many states and shares forward passes
+between them: N states cost one pass, or `ceil(N / batch_size)` of them, instead of N. It ports
+`ONNXAgent.predict_batch` and gives each state the result `predict` would, in input order:
+
+```rust,no_run
+# use laya::{BatchOptions, LayaEngine, QuestionSet};
+# let engine = LayaEngine::from_directory("x")?; let questions = QuestionSet::new();
+# let states: Vec<serde_json::Value> = vec![];
+let options = BatchOptions::new().batch_size(8).sort_by_length(true);
+let results = engine.predict_batch(&states, &questions, &options)?; // results[i] answers states[i]
+# Ok::<(), laya::LayaError>(())
+```
+
+- **`batch_size`.** The most states per forward pass. `None` or `0` sends them all in one pass, as
+ Python does for `None`, `0` and negative values.
+- **`sort_by_length`.** Groups similarly sized states inside windows of eight batches, so each batch
+ pads to a shorter maximum. It takes effect only with a `batch_size` above one and below the number
+ of states, and results keep input order. Different batch shapes can change floating-point
+ predictions slightly near a decision threshold.
+- **Edges.** No states give an empty list. No questions give one empty result per state, with no
+ state usage. A `null` state is `LayaError::InvalidState` (Python's `TypeError`); any other error
+ fails the whole call.
+- **Router.** `LayaRouter::predict_batch(&requests, &options)` takes `BatchRequest`s (state,
+ questions and the optional `model`, `task`, `lang` and `lang_guess` overrides), routes each,
+ groups them by checkpoint and then by question schema (order-sensitive, so option order is
+ positional), runs each group through `predict_batch` and returns the results in request order,
+ each with its `routing`. `LayaRouter::route_batch(&requests)` returns only the decisions and loads
+ nothing. A predictor without a batch capability (see `LayaPredictor::as_batch_predictor`) gives
+ `LayaError::NotSupported`.
+- **Not ported.** Per-call hooks, `min_confidence`, `lang`, per-request `max_len` / `head_max_len`
+ and the router's `diagnostics` report. `BatchOptions` and `BatchRequest` are `#[non_exhaustive]`
+ so those can be added without breaking callers.
+
+---
+
## Routing, language detection, email and shortlist
Runnable end-to-end example: [`samples/laya-routing`](https://github.com/NandhaKishorM/laya/tree/main/laya-rust/samples/laya-routing).
@@ -704,6 +741,7 @@ invocation that has the `download` feature enabled (the default) with no externa
| `src/tokenization/hf_tokenizer.rs` | HuggingFace tokenizer implementation | `HfTokenizer` |
| `src/laya_engine.rs` | Loads the ONNX model and answers questions | `LayaEngine` |
| `src/long_prediction.rs` | Window plan and aggregation for `predict_long` | `LongOptions` |
+| `src/batch_prediction.rs` | Chunking, sort windows and order restoration for `predict_batch` | `BatchOptions` |
| `src/error.rs` | Error type | `LayaError`, `Result<T>` — every fallible call in this crate returns `Result<T, LayaError>` |
Where the equivalent Python code is a module of related functions (`common.py`, `agent.py`), the
@@ -767,6 +805,6 @@ Rust port groups the corresponding functions into a unit struct with associated
- **Opt-in features are not ported yet:** abstention (`min_confidence`, including per-bucket
thresholds), histogram-binning calibration, and per-language temperatures
- (`lang_temperatures`). `predict_long` is ported, without hooks, per-language temperatures or
- the confidence gate. They are off by default in
+ (`lang_temperatures`). `predict_long` and `predict_batch` are ported, without hooks,
+ per-language temperatures or the confidence gate. They are off by default in
Python, so default predictions are unaffected. They are planned as follow-ups.
diff --git a/laya-rust/src/batch_prediction.rs b/laya-rust/src/batch_prediction.rs
new file mode 100644
index 00000000..f27e9b99
--- /dev/null
+++ b/laya-rust/src/batch_prediction.rs
@@ -0,0 +1,519 @@
+//! The plan behind `LayaEngine::predict_batch`: how many states go through one forward pass, when
+//! states are reordered to pad less, and how results come back in input order. Ports
+//! `ONNXAgent._infer_batch` (`laya/onnx_agent.py`), whose `batch_size` and `sort_by_length`
+//! handling is the same contract as `Agent.predict_batch`.
+//!
+//! The driver here is generic over what an encoded state is, so the chunking, the sort windows
+//! and the order restoration are testable without a model: the engine supplies the tokenizing and
+//! the forward pass.
+//!
+//! Python's per-call hooks, `min_confidence`, `lang`, `max_len` and `head_max_len` are not part of
+//! this port yet. [`BatchOptions`] is `#[non_exhaustive]` so they can be added as fields without a
+//! signature change.
+
+use crate::error::{LayaError, Result};
+use std::ops::Range;
+
+/// How many batches of states the `sort_by_length` lookahead spans. Bounds the tokenized
+/// lookahead independently of the input size, as Python does.
+const SORT_WINDOW_BATCHES: usize = 8;
+
+/// Options for `LayaEngine::predict_batch` and `LayaRouter::predict_batch`. Every field is
+/// optional; `Default` gives Python's defaults. Build one with the chained setters.
+///
+/// Per-call hooks, `min_confidence`, `lang`, `max_len` and `head_max_len`, which Python's
+/// `predict_batch` also accepts, are not part of this port yet.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+#[non_exhaustive]
+pub struct BatchOptions {
+ /// Most states per forward pass. `None` or `0` sends them all in one pass. Python treats a
+ /// `batch_size` of `None`, `0` or any negative number as unset; a negative size cannot be
+ /// written as a `usize`, so `0` stands for all of them here. Set it to bound peak memory when
+ /// batching many or long states.
+ pub batch_size: Option<usize>,
+ /// Group similarly sized encoded states within windows of eight batches, so each batch pads
+ /// to a shorter maximum. Takes effect only with a `batch_size` greater than one and smaller
+ /// than the number of states; otherwise it does nothing. Results keep input order. Changing
+ /// the batch shapes can slightly change floating-point predictions near decision thresholds.
+ pub sort_by_length: bool,
+}
+
+impl BatchOptions {
+ /// Python's defaults: one pass for every state, input order.
+ pub fn new() -> Self {
+ Self::default()
+ }
+
+ /// Set [`Self::batch_size`].
+ #[must_use]
+ pub fn batch_size(mut self, batch_size: usize) -> Self {
+ self.batch_size = Some(batch_size);
+ self
+ }
+
+ /// Set [`Self::sort_by_length`].
+ #[must_use]
+ pub fn sort_by_length(mut self, sort_by_length: bool) -> Self {
+ self.sort_by_length = sort_by_length;
+ self
+ }
+
+ /// States per forward pass for `count` states: `batch_size` when it is positive, else all.
+ fn chunk(&self, count: usize) -> usize {
+ match self.batch_size {
+ Some(size) if size > 0 => size,
+ _ => count,
+ }
+ }
+
+ /// Whether `sort_by_length` reorders `count` states: `1 < chunk < count`.
+ pub(crate) fn reorders(&self, count: usize) -> bool {
+ let chunk = self.chunk(count);
+ self.sort_by_length && 1 < chunk && chunk < count
+ }
+}
+
+/// Encodes, orders, chunks and runs `count` states, returning one result per state in input order.
+///
+/// * `encode` turns a range of input indices into their encoded states (all of a window at once,
+/// so a tokenizing error surfaces before any forward pass of that window, as in Python);
+/// * `row_len` is a state's longest encoded question row, the sort key;
+/// * `infer` runs one chunk of encoded states and returns one result per state of the chunk.
+///
+/// Without a reordering `sort_by_length` the windows are the chunks themselves. With one, each
+/// window spans eight chunks, its states are stably sorted by `row_len`, and the sorted states are
+/// cut into chunks.
+pub(crate) fn run<E, R>(
+ count: usize,
+ options: &BatchOptions,
+ mut encode: impl FnMut(Range<usize>) -> Result<Vec<E>>,
+ row_len: impl Fn(&E) -> usize,
+ mut infer: impl FnMut(Vec<E>) -> Result<Vec<R>>,
+) -> Result<Vec<R>> {
+ if count == 0 {
+ return Ok(Vec::new());
+ }
+ let chunk = options.chunk(count);
+ let reorder = options.reorders(count);
+ let window = if reorder {
+ chunk.saturating_mul(SORT_WINDOW_BATCHES)
+ } else {
+ chunk
+ };
+
+ let mut results = Vec::with_capacity(count);
+ for start in (0..count).step_by(window) {
+ let end = start.saturating_add(window).min(count);
+ let mut slots: Vec<Option<E>> = encode(start..end)?.into_iter().map(Some).collect();
+ let mut order: Vec<usize> = (0..slots.len()).collect();
+ if reorder {
+ order.sort_by_key(|&i| slots[i].as_ref().map_or(0, &row_len));
+ }
+ let mut window_results: Vec<Option<R>> = Vec::with_capacity(slots.len());
+ window_results.resize_with(slots.len(), || None);
+ for indices in order.chunks(chunk) {
+ let batch: Vec<E> = indices.iter().filter_map(|&i| slots[i].take()).collect();
+ let outputs = infer(batch)?;
+ if outputs.len() != indices.len() {
+ return Err(internal(format!(
+ "a forward pass returned {} results for {} states",
+ outputs.len(),
+ indices.len()
+ )));
+ }
+ for (&index, output) in indices.iter().zip(outputs) {
+ window_results[index] = Some(output);
+ }
+ }
+ for result in window_results {
+ results.push(result.ok_or_else(|| internal("a state was left unanswered".into()))?);
+ }
+ }
+ Ok(results)
+}
+
+fn internal(message: String) -> LayaError {
+ LayaError::Onnx(format!("internal error: {message}"))
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ /// An encoded state: its input index and its longest row.
+ type Fake = (usize, usize);
+
+ /// Runs the driver over states whose row lengths are `lens`, recording each forward pass as
+ /// the input indices it carried. The result for a state is its own index.
+ fn drive(lens: &[usize], options: &BatchOptions) -> (Vec<usize>, Vec<Vec<usize>>) {
+ let mut passes = Vec::new();
+ let results = run(
+ lens.len(),
+ options,
+ |range| Ok(range.map(|i| (i, lens[i])).collect::<Vec<Fake>>()),
+ |state| state.1,
+ |batch| {
+ passes.push(batch.iter().map(|s| s.0).collect::<Vec<_>>());
+ Ok(batch.into_iter().map(|s| s.0).collect())
+ },
+ )
+ .unwrap();
+ (results, passes)
+ }
+
+ fn sized(batch_size: usize, sort: bool) -> BatchOptions {
+ BatchOptions::new()
+ .batch_size(batch_size)
+ .sort_by_length(sort)
+ }
+
+ #[test]
+ fn no_batch_size_sends_every_state_in_one_pass() {
+ let (results, passes) = drive(&[5, 4, 3], &BatchOptions::new());
+ assert_eq!(results, [0, 1, 2]);
+ assert_eq!(passes, [vec![0, 1, 2]]);
+ }
+
+ #[test]
+ fn zero_batch_size_is_unset_as_python_treats_it() {
+ let (_, passes) = drive(&[5, 4, 3], &sized(0, true));
+ assert_eq!(passes, [vec![0, 1, 2]]);
+ }
+
+ #[test]
+ fn batch_size_cuts_chunks_in_input_order() {
+ let (results, passes) = drive(&[1, 2, 3, 4, 5], &sized(2, false));
+ assert_eq!(results, [0, 1, 2, 3, 4]);
+ assert_eq!(passes, [vec![0, 1], vec![2, 3], vec![4]]);
+ }
+
+ #[test]
+ fn sorting_groups_similar_lengths_and_restores_input_order() {
+ let (results, passes) = drive(&[9, 1, 8, 2, 7], &sized(2, true));
+ assert_eq!(results, [0, 1, 2, 3, 4]);
+ assert_eq!(passes, [vec![1, 3], vec![4, 2], vec![0]]);
+ }
+
+ #[test]
+ fn sorting_is_stable_for_equal_lengths() {
+ let (_, passes) = drive(&[3, 3, 3, 1, 3], &sized(2, true));
+ assert_eq!(passes, [vec![3, 0], vec![1, 2], vec![4]]);
+ }
+
+ #[test]
+ fn sorting_needs_a_batch_size_above_one_and_below_the_state_count() {
+ let lens = [9, 1, 8, 2];
+ // No batch size: one pass, unsorted.
+ assert_eq!(drive(&lens, &sized(0, true)).1, [vec![0, 1, 2, 3]]);
+ // A batch of one: nothing to group.
+ assert_eq!(
+ drive(&lens, &sized(1, true)).1,
+ [vec![0], vec![1], vec![2], vec![3]]
+ );
+ // A batch as large as the input, or larger: one pass, unsorted.
+ assert_eq!(drive(&lens, &sized(4, true)).1, [vec![0, 1, 2, 3]]);
+ assert_eq!(drive(&lens, &sized(10, true)).1, [vec![0, 1, 2, 3]]);
+ // Just under the count sorts.
+ assert_eq!(drive(&lens, &sized(3, true)).1, [vec![1, 3, 2], vec![0]]);
+ }
+
+ #[test]
+ fn reorders_follows_the_python_gate() {
+ assert!(sized(2, true).reorders(3));
+ assert!(!sized(2, false).reorders(3));
+ assert!(!sized(2, true).reorders(2));
+ assert!(!sized(1, true).reorders(3));
+ assert!(!BatchOptions::new().sort_by_length(true).reorders(3));
+ }
+
+ #[test]
+ fn sort_windows_span_eight_batches() {
+ // Batch size 2, so a window is 16 states. Descending lengths: the first window sorts only
+ // among its own 16 states, the next one among the remaining 4.
+ let lens: Vec<usize> = (0..20).rev().collect();
+ let (results, passes) = drive(&lens, &sized(2, true));
+ assert_eq!(results, (0..20).collect::<Vec<_>>());
+ assert_eq!(passes[0], [15, 14]);
+ assert_eq!(passes[7], [1, 0]);
+ assert_eq!(passes[8], [19, 18]);
+ assert_eq!(passes[9], [17, 16]);
+ assert_eq!(passes.len(), 10);
+ }
+
+ #[test]
+ fn unsorted_windows_are_the_chunks() {
+ let (_, passes) = drive(&[3, 2, 1], &sized(2, false));
+ assert_eq!(passes, [vec![0, 1], vec![2]]);
+ }
+
+ #[test]
+ fn no_states_run_nothing() {
+ let (results, passes) = drive(&[], &sized(2, true));
+ assert!(results.is_empty());
+ assert!(passes.is_empty());
+ }
+
+ #[test]
+ fn an_encode_error_stops_before_any_pass_of_its_window() {
+ let mut passes = 0;
+ let outcome: Result<Vec<usize>> = run(
+ 3,
+ &BatchOptions::new(),
+ |_| Err(LayaError::InvalidQuestion("boom".into())),
+ |_: &usize| 0,
+ |batch: Vec<usize>| {
+ passes += 1;
+ Ok(batch)
+ },
+ );
+ assert!(matches!(outcome, Err(LayaError::InvalidQuestion(_))));
+ assert_eq!(passes, 0);
+ }
+
+ #[test]
+ fn a_pass_returning_the_wrong_count_is_an_error_not_a_panic() {
+ let outcome: Result<Vec<usize>> = run(
+ 2,
+ &BatchOptions::new(),
+ |range| Ok(range.collect::<Vec<usize>>()),
+ |_: &usize| 0,
+ |_| Ok(vec![0usize]),
+ );
+ assert!(outcome.is_err());
+ }
+
+ #[test]
+ fn options_take_chained_setters() {
+ let options = BatchOptions::new().batch_size(4).sort_by_length(true);
+ assert_eq!(options.batch_size, Some(4));
+ assert!(options.sort_by_length);
+ assert_eq!(BatchOptions::default(), BatchOptions::new());
+ }
+
+ // ------------------------------------------------------------------ recorded session runs
+
+ /// Where the per-checkpoint goldens live: `LAYA_GOLDEN_DIR`, else `tests/golden`.
+ fn golden_root() -> std::path::PathBuf {
+ match std::env::var("LAYA_GOLDEN_DIR") {
+ Ok(dir) if !dir.trim().is_empty() => dir.into(),
+ _ => std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/golden"),
+ }
+ }
+
+ /// The fused artifact directory for a checkpoint, found as the integration tests find it:
+ /// `LAYA_ONNX_DIR` (multilingual only), `LAYA_ONNX_ROOT`, then `onnx/<checkpoint>` up the tree.
+ fn artifact_dir(checkpoint: crate::LayaCheckpoint) -> Option<std::path::PathBuf> {
+ use std::path::{Path, PathBuf};
+ let complete = |dir: &Path| {
+ dir.join("rl_agent_config.json").is_file()
+ && dir.join("model.onnx").is_file()
+ && dir.join("model.onnx.data").is_file()
+ && dir.join("tokenizer").join("tokenizer.json").is_file()
+ };
+ let name = checkpoint.subdir();
+ if checkpoint == crate::LayaCheckpoint::Multilingual
+ && let Ok(dir) = std::env::var("LAYA_ONNX_DIR")
+ && !dir.trim().is_empty()
+ {
+ let dir = PathBuf::from(dir);
+ return complete(&dir).then_some(dir);
+ }
+ if let Ok(root) = std::env::var("LAYA_ONNX_ROOT")
+ && !root.trim().is_empty()
+ {
+ let dir = PathBuf::from(root).join(name);
+ if complete(&dir) {
+ return Some(dir);
+ }
+ }
+ let mut dir = Some(Path::new(env!("CARGO_MANIFEST_DIR")).to_path_buf());
+ while let Some(d) = dir {
+ let candidate = d.join("onnx").join(name);
+ if complete(&candidate) {
+ return Some(candidate);
+ }
+ dir = d.parent().map(Path::to_path_buf);
+ }
+ None
+ }
+
+ /// The questions of a recorded case, from its `questions` definition (choice, score, noul).
+ fn recorded_questions(definition: &serde_json::Value) -> crate::QuestionSet {
+ use crate::{Question, QuestionSet};
+ use serde_json::Value;
+ let mut set = QuestionSet::new();
+ for (id, def) in definition.as_object().expect("questions object") {
+ let instructions = def["instructions"].clone();
+ let criteria = def.get("criteria");
+ let question = match def["type"].as_str().expect("question type") {
+ "choice" => {
+ let options: Vec<(String, Value)> = match criteria.expect("criteria") {
+ Value::Array(labels) => labels
+ .iter()
+ .map(|l| (l.as_str().expect("label").to_string(), Value::Null))
+ .collect(),
+ Value::Object(map) => {
+ map.iter().map(|(k, v)| (k.clone(), v.clone())).collect()
+ }
+ other => panic!("{id}: unusable criteria {other}"),
+ };
+ Question::choice(instructions, options).expect("choice question")
+ }
+ "score" => {
+ let levels = criteria.and_then(Value::as_array).expect("levels").clone();
+ Question::score(instructions, levels).expect("score question")
+ }
+ "noul" => {
+ let side = |key: &str| {
+ criteria
+ .and_then(|c| c.get(key))
+ .cloned()
+ .unwrap_or(Value::Null)
+ };
+ Question::noul(instructions, side("false"), side("true"))
+ }
+ other => panic!("{id}: unknown question type {other}"),
+ };
+ set.insert(id.clone(), question).expect("unique ids");
+ }
+ set
+ }
+
+ fn recorded_options(options: &serde_json::Value) -> BatchOptions {
+ BatchOptions {
+ // Python treats a batch_size that is zero or negative as unset.
+ batch_size: options
+ .get("batch_size")
+ .and_then(serde_json::Value::as_i64)
+ .and_then(|size| usize::try_from(size).ok())
+ .filter(|&size| size > 0),
+ sort_by_length: options
+ .get("sort_by_length")
+ .and_then(serde_json::Value::as_bool)
+ .unwrap_or(false),
+ }
+ }
+
+ /// Python's own session-run shapes against this port's chunk planning, over this port's own
+ /// token counts: each state's width (its longest question row) must equal Python's
+ /// `state_widths`, and the planned runs, `[rows, width]` per forward pass, must be the ones
+ /// Python made. Needs the tokenizer and the batch goldens, not a forward pass; skips like the
+ /// integration tests when either is missing.
+ #[test]
+ fn plans_the_recorded_session_runs() {
+ use crate::laya_config::LayaConfig;
+ use crate::model_artifacts::ModelArtifacts;
+ use crate::sequence_builder::SequenceBuilder;
+ use crate::tokenization::HfTokenizer;
+ use serde_json::Value;
+
+ let mut checked_any = false;
+ for checkpoint in [
+ crate::LayaCheckpoint::Multilingual,
+ crate::LayaCheckpoint::English,
+ crate::LayaCheckpoint::TypedDecisions,
+ ] {
+ let Some(dir) = artifact_dir(checkpoint) else {
+ continue;
+ };
+ let golden_dir = golden_root().join(checkpoint.subdir());
+ let Ok(index) = std::fs::read_to_string(golden_dir.join("index.json")) else {
+ eprintln!("skip: no golden data for '{}'", checkpoint.subdir());
+ continue;
+ };
+ let index: Value = serde_json::from_str(&index).expect("index.json");
+ let artifacts =
+ ModelArtifacts::from_directory(&dir, Some(checkpoint)).expect("artifacts");
+ let config = LayaConfig::load(&artifacts.config_path).expect("config");
+ let tokenizer = HfTokenizer::from_file_with_config(
+ &artifacts.tokenizer_path,
+ artifacts.tokenizer_config_path.as_deref(),
+ )
+ .expect("tokenizer");
+
+ for entry in index.as_array().expect("index array") {
+ let is_batch = entry["kind"].as_str() == Some("batch");
+ if !is_batch || entry["expects_error"].as_bool() == Some(true) {
+ continue;
+ }
+ let text =
+ std::fs::read_to_string(golden_dir.join(entry["file"].as_str().unwrap()))
+ .expect("case file");
+ let root: Value = serde_json::from_str(&text).expect("case json");
+ let case = format!(
+ "{}/{}",
+ checkpoint.subdir(),
+ entry["name"].as_str().unwrap()
+ );
+ let states = root["states"].as_array().expect("states");
+ let questions = recorded_questions(&root["questions"]);
+ if states.is_empty() || questions.is_empty() {
+ continue;
+ }
+ checked_any = true;
+
+ let widths: Vec<usize> = states
+ .iter()
+ .map(|state| {
+ questions
+ .iter()
+ .map(|(_, question)| {
+ SequenceBuilder::build_with_stats(
+ &tokenizer,
+ state,
+ question,
+ config.max_len,
+ config.head_max_len,
+ matches!(state, Value::Array(_)),
+ )
+ .expect("sequence")
+ .0
+ .len()
+ })
+ .max()
+ .unwrap_or(0)
+ })
+ .collect();
+ let recorded_widths: Vec<usize> = root["state_widths"]
+ .as_array()
+ .expect("state_widths")
+ .iter()
+ .map(|w| w.as_u64().expect("width") as usize)
+ .collect();
+ assert_eq!(recorded_widths, widths, "{case}: state widths");
+
+ let mut runs: Vec<[usize; 2]> = Vec::new();
+ run(
+ states.len(),
+ &recorded_options(&root["options"]),
+ |range| Ok(range.map(|i| widths[i]).collect::<Vec<usize>>()),
+ |width: &usize| *width,
+ |chunk: Vec<usize>| {
+ runs.push([
+ chunk.len() * questions.len(),
+ chunk.iter().copied().max().unwrap_or(0),
+ ]);
+ Ok(chunk)
+ },
+ )
+ .expect("plan");
+ let recorded_runs: Vec<[usize; 2]> = root["runs"]
+ .as_array()
+ .expect("runs")
+ .iter()
+ .map(|r| {
+ let r = r.as_array().expect("run");
+ [
+ r[0].as_u64().unwrap() as usize,
+ r[1].as_u64().unwrap() as usize,
+ ]
+ })
+ .collect();
+ assert_eq!(recorded_runs, runs, "{case}: planned session runs");
+ }
+ }
+ if !checked_any {
+ eprintln!("skip: no ONNX tokenizer + batch golden data found for any checkpoint");
+ }
+ }
+}
diff --git a/laya-rust/src/error.rs b/laya-rust/src/error.rs
index 4c7c93f7..93828e3a 100644
--- a/laya-rust/src/error.rs
+++ b/laya-rust/src/error.rs
@@ -103,6 +103,11 @@ pub enum LayaError {
max_len: usize,
},
+ /// A state or request was refused before any inference: a `null` state, which Python rejects
+ /// with a `TypeError` instead of answering a decision about the text `"null"`.
+ #[error("{0}")]
+ InvalidState(String),
+
/// `predict_long` was given an aggregation mode other than `"auto"`.
#[error("predict_long: only aggregate='auto' is supported")]
UnsupportedAggregate(String),
diff --git a/laya-rust/src/laya_engine.rs b/laya-rust/src/laya_engine.rs
index 532f1e68..f637102f 100644
--- a/laya-rust/src/laya_engine.rs
+++ b/laya-rust/src/laya_engine.rs
@@ -16,6 +16,7 @@ use crate::answers::{
ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScanUsage, ScoreAnswer, StateUsage,
Usage,
};
+use crate::batch_prediction::{self, BatchOptions};
use crate::calibration::Calibration;
use crate::collator::{CollatedBatch, Collator, SequenceItem};
use crate::error::{LayaError, Result};
@@ -56,6 +57,38 @@ pub struct LayaOutput {
pub act_logits: Vec<f32>,
}
+/// One state's tokenized question rows, plus what its truncation report says, ready to collate.
+struct EncodedState {
+ items: Vec<SequenceItem>,
+ usage: StateUsage,
+}
+
+impl EncodedState {
+ /// The longest question row, the key `sort_by_length` orders by.
+ fn longest_row(&self) -> usize {
+ self.items
+ .iter()
+ .map(|item| item.ids.len())
+ .max()
+ .unwrap_or(0)
+ }
+}
+
+/// Python's `TypeError` for a `None` state: scoring it would answer a decision about the text
+/// `"null"`, byte-identical to passing `"null"` and at full confidence.
+pub(crate) fn refuse_null_states(states: &[Value]) -> Result<()> {
+ if states.iter().any(Value::is_null) {
+ return Err(LayaError::InvalidState(
+ "state must not be None; pass a string, dict, or list".to_string(),
+ ));
+ }
+ Ok(())
+}
+
+fn question_ids(questions: &QuestionSet) -> Vec<String> {
+ questions.ids().map(str::to_string).collect()
+}
+
/// The loaded ONNX session(s) for whichever [`crate::model_artifacts::ModelLayout`] the artifact
/// directory used. `Fused` holds the single graph; `Split` holds the encoder and head graphs that
/// [`LayaEngine::run`] chains together, feeding the encoder's `last_hidden_state` output into the
@@ -156,6 +189,9 @@ impl LayaEngine {
/// [`crate::python_json::PythonJson::state`]). For a type that only implements
/// [`serde::Serialize`] (rather than `Into<Value>`), use [`Self::predict_serialize`] instead.
///
+ /// Like Python's `system_one`, this is a one-state [`Self::predict_batch`]: a `null` state is
+ /// [`LayaError::InvalidState`], and no questions give an empty result with zero usage.
+ ///
/// Returns [`LayaError::NoMarkers`] if a question's options cannot fit within
/// `Config::head_max_len`, rather than silently scoring fewer options than it was asked about.
pub fn predict(&self, state: impl Into<Value>, questions: &QuestionSet) -> Result<LayaResult> {
@@ -173,14 +209,15 @@ impl LayaEngine {
self.predict_value(value, questions)
}
+ /// Python's `system_one` is `predict_batch([state], ...)`, so this is a one-state batch: a
+ /// `null` state is refused and no questions give an empty result.
fn predict_value(&self, state: Value, questions: &QuestionSet) -> Result<LayaResult> {
- if questions.is_empty() {
- return Err(LayaError::InvalidQuestion(
- "no questions to answer".to_string(),
- ));
- }
- let mut results = self.predict_states(&[state], questions)?;
- Ok(results.remove(0))
+ let states = [state];
+ refuse_null_states(&states)?;
+ let mut results = self.predict_states(&states, questions)?;
+ results
+ .pop()
+ .ok_or_else(|| LayaError::Onnx("internal error: no result for the state".to_string()))
}
/// Scan a state longer than one sequence in overlapping token windows and keep, per question,
@@ -234,6 +271,8 @@ impl LayaEngine {
questions: &QuestionSet,
options: &LongOptions,
) -> Result<LayaResult> {
+ // Python checks the state before the aggregate mode.
+ refuse_null_states(std::slice::from_ref(&state))?;
options.check_aggregate()?;
let tokenizer = self.tokenizer.as_ref();
let explicit_window = options.window.unwrap_or(0);
@@ -289,6 +328,62 @@ impl LayaEngine {
))
}
+ /// Answer every question about every state, `options.batch_size` states per forward pass.
+ /// Ports `ONNXAgent.predict_batch`: the same questions are scored over each state, and the
+ /// result for state `i` is the one `predict` gives it, in input order.
+ ///
+ /// * `batch_size` of `None` or `0` sends every state in one pass; see [`BatchOptions`].
+ /// * `sort_by_length` groups similarly sized states inside windows of eight batches, and only
+ /// with a `batch_size` above one and below the number of states.
+ /// * No states give an empty list. No questions give one empty result per state, with no
+ /// state usage, as Python does.
+ ///
+ /// A `null` state is refused with [`LayaError::InvalidState`] (Python's `TypeError`: scoring it
+ /// would answer a decision about the text `"null"`). Everything else [`Self::predict`] returns
+ /// applies per state, and the first error fails the whole call.
+ pub fn predict_batch(
+ &self,
+ states: &[Value],
+ questions: &QuestionSet,
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ refuse_null_states(states)?;
+ if states.is_empty() {
+ return Ok(Vec::new());
+ }
+ let ids = question_ids(questions);
+ if ids.is_empty() {
+ return Ok(self.empty_results(states.len()));
+ }
+ batch_prediction::run(
+ states.len(),
+ options,
+ |range| {
+ states[range]
+ .iter()
+ .map(|state| self.encode_state(state, &ids, questions))
+ .collect()
+ },
+ EncodedState::longest_row,
+ |batch| self.infer_encoded(batch, &ids, questions),
+ )
+ }
+
+ /// [`Self::predict_batch`], for states that implement [`serde::Serialize`] but not
+ /// `Into<serde_json::Value>`.
+ pub fn predict_batch_serialize<S: Serialize>(
+ &self,
+ states: &[S],
+ questions: &QuestionSet,
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ let values = states
+ .iter()
+ .map(serde_json::to_value)
+ .collect::<std::result::Result<Vec<Value>, _>>()?;
+ self.predict_batch(&values, questions, options)
+ }
+
/// Scores every question against every state in one collate and one forward pass, returning
/// one result (with its own state usage) per state, in order. With no questions each result is
/// empty and carries no state usage. `predict` is this with one state.
@@ -297,72 +392,106 @@ impl LayaEngine {
states: &[Value],
questions: &QuestionSet,
) -> Result<Vec<LayaResult>> {
- let ids: Vec<String> = questions.ids().map(str::to_string).collect();
+ let ids = question_ids(questions);
if ids.is_empty() {
- return Ok(states
- .iter()
- .map(|_| LayaResult::new(MODEL_NAME, Vec::new(), HashMap::new(), Usage::default()))
- .collect());
+ return Ok(self.empty_results(states.len()));
}
+ let encoded = states
+ .iter()
+ .map(|state| self.encode_state(state, &ids, questions))
+ .collect::<Result<Vec<_>>>()?;
+ self.infer_encoded(encoded, &ids, questions)
+ }
- let mut items = Vec::with_capacity(ids.len() * states.len());
- let mut state_usages = Vec::with_capacity(states.len());
- for state in states {
- let mut state_usage = StateUsage {
- state_tokens: 0,
- state_tokens_dropped: 0,
- truncated: false,
- truncated_questions: Vec::new(),
- options: IndexMap::new(),
- };
-
- // A conversation-turn list is serialized newest-last (see `PythonJson::state`), so the
- // default right-truncation of an over-long state would silently drop the newest turn.
- // Truncate from the left for lists so the most recent turn survives instead. Ports
- // `Agent._encode_state`'s `truncate_left = isinstance(state, list)` (laya/agent.py,
- // laya 0.3.21) — 0.3.6 never set this, so every state was right-truncated regardless
- // of shape.
- let truncate_left = matches!(state, Value::Array(_));
+ /// The result for a state when there are no questions to answer.
+ fn empty_results(&self, count: usize) -> Vec<LayaResult> {
+ (0..count)
+ .map(|_| LayaResult::new(MODEL_NAME, Vec::new(), HashMap::new(), Usage::default()))
+ .collect()
+ }
- // The tokenizer here needs no lock: tokenizers::Tokenizer::encode takes &self.
- for id in &ids {
- let question = &questions[id.as_str()];
- let (seq, markers, stats) = SequenceBuilder::build_with_stats(
- self.tokenizer.as_ref(),
- state,
- question,
- self.config.max_len,
- self.config.head_max_len,
- truncate_left,
- )?;
- state_usage.state_tokens = stats.tokens;
- state_usage.state_tokens_dropped =
- state_usage.state_tokens_dropped.max(stats.dropped);
- if stats.dropped > 0 {
- state_usage.truncated = true;
- state_usage.truncated_questions.push(id.clone());
- }
- if let Some(options) = stats.options {
- state_usage.options.insert(id.clone(), options);
- }
+ /// One state's question rows and the truncation report for them, tokenizing the state for
+ /// each question. Ports `ONNXAgent._encode_state`.
+ fn encode_state(
+ &self,
+ state: &Value,
+ ids: &[String],
+ questions: &QuestionSet,
+ ) -> Result<EncodedState> {
+ let mut state_usage = StateUsage {
+ state_tokens: 0,
+ state_tokens_dropped: 0,
+ truncated: false,
+ truncated_questions: Vec::new(),
+ options: IndexMap::new(),
+ };
- // A marker dropped for landing past max_len means an option is not in the
- // sequence at all, so its logit would score whatever token happens to sit at
- // position 0.
- let expected_markers = SequenceBuilder::render_options(question).len();
- if markers.len() != expected_markers {
- return Err(LayaError::NoMarkers {
- id: id.clone(),
- head_max_len: self.config.head_max_len,
- actual_markers: markers.len(),
- expected_markers,
- max_len: self.config.max_len,
- });
- }
+ // A conversation-turn list is serialized newest-last (see `PythonJson::state`), so the
+ // default right-truncation of an over-long state would silently drop the newest turn.
+ // Truncate from the left for lists so the most recent turn survives instead. Ports
+ // `Agent._encode_state`'s `truncate_left = isinstance(state, list)` (laya/agent.py,
+ // laya 0.3.21) — 0.3.6 never set this, so every state was right-truncated regardless
+ // of shape.
+ let truncate_left = matches!(state, Value::Array(_));
+
+ let mut items = Vec::with_capacity(ids.len());
+ // The tokenizer here needs no lock: tokenizers::Tokenizer::encode takes &self.
+ for id in ids {
+ let question = &questions[id.as_str()];
+ let (seq, markers, stats) = SequenceBuilder::build_with_stats(
+ self.tokenizer.as_ref(),
+ state,
+ question,
+ self.config.max_len,
+ self.config.head_max_len,
+ truncate_left,
+ )?;
+ state_usage.state_tokens = stats.tokens;
+ state_usage.state_tokens_dropped = state_usage.state_tokens_dropped.max(stats.dropped);
+ if stats.dropped > 0 {
+ state_usage.truncated = true;
+ state_usage.truncated_questions.push(id.clone());
+ }
+ if let Some(options) = stats.options {
+ state_usage.options.insert(id.clone(), options);
+ }
- items.push(SequenceItem::new(seq, markers, question.question_type()));
+ // A marker dropped for landing past max_len means an option is not in the
+ // sequence at all, so its logit would score whatever token happens to sit at
+ // position 0.
+ let expected_markers = SequenceBuilder::render_options(question).len();
+ if markers.len() != expected_markers {
+ return Err(LayaError::NoMarkers {
+ id: id.clone(),
+ head_max_len: self.config.head_max_len,
+ actual_markers: markers.len(),
+ expected_markers,
+ max_len: self.config.max_len,
+ });
}
- state_usages.push(state_usage);
+
+ items.push(SequenceItem::new(seq, markers, question.question_type()));
+ }
+ Ok(EncodedState {
+ items,
+ usage: state_usage,
+ })
+ }
+
+ /// Collates the encoded states' rows into one batch, runs one forward pass and decodes one
+ /// result per state, in the order given. `ids` must be non-empty.
+ fn infer_encoded(
+ &self,
+ encoded: Vec<EncodedState>,
+ ids: &[String],
+ questions: &QuestionSet,
+ ) -> Result<Vec<LayaResult>> {
+ let state_count = encoded.len();
+ let mut items = Vec::with_capacity(ids.len() * state_count);
+ let mut state_usages = Vec::with_capacity(state_count);
+ for state in encoded {
+ items.extend(state.items);
+ state_usages.push(state.usage);
}
let batch = Collator::collate(&items, self.tokenizer.special().pad_id)?;
@@ -372,7 +501,7 @@ impl LayaEngine {
// hardcoded so this stays correct if the graph is re-exported with a wider action head.
let n_act = output.act_logits.len() / batch.count;
- let mut results = Vec::with_capacity(states.len());
+ let mut results = Vec::with_capacity(state_count);
for (state_index, state_usage) in state_usages.into_iter().enumerate() {
let offset = state_index * ids.len();
let mut answers = HashMap::with_capacity(ids.len());
@@ -414,7 +543,7 @@ impl LayaEngine {
results.push(
LayaResult::new(
MODEL_NAME,
- ids.clone(),
+ ids.to_vec(),
answers,
Usage {
input_tokens: input_tokens as u32,
@@ -848,6 +977,17 @@ mod tests {
use super::*;
use serde_json::json;
+ #[test]
+ fn a_null_state_is_refused_with_pythons_text() {
+ assert!(refuse_null_states(&[json!("a"), json!([null]), json!({})]).is_ok());
+ let error = refuse_null_states(&[json!("a"), Value::Null]).unwrap_err();
+ assert!(matches!(error, LayaError::InvalidState(_)), "{error:?}");
+ assert_eq!(
+ error.to_string(),
+ "state must not be None; pass a string, dict, or list"
+ );
+ }
+
#[test]
fn score_legends_render_structured_levels_as_strings() {
let question = Question::score(
diff --git a/laya-rust/src/laya_router.rs b/laya-rust/src/laya_router.rs
index 70784ceb..9180f9d9 100644
--- a/laya-rust/src/laya_router.rs
+++ b/laya-rust/src/laya_router.rs
@@ -37,9 +37,10 @@
//! blocks on the lock, then sees the checkpoint already resident).
use crate::answers::LayaResult;
+use crate::batch_prediction::BatchOptions;
use crate::error::{LayaError, Result};
use crate::language_detection::{LanguageAnalysis, LanguageDetection, prefix_chars, py_strip};
-use crate::laya_engine::LayaEngine;
+use crate::laya_engine::{LayaEngine, refuse_null_states};
use crate::laya_options::{LayaCheckpoint, LayaOptions};
use crate::long_prediction::LongOptions;
use crate::questions::QuestionSet;
@@ -68,6 +69,25 @@ pub trait LayaPredictor: Send + Sync {
fn as_long_predictor(&self) -> Option<&dyn LayaLongPredictor> {
None
}
+
+ /// This predictor's batch capability, if it has one. The default is `None`, so implementing
+ /// [`LayaPredictor`] elsewhere keeps compiling; [`LayaEngine`] and [`LayaRouter`] return
+ /// themselves. [`LayaRouter::predict_batch`] needs it.
+ fn as_batch_predictor(&self) -> Option<&dyn LayaBatchPredictor> {
+ None
+ }
+}
+
+/// A [`LayaPredictor`] that can score many states against the same questions in shared forward
+/// passes. The Rust counterpart of Python's router calling `agent.predict_batch`.
+pub trait LayaBatchPredictor: LayaPredictor {
+ /// Answer `questions` about every state, in input order. See [`LayaEngine::predict_batch`].
+ fn predict_batch(
+ &self,
+ states: &[Value],
+ questions: &QuestionSet,
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>>;
}
/// A [`LayaPredictor`] that can also scan a state longer than one sequence. The Rust counterpart
@@ -91,6 +111,21 @@ impl LayaPredictor for LayaEngine {
fn as_long_predictor(&self) -> Option<&dyn LayaLongPredictor> {
Some(self)
}
+
+ fn as_batch_predictor(&self) -> Option<&dyn LayaBatchPredictor> {
+ Some(self)
+ }
+}
+
+impl LayaBatchPredictor for LayaEngine {
+ fn predict_batch(
+ &self,
+ states: &[Value],
+ questions: &QuestionSet,
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ LayaEngine::predict_batch(self, states, questions, options)
+ }
}
impl LayaLongPredictor for LayaEngine {
@@ -262,6 +297,74 @@ impl std::fmt::Display for RouteDecision {
}
}
+/// One request of [`LayaRouter::route_batch`] / [`LayaRouter::predict_batch`]: a state, its
+/// questions and the same optional routing overrides [`LayaRouter::route_with_lang_guess`] takes.
+/// Python passes these as a request dict with the keys `state`, `questions`, `model`, `task`,
+/// `lang` and `lang_guess`.
+///
+/// Per-request `max_len` and `head_max_len` are not part of this port yet; the type is
+/// `#[non_exhaustive]` so they can be added as fields later. Build one with [`Self::new`] and the
+/// chained setters.
+#[derive(Debug, Clone)]
+#[non_exhaustive]
+pub struct BatchRequest {
+ /// What to decide about. A `null` state is refused, as Python's `TypeError`.
+ pub state: Value,
+ /// The questions to answer. Shared by `Arc`, so requests that ask the same questions need not
+ /// each hold a copy; requests with equal question schemas share a forward pass either way.
+ pub questions: Arc<QuestionSet>,
+ /// Route to this checkpoint (a name or alias), skipping detection.
+ pub model: Option<String>,
+ /// Route by task, such as `"typed_decisions"`.
+ pub task: Option<String>,
+ /// A language code that decides English or not, ahead of detection.
+ pub lang: Option<String>,
+ /// A per-request language hint; see [`LayaRouter::route_with_lang_guess`].
+ pub lang_guess: Option<LangGuess>,
+}
+
+impl BatchRequest {
+ /// A request with no routing overrides.
+ pub fn new(state: impl Into<Value>, questions: impl Into<Arc<QuestionSet>>) -> Self {
+ Self {
+ state: state.into(),
+ questions: questions.into(),
+ model: None,
+ task: None,
+ lang: None,
+ lang_guess: None,
+ }
+ }
+
+ /// Set [`Self::model`].
+ #[must_use]
+ pub fn model(mut self, model: impl Into<String>) -> Self {
+ self.model = Some(model.into());
+ self
+ }
+
+ /// Set [`Self::task`].
+ #[must_use]
+ pub fn task(mut self, task: impl Into<String>) -> Self {
+ self.task = Some(task.into());
+ self
+ }
+
+ /// Set [`Self::lang`].
+ #[must_use]
+ pub fn lang(mut self, lang: impl Into<String>) -> Self {
+ self.lang = Some(lang.into());
+ self
+ }
+
+ /// Set [`Self::lang_guess`].
+ #[must_use]
+ pub fn lang_guess(mut self, lang_guess: impl Into<LangGuess>) -> Self {
+ self.lang_guess = Some(lang_guess.into());
+ self
+ }
+}
+
/// Configuration for [`LayaRouter`]. `engine_options` is the template every loaded checkpoint's
/// [`LayaOptions`] is built from (provider, threads, download settings, cache, HF token); its own
/// `checkpoint` field is ignored, since each load overrides it with the checkpoint being built.
@@ -410,9 +513,9 @@ impl LayaRouter {
"multilingual" => Ok(LayaCheckpoint::Multilingual),
"typed-decisions" => Ok(LayaCheckpoint::TypedDecisions),
_ => Err(LayaError::InvalidConfig(format!(
- "unknown model {}; choose one of \"english\", \"multilingual\", \"typed-decisions\" \
- (or an alias: \"en\", \"laya\", \"default\", \"multi\", \"ml\", \"laya-multilingual\", \
- \"typed\", \"typed_decisions\", \"laya-typed-decisions\", \"decisions\")",
+ "unknown model {}; choose one of ['english', 'multilingual', 'typed-decisions'] \
+ (or an alias: ['decisions', 'default', 'en', 'laya', 'laya-multilingual', \
+ 'laya-typed-decisions', 'ml', 'multi', 'typed', 'typed_decisions'])",
python_repr_str(name)
))),
}
@@ -809,6 +912,8 @@ impl LayaRouter {
lang_guess: Option<&LangGuess>,
) -> Result<LayaResult> {
let state = state.into();
+ // Python's `Router.predict` refuses `None` before it routes or loads anything.
+ refuse_null_states(std::slice::from_ref(&state))?;
let decision =
self.route_with_lang_guess(&state, Some(questions), model, task, lang, lang_guess)?;
let engine = self.load_checkpoint(decision.model)?;
@@ -818,6 +923,131 @@ impl LayaRouter {
Ok(result.with_routing(decision))
}
+ /// Route a heterogeneous request batch without loading any checkpoint. Ports
+ /// `Router.route_batch`: each request is routed as [`Self::route_with_lang_guess`] routes it
+ /// from its own `model`, `task`, `lang` and `lang_guess`, and the decisions come back in
+ /// input order.
+ ///
+ /// A request whose state is `null` is refused with [`LayaError::InvalidState`], naming the
+ /// request's index as Python does; the first failing request fails the call.
+ pub fn route_batch(&self, requests: &[BatchRequest]) -> Result<Vec<RouteDecision>> {
+ requests
+ .iter()
+ .enumerate()
+ .map(|(index, request)| {
+ if request.state.is_null() {
+ return Err(LayaError::InvalidState(format!(
+ "request {index} 'state' must not be None; pass a string, dict, or list"
+ )));
+ }
+ self.route_with_lang_guess(
+ &request.state,
+ Some(&request.questions),
+ request.model.as_deref(),
+ request.task.as_deref(),
+ request.lang.as_deref(),
+ request.lang_guess.as_ref(),
+ )
+ })
+ .collect()
+ }
+
+ /// Route and answer a heterogeneous request batch with minimal model churn. Ports
+ /// `Router.predict_batch`: requests are routed first (nothing loads if routing fails), grouped
+ /// by checkpoint in order of first appearance, and within a checkpoint by question schema
+ /// (order-sensitive: ids, types, instructions and options), so the states of one group share
+ /// forward passes through [`LayaEngine::predict_batch`]. Each result carries its request's
+ /// [`LayaResult::routing`] and the results come back in input order.
+ ///
+ /// `options` goes to every group: `batch_size` caps states per forward pass and
+ /// `sort_by_length` reorders within a group only when that group clears the engine's own
+ /// `1 < batch_size < group size` gate. No requests give an empty list. The first failing group
+ /// fails the call. Returns [`LayaError::NotSupported`] when a chosen predictor (for example
+ /// one registered with [`Self::attach`]) has no batch capability.
+ pub fn predict_batch(
+ &self,
+ requests: &[BatchRequest],
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ let decisions = self.route_batch(requests)?;
+ if decisions.is_empty() {
+ return Ok(Vec::new());
+ }
+
+ // Insertion order keeps cache effects deterministic and collapses an interleaved
+ // workload to one load per routed checkpoint for this call.
+ let mut groups: Vec<(LayaCheckpoint, Vec<usize>)> = Vec::new();
+ for (index, decision) in decisions.iter().enumerate() {
+ match groups
+ .iter_mut()
+ .find(|(model, _)| *model == decision.model)
+ {
+ Some((_, indices)) => indices.push(index),
+ None => groups.push((decision.model, vec![index])),
+ }
+ }
+
+ let mut results: Vec<Option<LayaResult>> = Vec::with_capacity(requests.len());
+ results.resize_with(requests.len(), || None);
+ for (model, indices) in groups {
+ let predictor = self.load_checkpoint(model)?;
+ let batcher = predictor.as_batch_predictor().ok_or_else(|| {
+ LayaError::NotSupported(format!(
+ "the predictor for '{}' has no predict_batch, so a batch cannot share \
+ forward passes",
+ model.subdir()
+ ))
+ })?;
+
+ // Order-sensitive at every level: options are positional, so two equal schemas with
+ // different orders must not share a group.
+ let mut question_groups: Vec<(String, Vec<usize>)> = Vec::new();
+ let mut last: Option<(&Arc<QuestionSet>, String)> = None;
+ for &index in &indices {
+ let questions = &requests[index].questions;
+ let key = match &last {
+ Some((seen, key)) if Arc::ptr_eq(seen, questions) => key.clone(),
+ _ => questions.schema_key(),
+ };
+ last = Some((questions, key.clone()));
+ match question_groups.iter_mut().find(|(known, _)| *known == key) {
+ Some((_, items)) => items.push(index),
+ None => question_groups.push((key, vec![index])),
+ }
+ }
+
+ for (_, items) in question_groups {
+ let states: Vec<Value> = items
+ .iter()
+ .map(|&index| requests[index].state.clone())
+ .collect();
+ let batch =
+ batcher.predict_batch(&states, &requests[items[0]].questions, options)?;
+ if batch.len() != items.len() {
+ return Err(LayaError::Onnx(format!(
+ "internal error: predict_batch returned {} results for {} states",
+ batch.len(),
+ items.len()
+ )));
+ }
+ for (&index, result) in items.iter().zip(batch) {
+ results[index] = Some(result.with_routing(decisions[index].clone()));
+ }
+ }
+ }
+
+ results
+ .into_iter()
+ .map(|result| {
+ result.ok_or_else(|| {
+ LayaError::Onnx(
+ "internal error: batch execution did not produce every result".to_string(),
+ )
+ })
+ })
+ .collect()
+ }
+
/// Route once, on the original state, then scan the whole state on the chosen checkpoint with
/// [`LayaEngine::predict_long`]. The result's [`LayaResult::routing`] records the decision.
///
@@ -849,6 +1079,8 @@ impl LayaRouter {
lang_guess: Option<&LangGuess>,
) -> Result<LayaResult> {
let state = state.into();
+ // Python's `Router.predict` refuses `None` before it routes or loads anything.
+ refuse_null_states(std::slice::from_ref(&state))?;
let decision =
self.route_with_lang_guess(&state, Some(questions), model, task, lang, lang_guess)?;
let engine = self.load_checkpoint(decision.model)?;
@@ -874,6 +1106,28 @@ impl LayaPredictor for LayaRouter {
fn as_long_predictor(&self) -> Option<&dyn LayaLongPredictor> {
Some(self)
}
+
+ fn as_batch_predictor(&self) -> Option<&dyn LayaBatchPredictor> {
+ Some(self)
+ }
+}
+
+impl LayaBatchPredictor for LayaRouter {
+ /// [`LayaRouter::predict_batch`] with every override left at its default: each state is
+ /// routed on its own, as [`LayaRouter::predict`] routes one.
+ fn predict_batch(
+ &self,
+ states: &[Value],
+ questions: &QuestionSet,
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ let questions = Arc::new(questions.clone());
+ let requests: Vec<BatchRequest> = states
+ .iter()
+ .map(|state| BatchRequest::new(state.clone(), Arc::clone(&questions)))
+ .collect();
+ LayaRouter::predict_batch(self, &requests, options)
+ }
}
impl LayaLongPredictor for LayaRouter {
diff --git a/laya-rust/src/lib.rs b/laya-rust/src/lib.rs
index 78f92bfd..db35896b 100644
--- a/laya-rust/src/lib.rs
+++ b/laya-rust/src/lib.rs
@@ -41,6 +41,7 @@
//! records the clamped entries.
pub mod answers;
+pub mod batch_prediction;
pub mod calibration;
pub mod collator;
pub mod error;
@@ -64,6 +65,7 @@ pub use answers::{
ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScanUsage, ScoreAnswer, StateUsage,
Usage, WindowInfo,
};
+pub use batch_prediction::BatchOptions;
pub use calibration::Calibration;
pub use collator::{CollatedBatch, Collator, SequenceItem};
pub use error::{LayaError, Result};
@@ -76,8 +78,8 @@ pub use laya_options::{
};
pub use laya_presets::LayaPresets;
pub use laya_router::{
- GuessInput, LangGuess, LayaLongPredictor, LayaPredictor, LayaRouter, LayaRouterOptions,
- RouteDecision,
+ BatchRequest, GuessInput, LangGuess, LayaBatchPredictor, LayaLongPredictor, LayaPredictor,
+ LayaRouter, LayaRouterOptions, RouteDecision,
};
pub use laya_shortlist::{DEFAULT_SHORTLIST_K, LayaShortlist, ShortlistInfo};
pub use long_prediction::LongOptions;
diff --git a/laya-rust/src/questions.rs b/laya-rust/src/questions.rs
index 3e763cbc..d9abb578 100644
--- a/laya-rust/src/questions.rs
+++ b/laya-rust/src/questions.rs
@@ -317,6 +317,48 @@ impl QuestionSet {
}
}
+impl QuestionSet {
+ /// An order-sensitive signature of the set: two sets share it only when they have the same
+ /// ids, types, instructions and options in the same order. The router groups requests by it
+ /// so equal schemas share a forward pass. Ports `_question_schema` (`laya/router.py`), which
+ /// is `json.dumps(questions, sort_keys=False)`: option and key order are positional.
+ pub(crate) fn schema_key(&self) -> String {
+ let rows: Vec<Value> = self
+ .map
+ .iter()
+ .map(|(id, question)| {
+ let body = match question {
+ Question::Choice(q) => Value::Array(vec![
+ Value::from("choice"),
+ q.instructions.clone(),
+ Value::Array(
+ q.options
+ .iter()
+ .map(|(label, text)| {
+ Value::Array(vec![Value::from(label.as_str()), text.clone()])
+ })
+ .collect(),
+ ),
+ ]),
+ Question::Score(q) => Value::Array(vec![
+ Value::from("score"),
+ q.instructions.clone(),
+ Value::Array(q.levels.clone()),
+ ]),
+ Question::Noul(q) => Value::Array(vec![
+ Value::from("noul"),
+ q.instructions.clone(),
+ q.if_false.clone(),
+ q.if_true.clone(),
+ ]),
+ };
+ Value::Array(vec![Value::from(id.as_str()), body])
+ })
+ .collect();
+ Value::Array(rows).to_string()
+ }
+}
+
impl std::ops::Index<&str> for QuestionSet {
type Output = Question;
diff --git a/laya-rust/tests/common/golden_data.rs b/laya-rust/tests/common/golden_data.rs
index 8a912af5..688f2875 100644
--- a/laya-rust/tests/common/golden_data.rs
+++ b/laya-rust/tests/common/golden_data.rs
@@ -146,6 +146,22 @@ impl CheckpointGoldenData {
.collect()
}
+ /// Cases recorded with `"kind": "batch"` that return results (`ONNXAgent.predict_batch`).
+ pub fn batch_cases(&self) -> Vec<GoldenCaseInfo> {
+ self.index()
+ .into_iter()
+ .filter(|c| c.kind.as_deref() == Some("batch") && !c.expects_error)
+ .collect()
+ }
+
+ /// Cases recorded with `"kind": "batch"` that raise a Python `TypeError`.
+ pub fn batch_error_cases(&self) -> Vec<GoldenCaseInfo> {
+ self.index()
+ .into_iter()
+ .filter(|c| c.kind.as_deref() == Some("batch") && c.expects_error)
+ .collect()
+ }
+
/// Cases recorded with `"kind": "shortlist"` (`shortlist_many_options`): a hashing-embedder
/// reduced choice question run through the ordinary predict pipeline, not a plain
/// `Agent.predict` recording, so it is kept out of [`Self::success_cases`].
diff --git a/laya-rust/tests/golden/english/case_batch_chunked.json b/laya-rust/tests/golden/english/case_batch_chunked.json
new file mode 100644
index 00000000..224ad91e
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_chunked.json
@@ -0,0 +1,466 @@
+{
+ "name": "batch_chunked",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ],
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint. Turn 140: the agent restated the refund policy and the customer repeated the complaint. Turn 141: the agent restated the refund policy and the customer repeated the complaint. Turn 142: the agent restated the refund policy and the customer repeated the complaint. Turn 143: the agent restated the refund policy and the customer repeated the complaint. Turn 144: the agent restated the refund policy and the customer repeated the complaint. Turn 145: the agent restated the refund policy and the customer repeated the complaint. Turn 146: the agent restated the refund policy and the customer repeated the complaint. Turn 147: the agent restated the refund policy and the customer repeated the complaint. Turn 148: the agent restated the refund policy and the customer repeated the complaint. Turn 149: the agent restated the refund policy and the customer repeated the complaint. Turn 150: the agent restated the refund policy and the customer repeated the complaint. Turn 151: the agent restated the refund policy and the customer repeated the complaint. Turn 152: the agent restated the refund policy and the customer repeated the complaint. Turn 153: the agent restated the refund policy and the customer repeated the complaint. Turn 154: the agent restated the refund policy and the customer repeated the complaint. Turn 155: the agent restated the refund policy and the customer repeated the complaint. Turn 156: the agent restated the refund policy and the customer repeated the complaint. Turn 157: the agent restated the refund policy and the customer repeated the complaint. Turn 158: the agent restated the refund policy and the customer repeated the complaint. Turn 159: the agent restated the refund policy and the customer repeated the complaint. Turn 160: the agent restated the refund policy and the customer repeated the complaint. Turn 161: the agent restated the refund policy and the customer repeated the complaint. Turn 162: the agent restated the refund policy and the customer repeated the complaint. Turn 163: the agent restated the refund policy and the customer repeated the complaint. Turn 164: the agent restated the refund policy and the customer repeated the complaint. Turn 165: the agent restated the refund policy and the customer repeated the complaint. Turn 166: the agent restated the refund policy and the customer repeated the complaint. Turn 167: the agent restated the refund policy and the customer repeated the complaint. Turn 168: the agent restated the refund policy and the customer repeated the complaint. Turn 169: the agent restated the refund policy and the customer repeated the complaint. Turn 170: the agent restated the refund policy and the customer repeated the complaint. Turn 171: the agent restated the refund policy and the customer repeated the complaint. Turn 172: the agent restated the refund policy and the customer repeated the complaint. Turn 173: the agent restated the refund policy and the customer repeated the complaint. Turn 174: the agent restated the refund policy and the customer repeated the complaint. Turn 175: the agent restated the refund policy and the customer repeated the complaint. Turn 176: the agent restated the refund policy and the customer repeated the complaint. Turn 177: the agent restated the refund policy and the customer repeated the complaint. Turn 178: the agent restated the refund policy and the customer repeated the complaint. Turn 179: the agent restated the refund policy and the customer repeated the complaint. Turn 180: the agent restated the refund policy and the customer repeated the complaint. Turn 181: the agent restated the refund policy and the customer repeated the complaint. Turn 182: the agent restated the refund policy and the customer repeated the complaint. Turn 183: the agent restated the refund policy and the customer repeated the complaint. Turn 184: the agent restated the refund policy and the customer repeated the complaint. Turn 185: the agent restated the refund policy and the customer repeated the complaint. Turn 186: the agent restated the refund policy and the customer repeated the complaint. Turn 187: the agent restated the refund policy and the customer repeated the complaint. Turn 188: the agent restated the refund policy and the customer repeated the complaint. Turn 189: the agent restated the refund policy and the customer repeated the complaint. Turn 190: the agent restated the refund policy and the customer repeated the complaint. Turn 191: the agent restated the refund policy and the customer repeated the complaint. Turn 192: the agent restated the refund policy and the customer repeated the complaint. Turn 193: the agent restated the refund policy and the customer repeated the complaint. Turn 194: the agent restated the refund policy and the customer repeated the complaint. Turn 195: the agent restated the refund policy and the customer repeated the complaint. Turn 196: the agent restated the refund policy and the customer repeated the complaint. Turn 197: the agent restated the refund policy and the customer repeated the complaint. Turn 198: the agent restated the refund policy and the customer repeated the complaint. Turn 199: the agent restated the refund policy and the customer repeated the complaint. Turn 200: the agent restated the refund policy and the customer repeated the complaint. Turn 201: the agent restated the refund policy and the customer repeated the complaint. Turn 202: the agent restated the refund policy and the customer repeated the complaint. Turn 203: the agent restated the refund policy and the customer repeated the complaint. Turn 204: the agent restated the refund policy and the customer repeated the complaint. Turn 205: the agent restated the refund policy and the customer repeated the complaint. Turn 206: the agent restated the refund policy and the customer repeated the complaint. Turn 207: the agent restated the refund policy and the customer repeated the complaint. Turn 208: the agent restated the refund policy and the customer repeated the complaint. Turn 209: the agent restated the refund policy and the customer repeated the complaint. Turn 210: the agent restated the refund policy and the customer repeated the complaint. Turn 211: the agent restated the refund policy and the customer repeated the complaint. Turn 212: the agent restated the refund policy and the customer repeated the complaint. Turn 213: the agent restated the refund policy and the customer repeated the complaint. Turn 214: the agent restated the refund policy and the customer repeated the complaint. Turn 215: the agent restated the refund policy and the customer repeated the complaint. Turn 216: the agent restated the refund policy and the customer repeated the complaint. Turn 217: the agent restated the refund policy and the customer repeated the complaint. Turn 218: the agent restated the refund policy and the customer repeated the complaint. Turn 219: the agent restated the refund policy and the customer repeated the complaint. Turn 220: the agent restated the refund policy and the customer repeated the complaint. Turn 221: the agent restated the refund policy and the customer repeated the complaint. Turn 222: the agent restated the refund policy and the customer repeated the complaint. Turn 223: the agent restated the refund policy and the customer repeated the complaint. Turn 224: the agent restated the refund policy and the customer repeated the complaint. Turn 225: the agent restated the refund policy and the customer repeated the complaint. Turn 226: the agent restated the refund policy and the customer repeated the complaint. Turn 227: the agent restated the refund policy and the customer repeated the complaint. Turn 228: the agent restated the refund policy and the customer repeated the complaint. Turn 229: the agent restated the refund policy and the customer repeated the complaint. Turn 230: the agent restated the refund policy and the customer repeated the complaint. Turn 231: the agent restated the refund policy and the customer repeated the complaint. Turn 232: the agent restated the refund policy and the customer repeated the complaint. Turn 233: the agent restated the refund policy and the customer repeated the complaint. Turn 234: the agent restated the refund policy and the customer repeated the complaint. Turn 235: the agent restated the refund policy and the customer repeated the complaint. Turn 236: the agent restated the refund policy and the customer repeated the complaint. Turn 237: the agent restated the refund policy and the customer repeated the complaint. Turn 238: the agent restated the refund policy and the customer repeated the complaint. Turn 239: the agent restated the refund policy and the customer repeated the complaint. Turn 240: the agent restated the refund policy and the customer repeated the complaint. Turn 241: the agent restated the refund policy and the customer repeated the complaint. Turn 242: the agent restated the refund policy and the customer repeated the complaint. Turn 243: the agent restated the refund policy and the customer repeated the complaint. Turn 244: the agent restated the refund policy and the customer repeated the complaint. Turn 245: the agent restated the refund policy and the customer repeated the complaint. Turn 246: the agent restated the refund policy and the customer repeated the complaint. Turn 247: the agent restated the refund policy and the customer repeated the complaint. Turn 248: the agent restated the refund policy and the customer repeated the complaint. Turn 249: the agent restated the refund policy and the customer repeated the complaint. Turn 250: the agent restated the refund policy and the customer repeated the complaint. Turn 251: the agent restated the refund policy and the customer repeated the complaint. Turn 252: the agent restated the refund policy and the customer repeated the complaint. Turn 253: the agent restated the refund policy and the customer repeated the complaint. Turn 254: the agent restated the refund policy and the customer repeated the complaint. Turn 255: the agent restated the refund policy and the customer repeated the complaint. Turn 256: the agent restated the refund policy and the customer repeated the complaint. Turn 257: the agent restated the refund policy and the customer repeated the complaint. Turn 258: the agent restated the refund policy and the customer repeated the complaint. Turn 259: the agent restated the refund policy and the customer repeated the complaint. Turn 260: the agent restated the refund policy and the customer repeated the complaint. Turn 261: the agent restated the refund policy and the customer repeated the complaint. Turn 262: the agent restated the refund policy and the customer repeated the complaint. Turn 263: the agent restated the refund policy and the customer repeated the complaint. Turn 264: the agent restated the refund policy and the customer repeated the complaint. Turn 265: the agent restated the refund policy and the customer repeated the complaint. Turn 266: the agent restated the refund policy and the customer repeated the complaint. Turn 267: the agent restated the refund policy and the customer repeated the complaint. Turn 268: the agent restated the refund policy and the customer repeated the complaint. Turn 269: the agent restated the refund policy and the customer repeated the complaint. Turn 270: the agent restated the refund policy and the customer repeated the complaint. Turn 271: the agent restated the refund policy and the customer repeated the complaint. Turn 272: the agent restated the refund policy and the customer repeated the complaint. Turn 273: the agent restated the refund policy and the customer repeated the complaint. Turn 274: the agent restated the refund policy and the customer repeated the complaint. Turn 275: the agent restated the refund policy and the customer repeated the complaint. Turn 276: the agent restated the refund policy and the customer repeated the complaint. Turn 277: the agent restated the refund policy and the customer repeated the complaint. Turn 278: the agent restated the refund policy and the customer repeated the complaint. Turn 279: the agent restated the refund policy and the customer repeated the complaint. Turn 280: the agent restated the refund policy and the customer repeated the complaint. Turn 281: the agent restated the refund policy and the customer repeated the complaint. Turn 282: the agent restated the refund policy and the customer repeated the complaint. Turn 283: the agent restated the refund policy and the customer repeated the complaint. Turn 284: the agent restated the refund policy and the customer repeated the complaint. Turn 285: the agent restated the refund policy and the customer repeated the complaint. Turn 286: the agent restated the refund policy and the customer repeated the complaint. Turn 287: the agent restated the refund policy and the customer repeated the complaint. Turn 288: the agent restated the refund policy and the customer repeated the complaint. Turn 289: the agent restated the refund policy and the customer repeated the complaint. Turn 290: the agent restated the refund policy and the customer repeated the complaint. Turn 291: the agent restated the refund policy and the customer repeated the complaint. Turn 292: the agent restated the refund policy and the customer repeated the complaint. Turn 293: the agent restated the refund policy and the customer repeated the complaint. Turn 294: the agent restated the refund policy and the customer repeated the complaint. Turn 295: the agent restated the refund policy and the customer repeated the complaint. Turn 296: the agent restated the refund policy and the customer repeated the complaint. Turn 297: the agent restated the refund policy and the customer repeated the complaint. Turn 298: the agent restated the refund policy and the customer repeated the complaint. Turn 299: the agent restated the refund policy and the customer repeated the complaint. Turn 300: the agent restated the refund policy and the customer repeated the complaint. Turn 301: the agent restated the refund policy and the customer repeated the complaint. Turn 302: the agent restated the refund policy and the customer repeated the complaint. Turn 303: the agent restated the refund policy and the customer repeated the complaint. Turn 304: the agent restated the refund policy and the customer repeated the complaint. Turn 305: the agent restated the refund policy and the customer repeated the complaint. Turn 306: the agent restated the refund policy and the customer repeated the complaint. Turn 307: the agent restated the refund policy and the customer repeated the complaint. Turn 308: the agent restated the refund policy and the customer repeated the complaint. Turn 309: the agent restated the refund policy and the customer repeated the complaint. Turn 310: the agent restated the refund policy and the customer repeated the complaint. Turn 311: the agent restated the refund policy and the customer repeated the complaint. Turn 312: the agent restated the refund policy and the customer repeated the complaint. Turn 313: the agent restated the refund policy and the customer repeated the complaint. Turn 314: the agent restated the refund policy and the customer repeated the complaint. Turn 315: the agent restated the refund policy and the customer repeated the complaint. Turn 316: the agent restated the refund policy and the customer repeated the complaint. Turn 317: the agent restated the refund policy and the customer repeated the complaint. Turn 318: the agent restated the refund policy and the customer repeated the complaint. Turn 319: the agent restated the refund policy and the customer repeated the complaint. Turn 320: the agent restated the refund policy and the customer repeated the complaint. Turn 321: the agent restated the refund policy and the customer repeated the complaint. Turn 322: the agent restated the refund policy and the customer repeated the complaint. Turn 323: the agent restated the refund policy and the customer repeated the complaint. Turn 324: the agent restated the refund policy and the customer repeated the complaint. Turn 325: the agent restated the refund policy and the customer repeated the complaint. Turn 326: the agent restated the refund policy and the customer repeated the complaint. Turn 327: the agent restated the refund policy and the customer repeated the complaint. Turn 328: the agent restated the refund policy and the customer repeated the complaint. Turn 329: the agent restated the refund policy and the customer repeated the complaint. Turn 330: the agent restated the refund policy and the customer repeated the complaint. Turn 331: the agent restated the refund policy and the customer repeated the complaint. Turn 332: the agent restated the refund policy and the customer repeated the complaint. Turn 333: the agent restated the refund policy and the customer repeated the complaint. Turn 334: the agent restated the refund policy and the customer repeated the complaint. Turn 335: the agent restated the refund policy and the customer repeated the complaint. Turn 336: the agent restated the refund policy and the customer repeated the complaint. Turn 337: the agent restated the refund policy and the customer repeated the complaint. Turn 338: the agent restated the refund policy and the customer repeated the complaint. Turn 339: the agent restated the refund policy and the customer repeated the complaint. Turn 340: the agent restated the refund policy and the customer repeated the complaint. Turn 341: the agent restated the refund policy and the customer repeated the complaint. Turn 342: the agent restated the refund policy and the customer repeated the complaint. Turn 343: the agent restated the refund policy and the customer repeated the complaint. Turn 344: the agent restated the refund policy and the customer repeated the complaint. Turn 345: the agent restated the refund policy and the customer repeated the complaint. Turn 346: the agent restated the refund policy and the customer repeated the complaint. Turn 347: the agent restated the refund policy and the customer repeated the complaint. Turn 348: the agent restated the refund policy and the customer repeated the complaint. Turn 349: the agent restated the refund policy and the customer repeated the complaint. Turn 350: the agent restated the refund policy and the customer repeated the complaint. Turn 351: the agent restated the refund policy and the customer repeated the complaint. Turn 352: the agent restated the refund policy and the customer repeated the complaint. Turn 353: the agent restated the refund policy and the customer repeated the complaint. Turn 354: the agent restated the refund policy and the customer repeated the complaint. Turn 355: the agent restated the refund policy and the customer repeated the complaint. Turn 356: the agent restated the refund policy and the customer repeated the complaint. Turn 357: the agent restated the refund policy and the customer repeated the complaint. Turn 358: the agent restated the refund policy and the customer repeated the complaint. Turn 359: the agent restated the refund policy and the customer repeated the complaint. Turn 360: the agent restated the refund policy and the customer repeated the complaint. Turn 361: the agent restated the refund policy and the customer repeated the complaint. Turn 362: the agent restated the refund policy and the customer repeated the complaint. Turn 363: the agent restated the refund policy and the customer repeated the complaint. Turn 364: the agent restated the refund policy and the customer repeated the complaint. Turn 365: the agent restated the refund policy and the customer repeated the complaint. Turn 366: the agent restated the refund policy and the customer repeated the complaint. Turn 367: the agent restated the refund policy and the customer repeated the complaint. Turn 368: the agent restated the refund policy and the customer repeated the complaint. Turn 369: the agent restated the refund policy and the customer repeated the complaint. Turn 370: the agent restated the refund policy and the customer repeated the complaint. Turn 371: the agent restated the refund policy and the customer repeated the complaint. Turn 372: the agent restated the refund policy and the customer repeated the complaint. Turn 373: the agent restated the refund policy and the customer repeated the complaint. Turn 374: the agent restated the refund policy and the customer repeated the complaint. Turn 375: the agent restated the refund policy and the customer repeated the complaint. Turn 376: the agent restated the refund policy and the customer repeated the complaint. Turn 377: the agent restated the refund policy and the customer repeated the complaint. Turn 378: the agent restated the refund policy and the customer repeated the complaint. Turn 379: the agent restated the refund policy and the customer repeated the complaint. Turn 380: the agent restated the refund policy and the customer repeated the complaint. Turn 381: the agent restated the refund policy and the customer repeated the complaint. Turn 382: the agent restated the refund policy and the customer repeated the complaint. Turn 383: the agent restated the refund policy and the customer repeated the complaint. Turn 384: the agent restated the refund policy and the customer repeated the complaint. Turn 385: the agent restated the refund policy and the customer repeated the complaint. Turn 386: the agent restated the refund policy and the customer repeated the complaint. Turn 387: the agent restated the refund policy and the customer repeated the complaint. Turn 388: the agent restated the refund policy and the customer repeated the complaint. Turn 389: the agent restated the refund policy and the customer repeated the complaint. Turn 390: the agent restated the refund policy and the customer repeated the complaint. Turn 391: the agent restated the refund policy and the customer repeated the complaint. Turn 392: the agent restated the refund policy and the customer repeated the complaint. Turn 393: the agent restated the refund policy and the customer repeated the complaint. Turn 394: the agent restated the refund policy and the customer repeated the complaint. Turn 395: the agent restated the refund policy and the customer repeated the complaint. Turn 396: the agent restated the refund policy and the customer repeated the complaint. Turn 397: the agent restated the refund policy and the customer repeated the complaint. Turn 398: the agent restated the refund policy and the customer repeated the complaint. Turn 399: the agent restated the refund policy and the customer repeated the complaint.",
+ "The app crashes on login since yesterday's update.",
+ "ok"
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3
+ },
+ "state_widths": [
+ 91,
+ 44,
+ 157,
+ 74,
+ 512,
+ 44,
+ 35
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8178,
+ "technical": 0.1682,
+ "other": 0.014
+ },
+ "confidence": 0.5229,
+ "answer_confidence": 0.8178,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7428,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5839,
+ "1": 0.0894,
+ "2": 0.3267
+ },
+ "confidence": 0.1849,
+ "answer_confidence": 0.5839,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0144,
+ "confidence": 0.9856,
+ "answer_confidence": 0.9856,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7468,
+ "technical": 0.1514,
+ "other": 0.1017
+ },
+ "confidence": 0.3297,
+ "answer_confidence": 0.7468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6364,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1029,
+ "1": 0.1578,
+ "2": 0.7393
+ },
+ "confidence": 0.3185,
+ "answer_confidence": 0.7393,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9238,
+ "confidence": 0.9238,
+ "answer_confidence": 0.9238,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9067,
+ "technical": 0.0472,
+ "other": 0.0461
+ },
+ "confidence": 0.6589,
+ "answer_confidence": 0.9067,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8012,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3497,
+ "1": 0.4994,
+ "2": 0.1509
+ },
+ "confidence": 0.0902,
+ "answer_confidence": 0.4994,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1497,
+ "confidence": 0.8503,
+ "answer_confidence": 0.8503,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 214,
+ "output_tokens": 0,
+ "state_tokens": 40,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8204,
+ "technical": 0.0752,
+ "other": 0.1045
+ },
+ "confidence": 0.4603,
+ "answer_confidence": 0.8204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6454,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5702,
+ "1": 0.2141,
+ "2": 0.2156
+ },
+ "confidence": 0.1069,
+ "answer_confidence": 0.5702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3196,
+ "confidence": 0.6804,
+ "answer_confidence": 0.6804,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 6804,
+ "state_tokens_dropped": 6326,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.0483,
+ "technical": 0.9057,
+ "other": 0.046
+ },
+ "confidence": 0.6563,
+ "answer_confidence": 0.9057,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4741,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.17,
+ "1": 0.186,
+ "2": 0.6441
+ },
+ "confidence": 0.1831,
+ "answer_confidence": 0.6441,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.122,
+ "confidence": 0.878,
+ "answer_confidence": 0.878,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5728,
+ "technical": 0.2419,
+ "other": 0.1853
+ },
+ "confidence": 0.1126,
+ "answer_confidence": 0.5728,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.606,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5308,
+ "1": 0.3323,
+ "2": 0.1368
+ },
+ "confidence": 0.113,
+ "answer_confidence": 0.5308,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4482,
+ "confidence": 0.5518,
+ "answer_confidence": 0.5518,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 97,
+ "output_tokens": 0,
+ "state_tokens": 1,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 157
+ ],
+ [
+ 9,
+ 512
+ ],
+ [
+ 3,
+ 35
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_duplicate_states.json b/laya-rust/tests/golden/english/case_batch_duplicate_states.json
new file mode 100644
index 00000000..acc0038e
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_duplicate_states.json
@@ -0,0 +1,211 @@
+{
+ "name": "batch_duplicate_states",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 91,
+ 91,
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 91
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_empty_both.json b/laya-rust/tests/golden/english/case_batch_empty_both.json
new file mode 100644
index 00000000..8a31ab3d
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_empty_both.json
@@ -0,0 +1,9 @@
+{
+ "name": "batch_empty_both",
+ "states": [],
+ "questions": {},
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/english/case_batch_empty_questions.json b/laya-rust/tests/golden/english/case_batch_empty_questions.json
new file mode 100644
index 00000000..ea34701f
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_empty_questions.json
@@ -0,0 +1,29 @@
+{
+ "name": "batch_empty_questions",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ ""
+ ],
+ "questions": {},
+ "options": {},
+ "state_widths": [],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ }
+ ],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/english/case_batch_empty_questions_with_options.json b/laya-rust/tests/golden/english/case_batch_empty_questions_with_options.json
new file mode 100644
index 00000000..d646aae6
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_empty_questions_with_options.json
@@ -0,0 +1,45 @@
+{
+ "name": "batch_empty_questions_with_options",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ }
+ ],
+ "questions": {},
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ }
+ ],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/english/case_batch_empty_states.json b/laya-rust/tests/golden/english/case_batch_empty_states.json
new file mode 100644
index 00000000..eabd6424
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_empty_states.json
@@ -0,0 +1,32 @@
+{
+ "name": "batch_empty_states",
+ "states": [],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/english/case_batch_empty_states_invalid_questions.json b/laya-rust/tests/golden/english/case_batch_empty_states_invalid_questions.json
new file mode 100644
index 00000000..d8c82752
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_empty_states_invalid_questions.json
@@ -0,0 +1,316 @@
+{
+ "name": "batch_empty_states_invalid_questions",
+ "states": [],
+ "questions": {
+ "too_wide": {
+ "type": "choice",
+ "instructions": "Too many options to fit the sequence.",
+ "criteria": {
+ "opt_000": "description 0",
+ "opt_001": "description 1",
+ "opt_002": "description 2",
+ "opt_003": "description 3",
+ "opt_004": "description 4",
+ "opt_005": "description 5",
+ "opt_006": "description 6",
+ "opt_007": "description 7",
+ "opt_008": "description 8",
+ "opt_009": "description 9",
+ "opt_010": "description 10",
+ "opt_011": "description 11",
+ "opt_012": "description 12",
+ "opt_013": "description 13",
+ "opt_014": "description 14",
+ "opt_015": "description 15",
+ "opt_016": "description 16",
+ "opt_017": "description 17",
+ "opt_018": "description 18",
+ "opt_019": "description 19",
+ "opt_020": "description 20",
+ "opt_021": "description 21",
+ "opt_022": "description 22",
+ "opt_023": "description 23",
+ "opt_024": "description 24",
+ "opt_025": "description 25",
+ "opt_026": "description 26",
+ "opt_027": "description 27",
+ "opt_028": "description 28",
+ "opt_029": "description 29",
+ "opt_030": "description 30",
+ "opt_031": "description 31",
+ "opt_032": "description 32",
+ "opt_033": "description 33",
+ "opt_034": "description 34",
+ "opt_035": "description 35",
+ "opt_036": "description 36",
+ "opt_037": "description 37",
+ "opt_038": "description 38",
+ "opt_039": "description 39",
+ "opt_040": "description 40",
+ "opt_041": "description 41",
+ "opt_042": "description 42",
+ "opt_043": "description 43",
+ "opt_044": "description 44",
+ "opt_045": "description 45",
+ "opt_046": "description 46",
+ "opt_047": "description 47",
+ "opt_048": "description 48",
+ "opt_049": "description 49",
+ "opt_050": "description 50",
+ "opt_051": "description 51",
+ "opt_052": "description 52",
+ "opt_053": "description 53",
+ "opt_054": "description 54",
+ "opt_055": "description 55",
+ "opt_056": "description 56",
+ "opt_057": "description 57",
+ "opt_058": "description 58",
+ "opt_059": "description 59",
+ "opt_060": "description 60",
+ "opt_061": "description 61",
+ "opt_062": "description 62",
+ "opt_063": "description 63",
+ "opt_064": "description 64",
+ "opt_065": "description 65",
+ "opt_066": "description 66",
+ "opt_067": "description 67",
+ "opt_068": "description 68",
+ "opt_069": "description 69",
+ "opt_070": "description 70",
+ "opt_071": "description 71",
+ "opt_072": "description 72",
+ "opt_073": "description 73",
+ "opt_074": "description 74",
+ "opt_075": "description 75",
+ "opt_076": "description 76",
+ "opt_077": "description 77",
+ "opt_078": "description 78",
+ "opt_079": "description 79",
+ "opt_080": "description 80",
+ "opt_081": "description 81",
+ "opt_082": "description 82",
+ "opt_083": "description 83",
+ "opt_084": "description 84",
+ "opt_085": "description 85",
+ "opt_086": "description 86",
+ "opt_087": "description 87",
+ "opt_088": "description 88",
+ "opt_089": "description 89",
+ "opt_090": "description 90",
+ "opt_091": "description 91",
+ "opt_092": "description 92",
+ "opt_093": "description 93",
+ "opt_094": "description 94",
+ "opt_095": "description 95",
+ "opt_096": "description 96",
+ "opt_097": "description 97",
+ "opt_098": "description 98",
+ "opt_099": "description 99",
+ "opt_100": "description 100",
+ "opt_101": "description 101",
+ "opt_102": "description 102",
+ "opt_103": "description 103",
+ "opt_104": "description 104",
+ "opt_105": "description 105",
+ "opt_106": "description 106",
+ "opt_107": "description 107",
+ "opt_108": "description 108",
+ "opt_109": "description 109",
+ "opt_110": "description 110",
+ "opt_111": "description 111",
+ "opt_112": "description 112",
+ "opt_113": "description 113",
+ "opt_114": "description 114",
+ "opt_115": "description 115",
+ "opt_116": "description 116",
+ "opt_117": "description 117",
+ "opt_118": "description 118",
+ "opt_119": "description 119",
+ "opt_120": "description 120",
+ "opt_121": "description 121",
+ "opt_122": "description 122",
+ "opt_123": "description 123",
+ "opt_124": "description 124",
+ "opt_125": "description 125",
+ "opt_126": "description 126",
+ "opt_127": "description 127",
+ "opt_128": "description 128",
+ "opt_129": "description 129",
+ "opt_130": "description 130",
+ "opt_131": "description 131",
+ "opt_132": "description 132",
+ "opt_133": "description 133",
+ "opt_134": "description 134",
+ "opt_135": "description 135",
+ "opt_136": "description 136",
+ "opt_137": "description 137",
+ "opt_138": "description 138",
+ "opt_139": "description 139",
+ "opt_140": "description 140",
+ "opt_141": "description 141",
+ "opt_142": "description 142",
+ "opt_143": "description 143",
+ "opt_144": "description 144",
+ "opt_145": "description 145",
+ "opt_146": "description 146",
+ "opt_147": "description 147",
+ "opt_148": "description 148",
+ "opt_149": "description 149",
+ "opt_150": "description 150",
+ "opt_151": "description 151",
+ "opt_152": "description 152",
+ "opt_153": "description 153",
+ "opt_154": "description 154",
+ "opt_155": "description 155",
+ "opt_156": "description 156",
+ "opt_157": "description 157",
+ "opt_158": "description 158",
+ "opt_159": "description 159",
+ "opt_160": "description 160",
+ "opt_161": "description 161",
+ "opt_162": "description 162",
+ "opt_163": "description 163",
+ "opt_164": "description 164",
+ "opt_165": "description 165",
+ "opt_166": "description 166",
+ "opt_167": "description 167",
+ "opt_168": "description 168",
+ "opt_169": "description 169",
+ "opt_170": "description 170",
+ "opt_171": "description 171",
+ "opt_172": "description 172",
+ "opt_173": "description 173",
+ "opt_174": "description 174",
+ "opt_175": "description 175",
+ "opt_176": "description 176",
+ "opt_177": "description 177",
+ "opt_178": "description 178",
+ "opt_179": "description 179",
+ "opt_180": "description 180",
+ "opt_181": "description 181",
+ "opt_182": "description 182",
+ "opt_183": "description 183",
+ "opt_184": "description 184",
+ "opt_185": "description 185",
+ "opt_186": "description 186",
+ "opt_187": "description 187",
+ "opt_188": "description 188",
+ "opt_189": "description 189",
+ "opt_190": "description 190",
+ "opt_191": "description 191",
+ "opt_192": "description 192",
+ "opt_193": "description 193",
+ "opt_194": "description 194",
+ "opt_195": "description 195",
+ "opt_196": "description 196",
+ "opt_197": "description 197",
+ "opt_198": "description 198",
+ "opt_199": "description 199",
+ "opt_200": "description 200",
+ "opt_201": "description 201",
+ "opt_202": "description 202",
+ "opt_203": "description 203",
+ "opt_204": "description 204",
+ "opt_205": "description 205",
+ "opt_206": "description 206",
+ "opt_207": "description 207",
+ "opt_208": "description 208",
+ "opt_209": "description 209",
+ "opt_210": "description 210",
+ "opt_211": "description 211",
+ "opt_212": "description 212",
+ "opt_213": "description 213",
+ "opt_214": "description 214",
+ "opt_215": "description 215",
+ "opt_216": "description 216",
+ "opt_217": "description 217",
+ "opt_218": "description 218",
+ "opt_219": "description 219",
+ "opt_220": "description 220",
+ "opt_221": "description 221",
+ "opt_222": "description 222",
+ "opt_223": "description 223",
+ "opt_224": "description 224",
+ "opt_225": "description 225",
+ "opt_226": "description 226",
+ "opt_227": "description 227",
+ "opt_228": "description 228",
+ "opt_229": "description 229",
+ "opt_230": "description 230",
+ "opt_231": "description 231",
+ "opt_232": "description 232",
+ "opt_233": "description 233",
+ "opt_234": "description 234",
+ "opt_235": "description 235",
+ "opt_236": "description 236",
+ "opt_237": "description 237",
+ "opt_238": "description 238",
+ "opt_239": "description 239",
+ "opt_240": "description 240",
+ "opt_241": "description 241",
+ "opt_242": "description 242",
+ "opt_243": "description 243",
+ "opt_244": "description 244",
+ "opt_245": "description 245",
+ "opt_246": "description 246",
+ "opt_247": "description 247",
+ "opt_248": "description 248",
+ "opt_249": "description 249",
+ "opt_250": "description 250",
+ "opt_251": "description 251",
+ "opt_252": "description 252",
+ "opt_253": "description 253",
+ "opt_254": "description 254",
+ "opt_255": "description 255",
+ "opt_256": "description 256",
+ "opt_257": "description 257",
+ "opt_258": "description 258",
+ "opt_259": "description 259",
+ "opt_260": "description 260",
+ "opt_261": "description 261",
+ "opt_262": "description 262",
+ "opt_263": "description 263",
+ "opt_264": "description 264",
+ "opt_265": "description 265",
+ "opt_266": "description 266",
+ "opt_267": "description 267",
+ "opt_268": "description 268",
+ "opt_269": "description 269",
+ "opt_270": "description 270",
+ "opt_271": "description 271",
+ "opt_272": "description 272",
+ "opt_273": "description 273",
+ "opt_274": "description 274",
+ "opt_275": "description 275",
+ "opt_276": "description 276",
+ "opt_277": "description 277",
+ "opt_278": "description 278",
+ "opt_279": "description 279",
+ "opt_280": "description 280",
+ "opt_281": "description 281",
+ "opt_282": "description 282",
+ "opt_283": "description 283",
+ "opt_284": "description 284",
+ "opt_285": "description 285",
+ "opt_286": "description 286",
+ "opt_287": "description 287",
+ "opt_288": "description 288",
+ "opt_289": "description 289",
+ "opt_290": "description 290",
+ "opt_291": "description 291",
+ "opt_292": "description 292",
+ "opt_293": "description 293",
+ "opt_294": "description 294",
+ "opt_295": "description 295",
+ "opt_296": "description 296",
+ "opt_297": "description 297",
+ "opt_298": "description 298",
+ "opt_299": "description 299"
+ }
+ }
+ },
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/english/case_batch_mixed.json b/laya-rust/tests/golden/english/case_batch_mixed.json
new file mode 100644
index 00000000..426098fc
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_mixed.json
@@ -0,0 +1,515 @@
+{
+ "name": "batch_mixed",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7826,
+ "technical": 0.1583,
+ "other": 0.0591
+ },
+ "confidence": 0.4076,
+ "answer_confidence": 0.7826,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7512,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.1611,
+ "2": 0.795
+ },
+ "confidence": 0.4415,
+ "answer_confidence": 0.795,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9312,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7843,
+ "technical": 0.1543,
+ "other": 0.0614
+ },
+ "confidence": 0.4082,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0568,
+ "1": 0.179,
+ "2": 0.7642
+ },
+ "confidence": 0.3843,
+ "answer_confidence": 0.7642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8762,
+ "confidence": 0.8762,
+ "answer_confidence": 0.8762,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5445,
+ "technical": 0.2427,
+ "other": 0.2129
+ },
+ "confidence": 0.0861,
+ "answer_confidence": 0.5445,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6874,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4819,
+ "1": 0.3488,
+ "2": 0.1693
+ },
+ "confidence": 0.0717,
+ "answer_confidence": 0.4819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3894,
+ "confidence": 0.6106,
+ "answer_confidence": 0.6106,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2918,
+ "technical": 0.398,
+ "other": 0.3102
+ },
+ "confidence": 0.0086,
+ "answer_confidence": 0.398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5403,
+ "1": 0.1276,
+ "2": 0.3321
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5403,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0548,
+ "confidence": 0.9452,
+ "answer_confidence": 0.9452,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.573,
+ "technical": 0.1873,
+ "other": 0.2397
+ },
+ "confidence": 0.1124,
+ "answer_confidence": 0.573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8166,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0486,
+ "1": 0.0862,
+ "2": 0.8652
+ },
+ "confidence": 0.5599,
+ "answer_confidence": 0.8652,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8598,
+ "confidence": 0.8598,
+ "answer_confidence": 0.8598,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5426,
+ "technical": 0.3484,
+ "other": 0.109
+ },
+ "confidence": 0.1438,
+ "answer_confidence": 0.5426,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.516,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6281,
+ "1": 0.2278,
+ "2": 0.1441
+ },
+ "confidence": 0.1733,
+ "answer_confidence": 0.6281,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3136,
+ "confidence": 0.6864,
+ "answer_confidence": 0.6864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_none_state.json b/laya-rust/tests/golden/english/case_batch_none_state.json
new file mode 100644
index 00000000..560be81a
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_none_state.json
@@ -0,0 +1,34 @@
+{
+ "name": "batch_none_state",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ null
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "error_type": "TypeError",
+ "error_message": "state must not be None; pass a string, dict, or list"
+}
diff --git a/laya-rust/tests/golden/english/case_batch_one_state.json b/laya-rust/tests/golden/english/case_batch_one_state.json
new file mode 100644
index 00000000..41d01db1
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_one_state.json
@@ -0,0 +1,97 @@
+{
+ "name": "batch_one_state",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 91
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_options_exceed_head.json b/laya-rust/tests/golden/english/case_batch_options_exceed_head.json
new file mode 100644
index 00000000..367b9f55
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_options_exceed_head.json
@@ -0,0 +1,318 @@
+{
+ "name": "batch_options_exceed_head",
+ "states": [
+ "short",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "too_wide": {
+ "type": "choice",
+ "instructions": "Too many options to fit the sequence.",
+ "criteria": {
+ "opt_000": "description 0",
+ "opt_001": "description 1",
+ "opt_002": "description 2",
+ "opt_003": "description 3",
+ "opt_004": "description 4",
+ "opt_005": "description 5",
+ "opt_006": "description 6",
+ "opt_007": "description 7",
+ "opt_008": "description 8",
+ "opt_009": "description 9",
+ "opt_010": "description 10",
+ "opt_011": "description 11",
+ "opt_012": "description 12",
+ "opt_013": "description 13",
+ "opt_014": "description 14",
+ "opt_015": "description 15",
+ "opt_016": "description 16",
+ "opt_017": "description 17",
+ "opt_018": "description 18",
+ "opt_019": "description 19",
+ "opt_020": "description 20",
+ "opt_021": "description 21",
+ "opt_022": "description 22",
+ "opt_023": "description 23",
+ "opt_024": "description 24",
+ "opt_025": "description 25",
+ "opt_026": "description 26",
+ "opt_027": "description 27",
+ "opt_028": "description 28",
+ "opt_029": "description 29",
+ "opt_030": "description 30",
+ "opt_031": "description 31",
+ "opt_032": "description 32",
+ "opt_033": "description 33",
+ "opt_034": "description 34",
+ "opt_035": "description 35",
+ "opt_036": "description 36",
+ "opt_037": "description 37",
+ "opt_038": "description 38",
+ "opt_039": "description 39",
+ "opt_040": "description 40",
+ "opt_041": "description 41",
+ "opt_042": "description 42",
+ "opt_043": "description 43",
+ "opt_044": "description 44",
+ "opt_045": "description 45",
+ "opt_046": "description 46",
+ "opt_047": "description 47",
+ "opt_048": "description 48",
+ "opt_049": "description 49",
+ "opt_050": "description 50",
+ "opt_051": "description 51",
+ "opt_052": "description 52",
+ "opt_053": "description 53",
+ "opt_054": "description 54",
+ "opt_055": "description 55",
+ "opt_056": "description 56",
+ "opt_057": "description 57",
+ "opt_058": "description 58",
+ "opt_059": "description 59",
+ "opt_060": "description 60",
+ "opt_061": "description 61",
+ "opt_062": "description 62",
+ "opt_063": "description 63",
+ "opt_064": "description 64",
+ "opt_065": "description 65",
+ "opt_066": "description 66",
+ "opt_067": "description 67",
+ "opt_068": "description 68",
+ "opt_069": "description 69",
+ "opt_070": "description 70",
+ "opt_071": "description 71",
+ "opt_072": "description 72",
+ "opt_073": "description 73",
+ "opt_074": "description 74",
+ "opt_075": "description 75",
+ "opt_076": "description 76",
+ "opt_077": "description 77",
+ "opt_078": "description 78",
+ "opt_079": "description 79",
+ "opt_080": "description 80",
+ "opt_081": "description 81",
+ "opt_082": "description 82",
+ "opt_083": "description 83",
+ "opt_084": "description 84",
+ "opt_085": "description 85",
+ "opt_086": "description 86",
+ "opt_087": "description 87",
+ "opt_088": "description 88",
+ "opt_089": "description 89",
+ "opt_090": "description 90",
+ "opt_091": "description 91",
+ "opt_092": "description 92",
+ "opt_093": "description 93",
+ "opt_094": "description 94",
+ "opt_095": "description 95",
+ "opt_096": "description 96",
+ "opt_097": "description 97",
+ "opt_098": "description 98",
+ "opt_099": "description 99",
+ "opt_100": "description 100",
+ "opt_101": "description 101",
+ "opt_102": "description 102",
+ "opt_103": "description 103",
+ "opt_104": "description 104",
+ "opt_105": "description 105",
+ "opt_106": "description 106",
+ "opt_107": "description 107",
+ "opt_108": "description 108",
+ "opt_109": "description 109",
+ "opt_110": "description 110",
+ "opt_111": "description 111",
+ "opt_112": "description 112",
+ "opt_113": "description 113",
+ "opt_114": "description 114",
+ "opt_115": "description 115",
+ "opt_116": "description 116",
+ "opt_117": "description 117",
+ "opt_118": "description 118",
+ "opt_119": "description 119",
+ "opt_120": "description 120",
+ "opt_121": "description 121",
+ "opt_122": "description 122",
+ "opt_123": "description 123",
+ "opt_124": "description 124",
+ "opt_125": "description 125",
+ "opt_126": "description 126",
+ "opt_127": "description 127",
+ "opt_128": "description 128",
+ "opt_129": "description 129",
+ "opt_130": "description 130",
+ "opt_131": "description 131",
+ "opt_132": "description 132",
+ "opt_133": "description 133",
+ "opt_134": "description 134",
+ "opt_135": "description 135",
+ "opt_136": "description 136",
+ "opt_137": "description 137",
+ "opt_138": "description 138",
+ "opt_139": "description 139",
+ "opt_140": "description 140",
+ "opt_141": "description 141",
+ "opt_142": "description 142",
+ "opt_143": "description 143",
+ "opt_144": "description 144",
+ "opt_145": "description 145",
+ "opt_146": "description 146",
+ "opt_147": "description 147",
+ "opt_148": "description 148",
+ "opt_149": "description 149",
+ "opt_150": "description 150",
+ "opt_151": "description 151",
+ "opt_152": "description 152",
+ "opt_153": "description 153",
+ "opt_154": "description 154",
+ "opt_155": "description 155",
+ "opt_156": "description 156",
+ "opt_157": "description 157",
+ "opt_158": "description 158",
+ "opt_159": "description 159",
+ "opt_160": "description 160",
+ "opt_161": "description 161",
+ "opt_162": "description 162",
+ "opt_163": "description 163",
+ "opt_164": "description 164",
+ "opt_165": "description 165",
+ "opt_166": "description 166",
+ "opt_167": "description 167",
+ "opt_168": "description 168",
+ "opt_169": "description 169",
+ "opt_170": "description 170",
+ "opt_171": "description 171",
+ "opt_172": "description 172",
+ "opt_173": "description 173",
+ "opt_174": "description 174",
+ "opt_175": "description 175",
+ "opt_176": "description 176",
+ "opt_177": "description 177",
+ "opt_178": "description 178",
+ "opt_179": "description 179",
+ "opt_180": "description 180",
+ "opt_181": "description 181",
+ "opt_182": "description 182",
+ "opt_183": "description 183",
+ "opt_184": "description 184",
+ "opt_185": "description 185",
+ "opt_186": "description 186",
+ "opt_187": "description 187",
+ "opt_188": "description 188",
+ "opt_189": "description 189",
+ "opt_190": "description 190",
+ "opt_191": "description 191",
+ "opt_192": "description 192",
+ "opt_193": "description 193",
+ "opt_194": "description 194",
+ "opt_195": "description 195",
+ "opt_196": "description 196",
+ "opt_197": "description 197",
+ "opt_198": "description 198",
+ "opt_199": "description 199",
+ "opt_200": "description 200",
+ "opt_201": "description 201",
+ "opt_202": "description 202",
+ "opt_203": "description 203",
+ "opt_204": "description 204",
+ "opt_205": "description 205",
+ "opt_206": "description 206",
+ "opt_207": "description 207",
+ "opt_208": "description 208",
+ "opt_209": "description 209",
+ "opt_210": "description 210",
+ "opt_211": "description 211",
+ "opt_212": "description 212",
+ "opt_213": "description 213",
+ "opt_214": "description 214",
+ "opt_215": "description 215",
+ "opt_216": "description 216",
+ "opt_217": "description 217",
+ "opt_218": "description 218",
+ "opt_219": "description 219",
+ "opt_220": "description 220",
+ "opt_221": "description 221",
+ "opt_222": "description 222",
+ "opt_223": "description 223",
+ "opt_224": "description 224",
+ "opt_225": "description 225",
+ "opt_226": "description 226",
+ "opt_227": "description 227",
+ "opt_228": "description 228",
+ "opt_229": "description 229",
+ "opt_230": "description 230",
+ "opt_231": "description 231",
+ "opt_232": "description 232",
+ "opt_233": "description 233",
+ "opt_234": "description 234",
+ "opt_235": "description 235",
+ "opt_236": "description 236",
+ "opt_237": "description 237",
+ "opt_238": "description 238",
+ "opt_239": "description 239",
+ "opt_240": "description 240",
+ "opt_241": "description 241",
+ "opt_242": "description 242",
+ "opt_243": "description 243",
+ "opt_244": "description 244",
+ "opt_245": "description 245",
+ "opt_246": "description 246",
+ "opt_247": "description 247",
+ "opt_248": "description 248",
+ "opt_249": "description 249",
+ "opt_250": "description 250",
+ "opt_251": "description 251",
+ "opt_252": "description 252",
+ "opt_253": "description 253",
+ "opt_254": "description 254",
+ "opt_255": "description 255",
+ "opt_256": "description 256",
+ "opt_257": "description 257",
+ "opt_258": "description 258",
+ "opt_259": "description 259",
+ "opt_260": "description 260",
+ "opt_261": "description 261",
+ "opt_262": "description 262",
+ "opt_263": "description 263",
+ "opt_264": "description 264",
+ "opt_265": "description 265",
+ "opt_266": "description 266",
+ "opt_267": "description 267",
+ "opt_268": "description 268",
+ "opt_269": "description 269",
+ "opt_270": "description 270",
+ "opt_271": "description 271",
+ "opt_272": "description 272",
+ "opt_273": "description 273",
+ "opt_274": "description 274",
+ "opt_275": "description 275",
+ "opt_276": "description 276",
+ "opt_277": "description 277",
+ "opt_278": "description 278",
+ "opt_279": "description 279",
+ "opt_280": "description 280",
+ "opt_281": "description 281",
+ "opt_282": "description 282",
+ "opt_283": "description 283",
+ "opt_284": "description 284",
+ "opt_285": "description 285",
+ "opt_286": "description 286",
+ "opt_287": "description 287",
+ "opt_288": "description 288",
+ "opt_289": "description 289",
+ "opt_290": "description 290",
+ "opt_291": "description 291",
+ "opt_292": "description 292",
+ "opt_293": "description 293",
+ "opt_294": "description 294",
+ "opt_295": "description 295",
+ "opt_296": "description 296",
+ "opt_297": "description 297",
+ "opt_298": "description 298",
+ "opt_299": "description 299"
+ }
+ }
+ },
+ "options": {},
+ "error_type": "ValueError",
+ "error_message": "question 'too_wide': only 126 of its 300 option markers fit in max_len=512 with head_max_len=192 spent on the question; lower head_max_len, raise max_len, or use fewer options"
+}
diff --git a/laya-rust/tests/golden/english/case_batch_single_question.json b/laya-rust/tests/golden/english/case_batch_single_question.json
new file mode 100644
index 00000000..d935f4ee
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_single_question.json
@@ -0,0 +1,137 @@
+{
+ "name": "batch_single_question",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ]
+ ],
+ "questions": {
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 91,
+ 44,
+ 157,
+ 74
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 91,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0144,
+ "confidence": 0.9856,
+ "answer_confidence": 0.9856,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 44,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9238,
+ "confidence": 0.9238,
+ "answer_confidence": 0.9238,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 157,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1497,
+ "confidence": 0.8503,
+ "answer_confidence": 0.8503,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 74,
+ "output_tokens": 0,
+ "state_tokens": 40,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 2,
+ 91
+ ],
+ [
+ 2,
+ 157
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_size_1.json b/laya-rust/tests/golden/english/case_batch_size_1.json
new file mode 100644
index 00000000..cfe47670
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_size_1.json
@@ -0,0 +1,545 @@
+{
+ "name": "batch_size_1",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 1
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7826,
+ "technical": 0.1583,
+ "other": 0.0591
+ },
+ "confidence": 0.4076,
+ "answer_confidence": 0.7826,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7512,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.1611,
+ "2": 0.795
+ },
+ "confidence": 0.4415,
+ "answer_confidence": 0.795,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9312,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7843,
+ "technical": 0.1543,
+ "other": 0.0614
+ },
+ "confidence": 0.4082,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0568,
+ "1": 0.179,
+ "2": 0.7642
+ },
+ "confidence": 0.3843,
+ "answer_confidence": 0.7642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8762,
+ "confidence": 0.8762,
+ "answer_confidence": 0.8762,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5445,
+ "technical": 0.2427,
+ "other": 0.2129
+ },
+ "confidence": 0.0861,
+ "answer_confidence": 0.5445,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6874,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4819,
+ "1": 0.3488,
+ "2": 0.1693
+ },
+ "confidence": 0.0717,
+ "answer_confidence": 0.4819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3894,
+ "confidence": 0.6106,
+ "answer_confidence": 0.6106,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2918,
+ "technical": 0.398,
+ "other": 0.3102
+ },
+ "confidence": 0.0086,
+ "answer_confidence": 0.398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5403,
+ "1": 0.1276,
+ "2": 0.3321
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5403,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0548,
+ "confidence": 0.9452,
+ "answer_confidence": 0.9452,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.573,
+ "technical": 0.1873,
+ "other": 0.2397
+ },
+ "confidence": 0.1124,
+ "answer_confidence": 0.573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8166,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0486,
+ "1": 0.0862,
+ "2": 0.8652
+ },
+ "confidence": 0.5599,
+ "answer_confidence": 0.8652,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8598,
+ "confidence": 0.8598,
+ "answer_confidence": 0.8598,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5426,
+ "technical": 0.3484,
+ "other": 0.109
+ },
+ "confidence": 0.1438,
+ "answer_confidence": 0.5426,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.516,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6281,
+ "1": 0.2278,
+ "2": 0.1441
+ },
+ "confidence": 0.1733,
+ "answer_confidence": 0.6281,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3136,
+ "confidence": 0.6864,
+ "answer_confidence": 0.6864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 91
+ ],
+ [
+ 3,
+ 110
+ ],
+ [
+ 3,
+ 144
+ ],
+ [
+ 3,
+ 34
+ ],
+ [
+ 3,
+ 163
+ ],
+ [
+ 3,
+ 42
+ ],
+ [
+ 3,
+ 98
+ ],
+ [
+ 3,
+ 56
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_size_3.json b/laya-rust/tests/golden/english/case_batch_size_3.json
new file mode 100644
index 00000000..91881f04
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_size_3.json
@@ -0,0 +1,525 @@
+{
+ "name": "batch_size_3",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7826,
+ "technical": 0.1583,
+ "other": 0.0591
+ },
+ "confidence": 0.4076,
+ "answer_confidence": 0.7826,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7512,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.1611,
+ "2": 0.795
+ },
+ "confidence": 0.4415,
+ "answer_confidence": 0.795,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9312,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7843,
+ "technical": 0.1543,
+ "other": 0.0614
+ },
+ "confidence": 0.4082,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0568,
+ "1": 0.179,
+ "2": 0.7642
+ },
+ "confidence": 0.3843,
+ "answer_confidence": 0.7642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8762,
+ "confidence": 0.8762,
+ "answer_confidence": 0.8762,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5445,
+ "technical": 0.2427,
+ "other": 0.2129
+ },
+ "confidence": 0.0861,
+ "answer_confidence": 0.5445,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6874,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4819,
+ "1": 0.3488,
+ "2": 0.1693
+ },
+ "confidence": 0.0717,
+ "answer_confidence": 0.4819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3894,
+ "confidence": 0.6106,
+ "answer_confidence": 0.6106,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2918,
+ "technical": 0.398,
+ "other": 0.3102
+ },
+ "confidence": 0.0086,
+ "answer_confidence": 0.398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5403,
+ "1": 0.1276,
+ "2": 0.3321
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5403,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0548,
+ "confidence": 0.9452,
+ "answer_confidence": 0.9452,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.573,
+ "technical": 0.1873,
+ "other": 0.2397
+ },
+ "confidence": 0.1124,
+ "answer_confidence": 0.573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8166,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0486,
+ "1": 0.0862,
+ "2": 0.8652
+ },
+ "confidence": 0.5599,
+ "answer_confidence": 0.8652,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8598,
+ "confidence": 0.8598,
+ "answer_confidence": 0.8598,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5426,
+ "technical": 0.3484,
+ "other": 0.109
+ },
+ "confidence": 0.1438,
+ "answer_confidence": 0.5426,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.516,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6281,
+ "1": 0.2278,
+ "2": 0.1441
+ },
+ "confidence": 0.1733,
+ "answer_confidence": 0.6281,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3136,
+ "confidence": 0.6864,
+ "answer_confidence": 0.6864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 144
+ ],
+ [
+ 9,
+ 163
+ ],
+ [
+ 6,
+ 98
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_size_negative.json b/laya-rust/tests/golden/english/case_batch_size_negative.json
new file mode 100644
index 00000000..0a19f61d
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_size_negative.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_negative",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": -2
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7826,
+ "technical": 0.1583,
+ "other": 0.0591
+ },
+ "confidence": 0.4076,
+ "answer_confidence": 0.7826,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7512,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.1611,
+ "2": 0.795
+ },
+ "confidence": 0.4415,
+ "answer_confidence": 0.795,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9312,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7843,
+ "technical": 0.1543,
+ "other": 0.0614
+ },
+ "confidence": 0.4082,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0568,
+ "1": 0.179,
+ "2": 0.7642
+ },
+ "confidence": 0.3843,
+ "answer_confidence": 0.7642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8762,
+ "confidence": 0.8762,
+ "answer_confidence": 0.8762,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5445,
+ "technical": 0.2427,
+ "other": 0.2129
+ },
+ "confidence": 0.0861,
+ "answer_confidence": 0.5445,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6874,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4819,
+ "1": 0.3488,
+ "2": 0.1693
+ },
+ "confidence": 0.0717,
+ "answer_confidence": 0.4819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3894,
+ "confidence": 0.6106,
+ "answer_confidence": 0.6106,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2918,
+ "technical": 0.398,
+ "other": 0.3102
+ },
+ "confidence": 0.0086,
+ "answer_confidence": 0.398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5403,
+ "1": 0.1276,
+ "2": 0.3321
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5403,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0548,
+ "confidence": 0.9452,
+ "answer_confidence": 0.9452,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.573,
+ "technical": 0.1873,
+ "other": 0.2397
+ },
+ "confidence": 0.1124,
+ "answer_confidence": 0.573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8166,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0486,
+ "1": 0.0862,
+ "2": 0.8652
+ },
+ "confidence": 0.5599,
+ "answer_confidence": 0.8652,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8598,
+ "confidence": 0.8598,
+ "answer_confidence": 0.8598,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5426,
+ "technical": 0.3484,
+ "other": 0.109
+ },
+ "confidence": 0.1438,
+ "answer_confidence": 0.5426,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.516,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6281,
+ "1": 0.2278,
+ "2": 0.1441
+ },
+ "confidence": 0.1733,
+ "answer_confidence": 0.6281,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3136,
+ "confidence": 0.6864,
+ "answer_confidence": 0.6864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_size_over_len.json b/laya-rust/tests/golden/english/case_batch_size_over_len.json
new file mode 100644
index 00000000..a65fbefb
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_size_over_len.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_over_len",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 100
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7826,
+ "technical": 0.1583,
+ "other": 0.0591
+ },
+ "confidence": 0.4076,
+ "answer_confidence": 0.7826,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7512,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.1611,
+ "2": 0.795
+ },
+ "confidence": 0.4415,
+ "answer_confidence": 0.795,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9312,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7843,
+ "technical": 0.1543,
+ "other": 0.0614
+ },
+ "confidence": 0.4082,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0568,
+ "1": 0.179,
+ "2": 0.7642
+ },
+ "confidence": 0.3843,
+ "answer_confidence": 0.7642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8762,
+ "confidence": 0.8762,
+ "answer_confidence": 0.8762,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5445,
+ "technical": 0.2427,
+ "other": 0.2129
+ },
+ "confidence": 0.0861,
+ "answer_confidence": 0.5445,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6874,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4819,
+ "1": 0.3488,
+ "2": 0.1693
+ },
+ "confidence": 0.0717,
+ "answer_confidence": 0.4819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3894,
+ "confidence": 0.6106,
+ "answer_confidence": 0.6106,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2918,
+ "technical": 0.398,
+ "other": 0.3102
+ },
+ "confidence": 0.0086,
+ "answer_confidence": 0.398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5403,
+ "1": 0.1276,
+ "2": 0.3321
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5403,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0548,
+ "confidence": 0.9452,
+ "answer_confidence": 0.9452,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.573,
+ "technical": 0.1873,
+ "other": 0.2397
+ },
+ "confidence": 0.1124,
+ "answer_confidence": 0.573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8166,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0486,
+ "1": 0.0862,
+ "2": 0.8652
+ },
+ "confidence": 0.5599,
+ "answer_confidence": 0.8652,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8598,
+ "confidence": 0.8598,
+ "answer_confidence": 0.8598,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5426,
+ "technical": 0.3484,
+ "other": 0.109
+ },
+ "confidence": 0.1438,
+ "answer_confidence": 0.5426,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.516,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6281,
+ "1": 0.2278,
+ "2": 0.1441
+ },
+ "confidence": 0.1733,
+ "answer_confidence": 0.6281,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3136,
+ "confidence": 0.6864,
+ "answer_confidence": 0.6864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_size_zero.json b/laya-rust/tests/golden/english/case_batch_size_zero.json
new file mode 100644
index 00000000..f50beb52
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_size_zero.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_zero",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 0
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7826,
+ "technical": 0.1583,
+ "other": 0.0591
+ },
+ "confidence": 0.4076,
+ "answer_confidence": 0.7826,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7512,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.1611,
+ "2": 0.795
+ },
+ "confidence": 0.4415,
+ "answer_confidence": 0.795,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9312,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7843,
+ "technical": 0.1543,
+ "other": 0.0614
+ },
+ "confidence": 0.4082,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0568,
+ "1": 0.179,
+ "2": 0.7642
+ },
+ "confidence": 0.3843,
+ "answer_confidence": 0.7642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8762,
+ "confidence": 0.8762,
+ "answer_confidence": 0.8762,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5445,
+ "technical": 0.2427,
+ "other": 0.2129
+ },
+ "confidence": 0.0861,
+ "answer_confidence": 0.5445,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6874,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4819,
+ "1": 0.3488,
+ "2": 0.1693
+ },
+ "confidence": 0.0717,
+ "answer_confidence": 0.4819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3894,
+ "confidence": 0.6106,
+ "answer_confidence": 0.6106,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2918,
+ "technical": 0.398,
+ "other": 0.3102
+ },
+ "confidence": 0.0086,
+ "answer_confidence": 0.398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5403,
+ "1": 0.1276,
+ "2": 0.3321
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5403,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0548,
+ "confidence": 0.9452,
+ "answer_confidence": 0.9452,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.573,
+ "technical": 0.1873,
+ "other": 0.2397
+ },
+ "confidence": 0.1124,
+ "answer_confidence": 0.573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8166,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0486,
+ "1": 0.0862,
+ "2": 0.8652
+ },
+ "confidence": 0.5599,
+ "answer_confidence": 0.8652,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8598,
+ "confidence": 0.8598,
+ "answer_confidence": 0.8598,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5426,
+ "technical": 0.3484,
+ "other": 0.109
+ },
+ "confidence": 0.1438,
+ "answer_confidence": 0.5426,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.516,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6281,
+ "1": 0.2278,
+ "2": 0.1441
+ },
+ "confidence": 0.1733,
+ "answer_confidence": 0.6281,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3136,
+ "confidence": 0.6864,
+ "answer_confidence": 0.6864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_no_batch_size.json b/laya-rust/tests/golden/english/case_batch_sort_no_batch_size.json
new file mode 100644
index 00000000..1ec59464
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_no_batch_size.json
@@ -0,0 +1,1182 @@
+{
+ "name": "batch_sort_no_batch_size",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_off.json b/laya-rust/tests/golden/english/case_batch_sort_off.json
new file mode 100644
index 00000000..ff680065
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_off.json
@@ -0,0 +1,1218 @@
+{
+ "name": "batch_sort_off",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 199
+ ],
+ [
+ 6,
+ 145
+ ],
+ [
+ 6,
+ 175
+ ],
+ [
+ 6,
+ 157
+ ],
+ [
+ 6,
+ 187
+ ],
+ [
+ 6,
+ 163
+ ],
+ [
+ 6,
+ 205
+ ],
+ [
+ 6,
+ 169
+ ],
+ [
+ 6,
+ 193
+ ],
+ [
+ 6,
+ 181
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_on.json b/laya-rust/tests/golden/english/case_batch_sort_on.json
new file mode 100644
index 00000000..a9d75231
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_on.json
@@ -0,0 +1,1219 @@
+{
+ "name": "batch_sort_on",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 97
+ ],
+ [
+ 6,
+ 109
+ ],
+ [
+ 6,
+ 121
+ ],
+ [
+ 6,
+ 139
+ ],
+ [
+ 6,
+ 157
+ ],
+ [
+ 6,
+ 169
+ ],
+ [
+ 6,
+ 187
+ ],
+ [
+ 6,
+ 205
+ ],
+ [
+ 6,
+ 151
+ ],
+ [
+ 6,
+ 193
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_partial_window.json b/laya-rust/tests/golden/english/case_batch_sort_partial_window.json
new file mode 100644
index 00000000..ff70b938
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_partial_window.json
@@ -0,0 +1,218 @@
+{
+ "name": "batch_sort_partial_window",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 145
+ ],
+ [
+ 3,
+ 199
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_size_1.json b/laya-rust/tests/golden/english/case_batch_sort_size_1.json
new file mode 100644
index 00000000..93454a9c
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_size_1.json
@@ -0,0 +1,1259 @@
+{
+ "name": "batch_sort_size_1",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 1,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 199
+ ],
+ [
+ 3,
+ 103
+ ],
+ [
+ 3,
+ 145
+ ],
+ [
+ 3,
+ 91
+ ],
+ [
+ 3,
+ 175
+ ],
+ [
+ 3,
+ 121
+ ],
+ [
+ 3,
+ 157
+ ],
+ [
+ 3,
+ 97
+ ],
+ [
+ 3,
+ 187
+ ],
+ [
+ 3,
+ 133
+ ],
+ [
+ 3,
+ 109
+ ],
+ [
+ 3,
+ 163
+ ],
+ [
+ 3,
+ 205
+ ],
+ [
+ 3,
+ 115
+ ],
+ [
+ 3,
+ 139
+ ],
+ [
+ 3,
+ 169
+ ],
+ [
+ 3,
+ 127
+ ],
+ [
+ 3,
+ 193
+ ],
+ [
+ 3,
+ 151
+ ],
+ [
+ 3,
+ 181
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_size_3.json b/laya-rust/tests/golden/english/case_batch_sort_size_3.json
new file mode 100644
index 00000000..91bc90ee
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_size_3.json
@@ -0,0 +1,1207 @@
+{
+ "name": "batch_sort_size_3",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 103
+ ],
+ [
+ 9,
+ 121
+ ],
+ [
+ 9,
+ 139
+ ],
+ [
+ 9,
+ 157
+ ],
+ [
+ 9,
+ 175
+ ],
+ [
+ 9,
+ 193
+ ],
+ [
+ 6,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_size_eq_len.json b/laya-rust/tests/golden/english/case_batch_sort_size_eq_len.json
new file mode 100644
index 00000000..c77c742c
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_size_eq_len.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_eq_len",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 20,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_size_over_len.json b/laya-rust/tests/golden/english/case_batch_sort_size_over_len.json
new file mode 100644
index 00000000..6e2ebbc6
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_size_over_len.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_over_len",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 25,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sort_size_zero.json b/laya-rust/tests/golden/english/case_batch_sort_size_zero.json
new file mode 100644
index 00000000..33182c8d
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sort_size_zero.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_zero",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 0,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.896,
+ "technical": 0.0688,
+ "other": 0.0352
+ },
+ "confidence": 0.6355,
+ "answer_confidence": 0.896,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7317,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0621,
+ "1": 0.1442,
+ "2": 0.7937
+ },
+ "confidence": 0.4219,
+ "answer_confidence": 0.7937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8605,
+ "technical": 0.0968,
+ "other": 0.0428
+ },
+ "confidence": 0.5539,
+ "answer_confidence": 0.8605,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7395,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0594,
+ "1": 0.1416,
+ "2": 0.7989
+ },
+ "confidence": 0.432,
+ "answer_confidence": 0.7989,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9368,
+ "confidence": 0.9368,
+ "answer_confidence": 0.9368,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9048,
+ "technical": 0.064,
+ "other": 0.0312
+ },
+ "confidence": 0.6591,
+ "answer_confidence": 0.9048,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7761,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0368,
+ "1": 0.1503,
+ "2": 0.8129
+ },
+ "confidence": 0.4768,
+ "answer_confidence": 0.8129,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9441,
+ "confidence": 0.9441,
+ "answer_confidence": 0.9441,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8943,
+ "technical": 0.071,
+ "other": 0.0347
+ },
+ "confidence": 0.632,
+ "answer_confidence": 0.8943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7077,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0712,
+ "1": 0.1499,
+ "2": 0.7789
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7789,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.973,
+ "confidence": 0.973,
+ "answer_confidence": 0.973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8917,
+ "technical": 0.0738,
+ "other": 0.0345
+ },
+ "confidence": 0.6261,
+ "answer_confidence": 0.8917,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0487,
+ "1": 0.1333,
+ "2": 0.818
+ },
+ "confidence": 0.4719,
+ "answer_confidence": 0.818,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9284,
+ "confidence": 0.9284,
+ "answer_confidence": 0.9284,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8991,
+ "technical": 0.0687,
+ "other": 0.0322
+ },
+ "confidence": 0.6449,
+ "answer_confidence": 0.8991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7492,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0463,
+ "1": 0.1583,
+ "2": 0.7955
+ },
+ "confidence": 0.4393,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9494,
+ "confidence": 0.9494,
+ "answer_confidence": 0.9494,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8759,
+ "technical": 0.0843,
+ "other": 0.0398
+ },
+ "confidence": 0.5877,
+ "answer_confidence": 0.8759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6595,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0763,
+ "1": 0.1879,
+ "2": 0.7358
+ },
+ "confidence": 0.3298,
+ "answer_confidence": 0.7358,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9291,
+ "confidence": 0.9291,
+ "answer_confidence": 0.9291,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9152,
+ "technical": 0.0558,
+ "other": 0.0289
+ },
+ "confidence": 0.6862,
+ "answer_confidence": 0.9152,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7942,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0439,
+ "1": 0.118,
+ "2": 0.8381
+ },
+ "confidence": 0.5108,
+ "answer_confidence": 0.8381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9885,
+ "confidence": 0.9885,
+ "answer_confidence": 0.9885,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9167,
+ "technical": 0.0549,
+ "other": 0.0284
+ },
+ "confidence": 0.6903,
+ "answer_confidence": 0.9167,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.815,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0345,
+ "1": 0.116,
+ "2": 0.8495
+ },
+ "confidence": 0.5407,
+ "answer_confidence": 0.8495,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9104,
+ "confidence": 0.9104,
+ "answer_confidence": 0.9104,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8502,
+ "technical": 0.0984,
+ "other": 0.0514
+ },
+ "confidence": 0.5279,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7984,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0383,
+ "1": 0.1251,
+ "2": 0.8366
+ },
+ "confidence": 0.5138,
+ "answer_confidence": 0.8366,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9522,
+ "confidence": 0.9522,
+ "answer_confidence": 0.9522,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9119,
+ "technical": 0.0619,
+ "other": 0.0263
+ },
+ "confidence": 0.6797,
+ "answer_confidence": 0.9119,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8174,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0327,
+ "1": 0.1173,
+ "2": 0.85
+ },
+ "confidence": 0.5437,
+ "answer_confidence": 0.85,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9744,
+ "confidence": 0.9744,
+ "answer_confidence": 0.9744,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9076,
+ "technical": 0.0614,
+ "other": 0.031
+ },
+ "confidence": 0.6659,
+ "answer_confidence": 0.9076,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6686,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0787,
+ "1": 0.174,
+ "2": 0.7473
+ },
+ "confidence": 0.3428,
+ "answer_confidence": 0.7473,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9973,
+ "confidence": 0.9973,
+ "answer_confidence": 0.9973,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8711,
+ "technical": 0.084,
+ "other": 0.0449
+ },
+ "confidence": 0.5744,
+ "answer_confidence": 0.8711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8099,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0405,
+ "1": 0.1091,
+ "2": 0.8504
+ },
+ "confidence": 0.5363,
+ "answer_confidence": 0.8504,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9386,
+ "confidence": 0.9386,
+ "answer_confidence": 0.9386,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8908,
+ "technical": 0.074,
+ "other": 0.0352
+ },
+ "confidence": 0.6236,
+ "answer_confidence": 0.8908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8126,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0335,
+ "1": 0.1203,
+ "2": 0.8462
+ },
+ "confidence": 0.5358,
+ "answer_confidence": 0.8462,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9351,
+ "confidence": 0.9351,
+ "answer_confidence": 0.9351,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8958,
+ "technical": 0.071,
+ "other": 0.0333
+ },
+ "confidence": 0.6363,
+ "answer_confidence": 0.8958,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7433,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0599,
+ "1": 0.1369,
+ "2": 0.8032
+ },
+ "confidence": 0.4384,
+ "answer_confidence": 0.8032,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9824,
+ "confidence": 0.9824,
+ "answer_confidence": 0.9824,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.915,
+ "technical": 0.0552,
+ "other": 0.0298
+ },
+ "confidence": 0.6852,
+ "answer_confidence": 0.915,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8398,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0301,
+ "1": 0.1,
+ "2": 0.8699
+ },
+ "confidence": 0.584,
+ "answer_confidence": 0.8699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9285,
+ "confidence": 0.9285,
+ "answer_confidence": 0.9285,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.906,
+ "technical": 0.0606,
+ "other": 0.0334
+ },
+ "confidence": 0.6606,
+ "answer_confidence": 0.906,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7995,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0435,
+ "1": 0.1136,
+ "2": 0.8429
+ },
+ "confidence": 0.5199,
+ "answer_confidence": 0.8429,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9698,
+ "confidence": 0.9698,
+ "answer_confidence": 0.9698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8836,
+ "technical": 0.0799,
+ "other": 0.0365
+ },
+ "confidence": 0.6066,
+ "answer_confidence": 0.8836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7561,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0459,
+ "1": 0.1522,
+ "2": 0.8019
+ },
+ "confidence": 0.4494,
+ "answer_confidence": 0.8019,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9458,
+ "confidence": 0.9458,
+ "answer_confidence": 0.9458,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9298,
+ "technical": 0.0462,
+ "other": 0.024
+ },
+ "confidence": 0.7276,
+ "answer_confidence": 0.9298,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.752,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0625,
+ "1": 0.123,
+ "2": 0.8145
+ },
+ "confidence": 0.4555,
+ "answer_confidence": 0.8145,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9912,
+ "confidence": 0.9912,
+ "answer_confidence": 0.9912,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sorted.json b/laya-rust/tests/golden/english/case_batch_sorted.json
new file mode 100644
index 00000000..2dc3ddcb
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sorted.json
@@ -0,0 +1,471 @@
+{
+ "name": "batch_sorted",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ],
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint. Turn 140: the agent restated the refund policy and the customer repeated the complaint. Turn 141: the agent restated the refund policy and the customer repeated the complaint. Turn 142: the agent restated the refund policy and the customer repeated the complaint. Turn 143: the agent restated the refund policy and the customer repeated the complaint. Turn 144: the agent restated the refund policy and the customer repeated the complaint. Turn 145: the agent restated the refund policy and the customer repeated the complaint. Turn 146: the agent restated the refund policy and the customer repeated the complaint. Turn 147: the agent restated the refund policy and the customer repeated the complaint. Turn 148: the agent restated the refund policy and the customer repeated the complaint. Turn 149: the agent restated the refund policy and the customer repeated the complaint. Turn 150: the agent restated the refund policy and the customer repeated the complaint. Turn 151: the agent restated the refund policy and the customer repeated the complaint. Turn 152: the agent restated the refund policy and the customer repeated the complaint. Turn 153: the agent restated the refund policy and the customer repeated the complaint. Turn 154: the agent restated the refund policy and the customer repeated the complaint. Turn 155: the agent restated the refund policy and the customer repeated the complaint. Turn 156: the agent restated the refund policy and the customer repeated the complaint. Turn 157: the agent restated the refund policy and the customer repeated the complaint. Turn 158: the agent restated the refund policy and the customer repeated the complaint. Turn 159: the agent restated the refund policy and the customer repeated the complaint. Turn 160: the agent restated the refund policy and the customer repeated the complaint. Turn 161: the agent restated the refund policy and the customer repeated the complaint. Turn 162: the agent restated the refund policy and the customer repeated the complaint. Turn 163: the agent restated the refund policy and the customer repeated the complaint. Turn 164: the agent restated the refund policy and the customer repeated the complaint. Turn 165: the agent restated the refund policy and the customer repeated the complaint. Turn 166: the agent restated the refund policy and the customer repeated the complaint. Turn 167: the agent restated the refund policy and the customer repeated the complaint. Turn 168: the agent restated the refund policy and the customer repeated the complaint. Turn 169: the agent restated the refund policy and the customer repeated the complaint. Turn 170: the agent restated the refund policy and the customer repeated the complaint. Turn 171: the agent restated the refund policy and the customer repeated the complaint. Turn 172: the agent restated the refund policy and the customer repeated the complaint. Turn 173: the agent restated the refund policy and the customer repeated the complaint. Turn 174: the agent restated the refund policy and the customer repeated the complaint. Turn 175: the agent restated the refund policy and the customer repeated the complaint. Turn 176: the agent restated the refund policy and the customer repeated the complaint. Turn 177: the agent restated the refund policy and the customer repeated the complaint. Turn 178: the agent restated the refund policy and the customer repeated the complaint. Turn 179: the agent restated the refund policy and the customer repeated the complaint. Turn 180: the agent restated the refund policy and the customer repeated the complaint. Turn 181: the agent restated the refund policy and the customer repeated the complaint. Turn 182: the agent restated the refund policy and the customer repeated the complaint. Turn 183: the agent restated the refund policy and the customer repeated the complaint. Turn 184: the agent restated the refund policy and the customer repeated the complaint. Turn 185: the agent restated the refund policy and the customer repeated the complaint. Turn 186: the agent restated the refund policy and the customer repeated the complaint. Turn 187: the agent restated the refund policy and the customer repeated the complaint. Turn 188: the agent restated the refund policy and the customer repeated the complaint. Turn 189: the agent restated the refund policy and the customer repeated the complaint. Turn 190: the agent restated the refund policy and the customer repeated the complaint. Turn 191: the agent restated the refund policy and the customer repeated the complaint. Turn 192: the agent restated the refund policy and the customer repeated the complaint. Turn 193: the agent restated the refund policy and the customer repeated the complaint. Turn 194: the agent restated the refund policy and the customer repeated the complaint. Turn 195: the agent restated the refund policy and the customer repeated the complaint. Turn 196: the agent restated the refund policy and the customer repeated the complaint. Turn 197: the agent restated the refund policy and the customer repeated the complaint. Turn 198: the agent restated the refund policy and the customer repeated the complaint. Turn 199: the agent restated the refund policy and the customer repeated the complaint. Turn 200: the agent restated the refund policy and the customer repeated the complaint. Turn 201: the agent restated the refund policy and the customer repeated the complaint. Turn 202: the agent restated the refund policy and the customer repeated the complaint. Turn 203: the agent restated the refund policy and the customer repeated the complaint. Turn 204: the agent restated the refund policy and the customer repeated the complaint. Turn 205: the agent restated the refund policy and the customer repeated the complaint. Turn 206: the agent restated the refund policy and the customer repeated the complaint. Turn 207: the agent restated the refund policy and the customer repeated the complaint. Turn 208: the agent restated the refund policy and the customer repeated the complaint. Turn 209: the agent restated the refund policy and the customer repeated the complaint. Turn 210: the agent restated the refund policy and the customer repeated the complaint. Turn 211: the agent restated the refund policy and the customer repeated the complaint. Turn 212: the agent restated the refund policy and the customer repeated the complaint. Turn 213: the agent restated the refund policy and the customer repeated the complaint. Turn 214: the agent restated the refund policy and the customer repeated the complaint. Turn 215: the agent restated the refund policy and the customer repeated the complaint. Turn 216: the agent restated the refund policy and the customer repeated the complaint. Turn 217: the agent restated the refund policy and the customer repeated the complaint. Turn 218: the agent restated the refund policy and the customer repeated the complaint. Turn 219: the agent restated the refund policy and the customer repeated the complaint. Turn 220: the agent restated the refund policy and the customer repeated the complaint. Turn 221: the agent restated the refund policy and the customer repeated the complaint. Turn 222: the agent restated the refund policy and the customer repeated the complaint. Turn 223: the agent restated the refund policy and the customer repeated the complaint. Turn 224: the agent restated the refund policy and the customer repeated the complaint. Turn 225: the agent restated the refund policy and the customer repeated the complaint. Turn 226: the agent restated the refund policy and the customer repeated the complaint. Turn 227: the agent restated the refund policy and the customer repeated the complaint. Turn 228: the agent restated the refund policy and the customer repeated the complaint. Turn 229: the agent restated the refund policy and the customer repeated the complaint. Turn 230: the agent restated the refund policy and the customer repeated the complaint. Turn 231: the agent restated the refund policy and the customer repeated the complaint. Turn 232: the agent restated the refund policy and the customer repeated the complaint. Turn 233: the agent restated the refund policy and the customer repeated the complaint. Turn 234: the agent restated the refund policy and the customer repeated the complaint. Turn 235: the agent restated the refund policy and the customer repeated the complaint. Turn 236: the agent restated the refund policy and the customer repeated the complaint. Turn 237: the agent restated the refund policy and the customer repeated the complaint. Turn 238: the agent restated the refund policy and the customer repeated the complaint. Turn 239: the agent restated the refund policy and the customer repeated the complaint. Turn 240: the agent restated the refund policy and the customer repeated the complaint. Turn 241: the agent restated the refund policy and the customer repeated the complaint. Turn 242: the agent restated the refund policy and the customer repeated the complaint. Turn 243: the agent restated the refund policy and the customer repeated the complaint. Turn 244: the agent restated the refund policy and the customer repeated the complaint. Turn 245: the agent restated the refund policy and the customer repeated the complaint. Turn 246: the agent restated the refund policy and the customer repeated the complaint. Turn 247: the agent restated the refund policy and the customer repeated the complaint. Turn 248: the agent restated the refund policy and the customer repeated the complaint. Turn 249: the agent restated the refund policy and the customer repeated the complaint. Turn 250: the agent restated the refund policy and the customer repeated the complaint. Turn 251: the agent restated the refund policy and the customer repeated the complaint. Turn 252: the agent restated the refund policy and the customer repeated the complaint. Turn 253: the agent restated the refund policy and the customer repeated the complaint. Turn 254: the agent restated the refund policy and the customer repeated the complaint. Turn 255: the agent restated the refund policy and the customer repeated the complaint. Turn 256: the agent restated the refund policy and the customer repeated the complaint. Turn 257: the agent restated the refund policy and the customer repeated the complaint. Turn 258: the agent restated the refund policy and the customer repeated the complaint. Turn 259: the agent restated the refund policy and the customer repeated the complaint. Turn 260: the agent restated the refund policy and the customer repeated the complaint. Turn 261: the agent restated the refund policy and the customer repeated the complaint. Turn 262: the agent restated the refund policy and the customer repeated the complaint. Turn 263: the agent restated the refund policy and the customer repeated the complaint. Turn 264: the agent restated the refund policy and the customer repeated the complaint. Turn 265: the agent restated the refund policy and the customer repeated the complaint. Turn 266: the agent restated the refund policy and the customer repeated the complaint. Turn 267: the agent restated the refund policy and the customer repeated the complaint. Turn 268: the agent restated the refund policy and the customer repeated the complaint. Turn 269: the agent restated the refund policy and the customer repeated the complaint. Turn 270: the agent restated the refund policy and the customer repeated the complaint. Turn 271: the agent restated the refund policy and the customer repeated the complaint. Turn 272: the agent restated the refund policy and the customer repeated the complaint. Turn 273: the agent restated the refund policy and the customer repeated the complaint. Turn 274: the agent restated the refund policy and the customer repeated the complaint. Turn 275: the agent restated the refund policy and the customer repeated the complaint. Turn 276: the agent restated the refund policy and the customer repeated the complaint. Turn 277: the agent restated the refund policy and the customer repeated the complaint. Turn 278: the agent restated the refund policy and the customer repeated the complaint. Turn 279: the agent restated the refund policy and the customer repeated the complaint. Turn 280: the agent restated the refund policy and the customer repeated the complaint. Turn 281: the agent restated the refund policy and the customer repeated the complaint. Turn 282: the agent restated the refund policy and the customer repeated the complaint. Turn 283: the agent restated the refund policy and the customer repeated the complaint. Turn 284: the agent restated the refund policy and the customer repeated the complaint. Turn 285: the agent restated the refund policy and the customer repeated the complaint. Turn 286: the agent restated the refund policy and the customer repeated the complaint. Turn 287: the agent restated the refund policy and the customer repeated the complaint. Turn 288: the agent restated the refund policy and the customer repeated the complaint. Turn 289: the agent restated the refund policy and the customer repeated the complaint. Turn 290: the agent restated the refund policy and the customer repeated the complaint. Turn 291: the agent restated the refund policy and the customer repeated the complaint. Turn 292: the agent restated the refund policy and the customer repeated the complaint. Turn 293: the agent restated the refund policy and the customer repeated the complaint. Turn 294: the agent restated the refund policy and the customer repeated the complaint. Turn 295: the agent restated the refund policy and the customer repeated the complaint. Turn 296: the agent restated the refund policy and the customer repeated the complaint. Turn 297: the agent restated the refund policy and the customer repeated the complaint. Turn 298: the agent restated the refund policy and the customer repeated the complaint. Turn 299: the agent restated the refund policy and the customer repeated the complaint. Turn 300: the agent restated the refund policy and the customer repeated the complaint. Turn 301: the agent restated the refund policy and the customer repeated the complaint. Turn 302: the agent restated the refund policy and the customer repeated the complaint. Turn 303: the agent restated the refund policy and the customer repeated the complaint. Turn 304: the agent restated the refund policy and the customer repeated the complaint. Turn 305: the agent restated the refund policy and the customer repeated the complaint. Turn 306: the agent restated the refund policy and the customer repeated the complaint. Turn 307: the agent restated the refund policy and the customer repeated the complaint. Turn 308: the agent restated the refund policy and the customer repeated the complaint. Turn 309: the agent restated the refund policy and the customer repeated the complaint. Turn 310: the agent restated the refund policy and the customer repeated the complaint. Turn 311: the agent restated the refund policy and the customer repeated the complaint. Turn 312: the agent restated the refund policy and the customer repeated the complaint. Turn 313: the agent restated the refund policy and the customer repeated the complaint. Turn 314: the agent restated the refund policy and the customer repeated the complaint. Turn 315: the agent restated the refund policy and the customer repeated the complaint. Turn 316: the agent restated the refund policy and the customer repeated the complaint. Turn 317: the agent restated the refund policy and the customer repeated the complaint. Turn 318: the agent restated the refund policy and the customer repeated the complaint. Turn 319: the agent restated the refund policy and the customer repeated the complaint. Turn 320: the agent restated the refund policy and the customer repeated the complaint. Turn 321: the agent restated the refund policy and the customer repeated the complaint. Turn 322: the agent restated the refund policy and the customer repeated the complaint. Turn 323: the agent restated the refund policy and the customer repeated the complaint. Turn 324: the agent restated the refund policy and the customer repeated the complaint. Turn 325: the agent restated the refund policy and the customer repeated the complaint. Turn 326: the agent restated the refund policy and the customer repeated the complaint. Turn 327: the agent restated the refund policy and the customer repeated the complaint. Turn 328: the agent restated the refund policy and the customer repeated the complaint. Turn 329: the agent restated the refund policy and the customer repeated the complaint. Turn 330: the agent restated the refund policy and the customer repeated the complaint. Turn 331: the agent restated the refund policy and the customer repeated the complaint. Turn 332: the agent restated the refund policy and the customer repeated the complaint. Turn 333: the agent restated the refund policy and the customer repeated the complaint. Turn 334: the agent restated the refund policy and the customer repeated the complaint. Turn 335: the agent restated the refund policy and the customer repeated the complaint. Turn 336: the agent restated the refund policy and the customer repeated the complaint. Turn 337: the agent restated the refund policy and the customer repeated the complaint. Turn 338: the agent restated the refund policy and the customer repeated the complaint. Turn 339: the agent restated the refund policy and the customer repeated the complaint. Turn 340: the agent restated the refund policy and the customer repeated the complaint. Turn 341: the agent restated the refund policy and the customer repeated the complaint. Turn 342: the agent restated the refund policy and the customer repeated the complaint. Turn 343: the agent restated the refund policy and the customer repeated the complaint. Turn 344: the agent restated the refund policy and the customer repeated the complaint. Turn 345: the agent restated the refund policy and the customer repeated the complaint. Turn 346: the agent restated the refund policy and the customer repeated the complaint. Turn 347: the agent restated the refund policy and the customer repeated the complaint. Turn 348: the agent restated the refund policy and the customer repeated the complaint. Turn 349: the agent restated the refund policy and the customer repeated the complaint. Turn 350: the agent restated the refund policy and the customer repeated the complaint. Turn 351: the agent restated the refund policy and the customer repeated the complaint. Turn 352: the agent restated the refund policy and the customer repeated the complaint. Turn 353: the agent restated the refund policy and the customer repeated the complaint. Turn 354: the agent restated the refund policy and the customer repeated the complaint. Turn 355: the agent restated the refund policy and the customer repeated the complaint. Turn 356: the agent restated the refund policy and the customer repeated the complaint. Turn 357: the agent restated the refund policy and the customer repeated the complaint. Turn 358: the agent restated the refund policy and the customer repeated the complaint. Turn 359: the agent restated the refund policy and the customer repeated the complaint. Turn 360: the agent restated the refund policy and the customer repeated the complaint. Turn 361: the agent restated the refund policy and the customer repeated the complaint. Turn 362: the agent restated the refund policy and the customer repeated the complaint. Turn 363: the agent restated the refund policy and the customer repeated the complaint. Turn 364: the agent restated the refund policy and the customer repeated the complaint. Turn 365: the agent restated the refund policy and the customer repeated the complaint. Turn 366: the agent restated the refund policy and the customer repeated the complaint. Turn 367: the agent restated the refund policy and the customer repeated the complaint. Turn 368: the agent restated the refund policy and the customer repeated the complaint. Turn 369: the agent restated the refund policy and the customer repeated the complaint. Turn 370: the agent restated the refund policy and the customer repeated the complaint. Turn 371: the agent restated the refund policy and the customer repeated the complaint. Turn 372: the agent restated the refund policy and the customer repeated the complaint. Turn 373: the agent restated the refund policy and the customer repeated the complaint. Turn 374: the agent restated the refund policy and the customer repeated the complaint. Turn 375: the agent restated the refund policy and the customer repeated the complaint. Turn 376: the agent restated the refund policy and the customer repeated the complaint. Turn 377: the agent restated the refund policy and the customer repeated the complaint. Turn 378: the agent restated the refund policy and the customer repeated the complaint. Turn 379: the agent restated the refund policy and the customer repeated the complaint. Turn 380: the agent restated the refund policy and the customer repeated the complaint. Turn 381: the agent restated the refund policy and the customer repeated the complaint. Turn 382: the agent restated the refund policy and the customer repeated the complaint. Turn 383: the agent restated the refund policy and the customer repeated the complaint. Turn 384: the agent restated the refund policy and the customer repeated the complaint. Turn 385: the agent restated the refund policy and the customer repeated the complaint. Turn 386: the agent restated the refund policy and the customer repeated the complaint. Turn 387: the agent restated the refund policy and the customer repeated the complaint. Turn 388: the agent restated the refund policy and the customer repeated the complaint. Turn 389: the agent restated the refund policy and the customer repeated the complaint. Turn 390: the agent restated the refund policy and the customer repeated the complaint. Turn 391: the agent restated the refund policy and the customer repeated the complaint. Turn 392: the agent restated the refund policy and the customer repeated the complaint. Turn 393: the agent restated the refund policy and the customer repeated the complaint. Turn 394: the agent restated the refund policy and the customer repeated the complaint. Turn 395: the agent restated the refund policy and the customer repeated the complaint. Turn 396: the agent restated the refund policy and the customer repeated the complaint. Turn 397: the agent restated the refund policy and the customer repeated the complaint. Turn 398: the agent restated the refund policy and the customer repeated the complaint. Turn 399: the agent restated the refund policy and the customer repeated the complaint.",
+ "The app crashes on login since yesterday's update.",
+ "ok"
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 91,
+ 44,
+ 157,
+ 74,
+ 512,
+ 44,
+ 35
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8178,
+ "technical": 0.1682,
+ "other": 0.014
+ },
+ "confidence": 0.5229,
+ "answer_confidence": 0.8178,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7428,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5839,
+ "1": 0.0894,
+ "2": 0.3267
+ },
+ "confidence": 0.1849,
+ "answer_confidence": 0.5839,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0144,
+ "confidence": 0.9856,
+ "answer_confidence": 0.9856,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7468,
+ "technical": 0.1514,
+ "other": 0.1017
+ },
+ "confidence": 0.3297,
+ "answer_confidence": 0.7468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6364,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1029,
+ "1": 0.1578,
+ "2": 0.7393
+ },
+ "confidence": 0.3185,
+ "answer_confidence": 0.7393,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9238,
+ "confidence": 0.9238,
+ "answer_confidence": 0.9238,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9067,
+ "technical": 0.0472,
+ "other": 0.0461
+ },
+ "confidence": 0.6589,
+ "answer_confidence": 0.9067,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8012,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3497,
+ "1": 0.4994,
+ "2": 0.1509
+ },
+ "confidence": 0.0902,
+ "answer_confidence": 0.4994,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1497,
+ "confidence": 0.8503,
+ "answer_confidence": 0.8503,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 214,
+ "output_tokens": 0,
+ "state_tokens": 40,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8204,
+ "technical": 0.0752,
+ "other": 0.1045
+ },
+ "confidence": 0.4603,
+ "answer_confidence": 0.8204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6454,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5702,
+ "1": 0.2141,
+ "2": 0.2156
+ },
+ "confidence": 0.1069,
+ "answer_confidence": 0.5702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3196,
+ "confidence": 0.6804,
+ "answer_confidence": 0.6804,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 6804,
+ "state_tokens_dropped": 6326,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.0483,
+ "technical": 0.9057,
+ "other": 0.046
+ },
+ "confidence": 0.6563,
+ "answer_confidence": 0.9057,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4741,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.17,
+ "1": 0.186,
+ "2": 0.6441
+ },
+ "confidence": 0.1831,
+ "answer_confidence": 0.6441,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.122,
+ "confidence": 0.878,
+ "answer_confidence": 0.878,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5728,
+ "technical": 0.2419,
+ "other": 0.1853
+ },
+ "confidence": 0.1126,
+ "answer_confidence": 0.5728,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.606,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5308,
+ "1": 0.3323,
+ "2": 0.1368
+ },
+ "confidence": 0.113,
+ "answer_confidence": 0.5308,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4482,
+ "confidence": 0.5518,
+ "answer_confidence": 0.5518,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 97,
+ "output_tokens": 0,
+ "state_tokens": 1,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 44
+ ],
+ [
+ 6,
+ 74
+ ],
+ [
+ 6,
+ 157
+ ],
+ [
+ 3,
+ 512
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_sorted_windows.json b/laya-rust/tests/golden/english/case_batch_sorted_windows.json
new file mode 100644
index 00000000..6a5ed8e5
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_sorted_windows.json
@@ -0,0 +1,1101 @@
+{
+ "name": "batch_sorted_windows",
+ "states": [
+ "Routine status note 0. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 1. Routine status note 1. ",
+ "Routine status note 2. Routine status note 2. Routine status note 2. ",
+ "Routine status note 3. Routine status note 3. Routine status note 3. Routine status note 3. ",
+ "Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. ",
+ "Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. ",
+ "Routine status note 7. ",
+ "Routine status note 8. Routine status note 8. ",
+ "Routine status note 9. Routine status note 9. Routine status note 9. ",
+ "Routine status note 10. Routine status note 10. Routine status note 10. Routine status note 10. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. ",
+ "Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. ",
+ "Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. ",
+ "Routine status note 14. ",
+ "Routine status note 15. Routine status note 15. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 16. Routine status note 16. Routine status note 16. ",
+ "Routine status note 17. Routine status note 17. Routine status note 17. Routine status note 17. "
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 97,
+ 47,
+ 53,
+ 59,
+ 65,
+ 127,
+ 77,
+ 41,
+ 47,
+ 53,
+ 115,
+ 65,
+ 71,
+ 77,
+ 41,
+ 103,
+ 53,
+ 59
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8655,
+ "technical": 0.0902,
+ "other": 0.0442
+ },
+ "confidence": 0.5631,
+ "answer_confidence": 0.8655,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5635,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1496,
+ "1": 0.1374,
+ "2": 0.7131
+ },
+ "confidence": 0.2736,
+ "answer_confidence": 0.7131,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9215,
+ "confidence": 0.9215,
+ "answer_confidence": 0.9215,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2397,
+ "technical": 0.4202,
+ "other": 0.3402
+ },
+ "confidence": 0.0228,
+ "answer_confidence": 0.4202,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8313,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5109,
+ "1": 0.1468,
+ "2": 0.3423
+ },
+ "confidence": 0.0973,
+ "answer_confidence": 0.5109,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0808,
+ "confidence": 0.9192,
+ "answer_confidence": 0.9192,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1535,
+ "technical": 0.4669,
+ "other": 0.3797
+ },
+ "confidence": 0.0798,
+ "answer_confidence": 0.4669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9775,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4256,
+ "1": 0.1713,
+ "2": 0.4031
+ },
+ "confidence": 0.0606,
+ "answer_confidence": 0.4256,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.05,
+ "confidence": 0.95,
+ "answer_confidence": 0.95,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1547,
+ "technical": 0.468,
+ "other": 0.3773
+ },
+ "confidence": 0.079,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9046,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4522,
+ "1": 0.191,
+ "2": 0.3568
+ },
+ "confidence": 0.0508,
+ "answer_confidence": 0.4522,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.059,
+ "confidence": 0.941,
+ "answer_confidence": 0.941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1587,
+ "technical": 0.454,
+ "other": 0.3873
+ },
+ "confidence": 0.0734,
+ "answer_confidence": 0.454,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8131,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.493,
+ "1": 0.2008,
+ "2": 0.3062
+ },
+ "confidence": 0.0594,
+ "answer_confidence": 0.493,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0713,
+ "confidence": 0.9287,
+ "answer_confidence": 0.9287,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8316,
+ "technical": 0.1188,
+ "other": 0.0496
+ },
+ "confidence": 0.4944,
+ "answer_confidence": 0.8316,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7514,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.161,
+ "2": 0.7952
+ },
+ "confidence": 0.4418,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9343,
+ "confidence": 0.9343,
+ "answer_confidence": 0.9343,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1728,
+ "technical": 0.4389,
+ "other": 0.3884
+ },
+ "confidence": 0.0606,
+ "answer_confidence": 0.4389,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7076,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5471,
+ "1": 0.1982,
+ "2": 0.2547
+ },
+ "confidence": 0.0906,
+ "answer_confidence": 0.5471,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1004,
+ "confidence": 0.8996,
+ "answer_confidence": 0.8996,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2969,
+ "technical": 0.3791,
+ "other": 0.324
+ },
+ "confidence": 0.0047,
+ "answer_confidence": 0.3791,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.5766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6179,
+ "1": 0.1875,
+ "2": 0.1945
+ },
+ "confidence": 0.1536,
+ "answer_confidence": 0.6179,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0524,
+ "confidence": 0.9476,
+ "answer_confidence": 0.9476,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2135,
+ "technical": 0.4829,
+ "other": 0.3036
+ },
+ "confidence": 0.0505,
+ "answer_confidence": 0.4829,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8657,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4937,
+ "1": 0.1469,
+ "2": 0.3594
+ },
+ "confidence": 0.0916,
+ "answer_confidence": 0.4937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1542,
+ "technical": 0.4931,
+ "other": 0.3526
+ },
+ "confidence": 0.0857,
+ "answer_confidence": 0.4931,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7263,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5523,
+ "1": 0.1692,
+ "2": 0.2786
+ },
+ "confidence": 0.1038,
+ "answer_confidence": 0.5523,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0396,
+ "confidence": 0.9604,
+ "answer_confidence": 0.9604,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.811,
+ "technical": 0.1311,
+ "other": 0.0579
+ },
+ "confidence": 0.4528,
+ "answer_confidence": 0.811,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7229,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0615,
+ "1": 0.1542,
+ "2": 0.7843
+ },
+ "confidence": 0.4081,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9289,
+ "confidence": 0.9289,
+ "answer_confidence": 0.9289,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1664,
+ "technical": 0.4641,
+ "other": 0.3695
+ },
+ "confidence": 0.0692,
+ "answer_confidence": 0.4641,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7084,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5451,
+ "1": 0.2013,
+ "2": 0.2536
+ },
+ "confidence": 0.0885,
+ "answer_confidence": 0.5451,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0688,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1743,
+ "technical": 0.4648,
+ "other": 0.361
+ },
+ "confidence": 0.0639,
+ "answer_confidence": 0.4648,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7638,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5147,
+ "1": 0.2067,
+ "2": 0.2785
+ },
+ "confidence": 0.0681,
+ "answer_confidence": 0.5147,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0847,
+ "confidence": 0.9153,
+ "answer_confidence": 0.9153,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 205,
+ "output_tokens": 0,
+ "state_tokens": 37,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1576,
+ "technical": 0.4595,
+ "other": 0.3829
+ },
+ "confidence": 0.0751,
+ "answer_confidence": 0.4595,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7753,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5131,
+ "1": 0.1986,
+ "2": 0.2884
+ },
+ "confidence": 0.0697,
+ "answer_confidence": 0.5131,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0911,
+ "confidence": 0.9089,
+ "answer_confidence": 0.9089,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2859,
+ "technical": 0.3743,
+ "other": 0.3399
+ },
+ "confidence": 0.0055,
+ "answer_confidence": 0.3743,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6247,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5914,
+ "1": 0.1926,
+ "2": 0.216
+ },
+ "confidence": 0.1272,
+ "answer_confidence": 0.5914,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0601,
+ "confidence": 0.9399,
+ "answer_confidence": 0.9399,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8501,
+ "technical": 0.1039,
+ "other": 0.046
+ },
+ "confidence": 0.5314,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7509,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0536,
+ "1": 0.142,
+ "2": 0.8044
+ },
+ "confidence": 0.4457,
+ "answer_confidence": 0.8044,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.94,
+ "confidence": 0.94,
+ "answer_confidence": 0.94,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1586,
+ "technical": 0.4977,
+ "other": 0.3437
+ },
+ "confidence": 0.084,
+ "answer_confidence": 0.4977,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7191,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.557,
+ "1": 0.1668,
+ "2": 0.2761
+ },
+ "confidence": 0.1079,
+ "answer_confidence": 0.557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0613,
+ "confidence": 0.9387,
+ "answer_confidence": 0.9387,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1858,
+ "technical": 0.4573,
+ "other": 0.3568
+ },
+ "confidence": 0.055,
+ "answer_confidence": 0.4573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7483,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.534,
+ "1": 0.1837,
+ "2": 0.2823
+ },
+ "confidence": 0.0867,
+ "answer_confidence": 0.534,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0678,
+ "confidence": 0.9322,
+ "answer_confidence": 0.9322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 41
+ ],
+ [
+ 6,
+ 47
+ ],
+ [
+ 6,
+ 53
+ ],
+ [
+ 6,
+ 65
+ ],
+ [
+ 6,
+ 71
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 103
+ ],
+ [
+ 6,
+ 127
+ ],
+ [
+ 6,
+ 59
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_truncated.json b/laya-rust/tests/golden/english/case_batch_truncated.json
new file mode 100644
index 00000000..e463bdce
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_truncated.json
@@ -0,0 +1,854 @@
+{
+ "name": "batch_truncated",
+ "states": [
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ [
+ {
+ "role": "user",
+ "content": "Turn 0: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 1: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 2: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 3: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 4: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 5: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 6: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 7: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 8: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 9: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 10: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 11: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 12: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 13: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 14: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 15: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 16: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 17: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 18: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 19: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 20: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 21: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 22: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 23: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 24: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 25: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 26: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 27: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 28: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 29: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 30: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 31: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 32: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 33: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 34: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 35: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 36: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 37: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 38: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 39: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 40: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 41: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 42: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 43: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 44: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 45: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 46: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 47: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 48: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 49: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 50: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 51: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 52: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 53: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 54: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 55: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 56: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 57: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 58: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 59: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 60: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 61: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 62: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 63: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 64: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 65: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 66: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 67: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 68: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 69: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 70: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 71: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 72: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 73: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 74: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 75: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 76: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 77: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 78: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 79: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 80: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 81: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 82: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 83: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 84: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 85: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 86: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 87: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 88: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 89: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 90: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 91: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 92: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 93: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 94: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 95: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 96: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 97: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 98: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 99: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 100: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 101: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 102: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 103: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 104: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 105: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 106: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 107: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 108: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 109: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 110: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 111: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 112: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 113: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 114: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 115: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 116: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 117: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 118: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 119: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 120: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 121: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 122: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 123: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 124: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 125: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 126: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 127: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 128: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 129: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 130: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 131: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 132: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 133: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 134: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 135: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 136: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 137: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 138: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 139: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "I want a refund or I will cancel today."
+ }
+ ],
+ {
+ "notes": "Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 512,
+ 512,
+ 512,
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8204,
+ "technical": 0.0752,
+ "other": 0.1045
+ },
+ "confidence": 0.4603,
+ "answer_confidence": 0.8204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6454,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5702,
+ "1": 0.2141,
+ "2": 0.2156
+ },
+ "confidence": 0.1069,
+ "answer_confidence": 0.5702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3196,
+ "confidence": 0.6804,
+ "answer_confidence": 0.6804,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 2384,
+ "state_tokens_dropped": 1906,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7269,
+ "technical": 0.0916,
+ "other": 0.1815
+ },
+ "confidence": 0.3077,
+ "answer_confidence": 0.7269,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1608,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2363,
+ "1": 0.3666,
+ "2": 0.3971
+ },
+ "confidence": 0.021,
+ "answer_confidence": 0.3971,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.698,
+ "confidence": 0.698,
+ "answer_confidence": 0.698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 4012,
+ "state_tokens_dropped": 3534,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8348,
+ "technical": 0.0629,
+ "other": 0.1023
+ },
+ "confidence": 0.4922,
+ "answer_confidence": 0.8348,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.5638,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6079,
+ "1": 0.2203,
+ "2": 0.1718
+ },
+ "confidence": 0.1458,
+ "answer_confidence": 0.6079,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.5013,
+ "confidence": 0.5013,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 2448,
+ "state_tokens_dropped": 1970,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 512
+ ],
+ [
+ 6,
+ 512
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_truncated_sorted.json b/laya-rust/tests/golden/english/case_batch_truncated_sorted.json
new file mode 100644
index 00000000..f21a6c48
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_truncated_sorted.json
@@ -0,0 +1,855 @@
+{
+ "name": "batch_truncated_sorted",
+ "states": [
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ [
+ {
+ "role": "user",
+ "content": "Turn 0: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 1: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 2: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 3: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 4: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 5: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 6: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 7: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 8: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 9: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 10: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 11: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 12: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 13: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 14: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 15: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 16: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 17: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 18: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 19: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 20: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 21: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 22: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 23: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 24: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 25: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 26: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 27: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 28: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 29: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 30: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 31: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 32: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 33: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 34: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 35: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 36: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 37: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 38: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 39: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 40: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 41: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 42: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 43: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 44: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 45: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 46: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 47: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 48: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 49: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 50: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 51: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 52: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 53: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 54: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 55: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 56: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 57: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 58: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 59: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 60: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 61: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 62: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 63: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 64: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 65: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 66: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 67: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 68: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 69: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 70: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 71: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 72: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 73: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 74: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 75: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 76: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 77: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 78: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 79: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 80: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 81: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 82: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 83: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 84: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 85: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 86: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 87: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 88: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 89: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 90: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 91: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 92: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 93: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 94: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 95: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 96: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 97: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 98: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 99: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 100: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 101: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 102: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 103: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 104: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 105: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 106: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 107: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 108: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 109: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 110: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 111: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 112: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 113: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 114: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 115: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 116: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 117: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 118: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 119: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 120: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 121: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 122: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 123: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 124: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 125: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 126: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 127: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 128: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 129: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 130: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 131: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 132: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 133: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 134: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 135: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 136: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 137: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 138: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 139: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "I want a refund or I will cancel today."
+ }
+ ],
+ {
+ "notes": "Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 512,
+ 512,
+ 512,
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8204,
+ "technical": 0.0752,
+ "other": 0.1045
+ },
+ "confidence": 0.4603,
+ "answer_confidence": 0.8204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6454,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5702,
+ "1": 0.2141,
+ "2": 0.2156
+ },
+ "confidence": 0.1069,
+ "answer_confidence": 0.5702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3196,
+ "confidence": 0.6804,
+ "answer_confidence": 0.6804,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 2384,
+ "state_tokens_dropped": 1906,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7269,
+ "technical": 0.0916,
+ "other": 0.1815
+ },
+ "confidence": 0.3077,
+ "answer_confidence": 0.7269,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1608,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2363,
+ "1": 0.3666,
+ "2": 0.3971
+ },
+ "confidence": 0.021,
+ "answer_confidence": 0.3971,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.698,
+ "confidence": 0.698,
+ "answer_confidence": 0.698,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 4012,
+ "state_tokens_dropped": 3534,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8348,
+ "technical": 0.0629,
+ "other": 0.1023
+ },
+ "confidence": 0.4922,
+ "answer_confidence": 0.8348,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.5638,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6079,
+ "1": 0.2203,
+ "2": 0.1718
+ },
+ "confidence": 0.1458,
+ "answer_confidence": 0.6079,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.5013,
+ "confidence": 0.5013,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 1536,
+ "output_tokens": 0,
+ "state_tokens": 2448,
+ "state_tokens_dropped": 1970,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8355,
+ "technical": 0.1139,
+ "other": 0.0506
+ },
+ "confidence": 0.5008,
+ "answer_confidence": 0.8355,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8578,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0339,
+ "1": 0.0745,
+ "2": 0.8916
+ },
+ "confidence": 0.6264,
+ "answer_confidence": 0.8916,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.968,
+ "confidence": 0.968,
+ "answer_confidence": 0.968,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 512
+ ],
+ [
+ 3,
+ 512
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/case_batch_unsorted_windows.json b/laya-rust/tests/golden/english/case_batch_unsorted_windows.json
new file mode 100644
index 00000000..32240e81
--- /dev/null
+++ b/laya-rust/tests/golden/english/case_batch_unsorted_windows.json
@@ -0,0 +1,1100 @@
+{
+ "name": "batch_unsorted_windows",
+ "states": [
+ "Routine status note 0. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 1. Routine status note 1. ",
+ "Routine status note 2. Routine status note 2. Routine status note 2. ",
+ "Routine status note 3. Routine status note 3. Routine status note 3. Routine status note 3. ",
+ "Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. ",
+ "Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. ",
+ "Routine status note 7. ",
+ "Routine status note 8. Routine status note 8. ",
+ "Routine status note 9. Routine status note 9. Routine status note 9. ",
+ "Routine status note 10. Routine status note 10. Routine status note 10. Routine status note 10. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. ",
+ "Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. ",
+ "Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. ",
+ "Routine status note 14. ",
+ "Routine status note 15. Routine status note 15. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 16. Routine status note 16. Routine status note 16. ",
+ "Routine status note 17. Routine status note 17. Routine status note 17. Routine status note 17. "
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 97,
+ 47,
+ 53,
+ 59,
+ 65,
+ 127,
+ 77,
+ 41,
+ 47,
+ 53,
+ 115,
+ 65,
+ 71,
+ 77,
+ 41,
+ 103,
+ 53,
+ 59
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8655,
+ "technical": 0.0902,
+ "other": 0.0442
+ },
+ "confidence": 0.5631,
+ "answer_confidence": 0.8655,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5635,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1496,
+ "1": 0.1374,
+ "2": 0.7131
+ },
+ "confidence": 0.2736,
+ "answer_confidence": 0.7131,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9215,
+ "confidence": 0.9215,
+ "answer_confidence": 0.9215,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2397,
+ "technical": 0.4202,
+ "other": 0.3402
+ },
+ "confidence": 0.0228,
+ "answer_confidence": 0.4202,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8313,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5109,
+ "1": 0.1468,
+ "2": 0.3423
+ },
+ "confidence": 0.0973,
+ "answer_confidence": 0.5109,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0808,
+ "confidence": 0.9192,
+ "answer_confidence": 0.9192,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1535,
+ "technical": 0.4669,
+ "other": 0.3797
+ },
+ "confidence": 0.0798,
+ "answer_confidence": 0.4669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9775,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4256,
+ "1": 0.1713,
+ "2": 0.4031
+ },
+ "confidence": 0.0606,
+ "answer_confidence": 0.4256,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.05,
+ "confidence": 0.95,
+ "answer_confidence": 0.95,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1547,
+ "technical": 0.468,
+ "other": 0.3773
+ },
+ "confidence": 0.079,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9046,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4522,
+ "1": 0.191,
+ "2": 0.3568
+ },
+ "confidence": 0.0508,
+ "answer_confidence": 0.4522,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.059,
+ "confidence": 0.941,
+ "answer_confidence": 0.941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1587,
+ "technical": 0.454,
+ "other": 0.3873
+ },
+ "confidence": 0.0734,
+ "answer_confidence": 0.454,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8131,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.493,
+ "1": 0.2008,
+ "2": 0.3062
+ },
+ "confidence": 0.0594,
+ "answer_confidence": 0.493,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0713,
+ "confidence": 0.9287,
+ "answer_confidence": 0.9287,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8316,
+ "technical": 0.1188,
+ "other": 0.0496
+ },
+ "confidence": 0.4944,
+ "answer_confidence": 0.8316,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7514,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0438,
+ "1": 0.161,
+ "2": 0.7952
+ },
+ "confidence": 0.4418,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9343,
+ "confidence": 0.9343,
+ "answer_confidence": 0.9343,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1728,
+ "technical": 0.4389,
+ "other": 0.3884
+ },
+ "confidence": 0.0606,
+ "answer_confidence": 0.4389,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7076,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5471,
+ "1": 0.1982,
+ "2": 0.2547
+ },
+ "confidence": 0.0906,
+ "answer_confidence": 0.5471,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1004,
+ "confidence": 0.8996,
+ "answer_confidence": 0.8996,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2969,
+ "technical": 0.3791,
+ "other": 0.324
+ },
+ "confidence": 0.0047,
+ "answer_confidence": 0.3791,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.5766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.6179,
+ "1": 0.1875,
+ "2": 0.1945
+ },
+ "confidence": 0.1536,
+ "answer_confidence": 0.6179,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0524,
+ "confidence": 0.9476,
+ "answer_confidence": 0.9476,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2135,
+ "technical": 0.4829,
+ "other": 0.3036
+ },
+ "confidence": 0.0505,
+ "answer_confidence": 0.4829,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8657,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4937,
+ "1": 0.1469,
+ "2": 0.3594
+ },
+ "confidence": 0.0916,
+ "answer_confidence": 0.4937,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1542,
+ "technical": 0.4931,
+ "other": 0.3526
+ },
+ "confidence": 0.0857,
+ "answer_confidence": 0.4931,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7263,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5523,
+ "1": 0.1692,
+ "2": 0.2786
+ },
+ "confidence": 0.1038,
+ "answer_confidence": 0.5523,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0396,
+ "confidence": 0.9604,
+ "answer_confidence": 0.9604,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.811,
+ "technical": 0.1311,
+ "other": 0.0579
+ },
+ "confidence": 0.4528,
+ "answer_confidence": 0.811,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7229,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0615,
+ "1": 0.1542,
+ "2": 0.7843
+ },
+ "confidence": 0.4081,
+ "answer_confidence": 0.7843,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.9289,
+ "confidence": 0.9289,
+ "answer_confidence": 0.9289,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1664,
+ "technical": 0.4641,
+ "other": 0.3695
+ },
+ "confidence": 0.0692,
+ "answer_confidence": 0.4641,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7084,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5451,
+ "1": 0.2013,
+ "2": 0.2536
+ },
+ "confidence": 0.0885,
+ "answer_confidence": 0.5451,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0688,
+ "confidence": 0.9312,
+ "answer_confidence": 0.9312,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1743,
+ "technical": 0.4648,
+ "other": 0.361
+ },
+ "confidence": 0.0639,
+ "answer_confidence": 0.4648,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7638,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5147,
+ "1": 0.2067,
+ "2": 0.2785
+ },
+ "confidence": 0.0681,
+ "answer_confidence": 0.5147,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0847,
+ "confidence": 0.9153,
+ "answer_confidence": 0.9153,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 205,
+ "output_tokens": 0,
+ "state_tokens": 37,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1576,
+ "technical": 0.4595,
+ "other": 0.3829
+ },
+ "confidence": 0.0751,
+ "answer_confidence": 0.4595,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7753,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5131,
+ "1": 0.1986,
+ "2": 0.2884
+ },
+ "confidence": 0.0697,
+ "answer_confidence": 0.5131,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0911,
+ "confidence": 0.9089,
+ "answer_confidence": 0.9089,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.2859,
+ "technical": 0.3743,
+ "other": 0.3399
+ },
+ "confidence": 0.0055,
+ "answer_confidence": 0.3743,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.6247,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.5914,
+ "1": 0.1926,
+ "2": 0.216
+ },
+ "confidence": 0.1272,
+ "answer_confidence": 0.5914,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0601,
+ "confidence": 0.9399,
+ "answer_confidence": 0.9399,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8501,
+ "technical": 0.1039,
+ "other": 0.046
+ },
+ "confidence": 0.5314,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7509,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0536,
+ "1": 0.142,
+ "2": 0.8044
+ },
+ "confidence": 0.4457,
+ "answer_confidence": 0.8044,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.94,
+ "confidence": 0.94,
+ "answer_confidence": 0.94,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1586,
+ "technical": 0.4977,
+ "other": 0.3437
+ },
+ "confidence": 0.084,
+ "answer_confidence": 0.4977,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7191,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.557,
+ "1": 0.1668,
+ "2": 0.2761
+ },
+ "confidence": 0.1079,
+ "answer_confidence": 0.557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0613,
+ "confidence": 0.9387,
+ "answer_confidence": 0.9387,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1858,
+ "technical": 0.4573,
+ "other": 0.3568
+ },
+ "confidence": 0.055,
+ "answer_confidence": 0.4573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7483,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.534,
+ "1": 0.1837,
+ "2": 0.2823
+ },
+ "confidence": 0.0867,
+ "answer_confidence": 0.534,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0678,
+ "confidence": 0.9322,
+ "answer_confidence": 0.9322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 97
+ ],
+ [
+ 6,
+ 59
+ ],
+ [
+ 6,
+ 127
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 53
+ ],
+ [
+ 6,
+ 115
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 103
+ ],
+ [
+ 6,
+ 59
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/english/index.json b/laya-rust/tests/golden/english/index.json
index 5c2bf1ce..eb70e281 100644
--- a/laya-rust/tests/golden/english/index.json
+++ b/laya-rust/tests/golden/english/index.json
@@ -162,5 +162,191 @@
"file": "case_long_no_state_room.json",
"expects_error": true,
"kind": "long"
+ },
+ {
+ "name": "batch_one_state",
+ "file": "case_batch_one_state.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_mixed",
+ "file": "case_batch_mixed.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_1",
+ "file": "case_batch_size_1.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_3",
+ "file": "case_batch_size_3.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_over_len",
+ "file": "case_batch_size_over_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_zero",
+ "file": "case_batch_size_zero.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_negative",
+ "file": "case_batch_size_negative.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_on",
+ "file": "case_batch_sort_on.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_off",
+ "file": "case_batch_sort_off.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_3",
+ "file": "case_batch_sort_size_3.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_partial_window",
+ "file": "case_batch_sort_partial_window.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_no_batch_size",
+ "file": "case_batch_sort_no_batch_size.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_1",
+ "file": "case_batch_sort_size_1.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_eq_len",
+ "file": "case_batch_sort_size_eq_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_over_len",
+ "file": "case_batch_sort_size_over_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_zero",
+ "file": "case_batch_sort_size_zero.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_truncated",
+ "file": "case_batch_truncated.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_truncated_sorted",
+ "file": "case_batch_truncated_sorted.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_duplicate_states",
+ "file": "case_batch_duplicate_states.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_states",
+ "file": "case_batch_empty_states.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_states_invalid_questions",
+ "file": "case_batch_empty_states_invalid_questions.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_questions",
+ "file": "case_batch_empty_questions.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_both",
+ "file": "case_batch_empty_both.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sorted",
+ "file": "case_batch_sorted.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sorted_windows",
+ "file": "case_batch_sorted_windows.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_unsorted_windows",
+ "file": "case_batch_unsorted_windows.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_chunked",
+ "file": "case_batch_chunked.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_single_question",
+ "file": "case_batch_single_question.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_questions_with_options",
+ "file": "case_batch_empty_questions_with_options.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_none_state",
+ "file": "case_batch_none_state.json",
+ "expects_error": true,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_options_exceed_head",
+ "file": "case_batch_options_exceed_head.json",
+ "expects_error": true,
+ "kind": "batch"
}
]
diff --git a/laya-rust/tests/golden/multilingual/case_batch_chunked.json b/laya-rust/tests/golden/multilingual/case_batch_chunked.json
new file mode 100644
index 00000000..7289736f
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_chunked.json
@@ -0,0 +1,466 @@
+{
+ "name": "batch_chunked",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ],
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint. Turn 140: the agent restated the refund policy and the customer repeated the complaint. Turn 141: the agent restated the refund policy and the customer repeated the complaint. Turn 142: the agent restated the refund policy and the customer repeated the complaint. Turn 143: the agent restated the refund policy and the customer repeated the complaint. Turn 144: the agent restated the refund policy and the customer repeated the complaint. Turn 145: the agent restated the refund policy and the customer repeated the complaint. Turn 146: the agent restated the refund policy and the customer repeated the complaint. Turn 147: the agent restated the refund policy and the customer repeated the complaint. Turn 148: the agent restated the refund policy and the customer repeated the complaint. Turn 149: the agent restated the refund policy and the customer repeated the complaint. Turn 150: the agent restated the refund policy and the customer repeated the complaint. Turn 151: the agent restated the refund policy and the customer repeated the complaint. Turn 152: the agent restated the refund policy and the customer repeated the complaint. Turn 153: the agent restated the refund policy and the customer repeated the complaint. Turn 154: the agent restated the refund policy and the customer repeated the complaint. Turn 155: the agent restated the refund policy and the customer repeated the complaint. Turn 156: the agent restated the refund policy and the customer repeated the complaint. Turn 157: the agent restated the refund policy and the customer repeated the complaint. Turn 158: the agent restated the refund policy and the customer repeated the complaint. Turn 159: the agent restated the refund policy and the customer repeated the complaint. Turn 160: the agent restated the refund policy and the customer repeated the complaint. Turn 161: the agent restated the refund policy and the customer repeated the complaint. Turn 162: the agent restated the refund policy and the customer repeated the complaint. Turn 163: the agent restated the refund policy and the customer repeated the complaint. Turn 164: the agent restated the refund policy and the customer repeated the complaint. Turn 165: the agent restated the refund policy and the customer repeated the complaint. Turn 166: the agent restated the refund policy and the customer repeated the complaint. Turn 167: the agent restated the refund policy and the customer repeated the complaint. Turn 168: the agent restated the refund policy and the customer repeated the complaint. Turn 169: the agent restated the refund policy and the customer repeated the complaint. Turn 170: the agent restated the refund policy and the customer repeated the complaint. Turn 171: the agent restated the refund policy and the customer repeated the complaint. Turn 172: the agent restated the refund policy and the customer repeated the complaint. Turn 173: the agent restated the refund policy and the customer repeated the complaint. Turn 174: the agent restated the refund policy and the customer repeated the complaint. Turn 175: the agent restated the refund policy and the customer repeated the complaint. Turn 176: the agent restated the refund policy and the customer repeated the complaint. Turn 177: the agent restated the refund policy and the customer repeated the complaint. Turn 178: the agent restated the refund policy and the customer repeated the complaint. Turn 179: the agent restated the refund policy and the customer repeated the complaint. Turn 180: the agent restated the refund policy and the customer repeated the complaint. Turn 181: the agent restated the refund policy and the customer repeated the complaint. Turn 182: the agent restated the refund policy and the customer repeated the complaint. Turn 183: the agent restated the refund policy and the customer repeated the complaint. Turn 184: the agent restated the refund policy and the customer repeated the complaint. Turn 185: the agent restated the refund policy and the customer repeated the complaint. Turn 186: the agent restated the refund policy and the customer repeated the complaint. Turn 187: the agent restated the refund policy and the customer repeated the complaint. Turn 188: the agent restated the refund policy and the customer repeated the complaint. Turn 189: the agent restated the refund policy and the customer repeated the complaint. Turn 190: the agent restated the refund policy and the customer repeated the complaint. Turn 191: the agent restated the refund policy and the customer repeated the complaint. Turn 192: the agent restated the refund policy and the customer repeated the complaint. Turn 193: the agent restated the refund policy and the customer repeated the complaint. Turn 194: the agent restated the refund policy and the customer repeated the complaint. Turn 195: the agent restated the refund policy and the customer repeated the complaint. Turn 196: the agent restated the refund policy and the customer repeated the complaint. Turn 197: the agent restated the refund policy and the customer repeated the complaint. Turn 198: the agent restated the refund policy and the customer repeated the complaint. Turn 199: the agent restated the refund policy and the customer repeated the complaint. Turn 200: the agent restated the refund policy and the customer repeated the complaint. Turn 201: the agent restated the refund policy and the customer repeated the complaint. Turn 202: the agent restated the refund policy and the customer repeated the complaint. Turn 203: the agent restated the refund policy and the customer repeated the complaint. Turn 204: the agent restated the refund policy and the customer repeated the complaint. Turn 205: the agent restated the refund policy and the customer repeated the complaint. Turn 206: the agent restated the refund policy and the customer repeated the complaint. Turn 207: the agent restated the refund policy and the customer repeated the complaint. Turn 208: the agent restated the refund policy and the customer repeated the complaint. Turn 209: the agent restated the refund policy and the customer repeated the complaint. Turn 210: the agent restated the refund policy and the customer repeated the complaint. Turn 211: the agent restated the refund policy and the customer repeated the complaint. Turn 212: the agent restated the refund policy and the customer repeated the complaint. Turn 213: the agent restated the refund policy and the customer repeated the complaint. Turn 214: the agent restated the refund policy and the customer repeated the complaint. Turn 215: the agent restated the refund policy and the customer repeated the complaint. Turn 216: the agent restated the refund policy and the customer repeated the complaint. Turn 217: the agent restated the refund policy and the customer repeated the complaint. Turn 218: the agent restated the refund policy and the customer repeated the complaint. Turn 219: the agent restated the refund policy and the customer repeated the complaint. Turn 220: the agent restated the refund policy and the customer repeated the complaint. Turn 221: the agent restated the refund policy and the customer repeated the complaint. Turn 222: the agent restated the refund policy and the customer repeated the complaint. Turn 223: the agent restated the refund policy and the customer repeated the complaint. Turn 224: the agent restated the refund policy and the customer repeated the complaint. Turn 225: the agent restated the refund policy and the customer repeated the complaint. Turn 226: the agent restated the refund policy and the customer repeated the complaint. Turn 227: the agent restated the refund policy and the customer repeated the complaint. Turn 228: the agent restated the refund policy and the customer repeated the complaint. Turn 229: the agent restated the refund policy and the customer repeated the complaint. Turn 230: the agent restated the refund policy and the customer repeated the complaint. Turn 231: the agent restated the refund policy and the customer repeated the complaint. Turn 232: the agent restated the refund policy and the customer repeated the complaint. Turn 233: the agent restated the refund policy and the customer repeated the complaint. Turn 234: the agent restated the refund policy and the customer repeated the complaint. Turn 235: the agent restated the refund policy and the customer repeated the complaint. Turn 236: the agent restated the refund policy and the customer repeated the complaint. Turn 237: the agent restated the refund policy and the customer repeated the complaint. Turn 238: the agent restated the refund policy and the customer repeated the complaint. Turn 239: the agent restated the refund policy and the customer repeated the complaint. Turn 240: the agent restated the refund policy and the customer repeated the complaint. Turn 241: the agent restated the refund policy and the customer repeated the complaint. Turn 242: the agent restated the refund policy and the customer repeated the complaint. Turn 243: the agent restated the refund policy and the customer repeated the complaint. Turn 244: the agent restated the refund policy and the customer repeated the complaint. Turn 245: the agent restated the refund policy and the customer repeated the complaint. Turn 246: the agent restated the refund policy and the customer repeated the complaint. Turn 247: the agent restated the refund policy and the customer repeated the complaint. Turn 248: the agent restated the refund policy and the customer repeated the complaint. Turn 249: the agent restated the refund policy and the customer repeated the complaint. Turn 250: the agent restated the refund policy and the customer repeated the complaint. Turn 251: the agent restated the refund policy and the customer repeated the complaint. Turn 252: the agent restated the refund policy and the customer repeated the complaint. Turn 253: the agent restated the refund policy and the customer repeated the complaint. Turn 254: the agent restated the refund policy and the customer repeated the complaint. Turn 255: the agent restated the refund policy and the customer repeated the complaint. Turn 256: the agent restated the refund policy and the customer repeated the complaint. Turn 257: the agent restated the refund policy and the customer repeated the complaint. Turn 258: the agent restated the refund policy and the customer repeated the complaint. Turn 259: the agent restated the refund policy and the customer repeated the complaint. Turn 260: the agent restated the refund policy and the customer repeated the complaint. Turn 261: the agent restated the refund policy and the customer repeated the complaint. Turn 262: the agent restated the refund policy and the customer repeated the complaint. Turn 263: the agent restated the refund policy and the customer repeated the complaint. Turn 264: the agent restated the refund policy and the customer repeated the complaint. Turn 265: the agent restated the refund policy and the customer repeated the complaint. Turn 266: the agent restated the refund policy and the customer repeated the complaint. Turn 267: the agent restated the refund policy and the customer repeated the complaint. Turn 268: the agent restated the refund policy and the customer repeated the complaint. Turn 269: the agent restated the refund policy and the customer repeated the complaint. Turn 270: the agent restated the refund policy and the customer repeated the complaint. Turn 271: the agent restated the refund policy and the customer repeated the complaint. Turn 272: the agent restated the refund policy and the customer repeated the complaint. Turn 273: the agent restated the refund policy and the customer repeated the complaint. Turn 274: the agent restated the refund policy and the customer repeated the complaint. Turn 275: the agent restated the refund policy and the customer repeated the complaint. Turn 276: the agent restated the refund policy and the customer repeated the complaint. Turn 277: the agent restated the refund policy and the customer repeated the complaint. Turn 278: the agent restated the refund policy and the customer repeated the complaint. Turn 279: the agent restated the refund policy and the customer repeated the complaint. Turn 280: the agent restated the refund policy and the customer repeated the complaint. Turn 281: the agent restated the refund policy and the customer repeated the complaint. Turn 282: the agent restated the refund policy and the customer repeated the complaint. Turn 283: the agent restated the refund policy and the customer repeated the complaint. Turn 284: the agent restated the refund policy and the customer repeated the complaint. Turn 285: the agent restated the refund policy and the customer repeated the complaint. Turn 286: the agent restated the refund policy and the customer repeated the complaint. Turn 287: the agent restated the refund policy and the customer repeated the complaint. Turn 288: the agent restated the refund policy and the customer repeated the complaint. Turn 289: the agent restated the refund policy and the customer repeated the complaint. Turn 290: the agent restated the refund policy and the customer repeated the complaint. Turn 291: the agent restated the refund policy and the customer repeated the complaint. Turn 292: the agent restated the refund policy and the customer repeated the complaint. Turn 293: the agent restated the refund policy and the customer repeated the complaint. Turn 294: the agent restated the refund policy and the customer repeated the complaint. Turn 295: the agent restated the refund policy and the customer repeated the complaint. Turn 296: the agent restated the refund policy and the customer repeated the complaint. Turn 297: the agent restated the refund policy and the customer repeated the complaint. Turn 298: the agent restated the refund policy and the customer repeated the complaint. Turn 299: the agent restated the refund policy and the customer repeated the complaint. Turn 300: the agent restated the refund policy and the customer repeated the complaint. Turn 301: the agent restated the refund policy and the customer repeated the complaint. Turn 302: the agent restated the refund policy and the customer repeated the complaint. Turn 303: the agent restated the refund policy and the customer repeated the complaint. Turn 304: the agent restated the refund policy and the customer repeated the complaint. Turn 305: the agent restated the refund policy and the customer repeated the complaint. Turn 306: the agent restated the refund policy and the customer repeated the complaint. Turn 307: the agent restated the refund policy and the customer repeated the complaint. Turn 308: the agent restated the refund policy and the customer repeated the complaint. Turn 309: the agent restated the refund policy and the customer repeated the complaint. Turn 310: the agent restated the refund policy and the customer repeated the complaint. Turn 311: the agent restated the refund policy and the customer repeated the complaint. Turn 312: the agent restated the refund policy and the customer repeated the complaint. Turn 313: the agent restated the refund policy and the customer repeated the complaint. Turn 314: the agent restated the refund policy and the customer repeated the complaint. Turn 315: the agent restated the refund policy and the customer repeated the complaint. Turn 316: the agent restated the refund policy and the customer repeated the complaint. Turn 317: the agent restated the refund policy and the customer repeated the complaint. Turn 318: the agent restated the refund policy and the customer repeated the complaint. Turn 319: the agent restated the refund policy and the customer repeated the complaint. Turn 320: the agent restated the refund policy and the customer repeated the complaint. Turn 321: the agent restated the refund policy and the customer repeated the complaint. Turn 322: the agent restated the refund policy and the customer repeated the complaint. Turn 323: the agent restated the refund policy and the customer repeated the complaint. Turn 324: the agent restated the refund policy and the customer repeated the complaint. Turn 325: the agent restated the refund policy and the customer repeated the complaint. Turn 326: the agent restated the refund policy and the customer repeated the complaint. Turn 327: the agent restated the refund policy and the customer repeated the complaint. Turn 328: the agent restated the refund policy and the customer repeated the complaint. Turn 329: the agent restated the refund policy and the customer repeated the complaint. Turn 330: the agent restated the refund policy and the customer repeated the complaint. Turn 331: the agent restated the refund policy and the customer repeated the complaint. Turn 332: the agent restated the refund policy and the customer repeated the complaint. Turn 333: the agent restated the refund policy and the customer repeated the complaint. Turn 334: the agent restated the refund policy and the customer repeated the complaint. Turn 335: the agent restated the refund policy and the customer repeated the complaint. Turn 336: the agent restated the refund policy and the customer repeated the complaint. Turn 337: the agent restated the refund policy and the customer repeated the complaint. Turn 338: the agent restated the refund policy and the customer repeated the complaint. Turn 339: the agent restated the refund policy and the customer repeated the complaint. Turn 340: the agent restated the refund policy and the customer repeated the complaint. Turn 341: the agent restated the refund policy and the customer repeated the complaint. Turn 342: the agent restated the refund policy and the customer repeated the complaint. Turn 343: the agent restated the refund policy and the customer repeated the complaint. Turn 344: the agent restated the refund policy and the customer repeated the complaint. Turn 345: the agent restated the refund policy and the customer repeated the complaint. Turn 346: the agent restated the refund policy and the customer repeated the complaint. Turn 347: the agent restated the refund policy and the customer repeated the complaint. Turn 348: the agent restated the refund policy and the customer repeated the complaint. Turn 349: the agent restated the refund policy and the customer repeated the complaint. Turn 350: the agent restated the refund policy and the customer repeated the complaint. Turn 351: the agent restated the refund policy and the customer repeated the complaint. Turn 352: the agent restated the refund policy and the customer repeated the complaint. Turn 353: the agent restated the refund policy and the customer repeated the complaint. Turn 354: the agent restated the refund policy and the customer repeated the complaint. Turn 355: the agent restated the refund policy and the customer repeated the complaint. Turn 356: the agent restated the refund policy and the customer repeated the complaint. Turn 357: the agent restated the refund policy and the customer repeated the complaint. Turn 358: the agent restated the refund policy and the customer repeated the complaint. Turn 359: the agent restated the refund policy and the customer repeated the complaint. Turn 360: the agent restated the refund policy and the customer repeated the complaint. Turn 361: the agent restated the refund policy and the customer repeated the complaint. Turn 362: the agent restated the refund policy and the customer repeated the complaint. Turn 363: the agent restated the refund policy and the customer repeated the complaint. Turn 364: the agent restated the refund policy and the customer repeated the complaint. Turn 365: the agent restated the refund policy and the customer repeated the complaint. Turn 366: the agent restated the refund policy and the customer repeated the complaint. Turn 367: the agent restated the refund policy and the customer repeated the complaint. Turn 368: the agent restated the refund policy and the customer repeated the complaint. Turn 369: the agent restated the refund policy and the customer repeated the complaint. Turn 370: the agent restated the refund policy and the customer repeated the complaint. Turn 371: the agent restated the refund policy and the customer repeated the complaint. Turn 372: the agent restated the refund policy and the customer repeated the complaint. Turn 373: the agent restated the refund policy and the customer repeated the complaint. Turn 374: the agent restated the refund policy and the customer repeated the complaint. Turn 375: the agent restated the refund policy and the customer repeated the complaint. Turn 376: the agent restated the refund policy and the customer repeated the complaint. Turn 377: the agent restated the refund policy and the customer repeated the complaint. Turn 378: the agent restated the refund policy and the customer repeated the complaint. Turn 379: the agent restated the refund policy and the customer repeated the complaint. Turn 380: the agent restated the refund policy and the customer repeated the complaint. Turn 381: the agent restated the refund policy and the customer repeated the complaint. Turn 382: the agent restated the refund policy and the customer repeated the complaint. Turn 383: the agent restated the refund policy and the customer repeated the complaint. Turn 384: the agent restated the refund policy and the customer repeated the complaint. Turn 385: the agent restated the refund policy and the customer repeated the complaint. Turn 386: the agent restated the refund policy and the customer repeated the complaint. Turn 387: the agent restated the refund policy and the customer repeated the complaint. Turn 388: the agent restated the refund policy and the customer repeated the complaint. Turn 389: the agent restated the refund policy and the customer repeated the complaint. Turn 390: the agent restated the refund policy and the customer repeated the complaint. Turn 391: the agent restated the refund policy and the customer repeated the complaint. Turn 392: the agent restated the refund policy and the customer repeated the complaint. Turn 393: the agent restated the refund policy and the customer repeated the complaint. Turn 394: the agent restated the refund policy and the customer repeated the complaint. Turn 395: the agent restated the refund policy and the customer repeated the complaint. Turn 396: the agent restated the refund policy and the customer repeated the complaint. Turn 397: the agent restated the refund policy and the customer repeated the complaint. Turn 398: the agent restated the refund policy and the customer repeated the complaint. Turn 399: the agent restated the refund policy and the customer repeated the complaint.",
+ "The app crashes on login since yesterday's update.",
+ "ok"
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3
+ },
+ "state_widths": [
+ 90,
+ 44,
+ 147,
+ 73,
+ 1024,
+ 45,
+ 35
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9999,
+ "technical": 0.0,
+ "other": 0.0
+ },
+ "confidence": 0.9993,
+ "answer_confidence": 0.9999,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6231,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0528,
+ "1": 0.2714,
+ "2": 0.6759
+ },
+ "confidence": 0.2956,
+ "answer_confidence": 0.6759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0067,
+ "confidence": 0.9933,
+ "answer_confidence": 0.9933,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 126,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2098,
+ "technical": 0.0637,
+ "other": 0.7266
+ },
+ "confidence": 0.331,
+ "answer_confidence": 0.7266,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3977,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0308,
+ "1": 0.5407,
+ "2": 0.4285
+ },
+ "confidence": 0.2692,
+ "answer_confidence": 0.5407,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1873,
+ "confidence": 0.8127,
+ "answer_confidence": 0.8127,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 435,
+ "output_tokens": 0,
+ "state_tokens": 113,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6466,
+ "technical": 0.0903,
+ "other": 0.2631
+ },
+ "confidence": 0.2259,
+ "answer_confidence": 0.6466,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0493,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1356,
+ "1": 0.6796,
+ "2": 0.1849
+ },
+ "confidence": 0.2304,
+ "answer_confidence": 0.6796,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0166,
+ "confidence": 0.9834,
+ "answer_confidence": 0.9834,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 213,
+ "output_tokens": 0,
+ "state_tokens": 39,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2111,
+ "technical": 0.0685,
+ "other": 0.7204
+ },
+ "confidence": 0.319,
+ "answer_confidence": 0.7204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1106,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2027,
+ "1": 0.484,
+ "2": 0.3133
+ },
+ "confidence": 0.0549,
+ "answer_confidence": 0.484,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8812,
+ "confidence": 0.8812,
+ "answer_confidence": 0.8812,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 7894,
+ "state_tokens_dropped": 6904,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.0015,
+ "technical": 0.962,
+ "other": 0.0365
+ },
+ "confidence": 0.8474,
+ "answer_confidence": 0.962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1251,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0863,
+ "1": 0.7023,
+ "2": 0.2114
+ },
+ "confidence": 0.2826,
+ "answer_confidence": 0.7023,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0344,
+ "confidence": 0.9656,
+ "answer_confidence": 0.9656,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 129,
+ "output_tokens": 0,
+ "state_tokens": 11,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7941,
+ "technical": 0.0972,
+ "other": 0.1087
+ },
+ "confidence": 0.4075,
+ "answer_confidence": 0.7941,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.006,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2485,
+ "1": 0.4971,
+ "2": 0.2545
+ },
+ "confidence": 0.0518,
+ "answer_confidence": 0.4971,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0083,
+ "confidence": 0.9917,
+ "answer_confidence": 0.9917,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 99,
+ "output_tokens": 0,
+ "state_tokens": 1,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 147
+ ],
+ [
+ 9,
+ 1024
+ ],
+ [
+ 3,
+ 35
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_duplicate_states.json b/laya-rust/tests/golden/multilingual/case_batch_duplicate_states.json
new file mode 100644
index 00000000..0211716d
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_duplicate_states.json
@@ -0,0 +1,211 @@
+{
+ "name": "batch_duplicate_states",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 90,
+ 90,
+ 90
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 90
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_empty_both.json b/laya-rust/tests/golden/multilingual/case_batch_empty_both.json
new file mode 100644
index 00000000..8a31ab3d
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_empty_both.json
@@ -0,0 +1,9 @@
+{
+ "name": "batch_empty_both",
+ "states": [],
+ "questions": {},
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_empty_questions.json b/laya-rust/tests/golden/multilingual/case_batch_empty_questions.json
new file mode 100644
index 00000000..ea34701f
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_empty_questions.json
@@ -0,0 +1,29 @@
+{
+ "name": "batch_empty_questions",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ ""
+ ],
+ "questions": {},
+ "options": {},
+ "state_widths": [],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ }
+ ],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_empty_questions_with_options.json b/laya-rust/tests/golden/multilingual/case_batch_empty_questions_with_options.json
new file mode 100644
index 00000000..d646aae6
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_empty_questions_with_options.json
@@ -0,0 +1,45 @@
+{
+ "name": "batch_empty_questions_with_options",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ }
+ ],
+ "questions": {},
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ }
+ ],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_empty_states.json b/laya-rust/tests/golden/multilingual/case_batch_empty_states.json
new file mode 100644
index 00000000..eabd6424
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_empty_states.json
@@ -0,0 +1,32 @@
+{
+ "name": "batch_empty_states",
+ "states": [],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_empty_states_invalid_questions.json b/laya-rust/tests/golden/multilingual/case_batch_empty_states_invalid_questions.json
new file mode 100644
index 00000000..d8c82752
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_empty_states_invalid_questions.json
@@ -0,0 +1,316 @@
+{
+ "name": "batch_empty_states_invalid_questions",
+ "states": [],
+ "questions": {
+ "too_wide": {
+ "type": "choice",
+ "instructions": "Too many options to fit the sequence.",
+ "criteria": {
+ "opt_000": "description 0",
+ "opt_001": "description 1",
+ "opt_002": "description 2",
+ "opt_003": "description 3",
+ "opt_004": "description 4",
+ "opt_005": "description 5",
+ "opt_006": "description 6",
+ "opt_007": "description 7",
+ "opt_008": "description 8",
+ "opt_009": "description 9",
+ "opt_010": "description 10",
+ "opt_011": "description 11",
+ "opt_012": "description 12",
+ "opt_013": "description 13",
+ "opt_014": "description 14",
+ "opt_015": "description 15",
+ "opt_016": "description 16",
+ "opt_017": "description 17",
+ "opt_018": "description 18",
+ "opt_019": "description 19",
+ "opt_020": "description 20",
+ "opt_021": "description 21",
+ "opt_022": "description 22",
+ "opt_023": "description 23",
+ "opt_024": "description 24",
+ "opt_025": "description 25",
+ "opt_026": "description 26",
+ "opt_027": "description 27",
+ "opt_028": "description 28",
+ "opt_029": "description 29",
+ "opt_030": "description 30",
+ "opt_031": "description 31",
+ "opt_032": "description 32",
+ "opt_033": "description 33",
+ "opt_034": "description 34",
+ "opt_035": "description 35",
+ "opt_036": "description 36",
+ "opt_037": "description 37",
+ "opt_038": "description 38",
+ "opt_039": "description 39",
+ "opt_040": "description 40",
+ "opt_041": "description 41",
+ "opt_042": "description 42",
+ "opt_043": "description 43",
+ "opt_044": "description 44",
+ "opt_045": "description 45",
+ "opt_046": "description 46",
+ "opt_047": "description 47",
+ "opt_048": "description 48",
+ "opt_049": "description 49",
+ "opt_050": "description 50",
+ "opt_051": "description 51",
+ "opt_052": "description 52",
+ "opt_053": "description 53",
+ "opt_054": "description 54",
+ "opt_055": "description 55",
+ "opt_056": "description 56",
+ "opt_057": "description 57",
+ "opt_058": "description 58",
+ "opt_059": "description 59",
+ "opt_060": "description 60",
+ "opt_061": "description 61",
+ "opt_062": "description 62",
+ "opt_063": "description 63",
+ "opt_064": "description 64",
+ "opt_065": "description 65",
+ "opt_066": "description 66",
+ "opt_067": "description 67",
+ "opt_068": "description 68",
+ "opt_069": "description 69",
+ "opt_070": "description 70",
+ "opt_071": "description 71",
+ "opt_072": "description 72",
+ "opt_073": "description 73",
+ "opt_074": "description 74",
+ "opt_075": "description 75",
+ "opt_076": "description 76",
+ "opt_077": "description 77",
+ "opt_078": "description 78",
+ "opt_079": "description 79",
+ "opt_080": "description 80",
+ "opt_081": "description 81",
+ "opt_082": "description 82",
+ "opt_083": "description 83",
+ "opt_084": "description 84",
+ "opt_085": "description 85",
+ "opt_086": "description 86",
+ "opt_087": "description 87",
+ "opt_088": "description 88",
+ "opt_089": "description 89",
+ "opt_090": "description 90",
+ "opt_091": "description 91",
+ "opt_092": "description 92",
+ "opt_093": "description 93",
+ "opt_094": "description 94",
+ "opt_095": "description 95",
+ "opt_096": "description 96",
+ "opt_097": "description 97",
+ "opt_098": "description 98",
+ "opt_099": "description 99",
+ "opt_100": "description 100",
+ "opt_101": "description 101",
+ "opt_102": "description 102",
+ "opt_103": "description 103",
+ "opt_104": "description 104",
+ "opt_105": "description 105",
+ "opt_106": "description 106",
+ "opt_107": "description 107",
+ "opt_108": "description 108",
+ "opt_109": "description 109",
+ "opt_110": "description 110",
+ "opt_111": "description 111",
+ "opt_112": "description 112",
+ "opt_113": "description 113",
+ "opt_114": "description 114",
+ "opt_115": "description 115",
+ "opt_116": "description 116",
+ "opt_117": "description 117",
+ "opt_118": "description 118",
+ "opt_119": "description 119",
+ "opt_120": "description 120",
+ "opt_121": "description 121",
+ "opt_122": "description 122",
+ "opt_123": "description 123",
+ "opt_124": "description 124",
+ "opt_125": "description 125",
+ "opt_126": "description 126",
+ "opt_127": "description 127",
+ "opt_128": "description 128",
+ "opt_129": "description 129",
+ "opt_130": "description 130",
+ "opt_131": "description 131",
+ "opt_132": "description 132",
+ "opt_133": "description 133",
+ "opt_134": "description 134",
+ "opt_135": "description 135",
+ "opt_136": "description 136",
+ "opt_137": "description 137",
+ "opt_138": "description 138",
+ "opt_139": "description 139",
+ "opt_140": "description 140",
+ "opt_141": "description 141",
+ "opt_142": "description 142",
+ "opt_143": "description 143",
+ "opt_144": "description 144",
+ "opt_145": "description 145",
+ "opt_146": "description 146",
+ "opt_147": "description 147",
+ "opt_148": "description 148",
+ "opt_149": "description 149",
+ "opt_150": "description 150",
+ "opt_151": "description 151",
+ "opt_152": "description 152",
+ "opt_153": "description 153",
+ "opt_154": "description 154",
+ "opt_155": "description 155",
+ "opt_156": "description 156",
+ "opt_157": "description 157",
+ "opt_158": "description 158",
+ "opt_159": "description 159",
+ "opt_160": "description 160",
+ "opt_161": "description 161",
+ "opt_162": "description 162",
+ "opt_163": "description 163",
+ "opt_164": "description 164",
+ "opt_165": "description 165",
+ "opt_166": "description 166",
+ "opt_167": "description 167",
+ "opt_168": "description 168",
+ "opt_169": "description 169",
+ "opt_170": "description 170",
+ "opt_171": "description 171",
+ "opt_172": "description 172",
+ "opt_173": "description 173",
+ "opt_174": "description 174",
+ "opt_175": "description 175",
+ "opt_176": "description 176",
+ "opt_177": "description 177",
+ "opt_178": "description 178",
+ "opt_179": "description 179",
+ "opt_180": "description 180",
+ "opt_181": "description 181",
+ "opt_182": "description 182",
+ "opt_183": "description 183",
+ "opt_184": "description 184",
+ "opt_185": "description 185",
+ "opt_186": "description 186",
+ "opt_187": "description 187",
+ "opt_188": "description 188",
+ "opt_189": "description 189",
+ "opt_190": "description 190",
+ "opt_191": "description 191",
+ "opt_192": "description 192",
+ "opt_193": "description 193",
+ "opt_194": "description 194",
+ "opt_195": "description 195",
+ "opt_196": "description 196",
+ "opt_197": "description 197",
+ "opt_198": "description 198",
+ "opt_199": "description 199",
+ "opt_200": "description 200",
+ "opt_201": "description 201",
+ "opt_202": "description 202",
+ "opt_203": "description 203",
+ "opt_204": "description 204",
+ "opt_205": "description 205",
+ "opt_206": "description 206",
+ "opt_207": "description 207",
+ "opt_208": "description 208",
+ "opt_209": "description 209",
+ "opt_210": "description 210",
+ "opt_211": "description 211",
+ "opt_212": "description 212",
+ "opt_213": "description 213",
+ "opt_214": "description 214",
+ "opt_215": "description 215",
+ "opt_216": "description 216",
+ "opt_217": "description 217",
+ "opt_218": "description 218",
+ "opt_219": "description 219",
+ "opt_220": "description 220",
+ "opt_221": "description 221",
+ "opt_222": "description 222",
+ "opt_223": "description 223",
+ "opt_224": "description 224",
+ "opt_225": "description 225",
+ "opt_226": "description 226",
+ "opt_227": "description 227",
+ "opt_228": "description 228",
+ "opt_229": "description 229",
+ "opt_230": "description 230",
+ "opt_231": "description 231",
+ "opt_232": "description 232",
+ "opt_233": "description 233",
+ "opt_234": "description 234",
+ "opt_235": "description 235",
+ "opt_236": "description 236",
+ "opt_237": "description 237",
+ "opt_238": "description 238",
+ "opt_239": "description 239",
+ "opt_240": "description 240",
+ "opt_241": "description 241",
+ "opt_242": "description 242",
+ "opt_243": "description 243",
+ "opt_244": "description 244",
+ "opt_245": "description 245",
+ "opt_246": "description 246",
+ "opt_247": "description 247",
+ "opt_248": "description 248",
+ "opt_249": "description 249",
+ "opt_250": "description 250",
+ "opt_251": "description 251",
+ "opt_252": "description 252",
+ "opt_253": "description 253",
+ "opt_254": "description 254",
+ "opt_255": "description 255",
+ "opt_256": "description 256",
+ "opt_257": "description 257",
+ "opt_258": "description 258",
+ "opt_259": "description 259",
+ "opt_260": "description 260",
+ "opt_261": "description 261",
+ "opt_262": "description 262",
+ "opt_263": "description 263",
+ "opt_264": "description 264",
+ "opt_265": "description 265",
+ "opt_266": "description 266",
+ "opt_267": "description 267",
+ "opt_268": "description 268",
+ "opt_269": "description 269",
+ "opt_270": "description 270",
+ "opt_271": "description 271",
+ "opt_272": "description 272",
+ "opt_273": "description 273",
+ "opt_274": "description 274",
+ "opt_275": "description 275",
+ "opt_276": "description 276",
+ "opt_277": "description 277",
+ "opt_278": "description 278",
+ "opt_279": "description 279",
+ "opt_280": "description 280",
+ "opt_281": "description 281",
+ "opt_282": "description 282",
+ "opt_283": "description 283",
+ "opt_284": "description 284",
+ "opt_285": "description 285",
+ "opt_286": "description 286",
+ "opt_287": "description 287",
+ "opt_288": "description 288",
+ "opt_289": "description 289",
+ "opt_290": "description 290",
+ "opt_291": "description 291",
+ "opt_292": "description 292",
+ "opt_293": "description 293",
+ "opt_294": "description 294",
+ "opt_295": "description 295",
+ "opt_296": "description 296",
+ "opt_297": "description 297",
+ "opt_298": "description 298",
+ "opt_299": "description 299"
+ }
+ }
+ },
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_mixed.json b/laya-rust/tests/golden/multilingual/case_batch_mixed.json
new file mode 100644
index 00000000..bfe56925
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_mixed.json
@@ -0,0 +1,515 @@
+{
+ "name": "batch_mixed",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 90,
+ 113,
+ 143,
+ 34,
+ 150,
+ 42,
+ 90,
+ 48
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6527,
+ "technical": 0.0268,
+ "other": 0.3205
+ },
+ "confidence": 0.3264,
+ "answer_confidence": 0.6527,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4526,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0142,
+ "1": 0.519,
+ "2": 0.4668
+ },
+ "confidence": 0.3115,
+ "answer_confidence": 0.519,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3678,
+ "confidence": 0.6322,
+ "answer_confidence": 0.6322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 333,
+ "output_tokens": 0,
+ "state_tokens": 79,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2035,
+ "technical": 0.1274,
+ "other": 0.6691
+ },
+ "confidence": 0.2215,
+ "answer_confidence": 0.6691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3642,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0252,
+ "1": 0.5854,
+ "2": 0.3894
+ },
+ "confidence": 0.2959,
+ "answer_confidence": 0.5854,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0571,
+ "confidence": 0.9429,
+ "answer_confidence": 0.9429,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 423,
+ "output_tokens": 0,
+ "state_tokens": 109,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.806,
+ "technical": 0.1113,
+ "other": 0.0827
+ },
+ "confidence": 0.4317,
+ "answer_confidence": 0.806,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1501,
+ "1": 0.5664,
+ "2": 0.2835
+ },
+ "confidence": 0.1225,
+ "answer_confidence": 0.5664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0063,
+ "confidence": 0.9937,
+ "answer_confidence": 0.9937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 96,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.0595,
+ "technical": 0.0895,
+ "other": 0.851
+ },
+ "confidence": 0.5256,
+ "answer_confidence": 0.851,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3418,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1574,
+ "1": 0.3433,
+ "2": 0.4993
+ },
+ "confidence": 0.0853,
+ "answer_confidence": 0.4993,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0056,
+ "confidence": 0.9944,
+ "answer_confidence": 0.9944,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.468,
+ "technical": 0.301,
+ "other": 0.231
+ },
+ "confidence": 0.0395,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8737,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0305,
+ "1": 0.0652,
+ "2": 0.9042
+ },
+ "confidence": 0.6581,
+ "answer_confidence": 0.9042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4844,
+ "confidence": 0.5156,
+ "answer_confidence": 0.5156,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9822,
+ "technical": 0.0057,
+ "other": 0.012
+ },
+ "confidence": 0.9086,
+ "answer_confidence": 0.9822,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7021,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0303,
+ "1": 0.2373,
+ "2": 0.7324
+ },
+ "confidence": 0.3852,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1059,
+ "confidence": 0.8941,
+ "answer_confidence": 0.8941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 138,
+ "output_tokens": 0,
+ "state_tokens": 14,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 150
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_none_state.json b/laya-rust/tests/golden/multilingual/case_batch_none_state.json
new file mode 100644
index 00000000..560be81a
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_none_state.json
@@ -0,0 +1,34 @@
+{
+ "name": "batch_none_state",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ null
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "error_type": "TypeError",
+ "error_message": "state must not be None; pass a string, dict, or list"
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_one_state.json b/laya-rust/tests/golden/multilingual/case_batch_one_state.json
new file mode 100644
index 00000000..c000e321
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_one_state.json
@@ -0,0 +1,97 @@
+{
+ "name": "batch_one_state",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 90
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 90
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_options_exceed_head.json b/laya-rust/tests/golden/multilingual/case_batch_options_exceed_head.json
new file mode 100644
index 00000000..1c4ae386
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_options_exceed_head.json
@@ -0,0 +1,318 @@
+{
+ "name": "batch_options_exceed_head",
+ "states": [
+ "short",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "too_wide": {
+ "type": "choice",
+ "instructions": "Too many options to fit the sequence.",
+ "criteria": {
+ "opt_000": "description 0",
+ "opt_001": "description 1",
+ "opt_002": "description 2",
+ "opt_003": "description 3",
+ "opt_004": "description 4",
+ "opt_005": "description 5",
+ "opt_006": "description 6",
+ "opt_007": "description 7",
+ "opt_008": "description 8",
+ "opt_009": "description 9",
+ "opt_010": "description 10",
+ "opt_011": "description 11",
+ "opt_012": "description 12",
+ "opt_013": "description 13",
+ "opt_014": "description 14",
+ "opt_015": "description 15",
+ "opt_016": "description 16",
+ "opt_017": "description 17",
+ "opt_018": "description 18",
+ "opt_019": "description 19",
+ "opt_020": "description 20",
+ "opt_021": "description 21",
+ "opt_022": "description 22",
+ "opt_023": "description 23",
+ "opt_024": "description 24",
+ "opt_025": "description 25",
+ "opt_026": "description 26",
+ "opt_027": "description 27",
+ "opt_028": "description 28",
+ "opt_029": "description 29",
+ "opt_030": "description 30",
+ "opt_031": "description 31",
+ "opt_032": "description 32",
+ "opt_033": "description 33",
+ "opt_034": "description 34",
+ "opt_035": "description 35",
+ "opt_036": "description 36",
+ "opt_037": "description 37",
+ "opt_038": "description 38",
+ "opt_039": "description 39",
+ "opt_040": "description 40",
+ "opt_041": "description 41",
+ "opt_042": "description 42",
+ "opt_043": "description 43",
+ "opt_044": "description 44",
+ "opt_045": "description 45",
+ "opt_046": "description 46",
+ "opt_047": "description 47",
+ "opt_048": "description 48",
+ "opt_049": "description 49",
+ "opt_050": "description 50",
+ "opt_051": "description 51",
+ "opt_052": "description 52",
+ "opt_053": "description 53",
+ "opt_054": "description 54",
+ "opt_055": "description 55",
+ "opt_056": "description 56",
+ "opt_057": "description 57",
+ "opt_058": "description 58",
+ "opt_059": "description 59",
+ "opt_060": "description 60",
+ "opt_061": "description 61",
+ "opt_062": "description 62",
+ "opt_063": "description 63",
+ "opt_064": "description 64",
+ "opt_065": "description 65",
+ "opt_066": "description 66",
+ "opt_067": "description 67",
+ "opt_068": "description 68",
+ "opt_069": "description 69",
+ "opt_070": "description 70",
+ "opt_071": "description 71",
+ "opt_072": "description 72",
+ "opt_073": "description 73",
+ "opt_074": "description 74",
+ "opt_075": "description 75",
+ "opt_076": "description 76",
+ "opt_077": "description 77",
+ "opt_078": "description 78",
+ "opt_079": "description 79",
+ "opt_080": "description 80",
+ "opt_081": "description 81",
+ "opt_082": "description 82",
+ "opt_083": "description 83",
+ "opt_084": "description 84",
+ "opt_085": "description 85",
+ "opt_086": "description 86",
+ "opt_087": "description 87",
+ "opt_088": "description 88",
+ "opt_089": "description 89",
+ "opt_090": "description 90",
+ "opt_091": "description 91",
+ "opt_092": "description 92",
+ "opt_093": "description 93",
+ "opt_094": "description 94",
+ "opt_095": "description 95",
+ "opt_096": "description 96",
+ "opt_097": "description 97",
+ "opt_098": "description 98",
+ "opt_099": "description 99",
+ "opt_100": "description 100",
+ "opt_101": "description 101",
+ "opt_102": "description 102",
+ "opt_103": "description 103",
+ "opt_104": "description 104",
+ "opt_105": "description 105",
+ "opt_106": "description 106",
+ "opt_107": "description 107",
+ "opt_108": "description 108",
+ "opt_109": "description 109",
+ "opt_110": "description 110",
+ "opt_111": "description 111",
+ "opt_112": "description 112",
+ "opt_113": "description 113",
+ "opt_114": "description 114",
+ "opt_115": "description 115",
+ "opt_116": "description 116",
+ "opt_117": "description 117",
+ "opt_118": "description 118",
+ "opt_119": "description 119",
+ "opt_120": "description 120",
+ "opt_121": "description 121",
+ "opt_122": "description 122",
+ "opt_123": "description 123",
+ "opt_124": "description 124",
+ "opt_125": "description 125",
+ "opt_126": "description 126",
+ "opt_127": "description 127",
+ "opt_128": "description 128",
+ "opt_129": "description 129",
+ "opt_130": "description 130",
+ "opt_131": "description 131",
+ "opt_132": "description 132",
+ "opt_133": "description 133",
+ "opt_134": "description 134",
+ "opt_135": "description 135",
+ "opt_136": "description 136",
+ "opt_137": "description 137",
+ "opt_138": "description 138",
+ "opt_139": "description 139",
+ "opt_140": "description 140",
+ "opt_141": "description 141",
+ "opt_142": "description 142",
+ "opt_143": "description 143",
+ "opt_144": "description 144",
+ "opt_145": "description 145",
+ "opt_146": "description 146",
+ "opt_147": "description 147",
+ "opt_148": "description 148",
+ "opt_149": "description 149",
+ "opt_150": "description 150",
+ "opt_151": "description 151",
+ "opt_152": "description 152",
+ "opt_153": "description 153",
+ "opt_154": "description 154",
+ "opt_155": "description 155",
+ "opt_156": "description 156",
+ "opt_157": "description 157",
+ "opt_158": "description 158",
+ "opt_159": "description 159",
+ "opt_160": "description 160",
+ "opt_161": "description 161",
+ "opt_162": "description 162",
+ "opt_163": "description 163",
+ "opt_164": "description 164",
+ "opt_165": "description 165",
+ "opt_166": "description 166",
+ "opt_167": "description 167",
+ "opt_168": "description 168",
+ "opt_169": "description 169",
+ "opt_170": "description 170",
+ "opt_171": "description 171",
+ "opt_172": "description 172",
+ "opt_173": "description 173",
+ "opt_174": "description 174",
+ "opt_175": "description 175",
+ "opt_176": "description 176",
+ "opt_177": "description 177",
+ "opt_178": "description 178",
+ "opt_179": "description 179",
+ "opt_180": "description 180",
+ "opt_181": "description 181",
+ "opt_182": "description 182",
+ "opt_183": "description 183",
+ "opt_184": "description 184",
+ "opt_185": "description 185",
+ "opt_186": "description 186",
+ "opt_187": "description 187",
+ "opt_188": "description 188",
+ "opt_189": "description 189",
+ "opt_190": "description 190",
+ "opt_191": "description 191",
+ "opt_192": "description 192",
+ "opt_193": "description 193",
+ "opt_194": "description 194",
+ "opt_195": "description 195",
+ "opt_196": "description 196",
+ "opt_197": "description 197",
+ "opt_198": "description 198",
+ "opt_199": "description 199",
+ "opt_200": "description 200",
+ "opt_201": "description 201",
+ "opt_202": "description 202",
+ "opt_203": "description 203",
+ "opt_204": "description 204",
+ "opt_205": "description 205",
+ "opt_206": "description 206",
+ "opt_207": "description 207",
+ "opt_208": "description 208",
+ "opt_209": "description 209",
+ "opt_210": "description 210",
+ "opt_211": "description 211",
+ "opt_212": "description 212",
+ "opt_213": "description 213",
+ "opt_214": "description 214",
+ "opt_215": "description 215",
+ "opt_216": "description 216",
+ "opt_217": "description 217",
+ "opt_218": "description 218",
+ "opt_219": "description 219",
+ "opt_220": "description 220",
+ "opt_221": "description 221",
+ "opt_222": "description 222",
+ "opt_223": "description 223",
+ "opt_224": "description 224",
+ "opt_225": "description 225",
+ "opt_226": "description 226",
+ "opt_227": "description 227",
+ "opt_228": "description 228",
+ "opt_229": "description 229",
+ "opt_230": "description 230",
+ "opt_231": "description 231",
+ "opt_232": "description 232",
+ "opt_233": "description 233",
+ "opt_234": "description 234",
+ "opt_235": "description 235",
+ "opt_236": "description 236",
+ "opt_237": "description 237",
+ "opt_238": "description 238",
+ "opt_239": "description 239",
+ "opt_240": "description 240",
+ "opt_241": "description 241",
+ "opt_242": "description 242",
+ "opt_243": "description 243",
+ "opt_244": "description 244",
+ "opt_245": "description 245",
+ "opt_246": "description 246",
+ "opt_247": "description 247",
+ "opt_248": "description 248",
+ "opt_249": "description 249",
+ "opt_250": "description 250",
+ "opt_251": "description 251",
+ "opt_252": "description 252",
+ "opt_253": "description 253",
+ "opt_254": "description 254",
+ "opt_255": "description 255",
+ "opt_256": "description 256",
+ "opt_257": "description 257",
+ "opt_258": "description 258",
+ "opt_259": "description 259",
+ "opt_260": "description 260",
+ "opt_261": "description 261",
+ "opt_262": "description 262",
+ "opt_263": "description 263",
+ "opt_264": "description 264",
+ "opt_265": "description 265",
+ "opt_266": "description 266",
+ "opt_267": "description 267",
+ "opt_268": "description 268",
+ "opt_269": "description 269",
+ "opt_270": "description 270",
+ "opt_271": "description 271",
+ "opt_272": "description 272",
+ "opt_273": "description 273",
+ "opt_274": "description 274",
+ "opt_275": "description 275",
+ "opt_276": "description 276",
+ "opt_277": "description 277",
+ "opt_278": "description 278",
+ "opt_279": "description 279",
+ "opt_280": "description 280",
+ "opt_281": "description 281",
+ "opt_282": "description 282",
+ "opt_283": "description 283",
+ "opt_284": "description 284",
+ "opt_285": "description 285",
+ "opt_286": "description 286",
+ "opt_287": "description 287",
+ "opt_288": "description 288",
+ "opt_289": "description 289",
+ "opt_290": "description 290",
+ "opt_291": "description 291",
+ "opt_292": "description 292",
+ "opt_293": "description 293",
+ "opt_294": "description 294",
+ "opt_295": "description 295",
+ "opt_296": "description 296",
+ "opt_297": "description 297",
+ "opt_298": "description 298",
+ "opt_299": "description 299"
+ }
+ }
+ },
+ "options": {},
+ "error_type": "ValueError",
+ "error_message": "question 'too_wide': only 254 of its 300 option markers fit in max_len=1024 with head_max_len=256 spent on the question; lower head_max_len, raise max_len, or use fewer options"
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_single_question.json b/laya-rust/tests/golden/multilingual/case_batch_single_question.json
new file mode 100644
index 00000000..f5a311b6
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_single_question.json
@@ -0,0 +1,137 @@
+{
+ "name": "batch_single_question",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ]
+ ],
+ "questions": {
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 90,
+ 44,
+ 147,
+ 73
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 90,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0067,
+ "confidence": 0.9933,
+ "answer_confidence": 0.9933,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 44,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1873,
+ "confidence": 0.8127,
+ "answer_confidence": 0.8127,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 147,
+ "output_tokens": 0,
+ "state_tokens": 113,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0166,
+ "confidence": 0.9834,
+ "answer_confidence": 0.9834,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 73,
+ "output_tokens": 0,
+ "state_tokens": 39,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 2,
+ 90
+ ],
+ [
+ 2,
+ 147
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_size_1.json b/laya-rust/tests/golden/multilingual/case_batch_size_1.json
new file mode 100644
index 00000000..be4424ea
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_size_1.json
@@ -0,0 +1,545 @@
+{
+ "name": "batch_size_1",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 1
+ },
+ "state_widths": [
+ 90,
+ 113,
+ 143,
+ 34,
+ 150,
+ 42,
+ 90,
+ 48
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6527,
+ "technical": 0.0268,
+ "other": 0.3205
+ },
+ "confidence": 0.3264,
+ "answer_confidence": 0.6527,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4526,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0142,
+ "1": 0.519,
+ "2": 0.4668
+ },
+ "confidence": 0.3115,
+ "answer_confidence": 0.519,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3678,
+ "confidence": 0.6322,
+ "answer_confidence": 0.6322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 333,
+ "output_tokens": 0,
+ "state_tokens": 79,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2035,
+ "technical": 0.1274,
+ "other": 0.6691
+ },
+ "confidence": 0.2215,
+ "answer_confidence": 0.6691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3642,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0252,
+ "1": 0.5854,
+ "2": 0.3894
+ },
+ "confidence": 0.2959,
+ "answer_confidence": 0.5854,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0571,
+ "confidence": 0.9429,
+ "answer_confidence": 0.9429,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 423,
+ "output_tokens": 0,
+ "state_tokens": 109,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.806,
+ "technical": 0.1113,
+ "other": 0.0827
+ },
+ "confidence": 0.4317,
+ "answer_confidence": 0.806,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1501,
+ "1": 0.5664,
+ "2": 0.2835
+ },
+ "confidence": 0.1225,
+ "answer_confidence": 0.5664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0063,
+ "confidence": 0.9937,
+ "answer_confidence": 0.9937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 96,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.0595,
+ "technical": 0.0895,
+ "other": 0.851
+ },
+ "confidence": 0.5256,
+ "answer_confidence": 0.851,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3418,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1574,
+ "1": 0.3433,
+ "2": 0.4993
+ },
+ "confidence": 0.0853,
+ "answer_confidence": 0.4993,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0056,
+ "confidence": 0.9944,
+ "answer_confidence": 0.9944,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.468,
+ "technical": 0.301,
+ "other": 0.231
+ },
+ "confidence": 0.0395,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8737,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0305,
+ "1": 0.0652,
+ "2": 0.9042
+ },
+ "confidence": 0.6581,
+ "answer_confidence": 0.9042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4844,
+ "confidence": 0.5156,
+ "answer_confidence": 0.5156,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9822,
+ "technical": 0.0057,
+ "other": 0.012
+ },
+ "confidence": 0.9086,
+ "answer_confidence": 0.9822,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7021,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0303,
+ "1": 0.2373,
+ "2": 0.7324
+ },
+ "confidence": 0.3852,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1059,
+ "confidence": 0.8941,
+ "answer_confidence": 0.8941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 138,
+ "output_tokens": 0,
+ "state_tokens": 14,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 90
+ ],
+ [
+ 3,
+ 113
+ ],
+ [
+ 3,
+ 143
+ ],
+ [
+ 3,
+ 34
+ ],
+ [
+ 3,
+ 150
+ ],
+ [
+ 3,
+ 42
+ ],
+ [
+ 3,
+ 90
+ ],
+ [
+ 3,
+ 48
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_size_3.json b/laya-rust/tests/golden/multilingual/case_batch_size_3.json
new file mode 100644
index 00000000..c80284fa
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_size_3.json
@@ -0,0 +1,525 @@
+{
+ "name": "batch_size_3",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3
+ },
+ "state_widths": [
+ 90,
+ 113,
+ 143,
+ 34,
+ 150,
+ 42,
+ 90,
+ 48
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6527,
+ "technical": 0.0268,
+ "other": 0.3205
+ },
+ "confidence": 0.3264,
+ "answer_confidence": 0.6527,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4526,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0142,
+ "1": 0.519,
+ "2": 0.4668
+ },
+ "confidence": 0.3115,
+ "answer_confidence": 0.519,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3678,
+ "confidence": 0.6322,
+ "answer_confidence": 0.6322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 333,
+ "output_tokens": 0,
+ "state_tokens": 79,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2035,
+ "technical": 0.1274,
+ "other": 0.6691
+ },
+ "confidence": 0.2215,
+ "answer_confidence": 0.6691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3642,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0252,
+ "1": 0.5854,
+ "2": 0.3894
+ },
+ "confidence": 0.2959,
+ "answer_confidence": 0.5854,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0571,
+ "confidence": 0.9429,
+ "answer_confidence": 0.9429,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 423,
+ "output_tokens": 0,
+ "state_tokens": 109,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.806,
+ "technical": 0.1113,
+ "other": 0.0827
+ },
+ "confidence": 0.4317,
+ "answer_confidence": 0.806,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1501,
+ "1": 0.5664,
+ "2": 0.2835
+ },
+ "confidence": 0.1225,
+ "answer_confidence": 0.5664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0063,
+ "confidence": 0.9937,
+ "answer_confidence": 0.9937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 96,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.0595,
+ "technical": 0.0895,
+ "other": 0.851
+ },
+ "confidence": 0.5256,
+ "answer_confidence": 0.851,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3418,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1574,
+ "1": 0.3433,
+ "2": 0.4993
+ },
+ "confidence": 0.0853,
+ "answer_confidence": 0.4993,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0056,
+ "confidence": 0.9944,
+ "answer_confidence": 0.9944,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.468,
+ "technical": 0.301,
+ "other": 0.231
+ },
+ "confidence": 0.0395,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8737,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0305,
+ "1": 0.0652,
+ "2": 0.9042
+ },
+ "confidence": 0.6581,
+ "answer_confidence": 0.9042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4844,
+ "confidence": 0.5156,
+ "answer_confidence": 0.5156,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9822,
+ "technical": 0.0057,
+ "other": 0.012
+ },
+ "confidence": 0.9086,
+ "answer_confidence": 0.9822,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7021,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0303,
+ "1": 0.2373,
+ "2": 0.7324
+ },
+ "confidence": 0.3852,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1059,
+ "confidence": 0.8941,
+ "answer_confidence": 0.8941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 138,
+ "output_tokens": 0,
+ "state_tokens": 14,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 143
+ ],
+ [
+ 9,
+ 150
+ ],
+ [
+ 6,
+ 90
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_size_negative.json b/laya-rust/tests/golden/multilingual/case_batch_size_negative.json
new file mode 100644
index 00000000..8e676dd2
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_size_negative.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_negative",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": -2
+ },
+ "state_widths": [
+ 90,
+ 113,
+ 143,
+ 34,
+ 150,
+ 42,
+ 90,
+ 48
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6527,
+ "technical": 0.0268,
+ "other": 0.3205
+ },
+ "confidence": 0.3264,
+ "answer_confidence": 0.6527,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4526,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0142,
+ "1": 0.519,
+ "2": 0.4668
+ },
+ "confidence": 0.3115,
+ "answer_confidence": 0.519,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3678,
+ "confidence": 0.6322,
+ "answer_confidence": 0.6322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 333,
+ "output_tokens": 0,
+ "state_tokens": 79,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2035,
+ "technical": 0.1274,
+ "other": 0.6691
+ },
+ "confidence": 0.2215,
+ "answer_confidence": 0.6691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3642,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0252,
+ "1": 0.5854,
+ "2": 0.3894
+ },
+ "confidence": 0.2959,
+ "answer_confidence": 0.5854,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0571,
+ "confidence": 0.9429,
+ "answer_confidence": 0.9429,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 423,
+ "output_tokens": 0,
+ "state_tokens": 109,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.806,
+ "technical": 0.1113,
+ "other": 0.0827
+ },
+ "confidence": 0.4317,
+ "answer_confidence": 0.806,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1501,
+ "1": 0.5664,
+ "2": 0.2835
+ },
+ "confidence": 0.1225,
+ "answer_confidence": 0.5664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0063,
+ "confidence": 0.9937,
+ "answer_confidence": 0.9937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 96,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.0595,
+ "technical": 0.0895,
+ "other": 0.851
+ },
+ "confidence": 0.5256,
+ "answer_confidence": 0.851,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3418,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1574,
+ "1": 0.3433,
+ "2": 0.4993
+ },
+ "confidence": 0.0853,
+ "answer_confidence": 0.4993,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0056,
+ "confidence": 0.9944,
+ "answer_confidence": 0.9944,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.468,
+ "technical": 0.301,
+ "other": 0.231
+ },
+ "confidence": 0.0395,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8737,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0305,
+ "1": 0.0652,
+ "2": 0.9042
+ },
+ "confidence": 0.6581,
+ "answer_confidence": 0.9042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4844,
+ "confidence": 0.5156,
+ "answer_confidence": 0.5156,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9822,
+ "technical": 0.0057,
+ "other": 0.012
+ },
+ "confidence": 0.9086,
+ "answer_confidence": 0.9822,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7021,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0303,
+ "1": 0.2373,
+ "2": 0.7324
+ },
+ "confidence": 0.3852,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1059,
+ "confidence": 0.8941,
+ "answer_confidence": 0.8941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 138,
+ "output_tokens": 0,
+ "state_tokens": 14,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 150
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_size_over_len.json b/laya-rust/tests/golden/multilingual/case_batch_size_over_len.json
new file mode 100644
index 00000000..fbcc6dec
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_size_over_len.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_over_len",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 100
+ },
+ "state_widths": [
+ 90,
+ 113,
+ 143,
+ 34,
+ 150,
+ 42,
+ 90,
+ 48
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6527,
+ "technical": 0.0268,
+ "other": 0.3205
+ },
+ "confidence": 0.3264,
+ "answer_confidence": 0.6527,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4526,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0142,
+ "1": 0.519,
+ "2": 0.4668
+ },
+ "confidence": 0.3115,
+ "answer_confidence": 0.519,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3678,
+ "confidence": 0.6322,
+ "answer_confidence": 0.6322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 333,
+ "output_tokens": 0,
+ "state_tokens": 79,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2035,
+ "technical": 0.1274,
+ "other": 0.6691
+ },
+ "confidence": 0.2215,
+ "answer_confidence": 0.6691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3642,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0252,
+ "1": 0.5854,
+ "2": 0.3894
+ },
+ "confidence": 0.2959,
+ "answer_confidence": 0.5854,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0571,
+ "confidence": 0.9429,
+ "answer_confidence": 0.9429,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 423,
+ "output_tokens": 0,
+ "state_tokens": 109,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.806,
+ "technical": 0.1113,
+ "other": 0.0827
+ },
+ "confidence": 0.4317,
+ "answer_confidence": 0.806,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1501,
+ "1": 0.5664,
+ "2": 0.2835
+ },
+ "confidence": 0.1225,
+ "answer_confidence": 0.5664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0063,
+ "confidence": 0.9937,
+ "answer_confidence": 0.9937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 96,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.0595,
+ "technical": 0.0895,
+ "other": 0.851
+ },
+ "confidence": 0.5256,
+ "answer_confidence": 0.851,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3418,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1574,
+ "1": 0.3433,
+ "2": 0.4993
+ },
+ "confidence": 0.0853,
+ "answer_confidence": 0.4993,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0056,
+ "confidence": 0.9944,
+ "answer_confidence": 0.9944,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.468,
+ "technical": 0.301,
+ "other": 0.231
+ },
+ "confidence": 0.0395,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8737,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0305,
+ "1": 0.0652,
+ "2": 0.9042
+ },
+ "confidence": 0.6581,
+ "answer_confidence": 0.9042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4844,
+ "confidence": 0.5156,
+ "answer_confidence": 0.5156,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9822,
+ "technical": 0.0057,
+ "other": 0.012
+ },
+ "confidence": 0.9086,
+ "answer_confidence": 0.9822,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7021,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0303,
+ "1": 0.2373,
+ "2": 0.7324
+ },
+ "confidence": 0.3852,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1059,
+ "confidence": 0.8941,
+ "answer_confidence": 0.8941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 138,
+ "output_tokens": 0,
+ "state_tokens": 14,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 150
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_size_zero.json b/laya-rust/tests/golden/multilingual/case_batch_size_zero.json
new file mode 100644
index 00000000..40fcd782
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_size_zero.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_zero",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 0
+ },
+ "state_widths": [
+ 90,
+ 113,
+ 143,
+ 34,
+ 150,
+ 42,
+ 90,
+ 48
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6527,
+ "technical": 0.0268,
+ "other": 0.3205
+ },
+ "confidence": 0.3264,
+ "answer_confidence": 0.6527,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4526,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0142,
+ "1": 0.519,
+ "2": 0.4668
+ },
+ "confidence": 0.3115,
+ "answer_confidence": 0.519,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3678,
+ "confidence": 0.6322,
+ "answer_confidence": 0.6322,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 333,
+ "output_tokens": 0,
+ "state_tokens": 79,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2035,
+ "technical": 0.1274,
+ "other": 0.6691
+ },
+ "confidence": 0.2215,
+ "answer_confidence": 0.6691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3642,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0252,
+ "1": 0.5854,
+ "2": 0.3894
+ },
+ "confidence": 0.2959,
+ "answer_confidence": 0.5854,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0571,
+ "confidence": 0.9429,
+ "answer_confidence": 0.9429,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 423,
+ "output_tokens": 0,
+ "state_tokens": 109,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.806,
+ "technical": 0.1113,
+ "other": 0.0827
+ },
+ "confidence": 0.4317,
+ "answer_confidence": 0.806,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1501,
+ "1": 0.5664,
+ "2": 0.2835
+ },
+ "confidence": 0.1225,
+ "answer_confidence": 0.5664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0063,
+ "confidence": 0.9937,
+ "answer_confidence": 0.9937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 96,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.0595,
+ "technical": 0.0895,
+ "other": 0.851
+ },
+ "confidence": 0.5256,
+ "answer_confidence": 0.851,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3418,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1574,
+ "1": 0.3433,
+ "2": 0.4993
+ },
+ "confidence": 0.0853,
+ "answer_confidence": 0.4993,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0056,
+ "confidence": 0.9944,
+ "answer_confidence": 0.9944,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.468,
+ "technical": 0.301,
+ "other": 0.231
+ },
+ "confidence": 0.0395,
+ "answer_confidence": 0.468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8737,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0305,
+ "1": 0.0652,
+ "2": 0.9042
+ },
+ "confidence": 0.6581,
+ "answer_confidence": 0.9042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.4844,
+ "confidence": 0.5156,
+ "answer_confidence": 0.5156,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9822,
+ "technical": 0.0057,
+ "other": 0.012
+ },
+ "confidence": 0.9086,
+ "answer_confidence": 0.9822,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7021,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0303,
+ "1": 0.2373,
+ "2": 0.7324
+ },
+ "confidence": 0.3852,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1059,
+ "confidence": 0.8941,
+ "answer_confidence": 0.8941,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 138,
+ "output_tokens": 0,
+ "state_tokens": 14,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 150
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_no_batch_size.json b/laya-rust/tests/golden/multilingual/case_batch_sort_no_batch_size.json
new file mode 100644
index 00000000..e684e273
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_no_batch_size.json
@@ -0,0 +1,1182 @@
+{
+ "name": "batch_sort_no_batch_size",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 185
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_off.json b/laya-rust/tests/golden/multilingual/case_batch_sort_off.json
new file mode 100644
index 00000000..75e26b91
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_off.json
@@ -0,0 +1,1218 @@
+{
+ "name": "batch_sort_off",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 180
+ ],
+ [
+ 6,
+ 135
+ ],
+ [
+ 6,
+ 160
+ ],
+ [
+ 6,
+ 145
+ ],
+ [
+ 6,
+ 170
+ ],
+ [
+ 6,
+ 150
+ ],
+ [
+ 6,
+ 185
+ ],
+ [
+ 6,
+ 155
+ ],
+ [
+ 6,
+ 175
+ ],
+ [
+ 6,
+ 165
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_on.json b/laya-rust/tests/golden/multilingual/case_batch_sort_on.json
new file mode 100644
index 00000000..bfac7e57
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_on.json
@@ -0,0 +1,1219 @@
+{
+ "name": "batch_sort_on",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 95
+ ],
+ [
+ 6,
+ 105
+ ],
+ [
+ 6,
+ 115
+ ],
+ [
+ 6,
+ 130
+ ],
+ [
+ 6,
+ 145
+ ],
+ [
+ 6,
+ 155
+ ],
+ [
+ 6,
+ 170
+ ],
+ [
+ 6,
+ 185
+ ],
+ [
+ 6,
+ 140
+ ],
+ [
+ 6,
+ 175
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_partial_window.json b/laya-rust/tests/golden/multilingual/case_batch_sort_partial_window.json
new file mode 100644
index 00000000..de1cbb8f
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_partial_window.json
@@ -0,0 +1,218 @@
+{
+ "name": "batch_sort_partial_window",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 135
+ ],
+ [
+ 3,
+ 180
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_size_1.json b/laya-rust/tests/golden/multilingual/case_batch_sort_size_1.json
new file mode 100644
index 00000000..6e5b15b5
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_size_1.json
@@ -0,0 +1,1259 @@
+{
+ "name": "batch_sort_size_1",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 1,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 180
+ ],
+ [
+ 3,
+ 100
+ ],
+ [
+ 3,
+ 135
+ ],
+ [
+ 3,
+ 90
+ ],
+ [
+ 3,
+ 160
+ ],
+ [
+ 3,
+ 115
+ ],
+ [
+ 3,
+ 145
+ ],
+ [
+ 3,
+ 95
+ ],
+ [
+ 3,
+ 170
+ ],
+ [
+ 3,
+ 125
+ ],
+ [
+ 3,
+ 105
+ ],
+ [
+ 3,
+ 150
+ ],
+ [
+ 3,
+ 185
+ ],
+ [
+ 3,
+ 110
+ ],
+ [
+ 3,
+ 130
+ ],
+ [
+ 3,
+ 155
+ ],
+ [
+ 3,
+ 120
+ ],
+ [
+ 3,
+ 175
+ ],
+ [
+ 3,
+ 140
+ ],
+ [
+ 3,
+ 165
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_size_3.json b/laya-rust/tests/golden/multilingual/case_batch_sort_size_3.json
new file mode 100644
index 00000000..dd48171b
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_size_3.json
@@ -0,0 +1,1207 @@
+{
+ "name": "batch_sort_size_3",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 100
+ ],
+ [
+ 9,
+ 115
+ ],
+ [
+ 9,
+ 130
+ ],
+ [
+ 9,
+ 145
+ ],
+ [
+ 9,
+ 160
+ ],
+ [
+ 9,
+ 175
+ ],
+ [
+ 6,
+ 185
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_size_eq_len.json b/laya-rust/tests/golden/multilingual/case_batch_sort_size_eq_len.json
new file mode 100644
index 00000000..fd2b3384
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_size_eq_len.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_eq_len",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 20,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 185
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_size_over_len.json b/laya-rust/tests/golden/multilingual/case_batch_sort_size_over_len.json
new file mode 100644
index 00000000..ad88c803
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_size_over_len.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_over_len",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 25,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 185
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sort_size_zero.json b/laya-rust/tests/golden/multilingual/case_batch_sort_size_zero.json
new file mode 100644
index 00000000..72b95a87
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sort_size_zero.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_zero",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 0,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 180,
+ 100,
+ 135,
+ 90,
+ 160,
+ 115,
+ 145,
+ 95,
+ 170,
+ 125,
+ 105,
+ 150,
+ 185,
+ 110,
+ 130,
+ 155,
+ 120,
+ 175,
+ 140,
+ 165
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9823,
+ "technical": 0.0007,
+ "other": 0.017
+ },
+ "confidence": 0.916,
+ "answer_confidence": 0.9823,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3568,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0709,
+ "1": 0.5013,
+ "2": 0.4278
+ },
+ "confidence": 0.1834,
+ "answer_confidence": 0.5013,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0867,
+ "confidence": 0.9133,
+ "answer_confidence": 0.9133,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 534,
+ "output_tokens": 0,
+ "state_tokens": 146,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9664,
+ "technical": 0.0022,
+ "other": 0.0314
+ },
+ "confidence": 0.8589,
+ "answer_confidence": 0.9664,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0367,
+ "1": 0.7004,
+ "2": 0.2629
+ },
+ "confidence": 0.3429,
+ "answer_confidence": 0.7004,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0634,
+ "confidence": 0.9366,
+ "answer_confidence": 0.9366,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 294,
+ "output_tokens": 0,
+ "state_tokens": 66,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9708,
+ "technical": 0.0008,
+ "other": 0.0284
+ },
+ "confidence": 0.8764,
+ "answer_confidence": 0.9708,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6446,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0315,
+ "1": 0.2924,
+ "2": 0.6761
+ },
+ "confidence": 0.3327,
+ "answer_confidence": 0.6761,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0554,
+ "confidence": 0.9446,
+ "answer_confidence": 0.9446,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 399,
+ "output_tokens": 0,
+ "state_tokens": 101,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.972,
+ "technical": 0.0011,
+ "other": 0.0269
+ },
+ "confidence": 0.8798,
+ "answer_confidence": 0.972,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5259,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0331,
+ "1": 0.4079,
+ "2": 0.559
+ },
+ "confidence": 0.2684,
+ "answer_confidence": 0.559,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0972,
+ "confidence": 0.9028,
+ "answer_confidence": 0.9028,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 474,
+ "output_tokens": 0,
+ "state_tokens": 126,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9345,
+ "technical": 0.0011,
+ "other": 0.0644
+ },
+ "confidence": 0.7748,
+ "answer_confidence": 0.9345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3357,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0468,
+ "1": 0.5707,
+ "2": 0.3825
+ },
+ "confidence": 0.2436,
+ "answer_confidence": 0.5707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1013,
+ "confidence": 0.8987,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 339,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9765,
+ "technical": 0.0006,
+ "other": 0.0229
+ },
+ "confidence": 0.896,
+ "answer_confidence": 0.9765,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0311,
+ "1": 0.325,
+ "2": 0.6439
+ },
+ "confidence": 0.3113,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 429,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.8729,
+ "technical": 0.0066,
+ "other": 0.1206
+ },
+ "confidence": 0.6299,
+ "answer_confidence": 0.8729,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1993,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0317,
+ "1": 0.7373,
+ "2": 0.231
+ },
+ "confidence": 0.3878,
+ "answer_confidence": 0.7373,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0431,
+ "confidence": 0.9569,
+ "answer_confidence": 0.9569,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 279,
+ "output_tokens": 0,
+ "state_tokens": 61,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9385,
+ "technical": 0.0016,
+ "other": 0.0599
+ },
+ "confidence": 0.7827,
+ "answer_confidence": 0.9385,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3682,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0805,
+ "1": 0.4707,
+ "2": 0.4488
+ },
+ "confidence": 0.1652,
+ "answer_confidence": 0.4707,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2113,
+ "confidence": 0.7887,
+ "answer_confidence": 0.7887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 504,
+ "output_tokens": 0,
+ "state_tokens": 136,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9699,
+ "technical": 0.001,
+ "other": 0.0291
+ },
+ "confidence": 0.8732,
+ "answer_confidence": 0.9699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4181,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0389,
+ "1": 0.5042,
+ "2": 0.457
+ },
+ "confidence": 0.245,
+ "answer_confidence": 0.5042,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1113,
+ "confidence": 0.8887,
+ "answer_confidence": 0.8887,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 369,
+ "output_tokens": 0,
+ "state_tokens": 91,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.955,
+ "technical": 0.0021,
+ "other": 0.0428
+ },
+ "confidence": 0.8253,
+ "answer_confidence": 0.955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3941,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.5512,
+ "2": 0.4215
+ },
+ "confidence": 0.2801,
+ "answer_confidence": 0.5512,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.063,
+ "confidence": 0.937,
+ "answer_confidence": 0.937,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 309,
+ "output_tokens": 0,
+ "state_tokens": 71,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9867,
+ "technical": 0.0005,
+ "other": 0.0129
+ },
+ "confidence": 0.9336,
+ "answer_confidence": 0.9867,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5707,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0398,
+ "1": 0.3498,
+ "2": 0.6105
+ },
+ "confidence": 0.2746,
+ "answer_confidence": 0.6105,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0882,
+ "confidence": 0.9118,
+ "answer_confidence": 0.9118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 444,
+ "output_tokens": 0,
+ "state_tokens": 116,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9575,
+ "technical": 0.0014,
+ "other": 0.0411
+ },
+ "confidence": 0.8343,
+ "answer_confidence": 0.9575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.067,
+ "1": 0.4609,
+ "2": 0.472
+ },
+ "confidence": 0.1876,
+ "answer_confidence": 0.472,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0889,
+ "confidence": 0.9111,
+ "answer_confidence": 0.9111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 549,
+ "output_tokens": 0,
+ "state_tokens": 151,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9668,
+ "technical": 0.0012,
+ "other": 0.0319
+ },
+ "confidence": 0.8628,
+ "answer_confidence": 0.9668,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4042,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0443,
+ "1": 0.5073,
+ "2": 0.4484
+ },
+ "confidence": 0.2337,
+ "answer_confidence": 0.5073,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0683,
+ "confidence": 0.9317,
+ "answer_confidence": 0.9317,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 324,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9758,
+ "technical": 0.0008,
+ "other": 0.0234
+ },
+ "confidence": 0.8931,
+ "answer_confidence": 0.9758,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0272,
+ "1": 0.3085,
+ "2": 0.6643
+ },
+ "confidence": 0.3332,
+ "answer_confidence": 0.6643,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0753,
+ "confidence": 0.9247,
+ "answer_confidence": 0.9247,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 384,
+ "output_tokens": 0,
+ "state_tokens": 96,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9913,
+ "technical": 0.0005,
+ "other": 0.0083
+ },
+ "confidence": 0.9527,
+ "answer_confidence": 0.9913,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5763,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0328,
+ "1": 0.3582,
+ "2": 0.6091
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.6091,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.07,
+ "confidence": 0.93,
+ "answer_confidence": 0.93,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 459,
+ "output_tokens": 0,
+ "state_tokens": 121,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9702,
+ "technical": 0.0009,
+ "other": 0.0289
+ },
+ "confidence": 0.874,
+ "answer_confidence": 0.9702,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3405,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0412,
+ "1": 0.577,
+ "2": 0.3817
+ },
+ "confidence": 0.2569,
+ "answer_confidence": 0.577,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0905,
+ "confidence": 0.9095,
+ "answer_confidence": 0.9095,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 354,
+ "output_tokens": 0,
+ "state_tokens": 86,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9658,
+ "technical": 0.0016,
+ "other": 0.0326
+ },
+ "confidence": 0.8583,
+ "answer_confidence": 0.9658,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2456,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1208,
+ "1": 0.5128,
+ "2": 0.3664
+ },
+ "confidence": 0.121,
+ "answer_confidence": 0.5128,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1006,
+ "confidence": 0.8994,
+ "answer_confidence": 0.8994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 519,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9802,
+ "technical": 0.0008,
+ "other": 0.019
+ },
+ "confidence": 0.9085,
+ "answer_confidence": 0.9802,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4647,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0502,
+ "1": 0.435,
+ "2": 0.5148
+ },
+ "confidence": 0.2227,
+ "answer_confidence": 0.5148,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0599,
+ "confidence": 0.9401,
+ "answer_confidence": 0.9401,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 414,
+ "output_tokens": 0,
+ "state_tokens": 106,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9703,
+ "technical": 0.0013,
+ "other": 0.0285
+ },
+ "confidence": 0.8733,
+ "answer_confidence": 0.9703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5325,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0449,
+ "1": 0.3777,
+ "2": 0.5774
+ },
+ "confidence": 0.2498,
+ "answer_confidence": 0.5774,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1136,
+ "confidence": 0.8864,
+ "answer_confidence": 0.8864,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 489,
+ "output_tokens": 0,
+ "state_tokens": 131,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 185
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sorted.json b/laya-rust/tests/golden/multilingual/case_batch_sorted.json
new file mode 100644
index 00000000..939629e5
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sorted.json
@@ -0,0 +1,471 @@
+{
+ "name": "batch_sorted",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ],
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint. Turn 140: the agent restated the refund policy and the customer repeated the complaint. Turn 141: the agent restated the refund policy and the customer repeated the complaint. Turn 142: the agent restated the refund policy and the customer repeated the complaint. Turn 143: the agent restated the refund policy and the customer repeated the complaint. Turn 144: the agent restated the refund policy and the customer repeated the complaint. Turn 145: the agent restated the refund policy and the customer repeated the complaint. Turn 146: the agent restated the refund policy and the customer repeated the complaint. Turn 147: the agent restated the refund policy and the customer repeated the complaint. Turn 148: the agent restated the refund policy and the customer repeated the complaint. Turn 149: the agent restated the refund policy and the customer repeated the complaint. Turn 150: the agent restated the refund policy and the customer repeated the complaint. Turn 151: the agent restated the refund policy and the customer repeated the complaint. Turn 152: the agent restated the refund policy and the customer repeated the complaint. Turn 153: the agent restated the refund policy and the customer repeated the complaint. Turn 154: the agent restated the refund policy and the customer repeated the complaint. Turn 155: the agent restated the refund policy and the customer repeated the complaint. Turn 156: the agent restated the refund policy and the customer repeated the complaint. Turn 157: the agent restated the refund policy and the customer repeated the complaint. Turn 158: the agent restated the refund policy and the customer repeated the complaint. Turn 159: the agent restated the refund policy and the customer repeated the complaint. Turn 160: the agent restated the refund policy and the customer repeated the complaint. Turn 161: the agent restated the refund policy and the customer repeated the complaint. Turn 162: the agent restated the refund policy and the customer repeated the complaint. Turn 163: the agent restated the refund policy and the customer repeated the complaint. Turn 164: the agent restated the refund policy and the customer repeated the complaint. Turn 165: the agent restated the refund policy and the customer repeated the complaint. Turn 166: the agent restated the refund policy and the customer repeated the complaint. Turn 167: the agent restated the refund policy and the customer repeated the complaint. Turn 168: the agent restated the refund policy and the customer repeated the complaint. Turn 169: the agent restated the refund policy and the customer repeated the complaint. Turn 170: the agent restated the refund policy and the customer repeated the complaint. Turn 171: the agent restated the refund policy and the customer repeated the complaint. Turn 172: the agent restated the refund policy and the customer repeated the complaint. Turn 173: the agent restated the refund policy and the customer repeated the complaint. Turn 174: the agent restated the refund policy and the customer repeated the complaint. Turn 175: the agent restated the refund policy and the customer repeated the complaint. Turn 176: the agent restated the refund policy and the customer repeated the complaint. Turn 177: the agent restated the refund policy and the customer repeated the complaint. Turn 178: the agent restated the refund policy and the customer repeated the complaint. Turn 179: the agent restated the refund policy and the customer repeated the complaint. Turn 180: the agent restated the refund policy and the customer repeated the complaint. Turn 181: the agent restated the refund policy and the customer repeated the complaint. Turn 182: the agent restated the refund policy and the customer repeated the complaint. Turn 183: the agent restated the refund policy and the customer repeated the complaint. Turn 184: the agent restated the refund policy and the customer repeated the complaint. Turn 185: the agent restated the refund policy and the customer repeated the complaint. Turn 186: the agent restated the refund policy and the customer repeated the complaint. Turn 187: the agent restated the refund policy and the customer repeated the complaint. Turn 188: the agent restated the refund policy and the customer repeated the complaint. Turn 189: the agent restated the refund policy and the customer repeated the complaint. Turn 190: the agent restated the refund policy and the customer repeated the complaint. Turn 191: the agent restated the refund policy and the customer repeated the complaint. Turn 192: the agent restated the refund policy and the customer repeated the complaint. Turn 193: the agent restated the refund policy and the customer repeated the complaint. Turn 194: the agent restated the refund policy and the customer repeated the complaint. Turn 195: the agent restated the refund policy and the customer repeated the complaint. Turn 196: the agent restated the refund policy and the customer repeated the complaint. Turn 197: the agent restated the refund policy and the customer repeated the complaint. Turn 198: the agent restated the refund policy and the customer repeated the complaint. Turn 199: the agent restated the refund policy and the customer repeated the complaint. Turn 200: the agent restated the refund policy and the customer repeated the complaint. Turn 201: the agent restated the refund policy and the customer repeated the complaint. Turn 202: the agent restated the refund policy and the customer repeated the complaint. Turn 203: the agent restated the refund policy and the customer repeated the complaint. Turn 204: the agent restated the refund policy and the customer repeated the complaint. Turn 205: the agent restated the refund policy and the customer repeated the complaint. Turn 206: the agent restated the refund policy and the customer repeated the complaint. Turn 207: the agent restated the refund policy and the customer repeated the complaint. Turn 208: the agent restated the refund policy and the customer repeated the complaint. Turn 209: the agent restated the refund policy and the customer repeated the complaint. Turn 210: the agent restated the refund policy and the customer repeated the complaint. Turn 211: the agent restated the refund policy and the customer repeated the complaint. Turn 212: the agent restated the refund policy and the customer repeated the complaint. Turn 213: the agent restated the refund policy and the customer repeated the complaint. Turn 214: the agent restated the refund policy and the customer repeated the complaint. Turn 215: the agent restated the refund policy and the customer repeated the complaint. Turn 216: the agent restated the refund policy and the customer repeated the complaint. Turn 217: the agent restated the refund policy and the customer repeated the complaint. Turn 218: the agent restated the refund policy and the customer repeated the complaint. Turn 219: the agent restated the refund policy and the customer repeated the complaint. Turn 220: the agent restated the refund policy and the customer repeated the complaint. Turn 221: the agent restated the refund policy and the customer repeated the complaint. Turn 222: the agent restated the refund policy and the customer repeated the complaint. Turn 223: the agent restated the refund policy and the customer repeated the complaint. Turn 224: the agent restated the refund policy and the customer repeated the complaint. Turn 225: the agent restated the refund policy and the customer repeated the complaint. Turn 226: the agent restated the refund policy and the customer repeated the complaint. Turn 227: the agent restated the refund policy and the customer repeated the complaint. Turn 228: the agent restated the refund policy and the customer repeated the complaint. Turn 229: the agent restated the refund policy and the customer repeated the complaint. Turn 230: the agent restated the refund policy and the customer repeated the complaint. Turn 231: the agent restated the refund policy and the customer repeated the complaint. Turn 232: the agent restated the refund policy and the customer repeated the complaint. Turn 233: the agent restated the refund policy and the customer repeated the complaint. Turn 234: the agent restated the refund policy and the customer repeated the complaint. Turn 235: the agent restated the refund policy and the customer repeated the complaint. Turn 236: the agent restated the refund policy and the customer repeated the complaint. Turn 237: the agent restated the refund policy and the customer repeated the complaint. Turn 238: the agent restated the refund policy and the customer repeated the complaint. Turn 239: the agent restated the refund policy and the customer repeated the complaint. Turn 240: the agent restated the refund policy and the customer repeated the complaint. Turn 241: the agent restated the refund policy and the customer repeated the complaint. Turn 242: the agent restated the refund policy and the customer repeated the complaint. Turn 243: the agent restated the refund policy and the customer repeated the complaint. Turn 244: the agent restated the refund policy and the customer repeated the complaint. Turn 245: the agent restated the refund policy and the customer repeated the complaint. Turn 246: the agent restated the refund policy and the customer repeated the complaint. Turn 247: the agent restated the refund policy and the customer repeated the complaint. Turn 248: the agent restated the refund policy and the customer repeated the complaint. Turn 249: the agent restated the refund policy and the customer repeated the complaint. Turn 250: the agent restated the refund policy and the customer repeated the complaint. Turn 251: the agent restated the refund policy and the customer repeated the complaint. Turn 252: the agent restated the refund policy and the customer repeated the complaint. Turn 253: the agent restated the refund policy and the customer repeated the complaint. Turn 254: the agent restated the refund policy and the customer repeated the complaint. Turn 255: the agent restated the refund policy and the customer repeated the complaint. Turn 256: the agent restated the refund policy and the customer repeated the complaint. Turn 257: the agent restated the refund policy and the customer repeated the complaint. Turn 258: the agent restated the refund policy and the customer repeated the complaint. Turn 259: the agent restated the refund policy and the customer repeated the complaint. Turn 260: the agent restated the refund policy and the customer repeated the complaint. Turn 261: the agent restated the refund policy and the customer repeated the complaint. Turn 262: the agent restated the refund policy and the customer repeated the complaint. Turn 263: the agent restated the refund policy and the customer repeated the complaint. Turn 264: the agent restated the refund policy and the customer repeated the complaint. Turn 265: the agent restated the refund policy and the customer repeated the complaint. Turn 266: the agent restated the refund policy and the customer repeated the complaint. Turn 267: the agent restated the refund policy and the customer repeated the complaint. Turn 268: the agent restated the refund policy and the customer repeated the complaint. Turn 269: the agent restated the refund policy and the customer repeated the complaint. Turn 270: the agent restated the refund policy and the customer repeated the complaint. Turn 271: the agent restated the refund policy and the customer repeated the complaint. Turn 272: the agent restated the refund policy and the customer repeated the complaint. Turn 273: the agent restated the refund policy and the customer repeated the complaint. Turn 274: the agent restated the refund policy and the customer repeated the complaint. Turn 275: the agent restated the refund policy and the customer repeated the complaint. Turn 276: the agent restated the refund policy and the customer repeated the complaint. Turn 277: the agent restated the refund policy and the customer repeated the complaint. Turn 278: the agent restated the refund policy and the customer repeated the complaint. Turn 279: the agent restated the refund policy and the customer repeated the complaint. Turn 280: the agent restated the refund policy and the customer repeated the complaint. Turn 281: the agent restated the refund policy and the customer repeated the complaint. Turn 282: the agent restated the refund policy and the customer repeated the complaint. Turn 283: the agent restated the refund policy and the customer repeated the complaint. Turn 284: the agent restated the refund policy and the customer repeated the complaint. Turn 285: the agent restated the refund policy and the customer repeated the complaint. Turn 286: the agent restated the refund policy and the customer repeated the complaint. Turn 287: the agent restated the refund policy and the customer repeated the complaint. Turn 288: the agent restated the refund policy and the customer repeated the complaint. Turn 289: the agent restated the refund policy and the customer repeated the complaint. Turn 290: the agent restated the refund policy and the customer repeated the complaint. Turn 291: the agent restated the refund policy and the customer repeated the complaint. Turn 292: the agent restated the refund policy and the customer repeated the complaint. Turn 293: the agent restated the refund policy and the customer repeated the complaint. Turn 294: the agent restated the refund policy and the customer repeated the complaint. Turn 295: the agent restated the refund policy and the customer repeated the complaint. Turn 296: the agent restated the refund policy and the customer repeated the complaint. Turn 297: the agent restated the refund policy and the customer repeated the complaint. Turn 298: the agent restated the refund policy and the customer repeated the complaint. Turn 299: the agent restated the refund policy and the customer repeated the complaint. Turn 300: the agent restated the refund policy and the customer repeated the complaint. Turn 301: the agent restated the refund policy and the customer repeated the complaint. Turn 302: the agent restated the refund policy and the customer repeated the complaint. Turn 303: the agent restated the refund policy and the customer repeated the complaint. Turn 304: the agent restated the refund policy and the customer repeated the complaint. Turn 305: the agent restated the refund policy and the customer repeated the complaint. Turn 306: the agent restated the refund policy and the customer repeated the complaint. Turn 307: the agent restated the refund policy and the customer repeated the complaint. Turn 308: the agent restated the refund policy and the customer repeated the complaint. Turn 309: the agent restated the refund policy and the customer repeated the complaint. Turn 310: the agent restated the refund policy and the customer repeated the complaint. Turn 311: the agent restated the refund policy and the customer repeated the complaint. Turn 312: the agent restated the refund policy and the customer repeated the complaint. Turn 313: the agent restated the refund policy and the customer repeated the complaint. Turn 314: the agent restated the refund policy and the customer repeated the complaint. Turn 315: the agent restated the refund policy and the customer repeated the complaint. Turn 316: the agent restated the refund policy and the customer repeated the complaint. Turn 317: the agent restated the refund policy and the customer repeated the complaint. Turn 318: the agent restated the refund policy and the customer repeated the complaint. Turn 319: the agent restated the refund policy and the customer repeated the complaint. Turn 320: the agent restated the refund policy and the customer repeated the complaint. Turn 321: the agent restated the refund policy and the customer repeated the complaint. Turn 322: the agent restated the refund policy and the customer repeated the complaint. Turn 323: the agent restated the refund policy and the customer repeated the complaint. Turn 324: the agent restated the refund policy and the customer repeated the complaint. Turn 325: the agent restated the refund policy and the customer repeated the complaint. Turn 326: the agent restated the refund policy and the customer repeated the complaint. Turn 327: the agent restated the refund policy and the customer repeated the complaint. Turn 328: the agent restated the refund policy and the customer repeated the complaint. Turn 329: the agent restated the refund policy and the customer repeated the complaint. Turn 330: the agent restated the refund policy and the customer repeated the complaint. Turn 331: the agent restated the refund policy and the customer repeated the complaint. Turn 332: the agent restated the refund policy and the customer repeated the complaint. Turn 333: the agent restated the refund policy and the customer repeated the complaint. Turn 334: the agent restated the refund policy and the customer repeated the complaint. Turn 335: the agent restated the refund policy and the customer repeated the complaint. Turn 336: the agent restated the refund policy and the customer repeated the complaint. Turn 337: the agent restated the refund policy and the customer repeated the complaint. Turn 338: the agent restated the refund policy and the customer repeated the complaint. Turn 339: the agent restated the refund policy and the customer repeated the complaint. Turn 340: the agent restated the refund policy and the customer repeated the complaint. Turn 341: the agent restated the refund policy and the customer repeated the complaint. Turn 342: the agent restated the refund policy and the customer repeated the complaint. Turn 343: the agent restated the refund policy and the customer repeated the complaint. Turn 344: the agent restated the refund policy and the customer repeated the complaint. Turn 345: the agent restated the refund policy and the customer repeated the complaint. Turn 346: the agent restated the refund policy and the customer repeated the complaint. Turn 347: the agent restated the refund policy and the customer repeated the complaint. Turn 348: the agent restated the refund policy and the customer repeated the complaint. Turn 349: the agent restated the refund policy and the customer repeated the complaint. Turn 350: the agent restated the refund policy and the customer repeated the complaint. Turn 351: the agent restated the refund policy and the customer repeated the complaint. Turn 352: the agent restated the refund policy and the customer repeated the complaint. Turn 353: the agent restated the refund policy and the customer repeated the complaint. Turn 354: the agent restated the refund policy and the customer repeated the complaint. Turn 355: the agent restated the refund policy and the customer repeated the complaint. Turn 356: the agent restated the refund policy and the customer repeated the complaint. Turn 357: the agent restated the refund policy and the customer repeated the complaint. Turn 358: the agent restated the refund policy and the customer repeated the complaint. Turn 359: the agent restated the refund policy and the customer repeated the complaint. Turn 360: the agent restated the refund policy and the customer repeated the complaint. Turn 361: the agent restated the refund policy and the customer repeated the complaint. Turn 362: the agent restated the refund policy and the customer repeated the complaint. Turn 363: the agent restated the refund policy and the customer repeated the complaint. Turn 364: the agent restated the refund policy and the customer repeated the complaint. Turn 365: the agent restated the refund policy and the customer repeated the complaint. Turn 366: the agent restated the refund policy and the customer repeated the complaint. Turn 367: the agent restated the refund policy and the customer repeated the complaint. Turn 368: the agent restated the refund policy and the customer repeated the complaint. Turn 369: the agent restated the refund policy and the customer repeated the complaint. Turn 370: the agent restated the refund policy and the customer repeated the complaint. Turn 371: the agent restated the refund policy and the customer repeated the complaint. Turn 372: the agent restated the refund policy and the customer repeated the complaint. Turn 373: the agent restated the refund policy and the customer repeated the complaint. Turn 374: the agent restated the refund policy and the customer repeated the complaint. Turn 375: the agent restated the refund policy and the customer repeated the complaint. Turn 376: the agent restated the refund policy and the customer repeated the complaint. Turn 377: the agent restated the refund policy and the customer repeated the complaint. Turn 378: the agent restated the refund policy and the customer repeated the complaint. Turn 379: the agent restated the refund policy and the customer repeated the complaint. Turn 380: the agent restated the refund policy and the customer repeated the complaint. Turn 381: the agent restated the refund policy and the customer repeated the complaint. Turn 382: the agent restated the refund policy and the customer repeated the complaint. Turn 383: the agent restated the refund policy and the customer repeated the complaint. Turn 384: the agent restated the refund policy and the customer repeated the complaint. Turn 385: the agent restated the refund policy and the customer repeated the complaint. Turn 386: the agent restated the refund policy and the customer repeated the complaint. Turn 387: the agent restated the refund policy and the customer repeated the complaint. Turn 388: the agent restated the refund policy and the customer repeated the complaint. Turn 389: the agent restated the refund policy and the customer repeated the complaint. Turn 390: the agent restated the refund policy and the customer repeated the complaint. Turn 391: the agent restated the refund policy and the customer repeated the complaint. Turn 392: the agent restated the refund policy and the customer repeated the complaint. Turn 393: the agent restated the refund policy and the customer repeated the complaint. Turn 394: the agent restated the refund policy and the customer repeated the complaint. Turn 395: the agent restated the refund policy and the customer repeated the complaint. Turn 396: the agent restated the refund policy and the customer repeated the complaint. Turn 397: the agent restated the refund policy and the customer repeated the complaint. Turn 398: the agent restated the refund policy and the customer repeated the complaint. Turn 399: the agent restated the refund policy and the customer repeated the complaint.",
+ "The app crashes on login since yesterday's update.",
+ "ok"
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 90,
+ 44,
+ 147,
+ 73,
+ 1024,
+ 45,
+ 35
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9999,
+ "technical": 0.0,
+ "other": 0.0
+ },
+ "confidence": 0.9993,
+ "answer_confidence": 0.9999,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6231,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0528,
+ "1": 0.2714,
+ "2": 0.6759
+ },
+ "confidence": 0.2956,
+ "answer_confidence": 0.6759,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0067,
+ "confidence": 0.9933,
+ "answer_confidence": 0.9933,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 126,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2098,
+ "technical": 0.0637,
+ "other": 0.7266
+ },
+ "confidence": 0.331,
+ "answer_confidence": 0.7266,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3977,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0308,
+ "1": 0.5407,
+ "2": 0.4285
+ },
+ "confidence": 0.2692,
+ "answer_confidence": 0.5407,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1873,
+ "confidence": 0.8127,
+ "answer_confidence": 0.8127,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 435,
+ "output_tokens": 0,
+ "state_tokens": 113,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6466,
+ "technical": 0.0903,
+ "other": 0.2631
+ },
+ "confidence": 0.2259,
+ "answer_confidence": 0.6466,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0493,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1356,
+ "1": 0.6796,
+ "2": 0.1849
+ },
+ "confidence": 0.2304,
+ "answer_confidence": 0.6796,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0166,
+ "confidence": 0.9834,
+ "answer_confidence": 0.9834,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 213,
+ "output_tokens": 0,
+ "state_tokens": 39,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2111,
+ "technical": 0.0685,
+ "other": 0.7204
+ },
+ "confidence": 0.319,
+ "answer_confidence": 0.7204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1106,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2027,
+ "1": 0.484,
+ "2": 0.3133
+ },
+ "confidence": 0.0549,
+ "answer_confidence": 0.484,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8812,
+ "confidence": 0.8812,
+ "answer_confidence": 0.8812,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 7894,
+ "state_tokens_dropped": 6904,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.0015,
+ "technical": 0.962,
+ "other": 0.0365
+ },
+ "confidence": 0.8474,
+ "answer_confidence": 0.962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1251,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0863,
+ "1": 0.7023,
+ "2": 0.2114
+ },
+ "confidence": 0.2826,
+ "answer_confidence": 0.7023,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0344,
+ "confidence": 0.9656,
+ "answer_confidence": 0.9656,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 129,
+ "output_tokens": 0,
+ "state_tokens": 11,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7941,
+ "technical": 0.0972,
+ "other": 0.1087
+ },
+ "confidence": 0.4075,
+ "answer_confidence": 0.7941,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.006,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2485,
+ "1": 0.4971,
+ "2": 0.2545
+ },
+ "confidence": 0.0518,
+ "answer_confidence": 0.4971,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0083,
+ "confidence": 0.9917,
+ "answer_confidence": 0.9917,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 99,
+ "output_tokens": 0,
+ "state_tokens": 1,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 44
+ ],
+ [
+ 6,
+ 73
+ ],
+ [
+ 6,
+ 147
+ ],
+ [
+ 3,
+ 1024
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_sorted_windows.json b/laya-rust/tests/golden/multilingual/case_batch_sorted_windows.json
new file mode 100644
index 00000000..0a2d02e5
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_sorted_windows.json
@@ -0,0 +1,1101 @@
+{
+ "name": "batch_sorted_windows",
+ "states": [
+ "Routine status note 0. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 1. Routine status note 1. ",
+ "Routine status note 2. Routine status note 2. Routine status note 2. ",
+ "Routine status note 3. Routine status note 3. Routine status note 3. Routine status note 3. ",
+ "Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. ",
+ "Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. ",
+ "Routine status note 7. ",
+ "Routine status note 8. Routine status note 8. ",
+ "Routine status note 9. Routine status note 9. Routine status note 9. ",
+ "Routine status note 10. Routine status note 10. Routine status note 10. Routine status note 10. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. ",
+ "Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. ",
+ "Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. ",
+ "Routine status note 14. ",
+ "Routine status note 15. Routine status note 15. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 16. Routine status note 16. Routine status note 16. ",
+ "Routine status note 17. Routine status note 17. Routine status note 17. Routine status note 17. "
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 96,
+ 47,
+ 53,
+ 59,
+ 65,
+ 126,
+ 77,
+ 41,
+ 47,
+ 53,
+ 118,
+ 70,
+ 77,
+ 84,
+ 42,
+ 104,
+ 56,
+ 63
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9014,
+ "technical": 0.0068,
+ "other": 0.0918
+ },
+ "confidence": 0.6844,
+ "answer_confidence": 0.9014,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3022,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.027,
+ "1": 0.6439,
+ "2": 0.3292
+ },
+ "confidence": 0.3204,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0285,
+ "confidence": 0.9715,
+ "answer_confidence": 0.9715,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 282,
+ "output_tokens": 0,
+ "state_tokens": 62,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3729,
+ "technical": 0.159,
+ "other": 0.4682
+ },
+ "confidence": 0.0756,
+ "answer_confidence": 0.4682,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9772,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1331,
+ "1": 0.7566,
+ "2": 0.1103
+ },
+ "confidence": 0.3422,
+ "answer_confidence": 0.7566,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0016,
+ "confidence": 0.9984,
+ "answer_confidence": 0.9984,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 135,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5872,
+ "technical": 0.1169,
+ "other": 0.2959
+ },
+ "confidence": 0.159,
+ "answer_confidence": 0.5872,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0725,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1339,
+ "1": 0.6598,
+ "2": 0.2064
+ },
+ "confidence": 0.2088,
+ "answer_confidence": 0.6598,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0011,
+ "confidence": 0.9989,
+ "answer_confidence": 0.9989,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 153,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5939,
+ "technical": 0.1168,
+ "other": 0.2893
+ },
+ "confidence": 0.1634,
+ "answer_confidence": 0.5939,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0196,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1745,
+ "1": 0.6313,
+ "2": 0.1941
+ },
+ "confidence": 0.1687,
+ "answer_confidence": 0.6313,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0011,
+ "confidence": 0.9989,
+ "answer_confidence": 0.9989,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 171,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5894,
+ "technical": 0.1066,
+ "other": 0.3041
+ },
+ "confidence": 0.1697,
+ "answer_confidence": 0.5894,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.006,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1794,
+ "1": 0.6351,
+ "2": 0.1854
+ },
+ "confidence": 0.1726,
+ "answer_confidence": 0.6351,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0011,
+ "confidence": 0.9989,
+ "answer_confidence": 0.9989,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 189,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9691,
+ "technical": 0.002,
+ "other": 0.0289
+ },
+ "confidence": 0.8678,
+ "answer_confidence": 0.9691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.45,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0141,
+ "1": 0.5219,
+ "2": 0.464
+ },
+ "confidence": 0.3122,
+ "answer_confidence": 0.5219,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0538,
+ "confidence": 0.9462,
+ "answer_confidence": 0.9462,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 372,
+ "output_tokens": 0,
+ "state_tokens": 92,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5699,
+ "technical": 0.1313,
+ "other": 0.2988
+ },
+ "confidence": 0.1371,
+ "answer_confidence": 0.5699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1242,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1557,
+ "1": 0.5644,
+ "2": 0.2799
+ },
+ "confidence": 0.1181,
+ "answer_confidence": 0.5644,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.001,
+ "confidence": 0.999,
+ "answer_confidence": 0.999,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 225,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2209,
+ "technical": 0.1229,
+ "other": 0.6562
+ },
+ "confidence": 0.2102,
+ "answer_confidence": 0.6562,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9522,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2022,
+ "1": 0.6435,
+ "2": 0.1543
+ },
+ "confidence": 0.1851,
+ "answer_confidence": 0.6435,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0019,
+ "confidence": 0.9981,
+ "answer_confidence": 0.9981,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 117,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3429,
+ "technical": 0.1539,
+ "other": 0.5033
+ },
+ "confidence": 0.0892,
+ "answer_confidence": 0.5033,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9555,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1784,
+ "1": 0.6876,
+ "2": 0.1339
+ },
+ "confidence": 0.2406,
+ "answer_confidence": 0.6876,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.001,
+ "confidence": 0.999,
+ "answer_confidence": 0.999,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 135,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5255,
+ "technical": 0.1426,
+ "other": 0.3319
+ },
+ "confidence": 0.1062,
+ "answer_confidence": 0.5255,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0067,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.165,
+ "1": 0.6633,
+ "2": 0.1717
+ },
+ "confidence": 0.2062,
+ "answer_confidence": 0.6633,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0008,
+ "confidence": 0.9992,
+ "answer_confidence": 0.9992,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 153,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9448,
+ "technical": 0.0019,
+ "other": 0.0533
+ },
+ "confidence": 0.7979,
+ "answer_confidence": 0.9448,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2482,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0276,
+ "1": 0.6967,
+ "2": 0.2758
+ },
+ "confidence": 0.3573,
+ "answer_confidence": 0.6967,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0793,
+ "confidence": 0.9207,
+ "answer_confidence": 0.9207,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 348,
+ "output_tokens": 0,
+ "state_tokens": 84,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4408,
+ "technical": 0.1294,
+ "other": 0.4297
+ },
+ "confidence": 0.1001,
+ "answer_confidence": 0.4408,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1102,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1325,
+ "1": 0.6248,
+ "2": 0.2427
+ },
+ "confidence": 0.1759,
+ "answer_confidence": 0.6248,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0006,
+ "confidence": 0.9994,
+ "answer_confidence": 0.9994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 204,
+ "output_tokens": 0,
+ "state_tokens": 36,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4908,
+ "technical": 0.146,
+ "other": 0.3633
+ },
+ "confidence": 0.0915,
+ "answer_confidence": 0.4908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1341,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1639,
+ "1": 0.5381,
+ "2": 0.298
+ },
+ "confidence": 0.0983,
+ "answer_confidence": 0.5381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0005,
+ "confidence": 0.9995,
+ "answer_confidence": 0.9995,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 225,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4493,
+ "technical": 0.1573,
+ "other": 0.3934
+ },
+ "confidence": 0.0739,
+ "answer_confidence": 0.4493,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1835,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1551,
+ "1": 0.5063,
+ "2": 0.3386
+ },
+ "confidence": 0.0895,
+ "answer_confidence": 0.5063,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0004,
+ "confidence": 0.9996,
+ "answer_confidence": 0.9996,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 246,
+ "output_tokens": 0,
+ "state_tokens": 50,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.21,
+ "technical": 0.1104,
+ "other": 0.6796
+ },
+ "confidence": 0.2412,
+ "answer_confidence": 0.6796,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9855,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1809,
+ "1": 0.6526,
+ "2": 0.1664
+ },
+ "confidence": 0.1933,
+ "answer_confidence": 0.6526,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0013,
+ "confidence": 0.9987,
+ "answer_confidence": 0.9987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9237,
+ "technical": 0.0024,
+ "other": 0.0739
+ },
+ "confidence": 0.745,
+ "answer_confidence": 0.9237,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.292,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0184,
+ "1": 0.6711,
+ "2": 0.3104
+ },
+ "confidence": 0.3589,
+ "answer_confidence": 0.6711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0624,
+ "confidence": 0.9376,
+ "answer_confidence": 0.9376,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 306,
+ "output_tokens": 0,
+ "state_tokens": 70,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4746,
+ "technical": 0.1093,
+ "other": 0.4161
+ },
+ "confidence": 0.1257,
+ "answer_confidence": 0.4746,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0366,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1616,
+ "1": 0.6402,
+ "2": 0.1982
+ },
+ "confidence": 0.18,
+ "answer_confidence": 0.6402,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0007,
+ "confidence": 0.9993,
+ "answer_confidence": 0.9993,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 162,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.4059,
+ "technical": 0.0931,
+ "other": 0.501
+ },
+ "confidence": 0.1505,
+ "answer_confidence": 0.501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0468,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1712,
+ "1": 0.6109,
+ "2": 0.218
+ },
+ "confidence": 0.1487,
+ "answer_confidence": 0.6109,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0006,
+ "confidence": 0.9994,
+ "answer_confidence": 0.9994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 183,
+ "output_tokens": 0,
+ "state_tokens": 29,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 42
+ ],
+ [
+ 6,
+ 47
+ ],
+ [
+ 6,
+ 53
+ ],
+ [
+ 6,
+ 65
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 84
+ ],
+ [
+ 6,
+ 104
+ ],
+ [
+ 6,
+ 126
+ ],
+ [
+ 6,
+ 63
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_truncated.json b/laya-rust/tests/golden/multilingual/case_batch_truncated.json
new file mode 100644
index 00000000..5181ba6d
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_truncated.json
@@ -0,0 +1,854 @@
+{
+ "name": "batch_truncated",
+ "states": [
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ [
+ {
+ "role": "user",
+ "content": "Turn 0: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 1: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 2: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 3: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 4: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 5: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 6: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 7: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 8: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 9: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 10: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 11: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 12: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 13: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 14: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 15: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 16: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 17: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 18: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 19: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 20: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 21: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 22: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 23: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 24: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 25: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 26: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 27: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 28: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 29: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 30: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 31: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 32: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 33: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 34: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 35: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 36: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 37: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 38: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 39: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 40: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 41: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 42: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 43: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 44: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 45: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 46: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 47: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 48: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 49: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 50: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 51: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 52: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 53: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 54: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 55: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 56: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 57: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 58: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 59: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 60: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 61: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 62: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 63: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 64: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 65: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 66: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 67: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 68: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 69: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 70: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 71: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 72: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 73: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 74: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 75: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 76: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 77: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 78: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 79: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 80: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 81: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 82: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 83: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 84: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 85: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 86: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 87: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 88: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 89: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 90: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 91: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 92: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 93: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 94: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 95: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 96: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 97: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 98: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 99: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 100: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 101: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 102: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 103: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 104: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 105: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 106: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 107: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 108: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 109: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 110: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 111: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 112: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 113: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 114: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 115: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 116: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 117: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 118: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 119: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 120: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 121: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 122: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 123: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 124: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 125: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 126: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 127: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 128: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 129: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 130: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 131: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 132: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 133: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 134: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 135: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 136: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 137: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 138: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 139: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "I want a refund or I will cancel today."
+ }
+ ],
+ {
+ "notes": "Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 1024,
+ 1024,
+ 1024,
+ 90
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2111,
+ "technical": 0.0685,
+ "other": 0.7204
+ },
+ "confidence": 0.319,
+ "answer_confidence": 0.7204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1106,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2027,
+ "1": 0.484,
+ "2": 0.3133
+ },
+ "confidence": 0.0549,
+ "answer_confidence": 0.484,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8812,
+ "confidence": 0.8812,
+ "answer_confidence": 0.8812,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2694,
+ "state_tokens_dropped": 1704,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.978,
+ "technical": 0.0026,
+ "other": 0.0194
+ },
+ "confidence": 0.8967,
+ "answer_confidence": 0.978,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8829,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0158,
+ "1": 0.0855,
+ "2": 0.8987
+ },
+ "confidence": 0.6616,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.5785,
+ "confidence": 0.5785,
+ "answer_confidence": 0.5785,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 4252,
+ "state_tokens_dropped": 3262,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2488,
+ "technical": 0.0481,
+ "other": 0.703
+ },
+ "confidence": 0.3265,
+ "answer_confidence": 0.703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1362,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1841,
+ "1": 0.4956,
+ "2": 0.3203
+ },
+ "confidence": 0.0678,
+ "answer_confidence": 0.4956,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0491,
+ "confidence": 0.9509,
+ "answer_confidence": 0.9509,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2758,
+ "state_tokens_dropped": 1768,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 1024
+ ],
+ [
+ 6,
+ 1024
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_truncated_sorted.json b/laya-rust/tests/golden/multilingual/case_batch_truncated_sorted.json
new file mode 100644
index 00000000..0edcb6b6
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_truncated_sorted.json
@@ -0,0 +1,855 @@
+{
+ "name": "batch_truncated_sorted",
+ "states": [
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ [
+ {
+ "role": "user",
+ "content": "Turn 0: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 1: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 2: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 3: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 4: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 5: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 6: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 7: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 8: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 9: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 10: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 11: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 12: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 13: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 14: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 15: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 16: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 17: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 18: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 19: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 20: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 21: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 22: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 23: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 24: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 25: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 26: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 27: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 28: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 29: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 30: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 31: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 32: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 33: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 34: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 35: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 36: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 37: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 38: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 39: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 40: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 41: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 42: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 43: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 44: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 45: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 46: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 47: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 48: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 49: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 50: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 51: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 52: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 53: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 54: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 55: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 56: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 57: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 58: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 59: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 60: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 61: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 62: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 63: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 64: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 65: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 66: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 67: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 68: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 69: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 70: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 71: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 72: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 73: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 74: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 75: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 76: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 77: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 78: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 79: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 80: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 81: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 82: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 83: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 84: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 85: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 86: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 87: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 88: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 89: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 90: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 91: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 92: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 93: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 94: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 95: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 96: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 97: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 98: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 99: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 100: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 101: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 102: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 103: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 104: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 105: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 106: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 107: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 108: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 109: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 110: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 111: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 112: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 113: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 114: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 115: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 116: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 117: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 118: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 119: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 120: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 121: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 122: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 123: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 124: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 125: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 126: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 127: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 128: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 129: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 130: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 131: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 132: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 133: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 134: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 135: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 136: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 137: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 138: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 139: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "I want a refund or I will cancel today."
+ }
+ ],
+ {
+ "notes": "Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 1024,
+ 1024,
+ 1024,
+ 90
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2111,
+ "technical": 0.0685,
+ "other": 0.7204
+ },
+ "confidence": 0.319,
+ "answer_confidence": 0.7204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1106,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2027,
+ "1": 0.484,
+ "2": 0.3133
+ },
+ "confidence": 0.0549,
+ "answer_confidence": 0.484,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8812,
+ "confidence": 0.8812,
+ "answer_confidence": 0.8812,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2694,
+ "state_tokens_dropped": 1704,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.978,
+ "technical": 0.0026,
+ "other": 0.0194
+ },
+ "confidence": 0.8967,
+ "answer_confidence": 0.978,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8829,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0158,
+ "1": 0.0855,
+ "2": 0.8987
+ },
+ "confidence": 0.6616,
+ "answer_confidence": 0.8987,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.5785,
+ "confidence": 0.5785,
+ "answer_confidence": 0.5785,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 4252,
+ "state_tokens_dropped": 3262,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2488,
+ "technical": 0.0481,
+ "other": 0.703
+ },
+ "confidence": 0.3265,
+ "answer_confidence": 0.703,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1362,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1841,
+ "1": 0.4956,
+ "2": 0.3203
+ },
+ "confidence": 0.0678,
+ "answer_confidence": 0.4956,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0491,
+ "confidence": 0.9509,
+ "answer_confidence": 0.9509,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2758,
+ "state_tokens_dropped": 1768,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.836,
+ "technical": 0.0114,
+ "other": 0.1526
+ },
+ "confidence": 0.5562,
+ "answer_confidence": 0.836,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3713,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0231,
+ "1": 0.5824,
+ "2": 0.3945
+ },
+ "confidence": 0.3001,
+ "answer_confidence": 0.5824,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0414,
+ "confidence": 0.9586,
+ "answer_confidence": 0.9586,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 264,
+ "output_tokens": 0,
+ "state_tokens": 56,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 1024
+ ],
+ [
+ 3,
+ 1024
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/case_batch_unsorted_windows.json b/laya-rust/tests/golden/multilingual/case_batch_unsorted_windows.json
new file mode 100644
index 00000000..4b6ceddb
--- /dev/null
+++ b/laya-rust/tests/golden/multilingual/case_batch_unsorted_windows.json
@@ -0,0 +1,1100 @@
+{
+ "name": "batch_unsorted_windows",
+ "states": [
+ "Routine status note 0. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 1. Routine status note 1. ",
+ "Routine status note 2. Routine status note 2. Routine status note 2. ",
+ "Routine status note 3. Routine status note 3. Routine status note 3. Routine status note 3. ",
+ "Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. ",
+ "Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. ",
+ "Routine status note 7. ",
+ "Routine status note 8. Routine status note 8. ",
+ "Routine status note 9. Routine status note 9. Routine status note 9. ",
+ "Routine status note 10. Routine status note 10. Routine status note 10. Routine status note 10. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. ",
+ "Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. ",
+ "Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. ",
+ "Routine status note 14. ",
+ "Routine status note 15. Routine status note 15. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 16. Routine status note 16. Routine status note 16. ",
+ "Routine status note 17. Routine status note 17. Routine status note 17. Routine status note 17. "
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 96,
+ 47,
+ 53,
+ 59,
+ 65,
+ 126,
+ 77,
+ 41,
+ 47,
+ 53,
+ 118,
+ 70,
+ 77,
+ 84,
+ 42,
+ 104,
+ 56,
+ 63
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9014,
+ "technical": 0.0068,
+ "other": 0.0918
+ },
+ "confidence": 0.6844,
+ "answer_confidence": 0.9014,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.3022,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.027,
+ "1": 0.6439,
+ "2": 0.3292
+ },
+ "confidence": 0.3204,
+ "answer_confidence": 0.6439,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0285,
+ "confidence": 0.9715,
+ "answer_confidence": 0.9715,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 282,
+ "output_tokens": 0,
+ "state_tokens": 62,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3729,
+ "technical": 0.159,
+ "other": 0.4682
+ },
+ "confidence": 0.0756,
+ "answer_confidence": 0.4682,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9772,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1331,
+ "1": 0.7566,
+ "2": 0.1103
+ },
+ "confidence": 0.3422,
+ "answer_confidence": 0.7566,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0016,
+ "confidence": 0.9984,
+ "answer_confidence": 0.9984,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 135,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5872,
+ "technical": 0.1169,
+ "other": 0.2959
+ },
+ "confidence": 0.159,
+ "answer_confidence": 0.5872,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0725,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1339,
+ "1": 0.6598,
+ "2": 0.2064
+ },
+ "confidence": 0.2088,
+ "answer_confidence": 0.6598,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0011,
+ "confidence": 0.9989,
+ "answer_confidence": 0.9989,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 153,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5939,
+ "technical": 0.1168,
+ "other": 0.2893
+ },
+ "confidence": 0.1634,
+ "answer_confidence": 0.5939,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0196,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1745,
+ "1": 0.6313,
+ "2": 0.1941
+ },
+ "confidence": 0.1687,
+ "answer_confidence": 0.6313,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0011,
+ "confidence": 0.9989,
+ "answer_confidence": 0.9989,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 171,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5894,
+ "technical": 0.1066,
+ "other": 0.3041
+ },
+ "confidence": 0.1697,
+ "answer_confidence": 0.5894,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.006,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1794,
+ "1": 0.6351,
+ "2": 0.1854
+ },
+ "confidence": 0.1726,
+ "answer_confidence": 0.6351,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0011,
+ "confidence": 0.9989,
+ "answer_confidence": 0.9989,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 189,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9691,
+ "technical": 0.002,
+ "other": 0.0289
+ },
+ "confidence": 0.8678,
+ "answer_confidence": 0.9691,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.45,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0141,
+ "1": 0.5219,
+ "2": 0.464
+ },
+ "confidence": 0.3122,
+ "answer_confidence": 0.5219,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0538,
+ "confidence": 0.9462,
+ "answer_confidence": 0.9462,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 372,
+ "output_tokens": 0,
+ "state_tokens": 92,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5699,
+ "technical": 0.1313,
+ "other": 0.2988
+ },
+ "confidence": 0.1371,
+ "answer_confidence": 0.5699,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1242,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1557,
+ "1": 0.5644,
+ "2": 0.2799
+ },
+ "confidence": 0.1181,
+ "answer_confidence": 0.5644,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.001,
+ "confidence": 0.999,
+ "answer_confidence": 0.999,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 225,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2209,
+ "technical": 0.1229,
+ "other": 0.6562
+ },
+ "confidence": 0.2102,
+ "answer_confidence": 0.6562,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9522,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2022,
+ "1": 0.6435,
+ "2": 0.1543
+ },
+ "confidence": 0.1851,
+ "answer_confidence": 0.6435,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0019,
+ "confidence": 0.9981,
+ "answer_confidence": 0.9981,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 117,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3429,
+ "technical": 0.1539,
+ "other": 0.5033
+ },
+ "confidence": 0.0892,
+ "answer_confidence": 0.5033,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9555,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1784,
+ "1": 0.6876,
+ "2": 0.1339
+ },
+ "confidence": 0.2406,
+ "answer_confidence": 0.6876,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.001,
+ "confidence": 0.999,
+ "answer_confidence": 0.999,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 135,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5255,
+ "technical": 0.1426,
+ "other": 0.3319
+ },
+ "confidence": 0.1062,
+ "answer_confidence": 0.5255,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0067,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.165,
+ "1": 0.6633,
+ "2": 0.1717
+ },
+ "confidence": 0.2062,
+ "answer_confidence": 0.6633,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0008,
+ "confidence": 0.9992,
+ "answer_confidence": 0.9992,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 153,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9448,
+ "technical": 0.0019,
+ "other": 0.0533
+ },
+ "confidence": 0.7979,
+ "answer_confidence": 0.9448,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2482,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0276,
+ "1": 0.6967,
+ "2": 0.2758
+ },
+ "confidence": 0.3573,
+ "answer_confidence": 0.6967,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0793,
+ "confidence": 0.9207,
+ "answer_confidence": 0.9207,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 348,
+ "output_tokens": 0,
+ "state_tokens": 84,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4408,
+ "technical": 0.1294,
+ "other": 0.4297
+ },
+ "confidence": 0.1001,
+ "answer_confidence": 0.4408,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1102,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1325,
+ "1": 0.6248,
+ "2": 0.2427
+ },
+ "confidence": 0.1759,
+ "answer_confidence": 0.6248,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0006,
+ "confidence": 0.9994,
+ "answer_confidence": 0.9994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 204,
+ "output_tokens": 0,
+ "state_tokens": 36,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4908,
+ "technical": 0.146,
+ "other": 0.3633
+ },
+ "confidence": 0.0915,
+ "answer_confidence": 0.4908,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1341,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1639,
+ "1": 0.5381,
+ "2": 0.298
+ },
+ "confidence": 0.0983,
+ "answer_confidence": 0.5381,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0005,
+ "confidence": 0.9995,
+ "answer_confidence": 0.9995,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 225,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4493,
+ "technical": 0.1573,
+ "other": 0.3934
+ },
+ "confidence": 0.0739,
+ "answer_confidence": 0.4493,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.1835,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1551,
+ "1": 0.5063,
+ "2": 0.3386
+ },
+ "confidence": 0.0895,
+ "answer_confidence": 0.5063,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0004,
+ "confidence": 0.9996,
+ "answer_confidence": 0.9996,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 246,
+ "output_tokens": 0,
+ "state_tokens": 50,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.21,
+ "technical": 0.1104,
+ "other": 0.6796
+ },
+ "confidence": 0.2412,
+ "answer_confidence": 0.6796,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9855,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1809,
+ "1": 0.6526,
+ "2": 0.1664
+ },
+ "confidence": 0.1933,
+ "answer_confidence": 0.6526,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0013,
+ "confidence": 0.9987,
+ "answer_confidence": 0.9987,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 120,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.9237,
+ "technical": 0.0024,
+ "other": 0.0739
+ },
+ "confidence": 0.745,
+ "answer_confidence": 0.9237,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.292,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0184,
+ "1": 0.6711,
+ "2": 0.3104
+ },
+ "confidence": 0.3589,
+ "answer_confidence": 0.6711,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0624,
+ "confidence": 0.9376,
+ "answer_confidence": 0.9376,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 306,
+ "output_tokens": 0,
+ "state_tokens": 70,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4746,
+ "technical": 0.1093,
+ "other": 0.4161
+ },
+ "confidence": 0.1257,
+ "answer_confidence": 0.4746,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0366,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1616,
+ "1": 0.6402,
+ "2": 0.1982
+ },
+ "confidence": 0.18,
+ "answer_confidence": 0.6402,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0007,
+ "confidence": 0.9993,
+ "answer_confidence": 0.9993,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 162,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.4059,
+ "technical": 0.0931,
+ "other": 0.501
+ },
+ "confidence": 0.1505,
+ "answer_confidence": 0.501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0468,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1712,
+ "1": 0.6109,
+ "2": 0.218
+ },
+ "confidence": 0.1487,
+ "answer_confidence": 0.6109,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0006,
+ "confidence": 0.9994,
+ "answer_confidence": 0.9994,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 183,
+ "output_tokens": 0,
+ "state_tokens": 29,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 96
+ ],
+ [
+ 6,
+ 59
+ ],
+ [
+ 6,
+ 126
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 53
+ ],
+ [
+ 6,
+ 118
+ ],
+ [
+ 6,
+ 84
+ ],
+ [
+ 6,
+ 104
+ ],
+ [
+ 6,
+ 63
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/multilingual/index.json b/laya-rust/tests/golden/multilingual/index.json
index 5c2bf1ce..eb70e281 100644
--- a/laya-rust/tests/golden/multilingual/index.json
+++ b/laya-rust/tests/golden/multilingual/index.json
@@ -162,5 +162,191 @@
"file": "case_long_no_state_room.json",
"expects_error": true,
"kind": "long"
+ },
+ {
+ "name": "batch_one_state",
+ "file": "case_batch_one_state.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_mixed",
+ "file": "case_batch_mixed.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_1",
+ "file": "case_batch_size_1.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_3",
+ "file": "case_batch_size_3.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_over_len",
+ "file": "case_batch_size_over_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_zero",
+ "file": "case_batch_size_zero.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_negative",
+ "file": "case_batch_size_negative.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_on",
+ "file": "case_batch_sort_on.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_off",
+ "file": "case_batch_sort_off.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_3",
+ "file": "case_batch_sort_size_3.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_partial_window",
+ "file": "case_batch_sort_partial_window.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_no_batch_size",
+ "file": "case_batch_sort_no_batch_size.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_1",
+ "file": "case_batch_sort_size_1.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_eq_len",
+ "file": "case_batch_sort_size_eq_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_over_len",
+ "file": "case_batch_sort_size_over_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_zero",
+ "file": "case_batch_sort_size_zero.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_truncated",
+ "file": "case_batch_truncated.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_truncated_sorted",
+ "file": "case_batch_truncated_sorted.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_duplicate_states",
+ "file": "case_batch_duplicate_states.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_states",
+ "file": "case_batch_empty_states.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_states_invalid_questions",
+ "file": "case_batch_empty_states_invalid_questions.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_questions",
+ "file": "case_batch_empty_questions.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_both",
+ "file": "case_batch_empty_both.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sorted",
+ "file": "case_batch_sorted.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sorted_windows",
+ "file": "case_batch_sorted_windows.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_unsorted_windows",
+ "file": "case_batch_unsorted_windows.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_chunked",
+ "file": "case_batch_chunked.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_single_question",
+ "file": "case_batch_single_question.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_questions_with_options",
+ "file": "case_batch_empty_questions_with_options.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_none_state",
+ "file": "case_batch_none_state.json",
+ "expects_error": true,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_options_exceed_head",
+ "file": "case_batch_options_exceed_head.json",
+ "expects_error": true,
+ "kind": "batch"
}
]
diff --git a/laya-rust/tests/golden/routing/meta.json b/laya-rust/tests/golden/routing/meta.json
index b8f7a2d9..92814b6c 100644
--- a/laya-rust/tests/golden/routing/meta.json
+++ b/laya-rust/tests/golden/routing/meta.json
@@ -9,6 +9,8 @@
"counts": {
"lang_probe": 261,
"route_probe": 159,
+ "route_batch_probe": 10,
+ "route_batch_probe_errors": 6,
"email_probe_clean": 253,
"email_probe_state": 22,
"shortlist_probe": 22,
diff --git a/laya-rust/tests/golden/routing/route_batch_probe.json b/laya-rust/tests/golden/routing/route_batch_probe.json
new file mode 100644
index 00000000..f3470247
--- /dev/null
+++ b/laya-rust/tests/golden/routing/route_batch_probe.json
@@ -0,0 +1,1061 @@
+{
+ "cases": [
+ {
+ "label": "empty",
+ "router_kwargs": {},
+ "requests": [],
+ "decisions": []
+ },
+ {
+ "label": "single_english",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "I was charged twice, please refund.",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "model": "english",
+ "reason": "English Latin text",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "en",
+ "is_english": true,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "mixed_order",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "I was charged twice, please refund.",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": "Der Kunde wurde zweimal belastet und moechte eine Rueckerstattung fuer die Rechnung die nicht korrekt ist",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": {
+ "body": "मुझसे दो बार शुल्क लिया गया"
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": "I was charged twice, please refund.",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "model": "english",
+ "reason": "English Latin text",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "en",
+ "is_english": true,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "Latin script but language looks like 'de', not English",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "de",
+ "is_english": false,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "non-Latin script (devanagari, 100% of letters); the English checkpoint cannot read it",
+ "detection": {
+ "script": "devanagari",
+ "script_profile": {
+ "devanagari": 1.0
+ },
+ "language": null,
+ "is_english": false,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 1.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "English Latin text",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "en",
+ "is_english": true,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "per_request_overrides",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "hello",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "model": "ml"
+ },
+ {
+ "state": "hello",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "task": "typed_decisions"
+ },
+ {
+ "state": "मुझसे दो बार",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "en"
+ },
+ {
+ "state": "hello",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "de"
+ },
+ {
+ "state": "hello",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "C",
+ "lang_guess": "ro"
+ },
+ {
+ "state": "Care este ora in Tokyo?",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang_guess": "ro"
+ },
+ {
+ "state": "Care este ora in Tokyo?",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang_guess": "en"
+ },
+ {
+ "state": "Care este ora in Tokyo?",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang_guess": "und"
+ },
+ {
+ "state": "x",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "model": "english",
+ "lang_guess": "ro"
+ }
+ ],
+ "decisions": [
+ {
+ "model": "multilingual",
+ "reason": "explicit model='ml'",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "typed-decisions",
+ "reason": "explicit task='typed_decisions'",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "explicit lang='en'",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "explicit lang='de'",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "lang_guess: the caller identified this as non-English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "lang_guess: the caller identified this as non-English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "lang_guess: the caller identified this as English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "English Latin text",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "en",
+ "is_english": true,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "explicit model='english'",
+ "detection": null,
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "installed_lang_guess",
+ "router_kwargs": {
+ "lang_guess": "ro"
+ },
+ "requests": [
+ {
+ "state": "Care este ora in Tokyo?",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": "Care este ora in Tokyo?",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang_guess": "en"
+ },
+ {
+ "state": "Care este ora in Tokyo?",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang_guess": "C"
+ }
+ ],
+ "decisions": [
+ {
+ "model": "multilingual",
+ "reason": "Router(lang_guess=...): the caller identified this as non-English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "lang_guess: the caller identified this as English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "Router(lang_guess=...): the caller identified this as non-English text",
+ "detection": null,
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "auto_task_workflow",
+ "router_kwargs": {
+ "auto_task_detection": true
+ },
+ "requests": [
+ {
+ "state": {
+ "body": "I was charged twice"
+ },
+ "questions": {
+ "action": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "category": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "churn_risk": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "needs_human": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "urgency": {
+ "type": "noul",
+ "instructions": "x"
+ }
+ }
+ },
+ {
+ "state": {
+ "body": "I was charged twice"
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": "x",
+ "questions": {
+ "action": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "category": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "churn_risk": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "needs_human": {
+ "type": "noul",
+ "instructions": "x"
+ },
+ "urgency": {
+ "type": "noul",
+ "instructions": "x"
+ }
+ },
+ "lang_guess": "ro"
+ }
+ ],
+ "decisions": [
+ {
+ "model": "typed-decisions",
+ "reason": "question ids match the 'customer_service' typed-decisions workflow",
+ "detection": null,
+ "workflow": "customer_service"
+ },
+ {
+ "model": "english",
+ "reason": "English Latin text",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "en",
+ "is_english": true,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "typed-decisions",
+ "reason": "question ids match the 'customer_service' typed-decisions workflow",
+ "detection": null,
+ "workflow": "customer_service"
+ }
+ ]
+ },
+ {
+ "label": "custom_default",
+ "router_kwargs": {
+ "default": "english"
+ },
+ "requests": [
+ {
+ "state": "12345",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": "Hello",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "model": "english",
+ "reason": "no letters detected in state; using default (english)",
+ "detection": {
+ "script": "unknown",
+ "script_profile": {},
+ "language": null,
+ "is_english": true,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "Latin script, language not identified and no non-English letters; using default (english)",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": null,
+ "is_english": true,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "falsy_states",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": {},
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": [],
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": 0,
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ }
+ ],
+ "decisions": [
+ {
+ "model": "multilingual",
+ "reason": "no letters detected in state; using default (multilingual)",
+ "detection": {
+ "script": "unknown",
+ "script_profile": {},
+ "language": null,
+ "is_english": true,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "no letters detected in state; using default (multilingual)",
+ "detection": {
+ "script": "unknown",
+ "script_profile": {},
+ "language": null,
+ "is_english": true,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "no letters detected in state; using default (multilingual)",
+ "detection": {
+ "script": "unknown",
+ "script_profile": {},
+ "language": null,
+ "is_english": true,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "multilingual",
+ "reason": "no letters detected in state; using default (multilingual)",
+ "detection": {
+ "script": "unknown",
+ "script_profile": {},
+ "language": null,
+ "is_english": true,
+ "language_undecided": true,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "lang_variants",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": {
+ "body": "I was charged twice, please refund this order."
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "fr"
+ },
+ {
+ "state": {
+ "body": "मुझसे दो बार शुल्क लिया गया"
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "en-US"
+ },
+ {
+ "state": {
+ "body": "I was charged twice, please refund this order."
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "C"
+ },
+ {
+ "state": {
+ "body": "मुझसे दो बार शुल्क लिया गया"
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "model": "english"
+ }
+ ],
+ "decisions": [
+ {
+ "model": "multilingual",
+ "reason": "explicit lang='fr'",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "explicit lang='en-US'",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "English Latin text",
+ "detection": {
+ "script": "latin",
+ "script_profile": {
+ "latin": 1.0
+ },
+ "language": "en",
+ "is_english": true,
+ "language_undecided": false,
+ "diacritic_rate": 0.0,
+ "non_latin_fraction": 0.0,
+ "mixed_segment": null
+ },
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "explicit model='english'",
+ "detection": null,
+ "workflow": null
+ }
+ ]
+ },
+ {
+ "label": "installed_lang_guess_lang_wins",
+ "router_kwargs": {
+ "lang_guess": "de"
+ },
+ "requests": [
+ {
+ "state": {
+ "body": "I was charged twice, please refund this order."
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": {
+ "body": "I was charged twice, please refund this order."
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang_guess": "en"
+ },
+ {
+ "state": {
+ "body": "I was charged twice, please refund this order."
+ },
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "lang": "en"
+ }
+ ],
+ "decisions": [
+ {
+ "model": "multilingual",
+ "reason": "Router(lang_guess=...): the caller identified this as non-English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "lang_guess: the caller identified this as English text",
+ "detection": null,
+ "workflow": null
+ },
+ {
+ "model": "english",
+ "reason": "explicit lang='en'",
+ "detection": null,
+ "workflow": null
+ }
+ ]
+ }
+ ],
+ "errors": [
+ {
+ "label": "state_none",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "ok",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": null,
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ }
+ ],
+ "error_type": "TypeError",
+ "error_message": "request 1 'state' must not be None; pass a string, dict, or list"
+ },
+ {
+ "label": "state_missing",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ }
+ ],
+ "error_type": "ValueError",
+ "error_message": "request 0 is missing required key 'state'"
+ },
+ {
+ "label": "questions_missing",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "ok"
+ }
+ ],
+ "error_type": "ValueError",
+ "error_message": "request 0 is missing required key 'questions'"
+ },
+ {
+ "label": "questions_none",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "ok",
+ "questions": null
+ }
+ ],
+ "error_type": "TypeError",
+ "error_message": "request 0 'questions' must be a dict, got NoneType"
+ },
+ {
+ "label": "unknown_model",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "ok",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ {
+ "state": "ok",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ },
+ "model": "gpt"
+ }
+ ],
+ "error_type": "ValueError",
+ "error_message": "unknown model 'gpt'; choose one of ['english', 'multilingual', 'typed-decisions'] (or an alias: ['decisions', 'default', 'en', 'laya', 'laya-multilingual', 'laya-typed-decisions', 'ml', 'multi', 'typed', 'typed_decisions'])"
+ },
+ {
+ "label": "request_not_dict",
+ "router_kwargs": {},
+ "requests": [
+ {
+ "state": "ok",
+ "questions": {
+ "dept": {
+ "type": "choice",
+ "instructions": "Which team?",
+ "criteria": {
+ "billing": null,
+ "tech": null
+ }
+ }
+ }
+ },
+ "ok"
+ ],
+ "error_type": "TypeError",
+ "error_message": "request 1 must be a dict, got str"
+ }
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_chunked.json b/laya-rust/tests/golden/typed-decisions/case_batch_chunked.json
new file mode 100644
index 00000000..73a35af8
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_chunked.json
@@ -0,0 +1,466 @@
+{
+ "name": "batch_chunked",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ],
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint. Turn 140: the agent restated the refund policy and the customer repeated the complaint. Turn 141: the agent restated the refund policy and the customer repeated the complaint. Turn 142: the agent restated the refund policy and the customer repeated the complaint. Turn 143: the agent restated the refund policy and the customer repeated the complaint. Turn 144: the agent restated the refund policy and the customer repeated the complaint. Turn 145: the agent restated the refund policy and the customer repeated the complaint. Turn 146: the agent restated the refund policy and the customer repeated the complaint. Turn 147: the agent restated the refund policy and the customer repeated the complaint. Turn 148: the agent restated the refund policy and the customer repeated the complaint. Turn 149: the agent restated the refund policy and the customer repeated the complaint. Turn 150: the agent restated the refund policy and the customer repeated the complaint. Turn 151: the agent restated the refund policy and the customer repeated the complaint. Turn 152: the agent restated the refund policy and the customer repeated the complaint. Turn 153: the agent restated the refund policy and the customer repeated the complaint. Turn 154: the agent restated the refund policy and the customer repeated the complaint. Turn 155: the agent restated the refund policy and the customer repeated the complaint. Turn 156: the agent restated the refund policy and the customer repeated the complaint. Turn 157: the agent restated the refund policy and the customer repeated the complaint. Turn 158: the agent restated the refund policy and the customer repeated the complaint. Turn 159: the agent restated the refund policy and the customer repeated the complaint. Turn 160: the agent restated the refund policy and the customer repeated the complaint. Turn 161: the agent restated the refund policy and the customer repeated the complaint. Turn 162: the agent restated the refund policy and the customer repeated the complaint. Turn 163: the agent restated the refund policy and the customer repeated the complaint. Turn 164: the agent restated the refund policy and the customer repeated the complaint. Turn 165: the agent restated the refund policy and the customer repeated the complaint. Turn 166: the agent restated the refund policy and the customer repeated the complaint. Turn 167: the agent restated the refund policy and the customer repeated the complaint. Turn 168: the agent restated the refund policy and the customer repeated the complaint. Turn 169: the agent restated the refund policy and the customer repeated the complaint. Turn 170: the agent restated the refund policy and the customer repeated the complaint. Turn 171: the agent restated the refund policy and the customer repeated the complaint. Turn 172: the agent restated the refund policy and the customer repeated the complaint. Turn 173: the agent restated the refund policy and the customer repeated the complaint. Turn 174: the agent restated the refund policy and the customer repeated the complaint. Turn 175: the agent restated the refund policy and the customer repeated the complaint. Turn 176: the agent restated the refund policy and the customer repeated the complaint. Turn 177: the agent restated the refund policy and the customer repeated the complaint. Turn 178: the agent restated the refund policy and the customer repeated the complaint. Turn 179: the agent restated the refund policy and the customer repeated the complaint. Turn 180: the agent restated the refund policy and the customer repeated the complaint. Turn 181: the agent restated the refund policy and the customer repeated the complaint. Turn 182: the agent restated the refund policy and the customer repeated the complaint. Turn 183: the agent restated the refund policy and the customer repeated the complaint. Turn 184: the agent restated the refund policy and the customer repeated the complaint. Turn 185: the agent restated the refund policy and the customer repeated the complaint. Turn 186: the agent restated the refund policy and the customer repeated the complaint. Turn 187: the agent restated the refund policy and the customer repeated the complaint. Turn 188: the agent restated the refund policy and the customer repeated the complaint. Turn 189: the agent restated the refund policy and the customer repeated the complaint. Turn 190: the agent restated the refund policy and the customer repeated the complaint. Turn 191: the agent restated the refund policy and the customer repeated the complaint. Turn 192: the agent restated the refund policy and the customer repeated the complaint. Turn 193: the agent restated the refund policy and the customer repeated the complaint. Turn 194: the agent restated the refund policy and the customer repeated the complaint. Turn 195: the agent restated the refund policy and the customer repeated the complaint. Turn 196: the agent restated the refund policy and the customer repeated the complaint. Turn 197: the agent restated the refund policy and the customer repeated the complaint. Turn 198: the agent restated the refund policy and the customer repeated the complaint. Turn 199: the agent restated the refund policy and the customer repeated the complaint. Turn 200: the agent restated the refund policy and the customer repeated the complaint. Turn 201: the agent restated the refund policy and the customer repeated the complaint. Turn 202: the agent restated the refund policy and the customer repeated the complaint. Turn 203: the agent restated the refund policy and the customer repeated the complaint. Turn 204: the agent restated the refund policy and the customer repeated the complaint. Turn 205: the agent restated the refund policy and the customer repeated the complaint. Turn 206: the agent restated the refund policy and the customer repeated the complaint. Turn 207: the agent restated the refund policy and the customer repeated the complaint. Turn 208: the agent restated the refund policy and the customer repeated the complaint. Turn 209: the agent restated the refund policy and the customer repeated the complaint. Turn 210: the agent restated the refund policy and the customer repeated the complaint. Turn 211: the agent restated the refund policy and the customer repeated the complaint. Turn 212: the agent restated the refund policy and the customer repeated the complaint. Turn 213: the agent restated the refund policy and the customer repeated the complaint. Turn 214: the agent restated the refund policy and the customer repeated the complaint. Turn 215: the agent restated the refund policy and the customer repeated the complaint. Turn 216: the agent restated the refund policy and the customer repeated the complaint. Turn 217: the agent restated the refund policy and the customer repeated the complaint. Turn 218: the agent restated the refund policy and the customer repeated the complaint. Turn 219: the agent restated the refund policy and the customer repeated the complaint. Turn 220: the agent restated the refund policy and the customer repeated the complaint. Turn 221: the agent restated the refund policy and the customer repeated the complaint. Turn 222: the agent restated the refund policy and the customer repeated the complaint. Turn 223: the agent restated the refund policy and the customer repeated the complaint. Turn 224: the agent restated the refund policy and the customer repeated the complaint. Turn 225: the agent restated the refund policy and the customer repeated the complaint. Turn 226: the agent restated the refund policy and the customer repeated the complaint. Turn 227: the agent restated the refund policy and the customer repeated the complaint. Turn 228: the agent restated the refund policy and the customer repeated the complaint. Turn 229: the agent restated the refund policy and the customer repeated the complaint. Turn 230: the agent restated the refund policy and the customer repeated the complaint. Turn 231: the agent restated the refund policy and the customer repeated the complaint. Turn 232: the agent restated the refund policy and the customer repeated the complaint. Turn 233: the agent restated the refund policy and the customer repeated the complaint. Turn 234: the agent restated the refund policy and the customer repeated the complaint. Turn 235: the agent restated the refund policy and the customer repeated the complaint. Turn 236: the agent restated the refund policy and the customer repeated the complaint. Turn 237: the agent restated the refund policy and the customer repeated the complaint. Turn 238: the agent restated the refund policy and the customer repeated the complaint. Turn 239: the agent restated the refund policy and the customer repeated the complaint. Turn 240: the agent restated the refund policy and the customer repeated the complaint. Turn 241: the agent restated the refund policy and the customer repeated the complaint. Turn 242: the agent restated the refund policy and the customer repeated the complaint. Turn 243: the agent restated the refund policy and the customer repeated the complaint. Turn 244: the agent restated the refund policy and the customer repeated the complaint. Turn 245: the agent restated the refund policy and the customer repeated the complaint. Turn 246: the agent restated the refund policy and the customer repeated the complaint. Turn 247: the agent restated the refund policy and the customer repeated the complaint. Turn 248: the agent restated the refund policy and the customer repeated the complaint. Turn 249: the agent restated the refund policy and the customer repeated the complaint. Turn 250: the agent restated the refund policy and the customer repeated the complaint. Turn 251: the agent restated the refund policy and the customer repeated the complaint. Turn 252: the agent restated the refund policy and the customer repeated the complaint. Turn 253: the agent restated the refund policy and the customer repeated the complaint. Turn 254: the agent restated the refund policy and the customer repeated the complaint. Turn 255: the agent restated the refund policy and the customer repeated the complaint. Turn 256: the agent restated the refund policy and the customer repeated the complaint. Turn 257: the agent restated the refund policy and the customer repeated the complaint. Turn 258: the agent restated the refund policy and the customer repeated the complaint. Turn 259: the agent restated the refund policy and the customer repeated the complaint. Turn 260: the agent restated the refund policy and the customer repeated the complaint. Turn 261: the agent restated the refund policy and the customer repeated the complaint. Turn 262: the agent restated the refund policy and the customer repeated the complaint. Turn 263: the agent restated the refund policy and the customer repeated the complaint. Turn 264: the agent restated the refund policy and the customer repeated the complaint. Turn 265: the agent restated the refund policy and the customer repeated the complaint. Turn 266: the agent restated the refund policy and the customer repeated the complaint. Turn 267: the agent restated the refund policy and the customer repeated the complaint. Turn 268: the agent restated the refund policy and the customer repeated the complaint. Turn 269: the agent restated the refund policy and the customer repeated the complaint. Turn 270: the agent restated the refund policy and the customer repeated the complaint. Turn 271: the agent restated the refund policy and the customer repeated the complaint. Turn 272: the agent restated the refund policy and the customer repeated the complaint. Turn 273: the agent restated the refund policy and the customer repeated the complaint. Turn 274: the agent restated the refund policy and the customer repeated the complaint. Turn 275: the agent restated the refund policy and the customer repeated the complaint. Turn 276: the agent restated the refund policy and the customer repeated the complaint. Turn 277: the agent restated the refund policy and the customer repeated the complaint. Turn 278: the agent restated the refund policy and the customer repeated the complaint. Turn 279: the agent restated the refund policy and the customer repeated the complaint. Turn 280: the agent restated the refund policy and the customer repeated the complaint. Turn 281: the agent restated the refund policy and the customer repeated the complaint. Turn 282: the agent restated the refund policy and the customer repeated the complaint. Turn 283: the agent restated the refund policy and the customer repeated the complaint. Turn 284: the agent restated the refund policy and the customer repeated the complaint. Turn 285: the agent restated the refund policy and the customer repeated the complaint. Turn 286: the agent restated the refund policy and the customer repeated the complaint. Turn 287: the agent restated the refund policy and the customer repeated the complaint. Turn 288: the agent restated the refund policy and the customer repeated the complaint. Turn 289: the agent restated the refund policy and the customer repeated the complaint. Turn 290: the agent restated the refund policy and the customer repeated the complaint. Turn 291: the agent restated the refund policy and the customer repeated the complaint. Turn 292: the agent restated the refund policy and the customer repeated the complaint. Turn 293: the agent restated the refund policy and the customer repeated the complaint. Turn 294: the agent restated the refund policy and the customer repeated the complaint. Turn 295: the agent restated the refund policy and the customer repeated the complaint. Turn 296: the agent restated the refund policy and the customer repeated the complaint. Turn 297: the agent restated the refund policy and the customer repeated the complaint. Turn 298: the agent restated the refund policy and the customer repeated the complaint. Turn 299: the agent restated the refund policy and the customer repeated the complaint. Turn 300: the agent restated the refund policy and the customer repeated the complaint. Turn 301: the agent restated the refund policy and the customer repeated the complaint. Turn 302: the agent restated the refund policy and the customer repeated the complaint. Turn 303: the agent restated the refund policy and the customer repeated the complaint. Turn 304: the agent restated the refund policy and the customer repeated the complaint. Turn 305: the agent restated the refund policy and the customer repeated the complaint. Turn 306: the agent restated the refund policy and the customer repeated the complaint. Turn 307: the agent restated the refund policy and the customer repeated the complaint. Turn 308: the agent restated the refund policy and the customer repeated the complaint. Turn 309: the agent restated the refund policy and the customer repeated the complaint. Turn 310: the agent restated the refund policy and the customer repeated the complaint. Turn 311: the agent restated the refund policy and the customer repeated the complaint. Turn 312: the agent restated the refund policy and the customer repeated the complaint. Turn 313: the agent restated the refund policy and the customer repeated the complaint. Turn 314: the agent restated the refund policy and the customer repeated the complaint. Turn 315: the agent restated the refund policy and the customer repeated the complaint. Turn 316: the agent restated the refund policy and the customer repeated the complaint. Turn 317: the agent restated the refund policy and the customer repeated the complaint. Turn 318: the agent restated the refund policy and the customer repeated the complaint. Turn 319: the agent restated the refund policy and the customer repeated the complaint. Turn 320: the agent restated the refund policy and the customer repeated the complaint. Turn 321: the agent restated the refund policy and the customer repeated the complaint. Turn 322: the agent restated the refund policy and the customer repeated the complaint. Turn 323: the agent restated the refund policy and the customer repeated the complaint. Turn 324: the agent restated the refund policy and the customer repeated the complaint. Turn 325: the agent restated the refund policy and the customer repeated the complaint. Turn 326: the agent restated the refund policy and the customer repeated the complaint. Turn 327: the agent restated the refund policy and the customer repeated the complaint. Turn 328: the agent restated the refund policy and the customer repeated the complaint. Turn 329: the agent restated the refund policy and the customer repeated the complaint. Turn 330: the agent restated the refund policy and the customer repeated the complaint. Turn 331: the agent restated the refund policy and the customer repeated the complaint. Turn 332: the agent restated the refund policy and the customer repeated the complaint. Turn 333: the agent restated the refund policy and the customer repeated the complaint. Turn 334: the agent restated the refund policy and the customer repeated the complaint. Turn 335: the agent restated the refund policy and the customer repeated the complaint. Turn 336: the agent restated the refund policy and the customer repeated the complaint. Turn 337: the agent restated the refund policy and the customer repeated the complaint. Turn 338: the agent restated the refund policy and the customer repeated the complaint. Turn 339: the agent restated the refund policy and the customer repeated the complaint. Turn 340: the agent restated the refund policy and the customer repeated the complaint. Turn 341: the agent restated the refund policy and the customer repeated the complaint. Turn 342: the agent restated the refund policy and the customer repeated the complaint. Turn 343: the agent restated the refund policy and the customer repeated the complaint. Turn 344: the agent restated the refund policy and the customer repeated the complaint. Turn 345: the agent restated the refund policy and the customer repeated the complaint. Turn 346: the agent restated the refund policy and the customer repeated the complaint. Turn 347: the agent restated the refund policy and the customer repeated the complaint. Turn 348: the agent restated the refund policy and the customer repeated the complaint. Turn 349: the agent restated the refund policy and the customer repeated the complaint. Turn 350: the agent restated the refund policy and the customer repeated the complaint. Turn 351: the agent restated the refund policy and the customer repeated the complaint. Turn 352: the agent restated the refund policy and the customer repeated the complaint. Turn 353: the agent restated the refund policy and the customer repeated the complaint. Turn 354: the agent restated the refund policy and the customer repeated the complaint. Turn 355: the agent restated the refund policy and the customer repeated the complaint. Turn 356: the agent restated the refund policy and the customer repeated the complaint. Turn 357: the agent restated the refund policy and the customer repeated the complaint. Turn 358: the agent restated the refund policy and the customer repeated the complaint. Turn 359: the agent restated the refund policy and the customer repeated the complaint. Turn 360: the agent restated the refund policy and the customer repeated the complaint. Turn 361: the agent restated the refund policy and the customer repeated the complaint. Turn 362: the agent restated the refund policy and the customer repeated the complaint. Turn 363: the agent restated the refund policy and the customer repeated the complaint. Turn 364: the agent restated the refund policy and the customer repeated the complaint. Turn 365: the agent restated the refund policy and the customer repeated the complaint. Turn 366: the agent restated the refund policy and the customer repeated the complaint. Turn 367: the agent restated the refund policy and the customer repeated the complaint. Turn 368: the agent restated the refund policy and the customer repeated the complaint. Turn 369: the agent restated the refund policy and the customer repeated the complaint. Turn 370: the agent restated the refund policy and the customer repeated the complaint. Turn 371: the agent restated the refund policy and the customer repeated the complaint. Turn 372: the agent restated the refund policy and the customer repeated the complaint. Turn 373: the agent restated the refund policy and the customer repeated the complaint. Turn 374: the agent restated the refund policy and the customer repeated the complaint. Turn 375: the agent restated the refund policy and the customer repeated the complaint. Turn 376: the agent restated the refund policy and the customer repeated the complaint. Turn 377: the agent restated the refund policy and the customer repeated the complaint. Turn 378: the agent restated the refund policy and the customer repeated the complaint. Turn 379: the agent restated the refund policy and the customer repeated the complaint. Turn 380: the agent restated the refund policy and the customer repeated the complaint. Turn 381: the agent restated the refund policy and the customer repeated the complaint. Turn 382: the agent restated the refund policy and the customer repeated the complaint. Turn 383: the agent restated the refund policy and the customer repeated the complaint. Turn 384: the agent restated the refund policy and the customer repeated the complaint. Turn 385: the agent restated the refund policy and the customer repeated the complaint. Turn 386: the agent restated the refund policy and the customer repeated the complaint. Turn 387: the agent restated the refund policy and the customer repeated the complaint. Turn 388: the agent restated the refund policy and the customer repeated the complaint. Turn 389: the agent restated the refund policy and the customer repeated the complaint. Turn 390: the agent restated the refund policy and the customer repeated the complaint. Turn 391: the agent restated the refund policy and the customer repeated the complaint. Turn 392: the agent restated the refund policy and the customer repeated the complaint. Turn 393: the agent restated the refund policy and the customer repeated the complaint. Turn 394: the agent restated the refund policy and the customer repeated the complaint. Turn 395: the agent restated the refund policy and the customer repeated the complaint. Turn 396: the agent restated the refund policy and the customer repeated the complaint. Turn 397: the agent restated the refund policy and the customer repeated the complaint. Turn 398: the agent restated the refund policy and the customer repeated the complaint. Turn 399: the agent restated the refund policy and the customer repeated the complaint.",
+ "The app crashes on login since yesterday's update.",
+ "ok"
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3
+ },
+ "state_widths": [
+ 91,
+ 44,
+ 157,
+ 74,
+ 1024,
+ 44,
+ 35
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4925,
+ "technical": 0.3304,
+ "other": 0.1771
+ },
+ "confidence": 0.0704,
+ "answer_confidence": 0.4925,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2205,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2895,
+ "1": 0.2006,
+ "2": 0.51
+ },
+ "confidence": 0.0675,
+ "answer_confidence": 0.51,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0363,
+ "confidence": 0.9637,
+ "answer_confidence": 0.9637,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5409,
+ "technical": 0.2757,
+ "other": 0.1834
+ },
+ "confidence": 0.0909,
+ "answer_confidence": 0.5409,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.138,
+ "2": 0.8347
+ },
+ "confidence": 0.5244,
+ "answer_confidence": 0.8347,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7748,
+ "confidence": 0.7748,
+ "answer_confidence": 0.7748,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5116,
+ "technical": 0.1679,
+ "other": 0.3205
+ },
+ "confidence": 0.0832,
+ "answer_confidence": 0.5116,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2127,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1436,
+ "1": 0.5002,
+ "2": 0.3562
+ },
+ "confidence": 0.0963,
+ "answer_confidence": 0.5002,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2121,
+ "confidence": 0.7879,
+ "answer_confidence": 0.7879,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 214,
+ "output_tokens": 0,
+ "state_tokens": 40,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4553,
+ "technical": 0.1506,
+ "other": 0.3941
+ },
+ "confidence": 0.0804,
+ "answer_confidence": 0.4553,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9401,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3838,
+ "1": 0.2923,
+ "2": 0.3239
+ },
+ "confidence": 0.0058,
+ "answer_confidence": 0.3838,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1726,
+ "confidence": 0.8274,
+ "answer_confidence": 0.8274,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 6804,
+ "state_tokens_dropped": 5814,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.0616,
+ "technical": 0.7487,
+ "other": 0.1897
+ },
+ "confidence": 0.3594,
+ "answer_confidence": 0.7487,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4629,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1179,
+ "1": 0.3013,
+ "2": 0.5808
+ },
+ "confidence": 0.1543,
+ "answer_confidence": 0.5808,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.5343,
+ "confidence": 0.5343,
+ "answer_confidence": 0.5343,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.3803,
+ "technical": 0.2662,
+ "other": 0.3535
+ },
+ "confidence": 0.01,
+ "answer_confidence": 0.3803,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8546,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3938,
+ "1": 0.3579,
+ "2": 0.2483
+ },
+ "confidence": 0.0164,
+ "answer_confidence": 0.3938,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3012,
+ "confidence": 0.6988,
+ "answer_confidence": 0.6988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 97,
+ "output_tokens": 0,
+ "state_tokens": 1,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 157
+ ],
+ [
+ 9,
+ 1024
+ ],
+ [
+ 3,
+ 35
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_duplicate_states.json b/laya-rust/tests/golden/typed-decisions/case_batch_duplicate_states.json
new file mode 100644
index 00000000..849422ab
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_duplicate_states.json
@@ -0,0 +1,211 @@
+{
+ "name": "batch_duplicate_states",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 91,
+ 91,
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 91
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_empty_both.json b/laya-rust/tests/golden/typed-decisions/case_batch_empty_both.json
new file mode 100644
index 00000000..8a31ab3d
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_empty_both.json
@@ -0,0 +1,9 @@
+{
+ "name": "batch_empty_both",
+ "states": [],
+ "questions": {},
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_empty_questions.json b/laya-rust/tests/golden/typed-decisions/case_batch_empty_questions.json
new file mode 100644
index 00000000..ea34701f
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_empty_questions.json
@@ -0,0 +1,29 @@
+{
+ "name": "batch_empty_questions",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ ""
+ ],
+ "questions": {},
+ "options": {},
+ "state_widths": [],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ }
+ ],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_empty_questions_with_options.json b/laya-rust/tests/golden/typed-decisions/case_batch_empty_questions_with_options.json
new file mode 100644
index 00000000..d646aae6
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_empty_questions_with_options.json
@@ -0,0 +1,45 @@
+{
+ "name": "batch_empty_questions_with_options",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ }
+ ],
+ "questions": {},
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {},
+ "usage": {
+ "input_tokens": 0,
+ "output_tokens": 0
+ }
+ }
+ ],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_empty_states.json b/laya-rust/tests/golden/typed-decisions/case_batch_empty_states.json
new file mode 100644
index 00000000..eabd6424
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_empty_states.json
@@ -0,0 +1,32 @@
+{
+ "name": "batch_empty_states",
+ "states": [],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_empty_states_invalid_questions.json b/laya-rust/tests/golden/typed-decisions/case_batch_empty_states_invalid_questions.json
new file mode 100644
index 00000000..d8c82752
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_empty_states_invalid_questions.json
@@ -0,0 +1,316 @@
+{
+ "name": "batch_empty_states_invalid_questions",
+ "states": [],
+ "questions": {
+ "too_wide": {
+ "type": "choice",
+ "instructions": "Too many options to fit the sequence.",
+ "criteria": {
+ "opt_000": "description 0",
+ "opt_001": "description 1",
+ "opt_002": "description 2",
+ "opt_003": "description 3",
+ "opt_004": "description 4",
+ "opt_005": "description 5",
+ "opt_006": "description 6",
+ "opt_007": "description 7",
+ "opt_008": "description 8",
+ "opt_009": "description 9",
+ "opt_010": "description 10",
+ "opt_011": "description 11",
+ "opt_012": "description 12",
+ "opt_013": "description 13",
+ "opt_014": "description 14",
+ "opt_015": "description 15",
+ "opt_016": "description 16",
+ "opt_017": "description 17",
+ "opt_018": "description 18",
+ "opt_019": "description 19",
+ "opt_020": "description 20",
+ "opt_021": "description 21",
+ "opt_022": "description 22",
+ "opt_023": "description 23",
+ "opt_024": "description 24",
+ "opt_025": "description 25",
+ "opt_026": "description 26",
+ "opt_027": "description 27",
+ "opt_028": "description 28",
+ "opt_029": "description 29",
+ "opt_030": "description 30",
+ "opt_031": "description 31",
+ "opt_032": "description 32",
+ "opt_033": "description 33",
+ "opt_034": "description 34",
+ "opt_035": "description 35",
+ "opt_036": "description 36",
+ "opt_037": "description 37",
+ "opt_038": "description 38",
+ "opt_039": "description 39",
+ "opt_040": "description 40",
+ "opt_041": "description 41",
+ "opt_042": "description 42",
+ "opt_043": "description 43",
+ "opt_044": "description 44",
+ "opt_045": "description 45",
+ "opt_046": "description 46",
+ "opt_047": "description 47",
+ "opt_048": "description 48",
+ "opt_049": "description 49",
+ "opt_050": "description 50",
+ "opt_051": "description 51",
+ "opt_052": "description 52",
+ "opt_053": "description 53",
+ "opt_054": "description 54",
+ "opt_055": "description 55",
+ "opt_056": "description 56",
+ "opt_057": "description 57",
+ "opt_058": "description 58",
+ "opt_059": "description 59",
+ "opt_060": "description 60",
+ "opt_061": "description 61",
+ "opt_062": "description 62",
+ "opt_063": "description 63",
+ "opt_064": "description 64",
+ "opt_065": "description 65",
+ "opt_066": "description 66",
+ "opt_067": "description 67",
+ "opt_068": "description 68",
+ "opt_069": "description 69",
+ "opt_070": "description 70",
+ "opt_071": "description 71",
+ "opt_072": "description 72",
+ "opt_073": "description 73",
+ "opt_074": "description 74",
+ "opt_075": "description 75",
+ "opt_076": "description 76",
+ "opt_077": "description 77",
+ "opt_078": "description 78",
+ "opt_079": "description 79",
+ "opt_080": "description 80",
+ "opt_081": "description 81",
+ "opt_082": "description 82",
+ "opt_083": "description 83",
+ "opt_084": "description 84",
+ "opt_085": "description 85",
+ "opt_086": "description 86",
+ "opt_087": "description 87",
+ "opt_088": "description 88",
+ "opt_089": "description 89",
+ "opt_090": "description 90",
+ "opt_091": "description 91",
+ "opt_092": "description 92",
+ "opt_093": "description 93",
+ "opt_094": "description 94",
+ "opt_095": "description 95",
+ "opt_096": "description 96",
+ "opt_097": "description 97",
+ "opt_098": "description 98",
+ "opt_099": "description 99",
+ "opt_100": "description 100",
+ "opt_101": "description 101",
+ "opt_102": "description 102",
+ "opt_103": "description 103",
+ "opt_104": "description 104",
+ "opt_105": "description 105",
+ "opt_106": "description 106",
+ "opt_107": "description 107",
+ "opt_108": "description 108",
+ "opt_109": "description 109",
+ "opt_110": "description 110",
+ "opt_111": "description 111",
+ "opt_112": "description 112",
+ "opt_113": "description 113",
+ "opt_114": "description 114",
+ "opt_115": "description 115",
+ "opt_116": "description 116",
+ "opt_117": "description 117",
+ "opt_118": "description 118",
+ "opt_119": "description 119",
+ "opt_120": "description 120",
+ "opt_121": "description 121",
+ "opt_122": "description 122",
+ "opt_123": "description 123",
+ "opt_124": "description 124",
+ "opt_125": "description 125",
+ "opt_126": "description 126",
+ "opt_127": "description 127",
+ "opt_128": "description 128",
+ "opt_129": "description 129",
+ "opt_130": "description 130",
+ "opt_131": "description 131",
+ "opt_132": "description 132",
+ "opt_133": "description 133",
+ "opt_134": "description 134",
+ "opt_135": "description 135",
+ "opt_136": "description 136",
+ "opt_137": "description 137",
+ "opt_138": "description 138",
+ "opt_139": "description 139",
+ "opt_140": "description 140",
+ "opt_141": "description 141",
+ "opt_142": "description 142",
+ "opt_143": "description 143",
+ "opt_144": "description 144",
+ "opt_145": "description 145",
+ "opt_146": "description 146",
+ "opt_147": "description 147",
+ "opt_148": "description 148",
+ "opt_149": "description 149",
+ "opt_150": "description 150",
+ "opt_151": "description 151",
+ "opt_152": "description 152",
+ "opt_153": "description 153",
+ "opt_154": "description 154",
+ "opt_155": "description 155",
+ "opt_156": "description 156",
+ "opt_157": "description 157",
+ "opt_158": "description 158",
+ "opt_159": "description 159",
+ "opt_160": "description 160",
+ "opt_161": "description 161",
+ "opt_162": "description 162",
+ "opt_163": "description 163",
+ "opt_164": "description 164",
+ "opt_165": "description 165",
+ "opt_166": "description 166",
+ "opt_167": "description 167",
+ "opt_168": "description 168",
+ "opt_169": "description 169",
+ "opt_170": "description 170",
+ "opt_171": "description 171",
+ "opt_172": "description 172",
+ "opt_173": "description 173",
+ "opt_174": "description 174",
+ "opt_175": "description 175",
+ "opt_176": "description 176",
+ "opt_177": "description 177",
+ "opt_178": "description 178",
+ "opt_179": "description 179",
+ "opt_180": "description 180",
+ "opt_181": "description 181",
+ "opt_182": "description 182",
+ "opt_183": "description 183",
+ "opt_184": "description 184",
+ "opt_185": "description 185",
+ "opt_186": "description 186",
+ "opt_187": "description 187",
+ "opt_188": "description 188",
+ "opt_189": "description 189",
+ "opt_190": "description 190",
+ "opt_191": "description 191",
+ "opt_192": "description 192",
+ "opt_193": "description 193",
+ "opt_194": "description 194",
+ "opt_195": "description 195",
+ "opt_196": "description 196",
+ "opt_197": "description 197",
+ "opt_198": "description 198",
+ "opt_199": "description 199",
+ "opt_200": "description 200",
+ "opt_201": "description 201",
+ "opt_202": "description 202",
+ "opt_203": "description 203",
+ "opt_204": "description 204",
+ "opt_205": "description 205",
+ "opt_206": "description 206",
+ "opt_207": "description 207",
+ "opt_208": "description 208",
+ "opt_209": "description 209",
+ "opt_210": "description 210",
+ "opt_211": "description 211",
+ "opt_212": "description 212",
+ "opt_213": "description 213",
+ "opt_214": "description 214",
+ "opt_215": "description 215",
+ "opt_216": "description 216",
+ "opt_217": "description 217",
+ "opt_218": "description 218",
+ "opt_219": "description 219",
+ "opt_220": "description 220",
+ "opt_221": "description 221",
+ "opt_222": "description 222",
+ "opt_223": "description 223",
+ "opt_224": "description 224",
+ "opt_225": "description 225",
+ "opt_226": "description 226",
+ "opt_227": "description 227",
+ "opt_228": "description 228",
+ "opt_229": "description 229",
+ "opt_230": "description 230",
+ "opt_231": "description 231",
+ "opt_232": "description 232",
+ "opt_233": "description 233",
+ "opt_234": "description 234",
+ "opt_235": "description 235",
+ "opt_236": "description 236",
+ "opt_237": "description 237",
+ "opt_238": "description 238",
+ "opt_239": "description 239",
+ "opt_240": "description 240",
+ "opt_241": "description 241",
+ "opt_242": "description 242",
+ "opt_243": "description 243",
+ "opt_244": "description 244",
+ "opt_245": "description 245",
+ "opt_246": "description 246",
+ "opt_247": "description 247",
+ "opt_248": "description 248",
+ "opt_249": "description 249",
+ "opt_250": "description 250",
+ "opt_251": "description 251",
+ "opt_252": "description 252",
+ "opt_253": "description 253",
+ "opt_254": "description 254",
+ "opt_255": "description 255",
+ "opt_256": "description 256",
+ "opt_257": "description 257",
+ "opt_258": "description 258",
+ "opt_259": "description 259",
+ "opt_260": "description 260",
+ "opt_261": "description 261",
+ "opt_262": "description 262",
+ "opt_263": "description 263",
+ "opt_264": "description 264",
+ "opt_265": "description 265",
+ "opt_266": "description 266",
+ "opt_267": "description 267",
+ "opt_268": "description 268",
+ "opt_269": "description 269",
+ "opt_270": "description 270",
+ "opt_271": "description 271",
+ "opt_272": "description 272",
+ "opt_273": "description 273",
+ "opt_274": "description 274",
+ "opt_275": "description 275",
+ "opt_276": "description 276",
+ "opt_277": "description 277",
+ "opt_278": "description 278",
+ "opt_279": "description 279",
+ "opt_280": "description 280",
+ "opt_281": "description 281",
+ "opt_282": "description 282",
+ "opt_283": "description 283",
+ "opt_284": "description 284",
+ "opt_285": "description 285",
+ "opt_286": "description 286",
+ "opt_287": "description 287",
+ "opt_288": "description 288",
+ "opt_289": "description 289",
+ "opt_290": "description 290",
+ "opt_291": "description 291",
+ "opt_292": "description 292",
+ "opt_293": "description 293",
+ "opt_294": "description 294",
+ "opt_295": "description 295",
+ "opt_296": "description 296",
+ "opt_297": "description 297",
+ "opt_298": "description 298",
+ "opt_299": "description 299"
+ }
+ }
+ },
+ "options": {},
+ "state_widths": [],
+ "results": [],
+ "runs": []
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_mixed.json b/laya-rust/tests/golden/typed-decisions/case_batch_mixed.json
new file mode 100644
index 00000000..58532154
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_mixed.json
@@ -0,0 +1,515 @@
+{
+ "name": "batch_mixed",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6869,
+ "technical": 0.2079,
+ "other": 0.1052
+ },
+ "confidence": 0.2523,
+ "answer_confidence": 0.6869,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0204,
+ "1": 0.1675,
+ "2": 0.8121
+ },
+ "confidence": 0.5015,
+ "answer_confidence": 0.8121,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7602,
+ "confidence": 0.7602,
+ "answer_confidence": 0.7602,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6718,
+ "technical": 0.2244,
+ "other": 0.1038
+ },
+ "confidence": 0.2375,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7672,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0194,
+ "1": 0.194,
+ "2": 0.7866
+ },
+ "confidence": 0.4689,
+ "answer_confidence": 0.7866,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7234,
+ "confidence": 0.7234,
+ "answer_confidence": 0.7234,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3741,
+ "technical": 0.2268,
+ "other": 0.3991
+ },
+ "confidence": 0.0252,
+ "answer_confidence": 0.3991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9031,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.368,
+ "1": 0.361,
+ "2": 0.2711
+ },
+ "confidence": 0.0083,
+ "answer_confidence": 0.368,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2943,
+ "confidence": 0.7057,
+ "answer_confidence": 0.7057,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.142,
+ "technical": 0.3006,
+ "other": 0.5573
+ },
+ "confidence": 0.1222,
+ "answer_confidence": 0.5573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2804,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2652,
+ "1": 0.1891,
+ "2": 0.5456
+ },
+ "confidence": 0.092,
+ "answer_confidence": 0.5456,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0403,
+ "confidence": 0.9597,
+ "answer_confidence": 0.9597,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2504,
+ "technical": 0.3345,
+ "other": 0.4151
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4151,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0394,
+ "1": 0.1552,
+ "2": 0.8054
+ },
+ "confidence": 0.4622,
+ "answer_confidence": 0.8054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.776,
+ "confidence": 0.776,
+ "answer_confidence": 0.776,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1839,
+ "technical": 0.4141,
+ "other": 0.4019
+ },
+ "confidence": 0.0507,
+ "answer_confidence": 0.4141,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9808,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3293,
+ "1": 0.3606,
+ "2": 0.3101
+ },
+ "confidence": 0.0018,
+ "answer_confidence": 0.3606,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2948,
+ "confidence": 0.7052,
+ "answer_confidence": 0.7052,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_none_state.json b/laya-rust/tests/golden/typed-decisions/case_batch_none_state.json
new file mode 100644
index 00000000..560be81a
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_none_state.json
@@ -0,0 +1,34 @@
+{
+ "name": "batch_none_state",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ null
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "error_type": "TypeError",
+ "error_message": "state must not be None; pass a string, dict, or list"
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_one_state.json b/laya-rust/tests/golden/typed-decisions/case_batch_one_state.json
new file mode 100644
index 00000000..e04a3370
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_one_state.json
@@ -0,0 +1,97 @@
+{
+ "name": "batch_one_state",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {},
+ "state_widths": [
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 91
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_options_exceed_head.json b/laya-rust/tests/golden/typed-decisions/case_batch_options_exceed_head.json
new file mode 100644
index 00000000..1c4ae386
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_options_exceed_head.json
@@ -0,0 +1,318 @@
+{
+ "name": "batch_options_exceed_head",
+ "states": [
+ "short",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "too_wide": {
+ "type": "choice",
+ "instructions": "Too many options to fit the sequence.",
+ "criteria": {
+ "opt_000": "description 0",
+ "opt_001": "description 1",
+ "opt_002": "description 2",
+ "opt_003": "description 3",
+ "opt_004": "description 4",
+ "opt_005": "description 5",
+ "opt_006": "description 6",
+ "opt_007": "description 7",
+ "opt_008": "description 8",
+ "opt_009": "description 9",
+ "opt_010": "description 10",
+ "opt_011": "description 11",
+ "opt_012": "description 12",
+ "opt_013": "description 13",
+ "opt_014": "description 14",
+ "opt_015": "description 15",
+ "opt_016": "description 16",
+ "opt_017": "description 17",
+ "opt_018": "description 18",
+ "opt_019": "description 19",
+ "opt_020": "description 20",
+ "opt_021": "description 21",
+ "opt_022": "description 22",
+ "opt_023": "description 23",
+ "opt_024": "description 24",
+ "opt_025": "description 25",
+ "opt_026": "description 26",
+ "opt_027": "description 27",
+ "opt_028": "description 28",
+ "opt_029": "description 29",
+ "opt_030": "description 30",
+ "opt_031": "description 31",
+ "opt_032": "description 32",
+ "opt_033": "description 33",
+ "opt_034": "description 34",
+ "opt_035": "description 35",
+ "opt_036": "description 36",
+ "opt_037": "description 37",
+ "opt_038": "description 38",
+ "opt_039": "description 39",
+ "opt_040": "description 40",
+ "opt_041": "description 41",
+ "opt_042": "description 42",
+ "opt_043": "description 43",
+ "opt_044": "description 44",
+ "opt_045": "description 45",
+ "opt_046": "description 46",
+ "opt_047": "description 47",
+ "opt_048": "description 48",
+ "opt_049": "description 49",
+ "opt_050": "description 50",
+ "opt_051": "description 51",
+ "opt_052": "description 52",
+ "opt_053": "description 53",
+ "opt_054": "description 54",
+ "opt_055": "description 55",
+ "opt_056": "description 56",
+ "opt_057": "description 57",
+ "opt_058": "description 58",
+ "opt_059": "description 59",
+ "opt_060": "description 60",
+ "opt_061": "description 61",
+ "opt_062": "description 62",
+ "opt_063": "description 63",
+ "opt_064": "description 64",
+ "opt_065": "description 65",
+ "opt_066": "description 66",
+ "opt_067": "description 67",
+ "opt_068": "description 68",
+ "opt_069": "description 69",
+ "opt_070": "description 70",
+ "opt_071": "description 71",
+ "opt_072": "description 72",
+ "opt_073": "description 73",
+ "opt_074": "description 74",
+ "opt_075": "description 75",
+ "opt_076": "description 76",
+ "opt_077": "description 77",
+ "opt_078": "description 78",
+ "opt_079": "description 79",
+ "opt_080": "description 80",
+ "opt_081": "description 81",
+ "opt_082": "description 82",
+ "opt_083": "description 83",
+ "opt_084": "description 84",
+ "opt_085": "description 85",
+ "opt_086": "description 86",
+ "opt_087": "description 87",
+ "opt_088": "description 88",
+ "opt_089": "description 89",
+ "opt_090": "description 90",
+ "opt_091": "description 91",
+ "opt_092": "description 92",
+ "opt_093": "description 93",
+ "opt_094": "description 94",
+ "opt_095": "description 95",
+ "opt_096": "description 96",
+ "opt_097": "description 97",
+ "opt_098": "description 98",
+ "opt_099": "description 99",
+ "opt_100": "description 100",
+ "opt_101": "description 101",
+ "opt_102": "description 102",
+ "opt_103": "description 103",
+ "opt_104": "description 104",
+ "opt_105": "description 105",
+ "opt_106": "description 106",
+ "opt_107": "description 107",
+ "opt_108": "description 108",
+ "opt_109": "description 109",
+ "opt_110": "description 110",
+ "opt_111": "description 111",
+ "opt_112": "description 112",
+ "opt_113": "description 113",
+ "opt_114": "description 114",
+ "opt_115": "description 115",
+ "opt_116": "description 116",
+ "opt_117": "description 117",
+ "opt_118": "description 118",
+ "opt_119": "description 119",
+ "opt_120": "description 120",
+ "opt_121": "description 121",
+ "opt_122": "description 122",
+ "opt_123": "description 123",
+ "opt_124": "description 124",
+ "opt_125": "description 125",
+ "opt_126": "description 126",
+ "opt_127": "description 127",
+ "opt_128": "description 128",
+ "opt_129": "description 129",
+ "opt_130": "description 130",
+ "opt_131": "description 131",
+ "opt_132": "description 132",
+ "opt_133": "description 133",
+ "opt_134": "description 134",
+ "opt_135": "description 135",
+ "opt_136": "description 136",
+ "opt_137": "description 137",
+ "opt_138": "description 138",
+ "opt_139": "description 139",
+ "opt_140": "description 140",
+ "opt_141": "description 141",
+ "opt_142": "description 142",
+ "opt_143": "description 143",
+ "opt_144": "description 144",
+ "opt_145": "description 145",
+ "opt_146": "description 146",
+ "opt_147": "description 147",
+ "opt_148": "description 148",
+ "opt_149": "description 149",
+ "opt_150": "description 150",
+ "opt_151": "description 151",
+ "opt_152": "description 152",
+ "opt_153": "description 153",
+ "opt_154": "description 154",
+ "opt_155": "description 155",
+ "opt_156": "description 156",
+ "opt_157": "description 157",
+ "opt_158": "description 158",
+ "opt_159": "description 159",
+ "opt_160": "description 160",
+ "opt_161": "description 161",
+ "opt_162": "description 162",
+ "opt_163": "description 163",
+ "opt_164": "description 164",
+ "opt_165": "description 165",
+ "opt_166": "description 166",
+ "opt_167": "description 167",
+ "opt_168": "description 168",
+ "opt_169": "description 169",
+ "opt_170": "description 170",
+ "opt_171": "description 171",
+ "opt_172": "description 172",
+ "opt_173": "description 173",
+ "opt_174": "description 174",
+ "opt_175": "description 175",
+ "opt_176": "description 176",
+ "opt_177": "description 177",
+ "opt_178": "description 178",
+ "opt_179": "description 179",
+ "opt_180": "description 180",
+ "opt_181": "description 181",
+ "opt_182": "description 182",
+ "opt_183": "description 183",
+ "opt_184": "description 184",
+ "opt_185": "description 185",
+ "opt_186": "description 186",
+ "opt_187": "description 187",
+ "opt_188": "description 188",
+ "opt_189": "description 189",
+ "opt_190": "description 190",
+ "opt_191": "description 191",
+ "opt_192": "description 192",
+ "opt_193": "description 193",
+ "opt_194": "description 194",
+ "opt_195": "description 195",
+ "opt_196": "description 196",
+ "opt_197": "description 197",
+ "opt_198": "description 198",
+ "opt_199": "description 199",
+ "opt_200": "description 200",
+ "opt_201": "description 201",
+ "opt_202": "description 202",
+ "opt_203": "description 203",
+ "opt_204": "description 204",
+ "opt_205": "description 205",
+ "opt_206": "description 206",
+ "opt_207": "description 207",
+ "opt_208": "description 208",
+ "opt_209": "description 209",
+ "opt_210": "description 210",
+ "opt_211": "description 211",
+ "opt_212": "description 212",
+ "opt_213": "description 213",
+ "opt_214": "description 214",
+ "opt_215": "description 215",
+ "opt_216": "description 216",
+ "opt_217": "description 217",
+ "opt_218": "description 218",
+ "opt_219": "description 219",
+ "opt_220": "description 220",
+ "opt_221": "description 221",
+ "opt_222": "description 222",
+ "opt_223": "description 223",
+ "opt_224": "description 224",
+ "opt_225": "description 225",
+ "opt_226": "description 226",
+ "opt_227": "description 227",
+ "opt_228": "description 228",
+ "opt_229": "description 229",
+ "opt_230": "description 230",
+ "opt_231": "description 231",
+ "opt_232": "description 232",
+ "opt_233": "description 233",
+ "opt_234": "description 234",
+ "opt_235": "description 235",
+ "opt_236": "description 236",
+ "opt_237": "description 237",
+ "opt_238": "description 238",
+ "opt_239": "description 239",
+ "opt_240": "description 240",
+ "opt_241": "description 241",
+ "opt_242": "description 242",
+ "opt_243": "description 243",
+ "opt_244": "description 244",
+ "opt_245": "description 245",
+ "opt_246": "description 246",
+ "opt_247": "description 247",
+ "opt_248": "description 248",
+ "opt_249": "description 249",
+ "opt_250": "description 250",
+ "opt_251": "description 251",
+ "opt_252": "description 252",
+ "opt_253": "description 253",
+ "opt_254": "description 254",
+ "opt_255": "description 255",
+ "opt_256": "description 256",
+ "opt_257": "description 257",
+ "opt_258": "description 258",
+ "opt_259": "description 259",
+ "opt_260": "description 260",
+ "opt_261": "description 261",
+ "opt_262": "description 262",
+ "opt_263": "description 263",
+ "opt_264": "description 264",
+ "opt_265": "description 265",
+ "opt_266": "description 266",
+ "opt_267": "description 267",
+ "opt_268": "description 268",
+ "opt_269": "description 269",
+ "opt_270": "description 270",
+ "opt_271": "description 271",
+ "opt_272": "description 272",
+ "opt_273": "description 273",
+ "opt_274": "description 274",
+ "opt_275": "description 275",
+ "opt_276": "description 276",
+ "opt_277": "description 277",
+ "opt_278": "description 278",
+ "opt_279": "description 279",
+ "opt_280": "description 280",
+ "opt_281": "description 281",
+ "opt_282": "description 282",
+ "opt_283": "description 283",
+ "opt_284": "description 284",
+ "opt_285": "description 285",
+ "opt_286": "description 286",
+ "opt_287": "description 287",
+ "opt_288": "description 288",
+ "opt_289": "description 289",
+ "opt_290": "description 290",
+ "opt_291": "description 291",
+ "opt_292": "description 292",
+ "opt_293": "description 293",
+ "opt_294": "description 294",
+ "opt_295": "description 295",
+ "opt_296": "description 296",
+ "opt_297": "description 297",
+ "opt_298": "description 298",
+ "opt_299": "description 299"
+ }
+ }
+ },
+ "options": {},
+ "error_type": "ValueError",
+ "error_message": "question 'too_wide': only 254 of its 300 option markers fit in max_len=1024 with head_max_len=256 spent on the question; lower head_max_len, raise max_len, or use fewer options"
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_single_question.json b/laya-rust/tests/golden/typed-decisions/case_batch_single_question.json
new file mode 100644
index 00000000..924a90ca
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_single_question.json
@@ -0,0 +1,137 @@
+{
+ "name": "batch_single_question",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ]
+ ],
+ "questions": {
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 91,
+ 44,
+ 157,
+ 74
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 91,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0363,
+ "confidence": 0.9637,
+ "answer_confidence": 0.9637,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 44,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7748,
+ "confidence": 0.7748,
+ "answer_confidence": 0.7748,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 157,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2121,
+ "confidence": 0.7879,
+ "answer_confidence": 0.7879,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 74,
+ "output_tokens": 0,
+ "state_tokens": 40,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 2,
+ 91
+ ],
+ [
+ 2,
+ 157
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_size_1.json b/laya-rust/tests/golden/typed-decisions/case_batch_size_1.json
new file mode 100644
index 00000000..497223c9
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_size_1.json
@@ -0,0 +1,545 @@
+{
+ "name": "batch_size_1",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 1
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6869,
+ "technical": 0.2079,
+ "other": 0.1052
+ },
+ "confidence": 0.2523,
+ "answer_confidence": 0.6869,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0204,
+ "1": 0.1675,
+ "2": 0.8121
+ },
+ "confidence": 0.5015,
+ "answer_confidence": 0.8121,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7602,
+ "confidence": 0.7602,
+ "answer_confidence": 0.7602,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6718,
+ "technical": 0.2244,
+ "other": 0.1038
+ },
+ "confidence": 0.2375,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7672,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0194,
+ "1": 0.194,
+ "2": 0.7866
+ },
+ "confidence": 0.4689,
+ "answer_confidence": 0.7866,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7234,
+ "confidence": 0.7234,
+ "answer_confidence": 0.7234,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3741,
+ "technical": 0.2268,
+ "other": 0.3991
+ },
+ "confidence": 0.0252,
+ "answer_confidence": 0.3991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9031,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.368,
+ "1": 0.361,
+ "2": 0.2711
+ },
+ "confidence": 0.0083,
+ "answer_confidence": 0.368,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2943,
+ "confidence": 0.7057,
+ "answer_confidence": 0.7057,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.142,
+ "technical": 0.3006,
+ "other": 0.5573
+ },
+ "confidence": 0.1222,
+ "answer_confidence": 0.5573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2804,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2652,
+ "1": 0.1891,
+ "2": 0.5456
+ },
+ "confidence": 0.092,
+ "answer_confidence": 0.5456,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0403,
+ "confidence": 0.9597,
+ "answer_confidence": 0.9597,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2504,
+ "technical": 0.3345,
+ "other": 0.4151
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4151,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0394,
+ "1": 0.1552,
+ "2": 0.8054
+ },
+ "confidence": 0.4622,
+ "answer_confidence": 0.8054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.776,
+ "confidence": 0.776,
+ "answer_confidence": 0.776,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1839,
+ "technical": 0.4141,
+ "other": 0.4019
+ },
+ "confidence": 0.0507,
+ "answer_confidence": 0.4141,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9808,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3293,
+ "1": 0.3606,
+ "2": 0.3101
+ },
+ "confidence": 0.0018,
+ "answer_confidence": 0.3606,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2948,
+ "confidence": 0.7052,
+ "answer_confidence": 0.7052,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 91
+ ],
+ [
+ 3,
+ 110
+ ],
+ [
+ 3,
+ 144
+ ],
+ [
+ 3,
+ 34
+ ],
+ [
+ 3,
+ 163
+ ],
+ [
+ 3,
+ 42
+ ],
+ [
+ 3,
+ 98
+ ],
+ [
+ 3,
+ 56
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_size_3.json b/laya-rust/tests/golden/typed-decisions/case_batch_size_3.json
new file mode 100644
index 00000000..2a673d53
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_size_3.json
@@ -0,0 +1,525 @@
+{
+ "name": "batch_size_3",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6869,
+ "technical": 0.2079,
+ "other": 0.1052
+ },
+ "confidence": 0.2523,
+ "answer_confidence": 0.6869,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0204,
+ "1": 0.1675,
+ "2": 0.8121
+ },
+ "confidence": 0.5015,
+ "answer_confidence": 0.8121,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7602,
+ "confidence": 0.7602,
+ "answer_confidence": 0.7602,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6718,
+ "technical": 0.2244,
+ "other": 0.1038
+ },
+ "confidence": 0.2375,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7672,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0194,
+ "1": 0.194,
+ "2": 0.7866
+ },
+ "confidence": 0.4689,
+ "answer_confidence": 0.7866,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7234,
+ "confidence": 0.7234,
+ "answer_confidence": 0.7234,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3741,
+ "technical": 0.2268,
+ "other": 0.3991
+ },
+ "confidence": 0.0252,
+ "answer_confidence": 0.3991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9031,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.368,
+ "1": 0.361,
+ "2": 0.2711
+ },
+ "confidence": 0.0083,
+ "answer_confidence": 0.368,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2943,
+ "confidence": 0.7057,
+ "answer_confidence": 0.7057,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.142,
+ "technical": 0.3006,
+ "other": 0.5573
+ },
+ "confidence": 0.1222,
+ "answer_confidence": 0.5573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2804,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2652,
+ "1": 0.1891,
+ "2": 0.5456
+ },
+ "confidence": 0.092,
+ "answer_confidence": 0.5456,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0403,
+ "confidence": 0.9597,
+ "answer_confidence": 0.9597,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2504,
+ "technical": 0.3345,
+ "other": 0.4151
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4151,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0394,
+ "1": 0.1552,
+ "2": 0.8054
+ },
+ "confidence": 0.4622,
+ "answer_confidence": 0.8054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.776,
+ "confidence": 0.776,
+ "answer_confidence": 0.776,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1839,
+ "technical": 0.4141,
+ "other": 0.4019
+ },
+ "confidence": 0.0507,
+ "answer_confidence": 0.4141,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9808,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3293,
+ "1": 0.3606,
+ "2": 0.3101
+ },
+ "confidence": 0.0018,
+ "answer_confidence": 0.3606,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2948,
+ "confidence": 0.7052,
+ "answer_confidence": 0.7052,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 144
+ ],
+ [
+ 9,
+ 163
+ ],
+ [
+ 6,
+ 98
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_size_negative.json b/laya-rust/tests/golden/typed-decisions/case_batch_size_negative.json
new file mode 100644
index 00000000..7617fbb1
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_size_negative.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_negative",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": -2
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6869,
+ "technical": 0.2079,
+ "other": 0.1052
+ },
+ "confidence": 0.2523,
+ "answer_confidence": 0.6869,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0204,
+ "1": 0.1675,
+ "2": 0.8121
+ },
+ "confidence": 0.5015,
+ "answer_confidence": 0.8121,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7602,
+ "confidence": 0.7602,
+ "answer_confidence": 0.7602,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6718,
+ "technical": 0.2244,
+ "other": 0.1038
+ },
+ "confidence": 0.2375,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7672,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0194,
+ "1": 0.194,
+ "2": 0.7866
+ },
+ "confidence": 0.4689,
+ "answer_confidence": 0.7866,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7234,
+ "confidence": 0.7234,
+ "answer_confidence": 0.7234,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3741,
+ "technical": 0.2268,
+ "other": 0.3991
+ },
+ "confidence": 0.0252,
+ "answer_confidence": 0.3991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9031,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.368,
+ "1": 0.361,
+ "2": 0.2711
+ },
+ "confidence": 0.0083,
+ "answer_confidence": 0.368,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2943,
+ "confidence": 0.7057,
+ "answer_confidence": 0.7057,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.142,
+ "technical": 0.3006,
+ "other": 0.5573
+ },
+ "confidence": 0.1222,
+ "answer_confidence": 0.5573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2804,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2652,
+ "1": 0.1891,
+ "2": 0.5456
+ },
+ "confidence": 0.092,
+ "answer_confidence": 0.5456,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0403,
+ "confidence": 0.9597,
+ "answer_confidence": 0.9597,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2504,
+ "technical": 0.3345,
+ "other": 0.4151
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4151,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0394,
+ "1": 0.1552,
+ "2": 0.8054
+ },
+ "confidence": 0.4622,
+ "answer_confidence": 0.8054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.776,
+ "confidence": 0.776,
+ "answer_confidence": 0.776,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1839,
+ "technical": 0.4141,
+ "other": 0.4019
+ },
+ "confidence": 0.0507,
+ "answer_confidence": 0.4141,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9808,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3293,
+ "1": 0.3606,
+ "2": 0.3101
+ },
+ "confidence": 0.0018,
+ "answer_confidence": 0.3606,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2948,
+ "confidence": 0.7052,
+ "answer_confidence": 0.7052,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_size_over_len.json b/laya-rust/tests/golden/typed-decisions/case_batch_size_over_len.json
new file mode 100644
index 00000000..cdcab5f0
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_size_over_len.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_over_len",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 100
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6869,
+ "technical": 0.2079,
+ "other": 0.1052
+ },
+ "confidence": 0.2523,
+ "answer_confidence": 0.6869,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0204,
+ "1": 0.1675,
+ "2": 0.8121
+ },
+ "confidence": 0.5015,
+ "answer_confidence": 0.8121,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7602,
+ "confidence": 0.7602,
+ "answer_confidence": 0.7602,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6718,
+ "technical": 0.2244,
+ "other": 0.1038
+ },
+ "confidence": 0.2375,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7672,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0194,
+ "1": 0.194,
+ "2": 0.7866
+ },
+ "confidence": 0.4689,
+ "answer_confidence": 0.7866,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7234,
+ "confidence": 0.7234,
+ "answer_confidence": 0.7234,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3741,
+ "technical": 0.2268,
+ "other": 0.3991
+ },
+ "confidence": 0.0252,
+ "answer_confidence": 0.3991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9031,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.368,
+ "1": 0.361,
+ "2": 0.2711
+ },
+ "confidence": 0.0083,
+ "answer_confidence": 0.368,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2943,
+ "confidence": 0.7057,
+ "answer_confidence": 0.7057,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.142,
+ "technical": 0.3006,
+ "other": 0.5573
+ },
+ "confidence": 0.1222,
+ "answer_confidence": 0.5573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2804,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2652,
+ "1": 0.1891,
+ "2": 0.5456
+ },
+ "confidence": 0.092,
+ "answer_confidence": 0.5456,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0403,
+ "confidence": 0.9597,
+ "answer_confidence": 0.9597,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2504,
+ "technical": 0.3345,
+ "other": 0.4151
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4151,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0394,
+ "1": 0.1552,
+ "2": 0.8054
+ },
+ "confidence": 0.4622,
+ "answer_confidence": 0.8054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.776,
+ "confidence": 0.776,
+ "answer_confidence": 0.776,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1839,
+ "technical": 0.4141,
+ "other": 0.4019
+ },
+ "confidence": 0.0507,
+ "answer_confidence": 0.4141,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9808,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3293,
+ "1": 0.3606,
+ "2": 0.3101
+ },
+ "confidence": 0.0018,
+ "answer_confidence": 0.3606,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2948,
+ "confidence": 0.7052,
+ "answer_confidence": 0.7052,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_size_zero.json b/laya-rust/tests/golden/typed-decisions/case_batch_size_zero.json
new file mode 100644
index 00000000..17c4dc7d
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_size_zero.json
@@ -0,0 +1,517 @@
+{
+ "name": "batch_size_zero",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ {
+ "subject": "Double charged on invoice 88213",
+ "body": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ [
+ {
+ "role": "user",
+ "content": "Hi, I need help with my invoice."
+ },
+ {
+ "role": "assistant",
+ "content": "Of course, what is wrong?"
+ },
+ {
+ "role": "user",
+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ }
+ ],
+ "",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, that is all sorted now.",
+ {
+ "notes": "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. ",
+ "message": "Please cancel my account today."
+ },
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 0
+ },
+ "state_widths": [
+ 91,
+ 110,
+ 144,
+ 34,
+ 163,
+ 42,
+ 98,
+ 56
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6869,
+ "technical": 0.2079,
+ "other": 0.1052
+ },
+ "confidence": 0.2523,
+ "answer_confidence": 0.6869,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7918,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0204,
+ "1": 0.1675,
+ "2": 0.8121
+ },
+ "confidence": 0.5015,
+ "answer_confidence": 0.8121,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7602,
+ "confidence": 0.7602,
+ "answer_confidence": 0.7602,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 322,
+ "output_tokens": 0,
+ "state_tokens": 76,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6718,
+ "technical": 0.2244,
+ "other": 0.1038
+ },
+ "confidence": 0.2375,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7672,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0194,
+ "1": 0.194,
+ "2": 0.7866
+ },
+ "confidence": 0.4689,
+ "answer_confidence": 0.7866,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7234,
+ "confidence": 0.7234,
+ "answer_confidence": 0.7234,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 424,
+ "output_tokens": 0,
+ "state_tokens": 110,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.3741,
+ "technical": 0.2268,
+ "other": 0.3991
+ },
+ "confidence": 0.0252,
+ "answer_confidence": 0.3991,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9031,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.368,
+ "1": 0.361,
+ "2": 0.2711
+ },
+ "confidence": 0.0083,
+ "answer_confidence": 0.368,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2943,
+ "confidence": 0.7057,
+ "answer_confidence": 0.7057,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 94,
+ "output_tokens": 0,
+ "state_tokens": 0,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.142,
+ "technical": 0.3006,
+ "other": 0.5573
+ },
+ "confidence": 0.1222,
+ "answer_confidence": 0.5573,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2804,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2652,
+ "1": 0.1891,
+ "2": 0.5456
+ },
+ "confidence": 0.092,
+ "answer_confidence": 0.5456,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0403,
+ "confidence": 0.9597,
+ "answer_confidence": 0.9597,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 118,
+ "output_tokens": 0,
+ "state_tokens": 8,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.2504,
+ "technical": 0.3345,
+ "other": 0.4151
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4151,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.766,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0394,
+ "1": 0.1552,
+ "2": 0.8054
+ },
+ "confidence": 0.4622,
+ "answer_confidence": 0.8054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.776,
+ "confidence": 0.776,
+ "answer_confidence": 0.776,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 286,
+ "output_tokens": 0,
+ "state_tokens": 64,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.1839,
+ "technical": 0.4141,
+ "other": 0.4019
+ },
+ "confidence": 0.0507,
+ "answer_confidence": 0.4141,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9808,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3293,
+ "1": 0.3606,
+ "2": 0.3101
+ },
+ "confidence": 0.0018,
+ "answer_confidence": 0.3606,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2948,
+ "confidence": 0.7052,
+ "answer_confidence": 0.7052,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 160,
+ "output_tokens": 0,
+ "state_tokens": 22,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 24,
+ 163
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_no_batch_size.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_no_batch_size.json
new file mode 100644
index 00000000..bcecc977
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_no_batch_size.json
@@ -0,0 +1,1182 @@
+{
+ "name": "batch_sort_no_batch_size",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_off.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_off.json
new file mode 100644
index 00000000..dc8ed69f
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_off.json
@@ -0,0 +1,1218 @@
+{
+ "name": "batch_sort_off",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 199
+ ],
+ [
+ 6,
+ 145
+ ],
+ [
+ 6,
+ 175
+ ],
+ [
+ 6,
+ 157
+ ],
+ [
+ 6,
+ 187
+ ],
+ [
+ 6,
+ 163
+ ],
+ [
+ 6,
+ 205
+ ],
+ [
+ 6,
+ 169
+ ],
+ [
+ 6,
+ 193
+ ],
+ [
+ 6,
+ 181
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_on.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_on.json
new file mode 100644
index 00000000..f4522ea4
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_on.json
@@ -0,0 +1,1219 @@
+{
+ "name": "batch_sort_on",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 97
+ ],
+ [
+ 6,
+ 109
+ ],
+ [
+ 6,
+ 121
+ ],
+ [
+ 6,
+ 139
+ ],
+ [
+ 6,
+ 157
+ ],
+ [
+ 6,
+ 169
+ ],
+ [
+ 6,
+ 187
+ ],
+ [
+ 6,
+ 205
+ ],
+ [
+ 6,
+ 151
+ ],
+ [
+ 6,
+ 193
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_partial_window.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_partial_window.json
new file mode 100644
index 00000000..ee17f1a9
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_partial_window.json
@@ -0,0 +1,218 @@
+{
+ "name": "batch_sort_partial_window",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 145
+ ],
+ [
+ 3,
+ 199
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_1.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_1.json
new file mode 100644
index 00000000..5a02652a
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_1.json
@@ -0,0 +1,1259 @@
+{
+ "name": "batch_sort_size_1",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 1,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 3,
+ 199
+ ],
+ [
+ 3,
+ 103
+ ],
+ [
+ 3,
+ 145
+ ],
+ [
+ 3,
+ 91
+ ],
+ [
+ 3,
+ 175
+ ],
+ [
+ 3,
+ 121
+ ],
+ [
+ 3,
+ 157
+ ],
+ [
+ 3,
+ 97
+ ],
+ [
+ 3,
+ 187
+ ],
+ [
+ 3,
+ 133
+ ],
+ [
+ 3,
+ 109
+ ],
+ [
+ 3,
+ 163
+ ],
+ [
+ 3,
+ 205
+ ],
+ [
+ 3,
+ 115
+ ],
+ [
+ 3,
+ 139
+ ],
+ [
+ 3,
+ 169
+ ],
+ [
+ 3,
+ 127
+ ],
+ [
+ 3,
+ 193
+ ],
+ [
+ 3,
+ 151
+ ],
+ [
+ 3,
+ 181
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_3.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_3.json
new file mode 100644
index 00000000..ac140ff1
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_3.json
@@ -0,0 +1,1207 @@
+{
+ "name": "batch_sort_size_3",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 103
+ ],
+ [
+ 9,
+ 121
+ ],
+ [
+ 9,
+ 139
+ ],
+ [
+ 9,
+ 157
+ ],
+ [
+ 9,
+ 175
+ ],
+ [
+ 9,
+ 193
+ ],
+ [
+ 6,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_eq_len.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_eq_len.json
new file mode 100644
index 00000000..0be562ab
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_eq_len.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_eq_len",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 20,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_over_len.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_over_len.json
new file mode 100644
index 00000000..7b179cd7
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_over_len.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_over_len",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 25,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_zero.json b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_zero.json
new file mode 100644
index 00000000..59422370
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sort_size_zero.json
@@ -0,0 +1,1183 @@
+{
+ "name": "batch_sort_size_zero",
+ "states": [
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Routine account check completed. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 0,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 199,
+ 103,
+ 145,
+ 91,
+ 175,
+ 121,
+ 157,
+ 97,
+ 187,
+ 133,
+ 109,
+ 163,
+ 205,
+ 115,
+ 139,
+ 169,
+ 127,
+ 193,
+ 151,
+ 181
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7688,
+ "technical": 0.1498,
+ "other": 0.0814
+ },
+ "confidence": 0.3712,
+ "answer_confidence": 0.7688,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.839,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1276,
+ "2": 0.8557
+ },
+ "confidence": 0.5773,
+ "answer_confidence": 0.8557,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8118,
+ "confidence": 0.8118,
+ "answer_confidence": 0.8118,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 589,
+ "output_tokens": 0,
+ "state_tokens": 165,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7324,
+ "technical": 0.1799,
+ "other": 0.0877
+ },
+ "confidence": 0.3172,
+ "answer_confidence": 0.7324,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.6583,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0379,
+ "1": 0.2659,
+ "2": 0.6962
+ },
+ "confidence": 0.337,
+ "answer_confidence": 0.6962,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7988,
+ "confidence": 0.7988,
+ "answer_confidence": 0.7988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7732,
+ "technical": 0.1468,
+ "other": 0.08
+ },
+ "confidence": 0.3786,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8334,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0162,
+ "1": 0.1342,
+ "2": 0.8496
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8132,
+ "confidence": 0.8132,
+ "answer_confidence": 0.8132,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 427,
+ "output_tokens": 0,
+ "state_tokens": 111,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7458,
+ "technical": 0.1647,
+ "other": 0.0895
+ },
+ "confidence": 0.3338,
+ "answer_confidence": 0.7458,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8335,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1332,
+ "2": 0.8501
+ },
+ "confidence": 0.5678,
+ "answer_confidence": 0.8501,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.827,
+ "confidence": 0.827,
+ "answer_confidence": 0.827,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 517,
+ "output_tokens": 0,
+ "state_tokens": 141,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7837,
+ "technical": 0.1379,
+ "other": 0.0784
+ },
+ "confidence": 0.3958,
+ "answer_confidence": 0.7837,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8327,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0171,
+ "1": 0.1331,
+ "2": 0.8498
+ },
+ "confidence": 0.5665,
+ "answer_confidence": 0.8498,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7977,
+ "confidence": 0.7977,
+ "answer_confidence": 0.7977,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 355,
+ "output_tokens": 0,
+ "state_tokens": 87,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7287,
+ "technical": 0.1816,
+ "other": 0.0897
+ },
+ "confidence": 0.3111,
+ "answer_confidence": 0.7287,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8351,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0151,
+ "1": 0.1346,
+ "2": 0.8502
+ },
+ "confidence": 0.571,
+ "answer_confidence": 0.8502,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8224,
+ "confidence": 0.8224,
+ "answer_confidence": 0.8224,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7582,
+ "technical": 0.1488,
+ "other": 0.093
+ },
+ "confidence": 0.3499,
+ "answer_confidence": 0.7582,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.5545,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0538,
+ "1": 0.3378,
+ "2": 0.6083
+ },
+ "confidence": 0.2479,
+ "answer_confidence": 0.6083,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7899,
+ "confidence": 0.7899,
+ "answer_confidence": 0.7899,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7819,
+ "technical": 0.1394,
+ "other": 0.0787
+ },
+ "confidence": 0.3927,
+ "answer_confidence": 0.7819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8342,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0169,
+ "1": 0.1321,
+ "2": 0.8511
+ },
+ "confidence": 0.569,
+ "answer_confidence": 0.8511,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8239,
+ "confidence": 0.8239,
+ "answer_confidence": 0.8239,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 553,
+ "output_tokens": 0,
+ "state_tokens": 153,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7943,
+ "technical": 0.1323,
+ "other": 0.0734
+ },
+ "confidence": 0.4154,
+ "answer_confidence": 0.7943,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8503,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0167,
+ "1": 0.1164,
+ "2": 0.8669
+ },
+ "confidence": 0.5973,
+ "answer_confidence": 0.8669,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8072,
+ "confidence": 0.8072,
+ "answer_confidence": 0.8072,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 391,
+ "output_tokens": 0,
+ "state_tokens": 99,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7677,
+ "technical": 0.1443,
+ "other": 0.088
+ },
+ "confidence": 0.3663,
+ "answer_confidence": 0.7677,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7692,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0234,
+ "1": 0.184,
+ "2": 0.7926
+ },
+ "confidence": 0.4687,
+ "answer_confidence": 0.7926,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7909,
+ "confidence": 0.7909,
+ "answer_confidence": 0.7909,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 319,
+ "output_tokens": 0,
+ "state_tokens": 75,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7312,
+ "technical": 0.1851,
+ "other": 0.0838
+ },
+ "confidence": 0.3184,
+ "answer_confidence": 0.7312,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8478,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0137,
+ "1": 0.1248,
+ "2": 0.8615
+ },
+ "confidence": 0.5931,
+ "answer_confidence": 0.8615,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8255,
+ "confidence": 0.8255,
+ "answer_confidence": 0.8255,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 481,
+ "output_tokens": 0,
+ "state_tokens": 129,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7575,
+ "technical": 0.1576,
+ "other": 0.0849
+ },
+ "confidence": 0.3529,
+ "answer_confidence": 0.7575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8307,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0179,
+ "1": 0.1335,
+ "2": 0.8486
+ },
+ "confidence": 0.5629,
+ "answer_confidence": 0.8486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8174,
+ "confidence": 0.8174,
+ "answer_confidence": 0.8174,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 607,
+ "output_tokens": 0,
+ "state_tokens": 171,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7807,
+ "technical": 0.135,
+ "other": 0.0843
+ },
+ "confidence": 0.3883,
+ "answer_confidence": 0.7807,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8128,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0189,
+ "1": 0.1494,
+ "2": 0.8317
+ },
+ "confidence": 0.5337,
+ "answer_confidence": 0.8317,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7952,
+ "confidence": 0.7952,
+ "answer_confidence": 0.7952,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7583,
+ "technical": 0.1605,
+ "other": 0.0812
+ },
+ "confidence": 0.3561,
+ "answer_confidence": 0.7583,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8375,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0183,
+ "1": 0.1259,
+ "2": 0.8558
+ },
+ "confidence": 0.5746,
+ "answer_confidence": 0.8558,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8111,
+ "confidence": 0.8111,
+ "answer_confidence": 0.8111,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 409,
+ "output_tokens": 0,
+ "state_tokens": 105,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7181,
+ "technical": 0.192,
+ "other": 0.0898
+ },
+ "confidence": 0.2981,
+ "answer_confidence": 0.7181,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8396,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0154,
+ "1": 0.1295,
+ "2": 0.855
+ },
+ "confidence": 0.5785,
+ "answer_confidence": 0.855,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8286,
+ "confidence": 0.8286,
+ "answer_confidence": 0.8286,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 499,
+ "output_tokens": 0,
+ "state_tokens": 135,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7955,
+ "technical": 0.133,
+ "other": 0.0715
+ },
+ "confidence": 0.4184,
+ "answer_confidence": 0.7955,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8318,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0165,
+ "1": 0.1351,
+ "2": 0.8483
+ },
+ "confidence": 0.5651,
+ "answer_confidence": 0.8483,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.809,
+ "confidence": 0.809,
+ "answer_confidence": 0.809,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7676,
+ "technical": 0.1494,
+ "other": 0.083
+ },
+ "confidence": 0.3686,
+ "answer_confidence": 0.7676,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.832,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0176,
+ "1": 0.1328,
+ "2": 0.8496
+ },
+ "confidence": 0.5652,
+ "answer_confidence": 0.8496,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8189,
+ "confidence": 0.8189,
+ "answer_confidence": 0.8189,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 571,
+ "output_tokens": 0,
+ "state_tokens": 159,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7387,
+ "technical": 0.1748,
+ "other": 0.0865
+ },
+ "confidence": 0.3261,
+ "answer_confidence": 0.7387,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8236,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0164,
+ "1": 0.1436,
+ "2": 0.84
+ },
+ "confidence": 0.5517,
+ "answer_confidence": 0.84,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8144,
+ "confidence": 0.8144,
+ "answer_confidence": 0.8144,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 445,
+ "output_tokens": 0,
+ "state_tokens": 117,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7814,
+ "technical": 0.1415,
+ "other": 0.0771
+ },
+ "confidence": 0.3928,
+ "answer_confidence": 0.7814,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8262,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0175,
+ "1": 0.1389,
+ "2": 0.8436
+ },
+ "confidence": 0.5555,
+ "answer_confidence": 0.8436,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.8303,
+ "confidence": 0.8303,
+ "answer_confidence": 0.8303,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 535,
+ "output_tokens": 0,
+ "state_tokens": 147,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 60,
+ 205
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sorted.json b/laya-rust/tests/golden/typed-decisions/case_batch_sorted.json
new file mode 100644
index 00000000..7179adbb
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sorted.json
@@ -0,0 +1,471 @@
+{
+ "name": "batch_sorted",
+ "states": [
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Thanks, the refund arrived and everything works now.",
+ {
+ "ticket": 4821,
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "notes": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "
+ },
+ [
+ {
+ "role": "assistant",
+ "content": "How can I help?"
+ },
+ {
+ "role": "user",
+ "content": "I was billed twice this month and nobody has answered."
+ }
+ ],
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint. Turn 140: the agent restated the refund policy and the customer repeated the complaint. Turn 141: the agent restated the refund policy and the customer repeated the complaint. Turn 142: the agent restated the refund policy and the customer repeated the complaint. Turn 143: the agent restated the refund policy and the customer repeated the complaint. Turn 144: the agent restated the refund policy and the customer repeated the complaint. Turn 145: the agent restated the refund policy and the customer repeated the complaint. Turn 146: the agent restated the refund policy and the customer repeated the complaint. Turn 147: the agent restated the refund policy and the customer repeated the complaint. Turn 148: the agent restated the refund policy and the customer repeated the complaint. Turn 149: the agent restated the refund policy and the customer repeated the complaint. Turn 150: the agent restated the refund policy and the customer repeated the complaint. Turn 151: the agent restated the refund policy and the customer repeated the complaint. Turn 152: the agent restated the refund policy and the customer repeated the complaint. Turn 153: the agent restated the refund policy and the customer repeated the complaint. Turn 154: the agent restated the refund policy and the customer repeated the complaint. Turn 155: the agent restated the refund policy and the customer repeated the complaint. Turn 156: the agent restated the refund policy and the customer repeated the complaint. Turn 157: the agent restated the refund policy and the customer repeated the complaint. Turn 158: the agent restated the refund policy and the customer repeated the complaint. Turn 159: the agent restated the refund policy and the customer repeated the complaint. Turn 160: the agent restated the refund policy and the customer repeated the complaint. Turn 161: the agent restated the refund policy and the customer repeated the complaint. Turn 162: the agent restated the refund policy and the customer repeated the complaint. Turn 163: the agent restated the refund policy and the customer repeated the complaint. Turn 164: the agent restated the refund policy and the customer repeated the complaint. Turn 165: the agent restated the refund policy and the customer repeated the complaint. Turn 166: the agent restated the refund policy and the customer repeated the complaint. Turn 167: the agent restated the refund policy and the customer repeated the complaint. Turn 168: the agent restated the refund policy and the customer repeated the complaint. Turn 169: the agent restated the refund policy and the customer repeated the complaint. Turn 170: the agent restated the refund policy and the customer repeated the complaint. Turn 171: the agent restated the refund policy and the customer repeated the complaint. Turn 172: the agent restated the refund policy and the customer repeated the complaint. Turn 173: the agent restated the refund policy and the customer repeated the complaint. Turn 174: the agent restated the refund policy and the customer repeated the complaint. Turn 175: the agent restated the refund policy and the customer repeated the complaint. Turn 176: the agent restated the refund policy and the customer repeated the complaint. Turn 177: the agent restated the refund policy and the customer repeated the complaint. Turn 178: the agent restated the refund policy and the customer repeated the complaint. Turn 179: the agent restated the refund policy and the customer repeated the complaint. Turn 180: the agent restated the refund policy and the customer repeated the complaint. Turn 181: the agent restated the refund policy and the customer repeated the complaint. Turn 182: the agent restated the refund policy and the customer repeated the complaint. Turn 183: the agent restated the refund policy and the customer repeated the complaint. Turn 184: the agent restated the refund policy and the customer repeated the complaint. Turn 185: the agent restated the refund policy and the customer repeated the complaint. Turn 186: the agent restated the refund policy and the customer repeated the complaint. Turn 187: the agent restated the refund policy and the customer repeated the complaint. Turn 188: the agent restated the refund policy and the customer repeated the complaint. Turn 189: the agent restated the refund policy and the customer repeated the complaint. Turn 190: the agent restated the refund policy and the customer repeated the complaint. Turn 191: the agent restated the refund policy and the customer repeated the complaint. Turn 192: the agent restated the refund policy and the customer repeated the complaint. Turn 193: the agent restated the refund policy and the customer repeated the complaint. Turn 194: the agent restated the refund policy and the customer repeated the complaint. Turn 195: the agent restated the refund policy and the customer repeated the complaint. Turn 196: the agent restated the refund policy and the customer repeated the complaint. Turn 197: the agent restated the refund policy and the customer repeated the complaint. Turn 198: the agent restated the refund policy and the customer repeated the complaint. Turn 199: the agent restated the refund policy and the customer repeated the complaint. Turn 200: the agent restated the refund policy and the customer repeated the complaint. Turn 201: the agent restated the refund policy and the customer repeated the complaint. Turn 202: the agent restated the refund policy and the customer repeated the complaint. Turn 203: the agent restated the refund policy and the customer repeated the complaint. Turn 204: the agent restated the refund policy and the customer repeated the complaint. Turn 205: the agent restated the refund policy and the customer repeated the complaint. Turn 206: the agent restated the refund policy and the customer repeated the complaint. Turn 207: the agent restated the refund policy and the customer repeated the complaint. Turn 208: the agent restated the refund policy and the customer repeated the complaint. Turn 209: the agent restated the refund policy and the customer repeated the complaint. Turn 210: the agent restated the refund policy and the customer repeated the complaint. Turn 211: the agent restated the refund policy and the customer repeated the complaint. Turn 212: the agent restated the refund policy and the customer repeated the complaint. Turn 213: the agent restated the refund policy and the customer repeated the complaint. Turn 214: the agent restated the refund policy and the customer repeated the complaint. Turn 215: the agent restated the refund policy and the customer repeated the complaint. Turn 216: the agent restated the refund policy and the customer repeated the complaint. Turn 217: the agent restated the refund policy and the customer repeated the complaint. Turn 218: the agent restated the refund policy and the customer repeated the complaint. Turn 219: the agent restated the refund policy and the customer repeated the complaint. Turn 220: the agent restated the refund policy and the customer repeated the complaint. Turn 221: the agent restated the refund policy and the customer repeated the complaint. Turn 222: the agent restated the refund policy and the customer repeated the complaint. Turn 223: the agent restated the refund policy and the customer repeated the complaint. Turn 224: the agent restated the refund policy and the customer repeated the complaint. Turn 225: the agent restated the refund policy and the customer repeated the complaint. Turn 226: the agent restated the refund policy and the customer repeated the complaint. Turn 227: the agent restated the refund policy and the customer repeated the complaint. Turn 228: the agent restated the refund policy and the customer repeated the complaint. Turn 229: the agent restated the refund policy and the customer repeated the complaint. Turn 230: the agent restated the refund policy and the customer repeated the complaint. Turn 231: the agent restated the refund policy and the customer repeated the complaint. Turn 232: the agent restated the refund policy and the customer repeated the complaint. Turn 233: the agent restated the refund policy and the customer repeated the complaint. Turn 234: the agent restated the refund policy and the customer repeated the complaint. Turn 235: the agent restated the refund policy and the customer repeated the complaint. Turn 236: the agent restated the refund policy and the customer repeated the complaint. Turn 237: the agent restated the refund policy and the customer repeated the complaint. Turn 238: the agent restated the refund policy and the customer repeated the complaint. Turn 239: the agent restated the refund policy and the customer repeated the complaint. Turn 240: the agent restated the refund policy and the customer repeated the complaint. Turn 241: the agent restated the refund policy and the customer repeated the complaint. Turn 242: the agent restated the refund policy and the customer repeated the complaint. Turn 243: the agent restated the refund policy and the customer repeated the complaint. Turn 244: the agent restated the refund policy and the customer repeated the complaint. Turn 245: the agent restated the refund policy and the customer repeated the complaint. Turn 246: the agent restated the refund policy and the customer repeated the complaint. Turn 247: the agent restated the refund policy and the customer repeated the complaint. Turn 248: the agent restated the refund policy and the customer repeated the complaint. Turn 249: the agent restated the refund policy and the customer repeated the complaint. Turn 250: the agent restated the refund policy and the customer repeated the complaint. Turn 251: the agent restated the refund policy and the customer repeated the complaint. Turn 252: the agent restated the refund policy and the customer repeated the complaint. Turn 253: the agent restated the refund policy and the customer repeated the complaint. Turn 254: the agent restated the refund policy and the customer repeated the complaint. Turn 255: the agent restated the refund policy and the customer repeated the complaint. Turn 256: the agent restated the refund policy and the customer repeated the complaint. Turn 257: the agent restated the refund policy and the customer repeated the complaint. Turn 258: the agent restated the refund policy and the customer repeated the complaint. Turn 259: the agent restated the refund policy and the customer repeated the complaint. Turn 260: the agent restated the refund policy and the customer repeated the complaint. Turn 261: the agent restated the refund policy and the customer repeated the complaint. Turn 262: the agent restated the refund policy and the customer repeated the complaint. Turn 263: the agent restated the refund policy and the customer repeated the complaint. Turn 264: the agent restated the refund policy and the customer repeated the complaint. Turn 265: the agent restated the refund policy and the customer repeated the complaint. Turn 266: the agent restated the refund policy and the customer repeated the complaint. Turn 267: the agent restated the refund policy and the customer repeated the complaint. Turn 268: the agent restated the refund policy and the customer repeated the complaint. Turn 269: the agent restated the refund policy and the customer repeated the complaint. Turn 270: the agent restated the refund policy and the customer repeated the complaint. Turn 271: the agent restated the refund policy and the customer repeated the complaint. Turn 272: the agent restated the refund policy and the customer repeated the complaint. Turn 273: the agent restated the refund policy and the customer repeated the complaint. Turn 274: the agent restated the refund policy and the customer repeated the complaint. Turn 275: the agent restated the refund policy and the customer repeated the complaint. Turn 276: the agent restated the refund policy and the customer repeated the complaint. Turn 277: the agent restated the refund policy and the customer repeated the complaint. Turn 278: the agent restated the refund policy and the customer repeated the complaint. Turn 279: the agent restated the refund policy and the customer repeated the complaint. Turn 280: the agent restated the refund policy and the customer repeated the complaint. Turn 281: the agent restated the refund policy and the customer repeated the complaint. Turn 282: the agent restated the refund policy and the customer repeated the complaint. Turn 283: the agent restated the refund policy and the customer repeated the complaint. Turn 284: the agent restated the refund policy and the customer repeated the complaint. Turn 285: the agent restated the refund policy and the customer repeated the complaint. Turn 286: the agent restated the refund policy and the customer repeated the complaint. Turn 287: the agent restated the refund policy and the customer repeated the complaint. Turn 288: the agent restated the refund policy and the customer repeated the complaint. Turn 289: the agent restated the refund policy and the customer repeated the complaint. Turn 290: the agent restated the refund policy and the customer repeated the complaint. Turn 291: the agent restated the refund policy and the customer repeated the complaint. Turn 292: the agent restated the refund policy and the customer repeated the complaint. Turn 293: the agent restated the refund policy and the customer repeated the complaint. Turn 294: the agent restated the refund policy and the customer repeated the complaint. Turn 295: the agent restated the refund policy and the customer repeated the complaint. Turn 296: the agent restated the refund policy and the customer repeated the complaint. Turn 297: the agent restated the refund policy and the customer repeated the complaint. Turn 298: the agent restated the refund policy and the customer repeated the complaint. Turn 299: the agent restated the refund policy and the customer repeated the complaint. Turn 300: the agent restated the refund policy and the customer repeated the complaint. Turn 301: the agent restated the refund policy and the customer repeated the complaint. Turn 302: the agent restated the refund policy and the customer repeated the complaint. Turn 303: the agent restated the refund policy and the customer repeated the complaint. Turn 304: the agent restated the refund policy and the customer repeated the complaint. Turn 305: the agent restated the refund policy and the customer repeated the complaint. Turn 306: the agent restated the refund policy and the customer repeated the complaint. Turn 307: the agent restated the refund policy and the customer repeated the complaint. Turn 308: the agent restated the refund policy and the customer repeated the complaint. Turn 309: the agent restated the refund policy and the customer repeated the complaint. Turn 310: the agent restated the refund policy and the customer repeated the complaint. Turn 311: the agent restated the refund policy and the customer repeated the complaint. Turn 312: the agent restated the refund policy and the customer repeated the complaint. Turn 313: the agent restated the refund policy and the customer repeated the complaint. Turn 314: the agent restated the refund policy and the customer repeated the complaint. Turn 315: the agent restated the refund policy and the customer repeated the complaint. Turn 316: the agent restated the refund policy and the customer repeated the complaint. Turn 317: the agent restated the refund policy and the customer repeated the complaint. Turn 318: the agent restated the refund policy and the customer repeated the complaint. Turn 319: the agent restated the refund policy and the customer repeated the complaint. Turn 320: the agent restated the refund policy and the customer repeated the complaint. Turn 321: the agent restated the refund policy and the customer repeated the complaint. Turn 322: the agent restated the refund policy and the customer repeated the complaint. Turn 323: the agent restated the refund policy and the customer repeated the complaint. Turn 324: the agent restated the refund policy and the customer repeated the complaint. Turn 325: the agent restated the refund policy and the customer repeated the complaint. Turn 326: the agent restated the refund policy and the customer repeated the complaint. Turn 327: the agent restated the refund policy and the customer repeated the complaint. Turn 328: the agent restated the refund policy and the customer repeated the complaint. Turn 329: the agent restated the refund policy and the customer repeated the complaint. Turn 330: the agent restated the refund policy and the customer repeated the complaint. Turn 331: the agent restated the refund policy and the customer repeated the complaint. Turn 332: the agent restated the refund policy and the customer repeated the complaint. Turn 333: the agent restated the refund policy and the customer repeated the complaint. Turn 334: the agent restated the refund policy and the customer repeated the complaint. Turn 335: the agent restated the refund policy and the customer repeated the complaint. Turn 336: the agent restated the refund policy and the customer repeated the complaint. Turn 337: the agent restated the refund policy and the customer repeated the complaint. Turn 338: the agent restated the refund policy and the customer repeated the complaint. Turn 339: the agent restated the refund policy and the customer repeated the complaint. Turn 340: the agent restated the refund policy and the customer repeated the complaint. Turn 341: the agent restated the refund policy and the customer repeated the complaint. Turn 342: the agent restated the refund policy and the customer repeated the complaint. Turn 343: the agent restated the refund policy and the customer repeated the complaint. Turn 344: the agent restated the refund policy and the customer repeated the complaint. Turn 345: the agent restated the refund policy and the customer repeated the complaint. Turn 346: the agent restated the refund policy and the customer repeated the complaint. Turn 347: the agent restated the refund policy and the customer repeated the complaint. Turn 348: the agent restated the refund policy and the customer repeated the complaint. Turn 349: the agent restated the refund policy and the customer repeated the complaint. Turn 350: the agent restated the refund policy and the customer repeated the complaint. Turn 351: the agent restated the refund policy and the customer repeated the complaint. Turn 352: the agent restated the refund policy and the customer repeated the complaint. Turn 353: the agent restated the refund policy and the customer repeated the complaint. Turn 354: the agent restated the refund policy and the customer repeated the complaint. Turn 355: the agent restated the refund policy and the customer repeated the complaint. Turn 356: the agent restated the refund policy and the customer repeated the complaint. Turn 357: the agent restated the refund policy and the customer repeated the complaint. Turn 358: the agent restated the refund policy and the customer repeated the complaint. Turn 359: the agent restated the refund policy and the customer repeated the complaint. Turn 360: the agent restated the refund policy and the customer repeated the complaint. Turn 361: the agent restated the refund policy and the customer repeated the complaint. Turn 362: the agent restated the refund policy and the customer repeated the complaint. Turn 363: the agent restated the refund policy and the customer repeated the complaint. Turn 364: the agent restated the refund policy and the customer repeated the complaint. Turn 365: the agent restated the refund policy and the customer repeated the complaint. Turn 366: the agent restated the refund policy and the customer repeated the complaint. Turn 367: the agent restated the refund policy and the customer repeated the complaint. Turn 368: the agent restated the refund policy and the customer repeated the complaint. Turn 369: the agent restated the refund policy and the customer repeated the complaint. Turn 370: the agent restated the refund policy and the customer repeated the complaint. Turn 371: the agent restated the refund policy and the customer repeated the complaint. Turn 372: the agent restated the refund policy and the customer repeated the complaint. Turn 373: the agent restated the refund policy and the customer repeated the complaint. Turn 374: the agent restated the refund policy and the customer repeated the complaint. Turn 375: the agent restated the refund policy and the customer repeated the complaint. Turn 376: the agent restated the refund policy and the customer repeated the complaint. Turn 377: the agent restated the refund policy and the customer repeated the complaint. Turn 378: the agent restated the refund policy and the customer repeated the complaint. Turn 379: the agent restated the refund policy and the customer repeated the complaint. Turn 380: the agent restated the refund policy and the customer repeated the complaint. Turn 381: the agent restated the refund policy and the customer repeated the complaint. Turn 382: the agent restated the refund policy and the customer repeated the complaint. Turn 383: the agent restated the refund policy and the customer repeated the complaint. Turn 384: the agent restated the refund policy and the customer repeated the complaint. Turn 385: the agent restated the refund policy and the customer repeated the complaint. Turn 386: the agent restated the refund policy and the customer repeated the complaint. Turn 387: the agent restated the refund policy and the customer repeated the complaint. Turn 388: the agent restated the refund policy and the customer repeated the complaint. Turn 389: the agent restated the refund policy and the customer repeated the complaint. Turn 390: the agent restated the refund policy and the customer repeated the complaint. Turn 391: the agent restated the refund policy and the customer repeated the complaint. Turn 392: the agent restated the refund policy and the customer repeated the complaint. Turn 393: the agent restated the refund policy and the customer repeated the complaint. Turn 394: the agent restated the refund policy and the customer repeated the complaint. Turn 395: the agent restated the refund policy and the customer repeated the complaint. Turn 396: the agent restated the refund policy and the customer repeated the complaint. Turn 397: the agent restated the refund policy and the customer repeated the complaint. Turn 398: the agent restated the refund policy and the customer repeated the complaint. Turn 399: the agent restated the refund policy and the customer repeated the complaint.",
+ "The app crashes on login since yesterday's update.",
+ "ok"
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 91,
+ 44,
+ 157,
+ 74,
+ 1024,
+ 44,
+ 35
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4925,
+ "technical": 0.3304,
+ "other": 0.1771
+ },
+ "confidence": 0.0704,
+ "answer_confidence": 0.4925,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2205,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.2895,
+ "1": 0.2006,
+ "2": 0.51
+ },
+ "confidence": 0.0675,
+ "answer_confidence": 0.51,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0363,
+ "confidence": 0.9637,
+ "answer_confidence": 0.9637,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5409,
+ "technical": 0.2757,
+ "other": 0.1834
+ },
+ "confidence": 0.0909,
+ "answer_confidence": 0.5409,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8074,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0273,
+ "1": 0.138,
+ "2": 0.8347
+ },
+ "confidence": 0.5244,
+ "answer_confidence": 0.8347,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7748,
+ "confidence": 0.7748,
+ "answer_confidence": 0.7748,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 463,
+ "output_tokens": 0,
+ "state_tokens": 123,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.5116,
+ "technical": 0.1679,
+ "other": 0.3205
+ },
+ "confidence": 0.0832,
+ "answer_confidence": 0.5116,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.2127,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1436,
+ "1": 0.5002,
+ "2": 0.3562
+ },
+ "confidence": 0.0963,
+ "answer_confidence": 0.5002,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2121,
+ "confidence": 0.7879,
+ "answer_confidence": 0.7879,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 214,
+ "output_tokens": 0,
+ "state_tokens": 40,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4553,
+ "technical": 0.1506,
+ "other": 0.3941
+ },
+ "confidence": 0.0804,
+ "answer_confidence": 0.4553,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9401,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3838,
+ "1": 0.2923,
+ "2": 0.3239
+ },
+ "confidence": 0.0058,
+ "answer_confidence": 0.3838,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1726,
+ "confidence": 0.8274,
+ "answer_confidence": 0.8274,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 6804,
+ "state_tokens_dropped": 5814,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "technical",
+ "probabilities": {
+ "refund": 0.0616,
+ "technical": 0.7487,
+ "other": 0.1897
+ },
+ "confidence": 0.3594,
+ "answer_confidence": 0.7487,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.4629,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1179,
+ "1": 0.3013,
+ "2": 0.5808
+ },
+ "confidence": 0.1543,
+ "answer_confidence": 0.5808,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.5343,
+ "confidence": 0.5343,
+ "answer_confidence": 0.5343,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 124,
+ "output_tokens": 0,
+ "state_tokens": 10,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.3803,
+ "technical": 0.2662,
+ "other": 0.3535
+ },
+ "confidence": 0.01,
+ "answer_confidence": 0.3803,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8546,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3938,
+ "1": 0.3579,
+ "2": 0.2483
+ },
+ "confidence": 0.0164,
+ "answer_confidence": 0.3938,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.3012,
+ "confidence": 0.6988,
+ "answer_confidence": 0.6988,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 97,
+ "output_tokens": 0,
+ "state_tokens": 1,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 44
+ ],
+ [
+ 6,
+ 74
+ ],
+ [
+ 6,
+ 157
+ ],
+ [
+ 3,
+ 1024
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_sorted_windows.json b/laya-rust/tests/golden/typed-decisions/case_batch_sorted_windows.json
new file mode 100644
index 00000000..494c8f20
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_sorted_windows.json
@@ -0,0 +1,1101 @@
+{
+ "name": "batch_sorted_windows",
+ "states": [
+ "Routine status note 0. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 1. Routine status note 1. ",
+ "Routine status note 2. Routine status note 2. Routine status note 2. ",
+ "Routine status note 3. Routine status note 3. Routine status note 3. Routine status note 3. ",
+ "Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. ",
+ "Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. ",
+ "Routine status note 7. ",
+ "Routine status note 8. Routine status note 8. ",
+ "Routine status note 9. Routine status note 9. Routine status note 9. ",
+ "Routine status note 10. Routine status note 10. Routine status note 10. Routine status note 10. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. ",
+ "Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. ",
+ "Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. ",
+ "Routine status note 14. ",
+ "Routine status note 15. Routine status note 15. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 16. Routine status note 16. Routine status note 16. ",
+ "Routine status note 17. Routine status note 17. Routine status note 17. Routine status note 17. "
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 97,
+ 47,
+ 53,
+ 59,
+ 65,
+ 127,
+ 77,
+ 41,
+ 47,
+ 53,
+ 115,
+ 65,
+ 71,
+ 77,
+ 41,
+ 103,
+ 53,
+ 59
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7222,
+ "technical": 0.1761,
+ "other": 0.1017
+ },
+ "confidence": 0.2961,
+ "answer_confidence": 0.7222,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7209,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0356,
+ "1": 0.2079,
+ "2": 0.7565
+ },
+ "confidence": 0.4025,
+ "answer_confidence": 0.7565,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7675,
+ "confidence": 0.7675,
+ "answer_confidence": 0.7675,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.173,
+ "technical": 0.2872,
+ "other": 0.5397
+ },
+ "confidence": 0.0946,
+ "answer_confidence": 0.5397,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8082,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4468,
+ "1": 0.2981,
+ "2": 0.2551
+ },
+ "confidence": 0.0267,
+ "answer_confidence": 0.4468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1223,
+ "confidence": 0.8777,
+ "answer_confidence": 0.8777,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1405,
+ "technical": 0.2954,
+ "other": 0.5642
+ },
+ "confidence": 0.1272,
+ "answer_confidence": 0.5642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0013,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.345,
+ "1": 0.3087,
+ "2": 0.3463
+ },
+ "confidence": 0.0013,
+ "answer_confidence": 0.3463,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1068,
+ "confidence": 0.8932,
+ "answer_confidence": 0.8932,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1397,
+ "technical": 0.3011,
+ "other": 0.5591
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5591,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9601,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3593,
+ "1": 0.3212,
+ "2": 0.3195
+ },
+ "confidence": 0.0014,
+ "answer_confidence": 0.3593,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1071,
+ "confidence": 0.8929,
+ "answer_confidence": 0.8929,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1414,
+ "technical": 0.3011,
+ "other": 0.5575
+ },
+ "confidence": 0.1228,
+ "answer_confidence": 0.5575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8981,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3945,
+ "1": 0.313,
+ "2": 0.2926
+ },
+ "confidence": 0.0078,
+ "answer_confidence": 0.3945,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0955,
+ "confidence": 0.9045,
+ "answer_confidence": 0.9045,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7204,
+ "technical": 0.197,
+ "other": 0.0826
+ },
+ "confidence": 0.3061,
+ "answer_confidence": 0.7204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7992,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0198,
+ "1": 0.1612,
+ "2": 0.819
+ },
+ "confidence": 0.5126,
+ "answer_confidence": 0.819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7907,
+ "confidence": 0.7907,
+ "answer_confidence": 0.7907,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1401,
+ "technical": 0.319,
+ "other": 0.541
+ },
+ "confidence": 0.1151,
+ "answer_confidence": 0.541,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7457,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4863,
+ "1": 0.2818,
+ "2": 0.2319
+ },
+ "confidence": 0.0475,
+ "answer_confidence": 0.4863,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0991,
+ "confidence": 0.9009,
+ "answer_confidence": 0.9009,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1908,
+ "technical": 0.2627,
+ "other": 0.5465
+ },
+ "confidence": 0.0921,
+ "answer_confidence": 0.5465,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8267,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.431,
+ "1": 0.3113,
+ "2": 0.2577
+ },
+ "confidence": 0.0211,
+ "answer_confidence": 0.431,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1412,
+ "confidence": 0.8588,
+ "answer_confidence": 0.8588,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1548,
+ "technical": 0.2982,
+ "other": 0.547
+ },
+ "confidence": 0.1083,
+ "answer_confidence": 0.547,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8704,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4209,
+ "1": 0.2879,
+ "2": 0.2913
+ },
+ "confidence": 0.0151,
+ "answer_confidence": 0.4209,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1222,
+ "confidence": 0.8778,
+ "answer_confidence": 0.8778,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1351,
+ "technical": 0.3056,
+ "other": 0.5592
+ },
+ "confidence": 0.1282,
+ "answer_confidence": 0.5592,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8124,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.446,
+ "1": 0.2956,
+ "2": 0.2584
+ },
+ "confidence": 0.026,
+ "answer_confidence": 0.446,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1097,
+ "confidence": 0.8903,
+ "answer_confidence": 0.8903,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7135,
+ "technical": 0.1905,
+ "other": 0.096
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.7135,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8225,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0182,
+ "1": 0.1412,
+ "2": 0.8406
+ },
+ "confidence": 0.5493,
+ "answer_confidence": 0.8406,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7732,
+ "confidence": 0.7732,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1358,
+ "technical": 0.3244,
+ "other": 0.5398
+ },
+ "confidence": 0.1178,
+ "answer_confidence": 0.5398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.827,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4369,
+ "1": 0.2992,
+ "2": 0.2639
+ },
+ "confidence": 0.0221,
+ "answer_confidence": 0.4369,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1021,
+ "confidence": 0.8979,
+ "answer_confidence": 0.8979,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1337,
+ "technical": 0.3239,
+ "other": 0.5424
+ },
+ "confidence": 0.1208,
+ "answer_confidence": 0.5424,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8311,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4345,
+ "1": 0.2998,
+ "2": 0.2656
+ },
+ "confidence": 0.0211,
+ "answer_confidence": 0.4345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1031,
+ "confidence": 0.8969,
+ "answer_confidence": 0.8969,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 205,
+ "output_tokens": 0,
+ "state_tokens": 37,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1339,
+ "technical": 0.3298,
+ "other": 0.5363
+ },
+ "confidence": 0.1178,
+ "answer_confidence": 0.5363,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.74,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4864,
+ "1": 0.2872,
+ "2": 0.2264
+ },
+ "confidence": 0.0486,
+ "answer_confidence": 0.4864,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1028,
+ "confidence": 0.8972,
+ "answer_confidence": 0.8972,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1964,
+ "technical": 0.2657,
+ "other": 0.5379
+ },
+ "confidence": 0.0849,
+ "answer_confidence": 0.5379,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.836,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4238,
+ "1": 0.3164,
+ "2": 0.2598
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4238,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1404,
+ "confidence": 0.8596,
+ "answer_confidence": 0.8596,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6932,
+ "technical": 0.2125,
+ "other": 0.0943
+ },
+ "confidence": 0.2665,
+ "answer_confidence": 0.6932,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7768,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0217,
+ "1": 0.1798,
+ "2": 0.7985
+ },
+ "confidence": 0.48,
+ "answer_confidence": 0.7985,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7737,
+ "confidence": 0.7737,
+ "answer_confidence": 0.7737,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.134,
+ "technical": 0.3007,
+ "other": 0.5653
+ },
+ "confidence": 0.1325,
+ "answer_confidence": 0.5653,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8129,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4444,
+ "1": 0.2983,
+ "2": 0.2573
+ },
+ "confidence": 0.0256,
+ "answer_confidence": 0.4444,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1135,
+ "confidence": 0.8865,
+ "answer_confidence": 0.8865,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1429,
+ "technical": 0.3085,
+ "other": 0.5486
+ },
+ "confidence": 0.1168,
+ "answer_confidence": 0.5486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8114,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4437,
+ "1": 0.3012,
+ "2": 0.2551
+ },
+ "confidence": 0.0256,
+ "answer_confidence": 0.4437,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1153,
+ "confidence": 0.8847,
+ "answer_confidence": 0.8847,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 41
+ ],
+ [
+ 6,
+ 47
+ ],
+ [
+ 6,
+ 53
+ ],
+ [
+ 6,
+ 65
+ ],
+ [
+ 6,
+ 71
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 103
+ ],
+ [
+ 6,
+ 127
+ ],
+ [
+ 6,
+ 59
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_truncated.json b/laya-rust/tests/golden/typed-decisions/case_batch_truncated.json
new file mode 100644
index 00000000..8e225e6d
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_truncated.json
@@ -0,0 +1,854 @@
+{
+ "name": "batch_truncated",
+ "states": [
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ [
+ {
+ "role": "user",
+ "content": "Turn 0: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 1: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 2: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 3: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 4: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 5: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 6: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 7: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 8: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 9: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 10: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 11: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 12: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 13: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 14: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 15: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 16: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 17: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 18: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 19: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 20: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 21: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 22: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 23: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 24: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 25: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 26: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 27: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 28: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 29: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 30: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 31: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 32: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 33: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 34: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 35: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 36: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 37: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 38: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 39: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 40: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 41: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 42: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 43: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 44: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 45: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 46: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 47: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 48: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 49: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 50: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 51: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 52: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 53: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 54: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 55: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 56: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 57: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 58: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 59: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 60: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 61: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 62: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 63: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 64: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 65: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 66: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 67: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 68: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 69: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 70: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 71: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 72: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 73: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 74: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 75: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 76: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 77: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 78: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 79: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 80: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 81: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 82: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 83: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 84: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 85: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 86: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 87: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 88: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 89: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 90: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 91: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 92: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 93: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 94: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 95: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 96: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 97: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 98: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 99: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 100: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 101: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 102: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 103: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 104: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 105: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 106: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 107: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 108: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 109: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 110: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 111: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 112: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 113: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 114: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 115: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 116: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 117: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 118: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 119: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 120: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 121: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 122: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 123: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 124: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 125: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 126: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 127: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 128: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 129: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 130: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 131: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 132: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 133: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 134: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 135: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 136: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 137: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 138: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 139: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "I want a refund or I will cancel today."
+ }
+ ],
+ {
+ "notes": "Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 1024,
+ 1024,
+ 1024,
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4553,
+ "technical": 0.1506,
+ "other": 0.3941
+ },
+ "confidence": 0.0804,
+ "answer_confidence": 0.4553,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9401,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3838,
+ "1": 0.2923,
+ "2": 0.3239
+ },
+ "confidence": 0.0058,
+ "answer_confidence": 0.3838,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1726,
+ "confidence": 0.8274,
+ "answer_confidence": 0.8274,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2384,
+ "state_tokens_dropped": 1394,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7622,
+ "technical": 0.0639,
+ "other": 0.1739
+ },
+ "confidence": 0.3748,
+ "answer_confidence": 0.7622,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.562,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1098,
+ "1": 0.2184,
+ "2": 0.6718
+ },
+ "confidence": 0.2335,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.6263,
+ "confidence": 0.6263,
+ "answer_confidence": 0.6263,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 4012,
+ "state_tokens_dropped": 3022,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.444,
+ "technical": 0.1555,
+ "other": 0.4005
+ },
+ "confidence": 0.0749,
+ "answer_confidence": 0.444,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8498,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4282,
+ "1": 0.2938,
+ "2": 0.278
+ },
+ "confidence": 0.0179,
+ "answer_confidence": 0.4282,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2183,
+ "confidence": 0.7817,
+ "answer_confidence": 0.7817,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2448,
+ "state_tokens_dropped": 1458,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 1024
+ ],
+ [
+ 6,
+ 1024
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_truncated_sorted.json b/laya-rust/tests/golden/typed-decisions/case_batch_truncated_sorted.json
new file mode 100644
index 00000000..96581bb1
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_truncated_sorted.json
@@ -0,0 +1,855 @@
+{
+ "name": "batch_truncated_sorted",
+ "states": [
+ "Customer support transcript. Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ [
+ {
+ "role": "user",
+ "content": "Turn 0: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 1: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 2: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 3: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 4: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 5: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 6: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 7: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 8: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 9: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 10: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 11: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 12: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 13: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 14: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 15: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 16: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 17: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 18: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 19: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 20: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 21: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 22: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 23: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 24: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 25: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 26: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 27: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 28: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 29: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 30: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 31: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 32: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 33: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 34: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 35: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 36: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 37: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 38: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 39: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 40: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 41: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 42: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 43: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 44: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 45: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 46: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 47: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 48: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 49: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 50: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 51: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 52: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 53: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 54: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 55: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 56: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 57: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 58: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 59: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 60: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 61: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 62: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 63: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 64: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 65: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 66: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 67: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 68: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 69: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 70: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 71: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 72: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 73: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 74: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 75: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 76: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 77: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 78: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 79: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 80: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 81: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 82: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 83: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 84: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 85: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 86: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 87: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 88: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 89: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 90: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 91: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 92: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 93: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 94: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 95: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 96: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 97: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 98: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 99: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 100: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 101: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 102: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 103: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 104: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 105: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 106: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 107: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 108: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 109: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 110: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 111: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 112: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 113: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 114: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 115: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 116: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 117: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 118: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 119: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 120: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 121: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 122: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 123: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 124: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 125: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 126: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 127: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 128: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 129: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 130: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 131: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 132: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 133: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 134: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 135: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 136: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 137: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "Turn 138: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "assistant",
+ "content": "Turn 139: the agent restated the refund policy and the customer repeated the complaint."
+ },
+ {
+ "role": "user",
+ "content": "I want a refund or I will cancel today."
+ }
+ ],
+ {
+ "notes": "Turn 0: the agent restated the refund policy and the customer repeated the complaint. Turn 1: the agent restated the refund policy and the customer repeated the complaint. Turn 2: the agent restated the refund policy and the customer repeated the complaint. Turn 3: the agent restated the refund policy and the customer repeated the complaint. Turn 4: the agent restated the refund policy and the customer repeated the complaint. Turn 5: the agent restated the refund policy and the customer repeated the complaint. Turn 6: the agent restated the refund policy and the customer repeated the complaint. Turn 7: the agent restated the refund policy and the customer repeated the complaint. Turn 8: the agent restated the refund policy and the customer repeated the complaint. Turn 9: the agent restated the refund policy and the customer repeated the complaint. Turn 10: the agent restated the refund policy and the customer repeated the complaint. Turn 11: the agent restated the refund policy and the customer repeated the complaint. Turn 12: the agent restated the refund policy and the customer repeated the complaint. Turn 13: the agent restated the refund policy and the customer repeated the complaint. Turn 14: the agent restated the refund policy and the customer repeated the complaint. Turn 15: the agent restated the refund policy and the customer repeated the complaint. Turn 16: the agent restated the refund policy and the customer repeated the complaint. Turn 17: the agent restated the refund policy and the customer repeated the complaint. Turn 18: the agent restated the refund policy and the customer repeated the complaint. Turn 19: the agent restated the refund policy and the customer repeated the complaint. Turn 20: the agent restated the refund policy and the customer repeated the complaint. Turn 21: the agent restated the refund policy and the customer repeated the complaint. Turn 22: the agent restated the refund policy and the customer repeated the complaint. Turn 23: the agent restated the refund policy and the customer repeated the complaint. Turn 24: the agent restated the refund policy and the customer repeated the complaint. Turn 25: the agent restated the refund policy and the customer repeated the complaint. Turn 26: the agent restated the refund policy and the customer repeated the complaint. Turn 27: the agent restated the refund policy and the customer repeated the complaint. Turn 28: the agent restated the refund policy and the customer repeated the complaint. Turn 29: the agent restated the refund policy and the customer repeated the complaint. Turn 30: the agent restated the refund policy and the customer repeated the complaint. Turn 31: the agent restated the refund policy and the customer repeated the complaint. Turn 32: the agent restated the refund policy and the customer repeated the complaint. Turn 33: the agent restated the refund policy and the customer repeated the complaint. Turn 34: the agent restated the refund policy and the customer repeated the complaint. Turn 35: the agent restated the refund policy and the customer repeated the complaint. Turn 36: the agent restated the refund policy and the customer repeated the complaint. Turn 37: the agent restated the refund policy and the customer repeated the complaint. Turn 38: the agent restated the refund policy and the customer repeated the complaint. Turn 39: the agent restated the refund policy and the customer repeated the complaint. Turn 40: the agent restated the refund policy and the customer repeated the complaint. Turn 41: the agent restated the refund policy and the customer repeated the complaint. Turn 42: the agent restated the refund policy and the customer repeated the complaint. Turn 43: the agent restated the refund policy and the customer repeated the complaint. Turn 44: the agent restated the refund policy and the customer repeated the complaint. Turn 45: the agent restated the refund policy and the customer repeated the complaint. Turn 46: the agent restated the refund policy and the customer repeated the complaint. Turn 47: the agent restated the refund policy and the customer repeated the complaint. Turn 48: the agent restated the refund policy and the customer repeated the complaint. Turn 49: the agent restated the refund policy and the customer repeated the complaint. Turn 50: the agent restated the refund policy and the customer repeated the complaint. Turn 51: the agent restated the refund policy and the customer repeated the complaint. Turn 52: the agent restated the refund policy and the customer repeated the complaint. Turn 53: the agent restated the refund policy and the customer repeated the complaint. Turn 54: the agent restated the refund policy and the customer repeated the complaint. Turn 55: the agent restated the refund policy and the customer repeated the complaint. Turn 56: the agent restated the refund policy and the customer repeated the complaint. Turn 57: the agent restated the refund policy and the customer repeated the complaint. Turn 58: the agent restated the refund policy and the customer repeated the complaint. Turn 59: the agent restated the refund policy and the customer repeated the complaint. Turn 60: the agent restated the refund policy and the customer repeated the complaint. Turn 61: the agent restated the refund policy and the customer repeated the complaint. Turn 62: the agent restated the refund policy and the customer repeated the complaint. Turn 63: the agent restated the refund policy and the customer repeated the complaint. Turn 64: the agent restated the refund policy and the customer repeated the complaint. Turn 65: the agent restated the refund policy and the customer repeated the complaint. Turn 66: the agent restated the refund policy and the customer repeated the complaint. Turn 67: the agent restated the refund policy and the customer repeated the complaint. Turn 68: the agent restated the refund policy and the customer repeated the complaint. Turn 69: the agent restated the refund policy and the customer repeated the complaint. Turn 70: the agent restated the refund policy and the customer repeated the complaint. Turn 71: the agent restated the refund policy and the customer repeated the complaint. Turn 72: the agent restated the refund policy and the customer repeated the complaint. Turn 73: the agent restated the refund policy and the customer repeated the complaint. Turn 74: the agent restated the refund policy and the customer repeated the complaint. Turn 75: the agent restated the refund policy and the customer repeated the complaint. Turn 76: the agent restated the refund policy and the customer repeated the complaint. Turn 77: the agent restated the refund policy and the customer repeated the complaint. Turn 78: the agent restated the refund policy and the customer repeated the complaint. Turn 79: the agent restated the refund policy and the customer repeated the complaint. Turn 80: the agent restated the refund policy and the customer repeated the complaint. Turn 81: the agent restated the refund policy and the customer repeated the complaint. Turn 82: the agent restated the refund policy and the customer repeated the complaint. Turn 83: the agent restated the refund policy and the customer repeated the complaint. Turn 84: the agent restated the refund policy and the customer repeated the complaint. Turn 85: the agent restated the refund policy and the customer repeated the complaint. Turn 86: the agent restated the refund policy and the customer repeated the complaint. Turn 87: the agent restated the refund policy and the customer repeated the complaint. Turn 88: the agent restated the refund policy and the customer repeated the complaint. Turn 89: the agent restated the refund policy and the customer repeated the complaint. Turn 90: the agent restated the refund policy and the customer repeated the complaint. Turn 91: the agent restated the refund policy and the customer repeated the complaint. Turn 92: the agent restated the refund policy and the customer repeated the complaint. Turn 93: the agent restated the refund policy and the customer repeated the complaint. Turn 94: the agent restated the refund policy and the customer repeated the complaint. Turn 95: the agent restated the refund policy and the customer repeated the complaint. Turn 96: the agent restated the refund policy and the customer repeated the complaint. Turn 97: the agent restated the refund policy and the customer repeated the complaint. Turn 98: the agent restated the refund policy and the customer repeated the complaint. Turn 99: the agent restated the refund policy and the customer repeated the complaint. Turn 100: the agent restated the refund policy and the customer repeated the complaint. Turn 101: the agent restated the refund policy and the customer repeated the complaint. Turn 102: the agent restated the refund policy and the customer repeated the complaint. Turn 103: the agent restated the refund policy and the customer repeated the complaint. Turn 104: the agent restated the refund policy and the customer repeated the complaint. Turn 105: the agent restated the refund policy and the customer repeated the complaint. Turn 106: the agent restated the refund policy and the customer repeated the complaint. Turn 107: the agent restated the refund policy and the customer repeated the complaint. Turn 108: the agent restated the refund policy and the customer repeated the complaint. Turn 109: the agent restated the refund policy and the customer repeated the complaint. Turn 110: the agent restated the refund policy and the customer repeated the complaint. Turn 111: the agent restated the refund policy and the customer repeated the complaint. Turn 112: the agent restated the refund policy and the customer repeated the complaint. Turn 113: the agent restated the refund policy and the customer repeated the complaint. Turn 114: the agent restated the refund policy and the customer repeated the complaint. Turn 115: the agent restated the refund policy and the customer repeated the complaint. Turn 116: the agent restated the refund policy and the customer repeated the complaint. Turn 117: the agent restated the refund policy and the customer repeated the complaint. Turn 118: the agent restated the refund policy and the customer repeated the complaint. Turn 119: the agent restated the refund policy and the customer repeated the complaint. Turn 120: the agent restated the refund policy and the customer repeated the complaint. Turn 121: the agent restated the refund policy and the customer repeated the complaint. Turn 122: the agent restated the refund policy and the customer repeated the complaint. Turn 123: the agent restated the refund policy and the customer repeated the complaint. Turn 124: the agent restated the refund policy and the customer repeated the complaint. Turn 125: the agent restated the refund policy and the customer repeated the complaint. Turn 126: the agent restated the refund policy and the customer repeated the complaint. Turn 127: the agent restated the refund policy and the customer repeated the complaint. Turn 128: the agent restated the refund policy and the customer repeated the complaint. Turn 129: the agent restated the refund policy and the customer repeated the complaint. Turn 130: the agent restated the refund policy and the customer repeated the complaint. Turn 131: the agent restated the refund policy and the customer repeated the complaint. Turn 132: the agent restated the refund policy and the customer repeated the complaint. Turn 133: the agent restated the refund policy and the customer repeated the complaint. Turn 134: the agent restated the refund policy and the customer repeated the complaint. Turn 135: the agent restated the refund policy and the customer repeated the complaint. Turn 136: the agent restated the refund policy and the customer repeated the complaint. Turn 137: the agent restated the refund policy and the customer repeated the complaint. Turn 138: the agent restated the refund policy and the customer repeated the complaint. Turn 139: the agent restated the refund policy and the customer repeated the complaint.",
+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ },
+ "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 3,
+ "sort_by_length": true
+ },
+ "state_widths": [
+ 1024,
+ 1024,
+ 1024,
+ 91
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.4553,
+ "technical": 0.1506,
+ "other": 0.3941
+ },
+ "confidence": 0.0804,
+ "answer_confidence": 0.4553,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9401,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3838,
+ "1": 0.2923,
+ "2": 0.3239
+ },
+ "confidence": 0.0058,
+ "answer_confidence": 0.3838,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1726,
+ "confidence": 0.8274,
+ "answer_confidence": 0.8274,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2384,
+ "state_tokens_dropped": 1394,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7622,
+ "technical": 0.0639,
+ "other": 0.1739
+ },
+ "confidence": 0.3748,
+ "answer_confidence": 0.7622,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.562,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.1098,
+ "1": 0.2184,
+ "2": 0.6718
+ },
+ "confidence": 0.2335,
+ "answer_confidence": 0.6718,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.6263,
+ "confidence": 0.6263,
+ "answer_confidence": 0.6263,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 4012,
+ "state_tokens_dropped": 3022,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.444,
+ "technical": 0.1555,
+ "other": 0.4005
+ },
+ "confidence": 0.0749,
+ "answer_confidence": 0.444,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8498,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4282,
+ "1": 0.2938,
+ "2": 0.278
+ },
+ "confidence": 0.0179,
+ "answer_confidence": 0.4282,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.2183,
+ "confidence": 0.7817,
+ "answer_confidence": 0.7817,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 3072,
+ "output_tokens": 0,
+ "state_tokens": 2448,
+ "state_tokens_dropped": 1458,
+ "truncated": true,
+ "truncated_questions": [
+ "topic",
+ "urgency",
+ "cancel"
+ ]
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7054,
+ "technical": 0.1834,
+ "other": 0.1111
+ },
+ "confidence": 0.2705,
+ "answer_confidence": 0.7054,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8371,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0193,
+ "1": 0.1243,
+ "2": 0.8564
+ },
+ "confidence": 0.574,
+ "answer_confidence": 0.8564,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7914,
+ "confidence": 0.7914,
+ "answer_confidence": 0.7914,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 265,
+ "output_tokens": 0,
+ "state_tokens": 57,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 9,
+ 1024
+ ],
+ [
+ 3,
+ 1024
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/case_batch_unsorted_windows.json b/laya-rust/tests/golden/typed-decisions/case_batch_unsorted_windows.json
new file mode 100644
index 00000000..3702715d
--- /dev/null
+++ b/laya-rust/tests/golden/typed-decisions/case_batch_unsorted_windows.json
@@ -0,0 +1,1100 @@
+{
+ "name": "batch_unsorted_windows",
+ "states": [
+ "Routine status note 0. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 1. Routine status note 1. ",
+ "Routine status note 2. Routine status note 2. Routine status note 2. ",
+ "Routine status note 3. Routine status note 3. Routine status note 3. Routine status note 3. ",
+ "Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. Routine status note 4. ",
+ "Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Routine status note 5. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. Routine status note 6. ",
+ "Routine status note 7. ",
+ "Routine status note 8. Routine status note 8. ",
+ "Routine status note 9. Routine status note 9. Routine status note 9. ",
+ "Routine status note 10. Routine status note 10. Routine status note 10. Routine status note 10. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. Routine status note 11. ",
+ "Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. Routine status note 12. ",
+ "Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. Routine status note 13. ",
+ "Routine status note 14. ",
+ "Routine status note 15. Routine status note 15. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",
+ "Routine status note 16. Routine status note 16. Routine status note 16. ",
+ "Routine status note 17. Routine status note 17. Routine status note 17. Routine status note 17. "
+ ],
+ "questions": {
+ "topic": {
+ "type": "choice",
+ "instructions": "What is the request about?",
+ "criteria": {
+ "refund": "money back, duplicate charge",
+ "technical": "bugs and outages",
+ "other": null
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "instructions": "How urgent is this request?",
+ "criteria": [
+ "not urgent",
+ "this week",
+ "today"
+ ]
+ },
+ "cancel": {
+ "type": "noul",
+ "instructions": "The customer threatens to cancel the account."
+ }
+ },
+ "options": {
+ "batch_size": 2
+ },
+ "state_widths": [
+ 97,
+ 47,
+ 53,
+ 59,
+ 65,
+ 127,
+ 77,
+ 41,
+ 47,
+ 53,
+ 115,
+ 65,
+ 71,
+ 77,
+ 41,
+ 103,
+ 53,
+ 59
+ ],
+ "results": [
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7222,
+ "technical": 0.1761,
+ "other": 0.1017
+ },
+ "confidence": 0.2961,
+ "answer_confidence": 0.7222,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7209,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0356,
+ "1": 0.2079,
+ "2": 0.7565
+ },
+ "confidence": 0.4025,
+ "answer_confidence": 0.7565,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7675,
+ "confidence": 0.7675,
+ "answer_confidence": 0.7675,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 283,
+ "output_tokens": 0,
+ "state_tokens": 63,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.173,
+ "technical": 0.2872,
+ "other": 0.5397
+ },
+ "confidence": 0.0946,
+ "answer_confidence": 0.5397,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8082,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4468,
+ "1": 0.2981,
+ "2": 0.2551
+ },
+ "confidence": 0.0267,
+ "answer_confidence": 0.4468,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1223,
+ "confidence": 0.8777,
+ "answer_confidence": 0.8777,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1405,
+ "technical": 0.2954,
+ "other": 0.5642
+ },
+ "confidence": 0.1272,
+ "answer_confidence": 0.5642,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.0013,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.345,
+ "1": 0.3087,
+ "2": 0.3463
+ },
+ "confidence": 0.0013,
+ "answer_confidence": 0.3463,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1068,
+ "confidence": 0.8932,
+ "answer_confidence": 0.8932,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1397,
+ "technical": 0.3011,
+ "other": 0.5591
+ },
+ "confidence": 0.1249,
+ "answer_confidence": 0.5591,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.9601,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3593,
+ "1": 0.3212,
+ "2": 0.3195
+ },
+ "confidence": 0.0014,
+ "answer_confidence": 0.3593,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1071,
+ "confidence": 0.8929,
+ "answer_confidence": 0.8929,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1414,
+ "technical": 0.3011,
+ "other": 0.5575
+ },
+ "confidence": 0.1228,
+ "answer_confidence": 0.5575,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8981,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.3945,
+ "1": 0.313,
+ "2": 0.2926
+ },
+ "confidence": 0.0078,
+ "answer_confidence": 0.3945,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0955,
+ "confidence": 0.9045,
+ "answer_confidence": 0.9045,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7204,
+ "technical": 0.197,
+ "other": 0.0826
+ },
+ "confidence": 0.3061,
+ "answer_confidence": 0.7204,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7992,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0198,
+ "1": 0.1612,
+ "2": 0.819
+ },
+ "confidence": 0.5126,
+ "answer_confidence": 0.819,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7907,
+ "confidence": 0.7907,
+ "answer_confidence": 0.7907,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 373,
+ "output_tokens": 0,
+ "state_tokens": 93,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1401,
+ "technical": 0.319,
+ "other": 0.541
+ },
+ "confidence": 0.1151,
+ "answer_confidence": 0.541,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.7457,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4863,
+ "1": 0.2818,
+ "2": 0.2319
+ },
+ "confidence": 0.0475,
+ "answer_confidence": 0.4863,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.0991,
+ "confidence": 0.9009,
+ "answer_confidence": 0.9009,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1908,
+ "technical": 0.2627,
+ "other": 0.5465
+ },
+ "confidence": 0.0921,
+ "answer_confidence": 0.5465,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8267,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.431,
+ "1": 0.3113,
+ "2": 0.2577
+ },
+ "confidence": 0.0211,
+ "answer_confidence": 0.431,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1412,
+ "confidence": 0.8588,
+ "answer_confidence": 0.8588,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1548,
+ "technical": 0.2982,
+ "other": 0.547
+ },
+ "confidence": 0.1083,
+ "answer_confidence": 0.547,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8704,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4209,
+ "1": 0.2879,
+ "2": 0.2913
+ },
+ "confidence": 0.0151,
+ "answer_confidence": 0.4209,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1222,
+ "confidence": 0.8778,
+ "answer_confidence": 0.8778,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 133,
+ "output_tokens": 0,
+ "state_tokens": 13,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1351,
+ "technical": 0.3056,
+ "other": 0.5592
+ },
+ "confidence": 0.1282,
+ "answer_confidence": 0.5592,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8124,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.446,
+ "1": 0.2956,
+ "2": 0.2584
+ },
+ "confidence": 0.026,
+ "answer_confidence": 0.446,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1097,
+ "confidence": 0.8903,
+ "answer_confidence": 0.8903,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.7135,
+ "technical": 0.1905,
+ "other": 0.096
+ },
+ "confidence": 0.2884,
+ "answer_confidence": 0.7135,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.8225,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0182,
+ "1": 0.1412,
+ "2": 0.8406
+ },
+ "confidence": 0.5493,
+ "answer_confidence": 0.8406,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7732,
+ "confidence": 0.7732,
+ "answer_confidence": 0.7732,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 337,
+ "output_tokens": 0,
+ "state_tokens": 81,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1358,
+ "technical": 0.3244,
+ "other": 0.5398
+ },
+ "confidence": 0.1178,
+ "answer_confidence": 0.5398,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.827,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4369,
+ "1": 0.2992,
+ "2": 0.2639
+ },
+ "confidence": 0.0221,
+ "answer_confidence": 0.4369,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1021,
+ "confidence": 0.8979,
+ "answer_confidence": 0.8979,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 187,
+ "output_tokens": 0,
+ "state_tokens": 31,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1337,
+ "technical": 0.3239,
+ "other": 0.5424
+ },
+ "confidence": 0.1208,
+ "answer_confidence": 0.5424,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8311,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4345,
+ "1": 0.2998,
+ "2": 0.2656
+ },
+ "confidence": 0.0211,
+ "answer_confidence": 0.4345,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1031,
+ "confidence": 0.8969,
+ "answer_confidence": 0.8969,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 205,
+ "output_tokens": 0,
+ "state_tokens": 37,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1339,
+ "technical": 0.3298,
+ "other": 0.5363
+ },
+ "confidence": 0.1178,
+ "answer_confidence": 0.5363,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.74,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4864,
+ "1": 0.2872,
+ "2": 0.2264
+ },
+ "confidence": 0.0486,
+ "answer_confidence": 0.4864,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1028,
+ "confidence": 0.8972,
+ "answer_confidence": 0.8972,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 223,
+ "output_tokens": 0,
+ "state_tokens": 43,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1964,
+ "technical": 0.2657,
+ "other": 0.5379
+ },
+ "confidence": 0.0849,
+ "answer_confidence": 0.5379,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.836,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4238,
+ "1": 0.3164,
+ "2": 0.2598
+ },
+ "confidence": 0.0187,
+ "answer_confidence": 0.4238,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1404,
+ "confidence": 0.8596,
+ "answer_confidence": 0.8596,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 115,
+ "output_tokens": 0,
+ "state_tokens": 7,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "refund",
+ "probabilities": {
+ "refund": 0.6932,
+ "technical": 0.2125,
+ "other": 0.0943
+ },
+ "confidence": 0.2665,
+ "answer_confidence": 0.6932,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 1.7768,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.0217,
+ "1": 0.1798,
+ "2": 0.7985
+ },
+ "confidence": 0.48,
+ "answer_confidence": 0.7985,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.7737,
+ "confidence": 0.7737,
+ "answer_confidence": 0.7737,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 301,
+ "output_tokens": 0,
+ "state_tokens": 69,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.134,
+ "technical": 0.3007,
+ "other": 0.5653
+ },
+ "confidence": 0.1325,
+ "answer_confidence": 0.5653,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8129,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4444,
+ "1": 0.2983,
+ "2": 0.2573
+ },
+ "confidence": 0.0256,
+ "answer_confidence": 0.4444,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1135,
+ "confidence": 0.8865,
+ "answer_confidence": 0.8865,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 151,
+ "output_tokens": 0,
+ "state_tokens": 19,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ },
+ {
+ "model": "laya-rl-agent-onnx",
+ "answers": {
+ "topic": {
+ "type": "choice",
+ "choice": "other",
+ "probabilities": {
+ "refund": 0.1429,
+ "technical": 0.3085,
+ "other": 0.5486
+ },
+ "confidence": 0.1168,
+ "answer_confidence": 0.5486,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "urgency": {
+ "type": "score",
+ "score": 0.8114,
+ "legend": {
+ "0": "not urgent",
+ "1": "this week",
+ "2": "today"
+ },
+ "probabilities": {
+ "0": 0.4437,
+ "1": 0.3012,
+ "2": 0.2551
+ },
+ "confidence": 0.0256,
+ "answer_confidence": 0.4437,
+ "action": {
+ "act_probability": 1.0
+ }
+ },
+ "cancel": {
+ "type": "noul",
+ "noul": 0.1153,
+ "confidence": 0.8847,
+ "answer_confidence": 0.8847,
+ "action": {
+ "act_probability": 1.0
+ }
+ }
+ },
+ "usage": {
+ "input_tokens": 169,
+ "output_tokens": 0,
+ "state_tokens": 25,
+ "state_tokens_dropped": 0,
+ "truncated": false,
+ "truncated_questions": []
+ }
+ }
+ ],
+ "runs": [
+ [
+ 6,
+ 97
+ ],
+ [
+ 6,
+ 59
+ ],
+ [
+ 6,
+ 127
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 53
+ ],
+ [
+ 6,
+ 115
+ ],
+ [
+ 6,
+ 77
+ ],
+ [
+ 6,
+ 103
+ ],
+ [
+ 6,
+ 59
+ ]
+ ]
+}
diff --git a/laya-rust/tests/golden/typed-decisions/index.json b/laya-rust/tests/golden/typed-decisions/index.json
index 5c2bf1ce..eb70e281 100644
--- a/laya-rust/tests/golden/typed-decisions/index.json
+++ b/laya-rust/tests/golden/typed-decisions/index.json
@@ -162,5 +162,191 @@
"file": "case_long_no_state_room.json",
"expects_error": true,
"kind": "long"
+ },
+ {
+ "name": "batch_one_state",
+ "file": "case_batch_one_state.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_mixed",
+ "file": "case_batch_mixed.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_1",
+ "file": "case_batch_size_1.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_3",
+ "file": "case_batch_size_3.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_over_len",
+ "file": "case_batch_size_over_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_zero",
+ "file": "case_batch_size_zero.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_size_negative",
+ "file": "case_batch_size_negative.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_on",
+ "file": "case_batch_sort_on.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_off",
+ "file": "case_batch_sort_off.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_3",
+ "file": "case_batch_sort_size_3.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_partial_window",
+ "file": "case_batch_sort_partial_window.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_no_batch_size",
+ "file": "case_batch_sort_no_batch_size.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_1",
+ "file": "case_batch_sort_size_1.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_eq_len",
+ "file": "case_batch_sort_size_eq_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_over_len",
+ "file": "case_batch_sort_size_over_len.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sort_size_zero",
+ "file": "case_batch_sort_size_zero.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_truncated",
+ "file": "case_batch_truncated.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_truncated_sorted",
+ "file": "case_batch_truncated_sorted.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_duplicate_states",
+ "file": "case_batch_duplicate_states.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_states",
+ "file": "case_batch_empty_states.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_states_invalid_questions",
+ "file": "case_batch_empty_states_invalid_questions.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_questions",
+ "file": "case_batch_empty_questions.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_both",
+ "file": "case_batch_empty_both.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sorted",
+ "file": "case_batch_sorted.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_sorted_windows",
+ "file": "case_batch_sorted_windows.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_unsorted_windows",
+ "file": "case_batch_unsorted_windows.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_chunked",
+ "file": "case_batch_chunked.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_single_question",
+ "file": "case_batch_single_question.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_empty_questions_with_options",
+ "file": "case_batch_empty_questions_with_options.json",
+ "expects_error": false,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_none_state",
+ "file": "case_batch_none_state.json",
+ "expects_error": true,
+ "kind": "batch"
+ },
+ {
+ "name": "batch_options_exceed_head",
+ "file": "case_batch_options_exceed_head.json",
+ "expects_error": true,
+ "kind": "batch"
}
]
diff --git a/laya-rust/tests/laya_router_tests.rs b/laya-rust/tests/laya_router_tests.rs
index 4bab8e3d..b5e0ed98 100644
--- a/laya-rust/tests/laya_router_tests.rs
+++ b/laya-rust/tests/laya_router_tests.rs
@@ -10,7 +10,9 @@ use common::routing_golden;
use laya::answers::{LayaResult, Usage};
use laya::error::Result;
use laya::laya_options::LayaCheckpoint;
-use laya::laya_router::{GuessInput, LangGuess, LayaPredictor, LayaRouter, LayaRouterOptions};
+use laya::laya_router::{
+ BatchRequest, GuessInput, LangGuess, LayaPredictor, LayaRouter, LayaRouterOptions,
+};
use laya::questions::QuestionSet;
use serde_json::{Value, json};
use std::collections::HashMap;
@@ -557,6 +559,149 @@ fn route_probe_matches_every_entry() {
);
}
+// ---------------------------------------------------------------------- route_batch_probe.json
+
+fn batch_request(request: &Value) -> BatchRequest {
+ let questions = golden_data::build_questions(&request["questions"]).unwrap();
+ let mut built = BatchRequest::new(request["state"].clone(), questions);
+ built.model = request
+ .get("model")
+ .and_then(Value::as_str)
+ .map(String::from);
+ built.task = request
+ .get("task")
+ .and_then(Value::as_str)
+ .map(String::from);
+ built.lang = request
+ .get("lang")
+ .and_then(Value::as_str)
+ .map(String::from);
+ built.lang_guess = request
+ .get("lang_guess")
+ .and_then(Value::as_str)
+ .map(LangGuess::from);
+ built
+}
+
+fn batch_requests(case: &Value) -> Vec<BatchRequest> {
+ case["requests"]
+ .as_array()
+ .unwrap()
+ .iter()
+ .map(batch_request)
+ .collect()
+}
+
+#[test]
+fn route_batch_probe_matches_every_entry() {
+ let Some(probe) = routing_golden::load("route_batch_probe.json") else {
+ eprintln!("skip: routing golden not found (tests/golden/routing)");
+ return;
+ };
+
+ let cases = probe["cases"].as_array().unwrap();
+ for case in cases {
+ let label = case["label"].as_str().unwrap();
+ let router = router_from_kwargs(&case["router_kwargs"]);
+ let decisions = router
+ .route_batch(&batch_requests(case))
+ .unwrap_or_else(|e| panic!("{label}: route_batch failed: {e}"));
+ let expected = case["decisions"].as_array().unwrap();
+ assert_eq!(expected.len(), decisions.len(), "{label}: decision count");
+ for (index, (decision, want)) in decisions.iter().zip(expected).enumerate() {
+ assert_eq!(
+ checkpoint_name(decision.model),
+ want["model"].as_str().unwrap(),
+ "{label}[{index}]: model"
+ );
+ assert_eq!(
+ decision.reason,
+ want["reason"].as_str().unwrap(),
+ "{label}[{index}]: reason"
+ );
+ assert_eq!(
+ decision.workflow.as_deref(),
+ want["workflow"].as_str(),
+ "{label}[{index}]: workflow"
+ );
+ assert_eq!(
+ decision.detection.as_ref().map(|d| d.script.as_str()),
+ want["detection"]["script"].as_str(),
+ "{label}[{index}]: detection.script"
+ );
+ }
+ }
+ eprintln!(
+ "route_batch_probe_matches_every_entry: checked {} cases",
+ cases.len()
+ );
+}
+
+/// Python's request validation errors. A missing `state` or `questions`, a `questions` that is not
+/// a mapping and a request that is not a mapping cannot be written with `BatchRequest`, so those
+/// entries are unrepresentable in Rust and are checked to be exactly that set.
+#[test]
+fn route_batch_probe_errors_match_python_or_cannot_be_expressed() {
+ let Some(probe) = routing_golden::load("route_batch_probe.json") else {
+ eprintln!("skip: routing golden not found (tests/golden/routing)");
+ return;
+ };
+
+ let mut compared = Vec::new();
+ for case in probe["errors"].as_array().unwrap() {
+ let label = case["label"].as_str().unwrap();
+ let requests = case["requests"].as_array().unwrap();
+ let expressible = requests.iter().all(|request| {
+ request.is_object()
+ && request.get("state").is_some()
+ && request.get("questions").is_some_and(Value::is_object)
+ });
+ if !expressible {
+ continue;
+ }
+ let router = router_from_kwargs(&case["router_kwargs"]);
+ let error = router.route_batch(&batch_requests(case)).expect_err(label);
+ assert_eq!(
+ case["error_message"].as_str().unwrap(),
+ error.to_string(),
+ "{label}: error message"
+ );
+ compared.push(label.to_string());
+ }
+ assert_eq!(["state_none", "unknown_model"], compared.as_slice());
+}
+
+/// Python's `Router.predict` raises `TypeError` for a `None` state before it routes or loads, and
+/// `Router.predict_long` goes through `predict`, so neither may build a checkpoint for one.
+#[test]
+fn predict_and_predict_long_refuse_a_null_state_before_routing() {
+ let message = "state must not be None; pass a string, dict, or list";
+ let log = Arc::new(Mutex::new(Vec::new()));
+ let router = fake_router(2, log.clone());
+ let questions = noul_question_set(&["q"]);
+
+ let error = router
+ .predict(Value::Null, &questions, None, None, None)
+ .expect_err("predict refuses null");
+ assert_eq!(message, error.to_string());
+ let error = router
+ .predict(Value::Null, &questions, Some("bogus"), None, None)
+ .expect_err("the state is checked before the model name");
+ assert_eq!(message, error.to_string());
+ let error = router
+ .predict_long(
+ Value::Null,
+ &questions,
+ &laya::long_prediction::LongOptions::new(),
+ None,
+ None,
+ None,
+ )
+ .expect_err("predict_long refuses null");
+ assert_eq!(message, error.to_string());
+ assert!(log.lock().unwrap().is_empty(), "no checkpoint is loaded");
+}
+
// ---------------------------------------------------------------------- lang_guess (callbacks)
//
// The golden can only carry string hints; callbacks (a language-identification model) are pinned
diff --git a/laya-rust/tests/predict_batch_tests.rs b/laya-rust/tests/predict_batch_tests.rs
new file mode 100644
index 00000000..efc1adc1
--- /dev/null
+++ b/laya-rust/tests/predict_batch_tests.rs
@@ -0,0 +1,637 @@
+//! `predict_batch`: the model-free pieces (router grouping, order restoration, `route_batch`
+//! loading nothing, option forwarding) with fake predictors, and the model-backed parity of every
+//! recorded `"kind": "batch"` golden case.
+//!
+//! The goldens come from the genuine `ONNXAgent.predict_batch` over the same ONNX graph, so they pin
+//! the chunking, the `sort_by_length` windows and the order restoration along with the numbers.
+//!
+//! Run with `LAYA_ONNX_ROOT` (and `LAYA_ONNX_SPLIT_ROOT` for the split layout) set and
+//! `--test-threads=1`, as for the other model-backed tests.
+
+mod common;
+
+use common::engines::{self, CHECKPOINTS};
+use common::golden_data::{self, CheckpointGoldenData};
+use common::result_compare::{assert_result_matches, assert_state_usage_matches};
+use laya::answers::{LayaResult, Usage};
+use laya::batch_prediction::BatchOptions;
+use laya::error::{LayaError, Result};
+use laya::laya_engine::LayaEngine;
+use laya::laya_options::LayaCheckpoint;
+use laya::laya_router::{
+ BatchRequest, LangGuess, LayaBatchPredictor, LayaPredictor, LayaRouter, LayaRouterOptions,
+};
+use laya::questions::{Question, QuestionSet};
+use serde_json::{Value, json};
+use std::collections::HashMap;
+use std::sync::{Arc, Mutex};
+
+// ---------------------------------------------------------------------- model-free
+
+/// One call a fake predictor received.
+#[derive(Debug, Clone, PartialEq)]
+struct Call {
+ checkpoint: LayaCheckpoint,
+ states: Vec<Value>,
+ ids: Vec<String>,
+ options: BatchOptions,
+}
+
+/// Unmistakably English text, so detection alone routes it to the English checkpoint.
+const ENGLISH: &str = "Hello there, this is plain English text from a customer.";
+
+type CallLog = Arc<Mutex<Vec<Call>>>;
+
+/// A predictor with no ONNX weights that records each `predict_batch` call. A state's result
+/// carries the state's text length as `input_tokens`, so a test can tell results apart.
+struct FakeBatch {
+ checkpoint: LayaCheckpoint,
+ calls: CallLog,
+}
+
+impl LayaPredictor for FakeBatch {
+ fn predict(&self, _state: Value, questions: &QuestionSet) -> Result<LayaResult> {
+ Ok(fake_result(self.checkpoint, 0, questions))
+ }
+
+ fn as_batch_predictor(&self) -> Option<&dyn LayaBatchPredictor> {
+ Some(self)
+ }
+}
+
+impl LayaBatchPredictor for FakeBatch {
+ fn predict_batch(
+ &self,
+ states: &[Value],
+ questions: &QuestionSet,
+ options: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ self.calls.lock().unwrap().push(Call {
+ checkpoint: self.checkpoint,
+ states: states.to_vec(),
+ ids: questions.ids().map(String::from).collect(),
+ options: options.clone(),
+ });
+ Ok(states
+ .iter()
+ .map(|state| {
+ let size = state.as_str().map_or(0, str::len);
+ fake_result(self.checkpoint, size as u32, questions)
+ })
+ .collect())
+ }
+}
+
+fn fake_result(checkpoint: LayaCheckpoint, tag: u32, questions: &QuestionSet) -> LayaResult {
+ LayaResult::new(
+ checkpoint.subdir(),
+ questions.ids().map(String::from).collect(),
+ HashMap::new(),
+ Usage {
+ input_tokens: tag,
+ output_tokens: 0,
+ },
+ )
+}
+
+struct Fixture {
+ router: LayaRouter,
+ calls: CallLog,
+ builds: Arc<Mutex<Vec<LayaCheckpoint>>>,
+}
+
+fn fixture() -> Fixture {
+ let calls = CallLog::default();
+ let builds = Arc::new(Mutex::new(Vec::new()));
+ let (factory_calls, factory_builds) = (Arc::clone(&calls), Arc::clone(&builds));
+ let options = LayaRouterOptions {
+ max_loaded: 3,
+ ..LayaRouterOptions::default()
+ };
+ let router = LayaRouter::with_factory(options, move |checkpoint, _| {
+ factory_builds.lock().unwrap().push(checkpoint);
+ Ok(Arc::new(FakeBatch {
+ checkpoint,
+ calls: Arc::clone(&factory_calls),
+ }) as Arc<dyn LayaPredictor>)
+ })
+ .expect("a fake router never fails to construct");
+ Fixture {
+ router,
+ calls,
+ builds,
+ }
+}
+
+fn noul(id: &str) -> QuestionSet {
+ QuestionSet::new()
+ .with(id, Question::noul("Is it?", Value::Null, Value::Null))
+ .unwrap()
+}
+
+fn choice(options: &[&str]) -> QuestionSet {
+ QuestionSet::new()
+ .with(
+ "pick",
+ Question::choice_labels("Which?", options.iter().copied()).unwrap(),
+ )
+ .unwrap()
+}
+
+#[test]
+fn route_batch_loads_nothing_and_keeps_input_order() {
+ let f = fixture();
+ let questions = Arc::new(noul("flag"));
+ let requests = [
+ BatchRequest::new(ENGLISH, Arc::clone(&questions)),
+ BatchRequest::new("texte", Arc::clone(&questions)).model("multi"),
+ BatchRequest::new("anything", Arc::clone(&questions)).lang("en-US"),
+ BatchRequest::new("anything", Arc::clone(&questions)).lang("fr"),
+ BatchRequest::new("anything", Arc::clone(&questions)).lang_guess("de"),
+ BatchRequest::new("anything", Arc::clone(&questions)).task("typed_decisions"),
+ ];
+ let decisions = f.router.route_batch(&requests).unwrap();
+ let models: Vec<_> = decisions.iter().map(|d| d.model).collect();
+ assert_eq!(
+ models,
+ [
+ LayaCheckpoint::English,
+ LayaCheckpoint::Multilingual,
+ LayaCheckpoint::English,
+ LayaCheckpoint::Multilingual,
+ LayaCheckpoint::Multilingual,
+ LayaCheckpoint::TypedDecisions,
+ ]
+ );
+ assert!(
+ f.builds.lock().unwrap().is_empty(),
+ "route_batch must not load"
+ );
+ assert!(f.router.loaded().is_empty());
+}
+
+#[test]
+fn route_batch_honours_each_requests_lang_guess_and_the_installed_one() {
+ let questions = Arc::new(noul("flag"));
+ let f = fixture();
+ let requests = [
+ BatchRequest::new("anything", Arc::clone(&questions)).lang_guess("fr"),
+ BatchRequest::new("anything", Arc::clone(&questions))
+ .lang_guess(LangGuess::callback(|_| Some("en".to_string()))),
+ // A hint that names no language abstains; detection then decides.
+ BatchRequest::new(ENGLISH, Arc::clone(&questions)).lang_guess("und"),
+ ];
+ let models: Vec<_> = f
+ .router
+ .route_batch(&requests)
+ .unwrap()
+ .iter()
+ .map(|d| d.model)
+ .collect();
+ assert_eq!(
+ models,
+ [
+ LayaCheckpoint::Multilingual,
+ LayaCheckpoint::English,
+ LayaCheckpoint::English
+ ]
+ );
+}
+
+#[test]
+fn route_batch_refuses_a_null_state_by_index_like_python() {
+ let f = fixture();
+ let questions = Arc::new(noul("flag"));
+ let requests = [
+ BatchRequest::new("fine", Arc::clone(&questions)),
+ BatchRequest::new(Value::Null, Arc::clone(&questions)),
+ ];
+ let error = f.router.route_batch(&requests).unwrap_err();
+ assert!(matches!(error, LayaError::InvalidState(_)), "{error:?}");
+ assert_eq!(
+ error.to_string(),
+ "request 1 'state' must not be None; pass a string, dict, or list"
+ );
+ let error = f
+ .router
+ .predict_batch(&requests, &BatchOptions::new())
+ .unwrap_err();
+ assert!(matches!(error, LayaError::InvalidState(_)), "{error:?}");
+ assert!(
+ f.builds.lock().unwrap().is_empty(),
+ "nothing loads when routing fails"
+ );
+}
+
+#[test]
+fn predict_batch_without_requests_answers_nothing_and_loads_nothing() {
+ let f = fixture();
+ assert!(
+ f.router
+ .predict_batch(&[], &BatchOptions::new())
+ .unwrap()
+ .is_empty()
+ );
+ assert!(f.builds.lock().unwrap().is_empty());
+}
+
+#[test]
+fn predict_batch_groups_by_checkpoint_and_restores_input_order() {
+ let f = fixture();
+ let questions = Arc::new(noul("flag"));
+ // Interleaved checkpoints; each state's text length tags its result.
+ let requests = [
+ BatchRequest::new("a", Arc::clone(&questions)).model("en"),
+ BatchRequest::new("bb", Arc::clone(&questions)).model("multi"),
+ BatchRequest::new("ccc", Arc::clone(&questions)).model("en"),
+ BatchRequest::new("dddd", Arc::clone(&questions)).model("multi"),
+ BatchRequest::new("eeeee", Arc::clone(&questions)).model("en"),
+ ];
+ let results = f
+ .router
+ .predict_batch(&requests, &BatchOptions::new())
+ .unwrap();
+
+ let tags: Vec<u32> = results.iter().map(|r| r.usage().input_tokens).collect();
+ assert_eq!(tags, [1, 2, 3, 4, 5], "results follow the request order");
+ let models: Vec<&str> = results.iter().map(LayaResult::model).collect();
+ assert_eq!(
+ models,
+ [
+ "english",
+ "multilingual",
+ "english",
+ "multilingual",
+ "english"
+ ]
+ );
+ for (result, request) in results.iter().zip(&requests) {
+ let routing = result.routing().expect("every result carries its routing");
+ assert_eq!(
+ routing.reason,
+ format!(
+ "explicit model={}",
+ python_repr(request.model.as_deref().unwrap())
+ )
+ );
+ }
+
+ // One load and one forward-pass group per checkpoint, in order of first appearance.
+ assert_eq!(
+ *f.builds.lock().unwrap(),
+ [LayaCheckpoint::English, LayaCheckpoint::Multilingual]
+ );
+ let calls = f.calls.lock().unwrap();
+ assert_eq!(calls.len(), 2);
+ assert_eq!(calls[0].checkpoint, LayaCheckpoint::English);
+ assert_eq!(calls[0].states, [json!("a"), json!("ccc"), json!("eeeee")]);
+ assert_eq!(calls[1].checkpoint, LayaCheckpoint::Multilingual);
+ assert_eq!(calls[1].states, [json!("bb"), json!("dddd")]);
+}
+
+fn python_repr(text: &str) -> String {
+ format!("'{text}'")
+}
+
+#[test]
+fn predict_batch_splits_a_checkpoint_by_question_schema() {
+ let f = fixture();
+ let a = Arc::new(noul("flag"));
+ let b = Arc::new(noul("other"));
+ let ab = Arc::new(choice(&["x", "y"]));
+ let ba = Arc::new(choice(&["y", "x"]));
+ let requests = [
+ BatchRequest::new("1", Arc::clone(&a)).model("en"),
+ BatchRequest::new("22", Arc::clone(&b)).model("en"),
+ // Separately built but equal to `a`: shares a pass with request 0 and 4.
+ BatchRequest::new("333", noul("flag")).model("en"),
+ BatchRequest::new("4444", Arc::clone(&ab)).model("en"),
+ BatchRequest::new("55555", Arc::clone(&a)).model("en"),
+ // Same ids and labels as `ab` in the other order: options are positional.
+ BatchRequest::new("666666", Arc::clone(&ba)).model("en"),
+ ];
+ let results = f
+ .router
+ .predict_batch(&requests, &BatchOptions::new())
+ .unwrap();
+
+ let tags: Vec<u32> = results.iter().map(|r| r.usage().input_tokens).collect();
+ assert_eq!(tags, [1, 2, 3, 4, 5, 6]);
+ let calls = f.calls.lock().unwrap();
+ let shapes: Vec<(Vec<Value>, Vec<String>)> = calls
+ .iter()
+ .map(|c| (c.states.clone(), c.ids.clone()))
+ .collect();
+ assert_eq!(
+ shapes,
+ [
+ (
+ vec![json!("1"), json!("333"), json!("55555")],
+ vec!["flag".to_string()]
+ ),
+ (vec![json!("22")], vec!["other".to_string()]),
+ (vec![json!("4444")], vec!["pick".to_string()]),
+ (vec![json!("666666")], vec!["pick".to_string()]),
+ ]
+ );
+ assert_eq!(
+ f.builds.lock().unwrap().len(),
+ 1,
+ "one load for one checkpoint"
+ );
+}
+
+#[test]
+fn predict_batch_forwards_the_options_to_every_group() {
+ let f = fixture();
+ let questions = Arc::new(noul("flag"));
+ let requests = [
+ BatchRequest::new("a", Arc::clone(&questions)).model("en"),
+ BatchRequest::new("b", Arc::clone(&questions)).model("multi"),
+ ];
+ let options = BatchOptions::new().batch_size(2).sort_by_length(true);
+ f.router.predict_batch(&requests, &options).unwrap();
+ let calls = f.calls.lock().unwrap();
+ assert_eq!(calls.len(), 2);
+ assert!(calls.iter().all(|call| call.options == options));
+}
+
+#[test]
+fn predict_batch_through_the_predictor_trait_routes_each_state() {
+ let f = fixture();
+ let questions = noul("flag");
+ let batcher = LayaPredictor::as_batch_predictor(&f.router).expect("a router can batch");
+ let results = batcher
+ .predict_batch(
+ &[json!(ENGLISH), json!("Привет, как дела у тебя сегодня?")],
+ &questions,
+ &BatchOptions::new(),
+ )
+ .unwrap();
+ let models: Vec<&str> = results.iter().map(LayaResult::model).collect();
+ assert_eq!(models, ["english", "multilingual"]);
+}
+
+struct SingleOnly;
+
+impl LayaPredictor for SingleOnly {
+ fn predict(&self, _state: Value, questions: &QuestionSet) -> Result<LayaResult> {
+ Ok(fake_result(LayaCheckpoint::English, 0, questions))
+ }
+}
+
+#[test]
+fn a_predictor_without_predict_batch_is_reported_not_emulated() {
+ let router = LayaRouter::with_factory(LayaRouterOptions::default(), |_, _| {
+ Ok(Arc::new(SingleOnly) as Arc<dyn LayaPredictor>)
+ })
+ .unwrap();
+ let requests = [BatchRequest::new("state", noul("flag")).model("en")];
+ let error = router
+ .predict_batch(&requests, &BatchOptions::new())
+ .unwrap_err();
+ assert!(matches!(error, LayaError::NotSupported(_)), "{error:?}");
+ assert!(error.to_string().contains("no predict_batch"), "{error}");
+}
+
+#[test]
+fn a_failing_group_fails_the_call() {
+ struct Failing;
+ impl LayaPredictor for Failing {
+ fn predict(&self, _: Value, _: &QuestionSet) -> Result<LayaResult> {
+ Err(LayaError::InvalidConfig("nope".into()))
+ }
+ fn as_batch_predictor(&self) -> Option<&dyn LayaBatchPredictor> {
+ Some(self)
+ }
+ }
+ impl LayaBatchPredictor for Failing {
+ fn predict_batch(
+ &self,
+ _: &[Value],
+ _: &QuestionSet,
+ _: &BatchOptions,
+ ) -> Result<Vec<LayaResult>> {
+ Err(LayaError::InvalidConfig("nope".into()))
+ }
+ }
+ let router = LayaRouter::with_factory(LayaRouterOptions::default(), |_, _| {
+ Ok(Arc::new(Failing) as Arc<dyn LayaPredictor>)
+ })
+ .unwrap();
+ let requests = [BatchRequest::new("state", noul("flag")).model("en")];
+ let error = router
+ .predict_batch(&requests, &BatchOptions::new())
+ .unwrap_err();
+ assert_eq!(error.to_string(), "nope");
+}
+
+// ---------------------------------------------------------------------- model-backed
+
+fn batch_options(options: &Value) -> BatchOptions {
+ let mut built = BatchOptions::new();
+ // Python treats a batch_size that is zero or negative as unset.
+ built.batch_size = options
+ .get("batch_size")
+ .and_then(Value::as_i64)
+ .and_then(|size| usize::try_from(size).ok())
+ .filter(|&size| size > 0);
+ built.sort_by_length = options
+ .get("sort_by_length")
+ .and_then(Value::as_bool)
+ .unwrap_or(false);
+ built
+}
+
+fn check_batch_case(engine: &LayaEngine, where_case: &str, root: &Value) {
+ let states = root["states"].as_array().expect("states");
+ let questions = golden_data::build_questions(&root["questions"])
+ .unwrap_or_else(|e| panic!("{where_case}: {e}"));
+ let options = batch_options(&root["options"]);
+ let got = engine
+ .predict_batch(states, &questions, &options)
+ .unwrap_or_else(|e| panic!("{where_case}: predict_batch failed: {e}"));
+ let expected = root["results"].as_array().expect("results");
+ assert_eq!(expected.len(), got.len(), "{where_case}: result count");
+
+ for (index, (want, result)) in expected.iter().zip(&got).enumerate() {
+ let where_state = format!("{where_case}[{index}]");
+ // The SDK keeps its own model identifier instead of the ONNX agent's suffix.
+ assert_eq!("laya-rl-agent-onnx", want["model"].as_str().unwrap());
+ if questions.is_empty() {
+ assert_eq!("laya-rl-agent", result.model(), "{where_state}: model");
+ assert!(result.ids().is_empty(), "{where_state}: no answers");
+ assert!(
+ result.state_usage().is_none(),
+ "{where_state}: no state usage"
+ );
+ assert_eq!(
+ 0,
+ result.usage().input_tokens,
+ "{where_state}: input_tokens"
+ );
+ assert_eq!(
+ 0,
+ result.usage().output_tokens,
+ "{where_state}: output_tokens"
+ );
+ assert_eq!(
+ want["usage"].as_object().unwrap().len(),
+ 2,
+ "{where_state}: usage keys"
+ );
+ } else {
+ // The batch goldens come from Python's ONNXAgent, whose family tag is
+ // "laya-rl-agent-onnx"; this crate keeps the tag of Python's Agent (and of the
+ // plain goldens), "laya-rl-agent", for predict and predict_batch alike.
+ assert_eq!(
+ "laya-rl-agent-onnx", want["model"],
+ "{where_state}: golden model"
+ );
+ assert_eq!("laya-rl-agent", result.model(), "{where_state}: model");
+ let mut want = want.clone();
+ want["model"] = json!("laya-rl-agent");
+ assert_result_matches(&where_state, &want, result);
+ assert_state_usage_matches(&where_state, &want, result);
+ }
+ }
+}
+
+fn run_success_cases(
+ load: fn(LayaCheckpoint) -> Option<LayaEngine>,
+ label: &str,
+ unavailable: &str,
+) {
+ let mut checked_any = false;
+ for checkpoint in CHECKPOINTS {
+ let Some(engine) = load(checkpoint) else {
+ continue;
+ };
+ let golden = CheckpointGoldenData::for_checkpoint(checkpoint.subdir());
+ if !golden.available() {
+ eprintln!("skip: no golden data for '{}'", checkpoint.subdir());
+ continue;
+ }
+ for case in golden.batch_cases() {
+ checked_any = true;
+ let root = golden.load_case(&case);
+ check_batch_case(
+ &engine,
+ &format!("{}/{case}{label}", checkpoint.subdir()),
+ &root,
+ );
+ }
+ }
+ if !checked_any {
+ eprintln!("skip: {unavailable}");
+ }
+}
+
+#[test]
+fn reproduces_the_recorded_batch_predictions() {
+ run_success_cases(
+ engines::load,
+ "",
+ "no batch golden data + artifacts found for any checkpoint",
+ );
+}
+
+#[test]
+fn reproduces_the_recorded_batch_predictions_for_the_split_layout() {
+ run_success_cases(
+ engines::load_split,
+ " (split layout)",
+ "no split-layout ONNX artifacts + batch golden data found for any checkpoint",
+ );
+}
+
+#[test]
+fn rejects_the_same_batch_requests_python_rejects() {
+ let mut checked_any = false;
+ for checkpoint in CHECKPOINTS {
+ let Some(engine) = engines::load(checkpoint) else {
+ continue;
+ };
+ let golden = CheckpointGoldenData::for_checkpoint(checkpoint.subdir());
+ if !golden.available() {
+ continue;
+ }
+ for case in golden.batch_error_cases() {
+ checked_any = true;
+ let root = golden.load_case(&case);
+ let questions = golden_data::build_questions(&root["questions"])
+ .unwrap_or_else(|e| panic!("{}/{case}: {e}", checkpoint.subdir()));
+ let states = root["states"].as_array().expect("states");
+ let error = engine
+ .predict_batch(states, &questions, &batch_options(&root["options"]))
+ .expect_err(&format!(
+ "{}/{case}: expected predict_batch to fail",
+ checkpoint.subdir()
+ ));
+ // Python's TypeError is the refused null state; its ValueError is a question whose
+ // options do not fit, which Rust reports as NoMarkers with the same leading text.
+ let message = root["error_message"].as_str().unwrap();
+ match root["error_type"].as_str().unwrap() {
+ "TypeError" => {
+ assert!(
+ matches!(error, LayaError::InvalidState(_)),
+ "{}/{case}: wrong error variant: {error:?}",
+ checkpoint.subdir()
+ );
+ assert_eq!(message, error.to_string(), "{}/{case}", checkpoint.subdir());
+ }
+ "ValueError" => {
+ assert!(
+ matches!(error, LayaError::NoMarkers { .. }),
+ "{}/{case}: wrong error variant: {error:?}",
+ checkpoint.subdir()
+ );
+ assert!(
+ error.to_string().starts_with(message),
+ "{}/{case}: python: {message}
+ rust: {error}",
+ checkpoint.subdir()
+ );
+ }
+ other => panic!(
+ "{}/{case}: unexpected error type {other}",
+ checkpoint.subdir()
+ ),
+ }
+ }
+ }
+ if !checked_any {
+ eprintln!("skip: no batch error-case golden data + artifacts found for any checkpoint");
+ }
+}
+
+/// `predict` is a one-state batch in Python (`system_one` is `predict_batch([state], ...)`).
+#[test]
+fn predict_is_a_one_state_batch_for_null_states_and_empty_questions() {
+ let mut checked_any = false;
+ for checkpoint in CHECKPOINTS {
+ let Some(engine) = engines::load(checkpoint) else {
+ continue;
+ };
+ checked_any = true;
+ let questions = golden_data::build_questions(&json!({
+ "flag": {"type": "noul", "instructions": "Is it?"}
+ }))
+ .unwrap();
+ let error = engine.predict(Value::Null, &questions).unwrap_err();
+ assert!(matches!(error, LayaError::InvalidState(_)), "{error:?}");
+ assert_eq!(
+ error.to_string(),
+ "state must not be None; pass a string, dict, or list"
+ );
+
+ let empty = engine.predict("some state", &QuestionSet::new()).unwrap();
+ assert!(empty.ids().is_empty());
+ assert!(empty.state_usage().is_none());
+ assert_eq!(0, empty.usage().input_tokens);
+ assert_eq!(0, empty.usage().output_tokens);
+ }
+ if !checked_any {
+ eprintln!("skip: no ONNX artifacts found for any checkpoint");
+ }
+}
diff --git a/laya-rust/tools/dump_golden.py b/laya-rust/tools/dump_golden.py
index 21cebb49..b4625458 100644
--- a/laya-rust/tools/dump_golden.py
+++ b/laya-rust/tools/dump_golden.py
@@ -8,7 +8,8 @@ unmodified, so every number written here is what Python actually answers -- not
paraphrase of it. It also means no 1.3 GB safetensors checkpoint is needed: the exported
ONNX graph supplies the forward pass.
-Long cases run the genuine `ONNXAgent.predict_long`, sharing that same verified session.
+Long cases run the genuine `ONNXAgent.predict_long`, sharing that same verified session. Batch cases
+run the genuine `ONNXAgent.predict_batch` the same way.
Before writing anything the script checks option logits and unrounded action probabilities
against the PyTorch reference vectors bundled in `fixtures.npz`. Raw action logits are
@@ -39,7 +40,15 @@ REPO = os.path.dirname(os.path.dirname(TOOLS))
sys.path.insert(0, REPO)
sys.path.insert(0, TOOLS)
-from golden_cases import CASES, ERROR_CASES, LONG_CASES, LONG_ERROR_CASES, SHORTLIST_E2E_CASES # noqa: E402
+from golden_cases import ( # noqa: E402
+ BATCH_CASES,
+ BATCH_ERROR_CASES,
+ CASES,
+ ERROR_CASES,
+ LONG_CASES,
+ LONG_ERROR_CASES,
+ SHORTLIST_E2E_CASES,
+)
from repr_cases import build_repr_probe # noqa: E402
import hashing_embedder # noqa: E402
@@ -456,6 +465,60 @@ def record_long_case(agent: Agent, case: dict, expects_error: bool = False) -> d
return rec
+class RecordingSession:
+ """Forwards to the verified ONNX session and notes each run's input shape, so a batch case
+ can record how `predict_batch` actually chunked and ordered its states."""
+
+ def __init__(self, session: ort.InferenceSession):
+ self.session = session
+ self.runs = []
+
+ def run(self, names, feeds):
+ self.runs.append([int(d) for d in feeds["input_ids"].shape])
+ return self.session.run(names, feeds)
+
+
+def record_batch_case(agent: Agent, case: dict, expects_error: bool = False) -> dict:
+ """Run the genuine ONNXAgent.predict_batch, sharing the already verified session/tokenizer."""
+ from laya.onnx_agent import ONNXAgent
+
+ session = RecordingSession(agent.model.sess)
+ onnx = ONNXAgent.__new__(ONNXAgent)
+ onnx.cfg, onnx.tok, onnx.session = agent.cfg, agent.tok, session
+ onnx.temperature = agent.temperature
+ onnx.temperature_by_options = agent.temperature_by_options
+ onnx.lang_temperatures = {}
+ onnx.hooks = []
+ onnx.model_id = "golden"
+ states, questions = case["states"], case["questions"]
+ options = case.get("options", {})
+ rec = {"name": case["name"], "states": states, "questions": questions, "options": options}
+ if expects_error:
+ expected = {"TypeError": TypeError, "ValueError": ValueError}[case["error_type"]]
+ try:
+ onnx.predict_batch(states, questions, **options)
+ except expected as e:
+ rec.update(error_type=type(e).__name__, error_message=str(e))
+ return rec
+ raise SystemExit("batch case %r was expected to raise %s but did not" % (case["name"], case["error_type"]))
+
+ # Each state's widest question row: the key sort_by_length orders by. Recorded separately from
+ # the results so the C# chunk planning can be checked against Python's own token counts.
+ widths = []
+ if questions and states:
+ ids = list(questions.keys())
+ internal = {qid: onnx._to_internal(questions[qid]) for qid in ids}
+ for state in states:
+ items = onnx._encode_state(state, ids, internal, onnx.cfg.get("max_len", 512),
+ onnx.cfg.get("head_max_len", 192))
+ widths.append(max(len(item["ids"]) for item in items))
+ rec["state_widths"] = widths
+ session.runs.clear()
+ rec["results"] = onnx.predict_batch(states, questions, **options)
+ rec["runs"] = list(session.runs)
+ return rec
+
+
def dump(obj, path: str):
with open(path, "w", encoding="utf-8", newline="\n") as f:
json.dump(obj, f, ensure_ascii=False, indent=2, sort_keys=False)
@@ -558,6 +621,15 @@ def run_checkpoint(onnx_dir: str, out_dir: str) -> int:
"expects_error": expects_error, "kind": "long",
})
+ for cases, expects_error in ((BATCH_CASES, False), (BATCH_ERROR_CASES, True)):
+ for case in cases:
+ rec = record_batch_case(agent, case, expects_error)
+ dump(rec, os.path.join(out_dir, "case_%s.json" % case["name"]))
+ index.append({
+ "name": case["name"], "file": "case_%s.json" % case["name"],
+ "expects_error": expects_error, "kind": "batch",
+ })
+
dump(index, os.path.join(out_dir, "index.json"))
dump(build_repr_probe(), os.path.join(out_dir, "repr_probe.json"))
print("done: %d cases -> %s" % (len(index), out_dir))
diff --git a/laya-rust/tools/dump_routing_golden.py b/laya-rust/tools/dump_routing_golden.py
index b5fb126d..5e3f31fd 100644
--- a/laya-rust/tools/dump_routing_golden.py
+++ b/laya-rust/tools/dump_routing_golden.py
@@ -8,6 +8,8 @@ corpus from `laya-rust/tools/routing_cases.py` and writes one JSON file per prob
lang_probe.json lang.analyse / detect_script / script_profile /
latin_profile / state_text, for every LANG_STATES entry.
+ route_batch_probe.json Router.route_batch() decisions for every ROUTE_BATCH_CASES entry,
+ plus the errors it raises (ROUTE_BATCH_ERROR_CASES).
route_probe.json Router.route() decisions (model, reason, detection,
workflow -- `repo` is intentionally omitted: the ported
RouteDecision has no Repo field) for every ROUTE_CASES entry.
@@ -52,6 +54,8 @@ from routing_cases import ( # noqa: E402
EMAIL_CLEAN_CASES,
EMAIL_STATE_CASES,
LANG_STATES,
+ ROUTE_BATCH_CASES,
+ ROUTE_BATCH_ERROR_CASES,
ROUTE_CASES,
SAMPLE_BANKING_CRITERIA,
SAMPLE_BANKING_INSTRUCTIONS,
@@ -187,6 +191,33 @@ def build_route_probe():
return out
+def build_route_batch_probe():
+ cases = []
+ for label, router_kwargs, requests in ROUTE_BATCH_CASES:
+ decisions = Router(**router_kwargs).route_batch(requests)
+ cases.append({
+ "label": label,
+ "router_kwargs": router_kwargs,
+ "requests": requests,
+ "decisions": [_decision_payload(d) for d in decisions],
+ })
+ errors = []
+ for label, router_kwargs, requests in ROUTE_BATCH_ERROR_CASES:
+ try:
+ Router(**router_kwargs).route_batch(requests)
+ except (TypeError, ValueError) as e:
+ errors.append({
+ "label": label,
+ "router_kwargs": router_kwargs,
+ "requests": requests,
+ "error_type": type(e).__name__,
+ "error_message": str(e),
+ })
+ else:
+ raise SystemExit("route_batch case %r was expected to raise but did not" % label)
+ return {"cases": cases, "errors": errors}
+
+
# ============================================================================ email_probe
def build_email_probe():
clean = []
@@ -356,6 +387,7 @@ def main() -> int:
lang_probe = build_lang_probe()
route_probe = build_route_probe()
+ route_batch_probe = build_route_batch_probe()
email_probe = build_email_probe()
shortlist_probe = build_shortlist_probe()
hashing_probe = build_hashing_embedder_probe()
@@ -376,6 +408,8 @@ def main() -> int:
"counts": {
"lang_probe": len(lang_probe),
"route_probe": len(route_probe),
+ "route_batch_probe": len(route_batch_probe["cases"]),
+ "route_batch_probe_errors": len(route_batch_probe["errors"]),
"email_probe_clean": len(email_probe["clean"]),
"email_probe_state": len(email_probe["state"]),
"shortlist_probe": len(shortlist_probe),
@@ -386,6 +420,9 @@ def main() -> int:
results = {}
results["lang_probe.json"] = dump(lang_probe, os.path.join(out_dir, "lang_probe.json"), args.force)
results["route_probe.json"] = dump(route_probe, os.path.join(out_dir, "route_probe.json"), args.force)
+ results["route_batch_probe.json"] = dump(
+ route_batch_probe, os.path.join(out_dir, "route_batch_probe.json"), args.force
+ )
results["email_probe.json"] = dump(email_probe, os.path.join(out_dir, "email_probe.json"), args.force)
results["shortlist_probe.json"] = dump(shortlist_probe, os.path.join(out_dir, "shortlist_probe.json"), args.force)
results["hashing_embedder_probe.json"] = dump(
diff --git a/laya-rust/tools/golden_cases.py b/laya-rust/tools/golden_cases.py
index 903b4557..df3db938 100644
--- a/laya-rust/tools/golden_cases.py
+++ b/laya-rust/tools/golden_cases.py
@@ -387,3 +387,127 @@ LONG_ERROR_CASES = [
"options": {"aggregate": "mean"}},
{"name": "long_no_state_room", "state": "short", "questions": ERROR_CASES[0]["questions"]},
]
+
+# ONNXAgent.predict_batch: the same questions over many states, with the batching knobs.
+# States are the order-sensitive part: widths differ so `sort_by_length` has something to
+# reorder, and list states are long enough to be left-truncated. `options` go to predict_batch
+# verbatim (`batch_size`, `sort_by_length`).
+BATCH_QUESTIONS = LONG_QUESTIONS
+
+_BATCH_FILLER = "Routine account check completed. "
+_BATCH_TURNS = " ".join(
+ "Turn %d: the agent restated the refund policy and the customer repeated the complaint." % i
+ for i in range(140)
+)
+
+# Widths in the order given are deliberately shuffled, so a stable sort has real work to do and
+# repeated widths (k % 5) exercise its tie-breaking.
+BATCH_SORT_STATES = [_BATCH_FILLER * k + QUICKSTART_STATE for k in
+ (18, 2, 9, 0, 14, 5, 11, 1, 16, 7, 3, 12, 19, 4, 8, 13, 6, 17, 10, 15)]
+
+BATCH_MIXED_STATES = [
+ QUICKSTART_STATE,
+ {"subject": "Double charged on invoice 88213", "body": QUICKSTART_STATE},
+ [{"role": "user", "content": "Hi, I need help with my invoice."},
+ {"role": "assistant", "content": "Of course, what is wrong?"},
+ {"role": "user", "content": QUICKSTART_STATE}],
+ "",
+ _BATCH_FILLER * 12 + QUICKSTART_STATE,
+ "Thanks, that is all sorted now.",
+ {"notes": _BATCH_FILLER * 8, "message": "Please cancel my account today."},
+ "Mein Konto wurde zweimal belastet. Bitte erstatten Sie den Betrag.",
+]
+
+# Long enough to overflow every checkpoint's window: a list state keeps its newest turns
+# (left truncation), a string or dict keeps its start.
+BATCH_TRUNCATED_STATES = [
+ "Customer support transcript. " + _BATCH_TURNS,
+ [{"role": "user" if i % 2 == 0 else "assistant",
+ "content": "Turn %d: the agent restated the refund policy and the customer repeated the complaint." % i}
+ for i in range(140)] + [{"role": "user", "content": "I want a refund or I will cancel today."}],
+ {"notes": _BATCH_TURNS, "message": QUICKSTART_STATE},
+ QUICKSTART_STATE,
+]
+
+# Shared with the Rust SDK's corpus. A mix of kinds and widths, one of them long enough to truncate.
+BATCH_STATES = [
+ QUICKSTART_STATE,
+ "Thanks, the refund arrived and everything works now.",
+ {"ticket": 4821, "message": QUICKSTART_STATE, "notes": "Routine checks. " * 12},
+ [{"role": "assistant", "content": "How can I help?"},
+ {"role": "user", "content": "I was billed twice this month and nobody has answered."}],
+ LONG_STATE,
+ "The app crashes on login since yesterday's update.",
+ "ok",
+]
+
+# Eighteen states of varied length, so a batch size of two crosses the eight-batch sort window.
+BATCH_WINDOW_STATES = [
+ ("Routine status note %d. " % i) * (i % 7 + 1) + (QUICKSTART_STATE if i % 5 == 0 else "")
+ for i in range(18)
+]
+
+BATCH_CASES = [
+ {"name": "batch_one_state", "states": [QUICKSTART_STATE], "questions": BATCH_QUESTIONS},
+ {"name": "batch_mixed", "states": BATCH_MIXED_STATES, "questions": BATCH_QUESTIONS},
+ {"name": "batch_size_1", "states": BATCH_MIXED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 1}},
+ {"name": "batch_size_3", "states": BATCH_MIXED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 3}},
+ {"name": "batch_size_over_len", "states": BATCH_MIXED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 100}},
+ # `batch_size and batch_size > 0` is falsy for zero and negatives: one run, no error.
+ {"name": "batch_size_zero", "states": BATCH_MIXED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 0}},
+ {"name": "batch_size_negative", "states": BATCH_MIXED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": -2}},
+ # sort_by_length takes effect only for 1 < batch_size < len(states); the sort window is 8 batches.
+ {"name": "batch_sort_on", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2, "sort_by_length": True}},
+ {"name": "batch_sort_off", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2}},
+ {"name": "batch_sort_size_3", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 3, "sort_by_length": True}},
+ {"name": "batch_sort_partial_window", "states": BATCH_SORT_STATES[:3], "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2, "sort_by_length": True}},
+ {"name": "batch_sort_no_batch_size", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"sort_by_length": True}},
+ {"name": "batch_sort_size_1", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 1, "sort_by_length": True}},
+ {"name": "batch_sort_size_eq_len", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 20, "sort_by_length": True}},
+ {"name": "batch_sort_size_over_len", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 25, "sort_by_length": True}},
+ {"name": "batch_sort_size_zero", "states": BATCH_SORT_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 0, "sort_by_length": True}},
+ {"name": "batch_truncated", "states": BATCH_TRUNCATED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2}},
+ {"name": "batch_truncated_sorted", "states": BATCH_TRUNCATED_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 3, "sort_by_length": True}},
+ {"name": "batch_duplicate_states", "states": [QUICKSTART_STATE] * 3, "questions": BATCH_QUESTIONS},
+ # An empty list is answered without looking at the questions, even ones that cannot be asked.
+ {"name": "batch_empty_states", "states": [], "questions": BATCH_QUESTIONS},
+ {"name": "batch_empty_states_invalid_questions", "states": [], "questions": ERROR_CASES[0]["questions"]},
+ {"name": "batch_empty_questions", "states": [QUICKSTART_STATE, ""], "questions": {}},
+ {"name": "batch_empty_both", "states": [], "questions": {}},
+ {"name": "batch_sorted", "states": BATCH_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2, "sort_by_length": True}},
+ {"name": "batch_sorted_windows", "states": BATCH_WINDOW_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2, "sort_by_length": True}},
+ {"name": "batch_unsorted_windows", "states": BATCH_WINDOW_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 2}},
+ {"name": "batch_chunked", "states": BATCH_STATES, "questions": BATCH_QUESTIONS,
+ "options": {"batch_size": 3}},
+ # A single question per state.
+ {"name": "batch_single_question", "states": BATCH_STATES[:4],
+ "questions": {"cancel": BATCH_QUESTIONS["cancel"]}, "options": {"batch_size": 2}},
+ {"name": "batch_empty_questions_with_options", "states": BATCH_STATES[:3], "questions": {},
+ "options": {"batch_size": 2, "sort_by_length": True}},
+]
+
+BATCH_ERROR_CASES = [
+ {"name": "batch_none_state", "states": [QUICKSTART_STATE, None], "questions": BATCH_QUESTIONS,
+ "error_type": "TypeError"},
+ {"name": "batch_options_exceed_head", "states": ["short", QUICKSTART_STATE],
+ "questions": ERROR_CASES[0]["questions"], "error_type": "ValueError"},
+]
diff --git a/laya-rust/tools/routing_cases.py b/laya-rust/tools/routing_cases.py
index 78e93c64..862f6656 100644
--- a/laya-rust/tools/routing_cases.py
+++ b/laya-rust/tools/routing_cases.py
@@ -608,6 +608,77 @@ ROUTE_CASES = [
]
+# ============================================================================ route_batch_probe
+# Router.route_batch(requests): (label, Router kwargs, request dicts). Requests are JSON-able
+# (lang_guess as a language code) because the corpus is replayed from JSON by the .NET suite.
+# route_batch loads nothing, so these need no model.
+
+ROUTE_BATCH_CASES = [
+ ("empty", {}, []),
+ ("single_english", {}, [{"state": "I was charged twice, please refund.", "questions": Q_GENERIC}]),
+ ("mixed_order", {}, [
+ {"state": "I was charged twice, please refund.", "questions": Q_GENERIC},
+ {"state": "Der Kunde wurde zweimal belastet und moechte eine Rueckerstattung fuer die Rechnung "
+ "die nicht korrekt ist", "questions": Q_GENERIC},
+ {"state": {"body": "मुझसे दो बार शुल्क लिया गया"}, "questions": Q_GENERIC},
+ {"state": "I was charged twice, please refund.", "questions": Q_GENERIC},
+ ]),
+ ("per_request_overrides", {}, [
+ {"state": "hello", "questions": Q_GENERIC, "model": "ml"},
+ {"state": "hello", "questions": Q_GENERIC, "task": "typed_decisions"},
+ {"state": "मुझसे दो बार", "questions": Q_GENERIC, "lang": "en"},
+ {"state": "hello", "questions": Q_GENERIC, "lang": "de"},
+ {"state": "hello", "questions": Q_GENERIC, "lang": "C", "lang_guess": "ro"},
+ {"state": "Care este ora in Tokyo?", "questions": Q_GENERIC, "lang_guess": "ro"},
+ {"state": "Care este ora in Tokyo?", "questions": Q_GENERIC, "lang_guess": "en"},
+ {"state": "Care este ora in Tokyo?", "questions": Q_GENERIC, "lang_guess": "und"},
+ {"state": "x", "questions": Q_GENERIC, "model": "english", "lang_guess": "ro"},
+ ]),
+ ("installed_lang_guess", {"lang_guess": "ro"}, [
+ {"state": "Care este ora in Tokyo?", "questions": Q_GENERIC},
+ {"state": "Care este ora in Tokyo?", "questions": Q_GENERIC, "lang_guess": "en"},
+ {"state": "Care este ora in Tokyo?", "questions": Q_GENERIC, "lang_guess": "C"},
+ ]),
+ ("auto_task_workflow", {"auto_task_detection": True}, [
+ {"state": {"body": "I was charged twice"}, "questions": _td_questions(_TD_IDS["customer_service"])},
+ {"state": {"body": "I was charged twice"}, "questions": Q_GENERIC},
+ {"state": "x", "questions": _td_questions(_TD_IDS["customer_service"]), "lang_guess": "ro"},
+ ]),
+ ("custom_default", {"default": "english"}, [
+ {"state": "12345", "questions": Q_GENERIC},
+ {"state": "Hello", "questions": Q_GENERIC},
+ ]),
+ ("falsy_states", {}, [
+ {"state": "", "questions": Q_GENERIC},
+ {"state": {}, "questions": Q_GENERIC},
+ {"state": [], "questions": Q_GENERIC},
+ {"state": 0, "questions": Q_GENERIC},
+ ]),
+ ("lang_variants", {}, [
+ {"state": {"body": "I was charged twice, please refund this order."}, "questions": Q_GENERIC, "lang": "fr"},
+ {"state": {"body": "मुझसे दो बार शुल्क लिया गया"}, "questions": Q_GENERIC, "lang": "en-US"},
+ {"state": {"body": "I was charged twice, please refund this order."}, "questions": Q_GENERIC, "lang": "C"},
+ {"state": {"body": "मुझसे दो बार शुल्क लिया गया"}, "questions": Q_GENERIC, "model": "english"},
+ ]),
+ ("installed_lang_guess_lang_wins", {"lang_guess": "de"}, [
+ {"state": {"body": "I was charged twice, please refund this order."}, "questions": Q_GENERIC},
+ {"state": {"body": "I was charged twice, please refund this order."}, "questions": Q_GENERIC,
+ "lang_guess": "en"},
+ {"state": {"body": "I was charged twice, please refund this order."}, "questions": Q_GENERIC, "lang": "en"},
+ ]),
+]
+
+# Requests Python refuses before routing anything: (label, Router kwargs, request dicts).
+ROUTE_BATCH_ERROR_CASES = [
+ ("state_none", {}, [{"state": "ok", "questions": Q_GENERIC}, {"state": None, "questions": Q_GENERIC}]),
+ ("state_missing", {}, [{"questions": Q_GENERIC}]),
+ ("questions_missing", {}, [{"state": "ok"}]),
+ ("questions_none", {}, [{"state": "ok", "questions": None}]),
+ ("unknown_model", {}, [{"state": "ok", "questions": Q_GENERIC}, {"state": "ok", "questions": Q_GENERIC, "model": "gpt"}]),
+ ("request_not_dict", {}, [{"state": "ok", "questions": Q_GENERIC}, "ok"]),
+]
+
+
# ============================================================================ email_probe
_DISCLAIMER_TEXT = "This email is confidential and intended solely for the named addressee."