Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(rust): add predict_long, ported from ONNXAgent.predict_long
kevin-gatimu63b94cff147e
diff --git a/laya-rust/CHANGELOG.md b/laya-rust/CHANGELOG.mdindex 4ec6a227..b7e97ec9 100644--- a/laya-rust/CHANGELOG.md+++ b/laya-rust/CHANGELOG.md@@ -11,6 +11,16 @@ goldens for the supported paths (see the known gaps below).### Added+- `LayaEngine::predict_long` and `LayaRouter::predict_long` (`aggregate="auto"`): scan a state+ longer than one sequence in overlapping token windows and keep the most confident window per+ question, with `Answer::window()` attribution, `LayaResult::scan_usage()` and summed+ `state_usage()`. Matches Python's `ONNXAgent.predict_long` and the .NET SDK; hooks, per-language+ temperatures and the confidence gate are not part of it. New `LongOptions`, `LayaLongPredictor`,+ `LayaError::{UnsupportedAggregate, InvalidWindow, NotSupported}` and a default+ `LayaTokenizer::decode`. `LayaError` is not `#[non_exhaustive]`, so an exhaustive `match` on it+ outside the crate needs arms for the three new variants.+- `predict` now runs through a shared multi-state core that scores several states in one forward+ pass; its results are unchanged.- Advisory parity CI on every push and PR: regenerate fixtures from the current Python checkout,test all checkpoints, and reject skipped tests. No golden cache or crate publication.- Python float-repr parity probes, including exact half-even decimal ties and random bit patterns.@@ -35,4 +45,4 @@ goldens for the supported paths (see the known gaps below).- Routing, script counting and fused-line email disclaimers follow the regenerated Python 0.4.0probes. New mixed-field language heuristics are not yet ported.- Not yet ported: abstention (`min_confidence`, including per-bucket thresholds),- histogram-binning calibration, `predict_long` and `lang_temperatures`.+ histogram-binning calibration and `lang_temperatures`.diff --git a/laya-rust/README.md b/laya-rust/README.mdindex 7f8c9b2a..4711ab4c 100644--- a/laya-rust/README.md+++ b/laya-rust/README.md@@ -33,6 +33,7 @@ Mirrors the [Laya Python SDK](https://github.com/NandhaKishorM/laya/blob/main/RE- [Writing questions](#writing-questions)- [Built-in presets](#built-in-presets)- [Reading results](#reading-results)+- [Long prediction](#long-prediction)- [Thread safety](#thread-safety)- [Execution providers and cargo features](#execution-providers-and-cargo-features)- [Async usage](#async-usage)@@ -439,6 +440,57 @@ let options = LayaOptions {---+## Long prediction++`predict` reads one sequence: a state longer than the room the questions leave is truncated (the+start is kept, or the end for a conversation-turn list). `LayaEngine::predict_long` scans the whole+state instead, in overlapping token windows, and keeps one answer per question. It ports+`ONNXAgent.predict_long` with `aggregate="auto"`:++```rust,no_run+# use laya::{LayaEngine, LongOptions, QuestionSet};+# let engine = LayaEngine::from_directory("x")?; let questions = QuestionSet::new();+# let long_transcript = "";+let options = LongOptions::new().window(96).stride(48).batch_size(8);+let result = engine.predict_long(long_transcript, &questions, &options)?;++result.scan_usage().unwrap().windows; // windows scored (1 = no scan was needed)+let answer = &result["refund_requested"];+if let Some(window) = answer.window() { // None for a single-pass state+ println!("decided by window {}/{} (tokens {}..{})",+ window.index + 1, window.count, window.token_start, window.token_end);+}+# Ok::<(), laya::LayaError>(())+```++- **Aggregation.** `noul` takes the window with the greatest P(true). `choice` and `score` take the+ window with the greatest `answer_confidence`. A tie goes to the first window. The probabilities+ are that window's, **not** a calibrated number for the whole document.+- **One pass.** A state that fits the room the questions leave (with an explicit+ `LongOptions::window`, a state that fits that window) is scored by the ordinary `predict` pass:+ one window, no `Answer::window()`.+- **Options.** `window`, `stride` and `batch_size` are optional and `0` counts as unset. The default+ window is `max(64, max_len - head_max_len - 8)`, capped at the smallest state room any question+ leaves; the default stride is half the effective window. A stride that would skip tokens between+ windows is `LayaError::InvalidWindow`, as is a question set whose options leave no room for the+ state. `aggregate` accepts only `"auto"` (`LayaError::UnsupportedAggregate` otherwise).+- **Usage.** `result.usage().input_tokens` is summed over windows. `result.state_usage()` sums+ `state_tokens` and `state_tokens_dropped` over windows and lists the last window's truncated+ questions; `result.scan_usage()` gives the window count and how many windows lost state tokens.+- **Re-encoding.** Each window is decoded back to text with special tokens kept (Python's default+ `decode`) and scored as an ordinary state, so its tokens can differ slightly from the original+ after re-encoding, exactly as in Python. A tokenizer that does not override `LayaTokenizer::decode`+ makes `predict_long` fail with `LayaError::Tokenizer`.+- **Router.** `LayaRouter::predict_long(state, &questions, &options, model, task, lang)` routes once,+ on the original state, then scans on the chosen checkpoint. A predictor without a scan capability+ (see `LayaPredictor::as_long_predictor`) gives `LayaError::NotSupported` rather than truncating.+- **Not ported.** Per-call hooks, per-language temperatures and the confidence gate are not part of+ `predict_long` here, and the two `warnings.warn` notices Python emits when it caps the window or+ reduces the stride are not emitted (the crate has no logging dependency); the window and stride+ themselves match.++---+## Routing, language detection, email and shortlistRunnable end-to-end example: [`samples/laya-routing`](https://github.com/NandhaKishorM/laya/tree/main/laya-rust/samples/laya-routing).@@ -611,7 +663,7 @@ invocation that has the `download` feature enabled (the default) with no externa|---|---|---|| `src/python_json.rs` | Byte-for-byte `json.dumps` dialects | `PythonJson` || `src/questions.rs` | Typed question definitions | `QuestionType`, `Question`, `ChoiceQuestion`, `ScoreQuestion`, `NoulQuestion`, `QuestionSet` |-| `src/answers.rs` | Typed answer results | `Answer`, `ChoiceAnswer`, `ScoreAnswer`, `NoulAnswer`, `ActionInfo`, `LayaResult`, `Usage` |+| `src/answers.rs` | Typed answer results | `Answer`, `ChoiceAnswer`, `ScoreAnswer`, `NoulAnswer`, `ActionInfo`, `LayaResult`, `Usage`, `ScanUsage`, `StateUsage`, `WindowInfo` || `src/calibration.rs` | Temperature scaling and confidence math | `Calibration` || `src/laya_config.rs` | Parsed `rl_agent_config.json` | `LayaConfig` || `src/laya_options.rs` | Engine construction options | `LayaOptions`, `LayaCheckpoint`, `LayaExecutionProvider`, `ArtifactDownloadProgress` |@@ -622,6 +674,7 @@ invocation that has the `download` feature enabled (the default) with no externa| `src/tokenization/laya_tokenizer.rs` | Tokenizer abstraction | `LayaTokenizer` (trait) || `src/tokenization/hf_tokenizer.rs` | HuggingFace tokenizer implementation | `HfTokenizer` || `src/laya_engine.rs` | Loads the ONNX model and answers questions | `LayaEngine` |+| `src/long_prediction.rs` | Window plan and aggregation for `predict_long` | `LongOptions` || `src/error.rs` | Error type | `LayaError`, `Result<T>` — every fallible call in this crate returns `Result<T, LayaError>` |Where the equivalent Python code is a module of related functions (`common.py`, `agent.py`), the@@ -670,6 +723,7 @@ Rust port groups the corresponding functions into a unit struct with associateda follow-up; these probes are not a claim of full coverage of every new Python feature.- **Opt-in features are not ported yet:** abstention (`min_confidence`, including per-bucket- thresholds), histogram-binning calibration, `predict_long`, and per-language temperatures- (`lang_temperatures`). They are off by default in+ thresholds), histogram-binning calibration, and per-language temperatures+ (`lang_temperatures`). `predict_long` is ported, without hooks, per-language temperatures or+ the confidence gate. They are off by default inPython, so default predictions are unaffected. They are planned as follow-ups.diff --git a/laya-rust/src/answers.rs b/laya-rust/src/answers.rsindex 70db3075..cf182121 100644--- a/laya-rust/src/answers.rs+++ b/laya-rust/src/answers.rs@@ -25,6 +25,20 @@ pub struct ActionInfo {pub act_probability: f64,}+/// The scan window that decided an answer, from `LayaEngine::predict_long`. Offsets index the+/// tokenized state; the end is exclusive.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub struct WindowInfo {+ /// Zero-based index of the deciding window.+ pub index: usize,+ /// First state token in the window.+ pub token_start: usize,+ /// One past the last state token in the window.+ pub token_end: usize,+ /// Total windows scored.+ pub count: usize,+}+/// The answer to a [`QuestionType::Choice`] question.#[derive(Debug, Clone)]pub struct ChoiceAnswer {@@ -35,6 +49,7 @@ pub struct ChoiceAnswer {confidence: f64,answer_confidence: f64,action: ActionInfo,+ window: Option<WindowInfo>,}impl ChoiceAnswer {@@ -52,6 +67,7 @@ impl ChoiceAnswer {confidence,answer_confidence,action,+ window: None,}}@@ -75,6 +91,7 @@ pub struct ScoreAnswer {confidence: f64,answer_confidence: f64,action: ActionInfo,+ window: Option<WindowInfo>,}impl ScoreAnswer {@@ -94,6 +111,7 @@ impl ScoreAnswer {confidence,answer_confidence,action,+ window: None,}}@@ -118,6 +136,7 @@ pub struct NoulAnswer {confidence: f64,answer_confidence: f64,action: ActionInfo,+ window: Option<WindowInfo>,}impl NoulAnswer {@@ -133,6 +152,7 @@ impl NoulAnswer {confidence,answer_confidence,action,+ window: None,}}@@ -187,6 +207,26 @@ impl Answer {}}+ /// The scan window that decided this answer, or `None` for an ordinary `predict` and for a+ /// `predict_long` state short enough to be scored in one pass. The probabilities describe that+ /// window, not a calibrated whole-document probability.+ pub fn window(&self) -> Option<WindowInfo> {+ match self {+ Answer::Choice(c) => c.window,+ Answer::Score(s) => s.window,+ Answer::Noul(n) => n.window,+ }+ }++ pub(crate) fn with_window(mut self, window: WindowInfo) -> Self {+ match &mut self {+ Answer::Choice(c) => c.window = Some(window),+ Answer::Score(s) => s.window = Some(window),+ Answer::Noul(n) => n.window = Some(window),+ }+ self+ }+/// The action head's output.pub fn action(&self) -> ActionInfo {match self {@@ -247,6 +287,17 @@ pub struct StateUsage {pub options: IndexMap<String, OptionUsage>,}+/// How many scan windows `LayaEngine::predict_long` scored, and how many of them lost state+/// tokens. Kept beside [`Usage`] rather than in it so `Usage` struct literals keep compiling.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub struct ScanUsage {+ /// Windows scored; 1 when the state fit one ordinary pass.+ pub windows: usize,+ /// Windows in which at least one question lost state tokens (Python's `usage["truncated"]`+ /// for a scan; [`StateUsage::truncated`] is this being non-zero).+ pub truncated_windows: usize,+}+/// The result of one `predict` call: one answer per question, plus token usage.#[derive(Debug, Clone)]pub struct LayaResult {@@ -257,6 +308,7 @@ pub struct LayaResult {routing: Option<RouteDecision>,shortlist: Option<IndexMap<String, ShortlistInfo>>,state_usage: Option<StateUsage>,+ scan_usage: Option<ScanUsage>,}impl LayaResult {@@ -279,6 +331,7 @@ impl LayaResult {routing: None,shortlist: None,state_usage: None,+ scan_usage: None,}}@@ -325,6 +378,17 @@ impl LayaResult {self.state_usage.as_ref()}+ /// Window counts for a result from `LayaEngine::predict_long`; `None` for an ordinary+ /// `predict`. When present, [`Self::state_usage`] holds the sums across windows.+ pub fn scan_usage(&self) -> Option<ScanUsage> {+ self.scan_usage+ }++ pub(crate) fn with_scan_usage(mut self, usage: ScanUsage) -> Self {+ self.scan_usage = Some(usage);+ self+ }+pub(crate) fn with_state_usage(mut self, usage: StateUsage) -> Self {self.state_usage = Some(usage);selfdiff --git a/laya-rust/src/error.rs b/laya-rust/src/error.rsindex 3f306504..4c7c93f7 100644--- a/laya-rust/src/error.rs+++ b/laya-rust/src/error.rs@@ -102,6 +102,20 @@ pub enum LayaError {/// The configured total sequence budget.max_len: usize,},++ /// `predict_long` was given an aggregation mode other than `"auto"`.+ #[error("predict_long: only aggregate='auto' is supported")]+ UnsupportedAggregate(String),++ /// The scan window or stride cannot cover the state (no room for the state, or a stride that+ /// would leave tokens between windows unread). The text matches Python's `ValueError`.+ #[error("{0}")]+ InvalidWindow(String),++ /// The chosen predictor does not support the requested operation (a `predict_long` through a+ /// router whose predictor has no scan capability).+ #[error("{0}")]+ NotSupported(String),}#[cfg(test)]diff --git a/laya-rust/src/laya_engine.rs b/laya-rust/src/laya_engine.rsindex 3d2c3743..4b83792f 100644--- a/laya-rust/src/laya_engine.rs+++ b/laya-rust/src/laya_engine.rs@@ -13,13 +13,15 @@//! Tokenization needs no such lock: `tokenizers::Tokenizer::encode` takes `&self`.use crate::answers::{- ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScoreAnswer, StateUsage, Usage,+ ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScanUsage, ScoreAnswer, StateUsage,+ Usage,};use crate::calibration::Calibration;use crate::collator::{CollatedBatch, Collator, SequenceItem};use crate::error::{LayaError, Result};use crate::laya_config::LayaConfig;use crate::laya_options::{LayaCheckpoint, LayaExecutionProvider, LayaOptions};+use crate::long_prediction::{self, LongOptions};use crate::model_artifacts::{ModelArtifacts, ModelPaths};use crate::questions::{Question, QuestionSet};use crate::sequence_builder::SequenceBuilder;@@ -172,59 +174,186 @@ impl LayaEngine {"no questions to answer".to_string(),));}+ let mut results = self.predict_states(&[state], questions)?;+ Ok(results.remove(0))+ }- let ids: Vec<String> = questions.ids().map(str::to_string).collect();- let mut items = Vec::with_capacity(ids.len());- let mut state_usage = StateUsage {- state_tokens: 0,- state_tokens_dropped: 0,- truncated: false,- truncated_questions: Vec::new(),- options: IndexMap::new(),+ /// Scan a state longer than one sequence in overlapping token windows and keep, per question,+ /// the most confident window. Ports `ONNXAgent.predict_long` with `aggregate="auto"`:+ ///+ /// * `noul`: the window with the greatest P(true);+ /// * `choice` and `score`: the window with the greatest `answer_confidence`;+ /// * ties go to the first window.+ ///+ /// The returned probabilities are the deciding window's, **not** a calibrated number for the+ /// whole document. Each answer's [`Answer::window`] names that window, and+ /// [`LayaResult::scan_usage`] / [`LayaResult::state_usage`] sum the scan.+ ///+ /// A state the questions leave room for (or, with an explicit [`LongOptions::window`], a state+ /// that fits that window) takes the ordinary [`Self::predict`] pass: no scan, one window, no+ /// window attribution. Otherwise each window is decoded back to text (special tokens kept, as+ /// Python's default `decode`) and scored as an ordinary state, so a window's tokens may differ+ /// slightly from the original after re-encoding, exactly as in Python. Windows go through the+ /// model together unless [`LongOptions::batch_size`] or the automatic cap chunks them.+ ///+ /// Per-call hooks, per-language temperatures and the confidence gate are not part of this+ /// port. Errors: [`LayaError::UnsupportedAggregate`], [`LayaError::InvalidWindow`] (no room+ /// for the state, or a stride that leaves gaps), plus everything [`Self::predict`] returns.+ pub fn predict_long(+ &self,+ state: impl Into<Value>,+ questions: &QuestionSet,+ options: &LongOptions,+ ) -> Result<LayaResult> {+ self.predict_long_value(state.into(), questions, options)+ }++ /// [`Self::predict_long`], for a state that implements [`serde::Serialize`] but not+ /// `Into<serde_json::Value>`.+ pub fn predict_long_serialize(+ &self,+ state: &impl Serialize,+ questions: &QuestionSet,+ options: &LongOptions,+ ) -> Result<LayaResult> {+ let value = serde_json::to_value(state)?;+ self.predict_long_value(value, questions, options)+ }++ fn predict_long_value(+ &self,+ state: Value,+ questions: &QuestionSet,+ options: &LongOptions,+ ) -> Result<LayaResult> {+ options.check_aggregate()?;+ let tokenizer = self.tokenizer.as_ref();+ let explicit_window = options.window.unwrap_or(0);+ let budget = long_prediction::window_budget(+ tokenizer,+ questions,+ self.config.max_len,+ self.config.head_max_len,+ explicit_window,+ options.stride.unwrap_or(0),+ )?;+ let state_ids = SequenceBuilder::encode_state(tokenizer, &state)?;++ // `predict` reads the whole room the questions leave, so the default window must not force+ // a scan there; an explicit window still bounds the single pass (Python 3e9bda30).+ let one_pass = if explicit_window > 0 {+ budget.window+ } else {+ budget.room};+ if state_ids.len() <= one_pass {+ let single = self.predict_states(&[state], questions)?.remove(0);+ let truncated_windows = single.state_usage().is_some_and(|u| u.truncated) as usize;+ return Ok(single.with_scan_usage(ScanUsage {+ windows: 1,+ truncated_windows,+ }));+ }- // A conversation-turn list is serialized newest-last (see `PythonJson::state`), so the- // default right-truncation of an over-long state would silently drop the newest turn.- // Truncate from the left for lists so the most recent turn survives instead. Ports- // `Agent._encode_state`'s `truncate_left = isinstance(state, list)` (laya/agent.py, laya- // 0.3.21) — 0.3.6 never set this, so every state was right-truncated regardless of shape.- let truncate_left = matches!(state, Value::Array(_));-- // The tokenizer here needs no lock: tokenizers::Tokenizer::encode takes &self.- for id in &ids {- let question = &questions[id.as_str()];- let (seq, markers, stats) = SequenceBuilder::build_with_stats(- self.tokenizer.as_ref(),- &state,- question,- self.config.max_len,- self.config.head_max_len,- truncate_left,- )?;- state_usage.state_tokens = stats.tokens;- state_usage.state_tokens_dropped = state_usage.state_tokens_dropped.max(stats.dropped);- if stats.dropped > 0 {- state_usage.truncated = true;- state_usage.truncated_questions.push(id.clone());- }- if let Some(options) = stats.options {- state_usage.options.insert(id.clone(), options);- }+ let spans = long_prediction::spans(state_ids.len(), budget.window, budget.stride);+ let windows = spans+ .iter()+ .map(|span| {+ tokenizer+ .decode(&state_ids[span.token_start..span.token_end])+ .map(Value::String)+ })+ .collect::<Result<Vec<Value>>>()?;+ let cap = long_prediction::batch_cap(+ spans.len(),+ budget.window,+ long_prediction::default_window(self.config.max_len, self.config.head_max_len),+ options.batch_size.unwrap_or(0),+ )+ .unwrap_or(spans.len());+ let mut results = Vec::with_capacity(spans.len());+ for chunk in windows.chunks(cap) {+ results.extend(self.predict_states(chunk, questions)?);+ }+ Ok(long_prediction::aggregate(+ MODEL_NAME, questions, &results, &spans,+ ))+ }- // A marker dropped for landing past max_len means an option is not in the sequence at- // all, so its logit would score whatever token happens to sit at position 0.- let expected_markers = SequenceBuilder::render_options(question).len();- if markers.len() != expected_markers {- return Err(LayaError::NoMarkers {- id: id.clone(),- head_max_len: self.config.head_max_len,- actual_markers: markers.len(),- expected_markers,- max_len: self.config.max_len,- });- }+ /// Scores every question against every state in one collate and one forward pass, returning+ /// one result (with its own state usage) per state, in order. With no questions each result is+ /// empty and carries no state usage. `predict` is this with one state.+ pub(crate) fn predict_states(+ &self,+ states: &[Value],+ questions: &QuestionSet,+ ) -> Result<Vec<LayaResult>> {+ let ids: Vec<String> = questions.ids().map(str::to_string).collect();+ if ids.is_empty() {+ return Ok(states+ .iter()+ .map(|_| LayaResult::new(MODEL_NAME, Vec::new(), HashMap::new(), Usage::default()))+ .collect());+ }++ let mut items = Vec::with_capacity(ids.len() * states.len());+ let mut state_usages = Vec::with_capacity(states.len());+ for state in states {+ let mut state_usage = StateUsage {+ state_tokens: 0,+ state_tokens_dropped: 0,+ truncated: false,+ truncated_questions: Vec::new(),+ options: IndexMap::new(),+ };- items.push(SequenceItem::new(seq, markers, question.question_type()));+ // A conversation-turn list is serialized newest-last (see `PythonJson::state`), so the+ // default right-truncation of an over-long state would silently drop the newest turn.+ // Truncate from the left for lists so the most recent turn survives instead. Ports+ // `Agent._encode_state`'s `truncate_left = isinstance(state, list)` (laya/agent.py,+ // laya 0.3.21) — 0.3.6 never set this, so every state was right-truncated regardless+ // of shape.+ let truncate_left = matches!(state, Value::Array(_));++ // The tokenizer here needs no lock: tokenizers::Tokenizer::encode takes &self.+ for id in &ids {+ let question = &questions[id.as_str()];+ let (seq, markers, stats) = SequenceBuilder::build_with_stats(+ self.tokenizer.as_ref(),+ state,+ question,+ self.config.max_len,+ self.config.head_max_len,+ truncate_left,+ )?;+ state_usage.state_tokens = stats.tokens;+ state_usage.state_tokens_dropped =+ state_usage.state_tokens_dropped.max(stats.dropped);+ if stats.dropped > 0 {+ state_usage.truncated = true;+ state_usage.truncated_questions.push(id.clone());+ }+ if let Some(options) = stats.options {+ state_usage.options.insert(id.clone(), options);+ }++ // A marker dropped for landing past max_len means an option is not in the+ // sequence at all, so its logit would score whatever token happens to sit at+ // position 0.+ let expected_markers = SequenceBuilder::render_options(question).len();+ if markers.len() != expected_markers {+ return Err(LayaError::NoMarkers {+ id: id.clone(),+ head_max_len: self.config.head_max_len,+ actual_markers: markers.len(),+ expected_markers,+ max_len: self.config.max_len,+ });+ }++ items.push(SequenceItem::new(seq, markers, question.question_type()));+ }+ state_usages.push(state_usage);}let batch = Collator::collate(&items, self.tokenizer.special().pad_id)?;@@ -234,35 +363,48 @@ impl LayaEngine {// hardcoded so this stays correct if the graph is re-exported with a wider action head.let n_act = output.act_logits.len() / batch.count;- let mut answers = HashMap::with_capacity(ids.len());- for (row, id) in ids.iter().enumerate() {- let question = &questions[id.as_str()];- let k = batch.marker_counts[row];- let temperature =- Calibration::resolve_temperature(&self.config, question.question_type(), k);- let row_start = row * batch.marker_count;- let row_logits = &output.logits[row_start..row_start + batch.marker_count];- let p = Calibration::probabilities(row_logits, k, temperature);-- let act_start = row * n_act;- let act_row = &output.act_logits[act_start..act_start + n_act];- let action = ActionInfo {- act_probability: Calibration::round4(Calibration::softmax(act_row)[0]),- };+ let mut results = Vec::with_capacity(states.len());+ for (state_index, state_usage) in state_usages.into_iter().enumerate() {+ let offset = state_index * ids.len();+ let mut answers = HashMap::with_capacity(ids.len());+ for (q, id) in ids.iter().enumerate() {+ let row = offset + q;+ let question = &questions[id.as_str()];+ let k = batch.marker_counts[row];+ let temperature =+ Calibration::resolve_temperature(&self.config, question.question_type(), k);+ let row_start = row * batch.marker_count;+ let row_logits = &output.logits[row_start..row_start + batch.marker_count];+ let p = Calibration::probabilities(row_logits, k, temperature);++ let act_start = row * n_act;+ let act_row = &output.act_logits[act_start..act_start + n_act];+ let action = ActionInfo {+ act_probability: Calibration::round4(Calibration::softmax(act_row)[0]),+ };++ answers.insert(id.clone(), build_answer(question, &p, action));+ }- answers.insert(id.clone(), build_answer(question, &p, action));+ // This state's own non-padding tokens; with one state it equals `batch.input_tokens`.+ let input_tokens: usize = items[offset..offset + ids.len()]+ .iter()+ .map(|item| item.ids.len())+ .sum();+ results.push(+ LayaResult::new(+ MODEL_NAME,+ ids.clone(),+ answers,+ Usage {+ input_tokens: input_tokens as u32,+ output_tokens: 0,+ },+ )+ .with_state_usage(state_usage),+ );}-- Ok(LayaResult::new(- MODEL_NAME,- ids,- answers,- Usage {- input_tokens: batch.input_tokens as u32,- output_tokens: 0,- },- )- .with_state_usage(state_usage))+ Ok(results)}/// One forward pass over a collated batch, returning the raw graph outputs. See thediff --git a/laya-rust/src/laya_router.rs b/laya-rust/src/laya_router.rsindex edd1c1c6..1eb75b29 100644--- a/laya-rust/src/laya_router.rs+++ b/laya-rust/src/laya_router.rs@@ -41,6 +41,7 @@ use crate::error::{LayaError, Result};use crate::language_detection::{LanguageAnalysis, LanguageDetection};use crate::laya_engine::LayaEngine;use crate::laya_options::{LayaCheckpoint, LayaOptions};+use crate::long_prediction::LongOptions;use crate::questions::QuestionSet;use serde_json::Value;use std::collections::{HashMap, HashSet};@@ -58,12 +59,49 @@ use std::sync::{Arc, Mutex};pub trait LayaPredictor: Send + Sync {/// Answer every question in `questions` about `state`.fn predict(&self, state: Value, questions: &QuestionSet) -> Result<LayaResult>;++ /// This predictor's scan capability, if it has one. The default is `None`, so implementing+ /// [`LayaPredictor`] elsewhere keeps compiling; [`LayaEngine`] and [`LayaRouter`] return+ /// themselves. [`LayaRouter::predict_long`] needs it: Rust has no run-time interface check on+ /// a `dyn LayaPredictor`, and a predictor without it must fail loudly rather than silently+ /// truncate a long state.+ fn as_long_predictor(&self) -> Option<&dyn LayaLongPredictor> {+ None+ }+}++/// A [`LayaPredictor`] that can also scan a state longer than one sequence. The Rust counterpart+/// of Python's router checking `getattr(agent, "predict_long", None)`.+pub trait LayaLongPredictor: LayaPredictor {+ /// Scan overlapping state-token windows and select one answer per question. See+ /// [`LayaEngine::predict_long`].+ fn predict_long(+ &self,+ state: Value,+ questions: &QuestionSet,+ options: &LongOptions,+ ) -> Result<LayaResult>;}impl LayaPredictor for LayaEngine {fn predict(&self, state: Value, questions: &QuestionSet) -> Result<LayaResult> {LayaEngine::predict(self, state, questions)}++ fn as_long_predictor(&self) -> Option<&dyn LayaLongPredictor> {+ Some(self)+ }+}++impl LayaLongPredictor for LayaEngine {+ fn predict_long(+ &self,+ state: Value,+ questions: &QuestionSet,+ options: &LongOptions,+ ) -> Result<LayaResult> {+ LayaEngine::predict_long(self, state, questions, options)+ }}/// Question-id signatures of the four typed-decisions workflows, used only when@@ -554,6 +592,34 @@ impl LayaRouter {let result = engine.predict(state, questions)?;Ok(result.with_routing(decision))}++ /// Route once, on the original state, then scan the whole state on the chosen checkpoint with+ /// [`LayaEngine::predict_long`]. The result's [`LayaResult::routing`] records the decision.+ ///+ /// Returns [`LayaError::NotSupported`] when the chosen predictor (for example one registered+ /// with [`Self::attach`]) has no scan capability, rather than truncating the state.+ pub fn predict_long(+ &self,+ state: impl Into<Value>,+ questions: &QuestionSet,+ options: &LongOptions,+ model: Option<&str>,+ task: Option<&str>,+ lang: Option<&str>,+ ) -> Result<LayaResult> {+ let state = state.into();+ let decision = self.route(&state, Some(questions), model, task, lang)?;+ let engine = self.load_checkpoint(decision.model)?;+ let scanner = engine.as_long_predictor().ok_or_else(|| {+ LayaError::NotSupported(format!(+ "the predictor for '{}' has no predict_long, so a state longer than its window \+ cannot be scanned",+ decision.model.subdir()+ ))+ })?;+ let result = scanner.predict_long(state, questions, options)?;+ Ok(result.with_routing(decision))+ }}impl LayaPredictor for LayaRouter {@@ -562,6 +628,22 @@ impl LayaPredictor for LayaRouter {fn predict(&self, state: Value, questions: &QuestionSet) -> Result<LayaResult> {LayaRouter::predict(self, state, questions, None, None, None)}++ fn as_long_predictor(&self) -> Option<&dyn LayaLongPredictor> {+ Some(self)+ }+}++impl LayaLongPredictor for LayaRouter {+ /// [`LayaRouter::predict_long`] with every override left at its default.+ fn predict_long(+ &self,+ state: Value,+ questions: &QuestionSet,+ options: &LongOptions,+ ) -> Result<LayaResult> {+ LayaRouter::predict_long(self, state, questions, options, None, None, None)+ }}/// Python `repr()` for a string: single-quoted unless the string itself contains a `'` and nodiff --git a/laya-rust/src/lib.rs b/laya-rust/src/lib.rsindex c8f8ae3e..00ce9093 100644--- a/laya-rust/src/lib.rs+++ b/laya-rust/src/lib.rs@@ -50,13 +50,17 @@ pub mod laya_options;pub mod laya_presets;pub mod laya_router;pub mod laya_shortlist;+pub mod long_prediction;pub mod model_artifacts;pub mod python_json;pub mod questions;pub mod sequence_builder;pub mod tokenization;-pub use answers::{ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScoreAnswer, Usage};+pub use answers::{+ ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScanUsage, ScoreAnswer, StateUsage,+ Usage, WindowInfo,+};pub use calibration::Calibration;pub use collator::{CollatedBatch, Collator, SequenceItem};pub use error::{LayaError, Result};@@ -68,8 +72,11 @@ pub use laya_options::{ArtifactDownloadProgress, LayaCheckpoint, LayaExecutionProvider, LayaOptions,};pub use laya_presets::LayaPresets;-pub use laya_router::{LayaPredictor, LayaRouter, LayaRouterOptions, RouteDecision};+pub use laya_router::{+ LayaLongPredictor, LayaPredictor, LayaRouter, LayaRouterOptions, RouteDecision,+};pub use laya_shortlist::{DEFAULT_SHORTLIST_K, LayaShortlist, ShortlistInfo};+pub use long_prediction::LongOptions;pub use model_artifacts::{ModelArtifacts, ModelLayout, ModelPaths};pub use python_json::PythonJson;pub use questions::{diff --git a/laya-rust/src/long_prediction.rs b/laya-rust/src/long_prediction.rsnew file mode 100644index 00000000..b215a6ae--- /dev/null+++ b/laya-rust/src/long_prediction.rs@@ -0,0 +1,802 @@+//! The window plan behind `LayaEngine::predict_long`: how wide the scan windows are, where they+//! start, how many go through one forward pass, and how the per-window answers collapse into one.+//! Ports `window_budget`, `window_batch_cap` and `state_room` from `laya/common.py` and the+//! aggregation in `ONNXAgent.predict_long` (`laya/onnx_agent.py`), `aggregate="auto"` only.+//!+//! Everything here is pure arithmetic over token counts and finished results, so it is testable+//! without a model. The engine supplies the tokenizer-dependent inputs (the per-question state+//! room and the state's token ids) and runs the windows.+//!+//! Python also emits `warnings.warn` when it caps the window or reduces the stride. This crate has+//! no logging dependency (see the crate docs), so those two warnings are not emitted; the+//! resulting window and stride are the same.++use crate::answers::{Answer, LayaResult, OptionUsage, ScanUsage, StateUsage, Usage, WindowInfo};+use crate::error::{LayaError, Result};+use crate::questions::{QuestionSet, QuestionType};+use crate::sequence_builder::SequenceBuilder;+use crate::tokenization::LayaTokenizer;+use indexmap::IndexMap;+use std::collections::HashMap;++/// Options for `LayaEngine::predict_long`. Every field is optional; `Default` gives Python's+/// defaults. Build one with the chained setters.+///+/// Per-call hooks, per-language temperatures and the confidence gate that Python's `predict_long`+/// also accepts are not part of this port.+#[derive(Debug, Clone, Default, PartialEq, Eq)]+#[non_exhaustive]+pub struct LongOptions {+ /// State tokens per window. `None` or `0` uses the checkpoint default,+ /// `max(64, max_len - head_max_len - 8)`. Capped at the smallest state room the questions+ /// leave.+ pub window: Option<usize>,+ /// Token step between windows. `None` or `0` uses half the effective window. A step past the+ /// requested window is an error; a valid step past a window the questions' room clamped is+ /// reduced.+ pub stride: Option<usize>,+ /// Only `"auto"` (the default when `None`) is supported.+ pub aggregate: Option<String>,+ /// Maximum windows per forward pass. `None` or `0` uses Python's automatic cap.+ pub batch_size: Option<usize>,+}++impl LongOptions {+ /// Python's defaults: checkpoint window, half-window stride, `"auto"`, automatic batch cap.+ pub fn new() -> Self {+ Self::default()+ }++ /// Set [`Self::window`].+ pub fn window(mut self, window: usize) -> Self {+ self.window = Some(window);+ self+ }++ /// Set [`Self::stride`].+ pub fn stride(mut self, stride: usize) -> Self {+ self.stride = Some(stride);+ self+ }++ /// Set [`Self::aggregate`]. Anything but `"auto"` makes `predict_long` return+ /// [`LayaError::UnsupportedAggregate`].+ pub fn aggregate(mut self, aggregate: impl Into<String>) -> Self {+ self.aggregate = Some(aggregate.into());+ self+ }++ /// Set [`Self::batch_size`].+ pub fn batch_size(mut self, batch_size: usize) -> Self {+ self.batch_size = Some(batch_size);+ self+ }++ pub(crate) fn check_aggregate(&self) -> Result<()> {+ match self.aggregate.as_deref() {+ None | Some("auto") => Ok(()),+ Some(other) => Err(LayaError::UnsupportedAggregate(other.to_string())),+ }+ }+}++/// The resolved scan geometry, plus the room that decides the one-pass shortcut.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub struct WindowBudget {+ /// Tokens per window, after clamping to `room`.+ pub window: usize,+ /// Tokens between window starts.+ pub stride: usize,+ /// The smallest state room any question leaves.+ pub room: usize,+}++/// The checkpoint's default window: `max(64, max_len - head_max_len - 8)`, the 64 being a floor.+pub(crate) fn default_window(max_len: usize, head_max_len: usize) -> usize {+ max_len+ .saturating_sub(head_max_len)+ .saturating_sub(8)+ .max(64)+}++/// `window_budget` with the per-question rooms already measured. An empty `rooms` (no questions)+/// leaves the requested window as the room.+pub(crate) fn plan_budget(+ rooms: &[usize],+ max_len: usize,+ head_max_len: usize,+ window: usize,+ stride: usize,+) -> Result<WindowBudget> {+ let requested = if window > 0 {+ window+ } else {+ default_window(max_len, head_max_len)+ };+ let room = rooms.iter().copied().min().unwrap_or(requested);+ if room == 0 {+ return Err(LayaError::InvalidWindow(format!(+ "predict_long: the questions' options fill the whole sequence (max_len={max_len}, \+ head_max_len={head_max_len}), leaving no room for the state; no window can carry any \+ of it. A label set this large is what laya.shortlist.predict_shortlist is for"+ )));+ }+ let size = requested.min(room);+ let mut step = if stride > 0 {+ stride+ } else {+ (size / 2).max(1)+ };+ if step > size {+ if size < requested && stride <= requested {+ step = (size / 2).max(1);+ } else {+ return Err(LayaError::InvalidWindow(format!(+ "predict_long: stride={step} steps past the {size}-token window{}, so {} tokens \+ between every pair of windows would be read by no window at all; pass stride <= \+ {size}",+ if size < requested {+ " these questions leave room for"+ } else {+ ""+ },+ step - size+ )));+ }+ }+ Ok(WindowBudget {+ window: size,+ stride: step,+ room,+ })+}++/// `window_budget`: measures each question's state room, then plans.+///+/// Public only so the parity tests, a separate crate, can compare the effective window, stride and+/// room with Python's; hidden from the docs like `LayaOutput`.+#[doc(hidden)]+pub fn window_budget(+ tokenizer: &dyn LayaTokenizer,+ questions: &QuestionSet,+ max_len: usize,+ head_max_len: usize,+ window: usize,+ stride: usize,+) -> Result<WindowBudget> {+ let mut rooms = Vec::with_capacity(questions.len());+ for id in questions.ids() {+ rooms.push(SequenceBuilder::state_room(+ tokenizer,+ &questions[id],+ max_len,+ head_max_len,+ )?);+ }+ plan_budget(&rooms, max_len, head_max_len, window, stride)+}++/// The overlapping windows over `tokens` state tokens. A window that reaches the end is the last.+pub(crate) fn spans(tokens: usize, window: usize, stride: usize) -> Vec<WindowInfo> {+ let mut bounds = Vec::new();+ let mut start = 0;+ while start < tokens {+ bounds.push((start, (start + window).min(tokens)));+ if start + window >= tokens {+ break;+ }+ start += stride;+ }+ let count = bounds.len();+ bounds+ .into_iter()+ .enumerate()+ .map(|(index, (token_start, token_end))| WindowInfo {+ index,+ token_start,+ token_end,+ count,+ })+ .collect()+}++/// `window_batch_cap`: windows per forward pass, or `None` for all of them in one pass.+pub(crate) fn batch_cap(+ count: usize,+ window: usize,+ config_budget: usize,+ batch_size: usize,+) -> Option<usize> {+ if batch_size > 0 {+ return Some(batch_size);+ }+ if window >= config_budget {+ return None;+ }+ let step = (window / 2).max(1);+ let config_step = (config_budget / 2).max(1);+ let unclamped = (count * step).div_ceil(config_step).max(1);+ // Capping that multiplies the window count by 2 or less keeps the shared pass.+ (count > unclamped * 2).then_some(unclamped)+}++/// Collapse per-window results into one: for each question the window with the greatest evidence+/// (P(true) for noul, `answer_confidence` for choice and score) wins, the first window on ties.+pub(crate) fn aggregate(+ model: &str,+ questions: &QuestionSet,+ results: &[LayaResult],+ spans: &[WindowInfo],+) -> LayaResult {+ let ids: Vec<String> = questions.ids().map(str::to_string).collect();+ let mut answers: HashMap<String, Answer> = HashMap::with_capacity(ids.len());+ for id in &ids {+ let evidence = |index: usize| -> f64 {+ let answer = &results[index][id.as_str()];+ match (questions[id.as_str()].question_type(), answer.as_noul()) {+ (QuestionType::Noul, Some(noul)) => noul.probability,+ _ => answer.answer_confidence(),+ }+ };+ let mut best = 0;+ for index in 1..results.len() {+ if evidence(index) > evidence(best) {+ best = index;+ }+ }+ answers.insert(+ id.clone(),+ results[best][id.as_str()].clone().with_window(spans[best]),+ );+ }++ let usage = Usage {+ input_tokens: results.iter().map(|r| r.usage().input_tokens).sum(),+ output_tokens: 0,+ };+ let result = LayaResult::new(model, ids, answers, usage);+ let diagnostics: Vec<&StateUsage> = results.iter().filter_map(|r| r.state_usage()).collect();+ let truncated_windows = diagnostics.iter().filter(|u| u.truncated).count();+ let scan = ScanUsage {+ windows: results.len(),+ truncated_windows,+ };+ if diagnostics.is_empty() {+ return result.with_scan_usage(scan);+ }+ let mut options: IndexMap<String, OptionUsage> = IndexMap::new();+ for usage in &diagnostics {+ for (id, stats) in &usage.options {+ options.insert(id.clone(), *stats);+ }+ }+ result+ .with_state_usage(StateUsage {+ state_tokens: diagnostics.iter().map(|u| u.state_tokens).sum(),+ state_tokens_dropped: diagnostics.iter().map(|u| u.state_tokens_dropped).sum(),+ truncated: truncated_windows > 0,+ // The last window's list, as in Python's scan aggregation.+ truncated_questions: results+ .last()+ .and_then(|r| r.state_usage())+ .map(|u| u.truncated_questions.clone())+ .unwrap_or_default(),+ options,+ })+ .with_scan_usage(scan)+}++#[cfg(test)]+mod tests {+ use super::*;+ use crate::answers::{ActionInfo, ChoiceAnswer, NoulAnswer, ScoreAnswer};+ use crate::questions::Question;+ use serde_json::Value;++ fn budget(rooms: &[usize], window: usize, stride: usize) -> Result<WindowBudget> {+ plan_budget(rooms, 512, 192, window, stride)+ }++ fn message(result: Result<WindowBudget>) -> String {+ result.unwrap_err().to_string()+ }++ #[test]+ fn default_window_has_a_64_token_floor() {+ assert_eq!(default_window(512, 192), 312);+ assert_eq!(default_window(128, 120), 64);+ assert_eq!(default_window(64, 192), 64);+ }++ #[test]+ fn budget_defaults_to_the_checkpoint_window_and_half_stride() {+ let b = budget(&[478, 400], 0, 0).unwrap();+ assert_eq!((b.window, b.stride, b.room), (312, 156, 400));+ }++ #[test]+ fn budget_without_questions_uses_the_requested_window_as_room() {+ let b = budget(&[], 32, 0).unwrap();+ assert_eq!((b.window, b.stride, b.room), (32, 16, 32));+ }++ #[test]+ fn budget_clamps_the_window_to_the_room() {+ let b = budget(&[100], 4096, 0).unwrap();+ assert_eq!((b.window, b.stride, b.room), (100, 50, 100));+ let b = budget(&[100], 0, 0).unwrap();+ assert_eq!((b.window, b.stride), (100, 50));+ }++ #[test]+ fn budget_keeps_a_stride_inside_the_clamped_window() {+ let b = budget(&[100], 4096, 60).unwrap();+ assert_eq!(b.stride, 60);+ }++ #[test]+ fn budget_rejects_a_stride_past_the_window() {+ assert_eq!(+ message(budget(&[478], 32, 33)),+ "predict_long: stride=33 steps past the 32-token window, so 1 tokens between every \+ pair of windows would be read by no window at all; pass stride <= 32"+ );+ }++ #[test]+ fn budget_rejects_a_stride_past_the_requested_window_even_when_clamped() {+ assert_eq!(+ message(budget(&[100], 120, 130)),+ "predict_long: stride=130 steps past the 100-token window these questions leave room \+ for, so 30 tokens between every pair of windows would be read by no window at all; \+ pass stride <= 100"+ );+ }++ #[test]+ fn budget_reduces_a_stride_past_the_clamped_window_but_within_the_request() {+ let b = budget(&[100], 4096, 2048).unwrap();+ assert_eq!((b.window, b.stride), (100, 50));+ }++ #[test]+ fn budget_refuses_when_the_questions_leave_no_room() {+ assert_eq!(+ message(budget(&[300, 0], 0, 0)),+ "predict_long: the questions' options fill the whole sequence (max_len=512, \+ head_max_len=192), leaving no room for the state; no window can carry any of it. A \+ label set this large is what laya.shortlist.predict_shortlist is for"+ );+ }++ #[test]+ fn spans_overlap_and_stop_at_the_window_that_reaches_the_end() {+ let s = spans(100, 40, 20);+ let bounds: Vec<_> = s.iter().map(|w| (w.token_start, w.token_end)).collect();+ assert_eq!(bounds, [(0, 40), (20, 60), (40, 80), (60, 100)]);+ assert!(s.iter().all(|w| w.count == 4));+ assert_eq!(s[3].index, 3);+ }++ #[test]+ fn spans_clip_the_last_window_and_handle_an_exact_fit() {+ let s = spans(50, 40, 30);+ let bounds: Vec<_> = s.iter().map(|w| (w.token_start, w.token_end)).collect();+ assert_eq!(bounds, [(0, 40), (30, 50)]);+ assert_eq!(spans(40, 40, 20).len(), 1);+ assert!(spans(0, 40, 20).is_empty());+ }++ #[test]+ fn batch_cap_honours_an_explicit_size() {+ assert_eq!(batch_cap(10, 312, 312, 3), Some(3));+ assert_eq!(batch_cap(10, 23, 312, 3), Some(3));+ }++ #[test]+ fn batch_cap_is_off_unless_the_window_was_clamped_hard() {+ assert_eq!(batch_cap(29, 312, 312, 0), None);+ // A 64-token budget reduced to 43: under twice the windows, a mild cap.+ assert_eq!(batch_cap(20, 43, 64, 0), None);+ // 23 against 312: windows multiplied about 13x, bounded at the un-capped count.+ assert_eq!(batch_cap(413, 23, 312, 0), Some(30));+ }++ fn noul(p: f64) -> Answer {+ Answer::Noul(NoulAnswer::new(p, p, p, ActionInfo::default()))+ }++ fn choice(c: f64) -> Answer {+ let mut p = IndexMap::new();+ p.insert("a".to_string(), c);+ Answer::Choice(ChoiceAnswer::new("a", p, c, c, ActionInfo::default()))+ }++ fn window_result(+ noul_answer: Answer,+ choice_answer: Answer,+ input_tokens: u32,+ dropped: usize,+ ) -> LayaResult {+ let mut answers = HashMap::new();+ answers.insert("flag".to_string(), noul_answer);+ answers.insert("topic".to_string(), choice_answer);+ LayaResult::new(+ "m",+ vec!["flag".to_string(), "topic".to_string()],+ answers,+ Usage {+ input_tokens,+ output_tokens: 0,+ },+ )+ .with_state_usage(StateUsage {+ state_tokens: 40,+ state_tokens_dropped: dropped,+ truncated: dropped > 0,+ truncated_questions: if dropped > 0 {+ vec!["flag".to_string()]+ } else {+ Vec::new()+ },+ options: IndexMap::new(),+ })+ }++ fn question_set() -> QuestionSet {+ QuestionSet::new()+ .with("flag", Question::noul("flag?", Value::Null, Value::Null))+ .unwrap()+ .with(+ "topic",+ Question::choice_labels("topic?", ["a", "b"]).unwrap(),+ )+ .unwrap()+ }++ #[test]+ fn aggregate_takes_max_probability_for_noul_and_max_confidence_for_choice() {+ let results = vec![+ window_result(noul(0.2), choice(0.6), 10, 0),+ window_result(noul(0.9), choice(0.5), 11, 3),+ window_result(noul(0.4), choice(0.7), 12, 0),+ ];+ let all = spans(100, 40, 30);+ let out = aggregate("m", &question_set(), &results, &all[..3]);+ assert_eq!(out["flag"].window().unwrap().index, 1);+ assert_eq!(out["topic"].window().unwrap().index, 2);+ assert_eq!(out["flag"].as_noul().unwrap().probability, 0.9);+ assert_eq!(out.usage().input_tokens, 33);+ let state = out.state_usage().unwrap();+ assert_eq!(state.state_tokens, 120);+ assert_eq!(state.state_tokens_dropped, 3);+ assert!(state.truncated);+ // The last window's list, which lost nothing.+ assert!(state.truncated_questions.is_empty());+ assert_eq!(+ out.scan_usage(),+ Some(ScanUsage {+ windows: 3,+ truncated_windows: 1+ })+ );+ }++ #[test]+ fn aggregate_prefers_the_first_window_on_ties() {+ let results = vec![+ window_result(noul(0.7), choice(0.6), 10, 0),+ window_result(noul(0.7), choice(0.6), 10, 0),+ ];+ let all = spans(60, 40, 20);+ let out = aggregate("m", &question_set(), &results, &all);+ assert_eq!(out["flag"].window().unwrap().index, 0);+ assert_eq!(out["topic"].window().unwrap().index, 0);+ }++ #[test]+ fn aggregate_merges_options_with_later_windows_winning() {+ let usage = |distinct| OptionUsage {+ total: 3,+ distinct,+ tokens_per_option: Some(4),+ };+ let with_option = |distinct| {+ let result = window_result(noul(0.1), choice(0.5), 1, 0);+ let mut state = result.state_usage().unwrap().clone();+ state.options.insert("topic".into(), usage(distinct));+ result.with_state_usage(state)+ };+ let all = spans(60, 40, 20);+ let out = aggregate(+ "m",+ &question_set(),+ &[with_option(1), with_option(2)],+ &all,+ );+ assert_eq!(out.state_usage().unwrap().options["topic"], usage(2));+ }++ #[test]+ fn aggregate_without_questions_still_counts_windows() {+ let empty = || LayaResult::new("m", Vec::new(), HashMap::new(), Usage::default());+ let all = spans(60, 40, 20);+ let out = aggregate("m", &QuestionSet::new(), &[empty(), empty()], &all);+ assert!(out.is_empty());+ assert!(out.state_usage().is_none());+ assert_eq!(out.scan_usage().unwrap().windows, 2);+ }++ #[test]+ fn default_budget_without_questions_keeps_the_64_floor_for_a_widened_head() {+ let b = plan_budget(&[], 200, 192, 0, 0).unwrap();+ assert_eq!((b.window, b.stride, b.room), (64, 32, 64));+ let b = plan_budget(&[], 512, 192, 0, 0).unwrap();+ assert_eq!((b.window, b.stride, b.room), (312, 156, 312));+ // Nothing clamps the window, so a stride past it is the caller's error.+ let error = message(plan_budget(&[], 512, 192, 40, 41));+ assert!(+ error.contains("stride=41 steps past the 40-token"),+ "{error}"+ );+ assert!(!error.contains("these questions leave room for"), "{error}");+ }++ #[test]+ fn budget_room_may_exceed_the_default_window_and_an_explicit_one() {+ let b = budget(&[478], 0, 0).unwrap();+ assert_eq!((b.window, b.room), (312, 478));+ let b = budget(&[478], 100, 0).unwrap();+ assert_eq!((b.window, b.room), (100, 478));+ }++ #[test]+ fn budget_room_equals_the_window_when_the_room_caps_it() {+ for window in [0, 1000] {+ let b = budget(&[120, 200], window, 0).unwrap();+ assert_eq!((b.window, b.room), (120, 120));+ }+ }++ #[test]+ fn budget_stride_equal_to_the_requested_window_is_reduced_after_the_clamp() {+ let b = budget(&[100], 1000, 1000).unwrap();+ assert_eq!((b.window, b.stride, b.room), (100, 50, 100));+ let error = message(budget(&[100], 1000, 1001));+ assert!(error.contains("these questions leave room for"), "{error}");+ }++ #[test]+ fn budget_floors_the_default_stride_at_one_token() {+ let b = budget(&[], 1, 0).unwrap();+ assert_eq!((b.window, b.stride), (1, 1));+ }++ #[test]+ fn budget_with_zero_state_room_names_the_cause() {+ let error = message(plan_budget(&[0], 10, 192, 0, 0));+ assert!(+ error.starts_with(+ "predict_long: the questions' options fill the whole sequence (max_len=10, \+ head_max_len=192), leaving no room for the state"+ ),+ "{error}"+ );+ }++ #[test]+ fn spans_cover_every_token_and_stop_at_the_first_complete_window() {+ // (tokens, window, stride, count, last start, last end)+ let cases = [+ (64, 64, 32, 1, 0, 64),+ (65, 64, 32, 2, 32, 65),+ (128, 64, 32, 3, 64, 128),+ (129, 64, 32, 4, 96, 129),+ (129, 64, 64, 3, 128, 129),+ (5, 1, 1, 5, 4, 5),+ ];+ for (tokens, window, stride, count, last_start, last_end) in cases {+ let all = spans(tokens, window, stride);+ assert_eq!(all.len(), count, "{tokens}/{window}/{stride}");+ let last = all[count - 1];+ assert_eq!(+ (last.index, last.token_start, last.token_end, last.count),+ (count - 1, last_start, last_end, count)+ );+ assert_eq!(all[0].token_start, 0);+ for (i, w) in all.iter().enumerate() {+ assert_eq!((w.index, w.count), (i, count));+ if i > 0 {+ assert!(w.token_start <= all[i - 1].token_end);+ }+ }+ }+ }++ #[test]+ fn spans_of_a_short_state_are_one_span_and_end_at_the_length() {+ let one = spans(10, 64, 32);+ assert_eq!(one.len(), 1);+ assert_eq!(+ (+ one[0].index,+ one[0].token_start,+ one[0].token_end,+ one[0].count+ ),+ (0, 0, 10, 1)+ );+ let two = spans(100, 64, 48);+ let bounds: Vec<_> = two.iter().map(|w| (w.token_start, w.token_end)).collect();+ assert_eq!(bounds, [(0, 64), (48, 100)]);+ assert!(two.iter().all(|w| w.count == 2));+ }++ #[test]+ fn batch_cap_edge_cases_match_python() {+ // (count, window, budget, batch_size, expected)+ let cases = [+ (100, 312, 312, 7, Some(7)), // explicit size, even when the window is not capped+ (100, 50, 1, 0, None), // window >= budget; budget / 2 floors at 1+ (1, 1, 312, 0, None),+ (2, 50, 312, 0, None), // n == 2 * unclamped keeps the single pass+ (3, 50, 312, 0, Some(1)), // one past it chunks at unclamped+ (12, 50, 312, 0, Some(2)), // ceil(300 / 156) = 2; 12 > 4+ (34, 50, 312, 0, Some(6)), // ceil(850 / 156) = 6; 34 > 12+ (5, 200, 312, 0, None), // ceil(500 / 156) = 4; 5 <= 8+ (100, 1, 312, 0, Some(1)), // window / 2 floors at 1+ (100, 312, 312, 0, None),+ (100, 200, 312, 0, None),+ (100, 50, 312, 0, Some(17)),+ (100, 50, 312, 3, Some(3)),+ ];+ for (count, window, budget, batch, expected) in cases {+ assert_eq!(+ batch_cap(count, window, budget, batch),+ expected,+ "{count}/{window}/{budget}/{batch}"+ );+ }+ }++ fn score(level: f64, c: f64) -> Answer {+ Answer::Score(ScoreAnswer::new(+ level,+ vec![Value::from("low"), Value::from("high")],+ vec![1.0 - c, c],+ 0.1,+ c,+ ActionInfo::default(),+ ))+ }++ fn three_questions() -> QuestionSet {+ question_set()+ .with(+ "urgency",+ Question::score("urgent?", ["low", "high"]).unwrap(),+ )+ .unwrap()+ }++ fn triple(noul_answer: Answer, c: f64, s: f64, input: u32) -> LayaResult {+ let mut answers = HashMap::new();+ answers.insert("flag".to_string(), noul_answer);+ answers.insert("topic".to_string(), choice(c));+ answers.insert("urgency".to_string(), score(s, s));+ LayaResult::new(+ "m",+ vec!["flag".into(), "topic".into(), "urgency".into()],+ answers,+ Usage {+ input_tokens: input,+ output_tokens: 7,+ },+ )+ }++ #[test]+ fn aggregate_selects_evidence_not_entropy() {+ // Window 1's noul has the lower answer_confidence but the higher P(true): it must win.+ let low_p = Answer::Noul(NoulAnswer::new(0.05, 0.95, 0.95, ActionInfo::default()));+ let high_p = Answer::Noul(NoulAnswer::new(0.7, 0.7, 0.7, ActionInfo::default()));+ let all = spans(65, 64, 32);+ let out = aggregate(+ "m",+ &three_questions(),+ &[triple(low_p, 0.8, 0.6, 20), triple(high_p, 0.8, 0.9, 30)],+ &all,+ );+ assert_eq!(out["flag"].window(), Some(all[1]));+ assert_eq!(out["topic"].window(), Some(all[0])); // tie+ assert_eq!(out["urgency"].window(), Some(all[1]));+ assert_eq!(out["urgency"].as_score().unwrap().score, 0.9);+ assert_eq!(out.usage().input_tokens, 50);+ assert_eq!(out.usage().output_tokens, 0);+ }++ #[test]+ fn aggregate_ties_go_to_the_first_window_for_all_three_types() {+ let tie = || triple(noul(0.7), 0.8, 0.6, 20);+ let all = spans(65, 64, 32);+ let out = aggregate("m", &three_questions(), &[tie(), tie()], &all);+ for id in ["flag", "topic", "urgency"] {+ assert_eq!(out[id].window(), Some(all[0]), "{id}");+ }+ assert_eq!(out.scan_usage().unwrap().windows, 2);+ }++ #[test]+ fn aggregate_of_one_window_without_state_usage_leaves_the_fields_none() {+ let all = spans(10, 64, 32);+ let out = aggregate(+ "m",+ &three_questions(),+ &[triple(noul(0.7), 0.8, 0.6, 20)],+ &all,+ );+ let w = out["flag"].window().unwrap();+ assert_eq!(+ (w.index, w.token_start, w.token_end, w.count),+ (0, 0, 10, 1)+ );+ assert_eq!(out.usage().input_tokens, 20);+ assert_eq!(out.usage().output_tokens, 0);+ assert!(out.state_usage().is_none());+ assert_eq!(+ out.scan_usage(),+ Some(ScanUsage {+ windows: 1,+ truncated_windows: 0+ })+ );+ }++ #[test]+ fn aggregate_carries_the_last_windows_truncated_questions() {+ let truncated = |input, names: &[&str]| {+ triple(noul(0.7), 0.8, 0.6, input).with_state_usage(StateUsage {+ state_tokens: 10,+ state_tokens_dropped: 1,+ truncated: true,+ truncated_questions: names.iter().map(|n| n.to_string()).collect(),+ options: IndexMap::new(),+ })+ };+ let all = spans(65, 64, 32);+ let out = aggregate(+ "m",+ &three_questions(),+ &[truncated(20, &["urgency"]), truncated(30, &[])],+ &all,+ );+ let state = out.state_usage().unwrap();+ assert!(state.truncated_questions.is_empty());+ assert_eq!(state.state_tokens_dropped, 2);+ assert_eq!(out.scan_usage().unwrap().truncated_windows, 2);+ }++ #[test]+ fn long_options_default_to_python_and_check_aggregate() {+ let options = LongOptions::new();+ assert_eq!(+ (options.window, options.stride, options.batch_size),+ (None, None, None)+ );+ assert!(options.check_aggregate().is_ok());+ assert!(+ LongOptions::new()+ .aggregate("auto")+ .check_aggregate()+ .is_ok()+ );+ let error = LongOptions::new()+ .aggregate("max")+ .check_aggregate()+ .unwrap_err();+ assert!(+ matches!(error, LayaError::UnsupportedAggregate(_)),+ "{error:?}"+ );+ }+}diff --git a/laya-rust/src/sequence_builder.rs b/laya-rust/src/sequence_builder.rsindex 24c28ab7..4f8a9b9b 100644--- a/laya-rust/src/sequence_builder.rs+++ b/laya-rust/src/sequence_builder.rs@@ -23,6 +23,13 @@ pub(crate) struct StateTokenStats {pub options: Option<crate::answers::OptionUsage>,}+/// The question-specific start of a sequence, before any state.+struct Head {+ ids: Vec<u32>,+ markers: Vec<usize>,+ options: Option<crate::answers::OptionUsage>,+}+impl SequenceBuilder {/// One option fragment is capped at this many tokens, before the marker is prepended.const MAX_OPTION_TOKENS: usize = 48;@@ -136,14 +143,13 @@ impl SequenceBuilder {.map(|(ids, markers, _)| (ids, markers))}- pub(crate) fn build_with_stats(+ /// The question's own part of the sequence: everything up to and including the SEP that+ /// closes the options. Ports `build_head` from `laya/common.py`.+ fn build_head(tokenizer: &dyn LayaTokenizer,- state: &Value,question: &Question,- max_len: usize,head_max_len: usize,- truncate_left: bool,- ) -> Result<(Vec<u32>, Vec<usize>, StateTokenStats)> {+ ) -> Result<Head> {let special = tokenizer.special();let mask_token = special.mask_token.as_str();@@ -189,7 +195,7 @@ impl SequenceBuilder {header_ids.truncate(head_take);}- let mut ids: Vec<u32> = Vec::with_capacity(max_len);+ let mut ids: Vec<u32> = Vec::with_capacity(head_max_len + 2);ids.push(special.cls_id);ids.extend_from_slice(&header_ids);ids.push(special.sep_id);@@ -201,6 +207,58 @@ impl SequenceBuilder {}ids.push(special.sep_id);+ let distinct = option_ids+ .iter()+ .collect::<std::collections::HashSet<_>>()+ .len();+ let options = (distinct < option_ids.len()).then_some(crate::answers::OptionUsage {+ total: option_ids.len(),+ distinct,+ tokens_per_option,+ });+ Ok(Head {+ ids,+ markers,+ options,+ })+ }++ /// How many state tokens `question` leaves inside `max_len`: what `build_sequence` keeps of+ /// the state. Ports `state_room` from `laya/common.py`.+ pub(crate) fn state_room(+ tokenizer: &dyn LayaTokenizer,+ question: &Question,+ max_len: usize,+ head_max_len: usize,+ ) -> Result<usize> {+ let head = Self::build_head(tokenizer, question, head_max_len)?;+ // -1 reserves the SEP that closes the state.+ Ok((max_len as i64 - head.ids.len() as i64 - 1).max(0) as usize)+ }++ /// The state's token ids exactly as `build_with_stats` slices them: serialized, mask token+ /// scrubbed, encoded without special tokens.+ pub(crate) fn encode_state(tokenizer: &dyn LayaTokenizer, state: &Value) -> Result<Vec<u32>> {+ let mask_token = tokenizer.special().mask_token.as_str();+ tokenizer.encode(&Self::scrub(&PythonJson::state(state), mask_token))+ }++ pub(crate) fn build_with_stats(+ tokenizer: &dyn LayaTokenizer,+ state: &Value,+ question: &Question,+ max_len: usize,+ head_max_len: usize,+ truncate_left: bool,+ ) -> Result<(Vec<u32>, Vec<usize>, StateTokenStats)> {+ let special = tokenizer.special();+ let mask_token = special.mask_token.as_str();+ let Head {+ mut ids,+ mut markers,+ options,+ } = Self::build_head(tokenizer, question, head_max_len)?;+// -1 reserves the final SEP that closes the whole sequence, appended below.let room = (max_len as i64 - ids.len() as i64 - 1).max(0) as usize;let state_text = Self::scrub(&PythonJson::state(state), mask_token);@@ -225,14 +283,7 @@ impl SequenceBuilder {let stats = StateTokenStats {tokens: state_ids.len(),dropped: state_ids.len() - sliced.len(),- options: {- let distinct = option_ids.iter().collect::<std::collections::HashSet<_>>().len();- (distinct < option_ids.len()).then_some(crate::answers::OptionUsage {- total: option_ids.len(),- distinct,- tokens_per_option,- })- },+ options,};Ok((ids, markers, stats))}diff --git a/laya-rust/src/tokenization/hf_tokenizer.rs b/laya-rust/src/tokenization/hf_tokenizer.rsindex 9f91fce1..72938bc4 100644--- a/laya-rust/src/tokenization/hf_tokenizer.rs+++ b/laya-rust/src/tokenization/hf_tokenizer.rs@@ -91,6 +91,14 @@ impl LayaTokenizer for HfTokenizer {fn special(&self) -> &SpecialTokens {&self.special}++ fn decode(&self, ids: &[u32]) -> Result<String> {+ // skip_special_tokens = false keeps `[CLS]`, `<bos>`, ... in the text, as Python's default+ // `decode` does, so the re-encoded window sees the same tokens the scan cut out.+ self.inner+ .decode(ids, false)+ .map_err(|e| LayaError::Tokenizer(format!("failed to decode tokens: {e}")))+ }}// ── special-token resolution ─────────────────────────────────────────────────diff --git a/laya-rust/src/tokenization/laya_tokenizer.rs b/laya-rust/src/tokenization/laya_tokenizer.rsindex 17b739a1..3385543a 100644--- a/laya-rust/src/tokenization/laya_tokenizer.rs+++ b/laya-rust/src/tokenization/laya_tokenizer.rs@@ -36,4 +36,16 @@ pub trait LayaTokenizer: Send + Sync {/// The special token ids and mask token string this tokenizer resolved at load time.fn special(&self) -> &SpecialTokens;++ /// Decodes token ids back to text, **keeping** special tokens in the output: Python's default+ /// `tok.decode(ids)` (`skip_special_tokens=False`). Only `LayaEngine::predict_long` uses this,+ /// to turn a token window back into a state string.+ ///+ /// A default method (returning an error) rather than a required one, so a downstream+ /// implementation of this public trait keeps compiling; [`crate::HfTokenizer`] overrides it.+ fn decode(&self, _ids: &[u32]) -> Result<String> {+ Err(crate::error::LayaError::Tokenizer(+ "this tokenizer does not support decode, which predict_long needs".to_string(),+ ))+ }}diff --git a/laya-rust/tests/common/golden_data.rs b/laya-rust/tests/common/golden_data.rsindex 3988dae4..8a912af5 100644--- a/laya-rust/tests/common/golden_data.rs+++ b/laya-rust/tests/common/golden_data.rs@@ -130,6 +130,22 @@ impl CheckpointGoldenData {.collect()}+ /// Cases recorded with `"kind": "long"` that return a result (`ONNXAgent.predict_long`).+ pub fn long_cases(&self) -> Vec<GoldenCaseInfo> {+ self.index()+ .into_iter()+ .filter(|c| c.kind.as_deref() == Some("long") && !c.expects_error)+ .collect()+ }++ /// Cases recorded with `"kind": "long"` that raise a Python `ValueError`.+ pub fn long_error_cases(&self) -> Vec<GoldenCaseInfo> {+ self.index()+ .into_iter()+ .filter(|c| c.kind.as_deref() == Some("long") && c.expects_error)+ .collect()+ }+/// Cases recorded with `"kind": "shortlist"` (`shortlist_many_options`): a hashing-embedder/// reduced choice question run through the ordinary predict pipeline, not a plain/// `Agent.predict` recording, so it is kept out of [`Self::success_cases`].diff --git a/laya-rust/tests/common/result_compare.rs b/laya-rust/tests/common/result_compare.rsindex bba85d44..6869292e 100644--- a/laya-rust/tests/common/result_compare.rs+++ b/laya-rust/tests/common/result_compare.rs@@ -42,6 +42,12 @@ pub fn assert_result_matches(where_case: &str, expected: &Value, result: &LayaRe"{where_case}: usage.output_tokens");+ assert_answers_match(where_case, expected, result);+}++/// Assert the answer ids (in order) and every answer's type, confidences, action and per-type+/// probabilities match the golden `expected` result, ignoring its model name and usage.+pub fn assert_answers_match(where_case: &str, expected: &Value, result: &LayaResult) {let answers = expected["answers"].as_object().unwrap();let expected_ids: Vec<&str> = answers.keys().map(String::as_str).collect();assert_eq!(diff --git a/laya-rust/tests/golden/english/case_long_clamp.json b/laya-rust/tests/golden/english/case_long_clamp.jsonnew file mode 100644index 00000000..e0327d38--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_clamp.json@@ -0,0 +1,128 @@+{+ "name": "long_clamp",+ "state": "Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 4096,+ "stride": 2048,+ "batch_size": 2+ },+ "state_tokens": 857,+ "effective_window": 478,+ "effective_stride": 239,+ "effective_room": 478,+ "windows": [+ {+ "token_start": 0,+ "token_end": 478,+ "text": "Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account"+ },+ {+ "token_start": 239,+ "token_end": 717,+ "text": ". Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal"+ },+ {+ "token_start": 478,+ "token_end": 857,+ "text": " activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9124,+ "technical": 0.0563,+ "other": 0.0313+ },+ "confidence": 0.6778,+ "answer_confidence": 0.9124,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 478,+ "token_end": 857,+ "count": 3+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.4876,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.1355,+ "1": 0.2414,+ "2": 0.6231+ },+ "confidence": 0.1729,+ "answer_confidence": 0.6231,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 478,+ "token_end": 857,+ "count": 3+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.9655,+ "confidence": 0.9655,+ "answer_confidence": 0.9655,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 478,+ "token_end": 857,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 4287,+ "output_tokens": 0,+ "state_tokens": 1335,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 3+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_dict.json b/laya-rust/tests/golden/english/case_long_dict.jsonnew file mode 100644index 00000000..bb73d54d--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_dict.json@@ -0,0 +1,129 @@+{+ "name": "long_dict",+ "state": {+ "notes": "Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. ",+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ },+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 96+ },+ "state_tokens": 179,+ "effective_window": 96,+ "effective_stride": 48,+ "effective_room": 478,+ "windows": [+ {+ "token_start": 0,+ "token_end": 96,+ "text": "{\"notes\": \"Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine"+ },+ {+ "token_start": 48,+ "token_end": 144,+ "text": ". Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. \", \"message\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan"+ },+ {+ "token_start": 96,+ "token_end": 179,+ "text": " checks completed. Routine checks completed. Routine checks completed. Routine checks completed. \", \"message\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.\"}"+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8794,+ "technical": 0.0613,+ "other": 0.0593+ },+ "confidence": 0.5889,+ "answer_confidence": 0.8794,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 179,+ "count": 3+ }+ },+ "urgency": {+ "type": "score",+ "score": 0.4528,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.6652,+ "1": 0.2168,+ "2": 0.118+ },+ "confidence": 0.2219,+ "answer_confidence": 0.6652,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 0,+ "token_start": 0,+ "token_end": 96,+ "count": 3+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.8946,+ "confidence": 0.8946,+ "answer_confidence": 0.8946,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 179,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 1107,+ "output_tokens": 0,+ "state_tokens": 275,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 3+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_empty_questions.json b/laya-rust/tests/golden/english/case_long_empty_questions.jsonnew file mode 100644index 00000000..34e42d25--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_empty_questions.json@@ -0,0 +1,64 @@+{+ "name": "long_empty_questions",+ "state": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. ",+ "questions": {},+ "options": {+ "window": 32,+ "batch_size": 2+ },+ "state_tokens": 141,+ "effective_window": 32,+ "effective_stride": 16,+ "effective_room": 32,+ "windows": [+ {+ "token_start": 0,+ "token_end": 32,+ "text": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 16,+ "token_end": 48,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 32,+ "token_end": 64,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 48,+ "token_end": 80,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 64,+ "token_end": 96,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 80,+ "token_end": 112,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 96,+ "token_end": 128,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 112,+ "token_end": 141,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {},+ "usage": {+ "input_tokens": 0,+ "output_tokens": 0,+ "windows": 8+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_exact_limit.json b/laya-rust/tests/golden/english/case_long_exact_limit.jsonnew file mode 100644index 00000000..75a7361f--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_exact_limit.json@@ -0,0 +1,90 @@+{+ "name": "long_exact_limit",+ "state": "a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 312,+ "effective_window": 312,+ "effective_stride": 156,+ "effective_room": 478,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.5786,+ "technical": 0.2117,+ "other": 0.2097+ },+ "confidence": 0.1145,+ "answer_confidence": 0.5786,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 0.9141,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.3697,+ "1": 0.3466,+ "2": 0.2838+ },+ "confidence": 0.0055,+ "answer_confidence": 0.3697,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.1347,+ "confidence": 0.8653,+ "answer_confidence": 0.8653,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1030,+ "output_tokens": 0,+ "state_tokens": 312,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_invalid_aggregate.json b/laya-rust/tests/golden/english/case_long_invalid_aggregate.jsonnew file mode 100644index 00000000..1b1a6496--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_invalid_aggregate.json@@ -0,0 +1,33 @@+{+ "name": "long_invalid_aggregate",+ "state": "short",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "aggregate": "mean"+ },+ "error_type": "ValueError",+ "error_message": "predict_long: only aggregate='auto' is supported"+}diff --git a/laya-rust/tests/golden/english/case_long_invalid_stride.json b/laya-rust/tests/golden/english/case_long_invalid_stride.jsonnew file mode 100644index 00000000..d710ad47--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_invalid_stride.json@@ -0,0 +1,34 @@+{+ "name": "long_invalid_stride",+ "state": "short",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 32,+ "stride": 33+ },+ "error_type": "ValueError",+ "error_message": "predict_long: stride=33 steps past the 32-token window, so 1 tokens between every pair of windows would be read by no window at all; pass stride <= 32"+}diff --git a/laya-rust/tests/golden/english/case_long_list.json b/laya-rust/tests/golden/english/case_long_list.jsonnew file mode 100644index 00000000..9ed8257d--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_list.json@@ -0,0 +1,146 @@+{+ "name": "long_list",+ "state": [+ {+ "role": "assistant",+ "content": "Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. "+ },+ {+ "role": "user",+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 96,+ "stride": 48+ },+ "state_tokens": 244,+ "effective_window": 96,+ "effective_stride": 48,+ "effective_room": 478,+ "windows": [+ {+ "token_start": 0,+ "token_end": 96,+ "text": "[{\"role\": \"assistant\", \"content\": \"Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we"+ },+ {+ "token_start": 48,+ "token_end": 144,+ "text": " check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we"+ },+ {+ "token_start": 96,+ "token_end": 192,+ "text": " check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. \"}, {\"role\": \"user\", \"content\": \"Subject: Double charged on invoice 882"+ },+ {+ "token_start": 144,+ "token_end": 240,+ "text": " check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. \"}, {\"role\": \"user\", \"content\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling"+ },+ {+ "token_start": 192,+ "token_end": 244,+ "text": "13\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.\"}]"+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9092,+ "technical": 0.0544,+ "other": 0.0364+ },+ "confidence": 0.6674,+ "answer_confidence": 0.9092,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 144,+ "token_end": 240,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.8904,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.029,+ "1": 0.0516,+ "2": 0.9194+ },+ "confidence": 0.6969,+ "answer_confidence": 0.9194,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 192,+ "token_end": 244,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.9909,+ "confidence": 0.9909,+ "answer_confidence": 0.9909,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 192,+ "token_end": 244,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 1778,+ "output_tokens": 0,+ "state_tokens": 436,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_multi_chunk.json b/laya-rust/tests/golden/english/case_long_multi_chunk.jsonnew file mode 100644index 00000000..4b2cd74e--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_multi_chunk.json@@ -0,0 +1,176 @@+{+ "name": "long_multi_chunk",+ "state": "The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "batch_size": 2+ },+ "state_tokens": 2037,+ "effective_window": 312,+ "effective_stride": 156,+ "effective_room": 478,+ "windows": [+ {+ "token_start": 0,+ "token_end": 312,+ "text": "The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the"+ },+ {+ "token_start": 156,+ "token_end": 468,+ "text": " checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and"+ },+ {+ "token_start": 312,+ "token_end": 624,+ "text": " account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine"+ },+ {+ "token_start": 468,+ "token_end": 780,+ "text": " recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates"+ },+ {+ "token_start": 624,+ "token_end": 936,+ "text": " status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The"+ },+ {+ "token_start": 780,+ "token_end": 1092,+ "text": ". The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked"+ },+ {+ "token_start": 936,+ "token_end": 1248,+ "text": " agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account"+ },+ {+ "token_start": 1092,+ "token_end": 1404,+ "text": " the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded"+ },+ {+ "token_start": 1248,+ "token_end": 1560,+ "text": " and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status"+ },+ {+ "token_start": 1404,+ "token_end": 1716,+ "text": " routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates."+ },+ {+ "token_start": 1560,+ "token_end": 1872,+ "text": " updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent"+ },+ {+ "token_start": 1716,+ "token_end": 2028,+ "text": " The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed"+ },+ {+ "token_start": 1872,+ "token_end": 2037,+ "text": " checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8998,+ "technical": 0.0733,+ "other": 0.0268+ },+ "confidence": 0.6508,+ "answer_confidence": 0.8998,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 12,+ "token_start": 1872,+ "token_end": 2037,+ "count": 13+ }+ },+ "urgency": {+ "type": "score",+ "score": 0.5771,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.588,+ "1": 0.2468,+ "2": 0.1651+ },+ "confidence": 0.1308,+ "answer_confidence": 0.588,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 6,+ "token_start": 936,+ "token_end": 1248,+ "count": 13+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.9029,+ "confidence": 0.9029,+ "answer_confidence": 0.9029,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 12,+ "token_start": 1872,+ "token_end": 2037,+ "count": 13+ }+ }+ },+ "usage": {+ "input_tokens": 12949,+ "output_tokens": 0,+ "state_tokens": 3909,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 13+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_no_state_room.json b/laya-rust/tests/golden/english/case_long_no_state_room.jsonnew file mode 100644index 00000000..2c1bf91c--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_no_state_room.json@@ -0,0 +1,315 @@+{+ "name": "long_no_state_room",+ "state": "short",+ "questions": {+ "too_wide": {+ "type": "choice",+ "instructions": "Too many options to fit the sequence.",+ "criteria": {+ "opt_000": "description 0",+ "opt_001": "description 1",+ "opt_002": "description 2",+ "opt_003": "description 3",+ "opt_004": "description 4",+ "opt_005": "description 5",+ "opt_006": "description 6",+ "opt_007": "description 7",+ "opt_008": "description 8",+ "opt_009": "description 9",+ "opt_010": "description 10",+ "opt_011": "description 11",+ "opt_012": "description 12",+ "opt_013": "description 13",+ "opt_014": "description 14",+ "opt_015": "description 15",+ "opt_016": "description 16",+ "opt_017": "description 17",+ "opt_018": "description 18",+ "opt_019": "description 19",+ "opt_020": "description 20",+ "opt_021": "description 21",+ "opt_022": "description 22",+ "opt_023": "description 23",+ "opt_024": "description 24",+ "opt_025": "description 25",+ "opt_026": "description 26",+ "opt_027": "description 27",+ "opt_028": "description 28",+ "opt_029": "description 29",+ "opt_030": "description 30",+ "opt_031": "description 31",+ "opt_032": "description 32",+ "opt_033": "description 33",+ "opt_034": "description 34",+ "opt_035": "description 35",+ "opt_036": "description 36",+ "opt_037": "description 37",+ "opt_038": "description 38",+ "opt_039": "description 39",+ "opt_040": "description 40",+ "opt_041": "description 41",+ "opt_042": "description 42",+ "opt_043": "description 43",+ "opt_044": "description 44",+ "opt_045": "description 45",+ "opt_046": "description 46",+ "opt_047": "description 47",+ "opt_048": "description 48",+ "opt_049": "description 49",+ "opt_050": "description 50",+ "opt_051": "description 51",+ "opt_052": "description 52",+ "opt_053": "description 53",+ "opt_054": "description 54",+ "opt_055": "description 55",+ "opt_056": "description 56",+ "opt_057": "description 57",+ "opt_058": "description 58",+ "opt_059": "description 59",+ "opt_060": "description 60",+ "opt_061": "description 61",+ "opt_062": "description 62",+ "opt_063": "description 63",+ "opt_064": "description 64",+ "opt_065": "description 65",+ "opt_066": "description 66",+ "opt_067": "description 67",+ "opt_068": "description 68",+ "opt_069": "description 69",+ "opt_070": "description 70",+ "opt_071": "description 71",+ "opt_072": "description 72",+ "opt_073": "description 73",+ "opt_074": "description 74",+ "opt_075": "description 75",+ "opt_076": "description 76",+ "opt_077": "description 77",+ "opt_078": "description 78",+ "opt_079": "description 79",+ "opt_080": "description 80",+ "opt_081": "description 81",+ "opt_082": "description 82",+ "opt_083": "description 83",+ "opt_084": "description 84",+ "opt_085": "description 85",+ "opt_086": "description 86",+ "opt_087": "description 87",+ "opt_088": "description 88",+ "opt_089": "description 89",+ "opt_090": "description 90",+ "opt_091": "description 91",+ "opt_092": "description 92",+ "opt_093": "description 93",+ "opt_094": "description 94",+ "opt_095": "description 95",+ "opt_096": "description 96",+ "opt_097": "description 97",+ "opt_098": "description 98",+ "opt_099": "description 99",+ "opt_100": "description 100",+ "opt_101": "description 101",+ "opt_102": "description 102",+ "opt_103": "description 103",+ "opt_104": "description 104",+ "opt_105": "description 105",+ "opt_106": "description 106",+ "opt_107": "description 107",+ "opt_108": "description 108",+ "opt_109": "description 109",+ "opt_110": "description 110",+ "opt_111": "description 111",+ "opt_112": "description 112",+ "opt_113": "description 113",+ "opt_114": "description 114",+ "opt_115": "description 115",+ "opt_116": "description 116",+ "opt_117": "description 117",+ "opt_118": "description 118",+ "opt_119": "description 119",+ "opt_120": "description 120",+ "opt_121": "description 121",+ "opt_122": "description 122",+ "opt_123": "description 123",+ "opt_124": "description 124",+ "opt_125": "description 125",+ "opt_126": "description 126",+ "opt_127": "description 127",+ "opt_128": "description 128",+ "opt_129": "description 129",+ "opt_130": "description 130",+ "opt_131": "description 131",+ "opt_132": "description 132",+ "opt_133": "description 133",+ "opt_134": "description 134",+ "opt_135": "description 135",+ "opt_136": "description 136",+ "opt_137": "description 137",+ "opt_138": "description 138",+ "opt_139": "description 139",+ "opt_140": "description 140",+ "opt_141": "description 141",+ "opt_142": "description 142",+ "opt_143": "description 143",+ "opt_144": "description 144",+ "opt_145": "description 145",+ "opt_146": "description 146",+ "opt_147": "description 147",+ "opt_148": "description 148",+ "opt_149": "description 149",+ "opt_150": "description 150",+ "opt_151": "description 151",+ "opt_152": "description 152",+ "opt_153": "description 153",+ "opt_154": "description 154",+ "opt_155": "description 155",+ "opt_156": "description 156",+ "opt_157": "description 157",+ "opt_158": "description 158",+ "opt_159": "description 159",+ "opt_160": "description 160",+ "opt_161": "description 161",+ "opt_162": "description 162",+ "opt_163": "description 163",+ "opt_164": "description 164",+ "opt_165": "description 165",+ "opt_166": "description 166",+ "opt_167": "description 167",+ "opt_168": "description 168",+ "opt_169": "description 169",+ "opt_170": "description 170",+ "opt_171": "description 171",+ "opt_172": "description 172",+ "opt_173": "description 173",+ "opt_174": "description 174",+ "opt_175": "description 175",+ "opt_176": "description 176",+ "opt_177": "description 177",+ "opt_178": "description 178",+ "opt_179": "description 179",+ "opt_180": "description 180",+ "opt_181": "description 181",+ "opt_182": "description 182",+ "opt_183": "description 183",+ "opt_184": "description 184",+ "opt_185": "description 185",+ "opt_186": "description 186",+ "opt_187": "description 187",+ "opt_188": "description 188",+ "opt_189": "description 189",+ "opt_190": "description 190",+ "opt_191": "description 191",+ "opt_192": "description 192",+ "opt_193": "description 193",+ "opt_194": "description 194",+ "opt_195": "description 195",+ "opt_196": "description 196",+ "opt_197": "description 197",+ "opt_198": "description 198",+ "opt_199": "description 199",+ "opt_200": "description 200",+ "opt_201": "description 201",+ "opt_202": "description 202",+ "opt_203": "description 203",+ "opt_204": "description 204",+ "opt_205": "description 205",+ "opt_206": "description 206",+ "opt_207": "description 207",+ "opt_208": "description 208",+ "opt_209": "description 209",+ "opt_210": "description 210",+ "opt_211": "description 211",+ "opt_212": "description 212",+ "opt_213": "description 213",+ "opt_214": "description 214",+ "opt_215": "description 215",+ "opt_216": "description 216",+ "opt_217": "description 217",+ "opt_218": "description 218",+ "opt_219": "description 219",+ "opt_220": "description 220",+ "opt_221": "description 221",+ "opt_222": "description 222",+ "opt_223": "description 223",+ "opt_224": "description 224",+ "opt_225": "description 225",+ "opt_226": "description 226",+ "opt_227": "description 227",+ "opt_228": "description 228",+ "opt_229": "description 229",+ "opt_230": "description 230",+ "opt_231": "description 231",+ "opt_232": "description 232",+ "opt_233": "description 233",+ "opt_234": "description 234",+ "opt_235": "description 235",+ "opt_236": "description 236",+ "opt_237": "description 237",+ "opt_238": "description 238",+ "opt_239": "description 239",+ "opt_240": "description 240",+ "opt_241": "description 241",+ "opt_242": "description 242",+ "opt_243": "description 243",+ "opt_244": "description 244",+ "opt_245": "description 245",+ "opt_246": "description 246",+ "opt_247": "description 247",+ "opt_248": "description 248",+ "opt_249": "description 249",+ "opt_250": "description 250",+ "opt_251": "description 251",+ "opt_252": "description 252",+ "opt_253": "description 253",+ "opt_254": "description 254",+ "opt_255": "description 255",+ "opt_256": "description 256",+ "opt_257": "description 257",+ "opt_258": "description 258",+ "opt_259": "description 259",+ "opt_260": "description 260",+ "opt_261": "description 261",+ "opt_262": "description 262",+ "opt_263": "description 263",+ "opt_264": "description 264",+ "opt_265": "description 265",+ "opt_266": "description 266",+ "opt_267": "description 267",+ "opt_268": "description 268",+ "opt_269": "description 269",+ "opt_270": "description 270",+ "opt_271": "description 271",+ "opt_272": "description 272",+ "opt_273": "description 273",+ "opt_274": "description 274",+ "opt_275": "description 275",+ "opt_276": "description 276",+ "opt_277": "description 277",+ "opt_278": "description 278",+ "opt_279": "description 279",+ "opt_280": "description 280",+ "opt_281": "description 281",+ "opt_282": "description 282",+ "opt_283": "description 283",+ "opt_284": "description 284",+ "opt_285": "description 285",+ "opt_286": "description 286",+ "opt_287": "description 287",+ "opt_288": "description 288",+ "opt_289": "description 289",+ "opt_290": "description 290",+ "opt_291": "description 291",+ "opt_292": "description 292",+ "opt_293": "description 293",+ "opt_294": "description 294",+ "opt_295": "description 295",+ "opt_296": "description 296",+ "opt_297": "description 297",+ "opt_298": "description 298",+ "opt_299": "description 299"+ }+ }+ },+ "options": {},+ "error_type": "ValueError",+ "error_message": "predict_long: the questions' options fill the whole sequence (max_len=512, head_max_len=192), leaving no room for the state; no window can carry any of it. A label set this large is what laya.shortlist.predict_shortlist is for"+}diff --git a/laya-rust/tests/golden/english/case_long_punctuation.json b/laya-rust/tests/golden/english/case_long_punctuation.jsonnew file mode 100644index 00000000..a41c0fd6--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_punctuation.json@@ -0,0 +1,138 @@+{+ "name": "long_punctuation",+ "state": "I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I want a refund ! I 'll cancel today .",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 64,+ "stride": 64,+ "batch_size": 3+ },+ "state_tokens": 299,+ "effective_window": 64,+ "effective_stride": 64,+ "effective_room": 478,+ "windows": [+ {+ "token_start": 0,+ "token_end": 64,+ "text": "I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 64,+ "token_end": 128,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 128,+ "token_end": 192,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 192,+ "token_end": 256,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 256,+ "token_end": 299,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I want a refund ! I 'll cancel today ."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9501,+ "technical": 0.023,+ "other": 0.0269+ },+ "confidence": 0.7881,+ "answer_confidence": 0.9501,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 299,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.6098,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.1266,+ "1": 0.137,+ "2": 0.7364+ },+ "confidence": 0.3089,+ "answer_confidence": 0.7364,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 299,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.9687,+ "confidence": 0.9687,+ "answer_confidence": 0.9687,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 299,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 1367,+ "output_tokens": 0,+ "state_tokens": 299,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_room_fast_path.json b/laya-rust/tests/golden/english/case_long_room_fast_path.jsonnew file mode 100644index 00000000..ce8ad891--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_room_fast_path.json@@ -0,0 +1,90 @@+{+ "name": "long_room_fast_path",+ "state": "a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 478,+ "effective_window": 312,+ "effective_stride": 156,+ "effective_room": 478,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.5284,+ "technical": 0.2499,+ "other": 0.2217+ },+ "confidence": 0.0738,+ "answer_confidence": 0.5284,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 0.8467,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.4008,+ "1": 0.3518,+ "2": 0.2475+ },+ "confidence": 0.0173,+ "answer_confidence": 0.4008,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.1659,+ "confidence": 0.8341,+ "answer_confidence": 0.8341,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 1528,+ "output_tokens": 0,+ "state_tokens": 478,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_short.json b/laya-rust/tests/golden/english/case_long_short.jsonnew file mode 100644index 00000000..fcbd242a--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_short.json@@ -0,0 +1,90 @@+{+ "name": "long_short",+ "state": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 57,+ "effective_window": 312,+ "effective_stride": 156,+ "effective_room": 478,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8355,+ "technical": 0.1139,+ "other": 0.0506+ },+ "confidence": 0.5008,+ "answer_confidence": 0.8355,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.8578,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0339,+ "1": 0.0745,+ "2": 0.8916+ },+ "confidence": 0.6264,+ "answer_confidence": 0.8916,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.968,+ "confidence": 0.968,+ "answer_confidence": 0.968,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 265,+ "output_tokens": 0,+ "state_tokens": 57,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/english/case_long_special_tokens.json b/laya-rust/tests/golden/english/case_long_special_tokens.jsonnew file mode 100644index 00000000..1cae8031--- /dev/null+++ b/laya-rust/tests/golden/english/case_long_special_tokens.json@@ -0,0 +1,177 @@+{+ "name": "long_special_tokens",+ "state": "The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. ",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 64,+ "batch_size": 2+ },+ "state_tokens": 421,+ "effective_window": 64,+ "effective_stride": 32,+ "effective_room": 478,+ "windows": [+ {+ "token_start": 0,+ "token_end": 64,+ "text": "The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <"+ },+ {+ "token_start": 32,+ "token_end": 96,+ "text": "[CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <e"+ },+ {+ "token_start": 64,+ "token_end": 128,+ "text": "bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <"+ },+ {+ "token_start": 96,+ "token_end": 160,+ "text": "os> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and em"+ },+ {+ "token_start": 128,+ "token_end": 192,+ "text": "mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀"+ },+ {+ "token_start": 160,+ "token_end": 224,+ "text": "ojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀."+ },+ {+ "token_start": 192,+ "token_end": 256,+ "text": " 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes "+ },+ {+ "token_start": 224,+ "token_end": 288,+ "text": " The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <"+ },+ {+ "token_start": 256,+ "token_end": 320,+ "text": "[CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <e"+ },+ {+ "token_start": 288,+ "token_end": 352,+ "text": "bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <"+ },+ {+ "token_start": 320,+ "token_end": 384,+ "text": "os> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and em"+ },+ {+ "token_start": 352,+ "token_end": 416,+ "text": "mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀"+ },+ {+ "token_start": 384,+ "token_end": 421,+ "text": "ojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. "+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "other",+ "probabilities": {+ "refund": 0.1677,+ "technical": 0.2108,+ "other": 0.6215+ },+ "confidence": 0.1597,+ "answer_confidence": 0.6215,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 96,+ "token_end": 160,+ "count": 13+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.4285,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.2439,+ "1": 0.0837,+ "2": 0.6724+ },+ "confidence": 0.2549,+ "answer_confidence": 0.6724,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 12,+ "token_start": 384,+ "token_end": 421,+ "count": 13+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.6175,+ "confidence": 0.6175,+ "answer_confidence": 0.6175,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 96,+ "token_end": 160,+ "count": 13+ }+ }+ },+ "usage": {+ "input_tokens": 3637,+ "output_tokens": 0,+ "state_tokens": 805,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 13+ }+ }+}diff --git a/laya-rust/tests/golden/english/index.json b/laya-rust/tests/golden/english/index.jsonindex 27fdf54e..5c2bf1ce 100644--- a/laya-rust/tests/golden/english/index.json+++ b/laya-rust/tests/golden/english/index.json@@ -84,5 +84,83 @@"file": "case_shortlist_many_options.json","expects_error": false,"kind": "shortlist"+ },+ {+ "name": "long_short",+ "file": "case_long_short.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_exact_limit",+ "file": "case_long_exact_limit.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_room_fast_path",+ "file": "case_long_room_fast_path.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_multi_chunk",+ "file": "case_long_multi_chunk.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_punctuation",+ "file": "case_long_punctuation.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_dict",+ "file": "case_long_dict.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_list",+ "file": "case_long_list.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_clamp",+ "file": "case_long_clamp.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_special_tokens",+ "file": "case_long_special_tokens.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_empty_questions",+ "file": "case_long_empty_questions.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_invalid_stride",+ "file": "case_long_invalid_stride.json",+ "expects_error": true,+ "kind": "long"+ },+ {+ "name": "long_invalid_aggregate",+ "file": "case_long_invalid_aggregate.json",+ "expects_error": true,+ "kind": "long"+ },+ {+ "name": "long_no_state_room",+ "file": "case_long_no_state_room.json",+ "expects_error": true,+ "kind": "long"}]diff --git a/laya-rust/tests/golden/english/meta.json b/laya-rust/tests/golden/english/meta.jsonindex 24260c96..3b262b40 100644--- a/laya-rust/tests/golden/english/meta.json+++ b/laya-rust/tests/golden/english/meta.json@@ -1,7 +1,7 @@{"generator": "tools/dump_golden.py","onnx_dir": "english",- "laya_version": "0.4.0",+ "laya_version": "0.4.11","onnxruntime_version": "1.30.0","transformers_version": "5.17.0","config": {@@ -70,12 +70,12 @@},"onnx_self_check": {"single_noul": {- "logits_maxdiff": 2.3871660232543945e-05,- "act_maxdiff": 0.001953125,+ "logits_maxdiff": 9.953975677490234e-06,+ "act_maxdiff": 0.0009765625,"act_probs_maxdiff": 0.0},"triage_4q": {- "logits_maxdiff": 1.823902130126953e-05,+ "logits_maxdiff": 1.946091651916504e-05,"act_maxdiff": 0.00146484375,"act_probs_maxdiff": 0.0}diff --git a/laya-rust/tests/golden/multilingual/case_long_clamp.json b/laya-rust/tests/golden/multilingual/case_long_clamp.jsonnew file mode 100644index 00000000..3e1e643c--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_clamp.json@@ -0,0 +1,94 @@+{+ "name": "long_clamp",+ "state": "Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 4096,+ "stride": 2048,+ "batch_size": 2+ },+ "state_tokens": 856,+ "effective_window": 990,+ "effective_stride": 495,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8343,+ "technical": 0.0249,+ "other": 0.1408+ },+ "confidence": 0.5275,+ "answer_confidence": 0.8343,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.4964,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0886,+ "1": 0.3263,+ "2": 0.585+ },+ "confidence": 0.1864,+ "answer_confidence": 0.585,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 1.0,+ "confidence": 1.0,+ "answer_confidence": 1.0,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 2664,+ "output_tokens": 0,+ "state_tokens": 856,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_dict.json b/laya-rust/tests/golden/multilingual/case_long_dict.jsonnew file mode 100644index 00000000..364bf12f--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_dict.json@@ -0,0 +1,129 @@+{+ "name": "long_dict",+ "state": {+ "notes": "Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. ",+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ },+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 96+ },+ "state_tokens": 157,+ "effective_window": 96,+ "effective_stride": 48,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 96,+ "text": " {\"notes\": \"Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. \", \"message\":"+ },+ {+ "token_start": 48,+ "token_end": 144,+ "text": " Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. \", \"message\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in."+ },+ {+ "token_start": 96,+ "token_end": 157,+ "text": " \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.\"}"+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.7892,+ "technical": 0.0078,+ "other": 0.2029+ },+ "confidence": 0.5008,+ "answer_confidence": 0.7892,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 1,+ "token_start": 48,+ "token_end": 144,+ "count": 3+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.2335,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0218,+ "1": 0.7229,+ "2": 0.2553+ },+ "confidence": 0.3933,+ "answer_confidence": 0.7229,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 157,+ "count": 3+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.0739,+ "confidence": 0.9261,+ "answer_confidence": 0.9261,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 157,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 1047,+ "output_tokens": 0,+ "state_tokens": 253,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 3+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_empty_questions.json b/laya-rust/tests/golden/multilingual/case_long_empty_questions.jsonnew file mode 100644index 00000000..4aa1945a--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_empty_questions.json@@ -0,0 +1,54 @@+{+ "name": "long_empty_questions",+ "state": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. ",+ "questions": {},+ "options": {+ "window": 32,+ "batch_size": 2+ },+ "state_tokens": 106,+ "effective_window": 32,+ "effective_stride": 16,+ "effective_room": 32,+ "windows": [+ {+ "token_start": 0,+ "token_end": 32,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks"+ },+ {+ "token_start": 16,+ "token_end": 48,+ "text": " checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 32,+ "token_end": 64,+ "text": ". Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine"+ },+ {+ "token_start": 48,+ "token_end": 80,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks"+ },+ {+ "token_start": 64,+ "token_end": 96,+ "text": " checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 80,+ "token_end": 106,+ "text": ". Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {},+ "usage": {+ "input_tokens": 0,+ "output_tokens": 0,+ "windows": 6+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_exact_limit.json b/laya-rust/tests/golden/multilingual/case_long_exact_limit.jsonnew file mode 100644index 00000000..b632c882--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_exact_limit.json@@ -0,0 +1,90 @@+{+ "name": "long_exact_limit",+ "state": "a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 760,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "technical",+ "probabilities": {+ "refund": 0.0576,+ "technical": 0.9159,+ "other": 0.0265+ },+ "confidence": 0.6895,+ "answer_confidence": 0.9159,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 0.9952,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.2231,+ "1": 0.5585,+ "2": 0.2183+ },+ "confidence": 0.0968,+ "answer_confidence": 0.5585,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.7666,+ "confidence": 0.7666,+ "answer_confidence": 0.7666,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 2376,+ "output_tokens": 0,+ "state_tokens": 760,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_invalid_aggregate.json b/laya-rust/tests/golden/multilingual/case_long_invalid_aggregate.jsonnew file mode 100644index 00000000..1b1a6496--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_invalid_aggregate.json@@ -0,0 +1,33 @@+{+ "name": "long_invalid_aggregate",+ "state": "short",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "aggregate": "mean"+ },+ "error_type": "ValueError",+ "error_message": "predict_long: only aggregate='auto' is supported"+}diff --git a/laya-rust/tests/golden/multilingual/case_long_invalid_stride.json b/laya-rust/tests/golden/multilingual/case_long_invalid_stride.jsonnew file mode 100644index 00000000..d710ad47--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_invalid_stride.json@@ -0,0 +1,34 @@+{+ "name": "long_invalid_stride",+ "state": "short",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 32,+ "stride": 33+ },+ "error_type": "ValueError",+ "error_message": "predict_long: stride=33 steps past the 32-token window, so 1 tokens between every pair of windows would be read by no window at all; pass stride <= 32"+}diff --git a/laya-rust/tests/golden/multilingual/case_long_list.json b/laya-rust/tests/golden/multilingual/case_long_list.jsonnew file mode 100644index 00000000..613541c0--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_list.json@@ -0,0 +1,146 @@+{+ "name": "long_list",+ "state": [+ {+ "role": "assistant",+ "content": "Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. "+ },+ {+ "role": "user",+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 96,+ "stride": 48+ },+ "state_tokens": 243,+ "effective_window": 96,+ "effective_stride": 48,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 96,+ "text": " [{\"role\": \"assistant\", \"content\": \"Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check"+ },+ {+ "token_start": 48,+ "token_end": 144,+ "text": " the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check"+ },+ {+ "token_start": 96,+ "token_end": 192,+ "text": " the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. \"}, {\"role\": \"user\", \"content\": \"Subject: Double charged on invoice 88"+ },+ {+ "token_start": 144,+ "token_end": 240,+ "text": " the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. \"}, {\"role\": \"user\", \"content\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the"+ },+ {+ "token_start": 192,+ "token_end": 243,+ "text": "213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.\"}]"+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8964,+ "technical": 0.0208,+ "other": 0.0828+ },+ "confidence": 0.6496,+ "answer_confidence": 0.8964,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 192,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.8103,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0112,+ "1": 0.1673,+ "2": 0.8215+ },+ "confidence": 0.5349,+ "answer_confidence": 0.8215,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 144,+ "token_end": 240,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.092,+ "confidence": 0.908,+ "answer_confidence": 0.908,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 192,+ "token_end": 243,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 1788,+ "output_tokens": 0,+ "state_tokens": 436,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_multi_chunk.json b/laya-rust/tests/golden/multilingual/case_long_multi_chunk.jsonnew file mode 100644index 00000000..854fbc0c--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_multi_chunk.json@@ -0,0 +1,136 @@+{+ "name": "long_multi_chunk",+ "state": "The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "batch_size": 2+ },+ "state_tokens": 2036,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 760,+ "text": " The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The"+ },+ {+ "token_start": 380,+ "token_end": 1140,+ "text": " recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded"+ },+ {+ "token_start": 760,+ "token_end": 1520,+ "text": " agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent"+ },+ {+ "token_start": 1140,+ "token_end": 1900,+ "text": " routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine"+ },+ {+ "token_start": 1520,+ "token_end": 2036,+ "text": " checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\n Hi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "other",+ "probabilities": {+ "refund": 0.0021,+ "technical": 0.0006,+ "other": 0.9973+ },+ "confidence": 0.9819,+ "answer_confidence": 0.9973,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 1140,+ "token_end": 1900,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.4626,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.1332,+ "1": 0.2709,+ "2": 0.5958+ },+ "confidence": 0.1527,+ "answer_confidence": 0.5958,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1520,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.2205,+ "confidence": 0.7795,+ "answer_confidence": 0.7795,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 1520,+ "token_end": 2036,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 11148,+ "output_tokens": 0,+ "state_tokens": 3556,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_no_state_room.json b/laya-rust/tests/golden/multilingual/case_long_no_state_room.jsonnew file mode 100644index 00000000..feba06c7--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_no_state_room.json@@ -0,0 +1,315 @@+{+ "name": "long_no_state_room",+ "state": "short",+ "questions": {+ "too_wide": {+ "type": "choice",+ "instructions": "Too many options to fit the sequence.",+ "criteria": {+ "opt_000": "description 0",+ "opt_001": "description 1",+ "opt_002": "description 2",+ "opt_003": "description 3",+ "opt_004": "description 4",+ "opt_005": "description 5",+ "opt_006": "description 6",+ "opt_007": "description 7",+ "opt_008": "description 8",+ "opt_009": "description 9",+ "opt_010": "description 10",+ "opt_011": "description 11",+ "opt_012": "description 12",+ "opt_013": "description 13",+ "opt_014": "description 14",+ "opt_015": "description 15",+ "opt_016": "description 16",+ "opt_017": "description 17",+ "opt_018": "description 18",+ "opt_019": "description 19",+ "opt_020": "description 20",+ "opt_021": "description 21",+ "opt_022": "description 22",+ "opt_023": "description 23",+ "opt_024": "description 24",+ "opt_025": "description 25",+ "opt_026": "description 26",+ "opt_027": "description 27",+ "opt_028": "description 28",+ "opt_029": "description 29",+ "opt_030": "description 30",+ "opt_031": "description 31",+ "opt_032": "description 32",+ "opt_033": "description 33",+ "opt_034": "description 34",+ "opt_035": "description 35",+ "opt_036": "description 36",+ "opt_037": "description 37",+ "opt_038": "description 38",+ "opt_039": "description 39",+ "opt_040": "description 40",+ "opt_041": "description 41",+ "opt_042": "description 42",+ "opt_043": "description 43",+ "opt_044": "description 44",+ "opt_045": "description 45",+ "opt_046": "description 46",+ "opt_047": "description 47",+ "opt_048": "description 48",+ "opt_049": "description 49",+ "opt_050": "description 50",+ "opt_051": "description 51",+ "opt_052": "description 52",+ "opt_053": "description 53",+ "opt_054": "description 54",+ "opt_055": "description 55",+ "opt_056": "description 56",+ "opt_057": "description 57",+ "opt_058": "description 58",+ "opt_059": "description 59",+ "opt_060": "description 60",+ "opt_061": "description 61",+ "opt_062": "description 62",+ "opt_063": "description 63",+ "opt_064": "description 64",+ "opt_065": "description 65",+ "opt_066": "description 66",+ "opt_067": "description 67",+ "opt_068": "description 68",+ "opt_069": "description 69",+ "opt_070": "description 70",+ "opt_071": "description 71",+ "opt_072": "description 72",+ "opt_073": "description 73",+ "opt_074": "description 74",+ "opt_075": "description 75",+ "opt_076": "description 76",+ "opt_077": "description 77",+ "opt_078": "description 78",+ "opt_079": "description 79",+ "opt_080": "description 80",+ "opt_081": "description 81",+ "opt_082": "description 82",+ "opt_083": "description 83",+ "opt_084": "description 84",+ "opt_085": "description 85",+ "opt_086": "description 86",+ "opt_087": "description 87",+ "opt_088": "description 88",+ "opt_089": "description 89",+ "opt_090": "description 90",+ "opt_091": "description 91",+ "opt_092": "description 92",+ "opt_093": "description 93",+ "opt_094": "description 94",+ "opt_095": "description 95",+ "opt_096": "description 96",+ "opt_097": "description 97",+ "opt_098": "description 98",+ "opt_099": "description 99",+ "opt_100": "description 100",+ "opt_101": "description 101",+ "opt_102": "description 102",+ "opt_103": "description 103",+ "opt_104": "description 104",+ "opt_105": "description 105",+ "opt_106": "description 106",+ "opt_107": "description 107",+ "opt_108": "description 108",+ "opt_109": "description 109",+ "opt_110": "description 110",+ "opt_111": "description 111",+ "opt_112": "description 112",+ "opt_113": "description 113",+ "opt_114": "description 114",+ "opt_115": "description 115",+ "opt_116": "description 116",+ "opt_117": "description 117",+ "opt_118": "description 118",+ "opt_119": "description 119",+ "opt_120": "description 120",+ "opt_121": "description 121",+ "opt_122": "description 122",+ "opt_123": "description 123",+ "opt_124": "description 124",+ "opt_125": "description 125",+ "opt_126": "description 126",+ "opt_127": "description 127",+ "opt_128": "description 128",+ "opt_129": "description 129",+ "opt_130": "description 130",+ "opt_131": "description 131",+ "opt_132": "description 132",+ "opt_133": "description 133",+ "opt_134": "description 134",+ "opt_135": "description 135",+ "opt_136": "description 136",+ "opt_137": "description 137",+ "opt_138": "description 138",+ "opt_139": "description 139",+ "opt_140": "description 140",+ "opt_141": "description 141",+ "opt_142": "description 142",+ "opt_143": "description 143",+ "opt_144": "description 144",+ "opt_145": "description 145",+ "opt_146": "description 146",+ "opt_147": "description 147",+ "opt_148": "description 148",+ "opt_149": "description 149",+ "opt_150": "description 150",+ "opt_151": "description 151",+ "opt_152": "description 152",+ "opt_153": "description 153",+ "opt_154": "description 154",+ "opt_155": "description 155",+ "opt_156": "description 156",+ "opt_157": "description 157",+ "opt_158": "description 158",+ "opt_159": "description 159",+ "opt_160": "description 160",+ "opt_161": "description 161",+ "opt_162": "description 162",+ "opt_163": "description 163",+ "opt_164": "description 164",+ "opt_165": "description 165",+ "opt_166": "description 166",+ "opt_167": "description 167",+ "opt_168": "description 168",+ "opt_169": "description 169",+ "opt_170": "description 170",+ "opt_171": "description 171",+ "opt_172": "description 172",+ "opt_173": "description 173",+ "opt_174": "description 174",+ "opt_175": "description 175",+ "opt_176": "description 176",+ "opt_177": "description 177",+ "opt_178": "description 178",+ "opt_179": "description 179",+ "opt_180": "description 180",+ "opt_181": "description 181",+ "opt_182": "description 182",+ "opt_183": "description 183",+ "opt_184": "description 184",+ "opt_185": "description 185",+ "opt_186": "description 186",+ "opt_187": "description 187",+ "opt_188": "description 188",+ "opt_189": "description 189",+ "opt_190": "description 190",+ "opt_191": "description 191",+ "opt_192": "description 192",+ "opt_193": "description 193",+ "opt_194": "description 194",+ "opt_195": "description 195",+ "opt_196": "description 196",+ "opt_197": "description 197",+ "opt_198": "description 198",+ "opt_199": "description 199",+ "opt_200": "description 200",+ "opt_201": "description 201",+ "opt_202": "description 202",+ "opt_203": "description 203",+ "opt_204": "description 204",+ "opt_205": "description 205",+ "opt_206": "description 206",+ "opt_207": "description 207",+ "opt_208": "description 208",+ "opt_209": "description 209",+ "opt_210": "description 210",+ "opt_211": "description 211",+ "opt_212": "description 212",+ "opt_213": "description 213",+ "opt_214": "description 214",+ "opt_215": "description 215",+ "opt_216": "description 216",+ "opt_217": "description 217",+ "opt_218": "description 218",+ "opt_219": "description 219",+ "opt_220": "description 220",+ "opt_221": "description 221",+ "opt_222": "description 222",+ "opt_223": "description 223",+ "opt_224": "description 224",+ "opt_225": "description 225",+ "opt_226": "description 226",+ "opt_227": "description 227",+ "opt_228": "description 228",+ "opt_229": "description 229",+ "opt_230": "description 230",+ "opt_231": "description 231",+ "opt_232": "description 232",+ "opt_233": "description 233",+ "opt_234": "description 234",+ "opt_235": "description 235",+ "opt_236": "description 236",+ "opt_237": "description 237",+ "opt_238": "description 238",+ "opt_239": "description 239",+ "opt_240": "description 240",+ "opt_241": "description 241",+ "opt_242": "description 242",+ "opt_243": "description 243",+ "opt_244": "description 244",+ "opt_245": "description 245",+ "opt_246": "description 246",+ "opt_247": "description 247",+ "opt_248": "description 248",+ "opt_249": "description 249",+ "opt_250": "description 250",+ "opt_251": "description 251",+ "opt_252": "description 252",+ "opt_253": "description 253",+ "opt_254": "description 254",+ "opt_255": "description 255",+ "opt_256": "description 256",+ "opt_257": "description 257",+ "opt_258": "description 258",+ "opt_259": "description 259",+ "opt_260": "description 260",+ "opt_261": "description 261",+ "opt_262": "description 262",+ "opt_263": "description 263",+ "opt_264": "description 264",+ "opt_265": "description 265",+ "opt_266": "description 266",+ "opt_267": "description 267",+ "opt_268": "description 268",+ "opt_269": "description 269",+ "opt_270": "description 270",+ "opt_271": "description 271",+ "opt_272": "description 272",+ "opt_273": "description 273",+ "opt_274": "description 274",+ "opt_275": "description 275",+ "opt_276": "description 276",+ "opt_277": "description 277",+ "opt_278": "description 278",+ "opt_279": "description 279",+ "opt_280": "description 280",+ "opt_281": "description 281",+ "opt_282": "description 282",+ "opt_283": "description 283",+ "opt_284": "description 284",+ "opt_285": "description 285",+ "opt_286": "description 286",+ "opt_287": "description 287",+ "opt_288": "description 288",+ "opt_289": "description 289",+ "opt_290": "description 290",+ "opt_291": "description 291",+ "opt_292": "description 292",+ "opt_293": "description 293",+ "opt_294": "description 294",+ "opt_295": "description 295",+ "opt_296": "description 296",+ "opt_297": "description 297",+ "opt_298": "description 298",+ "opt_299": "description 299"+ }+ }+ },+ "options": {},+ "error_type": "ValueError",+ "error_message": "predict_long: the questions' options fill the whole sequence (max_len=1024, head_max_len=256), leaving no room for the state; no window can carry any of it. A label set this large is what laya.shortlist.predict_shortlist is for"+}diff --git a/laya-rust/tests/golden/multilingual/case_long_punctuation.json b/laya-rust/tests/golden/multilingual/case_long_punctuation.jsonnew file mode 100644index 00000000..2be95e0e--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_punctuation.json@@ -0,0 +1,138 @@+{+ "name": "long_punctuation",+ "state": "I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I want a refund ! I 'll cancel today .",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 64,+ "stride": 64,+ "batch_size": 3+ },+ "state_tokens": 317,+ "effective_window": 64,+ "effective_stride": 64,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 64,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n"+ },+ {+ "token_start": 64,+ "token_end": 128,+ "text": "'t we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen"+ },+ {+ "token_start": 128,+ "token_end": 192,+ "text": " this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking ."+ },+ {+ "token_start": 192,+ "token_end": 256,+ "text": " We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I"+ },+ {+ "token_start": 256,+ "token_end": 317,+ "text": " 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I want a refund ! I 'll cancel today ."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9993,+ "technical": 0.0004,+ "other": 0.0003+ },+ "confidence": 0.994,+ "answer_confidence": 0.9993,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 317,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.8986,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0148,+ "1": 0.0717,+ "2": 0.9135+ },+ "confidence": 0.6959,+ "answer_confidence": 0.9135,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 317,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.3804,+ "confidence": 0.6196,+ "answer_confidence": 0.6196,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 317,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 1431,+ "output_tokens": 0,+ "state_tokens": 317,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_room_fast_path.json b/laya-rust/tests/golden/multilingual/case_long_room_fast_path.jsonnew file mode 100644index 00000000..ce70fb12--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_room_fast_path.json@@ -0,0 +1,90 @@+{+ "name": "long_room_fast_path",+ "state": "a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 990,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "technical",+ "probabilities": {+ "refund": 0.0377,+ "technical": 0.9444,+ "other": 0.018+ },+ "confidence": 0.7726,+ "answer_confidence": 0.9444,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.0401,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.2246,+ "1": 0.5108,+ "2": 0.2646+ },+ "confidence": 0.0621,+ "answer_confidence": 0.5108,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.8864,+ "confidence": 0.8864,+ "answer_confidence": 0.8864,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 3066,+ "output_tokens": 0,+ "state_tokens": 990,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_short.json b/laya-rust/tests/golden/multilingual/case_long_short.jsonnew file mode 100644index 00000000..d987e17a--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_short.json@@ -0,0 +1,90 @@+{+ "name": "long_short",+ "state": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 56,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.836,+ "technical": 0.0114,+ "other": 0.1526+ },+ "confidence": 0.5562,+ "answer_confidence": 0.836,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.3713,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0231,+ "1": 0.5824,+ "2": 0.3945+ },+ "confidence": 0.3001,+ "answer_confidence": 0.5824,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.0414,+ "confidence": 0.9586,+ "answer_confidence": 0.9586,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 264,+ "output_tokens": 0,+ "state_tokens": 56,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/case_long_special_tokens.json b/laya-rust/tests/golden/multilingual/case_long_special_tokens.jsonnew file mode 100644index 00000000..2bfb8d5a--- /dev/null+++ b/laya-rust/tests/golden/multilingual/case_long_special_tokens.json@@ -0,0 +1,162 @@+{+ "name": "long_special_tokens",+ "state": "The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. ",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 64,+ "batch_size": 2+ },+ "state_tokens": 346,+ "effective_window": 64,+ "effective_stride": 32,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 64,+ "text": " The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] "+ },+ {+ "token_start": 32,+ "token_end": 96,+ "text": " <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes ["+ },+ {+ "token_start": 64,+ "token_end": 128,+ "text": " and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos>"+ },+ {+ "token_start": 96,+ "token_end": 160,+ "text": "CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀"+ },+ {+ "token_start": 128,+ "token_end": 192,+ "text": " [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP"+ },+ {+ "token_start": 160,+ "token_end": 224,+ "text": ". The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] "+ },+ {+ "token_start": 192,+ "token_end": 256,+ "text": "] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes"+ },+ {+ "token_start": 224,+ "token_end": 288,+ "text": " and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> "+ },+ {+ "token_start": 256,+ "token_end": 320,+ "text": " [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀"+ },+ {+ "token_start": 288,+ "token_end": 346,+ "text": "<eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] and emojis 😀 🚀. "+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "other",+ "probabilities": {+ "refund": 0.0523,+ "technical": 0.2418,+ "other": 0.7059+ },+ "confidence": 0.3232,+ "answer_confidence": 0.7059,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 6,+ "token_start": 192,+ "token_end": 256,+ "count": 10+ }+ },+ "urgency": {+ "type": "score",+ "score": 0.932,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.1737,+ "1": 0.7206,+ "2": 0.1057+ },+ "confidence": 0.2921,+ "answer_confidence": 0.7206,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 96,+ "token_end": 160,+ "count": 10+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.0034,+ "confidence": 0.9966,+ "answer_confidence": 0.9966,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 5,+ "token_start": 160,+ "token_end": 224,+ "count": 10+ }+ }+ },+ "usage": {+ "input_tokens": 2862,+ "output_tokens": 0,+ "state_tokens": 634,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 10+ }+ }+}diff --git a/laya-rust/tests/golden/multilingual/index.json b/laya-rust/tests/golden/multilingual/index.jsonindex 27fdf54e..5c2bf1ce 100644--- a/laya-rust/tests/golden/multilingual/index.json+++ b/laya-rust/tests/golden/multilingual/index.json@@ -84,5 +84,83 @@"file": "case_shortlist_many_options.json","expects_error": false,"kind": "shortlist"+ },+ {+ "name": "long_short",+ "file": "case_long_short.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_exact_limit",+ "file": "case_long_exact_limit.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_room_fast_path",+ "file": "case_long_room_fast_path.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_multi_chunk",+ "file": "case_long_multi_chunk.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_punctuation",+ "file": "case_long_punctuation.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_dict",+ "file": "case_long_dict.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_list",+ "file": "case_long_list.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_clamp",+ "file": "case_long_clamp.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_special_tokens",+ "file": "case_long_special_tokens.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_empty_questions",+ "file": "case_long_empty_questions.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_invalid_stride",+ "file": "case_long_invalid_stride.json",+ "expects_error": true,+ "kind": "long"+ },+ {+ "name": "long_invalid_aggregate",+ "file": "case_long_invalid_aggregate.json",+ "expects_error": true,+ "kind": "long"+ },+ {+ "name": "long_no_state_room",+ "file": "case_long_no_state_room.json",+ "expects_error": true,+ "kind": "long"}]diff --git a/laya-rust/tests/golden/multilingual/meta.json b/laya-rust/tests/golden/multilingual/meta.jsonindex e57fdd98..c8da4e70 100644--- a/laya-rust/tests/golden/multilingual/meta.json+++ b/laya-rust/tests/golden/multilingual/meta.json@@ -1,7 +1,7 @@{"generator": "tools/dump_golden.py","onnx_dir": "multilingual",- "laya_version": "0.4.0",+ "laya_version": "0.4.11","onnxruntime_version": "1.30.0","transformers_version": "5.17.0","config": {@@ -56,13 +56,13 @@},"onnx_self_check": {"single_noul": {- "logits_maxdiff": 1.5795230865478516e-06,- "act_maxdiff": 0.0001220703125,+ "logits_maxdiff": 3.337860107421875e-06,+ "act_maxdiff": 0.000244140625,"act_probs_maxdiff": 0.0},"triage_4q": {"logits_maxdiff": 1.3232231140136719e-05,- "act_maxdiff": 0.0009765625,+ "act_maxdiff": 0.0010986328125,"act_probs_maxdiff": 0.0}}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_clamp.json b/laya-rust/tests/golden/typed-decisions/case_long_clamp.jsonnew file mode 100644index 00000000..8b060c62--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_clamp.json@@ -0,0 +1,94 @@+{+ "name": "long_clamp",+ "state": "Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Normal account activity. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 4096,+ "stride": 2048,+ "batch_size": 2+ },+ "state_tokens": 857,+ "effective_window": 990,+ "effective_stride": 495,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8479,+ "technical": 0.0837,+ "other": 0.0685+ },+ "confidence": 0.5166,+ "answer_confidence": 0.8479,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.6689,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0372,+ "1": 0.2567,+ "2": 0.7061+ },+ "confidence": 0.3471,+ "answer_confidence": 0.7061,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.8296,+ "confidence": 0.8296,+ "answer_confidence": 0.8296,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 2665,+ "output_tokens": 0,+ "state_tokens": 857,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_dict.json b/laya-rust/tests/golden/typed-decisions/case_long_dict.jsonnew file mode 100644index 00000000..012b9e5a--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_dict.json@@ -0,0 +1,129 @@+{+ "name": "long_dict",+ "state": {+ "notes": "Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. ",+ "message": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ },+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 96+ },+ "state_tokens": 179,+ "effective_window": 96,+ "effective_stride": 48,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 96,+ "text": "{\"notes\": \"Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine"+ },+ {+ "token_start": 48,+ "token_end": 144,+ "text": ". Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. Routine checks completed. \", \"message\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan"+ },+ {+ "token_start": 96,+ "token_end": 179,+ "text": " checks completed. Routine checks completed. Routine checks completed. Routine checks completed. \", \"message\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.\"}"+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8143,+ "technical": 0.1062,+ "other": 0.0794+ },+ "confidence": 0.4478,+ "answer_confidence": 0.8143,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 179,+ "count": 3+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.6601,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.029,+ "1": 0.2819,+ "2": 0.6891+ },+ "confidence": 0.348,+ "answer_confidence": 0.6891,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 179,+ "count": 3+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.801,+ "confidence": 0.801,+ "answer_confidence": 0.801,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 2,+ "token_start": 96,+ "token_end": 179,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 1107,+ "output_tokens": 0,+ "state_tokens": 275,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 3+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_empty_questions.json b/laya-rust/tests/golden/typed-decisions/case_long_empty_questions.jsonnew file mode 100644index 00000000..34e42d25--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_empty_questions.json@@ -0,0 +1,64 @@+{+ "name": "long_empty_questions",+ "state": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. ",+ "questions": {},+ "options": {+ "window": 32,+ "batch_size": 2+ },+ "state_tokens": 141,+ "effective_window": 32,+ "effective_stride": 16,+ "effective_room": 32,+ "windows": [+ {+ "token_start": 0,+ "token_end": 32,+ "text": "Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 16,+ "token_end": 48,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 32,+ "token_end": 64,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 48,+ "token_end": 80,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 64,+ "token_end": 96,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 80,+ "token_end": 112,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 96,+ "token_end": 128,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks."+ },+ {+ "token_start": 112,+ "token_end": 141,+ "text": " Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. Routine checks. "+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {},+ "usage": {+ "input_tokens": 0,+ "output_tokens": 0,+ "windows": 8+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_exact_limit.json b/laya-rust/tests/golden/typed-decisions/case_long_exact_limit.jsonnew file mode 100644index 00000000..3583fcb8--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_exact_limit.json@@ -0,0 +1,90 @@+{+ "name": "long_exact_limit",+ "state": "a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 760,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "other",+ "probabilities": {+ "refund": 0.2296,+ "technical": 0.3781,+ "other": 0.3923+ },+ "confidence": 0.0236,+ "answer_confidence": 0.3923,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.1188,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.2602,+ "1": 0.3608,+ "2": 0.379+ },+ "confidence": 0.0116,+ "answer_confidence": 0.379,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.2368,+ "confidence": 0.7632,+ "answer_confidence": 0.7632,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 2374,+ "output_tokens": 0,+ "state_tokens": 760,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_invalid_aggregate.json b/laya-rust/tests/golden/typed-decisions/case_long_invalid_aggregate.jsonnew file mode 100644index 00000000..1b1a6496--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_invalid_aggregate.json@@ -0,0 +1,33 @@+{+ "name": "long_invalid_aggregate",+ "state": "short",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "aggregate": "mean"+ },+ "error_type": "ValueError",+ "error_message": "predict_long: only aggregate='auto' is supported"+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_invalid_stride.json b/laya-rust/tests/golden/typed-decisions/case_long_invalid_stride.jsonnew file mode 100644index 00000000..d710ad47--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_invalid_stride.json@@ -0,0 +1,34 @@+{+ "name": "long_invalid_stride",+ "state": "short",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 32,+ "stride": 33+ },+ "error_type": "ValueError",+ "error_message": "predict_long: stride=33 steps past the 32-token window, so 1 tokens between every pair of windows would be read by no window at all; pass stride <= 32"+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_list.json b/laya-rust/tests/golden/typed-decisions/case_long_list.jsonnew file mode 100644index 00000000..acbb6566--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_list.json@@ -0,0 +1,146 @@+{+ "name": "long_list",+ "state": [+ {+ "role": "assistant",+ "content": "Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. "+ },+ {+ "role": "user",+ "content": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 96,+ "stride": 48+ },+ "state_tokens": 244,+ "effective_window": 96,+ "effective_stride": 48,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 96,+ "text": "[{\"role\": \"assistant\", \"content\": \"Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we"+ },+ {+ "token_start": 48,+ "token_end": 144,+ "text": " check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we"+ },+ {+ "token_start": 96,+ "token_end": 192,+ "text": " check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. \"}, {\"role\": \"user\", \"content\": \"Subject: Double charged on invoice 882"+ },+ {+ "token_start": 144,+ "token_end": 240,+ "text": " check the account. Please wait while we check the account. Please wait while we check the account. Please wait while we check the account. \"}, {\"role\": \"user\", \"content\": \"Subject: Double charged on invoice 88213\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling"+ },+ {+ "token_start": 192,+ "token_end": 244,+ "text": "13\\n\\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.\"}]"+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8269,+ "technical": 0.0938,+ "other": 0.0793+ },+ "confidence": 0.4719,+ "answer_confidence": 0.8269,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 144,+ "token_end": 240,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.7764,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0266,+ "1": 0.1705,+ "2": 0.803+ },+ "confidence": 0.4773,+ "answer_confidence": 0.803,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 192,+ "token_end": 244,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.8002,+ "confidence": 0.8002,+ "answer_confidence": 0.8002,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 192,+ "token_end": 244,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 1778,+ "output_tokens": 0,+ "state_tokens": 436,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_multi_chunk.json b/laya-rust/tests/golden/typed-decisions/case_long_multi_chunk.jsonnew file mode 100644index 00000000..4eaa2bc1--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_multi_chunk.json@@ -0,0 +1,136 @@+{+ "name": "long_multi_chunk",+ "state": "The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "batch_size": 2+ },+ "state_tokens": 2037,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 760,+ "text": "The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The"+ },+ {+ "token_start": 380,+ "token_end": 1140,+ "text": " recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded"+ },+ {+ "token_start": 760,+ "token_end": 1520,+ "text": " agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent"+ },+ {+ "token_start": 1140,+ "token_end": 1900,+ "text": " routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine"+ },+ {+ "token_start": 1520,+ "token_end": 2037,+ "text": " checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. The agent checked the account and recorded routine status updates. Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8698,+ "technical": 0.0699,+ "other": 0.0603+ },+ "confidence": 0.5662,+ "answer_confidence": 0.8698,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 1520,+ "token_end": 2037,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.7495,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0314,+ "1": 0.1876,+ "2": 0.7809+ },+ "confidence": 0.4395,+ "answer_confidence": 0.7809,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 1520,+ "token_end": 2037,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.7882,+ "confidence": 0.7882,+ "answer_confidence": 0.7882,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 1520,+ "token_end": 2037,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 11141,+ "output_tokens": 0,+ "state_tokens": 3557,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_no_state_room.json b/laya-rust/tests/golden/typed-decisions/case_long_no_state_room.jsonnew file mode 100644index 00000000..feba06c7--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_no_state_room.json@@ -0,0 +1,315 @@+{+ "name": "long_no_state_room",+ "state": "short",+ "questions": {+ "too_wide": {+ "type": "choice",+ "instructions": "Too many options to fit the sequence.",+ "criteria": {+ "opt_000": "description 0",+ "opt_001": "description 1",+ "opt_002": "description 2",+ "opt_003": "description 3",+ "opt_004": "description 4",+ "opt_005": "description 5",+ "opt_006": "description 6",+ "opt_007": "description 7",+ "opt_008": "description 8",+ "opt_009": "description 9",+ "opt_010": "description 10",+ "opt_011": "description 11",+ "opt_012": "description 12",+ "opt_013": "description 13",+ "opt_014": "description 14",+ "opt_015": "description 15",+ "opt_016": "description 16",+ "opt_017": "description 17",+ "opt_018": "description 18",+ "opt_019": "description 19",+ "opt_020": "description 20",+ "opt_021": "description 21",+ "opt_022": "description 22",+ "opt_023": "description 23",+ "opt_024": "description 24",+ "opt_025": "description 25",+ "opt_026": "description 26",+ "opt_027": "description 27",+ "opt_028": "description 28",+ "opt_029": "description 29",+ "opt_030": "description 30",+ "opt_031": "description 31",+ "opt_032": "description 32",+ "opt_033": "description 33",+ "opt_034": "description 34",+ "opt_035": "description 35",+ "opt_036": "description 36",+ "opt_037": "description 37",+ "opt_038": "description 38",+ "opt_039": "description 39",+ "opt_040": "description 40",+ "opt_041": "description 41",+ "opt_042": "description 42",+ "opt_043": "description 43",+ "opt_044": "description 44",+ "opt_045": "description 45",+ "opt_046": "description 46",+ "opt_047": "description 47",+ "opt_048": "description 48",+ "opt_049": "description 49",+ "opt_050": "description 50",+ "opt_051": "description 51",+ "opt_052": "description 52",+ "opt_053": "description 53",+ "opt_054": "description 54",+ "opt_055": "description 55",+ "opt_056": "description 56",+ "opt_057": "description 57",+ "opt_058": "description 58",+ "opt_059": "description 59",+ "opt_060": "description 60",+ "opt_061": "description 61",+ "opt_062": "description 62",+ "opt_063": "description 63",+ "opt_064": "description 64",+ "opt_065": "description 65",+ "opt_066": "description 66",+ "opt_067": "description 67",+ "opt_068": "description 68",+ "opt_069": "description 69",+ "opt_070": "description 70",+ "opt_071": "description 71",+ "opt_072": "description 72",+ "opt_073": "description 73",+ "opt_074": "description 74",+ "opt_075": "description 75",+ "opt_076": "description 76",+ "opt_077": "description 77",+ "opt_078": "description 78",+ "opt_079": "description 79",+ "opt_080": "description 80",+ "opt_081": "description 81",+ "opt_082": "description 82",+ "opt_083": "description 83",+ "opt_084": "description 84",+ "opt_085": "description 85",+ "opt_086": "description 86",+ "opt_087": "description 87",+ "opt_088": "description 88",+ "opt_089": "description 89",+ "opt_090": "description 90",+ "opt_091": "description 91",+ "opt_092": "description 92",+ "opt_093": "description 93",+ "opt_094": "description 94",+ "opt_095": "description 95",+ "opt_096": "description 96",+ "opt_097": "description 97",+ "opt_098": "description 98",+ "opt_099": "description 99",+ "opt_100": "description 100",+ "opt_101": "description 101",+ "opt_102": "description 102",+ "opt_103": "description 103",+ "opt_104": "description 104",+ "opt_105": "description 105",+ "opt_106": "description 106",+ "opt_107": "description 107",+ "opt_108": "description 108",+ "opt_109": "description 109",+ "opt_110": "description 110",+ "opt_111": "description 111",+ "opt_112": "description 112",+ "opt_113": "description 113",+ "opt_114": "description 114",+ "opt_115": "description 115",+ "opt_116": "description 116",+ "opt_117": "description 117",+ "opt_118": "description 118",+ "opt_119": "description 119",+ "opt_120": "description 120",+ "opt_121": "description 121",+ "opt_122": "description 122",+ "opt_123": "description 123",+ "opt_124": "description 124",+ "opt_125": "description 125",+ "opt_126": "description 126",+ "opt_127": "description 127",+ "opt_128": "description 128",+ "opt_129": "description 129",+ "opt_130": "description 130",+ "opt_131": "description 131",+ "opt_132": "description 132",+ "opt_133": "description 133",+ "opt_134": "description 134",+ "opt_135": "description 135",+ "opt_136": "description 136",+ "opt_137": "description 137",+ "opt_138": "description 138",+ "opt_139": "description 139",+ "opt_140": "description 140",+ "opt_141": "description 141",+ "opt_142": "description 142",+ "opt_143": "description 143",+ "opt_144": "description 144",+ "opt_145": "description 145",+ "opt_146": "description 146",+ "opt_147": "description 147",+ "opt_148": "description 148",+ "opt_149": "description 149",+ "opt_150": "description 150",+ "opt_151": "description 151",+ "opt_152": "description 152",+ "opt_153": "description 153",+ "opt_154": "description 154",+ "opt_155": "description 155",+ "opt_156": "description 156",+ "opt_157": "description 157",+ "opt_158": "description 158",+ "opt_159": "description 159",+ "opt_160": "description 160",+ "opt_161": "description 161",+ "opt_162": "description 162",+ "opt_163": "description 163",+ "opt_164": "description 164",+ "opt_165": "description 165",+ "opt_166": "description 166",+ "opt_167": "description 167",+ "opt_168": "description 168",+ "opt_169": "description 169",+ "opt_170": "description 170",+ "opt_171": "description 171",+ "opt_172": "description 172",+ "opt_173": "description 173",+ "opt_174": "description 174",+ "opt_175": "description 175",+ "opt_176": "description 176",+ "opt_177": "description 177",+ "opt_178": "description 178",+ "opt_179": "description 179",+ "opt_180": "description 180",+ "opt_181": "description 181",+ "opt_182": "description 182",+ "opt_183": "description 183",+ "opt_184": "description 184",+ "opt_185": "description 185",+ "opt_186": "description 186",+ "opt_187": "description 187",+ "opt_188": "description 188",+ "opt_189": "description 189",+ "opt_190": "description 190",+ "opt_191": "description 191",+ "opt_192": "description 192",+ "opt_193": "description 193",+ "opt_194": "description 194",+ "opt_195": "description 195",+ "opt_196": "description 196",+ "opt_197": "description 197",+ "opt_198": "description 198",+ "opt_199": "description 199",+ "opt_200": "description 200",+ "opt_201": "description 201",+ "opt_202": "description 202",+ "opt_203": "description 203",+ "opt_204": "description 204",+ "opt_205": "description 205",+ "opt_206": "description 206",+ "opt_207": "description 207",+ "opt_208": "description 208",+ "opt_209": "description 209",+ "opt_210": "description 210",+ "opt_211": "description 211",+ "opt_212": "description 212",+ "opt_213": "description 213",+ "opt_214": "description 214",+ "opt_215": "description 215",+ "opt_216": "description 216",+ "opt_217": "description 217",+ "opt_218": "description 218",+ "opt_219": "description 219",+ "opt_220": "description 220",+ "opt_221": "description 221",+ "opt_222": "description 222",+ "opt_223": "description 223",+ "opt_224": "description 224",+ "opt_225": "description 225",+ "opt_226": "description 226",+ "opt_227": "description 227",+ "opt_228": "description 228",+ "opt_229": "description 229",+ "opt_230": "description 230",+ "opt_231": "description 231",+ "opt_232": "description 232",+ "opt_233": "description 233",+ "opt_234": "description 234",+ "opt_235": "description 235",+ "opt_236": "description 236",+ "opt_237": "description 237",+ "opt_238": "description 238",+ "opt_239": "description 239",+ "opt_240": "description 240",+ "opt_241": "description 241",+ "opt_242": "description 242",+ "opt_243": "description 243",+ "opt_244": "description 244",+ "opt_245": "description 245",+ "opt_246": "description 246",+ "opt_247": "description 247",+ "opt_248": "description 248",+ "opt_249": "description 249",+ "opt_250": "description 250",+ "opt_251": "description 251",+ "opt_252": "description 252",+ "opt_253": "description 253",+ "opt_254": "description 254",+ "opt_255": "description 255",+ "opt_256": "description 256",+ "opt_257": "description 257",+ "opt_258": "description 258",+ "opt_259": "description 259",+ "opt_260": "description 260",+ "opt_261": "description 261",+ "opt_262": "description 262",+ "opt_263": "description 263",+ "opt_264": "description 264",+ "opt_265": "description 265",+ "opt_266": "description 266",+ "opt_267": "description 267",+ "opt_268": "description 268",+ "opt_269": "description 269",+ "opt_270": "description 270",+ "opt_271": "description 271",+ "opt_272": "description 272",+ "opt_273": "description 273",+ "opt_274": "description 274",+ "opt_275": "description 275",+ "opt_276": "description 276",+ "opt_277": "description 277",+ "opt_278": "description 278",+ "opt_279": "description 279",+ "opt_280": "description 280",+ "opt_281": "description 281",+ "opt_282": "description 282",+ "opt_283": "description 283",+ "opt_284": "description 284",+ "opt_285": "description 285",+ "opt_286": "description 286",+ "opt_287": "description 287",+ "opt_288": "description 288",+ "opt_289": "description 289",+ "opt_290": "description 290",+ "opt_291": "description 291",+ "opt_292": "description 292",+ "opt_293": "description 293",+ "opt_294": "description 294",+ "opt_295": "description 295",+ "opt_296": "description 296",+ "opt_297": "description 297",+ "opt_298": "description 298",+ "opt_299": "description 299"+ }+ }+ },+ "options": {},+ "error_type": "ValueError",+ "error_message": "predict_long: the questions' options fill the whole sequence (max_len=1024, head_max_len=256), leaving no room for the state; no window can carry any of it. A label set this large is what laya.shortlist.predict_shortlist is for"+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_punctuation.json b/laya-rust/tests/golden/typed-decisions/case_long_punctuation.jsonnew file mode 100644index 00000000..3bd38b9a--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_punctuation.json@@ -0,0 +1,138 @@+{+ "name": "long_punctuation",+ "state": "I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I want a refund ! I 'll cancel today .",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 64,+ "stride": 64,+ "batch_size": 3+ },+ "state_tokens": 299,+ "effective_window": 64,+ "effective_stride": 64,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 64,+ "text": "I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 64,+ "token_end": 128,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 128,+ "token_end": 192,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 192,+ "token_end": 256,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ?"+ },+ {+ "token_start": 256,+ "token_end": 299,+ "text": " I 'm checking . We 've seen this , have n't we ? I 'm checking . We 've seen this , have n't we ? I want a refund ! I 'll cancel today ."+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.817,+ "technical": 0.082,+ "other": 0.101+ },+ "confidence": 0.4523,+ "answer_confidence": 0.817,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 299,+ "count": 5+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.7239,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0515,+ "1": 0.173,+ "2": 0.7755+ },+ "confidence": 0.4051,+ "answer_confidence": 0.7755,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 299,+ "count": 5+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.785,+ "confidence": 0.785,+ "answer_confidence": 0.785,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 4,+ "token_start": 256,+ "token_end": 299,+ "count": 5+ }+ }+ },+ "usage": {+ "input_tokens": 1367,+ "output_tokens": 0,+ "state_tokens": 299,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 5+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_room_fast_path.json b/laya-rust/tests/golden/typed-decisions/case_long_room_fast_path.jsonnew file mode 100644index 00000000..284268f2--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_room_fast_path.json@@ -0,0 +1,90 @@+{+ "name": "long_room_fast_path",+ "state": "a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 990,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "technical",+ "probabilities": {+ "refund": 0.2548,+ "technical": 0.3772,+ "other": 0.368+ },+ "confidence": 0.0133,+ "answer_confidence": 0.3772,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.0802,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.2808,+ "1": 0.3582,+ "2": 0.361+ },+ "confidence": 0.0058,+ "answer_confidence": 0.361,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.2487,+ "confidence": 0.7513,+ "answer_confidence": 0.7513,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 3064,+ "output_tokens": 0,+ "state_tokens": 990,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_short.json b/laya-rust/tests/golden/typed-decisions/case_long_short.jsonnew file mode 100644index 00000000..da0ce3a8--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_short.json@@ -0,0 +1,90 @@+{+ "name": "long_short",+ "state": "Subject: Double charged on invoice 88213\n\nHi, I was billed twice this month for the Pro plan and the second charge has not been refunded. This is the third time I have written in. If it is not fixed today I am cancelling the account.",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {},+ "state_tokens": 57,+ "effective_window": 760,+ "effective_stride": 380,+ "effective_room": 990,+ "windows": [],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.7054,+ "technical": 0.1834,+ "other": 0.1111+ },+ "confidence": 0.2705,+ "answer_confidence": 0.7054,+ "action": {+ "act_probability": 1.0+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.8371,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.0193,+ "1": 0.1243,+ "2": 0.8564+ },+ "confidence": 0.574,+ "answer_confidence": 0.8564,+ "action": {+ "act_probability": 1.0+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.7914,+ "confidence": 0.7914,+ "answer_confidence": 0.7914,+ "action": {+ "act_probability": 1.0+ }+ }+ },+ "usage": {+ "input_tokens": 265,+ "output_tokens": 0,+ "state_tokens": 57,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/case_long_special_tokens.json b/laya-rust/tests/golden/typed-decisions/case_long_special_tokens.jsonnew file mode 100644index 00000000..e4c27745--- /dev/null+++ b/laya-rust/tests/golden/typed-decisions/case_long_special_tokens.json@@ -0,0 +1,177 @@+{+ "name": "long_special_tokens",+ "state": "The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. ",+ "questions": {+ "topic": {+ "type": "choice",+ "instructions": "What is the request about?",+ "criteria": {+ "refund": "money back, duplicate charge",+ "technical": "bugs and outages",+ "other": null+ }+ },+ "urgency": {+ "type": "score",+ "instructions": "How urgent is this request?",+ "criteria": [+ "not urgent",+ "this week",+ "today"+ ]+ },+ "cancel": {+ "type": "noul",+ "instructions": "The customer threatens to cancel the account."+ }+ },+ "options": {+ "window": 64,+ "batch_size": 2+ },+ "state_tokens": 421,+ "effective_window": 64,+ "effective_stride": 32,+ "effective_room": 990,+ "windows": [+ {+ "token_start": 0,+ "token_end": 64,+ "text": "The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <"+ },+ {+ "token_start": 32,+ "token_end": 96,+ "text": "[CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <e"+ },+ {+ "token_start": 64,+ "token_end": 128,+ "text": "bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <"+ },+ {+ "token_start": 96,+ "token_end": 160,+ "text": "os> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and em"+ },+ {+ "token_start": 128,+ "token_end": 192,+ "text": "mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀"+ },+ {+ "token_start": 160,+ "token_end": 224,+ "text": "ojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀."+ },+ {+ "token_start": 192,+ "token_end": 256,+ "text": " 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes "+ },+ {+ "token_start": 224,+ "token_end": 288,+ "text": " The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <"+ },+ {+ "token_start": 256,+ "token_end": 320,+ "text": "[CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <e"+ },+ {+ "token_start": 288,+ "token_end": 352,+ "text": "bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <"+ },+ {+ "token_start": 320,+ "token_end": 384,+ "text": "os> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and em"+ },+ {+ "token_start": 352,+ "token_end": 416,+ "text": "mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀"+ },+ {+ "token_start": 384,+ "token_end": 421,+ "text": "ojis 😀 🚀. The text includes [CLS] [SEP] <bos> <eos> <mask> and emojis 😀 🚀. "+ }+ ],+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "topic": {+ "type": "choice",+ "choice": "other",+ "probabilities": {+ "refund": 0.0893,+ "technical": 0.259,+ "other": 0.6516+ },+ "confidence": 0.231,+ "answer_confidence": 0.6516,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 3,+ "token_start": 96,+ "token_end": 160,+ "count": 13+ }+ },+ "urgency": {+ "type": "score",+ "score": 1.3344,+ "legend": {+ "0": "not urgent",+ "1": "this week",+ "2": "today"+ },+ "probabilities": {+ "0": 0.2168,+ "1": 0.2319,+ "2": 0.5512+ },+ "confidence": 0.091,+ "answer_confidence": 0.5512,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 12,+ "token_start": 384,+ "token_end": 421,+ "count": 13+ }+ },+ "cancel": {+ "type": "noul",+ "noul": 0.2017,+ "confidence": 0.7983,+ "answer_confidence": 0.7983,+ "action": {+ "act_probability": 1.0+ },+ "window": {+ "index": 12,+ "token_start": 384,+ "token_end": 421,+ "count": 13+ }+ }+ },+ "usage": {+ "input_tokens": 3637,+ "output_tokens": 0,+ "state_tokens": 805,+ "state_tokens_dropped": 0,+ "truncated": 0,+ "truncated_questions": [],+ "windows": 13+ }+ }+}diff --git a/laya-rust/tests/golden/typed-decisions/index.json b/laya-rust/tests/golden/typed-decisions/index.jsonindex 27fdf54e..5c2bf1ce 100644--- a/laya-rust/tests/golden/typed-decisions/index.json+++ b/laya-rust/tests/golden/typed-decisions/index.json@@ -84,5 +84,83 @@"file": "case_shortlist_many_options.json","expects_error": false,"kind": "shortlist"+ },+ {+ "name": "long_short",+ "file": "case_long_short.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_exact_limit",+ "file": "case_long_exact_limit.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_room_fast_path",+ "file": "case_long_room_fast_path.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_multi_chunk",+ "file": "case_long_multi_chunk.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_punctuation",+ "file": "case_long_punctuation.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_dict",+ "file": "case_long_dict.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_list",+ "file": "case_long_list.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_clamp",+ "file": "case_long_clamp.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_special_tokens",+ "file": "case_long_special_tokens.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_empty_questions",+ "file": "case_long_empty_questions.json",+ "expects_error": false,+ "kind": "long"+ },+ {+ "name": "long_invalid_stride",+ "file": "case_long_invalid_stride.json",+ "expects_error": true,+ "kind": "long"+ },+ {+ "name": "long_invalid_aggregate",+ "file": "case_long_invalid_aggregate.json",+ "expects_error": true,+ "kind": "long"+ },+ {+ "name": "long_no_state_room",+ "file": "case_long_no_state_room.json",+ "expects_error": true,+ "kind": "long"}]diff --git a/laya-rust/tests/golden/typed-decisions/meta.json b/laya-rust/tests/golden/typed-decisions/meta.jsonindex fbb10bf1..308ce341 100644--- a/laya-rust/tests/golden/typed-decisions/meta.json+++ b/laya-rust/tests/golden/typed-decisions/meta.json@@ -1,7 +1,7 @@{"generator": "tools/dump_golden.py","onnx_dir": "typed-decisions",- "laya_version": "0.4.0",+ "laya_version": "0.4.11","onnxruntime_version": "1.30.0","transformers_version": "5.17.0","config": {@@ -73,12 +73,12 @@},"onnx_self_check": {"single_noul": {- "logits_maxdiff": 4.470348358154297e-07,- "act_maxdiff": 0.00146484375,+ "logits_maxdiff": 1.817941665649414e-06,+ "act_maxdiff": 0.0009765625,"act_probs_maxdiff": 0.0},"triage_4q": {- "logits_maxdiff": 2.6226043701171875e-06,+ "logits_maxdiff": 2.86102294921875e-06,"act_maxdiff": 0.00146484375,"act_probs_maxdiff": 0.0}diff --git a/laya-rust/tests/predict_long_tests.rs b/laya-rust/tests/predict_long_tests.rsnew file mode 100644index 00000000..7483416d--- /dev/null+++ b/laya-rust/tests/predict_long_tests.rs@@ -0,0 +1,457 @@+//! `predict_long`: the model-free pieces (router capability check, tokenizer trait default) and the+//! model-backed parity of every recorded `"kind": "long"` golden case.+//!+//! The goldens come from the genuine `ONNXAgent.predict_long` over the same ONNX graph. Besides the+//! answers, window attribution and usage, the cases record the tokenized state length, the+//! effective window/stride/room and each window's decoded text, which pin the token offsets, the+//! decoder's special-token retention and the budget arithmetic independently of rounded scores.+//!+//! Run with `LAYA_ONNX_ROOT` (and `LAYA_ONNX_SPLIT_ROOT` for the split layout) set and+//! `--test-threads=1`, as for the other model-backed tests.++mod common;++use common::engines::{self, CHECKPOINTS};+use common::golden_data::{self, CheckpointGoldenData};+use common::result_compare::assert_answers_match;+use common::stub_tokenizer::StubTokenizer;+use laya::answers::{LayaResult, Usage, WindowInfo};+use laya::error::LayaError;+use laya::laya_engine::LayaEngine;+use laya::laya_options::LayaCheckpoint;+use laya::laya_router::{LayaPredictor, LayaRouter, LayaRouterOptions};+use laya::long_prediction::{LongOptions, window_budget};+use laya::python_json::PythonJson;+use laya::questions::{Question, QuestionSet};+use laya::tokenization::{HfTokenizer, LayaTokenizer};+use serde_json::Value;+use std::collections::HashMap;+use std::sync::Arc;++// ---------------------------------------------------------------------- model-free++#[test]+fn a_tokenizer_without_decode_reports_it_instead_of_breaking_the_trait() {+ // `StubTokenizer` implements only the required methods; `decode` is the trait's default.+ let error = StubTokenizer::new().decode(&[1, 2, 3]).unwrap_err();+ assert!(matches!(error, LayaError::Tokenizer(_)), "got {error:?}");+}++struct EncodeOnlyPredictor;++impl LayaPredictor for EncodeOnlyPredictor {+ fn predict(&self, _state: Value, questions: &QuestionSet) -> laya::Result<LayaResult> {+ Ok(LayaResult::new(+ "fake",+ questions.ids().map(String::from).collect(),+ HashMap::new(),+ Usage::default(),+ ))+ }+}++#[test]+fn a_router_never_silently_truncates_through_a_predictor_without_predict_long() {+ let router = LayaRouter::with_factory(LayaRouterOptions::default(), |_, _| {+ Ok(Arc::new(EncodeOnlyPredictor) as Arc<dyn LayaPredictor>)+ })+ .expect("router with a fake factory");+ let questions = QuestionSet::new()+ .with("flag", Question::noul("Is it?", Value::Null, Value::Null))+ .unwrap();++ let error = router+ .predict_long(+ "some state",+ &questions,+ &LongOptions::new(),+ Some("en"),+ None,+ None,+ )+ .unwrap_err();+ assert!(+ matches!(error, LayaError::NotSupported(_)),+ "expected NotSupported, got {error:?}"+ );+ assert!(error.to_string().contains("no predict_long"), "{error}");+}++#[test]+fn long_options_treat_zero_like_unset_and_take_chained_setters() {+ let options = LongOptions::new().window(64).stride(32).batch_size(2);+ assert_eq!(options.window, Some(64));+ assert_eq!(options.stride, Some(32));+ assert_eq!(options.batch_size, Some(2));+ assert_eq!(options.aggregate, None);+ assert_eq!(LongOptions::default(), LongOptions::new());+}++// ---------------------------------------------------------------------- model-backed parity++fn option_usize(options: &Value, name: &str) -> Option<usize> {+ options+ .get(name)+ .and_then(Value::as_u64)+ .map(|v| v as usize)+}++fn long_options(options: &Value) -> LongOptions {+ let mut built = LongOptions::new();+ built.window = option_usize(options, "window");+ built.stride = option_usize(options, "stride");+ built.batch_size = option_usize(options, "batch_size");+ built.aggregate = options+ .get("aggregate")+ .and_then(Value::as_str)+ .map(str::to_string);+ built+}++fn expect_usize(value: &Value, what: &str) -> usize {+ value+ .as_u64()+ .unwrap_or_else(|| panic!("{what}: expected a non-negative integer, got {value}"))+ as usize+}++fn check_long_usage(where_case: &str, want: &Value, got: &LayaResult) {+ assert_eq!(+ expect_usize(&want["input_tokens"], "input_tokens") as u32,+ got.usage().input_tokens,+ "{where_case}: usage.input_tokens"+ );+ assert_eq!(+ expect_usize(&want["output_tokens"], "output_tokens") as u32,+ got.usage().output_tokens,+ "{where_case}: usage.output_tokens"+ );+ let scan = got+ .scan_usage()+ .unwrap_or_else(|| panic!("{where_case}: predict_long must report scan usage"));+ assert_eq!(+ expect_usize(&want["windows"], "windows"),+ scan.windows,+ "{where_case}: usage.windows"+ );++ let Some(state) = got.state_usage() else {+ // Only a call with no questions has no per-state diagnostics, and Python then omits them.+ for key in [+ "state_tokens",+ "state_tokens_dropped",+ "truncated",+ "truncated_questions",+ ] {+ assert!(+ want.get(key).is_none(),+ "{where_case}: usage.{key} missing from result"+ );+ }+ return;+ };+ assert_eq!(+ expect_usize(&want["state_tokens"], "state_tokens"),+ state.state_tokens,+ "{where_case}: usage.state_tokens"+ );+ assert_eq!(+ expect_usize(&want["state_tokens_dropped"], "state_tokens_dropped"),+ state.state_tokens_dropped,+ "{where_case}: usage.state_tokens_dropped"+ );+ // Python reports a bool for one pass and a count of truncated windows for a scan.+ let truncated = match &want["truncated"] {+ Value::Bool(b) => usize::from(*b),+ other => expect_usize(other, "truncated"),+ };+ assert_eq!(+ truncated, scan.truncated_windows,+ "{where_case}: usage.truncated"+ );+ assert_eq!(+ truncated > 0,+ state.truncated,+ "{where_case}: truncated flag"+ );+ let ids: Vec<&str> = want["truncated_questions"]+ .as_array()+ .unwrap()+ .iter()+ .map(|id| id.as_str().unwrap())+ .collect();+ let got_ids: Vec<&str> = state+ .truncated_questions+ .iter()+ .map(String::as_str)+ .collect();+ assert_eq!(ids, got_ids, "{where_case}: usage.truncated_questions");++ let want_options = want.get("options").and_then(Value::as_object);+ assert_eq!(+ want_options.map_or(0, |o| o.len()),+ state.options.len(),+ "{where_case}: usage.options count"+ );+ if let Some(want_options) = want_options {+ let want_ids: Vec<&str> = want_options.keys().map(String::as_str).collect();+ let got_ids: Vec<&str> = state.options.keys().map(String::as_str).collect();+ assert_eq!(want_ids, got_ids, "{where_case}: usage.options ids");+ for (id, expected) in want_options {+ let got = &state.options[id];+ assert_eq!(+ expect_usize(&expected["total"], "total"),+ got.total,+ "{where_case}/{id}: options.total"+ );+ assert_eq!(+ expect_usize(&expected["distinct"], "distinct"),+ got.distinct,+ "{where_case}/{id}: options.distinct"+ );+ assert_eq!(+ expected["tokens_per_option"].as_u64().map(|v| v as usize),+ got.tokens_per_option,+ "{where_case}/{id}: options.tokens_per_option"+ );+ }+ }+}++fn check_windows(where_case: &str, want: &Value, got: &LayaResult) {+ let answers = want["answers"].as_object().unwrap();+ for (id, expected) in answers {+ let answer = got+ .get(id)+ .unwrap_or_else(|| panic!("{where_case}/{id}: missing answer"));+ match expected.get("window") {+ Some(window) => assert_eq!(+ Some(WindowInfo {+ index: expect_usize(&window["index"], "index"),+ token_start: expect_usize(&window["token_start"], "token_start"),+ token_end: expect_usize(&window["token_end"], "token_end"),+ count: expect_usize(&window["count"], "count"),+ }),+ answer.window(),+ "{where_case}/{id}: window attribution"+ ),+ None => assert_eq!(+ None,+ answer.window(),+ "{where_case}/{id}: single-pass answer must carry no window"+ ),+ }+ }+}++fn check_long_case(engine: &LayaEngine, where_case: &str, root: &Value) {+ let state = root["state"].clone();+ let questions = golden_data::build_questions(&root["questions"])+ .unwrap_or_else(|e| panic!("{where_case}: {e}"));+ let options = &root["options"];+ let got = engine+ .predict_long(state.clone(), &questions, &long_options(options))+ .unwrap_or_else(|e| panic!("{where_case}: predict_long failed: {e}"));+ let expected = &root["result"];++ // The SDK keeps its own model identifier instead of the ONNX agent's suffix.+ assert_eq!("laya-rl-agent-onnx", expected["model"].as_str().unwrap());+ assert_eq!("laya-rl-agent", got.model(), "{where_case}: model");+ check_long_usage(where_case, &expected["usage"], &got);+ assert_answers_match(where_case, expected, &got);+ check_windows(where_case, expected, &got);++ // These comparisons pin token offsets, decoder cleanup and special-token retention+ // independently of model probabilities (a dropped token could otherwise round away).+ let artifacts = engine.artifacts();+ let tokenizer = HfTokenizer::from_file_with_config(+ &artifacts.tokenizer_path,+ artifacts.tokenizer_config_path.as_deref(),+ )+ .unwrap_or_else(|e| panic!("{where_case}: tokenizer: {e}"));+ let mask_token = tokenizer.special().mask_token.clone();+ let state_ids = tokenizer+ .encode(&PythonJson::state(&state).replace(&mask_token, " "))+ .unwrap();+ assert_eq!(+ expect_usize(&root["state_tokens"], "state_tokens"),+ state_ids.len(),+ "{where_case}: state token count"+ );+ let config = engine.config();+ let budget = window_budget(+ &tokenizer,+ &questions,+ config.max_len,+ config.head_max_len,+ option_usize(options, "window").unwrap_or(0),+ option_usize(options, "stride").unwrap_or(0),+ )+ .unwrap_or_else(|e| panic!("{where_case}: window_budget: {e}"));+ assert_eq!(+ expect_usize(&root["effective_window"], "effective_window"),+ budget.window,+ "{where_case}: effective window"+ );+ assert_eq!(+ expect_usize(&root["effective_stride"], "effective_stride"),+ budget.stride,+ "{where_case}: effective stride"+ );+ assert_eq!(+ expect_usize(&root["effective_room"], "effective_room"),+ budget.room,+ "{where_case}: effective room"+ );+ for window in root["windows"].as_array().unwrap() {+ let start = expect_usize(&window["token_start"], "token_start");+ let end = expect_usize(&window["token_end"], "token_end");+ assert_eq!(+ window["text"].as_str().unwrap(),+ tokenizer.decode(&state_ids[start..end]).unwrap(),+ "{where_case}: decoded window {start}..{end}"+ );+ }++ let windows = got.scan_usage().unwrap().windows;+ match root["name"].as_str().unwrap() {+ "long_exact_limit" => {+ assert_eq!(+ budget.window,+ state_ids.len(),+ "{where_case}: state at limit"+ );+ assert_eq!(1, windows, "{where_case}: exact-limit state is one pass");+ }+ "long_room_fast_path" => {+ assert!(+ state_ids.len() > budget.window,+ "{where_case}: state must exceed the default window"+ );+ assert_eq!(budget.room, state_ids.len(), "{where_case}: state at room");+ assert_eq!(1, windows, "{where_case}: room-sized state is one pass");+ }+ _ => {}+ }++ if windows == 1 && !questions.is_empty() {+ // The one-pass shortcut is the ordinary predict path, so it must agree with it exactly.+ let plain = engine.predict(state, &questions).expect("plain predict");+ assert_eq!(plain.usage(), got.usage(), "{where_case}: one-pass usage");+ for id in questions.ids() {+ assert_eq!(+ plain[id].confidence(),+ got[id].confidence(),+ "{where_case}/{id}: one-pass confidence"+ );+ assert_eq!(+ plain[id].answer_confidence(),+ got[id].answer_confidence(),+ "{where_case}/{id}: one-pass answer_confidence"+ );+ assert_eq!(+ plain[id].action(),+ got[id].action(),+ "{where_case}/{id}: one-pass action"+ );+ }+ }+}++fn run_success_cases(+ load: fn(LayaCheckpoint) -> Option<LayaEngine>,+ label: &str,+ unavailable: &str,+) {+ let mut checked_any = false;+ for checkpoint in CHECKPOINTS {+ let Some(engine) = load(checkpoint) else {+ continue;+ };+ let golden = CheckpointGoldenData::for_checkpoint(checkpoint.subdir());+ if !golden.available() {+ eprintln!("skip: no golden data for '{}'", checkpoint.subdir());+ continue;+ }+ for case in golden.long_cases() {+ checked_any = true;+ let root = golden.load_case(&case);+ check_long_case(+ &engine,+ &format!("{}/{case}{label}", checkpoint.subdir()),+ &root,+ );+ }+ }+ if !checked_any {+ eprintln!("skip: {unavailable}");+ }+}++#[test]+fn reproduces_the_recorded_long_predictions() {+ run_success_cases(+ engines::load,+ "",+ "no long golden data + artifacts found for any checkpoint",+ );+}++#[test]+fn reproduces_the_recorded_long_predictions_for_the_split_layout() {+ run_success_cases(+ engines::load_split,+ " (split layout)",+ "no split-layout ONNX artifacts + long golden data found for any checkpoint",+ );+}++#[test]+fn rejects_the_same_long_requests_python_rejects() {+ let mut checked_any = false;+ for checkpoint in CHECKPOINTS {+ let Some(engine) = engines::load(checkpoint) else {+ continue;+ };+ let golden = CheckpointGoldenData::for_checkpoint(checkpoint.subdir());+ if !golden.available() {+ continue;+ }+ for case in golden.long_error_cases() {+ checked_any = true;+ let root = golden.load_case(&case);+ let questions = golden_data::build_questions(&root["questions"])+ .unwrap_or_else(|e| panic!("{}/{case}: {e}", checkpoint.subdir()));+ let error = engine+ .predict_long(+ root["state"].clone(),+ &questions,+ &long_options(&root["options"]),+ )+ .expect_err(&format!(+ "{}/{case}: expected predict_long to fail",+ checkpoint.subdir()+ ));+ let expected_message = root["error_message"].as_str().unwrap();+ let actual_message = error.to_string();+ assert!(+ actual_message.starts_with(expected_message),+ "{}/{case}: error message mismatch\n python: {expected_message}\n rust: {actual_message}",+ checkpoint.subdir()+ );+ let name = root["name"].as_str().unwrap();+ let right_variant = match name {+ "long_invalid_aggregate" => matches!(error, LayaError::UnsupportedAggregate(_)),+ _ => matches!(error, LayaError::InvalidWindow(_)),+ };+ assert!(+ right_variant,+ "{}/{case}: wrong error variant: {error:?}",+ checkpoint.subdir()+ );+ }+ }+ if !checked_any {+ eprintln!("skip: no long error-case golden data + artifacts found for any checkpoint");+ }+}diff --git a/laya-rust/tools/dump_golden.py b/laya-rust/tools/dump_golden.pyindex 6c410e46..21cebb49 100644--- a/laya-rust/tools/dump_golden.py+++ b/laya-rust/tools/dump_golden.py@@ -8,6 +8,8 @@ unmodified, so every number written here is what Python actually answers -- notparaphrase of it. It also means no 1.3 GB safetensors checkpoint is needed: the exportedONNX graph supplies the forward pass.+Long cases run the genuine `ONNXAgent.predict_long`, sharing that same verified session.+Before writing anything the script checks option logits and unrounded action probabilitiesagainst the PyTorch reference vectors bundled in `fixtures.npz`. Raw action logits areretained in the goldens and the self-check report for diagnostics.@@ -37,7 +39,7 @@ REPO = os.path.dirname(os.path.dirname(TOOLS))sys.path.insert(0, REPO)sys.path.insert(0, TOOLS)-from golden_cases import CASES, ERROR_CASES, SHORTLIST_E2E_CASES # noqa: E402+from golden_cases import CASES, ERROR_CASES, LONG_CASES, LONG_ERROR_CASES, SHORTLIST_E2E_CASES # noqa: E402from repr_cases import build_repr_probe # noqa: E402import hashing_embedder # noqa: E402@@ -399,6 +401,61 @@ def record_error_case(agent: Agent, case: dict) -> dict:raise SystemExit("case %r was expected to raise ValueError but did not" % case["name"])+def record_long_case(agent: Agent, case: dict, expects_error: bool = False) -> dict:+ """Run the genuine ONNXAgent scan, sharing the already verified session/tokenizer."""+ from laya.common import encode_text, window_budget+ from laya.onnx_agent import ONNXAgent++ onnx = ONNXAgent.__new__(ONNXAgent)+ onnx.cfg, onnx.tok, onnx.session = agent.cfg, agent.tok, agent.model.sess+ onnx.temperature = agent.temperature+ onnx.temperature_by_options = agent.temperature_by_options+ onnx.lang_temperatures = {}+ options = case.get("options", {})+ state, questions = case.get("state"), case["questions"]+ rec = {"name": case["name"], "state": state, "questions": questions, "options": options}+ if expects_error:+ try:+ onnx.predict_long(state, questions, **options)+ except ValueError as e:+ rec.update(error_type="ValueError", error_message=str(e))+ return rec+ raise SystemExit("long case %r was expected to raise ValueError but did not" % case["name"])++ internal = [onnx._to_internal(q) for q in questions.values()]+ budget, step, room = window_budget(+ onnx.tok, internal, onnx.cfg.get("max_len", 512), onnx.cfg.get("head_max_len", 192),+ window=options.get("window"), stride=options.get("stride"),+ )+ # predict_long's one-pass bound: an explicit window, else the room the questions leave.+ single = budget if (options.get("window") or 0) > 0 else room+ if case.get("state_at_room") and room <= budget:+ raise SystemExit("long case %r needs room (%d) > window (%d)" % (case["name"], room, budget))+ target = budget if case.get("state_at_limit") else room if case.get("state_at_room") else None+ if target is not None:+ state = " ".join(["a"] * target)+ rec["state"] = state+ state_ids = encode_text(+ onnx.tok, serialize_state(state).replace(onnx.tok.mask_token, " "), add_special_tokens=False,+ )["input_ids"]+ if target is not None and len(state_ids) != target:+ raise SystemExit("boundary state is %d tokens, expected exactly %d" % (len(state_ids), target))+ rec.update(state_tokens=len(state_ids), effective_window=budget, effective_stride=step,+ effective_room=room)+ # Record decoded windows too: this pins cleanup, special tokens and boundary drift independently+ # of rounded model scores. The single-window fast path must not decode/re-encode the state.+ windows = []+ if len(state_ids) > single:+ for start in range(0, len(state_ids), step):+ windows.append({"token_start": start, "token_end": min(start + budget, len(state_ids)),+ "text": onnx.tok.decode(state_ids[start:start + budget])})+ if start + budget >= len(state_ids):+ break+ rec["windows"] = windows+ rec["result"] = onnx.predict_long(state, questions, **options)+ return rec++def dump(obj, path: str):with open(path, "w", encoding="utf-8", newline="\n") as f:json.dump(obj, f, ensure_ascii=False, indent=2, sort_keys=False)@@ -492,6 +549,15 @@ def run_checkpoint(onnx_dir: str, out_dir: str) -> int:"expects_error": False, "kind": "shortlist",})+ for cases, expects_error in ((LONG_CASES, False), (LONG_ERROR_CASES, True)):+ for case in cases:+ rec = record_long_case(agent, case, expects_error)+ dump(rec, os.path.join(out_dir, "case_%s.json" % case["name"]))+ index.append({+ "name": case["name"], "file": "case_%s.json" % case["name"],+ "expects_error": expects_error, "kind": "long",+ })+dump(index, os.path.join(out_dir, "index.json"))dump(build_repr_probe(), os.path.join(out_dir, "repr_probe.json"))print("done: %d cases -> %s" % (len(index), out_dir))diff --git a/laya-rust/tools/golden_cases.py b/laya-rust/tools/golden_cases.pyindex eef195ec..903b4557 100644--- a/laya-rust/tools/golden_cases.py+++ b/laya-rust/tools/golden_cases.py@@ -316,3 +316,74 @@ SHORTLIST_E2E_CASES = ["k": 20,},]++# ONNXAgent.predict_long, not the torch Agent: the Rust engine is an ONNX runtime too.+LONG_QUESTIONS = {+ "topic": {+ "type": "choice", "instructions": "What is the request about?",+ "criteria": {"refund": "money back, duplicate charge", "technical": "bugs and outages", "other": None},+ },+ "urgency": {+ "type": "score", "instructions": "How urgent is this request?",+ "criteria": ["not urgent", "this week", "today"],+ },+ "cancel": {"type": "noul", "instructions": "The customer threatens to cancel the account."},+}++LONG_CASES = [+ {"name": "long_short", "state": QUICKSTART_STATE, "questions": LONG_QUESTIONS},+ # The dump computes this state from the actual tokenizer and the effective default budget.+ {"name": "long_exact_limit", "state_at_limit": True, "questions": LONG_QUESTIONS},+ # Past the default window but within the questions' room: one Predict pass, no scan.+ {"name": "long_room_fast_path", "state_at_room": True, "questions": LONG_QUESTIONS},+ {+ "name": "long_multi_chunk",+ "state": ("The agent checked the account and recorded routine status updates. " * 180+ + QUICKSTART_STATE),+ "questions": LONG_QUESTIONS,+ "options": {"batch_size": 2},+ },+ {+ "name": "long_punctuation",+ "state": ("I 'm checking . We 've seen this , have n't we ? " * 18+ + "I want a refund ! I 'll cancel today ."),+ "questions": LONG_QUESTIONS,+ "options": {"window": 64, "stride": 64, "batch_size": 3},+ },+ {+ "name": "long_dict",+ "state": {"notes": "Routine checks completed. " * 22, "message": QUICKSTART_STATE},+ "questions": LONG_QUESTIONS,+ "options": {"window": 96},+ },+ {+ "name": "long_list",+ "state": [{"role": "assistant", "content": "Please wait while we check the account. " * 20},+ {"role": "user", "content": QUICKSTART_STATE}],+ "questions": LONG_QUESTIONS,+ "options": {"window": 96, "stride": 48},+ },+ {+ "name": "long_clamp",+ "state": "Normal account activity. " * 200 + QUICKSTART_STATE,+ "questions": LONG_QUESTIONS,+ "options": {"window": 4096, "stride": 2048, "batch_size": 2},+ },+ # Tokenizer.decode must retain special tokens, just as Python's default decode does.+ {+ "name": "long_special_tokens",+ "state": ("The text includes [CLS] [SEP] <bos> <eos> [MASK] <mask> and emojis 😀 🚀. " * 15),+ "questions": LONG_QUESTIONS,+ "options": {"window": 64, "batch_size": 2},+ },+ {"name": "long_empty_questions", "state": "Routine checks. " * 35, "questions": {},+ "options": {"window": 32, "batch_size": 2}},+]++LONG_ERROR_CASES = [+ {"name": "long_invalid_stride", "state": "short", "questions": LONG_QUESTIONS,+ "options": {"window": 32, "stride": 33}},+ {"name": "long_invalid_aggregate", "state": "short", "questions": LONG_QUESTIONS,+ "options": {"aggregate": "mean"}},+ {"name": "long_no_state_room", "state": "short", "questions": ERROR_CASES[0]["questions"]},+]