Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(rust): match Python result and truncation error schemas
kevin-gatimu284e170a0e1e
diff --git a/laya-rust/CHANGELOG.md b/laya-rust/CHANGELOG.mdindex e1edbaa6..4496ba77 100644--- a/laya-rust/CHANGELOG.md+++ b/laya-rust/CHANGELOG.md@@ -6,12 +6,18 @@ All notable changes to the `laya-onnx` crate are documented here. The format fol## [0.1.0] - unreleased-First release: a port of the Laya Python SDK, tested against laya (Python) 0.3.21.+First release: a port of the Laya Python SDK, checked against regenerated laya (Python) 0.4.0+goldens for the supported paths (see the known gaps below).### Added- Advisory parity CI on every push and PR: regenerate fixtures from the current Python checkout,test all checkpoints, and reject skipped tests. No golden cache or crate publication.+- `LayaResult::state_usage()` reports state-token truncation across question budgets, matching+ Python 0.4.0 without changing the existing `Usage` totals or result constructor.+- Engine score legends render structured criteria as strings, matching Python 0.4.0.+- `NoMarkers` errors include actual/requested marker counts and the total sequence budget,+ matching Python's actionable truncation diagnostic.- `LayaEngine`: `choice`, `score` and `noul` predictions over the exported ONNX checkpoints(english, multilingual, typed-decisions); all questions for a state run in one batched forward pass.- Loads both ONNX layouts: the fused `model.onnx` and the split `encoder.onnx` + `head.onnx`.@@ -26,5 +32,5 @@ First release: a port of the Laya Python SDK, tested against laya (Python) 0.3.2- Routing, script counting and fused-line email disclaimers follow the regenerated Python 0.4.0probes. New mixed-field language heuristics are not yet ported.-- Not yet ported from Python 0.3.21: abstention (`min_confidence`), `predict_long` and- `lang_temperatures`.+- Not yet ported: abstention (`min_confidence`, including per-bucket thresholds),+ histogram-binning calibration, `predict_long` and `lang_temperatures`.diff --git a/laya-rust/README.md b/laya-rust/README.mdindex 1298ca33..1845f1a9 100644--- a/laya-rust/README.md+++ b/laya-rust/README.md@@ -558,6 +558,10 @@ Goldens are never cached. A `skip:` diagnostic or ignored test makes the lane reCargo counts a missing-model early return as a pass. Job-level `continue-on-error` keeps Pythonimprovements unblocked; Rust drift needs a follow-up port. This workflow never publishes crates.+Engine results expose Python 0.4.0's state-budget diagnostics via `LayaResult::state_usage()`:+original state tokens, worst-case dropped tokens, whether truncation occurred, and the affected+question ids. The existing `Usage` totals and result constructor remain source-compatible.+To reproduce it, install CPU torch and [`tools/requirements-regen.txt`](tools/requirements-regen.txt),then run from the repository root:@@ -651,6 +655,7 @@ Rust port groups the corresponding functions into a unit struct with associatedand checked against freshly regenerated probes. New mixed-field language heuristics remaina follow-up; these probes are not a claim of full coverage of every new Python feature.-- **Opt-in features added in Python 0.3.21 are not ported yet:** abstention (`min_confidence`),- `predict_long`, and per-language temperatures (`lang_temperatures`). They are off by default in+- **Opt-in features are not ported yet:** abstention (`min_confidence`, including per-bucket+ thresholds), histogram-binning calibration, `predict_long`, and per-language temperatures+ (`lang_temperatures`). They are off by default inPython, so default predictions are unaffected. They are planned as follow-ups.diff --git a/laya-rust/src/answers.rs b/laya-rust/src/answers.rsindex bc909276..bb483c67 100644--- a/laya-rust/src/answers.rs+++ b/laya-rust/src/answers.rs@@ -67,7 +67,8 @@ pub struct ScoreAnswer {/// Expected level: the probability-weighted mean of the level indices, a fractional value/// between 0 and `legend.len() - 1`. Rounded to 4 decimals.pub score: f64,- /// What each level index means, as supplied on the question.+ /// What each level index means. Engine-produced legends render each criterion as a string,+ /// matching Python 0.4.0, including structured dict/list levels.pub legend: Vec<Value>,/// Probability per level index. Rounded to 4 decimals.pub probabilities: Vec<f64>,@@ -220,6 +221,19 @@ impl Answer {}}+/// State-budget diagnostics reported by Python's `usage` since 0.4.0.+#[derive(Debug, Clone, PartialEq, Eq)]+pub struct StateUsage {+ /// Tokens in the untruncated state, counted once rather than once per question.+ pub state_tokens: usize,+ /// The largest state-token loss across all question heads.+ pub state_tokens_dropped: usize,+ /// Whether at least one question lost state tokens.+ pub truncated: bool,+ /// Ids of questions that lost state tokens, in question order.+ pub truncated_questions: Vec<String>,+}+/// The result of one `predict` call: one answer per question, plus token usage.#[derive(Debug, Clone)]pub struct LayaResult {@@ -229,6 +243,7 @@ pub struct LayaResult {by_id: HashMap<String, Answer>,routing: Option<RouteDecision>,shortlist: Option<IndexMap<String, ShortlistInfo>>,+ state_usage: Option<StateUsage>,}impl LayaResult {@@ -250,6 +265,7 @@ impl LayaResult {by_id,routing: None,shortlist: None,+ state_usage: None,}}@@ -290,6 +306,17 @@ impl LayaResult {self.usage}+ /// State-budget diagnostics for engine-produced answers. Manually constructed results+ /// retain `None`, preserving the existing constructor and the `Copy` token-total `Usage`.+ pub fn state_usage(&self) -> Option<&StateUsage> {+ self.state_usage.as_ref()+ }++ pub(crate) fn with_state_usage(mut self, usage: StateUsage) -> Self {+ self.state_usage = Some(usage);+ self+ }+/// The question ids, in the order they were asked.pub fn ids(&self) -> &[String] {&self.orderdiff --git a/laya-rust/src/error.rs b/laya-rust/src/error.rsindex 9b99f2c2..3f306504 100644--- a/laya-rust/src/error.rs+++ b/laya-rust/src/error.rs@@ -84,11 +84,44 @@ pub enum LayaError {/// Ports the "silently answering a truncated question" failure mode called out in/// `sequence-construction.md`, and matches Python's wording exactly (parity tests compare/// this text against golden `error_message` values byte-for-byte).- #[error("question '{id}' options exceed head_max_len={head_max_len}")]+ #[error(+ "question {}: only {actual_markers} of its {expected_markers} option markers fit in \+ max_len={max_len} with head_max_len={head_max_len} spent on the question; \+ lower head_max_len, raise max_len, or use fewer options",+ crate::laya_router::python_repr_str(.id)+ )]NoMarkers {/// The question id that lost one or more markers.id: String,/// The configured head budget the question's options did not fit within.head_max_len: usize,+ /// Markers that survived the total sequence budget.+ actual_markers: usize,+ /// Markers requested by the question.+ expected_markers: usize,+ /// The configured total sequence budget.+ max_len: usize,},}++#[cfg(test)]+mod tests {+ use super::LayaError;++ #[test]+ fn marker_truncation_error_matches_python_and_quotes_question_ids() {+ let error = LayaError::NoMarkers {+ id: "user's question".to_string(),+ head_max_len: 192,+ actual_markers: 125,+ expected_markers: 300,+ max_len: 512,+ };+ assert_eq!(+ error.to_string(),+ "question \"user's question\": only 125 of its 300 option markers fit in max_len=512 \+ with head_max_len=192 spent on the question; lower head_max_len, raise max_len, or \+ use fewer options"+ );+ }+}diff --git a/laya-rust/src/laya_engine.rs b/laya-rust/src/laya_engine.rsindex e90d68b1..0bb14304 100644--- a/laya-rust/src/laya_engine.rs+++ b/laya-rust/src/laya_engine.rs@@ -13,7 +13,7 @@//! Tokenization needs no such lock: `tokenizers::Tokenizer::encode` takes `&self`.use crate::answers::{- ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScoreAnswer, Usage,+ ActionInfo, Answer, ChoiceAnswer, LayaResult, NoulAnswer, ScoreAnswer, StateUsage, Usage,};use crate::calibration::Calibration;use crate::collator::{CollatedBatch, Collator, SequenceItem};@@ -175,6 +175,12 @@ impl LayaEngine {let ids: Vec<String> = questions.ids().map(str::to_string).collect();let mut items = Vec::with_capacity(ids.len());+ let mut state_usage = StateUsage {+ state_tokens: 0,+ state_tokens_dropped: 0,+ truncated: false,+ truncated_questions: Vec::new(),+ };// A conversation-turn list is serialized newest-last (see `PythonJson::state`), so the// default right-truncation of an over-long state would silently drop the newest turn.@@ -186,7 +192,7 @@ impl LayaEngine {// The tokenizer here needs no lock: tokenizers::Tokenizer::encode takes &self.for id in &ids {let question = &questions[id.as_str()];- let (seq, markers) = SequenceBuilder::build(+ let (seq, markers, stats) = SequenceBuilder::build_with_stats(self.tokenizer.as_ref(),&state,question,@@ -194,13 +200,23 @@ impl LayaEngine {self.config.head_max_len,truncate_left,)?;+ state_usage.state_tokens = stats.tokens;+ state_usage.state_tokens_dropped = state_usage.state_tokens_dropped.max(stats.dropped);+ if stats.dropped > 0 {+ state_usage.truncated = true;+ state_usage.truncated_questions.push(id.clone());+ }// A marker dropped for landing past max_len means an option is not in the sequence at// all, so its logit would score whatever token happens to sit at position 0.- if markers.len() != SequenceBuilder::render_options(question).len() {+ let expected_markers = SequenceBuilder::render_options(question).len();+ if markers.len() != expected_markers {return Err(LayaError::NoMarkers {id: id.clone(),head_max_len: self.config.head_max_len,+ actual_markers: markers.len(),+ expected_markers,+ max_len: self.config.max_len,});}@@ -241,7 +257,8 @@ impl LayaEngine {input_tokens: batch.input_tokens as u32,output_tokens: 0,},- ))+ )+ .with_state_usage(state_usage))}/// One forward pass over a collated batch, returning the raw graph outputs. See the@@ -526,7 +543,9 @@ fn build_answer(question: &Question, p: &[f64], action: ActionInfo) -> Answer {let expected = Calibration::round4(Calibration::expected_score(p));Answer::Score(ScoreAnswer::new(expected,- score.levels().to_vec(),+ score.levels().iter()+ .map(|level| Value::String(crate::python_json::PythonJson::criterion(level)))+ .collect(),probabilities,confidence,answer_confidence,@@ -661,6 +680,28 @@ mod tests {use super::*;use serde_json::json;+ #[test]+ fn score_legends_render_structured_levels_as_strings() {+ let question = Question::score(+ "severity",+ [json!({"label": "info", "cvss": 0.0}), json!({"label": "high", "cvss": 7.5})],+ ).unwrap();+ let Answer::Score(answer) = build_answer(+ &question,+ &[0.25, 0.75],+ ActionInfo { act_probability: 1.0 },+ ) else {+ panic!("score question must produce a score answer");+ };+ assert_eq!(+ answer.legend,+ vec![+ json!("{\"label\": \"info\", \"cvss\": 0.0}"),+ json!("{\"label\": \"high\", \"cvss\": 7.5}"),+ ],+ );+ }+/// A tiny two-row batch, just enough shape/data to exercise `value_for_input` without a real/// ONNX session.fn tiny_batch() -> CollatedBatch {diff --git a/laya-rust/src/laya_router.rs b/laya-rust/src/laya_router.rsindex ad3ab50b..edd1c1c6 100644--- a/laya-rust/src/laya_router.rs+++ b/laya-rust/src/laya_router.rs@@ -569,7 +569,7 @@ impl LayaPredictor for LayaRouter {/// the common control characters are escaped. Every model/task/lang name recorded in the routing/// golden vectors is plain ASCII, so this covers the cases that matter without attempting a full/// `repr()` (which would also need to classify every Unicode character as printable or not).-fn python_repr_str(s: &str) -> String {+pub(crate) fn python_repr_str(s: &str) -> String {let quote = if s.contains('\'') && !s.contains('"') {'"'} else {diff --git a/laya-rust/src/sequence_builder.rs b/laya-rust/src/sequence_builder.rsindex 9f1e8bc7..38519fd2 100644--- a/laya-rust/src/sequence_builder.rs+++ b/laya-rust/src/sequence_builder.rs@@ -16,6 +16,12 @@ use serde_json::Value;/// Namespace for sequence construction. See the module docs for the source this ports.pub struct SequenceBuilder;+#[derive(Debug, Clone, Copy)]+pub(crate) struct StateTokenStats {+ pub tokens: usize,+ pub dropped: usize,+}+impl SequenceBuilder {/// One option fragment is capped at this many tokens, before the marker is prepended.const MAX_OPTION_TOKENS: usize = 48;@@ -125,6 +131,18 @@ impl SequenceBuilder {head_max_len: usize,truncate_left: bool,) -> Result<(Vec<u32>, Vec<usize>)> {+ Self::build_with_stats(tokenizer, state, question, max_len, head_max_len, truncate_left)+ .map(|(ids, markers, _)| (ids, markers))+ }++ pub(crate) fn build_with_stats(+ tokenizer: &dyn LayaTokenizer,+ state: &Value,+ question: &Question,+ max_len: usize,+ head_max_len: usize,+ truncate_left: bool,+ ) -> Result<(Vec<u32>, Vec<usize>, StateTokenStats)> {let special = tokenizer.special();let mask_token = special.mask_token.as_str();@@ -201,6 +219,10 @@ impl SequenceBuilder {}markers.retain(|&m| m < max_len);- Ok((ids, markers))+ let stats = StateTokenStats {+ tokens: state_ids.len(),+ dropped: state_ids.len() - sliced.len(),+ };+ Ok((ids, markers, stats))}}diff --git a/laya-rust/tests/common/result_compare.rs b/laya-rust/tests/common/result_compare.rsindex e320fc68..9be5925a 100644--- a/laya-rust/tests/common/result_compare.rs+++ b/laya-rust/tests/common/result_compare.rs@@ -5,7 +5,6 @@use laya::LayaResult;use laya::answers::{Answer, ChoiceAnswer, NoulAnswer, ScoreAnswer};-use laya::python_json::PythonJson;use laya::sequence_builder::SequenceBuilder;use serde_json::Value;@@ -26,6 +25,7 @@ fn assert_close(expected: f64, actual: f64, tolerance: f64, what: &str) {/// action/choice/score) matches the recorded golden `expected` (a case's `"result"` field) within/// the standard parity tolerances. `where_case` prefixes every assertion message.pub fn assert_result_matches(where_case: &str, expected: &Value, result: &LayaResult) {+ assert_state_usage_matches(where_case, expected, result);assert_eq!(expected["model"].as_str().unwrap(),result.model(),@@ -36,6 +36,11 @@ pub fn assert_result_matches(where_case: &str, expected: &Value, result: &LayaReresult.usage().input_tokens,"{where_case}: usage.input_tokens");+ assert_eq!(+ expected["usage"]["output_tokens"].as_u64().unwrap() as u32,+ result.usage().output_tokens,+ "{where_case}: usage.output_tokens"+ );let answers = expected["answers"].as_object().unwrap();let expected_ids: Vec<&str> = answers.keys().map(String::as_str).collect();@@ -62,6 +67,12 @@ pub fn assert_result_matches(where_case: &str, expected: &Value, result: &LayaRePROB_TOLERANCE,&format!("{where_q}.confidence"),);+ assert_close(+ want["answer_confidence"].as_f64().unwrap(),+ got.answer_confidence(),+ PROB_TOLERANCE,+ &format!("{where_q}.answer_confidence"),+ );assert_close(want["action"]["act_probability"].as_f64().unwrap(),got.action().act_probability,@@ -77,6 +88,29 @@ pub fn assert_result_matches(where_case: &str, expected: &Value, result: &LayaRe}}+pub fn assert_state_usage_matches(where_case: &str, expected: &Value, result: &LayaResult) {+ let usage = result.state_usage().expect("engine result must report state usage");+ let want = &expected["usage"];+ assert_eq!(want.as_object().unwrap().len(), 6,+ "{where_case}: new Python usage fields need a Rust port");+ assert_eq!(+ want["state_tokens"].as_u64().unwrap() as usize, usage.state_tokens,+ "{where_case}: usage.state_tokens"+ );+ assert_eq!(+ want["state_tokens_dropped"].as_u64().unwrap() as usize, usage.state_tokens_dropped,+ "{where_case}: usage.state_tokens_dropped"+ );+ assert_eq!(+ want["truncated"].as_bool().unwrap(), usage.truncated,+ "{where_case}: usage.truncated"+ );+ let ids: Vec<&str> = want["truncated_questions"].as_array().unwrap()+ .iter().map(|id| id.as_str().unwrap()).collect();+ let got: Vec<&str> = usage.truncated_questions.iter().map(String::as_str).collect();+ assert_eq!(ids, got, "{where_case}: usage.truncated_questions");+}+fn assert_choice(want: &Value, got: &ChoiceAnswer, where_q: &str) {let probabilities = want["probabilities"].as_object().unwrap();let expected_labels: Vec<&str> = probabilities.keys().map(String::as_str).collect();@@ -106,8 +140,8 @@ fn assert_score(want: &Value, got: &ScoreAnswer, where_q: &str) {for i in 0..got.legend.len() {let expected_level = &legend[&i.to_string()];assert_eq!(- PythonJson::criterion(expected_level),- PythonJson::criterion(&got.legend[i]),+ expected_level,+ &got.legend[i],"{where_q}: legend[{i}]");}diff --git a/laya-rust/tests/predict_parity_tests.rs b/laya-rust/tests/predict_parity_tests.rsindex f4466875..36cac94a 100644--- a/laya-rust/tests/predict_parity_tests.rs+++ b/laya-rust/tests/predict_parity_tests.rs@@ -18,7 +18,6 @@ use common::golden_data::{self, CheckpointGoldenData};use laya::answers::{Answer, ChoiceAnswer, NoulAnswer, ScoreAnswer};use laya::error::LayaError;use laya::laya_options::LayaCheckpoint;-use laya::python_json::PythonJson;use laya::questions::QuestionSet;use laya::sequence_builder::SequenceBuilder;use serde_json::Value;@@ -74,6 +73,7 @@ fn reproduces_the_recorded_answers() {.unwrap_or_else(|e| panic!("{}/{case}: predict failed: {e}", checkpoint.subdir()));let expected = &root["result"];let where_case = format!("{}/{case}", checkpoint.subdir());+ common::result_compare::assert_state_usage_matches(&where_case, expected, &result);assert_eq!(expected["model"].as_str().unwrap(),@@ -119,6 +119,12 @@ fn reproduces_the_recorded_answers() {PROB_TOLERANCE,&format!("{where_q}.confidence"),);+ assert_close(+ want["answer_confidence"].as_f64().unwrap(),+ got.answer_confidence(),+ PROB_TOLERANCE,+ &format!("{where_q}.answer_confidence"),+ );assert_close(want["action"]["act_probability"].as_f64().unwrap(),got.action().act_probability,@@ -176,6 +182,7 @@ fn reproduces_the_recorded_answers_for_the_split_layout() {.unwrap_or_else(|e| panic!("{}/{case}: predict failed: {e}", checkpoint.subdir()));let expected = &root["result"];let where_case = format!("{}/{case} (split layout)", checkpoint.subdir());+ common::result_compare::assert_state_usage_matches(&where_case, expected, &result);assert_eq!(expected["model"].as_str().unwrap(),@@ -204,6 +211,12 @@ fn reproduces_the_recorded_answers_for_the_split_layout() {&format!("{where_q}.confidence"),);+ assert_close(+ want["answer_confidence"].as_f64().unwrap(),+ got.answer_confidence(),+ PROB_TOLERANCE,+ &format!("{where_q}.answer_confidence"),+ );match got {Answer::Choice(choice) => assert_choice(want, choice, &where_q),Answer::Score(score) => assert_score(want, score, &where_q),@@ -428,8 +441,8 @@ fn assert_score(want: &Value, got: &ScoreAnswer, where_q: &str) {for i in 0..got.legend.len() {let expected_level = &legend[&i.to_string()];assert_eq!(- PythonJson::criterion(expected_level),- PythonJson::criterion(&got.legend[i]),+ expected_level,+ &got.legend[i],"{where_q}: legend[{i}]");}