Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(go): harden the predict path's public contract after review
Adeshina Lasisie0b510e86a4e
diff --git a/.github/workflows/go.yml b/.github/workflows/go.ymlindex 4cbe5182..712dc702 100644--- a/.github/workflows/go.yml+++ b/.github/workflows/go.yml@@ -127,11 +127,11 @@ jobs:LAYA_GO_FIXTURES: ${{ github.workspace }}/.go-goldensLAYA_CHECKPOINTS: ${{ github.workspace }}/laya-java/.work/checkpointsrun: go test -race -count=1 -json ./... > "$RUNNER_TEMP/go-test.json"- # Exactly the measured count (1254 tests and subtests, on both Go versions): dropping a+ # Exactly the measured count (1307 tests and subtests, on both Go versions): dropping a# case fails here, and adding one means raising this number.- name: No test was skippedif: ${{ !cancelled() }}- run: python laya-go/scripts/check_test_json.py "$RUNNER_TEMP/go-test.json" --min-tests 1254+ run: python laya-go/scripts/check_test_json.py "$RUNNER_TEMP/go-test.json" --min-tests 1307# go.mod declares go 1.22, so that is what a consumer may build with.- name: Run the tests on the oldest supported Go@@ -143,7 +143,7 @@ jobs:run: go test -count=1 -json ./... > "$RUNNER_TEMP/go-test-oldest.json"- name: No test was skipped on the oldest Go eitherif: ${{ !cancelled() }}- run: python laya-go/scripts/check_test_json.py "$RUNNER_TEMP/go-test-oldest.json" --min-tests 1254+ run: python laya-go/scripts/check_test_json.py "$RUNNER_TEMP/go-test-oldest.json" --min-tests 1307- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2if: ${{ failure() }}diff --git a/laya-go/README.md b/laya-go/README.mdindex 324b9ddb..43509b8f 100644--- a/laya-go/README.md+++ b/laya-go/README.md@@ -2,8 +2,8 @@A pure-Go port of [laya](../README.md). Standard library only: no cgo, no third-party modules.-This first slice is the foundation the rest of the port stands on. The routing and model layers-(ONNX, agent, router) come in later changes.+The foundation (language detection, questions, sequences, the tokenizer) and the predict path+(`infer`, `agent`) are here. The native ONNX session and the router come in later changes.| Package | What it is || --- | --- |@@ -11,7 +11,7 @@ This first slice is the foundation the rest of the port stands on. The routing a| `schema` | Questions (`Choice`, `ChoiceOf`, `Score`, `Noul`, `NoulWith`, `Questions`) and results (`Answer`, `Prediction`, `Usage`) every other package shares. Questions are refused with laya's own wording and read or written in the Python wire form. || `sequence` | `build_sequence`, `build_head`, `state_room`, `window_budget` and `window_batch_cap`: `EncodeState`, `Build`, `StateRoom`, `PlanWindows`, `BatchCap`. || `infer` | The seam to a model runtime: `Batch` (the graph's five inputs), `Output` (its two), `Session`, and the shape checks `Batch.Validate` and `Output.Check`. |-| `agent` | `ONNXAgent.predict` and `predict_batch` over an `infer.Session`: `Open`, `Predict`, `PredictBatch`, `LoadConfig`, with batch size, sort-by-length and context cancellation. |+| `agent` | `ONNXAgent.predict` and `predict_batch` over an `infer.Session`: `Open`, `Predict`, `PredictBatch`, `LoadConfig`, with batch size, sort-by-length and context cancellation between runs. || `tokenizer` | A reader and runner for Hugging Face `tokenizer.json` (BPE): normalizer, pre-tokenizer, added tokens, byte fallback, decoder. || `internal/unicodedata` | The character predicates `laya/lang.py` leans on, recorded from CPython. || `internal/pyjson` | `json.dumps` and `repr` spelled as CPython spells them. |@@ -39,11 +39,13 @@ order-sensitive state. `Analyze` reads every leaf, so its cost grows with the nu## Layout- No root package: the module path ends in `laya-go`, so tags are `laya-go/vX.Y.Z`.-- Pure-Go packages are `lang` now and `router`, `presets` and `shortlist` later.-- cgo, if ever needed, lives only in a future `onnx` package, with a `//go:build !cgo` stub.- Importing `lang` never compiles cgo.+- Every package in this module is pure Go: `lang`, `schema`, `sequence`, `tokenizer`, `infer` and+ `agent` now, and `router`, `presets` and `shortlist` later.+- cgo will live only in an `onnx` package, a nested module with its own `go.mod`, so `go test ./...`+ here never compiles it. Importing `lang` or `agent` never compiles cgo.- `tokenizer` is public (`LoadFS`, `Parse`); `internal/unicodedata`, `internal/pyjson`,- `internal/decode` and `internal/collate` stay internal.+ `internal/decode` and `internal/collate` stay internal. `infer.Batch` is the public name of+ what `collate` builds.## Parity## Paritydiff --git a/laya-go/agent/agent.go b/laya-go/agent/agent.goindex 8a37a16e..48788796 100644--- a/laya-go/agent/agent.go+++ b/laya-go/agent/agent.go@@ -5,19 +5,37 @@ import ("errors""fmt""io/fs"+ "reflect""sync"+ "sync/atomic""github.com/NandhaKishorM/laya/laya-go/infer""github.com/NandhaKishorM/laya/laya-go/internal/decode""github.com/NandhaKishorM/laya/laya-go/schema"+ "github.com/NandhaKishorM/laya/laya-go/sequence""github.com/NandhaKishorM/laya/laya-go/tokenizer")-// ErrClosed is returned by an Agent that has been closed.-var ErrClosed = errors.New("laya: agent is closed")+// ErrClosed is returned by an Agent that has been closed. A session closed underneath an open+// agent yields a wrapped infer.ErrClosed instead.+var ErrClosed = errors.New("agent: agent is closed")++// StateError identifies the state at fault in a PredictBatch call.+type StateError struct {+ // Index is the state's position in the states slice.+ Index int+ Err error+}++// Error names the state and the cause.+func (e *StateError) Error() string { return fmt.Sprintf("agent: state %d: %v", e.Index, e.Err) }++// Unwrap returns the cause.+func (e *StateError) Unwrap() error { return e.Err }// Agent answers typed questions about a state with a laya checkpoint, through an infer.Session.-// It is the port of laya's ONNXAgent, and it is safe for concurrent use.+// It is the port of laya's ONNXAgent, and it is safe for concurrent use. Use Open: the zero+// Agent is closed.type Agent struct {tok *tokenizer.TokenizerpadID int32@@ -26,56 +44,81 @@ type Agent struct {sess infer.Sessionmu sync.RWMutex // held for reading by every call, for writing by Close- closed bool+ closed atomic.Bool // set when Close starts, so new and queued calls return at once}+// Option adjusts an Agent at Open. No options exist yet; the parameter is here so adding one is+// not a breaking change.+type Option func(*options)++type options struct{}+// Open loads rl_agent_config.json and the tokenizer from checkpoint, a directory holding a// laya checkpoint, and takes ownership of s: on error it closes s, and otherwise Close does.-// The session must run the graph exported from that same checkpoint; nothing here can tell.-func Open(checkpoint fs.FS, s infer.Session) (*Agent, error) {+// The tokenizer must name the pad, CLS, SEP and mask tokens. The session must run the graph+// exported from that same checkpoint; nothing here can tell.+func Open(checkpoint fs.FS, s infer.Session, opts ...Option) (*Agent, error) {if s == nil {- return nil, errors.New("laya: Open needs a session")+ return nil, errors.New("agent: Open needs a session")+ }+ var o options+ for _, opt := range opts {+ if opt != nil {+ opt(&o)+ }}a, err := open(checkpoint, s)if err != nil {- _ = s.Close()- return nil, err+ return nil, errors.Join(err, s.Close())}return a, nil}func open(checkpoint fs.FS, s infer.Session) (*Agent, error) {+ if checkpoint == nil {+ return nil, errors.New("agent: Open needs a checkpoint")+ }cfg, temps, err := loadConfig(checkpoint)if err != nil {return nil, err}tok, err := tokenizer.LoadFS(checkpoint, ".")if err != nil {- return nil, fmt.Errorf("laya: tokenizer: %w", err)+ return nil, &ConfigError{Field: tokenizerField, Err: err}}- pad, ok := tok.Pad()- if !ok {- return nil, errors.New("laya: tokenizer names no pad token")+ var pad int32+ for _, t := range []struct {+ name string+ id func() (int32, bool)+ }{{"pad", tok.Pad}, {"CLS", tok.CLS}, {"SEP", tok.SEP}, {"mask", tok.Mask}} {+ id, ok := t.id()+ if !ok {+ return nil, fmt.Errorf("agent: %w: no %s token", sequence.ErrMissingSpecialToken, t.name)+ }+ if t.name == "pad" {+ pad = id+ }}return &Agent{tok: tok, padID: pad, cfg: cfg, temps: temps, sess: s}, nil}-// Config is the checkpoint configuration in use, after clamping.+// Config returns a copy of the checkpoint configuration the agent runs with.func (a *Agent) Config() Config {cfg := a.cfgcfg.Warnings = append([]string(nil), cfg.Warnings...)return cfg}-// Close waits for calls in flight, then closes the session. It is idempotent; a later Predict-// returns ErrClosed.+// Close marks the agent closed, so new and queued calls return ErrClosed at once, then waits for+// calls in flight and closes the session. Close blocks until calls in flight return; cancel+// their contexts to bound it. It is idempotent.func (a *Agent) Close() error {+ first := a.closed.CompareAndSwap(false, true)a.mu.Lock()defer a.mu.Unlock()- if a.closed {+ if !first || a.sess == nil {return nil}- a.closed = truereturn a.sess.Close()}@@ -96,15 +139,21 @@ func WithLanguage(tag string) CallOption { return func(c *call) { c.lang = tag }// holds states x questions rows.func WithBatchSize(states int) CallOption { return func(c *call) { c.batch = states } }-// WithSortByLength pads less: states are ordered by length inside a lookahead of eight batches-// before they are chunked. It needs a batch size above 1 and below the number of states, and-// results stay in input order. A different batch shape can move a probability by a rounding step.-func WithSortByLength() CallOption { return func(c *call) { c.sort = true } }+// WithSortByLength(true) pads less: states are ordered by length inside a lookahead of eight+// batches before they are chunked. It needs a batch size above 1 and below the number of states,+// and results stay in input order. A different batch shape can move a probability by a rounding+// step. WithSortByLength(false) is the default.+func WithSortByLength(on bool) CallOption { return func(c *call) { c.sort = on } }-// Predict answers every question about one state. Its errors are those of PredictBatch.+// Predict answers every question about one state. Its errors are those of PredictBatch, except+// that a failing state's error is returned without the *StateError around it.func (a *Agent) Predict(ctx context.Context, state any, qs schema.Questions, opts ...CallOption) (schema.Prediction, error) {preds, err := a.PredictBatch(ctx, []any{state}, qs, opts...)if err != nil {+ var se *StateError+ if errors.As(err, &se) {+ err = se.Err+ }return schema.Prediction{}, err}return preds[0], nil@@ -112,9 +161,17 @@ func (a *Agent) Predict(ctx context.Context, state any, qs schema.Questions, opt// PredictBatch answers the questions for each state; results are in input order. A state is a// string, or any value sequence.SerializeState accepts: a slice is a conversation whose end-// survives truncation. Errors pass through unchanged: *schema.QuestionError,-// *sequence.MarkersError (ID set), sequence.ErrUnsupportedState and ctx.Err(). With no questions-// each state gets an empty answer and zero usage, with no model run.+// survives truncation. A nil state or nil pointer is refused; a nil slice or map is an empty+// list or dict. With no questions each state gets an empty answer and zero usage, with no model+// run.+//+// Errors may be wrapped; match them with errors.Is and errors.As:+// - ErrClosed, once Close has been called;+// - ctx.Err(), checked on entry, before each state is encoded and between runs;+// - *StateError, naming the state, around sequence.ErrUnsupportedState (also for a nil+// state), sequence.ErrMissingSpecialToken or a *sequence.MarkersError with its ID set;+// - *RunError, around whatever the session returned (so errors.Is finds context.Canceled and+// infer.ErrClosed through it), a *infer.ShapeError, or a logit that cannot be decoded.func (a *Agent) PredictBatch(ctx context.Context, states []any, qs schema.Questions, opts ...CallOption) ([]schema.Prediction, error) {var c callfor _, opt := range opts {@@ -122,14 +179,20 @@ func (a *Agent) PredictBatch(ctx context.Context, states []any, qs schema.Questiopt(&c)}}+ if err := ctx.Err(); err != nil {+ return nil, err+ }+ if a.sess == nil || a.closed.Load() {+ return nil, ErrClosed+ }a.mu.RLock()defer a.mu.RUnlock()- if a.closed {+ if a.closed.Load() {return nil, ErrClosed}for i, state := range states {- if state == nil {- return nil, fmt.Errorf("laya: state %d is nil; pass a string, a map or a slice", i)+ if isNil(state) {+ return nil, &StateError{Index: i, Err: fmt.Errorf("is nil; pass a string, a map or a slice: %w", sequence.ErrUnsupportedState)}}}if len(states) == 0 {@@ -143,5 +206,14 @@ func (a *Agent) PredictBatch(ctx context.Context, states []any, qs schema.Questi}return preds, nil}- return a.infer(ctx, states, qs, c)+ return a.predictChunks(ctx, states, qs, c)+}++// isNil reports an untyped nil or a nil pointer; a nil slice or map is an empty list or dict.+func isNil(state any) bool {+ if state == nil {+ return true+ }+ rv := reflect.ValueOf(state)+ return rv.Kind() == reflect.Pointer && rv.IsNil()}diff --git a/laya-go/agent/agent_test.go b/laya-go/agent/agent_test.goindex 4a4ed1f4..c09d6869 100644--- a/laya-go/agent/agent_test.go+++ b/laya-go/agent/agent_test.go@@ -7,6 +7,7 @@ import ("fmt""io/fs""reflect"+ "slices""strings""sync""testing"@@ -35,18 +36,25 @@ type run struct {type fakeSession struct {mu sync.Mutexruns []run+ pads [][]int64 // per run, the distinct ids under a zero maskcloses int// before, when set, runs at the start of each Run with the number of runs so far.before func(n int) error// shape, when set, edits the output before it is returned.shape func(*infer.Output)+ // onCtx, when set, sees the context each Run was given.+ onCtx func(context.Context)}-func (s *fakeSession) Run(_ context.Context, b infer.Batch) (infer.Output, error) {+func (s *fakeSession) Run(ctx context.Context, b infer.Batch) (infer.Output, error) {+ if s.onCtx != nil {+ s.onCtx(ctx)+ }if err := b.Validate(); err != nil {return infer.Output{}, err}r := run{Rows: b.Rows, Length: b.Length, Markers: b.Markers}+ var pads []int64out := infer.Output{Logits: make([]float32, b.Rows*b.Markers), ActLogits: make([]float32, b.Rows*2)}for i := 0; i < b.Rows; i++ {var sum, tokens, markers int64@@ -54,6 +62,8 @@ func (s *fakeSession) Run(_ context.Context, b infer.Batch) (infer.Output, errorif b.AttentionMask[i*b.Length+j] == 1 {sum += b.InputIDs[i*b.Length+j]tokens+++ } else if id := b.InputIDs[i*b.Length+j]; !slices.Contains(pads, id) {+ pads = append(pads, id)}}for j := 0; j < b.Markers; j++ {@@ -73,6 +83,7 @@ func (s *fakeSession) Run(_ context.Context, b infer.Batch) (infer.Output, errors.mu.Lock()n := len(s.runs)s.runs = append(s.runs, r)+ s.pads = append(s.pads, pads)before, shape := s.before, s.shapes.mu.Unlock()if before != nil {@@ -346,7 +357,7 @@ func TestEmptyQuestionsAndStates(t *testing.T) {func TestStateErrors(t *testing.T) {a, s := open(t, `{}`)ctx := context.Background()- if _, err := a.PredictBatch(ctx, []any{"fine", nil}, questions(t)); err == nil || !strings.Contains(err.Error(), "state 1 is nil") {+ if _, err := a.PredictBatch(ctx, []any{"fine", nil}, questions(t)); err == nil || !strings.Contains(err.Error(), "state 1: is nil") {t.Errorf("nil state: %v", err)}if _, err := a.Predict(ctx, struct{ X int }{1}, questions(t)); !errors.Is(err, sequence.ErrUnsupportedState) {@@ -458,7 +469,7 @@ func TestResultsAreTheSameWhateverTheBatching(t *testing.T) {for _, sorted := range []bool{false, true} {opts := []agent.CallOption{agent.WithBatchSize(size)}if sorted {- opts = append(opts, agent.WithSortByLength())+ opts = append(opts, agent.WithSortByLength(true))}got, err := a.PredictBatch(ctx, states, qs, opts...)if err != nil || len(got) != len(states) {@@ -507,13 +518,13 @@ func TestChunkingAndSorting(t *testing.T) {// Sorting reorders only when 1 < chunk < states: otherwise the runs are the unsorted ones.for _, size := range []int{1, 11, 40} {plain := record(agent.WithBatchSize(size))- sorted := record(agent.WithBatchSize(size), agent.WithSortByLength())+ sorted := record(agent.WithBatchSize(size), agent.WithSortByLength(true))if !reflect.DeepEqual(plain, sorted) {t.Errorf("batch %d: sorting changed the runs", size)}}unsorted := lengths(record(agent.WithBatchSize(3)))- sorted := lengths(record(agent.WithBatchSize(3), agent.WithSortByLength()))+ sorted := lengths(record(agent.WithBatchSize(3), agent.WithSortByLength(true)))for i := 1; i < len(sorted); i++ {if sorted[i] < sorted[i-1] {t.Errorf("sorted run lengths %v are not ascending", sorted)@@ -650,7 +661,7 @@ func TestContextStopsTheBatchBetweenChunks(t *testing.T) {ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond)defer cancel()s.before = func(int) error { <-ctx.Done(); return nil }- if _, err := a.PredictBatch(ctx, states, qs, agent.WithBatchSize(2), agent.WithSortByLength()); !errors.Is(err, context.DeadlineExceeded) {+ if _, err := a.PredictBatch(ctx, states, qs, agent.WithBatchSize(2), agent.WithSortByLength(true)); !errors.Is(err, context.DeadlineExceeded) {t.Fatalf("PredictBatch = %v", err)}if n := len(s.seen()); n != 1 {@@ -687,7 +698,7 @@ func TestConcurrentPredict(t *testing.T) {}continue}- got, err := a.PredictBatch(context.Background(), states, qs, agent.WithBatchSize(3+g), agent.WithSortByLength())+ got, err := a.PredictBatch(context.Background(), states, qs, agent.WithBatchSize(3+g), agent.WithSortByLength(true))if err != nil {errs <- errreturndiff --git a/laya-go/agent/batch.go b/laya-go/agent/batch.goindex b889955a..ece9514f 100644--- a/laya-go/agent/batch.go+++ b/laya-go/agent/batch.go@@ -12,13 +12,29 @@ import ("github.com/NandhaKishorM/laya/laya-go/sequence")+// RunError reports a failed model run: the session's error, an output that does not fit the+// batch, or a logit that cannot be decoded.+type RunError struct {+ // Rows, Length and Markers are the dimensions of the batch that was run.+ Rows, Length, Markers int+ Err error+}++// Error names the batch and the cause.+func (e *RunError) Error() string {+ return fmt.Sprintf("agent: running %d rows of %d tokens with %d marker slots: %v", e.Rows, e.Length, e.Markers, e.Err)+}++// Unwrap returns the cause.+func (e *RunError) Unwrap() error { return e.Err }+// lookaheadBatches is how many batches of states are encoded and sorted together.const lookaheadBatches = 8-// infer is ONNXAgent._infer_batch: encode, collate, run and decode in chunks of c.batch states.+// predictChunks is ONNXAgent._infer_batch: encode, collate, run and decode in chunks of c.batch states.// With c.sort the states are ordered by their longest question row inside a window of eight// chunks, so a chunk pads to a shorter length; results stay in input order.-func (a *Agent) infer(ctx context.Context, states []any, qs schema.Questions, c call) ([]schema.Prediction, error) {+func (a *Agent) predictChunks(ctx context.Context, states []any, qs schema.Questions, c call) ([]schema.Prediction, error) {chunk := len(states)if c.batch > 0 {chunk = min(c.batch, len(states))@@ -33,9 +49,12 @@ func (a *Agent) infer(ctx context.Context, states []any, qs schema.Questions, cpart := states[start:min(start+window, len(states))]encoded := make([][]sequence.Sequence, len(part))for i, state := range part {+ if err := ctx.Err(); err != nil {+ return nil, err+ }var err errorif encoded[i], err = a.encode(state, qs); err != nil {- return nil, err+ return nil, &StateError{Index: start + i, Err: err}}}order := make([]int, len(part))@@ -55,6 +74,9 @@ func (a *Agent) infer(ctx context.Context, states []any, qs schema.Questions, cif err := ctx.Err(); err != nil {return nil, err}+ if a.closed.Load() {+ return nil, ErrClosed+ }idx := order[off:min(off+chunk, len(order))]preds, err := a.run(ctx, qs, encoded, idx, c.lang)if err != nil {@@ -100,14 +122,15 @@ func (a *Agent) run(ctx context.Context, qs schema.Questions, encoded [][]sequen}b, err := collate.Collate(rows, a.padID)if err != nil {- return nil, err+ return nil, fmt.Errorf("agent: %w", err)}+ fail := func(err error) error { return &RunError{Rows: b.Rows, Length: b.Length, Markers: b.Markers, Err: err} }out, err := a.sess.Run(ctx, b)if err != nil {- return nil, fmt.Errorf("laya: running %d rows of %d tokens: %w", b.Rows, b.Length, err)+ return nil, fail(err)}if err := out.Check(b); err != nil {- return nil, err+ return nil, fail(err)}preds := make([]schema.Prediction, len(idx))row := 0@@ -118,13 +141,13 @@ func (a *Agent) run(ctx context.Context, qs schema.Questions, encoded [][]sequenat, e := row+r, qs.At(r)act, err := decode.ActionProbabilities(out.ActLogits[at*2 : at*2+2])if err != nil {- return nil, err+ return nil, fail(fmt.Errorf("action logits of question %s: %w", e.ID, err))}ans, err := decode.Decode(e.Question, decode.Row{Logits: out.Logits[at*b.Markers : (at+1)*b.Markers], K: len(s.Markers), Act: act,}, lang, a.temps)if err != nil {- return nil, fmt.Errorf("laya: question %s: %w", e.ID, err)+ return nil, fail(fmt.Errorf("question %s: %w", e.ID, err))}answers[r] = schema.Answered{ID: e.ID, Answer: ans}}diff --git a/laya-go/agent/config.go b/laya-go/agent/config.goindex 97858cee..300b6f62 100644--- a/laya-go/agent/config.go+++ b/laya-go/agent/config.go@@ -11,6 +11,7 @@ import ("github.com/NandhaKishorM/laya/laya-go/internal/decode""github.com/NandhaKishorM/laya/laya-go/internal/pyjson""github.com/NandhaKishorM/laya/laya-go/internal/pyvalue"+ "github.com/NandhaKishorM/laya/laya-go/internal/unicodedata")// configFile is the checkpoint file the configuration is read from.@@ -25,7 +26,7 @@ const (// ErrUnsupportedLayout is wrapped when a checkpoint's option_layout is not "sequential", the// only layout the exported graph takes.-var ErrUnsupportedLayout = errors.New("laya: option_layout is not supported by the ONNX runtime")+var ErrUnsupportedLayout = errors.New("agent: option_layout is not supported by the ONNX runtime")// ConfigError reports a missing or malformed rl_agent_config.json.type ConfigError struct {@@ -34,18 +35,29 @@ type ConfigError struct {Err error}+// tokenizerField is the ConfigError.Field of a tokenizer that could not be loaded.+const tokenizerField = "tokenizer"+// Error names the file, the field when there is one, and the cause.func (e *ConfigError) Error() string {- if e.Field == "" {- return "laya: " + configFile + ": " + e.Err.Error()+ cause := "<nil>"+ if e.Err != nil {+ cause = e.Err.Error()+ }+ switch e.Field {+ case "":+ return "agent: " + configFile + ": " + cause+ case tokenizerField:+ return "agent: tokenizer: " + cause // not a field of the config file}- return "laya: " + configFile + ": " + e.Field + ": " + e.Err.Error()+ return "agent: " + configFile + ": " + e.Field + ": " + cause}// Unwrap returns the cause.func (e *ConfigError) Unwrap() error { return e.Err }-// Config is the checkpoint configuration an Agent runs with.+// Config is the checkpoint configuration an Agent runs with. Temperatures are not in it; the+// Agent holds them clamped.type Config struct {// MaxLen is the whole sequence budget: max_len, default 512.MaxLen int@@ -59,7 +71,7 @@ type Config struct {}// LoadConfig reads and checks rl_agent_config.json from the root of checkpoint, as ONNXAgent-// does: max_len, head_max_len and option_layout, with the temperatures clamped.+// does: max_len, head_max_len and option_layout, and a warning for every temperature clamped.func LoadConfig(checkpoint fs.FS) (Config, error) {cfg, _, err := loadConfig(checkpoint)return cfg, err@@ -103,6 +115,11 @@ func loadConfig(checkpoint fs.FS) (Config, *decode.Temperatures, error) {}n, isNumber := raw.(json.Number)size, err := strconv.Atoi(string(n))+ var numErr *strconv.NumError+ if errors.As(err, &numErr) && numErr.Err == strconv.ErrRange && !strings.HasPrefix(string(n), "-") {+ return Config{}, nil, &ConfigError{Field: f.key,+ Err: fmt.Errorf("%s is too large for an int", n)}+ }if !isNumber || err != nil || size <= 0 {return Config{}, nil, &ConfigError{Field: f.key,Err: fmt.Errorf("want a positive integer, got %s", pyvalue.Repr(raw))}@@ -160,9 +177,17 @@ func loadConfig(checkpoint fs.FS) (Config, *decode.Temperatures, error) {return cfg, temps, nil}-// sameNumber is float(raw) == applied: false for anything that is not a number.+// sameNumber is float(raw) == applied: false for anything float() refuses. A string is read as+// decode.Clamp reads it: stripped, and without the hex and underscore spellings Python rejects.func sameNumber(raw any, applied float64) bool {switch x := raw.(type) {+ case string:+ s := unicodedata.Strip(x)+ if strings.ContainsAny(s, "xXpP_") {+ return false+ }+ f, err := strconv.ParseFloat(s, 64)+ return err == nil && f == appliedcase json.Number:f, err := strconv.ParseFloat(string(x), 64)return err == nil && f == applieddiff --git a/laya-go/agent/contract_test.go b/laya-go/agent/contract_test.gonew file mode 100644index 00000000..c5ec0e14--- /dev/null+++ b/laya-go/agent/contract_test.go@@ -0,0 +1,539 @@+package agent_test++import (+ "context"+ "errors"+ "fmt"+ "math"+ "os"+ "path/filepath"+ "reflect"+ "slices"+ "strings"+ "sync"+ "testing"+ "testing/fstest"+ "time"++ "github.com/NandhaKishorM/laya/laya-go/agent"+ "github.com/NandhaKishorM/laya/laya-go/infer"+ "github.com/NandhaKishorM/laya/laya-go/internal/fixtures"+ "github.com/NandhaKishorM/laya/laya-go/schema"+ "github.com/NandhaKishorM/laya/laya-go/sequence"+)++// within fails the test if f does not return in a second, and returns its error.+func within(t *testing.T, what string, f func() error) error {+ t.Helper()+ done := make(chan error, 1)+ go func() { done <- f() }()+ select {+ case err := <-done:+ return err+ case <-time.After(time.Second):+ t.Fatalf("%s did not return within a second", what)+ return nil+ }+}++// blockedClose opens an agent whose first Run blocks until release is closed, starts a Predict+// into that Run, then starts Close behind it and waits until the agent reports itself closed.+func blockedClose(t *testing.T, s *fakeSession, states []any, opts ...agent.CallOption) (a *agent.Agent, release chan struct{}, predict chan error, closed chan struct{}) {+ t.Helper()+ a, err := agent.Open(toyCheckpoint(`{}`), s)+ if err != nil {+ t.Fatal(err)+ }+ started := make(chan struct{})+ release = make(chan struct{})+ s.before = func(n int) error {+ if n == 0 {+ close(started)+ <-release+ }+ return nil+ }+ predict = make(chan error, 1)+ go func() { _, err := a.PredictBatch(context.Background(), states, questions(t), opts...); predict <- err }()+ <-started+ closed = make(chan struct{})+ go func() { a.Close(); close(closed) }()+ for deadline := time.Now().Add(time.Second); ; time.Sleep(time.Millisecond) {+ err := within(t, "a probe Predict", func() error {+ _, err := a.Predict(context.Background(), "probe", questions(t))+ return err+ })+ if errors.Is(err, agent.ErrClosed) {+ return a, release, predict, closed+ }+ if time.Now().After(deadline) {+ t.Fatal("Close never marked the agent closed")+ }+ }+}++// TestCloseBehindABlockedRunDoesNotStallNewCalls is the reviewer's scenario: Close waits on a+// call stuck in the session, and a new call, even one with a dead context, must not wait too.+func TestCloseBehindABlockedRunDoesNotStallNewCalls(t *testing.T) {+ s := &fakeSession{}+ a, release, predict, closed := blockedClose(t, s, []any{"state"})+ err := within(t, "Predict with a live context", func() error {+ _, err := a.Predict(context.Background(), "x", questions(t))+ return err+ })+ if !errors.Is(err, agent.ErrClosed) {+ t.Errorf("new Predict = %v, want ErrClosed", err)+ }+ ctx, cancel := context.WithCancel(context.Background())+ cancel()+ err = within(t, "Predict with a cancelled context", func() error {+ _, err := a.Predict(ctx, "x", questions(t))+ return err+ })+ if err != ctx.Err() {+ t.Errorf("Predict with a cancelled context = %v, want ctx.Err()", err)+ }+ select {+ case <-closed:+ t.Fatal("Close returned while a call was in the session")+ default:+ }+ close(release)+ if err := <-predict; err != nil {+ t.Errorf("the call in flight = %v", err)+ }+ <-closed+}++func TestCloseStopsAMultiChunkCallBetweenRuns(t *testing.T) {+ s := &fakeSession{}+ _, release, predict, closed := blockedClose(t, s, []any{"a", "b", "c"}, agent.WithBatchSize(1))+ close(release)+ if err := <-predict; !errors.Is(err, agent.ErrClosed) {+ t.Errorf("the call in flight = %v, want ErrClosed after its first run", err)+ }+ <-closed+ // The call's first run, and at most the probes that ran before Close began.+ for i, r := range s.seen() {+ if r.Rows == 9 {+ t.Errorf("run %d held 3 states: the batch size was lost", i)+ }+ }+ if got := len(s.seen()); got > 4 {+ t.Errorf("%d runs: the call went on after Close", got)+ }+}++func TestZeroAgentIsClosed(t *testing.T) {+ var a agent.Agent+ if err := a.Close(); err != nil {+ t.Errorf("Close = %v", err)+ }+ if err := a.Close(); err != nil {+ t.Errorf("second Close = %v", err)+ }+ if _, err := a.Predict(context.Background(), "x", questions(t)); !errors.Is(err, agent.ErrClosed) {+ t.Errorf("Predict = %v", err)+ }+ if cfg := a.Config(); cfg.MaxLen != 0 {+ t.Errorf("Config = %+v", cfg)+ }+}++func TestConfigErrorWithoutACause(t *testing.T) {+ for _, e := range []*agent.ConfigError{{}, {Field: "max_len"}} {+ if msg := e.Error(); !strings.HasPrefix(msg, "agent: rl_agent_config.json") {+ t.Errorf("Error = %q", msg)+ }+ }+}++type failingClose struct {+ fakeSession+ err error+}++func (f *failingClose) Close() error { f.fakeSession.Close(); return f.err }++func TestOpenReportsTheSessionsCloseError(t *testing.T) {+ boom := errors.New("close failed")+ s := &failingClose{err: boom}+ _, err := agent.Open(toyCheckpoint(`{"option_layout": "dual"}`), s)+ if !errors.Is(err, agent.ErrUnsupportedLayout) || !errors.Is(err, boom) {+ t.Fatalf("Open = %v, want both the load error and the close error", err)+ }+ if s.closes != 1 {+ t.Errorf("closed %d times", s.closes)+ }+}++func TestOpenRefusesWhatCannotRun(t *testing.T) {+ t.Run("nil checkpoint", func(t *testing.T) {+ s := &fakeSession{}+ if a, err := agent.Open(nil, s); err == nil || a != nil || s.closes != 1 {+ t.Fatalf("Open = %v, %v with %d closes", a, err, s.closes)+ }+ })+ roles := map[string]string{"pad_token": "[PAD]", "cls_token": "[CLS]", "sep_token": "[SEP]", "mask_token": "[MASK]"}+ for missing := range roles {+ t.Run("no "+missing, func(t *testing.T) {+ var parts []string+ for role, content := range roles {+ if role != missing {+ parts = append(parts, fmt.Sprintf("%q:%q", role, content))+ }+ }+ slices.Sort(parts)+ fsys := toyCheckpoint(`{}`)+ fsys["tokenizer/tokenizer_config.json"].Data = []byte("{" + strings.Join(parts, ",") + "}")+ s := &fakeSession{}+ _, err := agent.Open(fsys, s)+ if !errors.Is(err, sequence.ErrMissingSpecialToken) || s.closes != 1 {+ t.Fatalf("Open = %v with %d closes", err, s.closes)+ }+ })+ }+ t.Run("tokenizer load error", func(t *testing.T) {+ fsys := toyCheckpoint(`{}`)+ fsys["tokenizer/tokenizer.json"].Data = []byte(`{`)+ var ce *agent.ConfigError+ if _, err := agent.Open(fsys, &fakeSession{}); !errors.As(err, &ce) || ce.Field != "tokenizer" {+ t.Fatalf("Open = %v, want a ConfigError for the tokenizer", err)+ }+ })+}++func TestNilOptionsAreSkipped(t *testing.T) {+ s := &fakeSession{}+ a, err := agent.Open(toyCheckpoint(`{}`), s, nil)+ if err != nil {+ t.Fatal(err)+ }+ defer a.Close()+ if _, err := a.PredictBatch(context.Background(), []any{"x", "y"}, questions(t), nil, agent.WithBatchSize(1), nil); err != nil {+ t.Fatal(err)+ }+ if n := len(s.seen()); n != 2 {+ t.Errorf("%d runs, want 2: the option after the nil one was lost", n)+ }+}++func TestSortByLengthCanBeTurnedOff(t *testing.T) {+ states := varied()[:11]+ runs := func(opts ...agent.CallOption) []int {+ a, s := open(t, `{}`)+ if _, err := a.PredictBatch(context.Background(), states, questions(t), opts...); err != nil {+ t.Fatal(err)+ }+ return lengths(s.seen())+ }+ plain := runs(agent.WithBatchSize(3))+ if got := runs(agent.WithBatchSize(3), agent.WithSortByLength(true)); reflect.DeepEqual(got, plain) {+ t.Fatal("sorting changed nothing; the test proves nothing")+ }+ if got := runs(agent.WithBatchSize(3), agent.WithSortByLength(true), agent.WithSortByLength(false)); !reflect.DeepEqual(got, plain) {+ t.Errorf("WithSortByLength(false) left runs %v, want %v", got, plain)+ }+}++func TestRunErrors(t *testing.T) {+ boom := errors.New("boom")+ cases := []struct {+ name string+ setup func(*fakeSession)+ is error+ as any // a pointer to the typed error that must also be found+ }{+ {"the session fails", func(s *fakeSession) { s.before = func(int) error { return boom } }, boom, nil},+ {"the session reports a cancel", func(s *fakeSession) { s.before = func(int) error { return context.Canceled } }, context.Canceled, nil},+ {"the session is closed under the agent", func(s *fakeSession) { s.before = func(int) error { return infer.ErrClosed } }, infer.ErrClosed, nil},+ {"a wrong-shaped output", func(s *fakeSession) { s.shape = func(o *infer.Output) { o.Logits = o.Logits[1:] } }, nil, new(*infer.ShapeError)},+ {"a NaN logit", func(s *fakeSession) { s.shape = func(o *infer.Output) { o.Logits[0] = float32(nan()) } }, nil, nil},+ {"an infinite action logit", func(s *fakeSession) {+ s.shape = func(o *infer.Output) { o.ActLogits[1] = float32(math.Inf(1)) }+ }, nil, nil},+ }+ for _, c := range cases {+ t.Run(c.name, func(t *testing.T) {+ a, s := open(t, `{}`)+ c.setup(s)+ _, err := a.Predict(context.Background(), "state", questions(t))+ var re *agent.RunError+ if !errors.As(err, &re) {+ t.Fatalf("Predict = %v, want a *RunError", err)+ }+ if r := s.seen()[0]; re.Rows != r.Rows || re.Length != r.Length || re.Markers != r.Markers || re.Rows != 3 {+ t.Errorf("RunError = %+v, run was %+v", re, r)+ }+ if c.is != nil && !errors.Is(err, c.is) {+ t.Errorf("Predict = %v, want it to wrap %v", err, c.is)+ }+ if c.as != nil && !errors.As(err, c.as) {+ t.Errorf("Predict = %v, want %T", err, c.as)+ }+ if !strings.HasPrefix(err.Error(), "agent: ") {+ t.Errorf("message %q lacks the package prefix", err)+ }+ })+ }+}++// TestActionLogitFailureNamesTheQuestion pins the message of C8: the question id and the action+// logits, not "logits" alone.+func TestActionLogitFailureNamesTheQuestion(t *testing.T) {+ a, s := open(t, `{}`)+ s.shape = func(o *infer.Output) { o.ActLogits[2] = float32(nan()) } // row 1: urgency+ _, err := a.Predict(context.Background(), "state", questions(t))+ var re *agent.RunError+ if !errors.As(err, &re) || !strings.Contains(err.Error(), "action logits of question urgency") {+ t.Fatalf("Predict = %v", err)+ }+}++func TestStateErrors_Batch(t *testing.T) {+ a, _ := open(t, `{}`)+ ctx := context.Background()+ type S struct{ X int }+ var nilPtr *string+ cases := []struct {+ name string+ states []any+ index int+ opts []agent.CallOption+ }{+ {"unsupported", []any{"a", "b", S{1}, "d"}, 2, nil},+ {"unsupported in a later window", []any{"a", "b", "c", S{1}, "e"}, 3, []agent.CallOption{agent.WithBatchSize(1)}},+ {"untyped nil", []any{"a", nil}, 1, nil},+ {"nil pointer", []any{"a", "b", nilPtr}, 2, nil},+ }+ for _, c := range cases {+ t.Run(c.name, func(t *testing.T) {+ _, err := a.PredictBatch(ctx, c.states, questions(t), c.opts...)+ var se *agent.StateError+ if !errors.As(err, &se) || se.Index != c.index || !errors.Is(err, sequence.ErrUnsupportedState) {+ t.Fatalf("PredictBatch = %v, want a StateError at %d wrapping ErrUnsupportedState", err, c.index)+ }+ if !strings.Contains(err.Error(), fmt.Sprintf("state %d", c.index)) {+ t.Errorf("message %q does not name the state", err)+ }+ })+ }+ // A nil slice and a nil map are an empty list and dict, as in the SDK's other packages.+ if _, err := a.PredictBatch(ctx, []any{[]any(nil), map[string]any(nil)}, questions(t)); err != nil {+ t.Errorf("nil slice and map: %v", err)+ }+ // A failing question names its state too; Predict returns the inner error alone.+ wide := make([]string, 12)+ for i := range wide {+ wide[i] = fmt.Sprintf("option number %d", i)+ }+ qs, err := schema.NewQuestions(schema.Ask("wide", schema.MustQuestion(schema.ChoiceOf("pick", wide...))))+ if err != nil {+ t.Fatal(err)+ }+ narrow, _ := open(t, `{"max_len": 40, "head_max_len": 30}`)+ var se *agent.StateError+ var me *sequence.MarkersError+ if _, err := narrow.PredictBatch(ctx, []any{"x", "y"}, qs); !errors.As(err, &se) || se.Index != 0 || !errors.As(err, &me) || me.ID != "wide" {+ t.Errorf("PredictBatch = %v", err)+ }+ _, err = narrow.Predict(ctx, "x", qs)+ if errors.As(err, &se) || !errors.As(err, &me) {+ t.Errorf("Predict = %v, want the MarkersError without a StateError", err)+ }+ _, err = a.Predict(ctx, S{1}, questions(t))+ if errors.As(err, &se) || !errors.Is(err, sequence.ErrUnsupportedState) {+ t.Errorf("Predict(struct) = %v, want the bare error", err)+ }+}++// TestNilStateIsRefusedWithoutQuestions pins that Go refuses the nil state Python raises a+// TypeError for, even when there is nothing to ask and nothing would run.+func TestNilStateIsRefusedWithoutQuestions(t *testing.T) {+ a, _ := open(t, `{}`)+ if _, err := a.PredictBatch(context.Background(), []any{"fine", nil}, schema.Questions{}); !errors.Is(err, sequence.ErrUnsupportedState) {+ t.Fatalf("PredictBatch = %v", err)+ }+}++func TestContextIsCheckedOnEntry(t *testing.T) {+ a, s := open(t, `{}`)+ ctx, cancel := context.WithCancel(context.Background())+ cancel()+ if _, err := a.PredictBatch(ctx, nil, questions(t)); err != ctx.Err() {+ t.Errorf("no states: %v, want ctx.Err()", err)+ }+ if _, err := a.PredictBatch(ctx, []any{"x"}, schema.Questions{}); err != ctx.Err() {+ t.Errorf("no questions: %v, want ctx.Err()", err)+ }+ if len(s.seen()) != 0 {+ t.Error("ran with a cancelled context")+ }+}++type ctxKey struct{}++// TestSessionSeesTheCallersContext pins that the context is forwarded to Run: a value on it,+// and its cancellation, both reach the session.+func TestSessionSeesTheCallersContext(t *testing.T) {+ a, s := open(t, `{}`)+ var mu sync.Mutex+ var got []any+ s.onCtx = func(ctx context.Context) {+ mu.Lock()+ got = append(got, ctx.Value(ctxKey{}))+ mu.Unlock()+ }+ ctx := context.WithValue(context.Background(), ctxKey{}, "mine")+ if _, err := a.Predict(ctx, "state", questions(t)); err != nil {+ t.Fatal(err)+ }+ if !reflect.DeepEqual(got, []any{"mine"}) {+ t.Fatalf("the session saw context values %v", got)+ }+ cctx, cancel := context.WithCancel(context.Background())+ s.onCtx = func(c context.Context) { cancel(); <-c.Done() }+ s.before = func(int) error { return cctx.Err() }+ if _, err := within2(t, a, cctx); !errors.Is(err, context.Canceled) {+ t.Errorf("Predict = %v, want the cancel to have reached the session", err)+ }+}++// within2 runs one Predict that would hang forever if the session's context were not cancelled.+func within2(t *testing.T, a *agent.Agent, ctx context.Context) (schema.Prediction, error) {+ t.Helper()+ var p schema.Prediction+ err := within(t, "Predict", func() (err error) { p, err = a.Predict(ctx, "state", questions(t)); return })+ return p, err+}++// TestMaskedPositionsHoldThePadID pins the pad id the collator is given, on the toy tokenizer+// and on the shipped English one (pad id 50283), neither of which is 0.+func TestMaskedPositionsHoldThePadID(t *testing.T) {+ check := func(t *testing.T, a *agent.Agent, s *fakeSession, pad int64) {+ t.Helper()+ if _, err := a.PredictBatch(context.Background(), varied(), questions(t)); err != nil {+ t.Fatal(err)+ }+ var padded bool+ s.mu.Lock()+ defer s.mu.Unlock()+ for _, pads := range s.pads {+ if len(pads) > 0 {+ padded = true+ if !reflect.DeepEqual(pads, []int64{pad}) {+ t.Errorf("masked positions hold %v, want only %d", pads, pad)+ }+ }+ }+ if !padded {+ t.Fatal("no row was padded; the test proves nothing")+ }+ }+ t.Run("toy", func(t *testing.T) {+ a, s := open(t, `{}`)+ check(t, a, s, fixtures.ToyPad)+ })+ t.Run("english", func(t *testing.T) {+ root := fixtures.Checkpoints()+ if root == "" {+ t.Skip("set LAYA_CHECKPOINTS to a directory holding the english and multilingual checkpoints")+ }+ s := &fakeSession{}+ a, err := agent.Open(os.DirFS(filepath.Join(root, "english")), s)+ if err != nil {+ t.Fatal(err)+ }+ defer a.Close()+ check(t, a, s, 50283)+ })+}++func TestEmptyQuestionsUsageIsAnEmptyList(t *testing.T) {+ a, _ := open(t, `{}`)+ preds, err := a.PredictBatch(context.Background(), []any{"x"}, schema.Questions{})+ if err != nil {+ t.Fatal(err)+ }+ if tq := preds[0].Usage.TruncatedQuestions; tq == nil || len(tq) != 0 {+ t.Fatalf("TruncatedQuestions = %#v, want an empty non-nil slice", tq)+ }+}++// pyWarning is the text ONNXAgent.__init__ warns with, less its "laya ONNX: " prefix. The+// expected strings below were produced by running laya.onnx_agent.ONNXAgent on each config with+// onnxruntime, transformers and the tokenizer fix patched out, and reading the RuntimeWarning.+const pyWarning = "this checkpoint ships temperatures outside [0.5, 5] which would distort confidence; clamping %s. " ++ "Treat confidence from the affected buckets as uncalibrated."++func TestClampWarningsMatchPython(t *testing.T) {+ cases := []struct {+ name, config, clamped string // clamped "" means no warning+ }{+ // A bucket and a type entry both clamped: Python lists buckets first.+ {"buckets before types", `{"temperature": [1.0, 9, 1.0], "temperature_by_options": {"choice:11+": 0.1006}}`,+ "choice:11+=0.1006 -> 0.5, temperature[1]=9 -> 5"},+ // float(False) is 0.0, not the 1.0 applied; float(True) is 1.0 and is quiet.+ {"a bool temperature", `{"temperature": [false, true, 1.0]}`, "temperature[0]=False -> 1"},+ // float("2.0") is 2.0, applied unchanged; float("9") is 9.0, clamped to 5.+ {"numeric strings", `{"temperature": ["2.0", "9", 1.0]}`, "temperature[1]='9' -> 5"},+ }+ for _, c := range cases {+ t.Run(c.name, func(t *testing.T) {+ cfg, err := agent.LoadConfig(toyCheckpoint(c.config))+ if err != nil {+ t.Fatal(err)+ }+ if want := []string{fmt.Sprintf(pyWarning, c.clamped)}; !reflect.DeepEqual(cfg.Warnings, want) {+ t.Fatalf("warnings = %q\nwant %q", cfg.Warnings, want)+ }+ })+ }+ for _, quiet := range []string{`{"temperature": ["2.0", " 1.5 ", 1]}`, `{"temperature": [true, 1, 1]}`} {+ if cfg, err := agent.LoadConfig(toyCheckpoint(quiet)); err != nil || len(cfg.Warnings) != 0 {+ t.Errorf("%s: %v, %q, want no warning", quiet, err, cfg.Warnings)+ }+ }+}++func TestMaxLenTooLarge(t *testing.T) {+ _, err := agent.LoadConfig(toyCheckpoint(`{"max_len": 99999999999999999999}`))+ var ce *agent.ConfigError+ if !errors.As(err, &ce) || ce.Field != "max_len" || !strings.Contains(err.Error(), "99999999999999999999 is too large") {+ t.Fatalf("LoadConfig = %v", err)+ }+ _, err = agent.LoadConfig(toyCheckpoint(`{"head_max_len": -99999999999999999999}`))+ if !errors.As(err, &ce) || strings.Contains(err.Error(), "too large") || !strings.Contains(err.Error(), "positive integer") {+ t.Fatalf("LoadConfig = %v, want the positive-integer message", err)+ }+}++func TestMessagesCarryThePackageName(t *testing.T) {+ for want, err := range map[string]error{+ "agent: ": agent.ErrClosed, "infer: ": infer.ErrClosed,+ } {+ if !strings.HasPrefix(err.Error(), want) {+ t.Errorf("%q lacks the prefix %q", err, want)+ }+ }+ if !strings.HasPrefix(agent.ErrUnsupportedLayout.Error(), "agent: ") {+ t.Errorf("%q", agent.ErrUnsupportedLayout)+ }+ if _, err := agent.LoadConfig(fstestEmpty()); !strings.HasPrefix(err.Error(), "agent: ") {+ t.Errorf("%q", err)+ }+ if got := (&agent.StateError{Index: 3, Err: errors.New("x")}).Error(); got != "agent: state 3: x" {+ t.Errorf("StateError = %q", got)+ }+}++func fstestEmpty() fstest.MapFS { return fstest.MapFS{} }++// TestTokenizerErrorDoesNotBlameTheConfigFile: a tokenizer that cannot be loaded is not a field of+// rl_agent_config.json, so its message must not point there.+func TestTokenizerErrorDoesNotBlameTheConfigFile(t *testing.T) {+ err := &agent.ConfigError{Field: "tokenizer", Err: errors.New("no tokenizer.json")}+ if got, want := err.Error(), "agent: tokenizer: no tokenizer.json"; got != want {+ t.Errorf("got %q, want %q", got, want)+ }+}diff --git a/laya-go/agent/doc.go b/laya-go/agent/doc.goindex 90a571d1..00b1b714 100644--- a/laya-go/agent/doc.go+++ b/laya-go/agent/doc.go@@ -15,6 +15,23 @@// or sorting. Batch shape does change padding, and with it a probability's last digits, so// results are compared with a tolerance across shapes, not byte for byte.//+// # Differences from Python+//+// With no questions, Python writes a usage of two keys (input_tokens, output_tokens); schema.Usage+// writes all seven, and so does this package.+//+// A nil state is refused, with or without questions, where Python raises a TypeError only when+// there are questions. A nil slice or map is an empty list or dict, as in Python.+//+// max_len and head_max_len are checked at load and must be positive integers; Python reads them+// with cfg.get and checks nothing until a call uses them.+//+// A temperature that is not a list of exactly 3 numbers is refused at load, as Python's torch+// Agent refuses it; ONNXAgent loads it.+//+// A NaN or infinite logit is refused, including a -Inf for a real option, where Python would+// answer probability 0 for that option. A model that emits one is broken.+//// # Scope//// Not here yet: per-language temperature overrides, the calibration file, min_confidence,diff --git a/laya-go/agent/example_test.go b/laya-go/agent/example_test.goindex 74b79aff..ee288cbc 100644--- a/laya-go/agent/example_test.go+++ b/laya-go/agent/example_test.go@@ -5,36 +5,16 @@ import ("fmt""log""os"- "testing/fstest""github.com/NandhaKishorM/laya/laya-go/agent""github.com/NandhaKishorM/laya/laya-go/infer"- "github.com/NandhaKishorM/laya/laya-go/internal/fixtures""github.com/NandhaKishorM/laya/laya-go/schema")-// scorer is a stand-in for a native session: it prefers the second option of every question.-type scorer struct{}--func (scorer) Run(_ context.Context, b infer.Batch) (infer.Output, error) {- out := infer.Output{Logits: make([]float32, b.Rows*b.Markers), ActLogits: make([]float32, b.Rows*2)}- for i := 0; i < b.Rows; i++ {- out.Logits[i*b.Markers+1] = 2- }- return out, nil-}--func (scorer) Close() error { return nil }-+// Example is compiled and not run: it needs a checkpoint and a native session.func Example() {- // A checkpoint is a directory with rl_agent_config.json and tokenizer/. This one is a toy.- spec, cfg := fixtures.ToyTokenizer()- checkpoint := fstest.MapFS{- "rl_agent_config.json": {Data: []byte(`{"max_len": 128, "head_max_len": 64}`)},- "tokenizer/tokenizer.json": {Data: spec},- "tokenizer/tokenizer_config.json": {Data: cfg},- }- a, err := agent.Open(checkpoint, scorer{}) // a real session comes from the onnx package+ var session infer.Session // from the onnx package+ a, err := agent.Open(os.DirFS("checkpoints/english"), session)if err != nil {log.Fatal(err)}@@ -48,7 +28,7 @@ func Example() {log.Fatal(err)}preds, err := a.PredictBatch(context.Background(), []any{"I was charged twice", "Where is my order?"}, qs,- agent.WithBatchSize(2), agent.WithSortByLength())+ agent.WithBatchSize(2), agent.WithSortByLength(true), agent.WithLanguage("en"))if err != nil {log.Fatal(err)}@@ -56,23 +36,4 @@ func Example() {intent, _ := p.Answers.Get("intent")fmt.Println(intent.(*schema.ChoiceAnswer).Choice, p.Usage.InputTokens, p.Usage.Truncated)}- fmt.Println(a.Config().MaxLen)- // Output:- // refund 170 false- // refund 168 false- // 128-}--// ExampleOpen_checkpoint is compiled and not run: it needs a checkpoint and a native session.-func ExampleOpen_checkpoint() {- var session infer.Session // from the onnx package- a, err := agent.Open(os.DirFS("checkpoints/english"), session)- if err != nil {- log.Fatal(err)- }- defer a.Close()- qs := schema.Questions{}- if _, err := a.Predict(context.Background(), "I was charged twice", qs, agent.WithLanguage("en")); err != nil {- log.Fatal(err)- }}diff --git a/laya-go/agent/plan_golden_test.go b/laya-go/agent/plan_golden_test.goindex 4fc18b87..b847560b 100644--- a/laya-go/agent/plan_golden_test.go+++ b/laya-go/agent/plan_golden_test.go@@ -208,7 +208,7 @@ func TestPredictPlanGolden(t *testing.T) {opts = append(opts, agent.WithBatchSize(fixtures.Int(t, n)))}if fixtures.Bool(t, rec.Must("sort")) {- opts = append(opts, agent.WithSortByLength())+ opts = append(opts, agent.WithSortByLength(true))}if tag := rec.Must("lang"); tag != nil {opts = append(opts, agent.WithLanguage(fixtures.Str(t, tag)))diff --git a/laya-go/agent/toy_end_to_end_test.go b/laya-go/agent/toy_end_to_end_test.gonew file mode 100644index 00000000..da76b25f--- /dev/null+++ b/laya-go/agent/toy_end_to_end_test.go@@ -0,0 +1,55 @@+package agent_test++import (+ "context"+ "fmt"+ "testing"++ "github.com/NandhaKishorM/laya/laya-go/agent"+ "github.com/NandhaKishorM/laya/laya-go/infer"+ "github.com/NandhaKishorM/laya/laya-go/schema"+)++// scorer is a stand-in for a native session: it prefers the second option of every question.+type scorer struct{}++func (scorer) Run(_ context.Context, b infer.Batch) (infer.Output, error) {+ out := infer.Output{Logits: make([]float32, b.Rows*b.Markers), ActLogits: make([]float32, b.Rows*2)}+ for i := 0; i < b.Rows; i++ {+ out.Logits[i*b.Markers+1] = 2+ }+ return out, nil+}++func (scorer) Close() error { return nil }++// TestToyCheckpointEndToEnd runs the documented example on the toy tokenizer.+func TestToyCheckpointEndToEnd(t *testing.T) {+ a, err := agent.Open(toyCheckpoint(`{"max_len": 128, "head_max_len": 64}`), scorer{})+ if err != nil {+ t.Fatal(err)+ }+ defer a.Close()+ qs, err := schema.NewQuestions(+ schema.Ask("intent", schema.MustQuestion(schema.ChoiceOf("What does the customer want?", "status", "refund"))),+ schema.Ask("angry", schema.MustQuestion(schema.Noul("Is the customer angry?"))),+ )+ if err != nil {+ t.Fatal(err)+ }+ preds, err := a.PredictBatch(context.Background(), []any{"I was charged twice", "Where is my order?"}, qs,+ agent.WithBatchSize(2), agent.WithSortByLength(true))+ if err != nil {+ t.Fatal(err)+ }+ var got []string+ for _, p := range preds {+ intent, _ := p.Answers.Get("intent")+ got = append(got, fmt.Sprint(intent.(*schema.ChoiceAnswer).Choice, " ", p.Usage.InputTokens, " ", p.Usage.Truncated))+ }+ got = append(got, fmt.Sprint(a.Config().MaxLen))+ want := []string{"refund 170 false", "refund 168 false", "128"}+ if fmt.Sprint(got) != fmt.Sprint(want) {+ t.Fatalf("got %q, want %q", got, want)+ }+}diff --git a/laya-go/infer/infer.go b/laya-go/infer/infer.goindex 88c5c38b..4d4e0b51 100644--- a/laya-go/infer/infer.go+++ b/laya-go/infer/infer.go@@ -4,10 +4,22 @@ import ("context""errors""fmt"+ "math")// ErrClosed is returned by a Session that has been closed.-var ErrClosed = errors.New("laya: session is closed")+var ErrClosed = errors.New("infer: session is closed")++// The graph's tensor names, as ShapeError.Name carries them and as a native session binds them.+const (+ NameInputIDs = "input_ids"+ NameAttentionMask = "attention_mask"+ NameMarkerPos = "marker_pos"+ NameMarkerMask = "marker_mask"+ NameQType = "qtype"+ NameLogits = "logits"+ NameActLogits = "act_logits"+)// Batch is the graph's five inputs, each flattened row-major, as onnxruntime tensors take them.type Batch struct {@@ -24,9 +36,12 @@ type Output struct {Logits, ActLogits []float32}-// Session runs one collated batch. Implementations must be safe for concurrent Run.+// Session runs one collated batch. Implementations must be safe for concurrent Run. The agent+// never calls Close concurrently with Run.type Session interface {// Run returns the logits for b. It should return ctx.Err() if ctx ends before the run does.+ // Run must not modify b's slices or retain them after it returns. The returned Output's+ // slices belong to the caller, and the session must not reuse them.Run(ctx context.Context, b Batch) (Output, error)// Close releases the runtime. It is idempotent; Run after Close returns ErrClosed.Close() error@@ -34,7 +49,7 @@ type Session interface {// ShapeError is returned when a slice's length does not match the batch it belongs to.type ShapeError struct {- // Name is the tensor: "input_ids", "logits" and so on.+ // Name is the tensor: NameInputIDs, NameLogits and so on.Name string// Got is the slice's length, Want the length the batch's dimensions call for.Got, Want int@@ -42,32 +57,52 @@ type ShapeError struct {// Error names the tensor and both lengths.func (e *ShapeError) Error() string {- return fmt.Sprintf("laya: %s has %d values, want %d", e.Name, e.Got, e.Want)+ return fmt.Sprintf("infer: %s has %d values, want %d", e.Name, e.Got, e.Want)}-// Validate reports a *ShapeError when a slice's length disagrees with Rows, Length and Markers,-// and a plain error when a dimension is negative or Markers is below 2.+// Validate checks, in order, that:+// - no dimension is negative;+// - a batch with rows has Length of at least 1 and Markers of at least 2, since the exported+// head takes topk(2);+// - Rows*Length and Rows*Markers do not overflow an int (Markers of at least 2 keeps Rows*2 safe too);+// - every slice's length matches Rows, Length and Markers, else a *ShapeError;+// - every MarkerPos lies in [0, Length).+//+// All but the ShapeError are plain errors.func (b Batch) Validate() error {if b.Rows < 0 || b.Length < 0 || b.Markers < 0 {- return fmt.Errorf("laya: batch has negative dimensions %d x %d x %d", b.Rows, b.Length, b.Markers)+ return fmt.Errorf("infer: batch has negative dimensions %d x %d x %d", b.Rows, b.Length, b.Markers)}- if b.Rows > 0 && b.Markers < 2 {- return fmt.Errorf("laya: batch has %d marker slots, the graph needs at least 2", b.Markers)+ if b.Rows > 0 {+ if b.Length == 0 {+ return fmt.Errorf("infer: batch has %d rows of 0 tokens", b.Rows)+ }+ if b.Markers < 2 {+ return fmt.Errorf("infer: batch has %d marker slots, the graph needs at least 2", b.Markers)+ }+ if b.Length > math.MaxInt/b.Rows || b.Markers > math.MaxInt/b.Rows {+ return fmt.Errorf("infer: batch dimensions %d x %d x %d overflow", b.Rows, b.Length, b.Markers)+ }}for _, c := range []struct {name stringgot, want int}{- {"input_ids", len(b.InputIDs), b.Rows * b.Length},- {"attention_mask", len(b.AttentionMask), b.Rows * b.Length},- {"marker_pos", len(b.MarkerPos), b.Rows * b.Markers},- {"marker_mask", len(b.MarkerMask), b.Rows * b.Markers},- {"qtype", len(b.QType), b.Rows},+ {NameInputIDs, len(b.InputIDs), b.Rows * b.Length},+ {NameAttentionMask, len(b.AttentionMask), b.Rows * b.Length},+ {NameMarkerPos, len(b.MarkerPos), b.Rows * b.Markers},+ {NameMarkerMask, len(b.MarkerMask), b.Rows * b.Markers},+ {NameQType, len(b.QType), b.Rows},} {if c.got != c.want {return &ShapeError{Name: c.name, Got: c.got, Want: c.want}}}+ for i, p := range b.MarkerPos {+ if p < 0 || p >= int64(b.Length) {+ return fmt.Errorf("infer: marker_pos[%d] is %d, outside [0, %d)", i, p, b.Length)+ }+ }return nil}@@ -75,10 +110,10 @@ func (b Batch) Validate() error {// for b.func (o Output) Check(b Batch) error {if len(o.Logits) != b.Rows*b.Markers {- return &ShapeError{Name: "logits", Got: len(o.Logits), Want: b.Rows * b.Markers}+ return &ShapeError{Name: NameLogits, Got: len(o.Logits), Want: b.Rows * b.Markers}}if len(o.ActLogits) != b.Rows*2 {- return &ShapeError{Name: "act_logits", Got: len(o.ActLogits), Want: b.Rows * 2}+ return &ShapeError{Name: NameActLogits, Got: len(o.ActLogits), Want: b.Rows * 2}}return nil}diff --git a/laya-go/infer/infer_test.go b/laya-go/infer/infer_test.goindex 34f54cd3..fd7d8c19 100644--- a/laya-go/infer/infer_test.go+++ b/laya-go/infer/infer_test.go@@ -2,6 +2,8 @@ package infer_testimport ("errors"+ "math"+ "strings""testing""github.com/NandhaKishorM/laya/laya-go/infer"@@ -31,7 +33,18 @@ func TestBatchValidate(t *testing.T) {{"short marker_mask", func(b *infer.Batch) { b.MarkerMask = b.MarkerMask[:1] }, "marker_mask"},{"short qtype", func(b *infer.Batch) { b.QType = nil }, "qtype"},{"one marker slot", func(b *infer.Batch) { *b = batch(3, 4, 1) }, "other"},+ {"no marker slots", func(b *infer.Batch) { *b = batch(3, 4, 0) }, "other"},{"negative rows", func(b *infer.Batch) { b.Rows = -1 }, "other"},+ {"negative length", func(b *infer.Batch) { b.Length = -1 }, "other"},+ {"negative markers", func(b *infer.Batch) { b.Markers = -2 }, "other"},+ {"rows of no tokens", func(b *infer.Batch) { *b = batch(3, 0, 2) }, "other"},+ {"marker position below zero", func(b *infer.Batch) { b.MarkerPos[2] = -1 }, "other"},+ {"marker position at the length", func(b *infer.Batch) { b.MarkerPos[5] = 4 }, "other"},+ {"marker position in range", func(b *infer.Batch) { b.MarkerPos[5] = 3 }, ""},+ // 2 * 2^62 wraps to the most negative int, which no slice length equals.+ {"rows x length overflows", func(b *infer.Batch) { *b = infer.Batch{Rows: 2, Length: math.MaxInt/2 + 1, Markers: 2} }, "overflow"},+ {"rows x markers overflows", func(b *infer.Batch) { *b = infer.Batch{Rows: 4, Length: 1, Markers: math.MaxInt/2 + 1} }, "overflow"},+ {"rows x length wraps to zero", func(b *infer.Batch) { *b = infer.Batch{Rows: math.MaxInt/2 + 1, Length: 2, Markers: 2} }, "overflow"},}for _, c := range cases {t.Run(c.name, func(t *testing.T) {@@ -42,9 +55,11 @@ func TestBatchValidate(t *testing.T) {switch {case c.field == "" && err != nil:t.Fatalf("Validate = %v, want nil", err)+ case c.field == "overflow" && (err == nil || errors.As(err, &se) || !strings.Contains(err.Error(), "overflow")):+ t.Fatalf("Validate = %v, want an overflow error", err)case c.field == "other" && (err == nil || errors.As(err, &se)):t.Fatalf("Validate = %v, want a plain error", err)- case c.field != "" && c.field != "other" && (!errors.As(err, &se) || se.Name != c.field):+ case c.field != "" && c.field != "other" && c.field != "overflow" && (!errors.As(err, &se) || se.Name != c.field):t.Fatalf("Validate = %v, want a ShapeError for %s", err, c.field)}})@@ -70,6 +85,25 @@ func TestOutputCheck(t *testing.T) {}}+func TestMessagesAndNamesCarryThePackage(t *testing.T) {+ for _, err := range []error{+ infer.ErrClosed, &infer.ShapeError{Name: infer.NameLogits, Got: 1, Want: 2},+ infer.Batch{Rows: -1}.Validate(), batch(1, 1, 1).Validate(),+ } {+ if !strings.HasPrefix(err.Error(), "infer: ") {+ t.Errorf("%q lacks the package prefix", err)+ }+ }+ for got, want := range map[string]string{+ infer.NameInputIDs: "input_ids", infer.NameAttentionMask: "attention_mask", infer.NameMarkerPos: "marker_pos",+ infer.NameMarkerMask: "marker_mask", infer.NameQType: "qtype", infer.NameLogits: "logits", infer.NameActLogits: "act_logits",+ } {+ if got != want {+ t.Errorf("tensor name %q, want %q", got, want)+ }+ }+}+// TestCollateBatchIsInferBatch pins the alias: a Batch the collator builds is a valid infer.Batch.func TestCollateBatchIsInferBatch(t *testing.T) {b, err := collate.Collate([]collate.Row{@@ -84,3 +118,12 @@ func TestCollateBatchIsInferBatch(t *testing.T) {t.Fatalf("Validate = %v on %+v", err, ib)}}++// TestValidateNamesTheEmptyRow pins that rows of no tokens are refused for that reason, not as a+// side effect of the marker range check.+func TestValidateNamesTheEmptyRow(t *testing.T) {+ err := batch(3, 0, 2).Validate()+ if err == nil || !strings.Contains(err.Error(), "0 tokens") {+ t.Fatalf("Validate = %v", err)+ }+}diff --git a/laya-go/internal/collate/collate.go b/laya-go/internal/collate/collate.goindex e33eabdc..baccac14 100644--- a/laya-go/internal/collate/collate.go+++ b/laya-go/internal/collate/collate.go@@ -1,5 +1,5 @@// Package collate pads built sequences into the rectangular tensors the graph takes. The-// tensor layout is the ONNX graph's contract, not a caller's, so this stays internal.+// tensor layout is the ONNX graph's contract; callers see the result as an infer.Batch.//// One row per question, not per state: four questions about one document are four rows that// share a state. Rows are padded to the longest sequence and markers to the most options.diff --git a/laya-go/scripts/gen_go_fixtures.py b/laya-go/scripts/gen_go_fixtures.pyindex 600b8eb9..fc64125f 100644--- a/laya-go/scripts/gen_go_fixtures.py+++ b/laya-go/scripts/gen_go_fixtures.py@@ -961,13 +961,37 @@ def build():EXP_DEPENDENT = ("label_answers", "answers_fuzz")+def _unnumber(value):+ """value with every number replaced by 0, so only its structure is left."""+ if isinstance(value, bool):+ return value+ if isinstance(value, (int, float)):+ return 0+ if isinstance(value, dict):+ return {k: _unnumber(v) for k, v in value.items()}+ if isinstance(value, list):+ return [_unnumber(v) for v in value]+ return value+++def _answers_shape(answers):+ """Each answer's type, key sets and order, legend and ids; no number, and no chosen value."""+ out = {}+ for question_id, answer in answers.items():+ shaped = _unnumber(answer)+ if "choice" in shaped:+ shaped["choice"] = None+ out[question_id] = shaped+ return out++def portable(payload):out = dict(payload)for name in EXP_DEPENDENT:out[name] = {"cases": [{k: v for k, v in case.items() if k != "answer_text"}for case in payload[name]["cases"]]}out["predict_plan"] = {- name: ({"cases": [{**case, "results": [{k: v for k, v in r.items() if k != "answers"}+ name: ({"cases": [{**case, "results": [{**r, "answers": _answers_shape(r["answers"])}for r in case["results"]]}for case in value["cases"]]} if name != "questions" else value)for name, value in payload["predict_plan"].items()}