Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

Merge pull request #601 from aashish254/fix-mcp-auto-task

Nandakishorde8900610d9d
diff --cc README.md
index bc023e05,93705ce7..7d0f74db
--- a/README.md
+++ b/README.md
@@@ -902,22 -805,10 +902,23 @@@ package
| `LAYA_PRELOAD` | `1` | Same as `laya.serve`: build the checkpoints at startup, not lazily |
| `LAYA_MODELS` | `english,multilingual` | Comma list to preload (serve contract). MCP difference: an empty value preloads `english,multilingual` so `typed-decisions` stays lazy; in `laya.serve` empty means every checkpoint |
| `LAYA_THREADS` | (torch default) | Same as `laya.serve`: cap torch intra-op threads for CPU inference; keep it at or below the physical core count |
+ | `LAYA_AUTO_TASK` | `0` | Same as `laya.serve`: `1` lets a request whose question ids match a typed-decisions workflow route to that checkpoint, which is then loaded on demand; it never joins the preload list |
The tools return structured JSON (answers with probabilities, routing metadata, device,
-`latency_ms`). `laya_shortlist` is the MCP form of [`predict_shortlist`](#honest-limits):
+`latency_ms`). `laya_predict_batch` and `laya_route_batch` are the MCP form of
+[`Router.predict_batch` / `route_batch`](#batch-mode-score-many-states-in-one-forward-pass):
+one tool call takes an array of `{state, questions, model?, lang?}` requests, routes them
+first, groups them by checkpoint, and shares forward passes between requests with the same
+question schema, returning the answers in input order. On 16 mixed-language tickets through
+the tool functions themselves, one batch call beat 16 `laya_predict` calls by **2.1-2.3x on
+MPS** (983-1082 ms -> 467-477 ms) and **~1.25x on CPU** (1861-2471 ms -> 1470-1911 ms), with
+**0/16 decision flips** (choice label, rounded score, noul sign) against the loop. Prefer it
+whenever a client has more than a few requests: each saved round trip is also an MCP
+request/response. `laya_decide` is the MCP form of [`laya.decide`](#schema-driven-decisions): it
+takes a JSON schema (enum choices, booleans, bounded integers) instead of hand-written
+questions and returns the decided `values` projected onto that schema -- enum member, integer
+level, boolean -- beside per-field `confidence` and `probabilities`, so a client that already
+knows the answer shape never parses an answer map by hand. `laya_shortlist` is the MCP form of [`predict_shortlist`](#honest-limits):
it shortlists a many-option choice question to its `k` most likely labels by embedding
similarity (mean-pooled from the answering checkpoint's own encoder, so no extra model is
downloaded), answers in one forward pass, and returns per-question shortlist metadata
diff --cc tests/test_mcp.py
index 44e27f6e,97c09e53..bf334fd2
--- a/tests/test_mcp.py
+++ b/tests/test_mcp.py
@@@ -1730,22 -629,9 +1796,23 @@@ test_model_forwarding(
test_shape()
test_question_forwarding()
test_shortlist()
+test_batch_validation()
+test_batch_predict()
+test_batch_route()
+test_decide()
+test_shortlist_lang_parity()
+test_shortlist_embed_cache()
+test_controls_validation()
+test_controls_predict()
+test_controls_route()
+test_controls_shortlist()
+test_controls_preset()
+test_controls_signature_and_schema()
+test_question_validation_matches_the_agent()
+test_a_bad_question_is_a_caller_error_not_a_server_fault()
test_timeout_removed()
test_models_from_env()
+ test_auto_task_env()
test_server_registration()
print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))