Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

fix(ts): stringify score legend values and document serve choice limit (#302)

Swatantra Yadav655a4821aa82
diff --git a/README.md b/README.md
index 54c20cf9..c5045ed2 100644
--- a/README.md
+++ b/README.md
@@ -464,7 +464,7 @@ the router auto-selects by script/language.
Three things differ from Jev when you port a client:
-* **Options per question.** A question's options share the checkpoint's option budget, `head_max_len` (192 tokens on `laya`, 256 on the other two), not Jev's cap of 255 options. Once they overflow it, around 20 options with a short description each, every option is trimmed to fit, so long or similar labels can reach the model reading the same ([Where Jev leads](#where-jev-leads)). Once they no longer fit the window at all, the request is rejected with 422. With short labels such as `Queue 042: Tickets routed to queue 42` that happens above 126 options on `laya` and 254 on the other two; the exact point moves with the length of the instructions and labels. For more candidates, narrow them first with `predict_shortlist` ([Honest limits](#honest-limits)).
+* **Options per question.** A question's options share the checkpoint's option budget, `head_max_len` (192 tokens on `laya`, 256 on the other two), not Jev's cap of 255 options. In addition, the HTTP server (`laya.serve`) enforces an amplification guard of at most 100 choice options per question (`MAX_CHOICE_OPTIONS = 100`, rejected with 413 before inference). Once options overflow the token budget, around 20 options with a short description each, every option is trimmed to fit, so long or similar labels can reach the model reading the same ([Where Jev leads](#where-jev-leads)). Once they no longer fit the window at all, the library rejects the request with 422. For more candidates, narrow them first with `predict_shortlist` ([Honest limits](#honest-limits)).
* **Score levels.** Every level needs a description. A `null` level is rejected with 422 rather than scored and echoed back in `legend`.
* **`confidence`** on `choice` and `score` answers is 1 minus normalised entropy, a measure of how concentrated the distribution is, not Jev's `(n·p_max − 1)/(n − 1)`. A threshold carried over from Jev does not transfer. For one calibrated number on every question type, gate on `answer_confidence`, the probability of the reported answer.
diff --git a/laya-ts/src/agent.ts b/laya-ts/src/agent.ts
index f5bc8e69..a563f6f6 100644
--- a/laya-ts/src/agent.ts
+++ b/laya-ts/src/agent.ts
@@ -51,7 +51,7 @@ export interface ChoiceAnswer {
export interface ScoreAnswer {
type: "score";
score: number;
- legend: Record<string, unknown>;
+ legend: Record<string, string>;
probabilities: Record<string, number>;
confidence: number;
answer_confidence: number;
@@ -487,7 +487,7 @@ export class Agent extends HookRegistry {
answers[qid] = {
type: "score",
score: r4(exp),
- legend: Object.fromEntries((q.crit as unknown[]).map((c, i) => [String(i), c])),
+ legend: Object.fromEntries((q.crit as unknown[]).map((c, i) => [String(i), String(c)])),
probabilities: Object.fromEntries(p.map((v, i) => [String(i), r4(v)])),
confidence: r4(confidenceFromProbs(p)),
answer_confidence: ansConf,
diff --git a/laya-ts/tests/agent.test.ts b/laya-ts/tests/agent.test.ts
index 76c810fd..583c0d01 100644
--- a/laya-ts/tests/agent.test.ts
+++ b/laya-ts/tests/agent.test.ts
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
import { Agent } from "../src/agent.js";
const fakeProvider = () => ({
async runEncoder(_b: any) { return { lastHidden: [[1, 0], [0, 1]] }; },
- async runHead(_h: any) { return { logits: [[2, 0]], act: [[3, 0]] }; },
+ async runHead(_h: any) { return { logits: [[2, 0, 1, 0]], act: [[3, 0]] }; },
});
describe("agent", () => {
it("empty questions skip forward pass", async () => {
@@ -21,4 +21,14 @@ describe("agent", () => {
const a = new Agent({ provider: fakeProvider() } as any);
await expect(a.systemOne("hi", { q: { type: "nope" } as any })).rejects.toThrow('question "q"');
});
+ it("score legend stringifies non-string levels (#302)", async () => {
+ const a = new Agent({ provider: fakeProvider() } as any);
+ const r: any = await a.systemOne("hi", {
+ s: { type: "score", instructions: "scale?", criteria: [1, 2, 3] as any } });
+ const ans = r.answers.s;
+ expect(ans.legend).toEqual({ "0": "1", "1": "2", "2": "3" });
+ for (const val of Object.values(ans.legend)) {
+ expect(typeof val).toBe("string");
+ }
+ });
});