Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

13.7 KB
Raw Blame

laya-client

A dependency-free HTTP client for a self-hosted Laya laya-serve /v1/systemone endpoint. Supports ESM and CommonJS on Node.js 22+, and browsers with fetch, AbortController, and structuredClone. TypeScript consumers need TypeScript 5 or newer.

Inference runs in your self-hosted laya-serve process. JavaScript clients do not need Python, PyTorch, or model weights installed; the server does. This package does not run models directly in a browser or Node.js.

Run from this repository

In a terminal at the repository root, start the server:

python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[serve]'
LAYA_HOST=127.0.0.1 LAYA_MODELS=english laya-serve

In a second terminal, build the SDK and run the plain JavaScript example:

cd sdk/typescript
npm ci
npm run build
node examples/triage.mjs

The server preloads checkpoints at startup by default. The command above loads the English checkpoint; the router can load others when needed. To preload both English and multilingual on CPU:

LAYA_HOST=127.0.0.1 LAYA_MODELS=english,multilingual LAYA_DEVICE=cpu laya-serve
# Set LAYA_DEVICE=cuda or mps for a suitable GPU installation.

Set LAYA_PRELOAD=0 to defer loading until the first prediction. Initial loads download weights and need sufficient memory and disk space. The client timeout defaults to 120 seconds; the example allows 10 minutes for a cold load. Server configuration uses environment variables, not command-line flags. See the server guide.

Install in another JavaScript project

After the first release is published, install laya-client with:

npm install laya-client

You can also install the locally built tarball before publication:

# From sdk/typescript:
npm pack

# From your app, substitute the actual checkout path:
npm install /path/to/laya/sdk/typescript/laya-client-0.1.0.tgz
import { Laya, triageQuestions } from 'laya-client';

const laya = new Laya({ baseURL: 'http://127.0.0.1:8000' });
const result = await laya.predict(
  { message: 'I was charged twice. Please refund the duplicate.' },
  triageQuestions(),
);

console.log(result.answers.intent.choice);
console.log(result.answers.refund_requested.noul); // P(true)
console.log(result.routing?.model); // Laya-only metadata

Choice and score answers may also contain x_jev_confidence, the HTTP server's Jev-compatible confidence measure. It is distinct from Laya's confidence and answer_confidence, and is absent on older servers, in strict Jev mode, and on noul answers. The client preserves it and checks that it is a finite number in [0, 1] when present; it does not compute or substitute it when absent.

CommonJS works too:

const { Laya, triageQuestions } = require('laya-client');
const laya = new Laya();
laya.predict({ message: 'Please refund my order' }, triageQuestions())
  .then(result => console.log(result.answers));

Typed custom questions

Question IDs, primitive types, and choice labels are inferred. Use defineQuestions to preserve literal types when storing a schema in a variable. Inline schemas also infer automatically.

import { Laya, defineQuestions } from 'laya-client';

const laya = new Laya();
const questions = defineQuestions({
  department: {
    type: 'choice',
    instructions: 'Which department should handle this request?',
    criteria: {
      billing: 'invoices, payments, refunds',
      technical: 'bugs, outages, integrations',
    },
  },
  urgency: {
    type: 'score',
    instructions: 'How urgent is this request?',
    criteria: ['no time pressure', 'soon', 'blocking issue or deadline'],
  },
  refund: { type: 'noul', instructions: 'Does the user ask for a refund?' },
});

const result = await laya.predict({ body: 'Refund the duplicate charge today' }, questions);
result.answers.department.choice; // 'billing' | 'technical'
result.answers.urgency.score;      // number: expected rubric index, from 0 to 2
result.answers.refund.noul;        // number: P(true), from 0 to 1
result.answers.department.probabilities.billing;
result.answers.department.confidence;
result.answers.department.answer_confidence; // number: max(p), the same quantity on every type
result.answers.urgency.abstention;           // 'passed' | 'abstained' | 'unevaluated', with minConfidence
result.answers.urgency.low_confidence;       // true only on an answer that fell below it
result.answers.department.action?.act_probability; // Laya-only metadata
result.usage.input_tokens;
result.usage.truncated;              // boolean: the state did not fit the room left for it
result.usage.truncated_questions;    // string[]: the questions whose own window was cut

Choice criteria also accept an array of unique labels, serialized as an option-to-null map for the shared protocol. Structured JSON values are supported in instructions and descriptions, as in Python. State may be text, a JSON object, an array (including chat turns), or null. Undefined values, dates, non-finite numbers, BigInts and cycles are rejected locally instead of silently changing during serialization. Empty questions or criteria are rejected.

Response fields retain Python's spelling (input_tokens, act_probability, etc.). Scores use zero-based rubric levels. Confidence and accuracy have the same calibration limits as the Python model; see the root README.

usage is the whole record /v1/systemone reports: input_tokens, output_tokens (always 0), state_tokens, state_tokens_dropped, truncated and truncated_questions. How much of a state the model reads is a token budget that moves with max_len, head_max_len and each question's own option prompt, so these are the only fields that show a cut: the answer itself is unchanged when evidence is dropped. options is added only when some question's options no longer have a token span each. See the usage table in docs/http-api.md for what each key measures.

Local model selection

By default, laya-client sends no model property. laya-serve then chooses the appropriate local checkpoint automatically. Set a client-wide model or override it per prediction only when a specific local checkpoint is required:

const laya = new Laya({ model: 'english' });
await laya.predict(state, questions, { model: 'typed-decisions' });
await laya.predict(state, questions, { model: 'multilingual' });

Laya accepts its local checkpoint aliases. The HTTP endpoint forwards the per-request controls documented in the HTTP API reference, and this client exposes them as options on predict():

option wire field meaning
model model checkpoint name or alias for this request; a path or unpublished Hub id is a 422 from the server
task task force a workflow; an unknown name is a 422 naming it
lang lang a language code that skips detection when it names a language
langGuess lang_guess a code from your own LID, consulted before detection
maxLen max_len total token window for this request (server-capped)
headMaxLen head_max_len token window the option prompt shares, same cap
minConfidence min_confidence abstention gate: one threshold in [0, 1], or a per-bucket map of option-count bucket to threshold; low-confidence answers are marked, not dropped
await laya.predict(state, questions, { task: 'typed', minConfidence: 0.8 });
await laya.predict(state, questions, { lang: 'de', maxLen: 4096, headMaxLen: 512 });

Absent options stay absent, so the deployment's own Router(...) settings remain in charge. The endpoint does not expose a standalone route() call. Laya still detects language automatically; set LAYA_AUTO_TASK=1 on the server to enable automatic workflow routing.

Laya returns model, answers, and usage, with optional routing, answer action, and Noul confidence metadata. Those fields are validated when present. Choice and Score confidence remains required. Every answer also carries answer_confidence (max(p)), the one confidence that means the same thing on all three question types; when you pass minConfidence, each answer of that call additionally reports abstention and abstention_threshold, and the ones below the threshold carry low_confidence: true. With minConfidence unset none of those three keys appear — absence is how a caller tells an ungated run from a cleared gate. health() is never called by predict().

Which JavaScript client?

Use laya-client when a JavaScript or TypeScript application talks over HTTP to self-hosted Python laya-serve. Use laya-ts when inference must run directly inside JavaScript through its local ONNX runtime, without a Python server.

Presets

Available preset functions: triageQuestions(), emailQuestions(categories?), guardQuestions(), moderationQuestions(), and routerQuestions(). Each returns a fresh schema. Email categories can be customized:

import { emailQuestions } from 'laya-client';
const questions = emailQuestions({ finance: 'payments', engineering: 'bugs' });

Errors, cancellation, authentication

import { Laya, LayaAPIError, LayaTimeoutError } from 'laya-client';

const client = new Laya({
  baseURL: 'http://127.0.0.1:8000',
  apiKey: process.env.LAYA_API_KEY,
  timeoutMs: 120_000,
});
const controller = new AbortController();
try {
  await client.predict(state, questions, { signal: controller.signal, timeoutMs: 300_000 });
} catch (error) {
  if (error instanceof LayaAPIError) console.error(error.status, error.code, error.details);
  else if (error instanceof LayaTimeoutError) console.error('Request timed out');
  else throw error;
}

timeoutMs: 0 disables the deadline. Calling controller.abort() stops waiting; inference already running on the server may still finish. Requests are never automatically retried. Other errors extend LayaError: LayaValidationError, LayaConnectionError, LayaAbortError, and LayaResponseError (invalid JSON or incompatible response structure).

The client accepts custom headers and a Fetch-compatible fetch implementation. Keep API keys on your application's backend. For browser use, put a same-origin application proxy in front of a protected server, or configure an appropriate authenticated gateway. A shared server key embedded in browser code is public.

Laya's server binds to 0.0.0.0 by default; the quickstart sets LAYA_HOST=127.0.0.1 for local use. Set LAYA_API_KEY in its environment to require Bearer authentication for predictions. /health remains public for liveness; loaded and device need the bearer. HF_TOKEN, if needed for checkpoint downloads, is a separate server-only credential. Configure CORS and TLS at your application proxy or gateway.

Python service contract

Endpoint Request Response
GET /health None for liveness {status}, plus loaded and device with the bearer
POST /v1/systemone {state, questions, model?} {model, answers, usage} plus optional routing

Laya errors use FastAPI's {detail: ...} body: 400 for malformed requests, 401 for authentication, 413 for request limits, 422 for invalid questions, and 500 for inference failures. The SDK preserves the status and details in LayaAPIError; string details become the error message. It also accepts {error: {code, message, details?}} from compatible services. OpenAPI is available at /openapi.json and interactive request documentation at /docs.

For custom checkpoint paths or an existing agent, configure Python's router:

import uvicorn
from laya import Router
from laya.serve import create_app

router = Router(models={"english": "/path/to/checkpoint"}, device="cpu")
router.preload(["english"])
app = create_app(router)  # Reads LAYA_API_KEY from the environment.
uvicorn.run(app, host="127.0.0.1", port=8000)

This SDK uses the repository's existing server without changing its Python implementation. The integration fixture injects a router into that same server.

Development

# In sdk/typescript:
npm ci
npm test
PYTHON=../../.venv/bin/python npm run test:integration
python3 scripts/sync_presets.py --check

Integration testing uses real local PyTorch inference with a tiny random test checkpoint, compares answers against direct Python calls, and downloads no model weights. It verifies transport and runtime compatibility, not pretrained model accuracy. Run the repository's pretrained model tests separately when those weights are available. Presets are generated from laya/presets.py; regenerate with python3 scripts/sync_presets.py after editing that source.

Publishing

Publish laya-client from sdk/typescript using an npm account with publishing rights for that package name.

npm login --registry=https://registry.npmjs.org
npm whoami --registry=https://registry.npmjs.org
npm ci
npm test
PYTHON=../../.venv/bin/python npm run test:integration
python3 scripts/sync_presets.py --check
npm publish --dry-run
npm publish

publishConfig sets public access and the official npm registry. Publishing runs the build, type checks, and SDK tests through prepublishOnly; prepack rebuilds the distributable. Only dist/, this README, the license, and package metadata are included. Complete any authentication challenge npm presents locally.

The first version is 0.1.0. For later releases, run npm version patch --no-git-tag-version (or minor / major) here, then repeat the checks and publishing commands. Keep SDK versions independent of the Python package. npm does not allow republishing an already-used name/version pair.

After publication, verify the release and install it into a clean project:

npm view laya-client version --registry=https://registry.npmjs.org
npm install laya-client