Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

8.6 KB
Raw Blame

Python HTTP client

A synchronous, standard-library client for a self-hosted laya-serve. The application process does not install or import laya, PyTorch, Transformers, or NumPy. Use the core laya package when inference must run in your own process.

Install from this checkout

This distribution is prepared as laya-client; it has not been published to PyPI. In the application environment, from the repository root:

python -m pip install ./sdk/python

This uses setuptools to build the package. The installed SDK has no runtime dependencies and supports Python 3.10–3.13.

Run the server in a separate environment using the existing self-hosting instructions. Installing laya[serve] in the application environment would install inference dependencies again.

Predict

import os
from laya_client import LayaClient, LayaAPIError, LayaTimeoutError

client = LayaClient(
    base_url="http://127.0.0.1:8000",
    api_key=os.environ.get("LAYA_API_KEY"),
    timeout=120.0,
)
questions = {
    "team": {
        "type": "choice",
        "instructions": "Which team should handle this message?",
        "criteria": {"billing": "Payments and refunds", "technical": "Product support"},
    },
}
try:
    result = client.predict({"message": "I was charged twice"}, questions)
    print(result["answers"]["team"]["choice"])
except LayaAPIError as error:
    print(error.status_code, error.detail)
except LayaTimeoutError:
    print("Stopped waiting; the server may still be processing the request.")

predict(state, questions, *, model=None, task=None, lang=None, lang_guess=None, max_len=None, head_max_len=None, min_confidence=None, timeout=None) calls POST /v1/systemone. State is text, a JSON object, or an array; questions use the existing HTTP contract. Choice, score and noul answers are returned as ordinary dictionaries. Types are available in laya_client.types.

Controls left as None are omitted so server defaults remain in effect. In particular, model is unset by default. Explicit model names, workflow names, language hints and threshold-map keys are resolved by the server. The client does not duplicate its model registry or impose its configurable request limits. Question extensions such as option_order pass through unchanged.

Results retain server extensions and unknown JSON fields. Strict Jev responses are accepted without Laya metadata. If a strict server omits abstention, this does not mean a requested threshold passed. Use non-strict responses when your application needs that information. Tournament probabilities may contain zeros for eliminated options; the reported confidence still belongs to the final round and is not recalculated by the client.

client.health() calls GET /health. Even a server with an API key can return {"status": "ok"} for an anonymous or incorrectly authenticated probe. Liveness does not prove authentication or that checkpoints have been loaded.

Connection and error behavior

  • URLs may include a reverse-proxy prefix, e.g. https://host.example/laya. Use ASCII URLs (percent-encode paths and use IDNA hostnames where necessary), without credentials, whitespace, query strings or fragments.
  • The optional API key is a printable ASCII bearer token without whitespace. It is passed explicitly; the SDK does not read environment variables itself.
  • TLS certificate verification and Python's standard proxy settings remain enabled. Redirects are refused: configure the final server URL.
  • Calls block the calling thread. Timeout is in seconds, positive and finite, subject to the platform timeout limit. A per-call None inherits the client's 120-second default. This is a blocking I/O timeout, not a total request deadline or a server cancellation mechanism.
  • Requests are never automatically retried, including on 503 or timeout.
  • There is no connection pool or open client session to close. Each response is closed before the method returns or raises.

All client errors inherit LayaError:

Exception Meaning
LayaValidationError Invalid settings or a request rejected locally before sending
LayaAPIError Non-2xx response; status_code and detail preserve HTTP diagnostics
LayaConnectionError Connection, DNS, TLS or interrupted-response failure
LayaTimeoutError A blocking operation timed out; also a LayaConnectionError
LayaResponseError A 2xx response contained invalid JSON or required prediction fields

HTTP error details can be strings, objects or arrays. Error bodies are read up to 64 KiB; bodies that cannot be decoded as JSON are returned as text. The SDK does not log requests or credentials; applications should handle server diagnostics according to their own logging policy.

The client exposes predict, predict_batch and health. It does not implement async, local routing, model loading or Python prediction hooks.

Batch predictions

predict_batch sends one POST /v1/systemone/batch request with a nonempty list of states and one shared question set:

states = ["Please refund my order", {"message": "Where is my order?"}]
questions = {
    "refund": {"type": "noul", "instructions": "Is the customer requesting a refund?"},
}
batch = client.predict_batch(states, questions)
for state, result in zip(states, batch["results"]):
    print(state, result["answers"]["refund"]["noul"])
print(batch["total_usage"])

The return type is laya_client.types.BatchPrediction, an ordinary dictionary with results: list[Prediction] and total_usage: Usage. Result at index i belongs to state at index i; the client preserves server order and unknown fields. It requires exactly one valid prediction per state and nonnegative integer input_tokens and output_tokens in total_usage. An invalid envelope, count or individual result raises LayaResponseError; no partial list is returned.

All predict controls apply to the whole batch. The additional optional batch_size (positive integer) and sort_by_length (boolean) control server-side inference batching, not HTTP request splitting. Length sorting still returns results in input order. Changing inference batch shapes can change floating-point results; the client does not recalculate predictions or confidence.

Omitted controls inherit server settings. Empty lists and invalid JSON inputs raise LayaValidationError before sending. Server limits remain authoritative: the current server accepts at most 64 states, with its existing per-state, question, token-budget and request-body limits. A larger list is sent once and the server's refusal (normally HTTP 413) is exposed as LayaAPIError.

The batch endpoint fails the whole request; there is no per-item error contract. On a server without this endpoint, HTTP 404/405 raises LayaAPIError with the original status and detail. The SDK never falls back to individual requests, splits a request, or retries it, including after a timeout. Existing predict and health behavior is unchanged.

Development and verification

From the repository root, in a build environment using a current patched Python:

python -m pip install build setuptools wheel
python -m build --no-isolation sdk/python
python sdk/python/tests/verify_package.py sdk/python/dist

The verifier builds an isolated client environment, installs the wheel with normal dependency resolution and no package index, then runs the standard-library tests against that installed package. It also checks the sdist-built wheel. Source archives are extracted with tarfile's data filter and must contain only regular files and directories within the extraction directory. The verifier refuses to extract on older Python builds without tarfile.data_filter; this requirement applies to the verification tool, not the installed client's runtime.

For live integration, supply a Python with laya[serve] dependencies installed:

python sdk/python/tests/verify_package.py sdk/python/dist --server-python /path/to/server/python

A tiny offline checkpoint exercises actual HTTP inference. The client process has only the SDK installed. This verifies transport and inference parity, not pretrained model quality. CI exercises the current server and the 0.4.7 fixture; version compatibility is determined by those checks, not by a version handshake.

Publishing

Package ownership, versioning and PyPI publication are maintained separately from the core distribution. Build and inspect the wheel/sdist before release. The core release workflow is not changed by this SDK.