Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(sdk): add a standalone Python HTTP client

xp1791e82be832
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index ed0b734c..3797c07a 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -270,10 +270,59 @@ jobs:
# second. Widen the second list to the first's rule set once that debt is paid.
run: |
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
+ ruff check sdk/python/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
ruff check laya-java/scripts/ laya-dotnet/tools/ laya-ts/scripts/ \
laya-go/scripts/ --select=E9,F63,F7,F82 --line-length=120
- name: Byte-compile every module
- run: python -m compileall -q laya/ tests/
+ run: python -m compileall -q laya/ tests/ sdk/python/
+
+ python-sdk:
+ name: Python SDK (${{ matrix.os }}, py${{ matrix.python }})
+ runs-on: ${{ matrix.os }}
+ timeout-minutes: 10
+ strategy:
+ fail-fast: false
+ matrix:
+ os: [ubuntu-latest]
+ python: ["3.10", "3.11", "3.12", "3.13"]
+ include:
+ - os: windows-latest
+ python: "3.11"
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: ${{ matrix.python }}
+ - name: Build standalone distributions
+ run: |
+ python -m pip install --upgrade build setuptools wheel
+ python -m build --no-isolation sdk/python
+ - name: Verify installed wheel and sdist without inference dependencies
+ run: python sdk/python/tests/verify_package.py sdk/python/dist
+
+ python-sdk-integration:
+ name: Python SDK (live HTTP)
+ runs-on: ubuntu-latest
+ timeout-minutes: 15
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+ with:
+ ref: 697d788e1c16e9cb7f736471897ab758a79c1df7 # v0.4.7
+ path: .sdk-server-0.4.7
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: "3.11"
+ - name: Install server and build tools
+ run: |
+ python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
+ python -m pip install -e '.[serve]' build setuptools wheel
+ python -m build --no-isolation sdk/python
+ - name: Real inference from an isolated client environment
+ run: |
+ python sdk/python/tests/verify_package.py sdk/python/dist \
+ --server-python "$(command -v python)" \
+ --legacy-server-fixture .sdk-server-0.4.7/tests/sdk_server_fixture.py
typescript-sdk:
name: TypeScript SDK (node${{ matrix.node }})
diff --git a/README.md b/README.md
index 01468027..bf3196d2 100644
--- a/README.md
+++ b/README.md
@@ -441,6 +441,15 @@ Measured on the running server: 64 real reviews of 91–2,293 characters, one ch
`batch_size: 8` on its own was 4.95s, so the length grouping is what earns the second factor:
1.66x over the sized batch.
+### Python HTTP client
+
+Applications that call a separate `laya-serve` can use the
+[Python HTTP client](https://github.com/NandhaKishorM/laya/blob/main/sdk/python/README.md)
+without installing the inference dependencies. It provides synchronous `predict()` and
+`health()` calls using the standard library. From this checkout, install it with
+`python -m pip install ./sdk/python` in the application environment; keep `laya[serve]`
+in the server environment. The separate `laya-client` distribution is not yet published to PyPI.
+
### JavaScript / TypeScript
[`laya-client`](https://github.com/NandhaKishorM/laya/blob/main/sdk/typescript/README.md) supports JavaScript and TypeScript
diff --git a/docs/http-api.md b/docs/http-api.md
index 999e4e44..6e3a22f2 100644
--- a/docs/http-api.md
+++ b/docs/http-api.md
@@ -5,6 +5,10 @@ against Jev -- `hs-jev`, `typesafe-sdk`, or your own -- can point its base URL a
keep working: Laya's `predict()` output is already schema-compatible, and the server adds only the
HTTP surface: one decision route, a health probe, an optional bearer check and request limits.
+The [Python HTTP client](https://github.com/NandhaKishorM/laya/blob/main/sdk/python/README.md)
+provides synchronous `predict` and `health` calls with no runtime dependencies. Install it
+from `sdk/python` in the application environment to keep the inference stack on the server.
+
A client that targets `laya-serve` rather than the Jev API exists for PHP:
[`marcreichel/laya-php`](https://github.com/marcreichel/laya-php) is a Composer SDK (PHP 8.4+)
that maps a class of enums and attributes onto questions and returns an instance, reads `GET /health`
diff --git a/sdk/python/LICENSE b/sdk/python/LICENSE
new file mode 100644
index 00000000..d9a10c0d
--- /dev/null
+++ b/sdk/python/LICENSE
@@ -0,0 +1,176 @@
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
diff --git a/sdk/python/README.md b/sdk/python/README.md
new file mode 100644
index 00000000..de32c1b4
--- /dev/null
+++ b/sdk/python/README.md
@@ -0,0 +1,138 @@
+# Python HTTP client
+
+A synchronous, standard-library client for a self-hosted `laya-serve`.
+The application process does not install or import `laya`, PyTorch, Transformers,
+or NumPy. Use the core `laya` package when inference must run in your own process.
+
+## Install from this checkout
+
+This distribution is prepared as `laya-client`; it has not been published to PyPI.
+In the application environment, from the repository root:
+
+```bash
+python -m pip install ./sdk/python
+```
+
+This uses setuptools to build the package. The installed SDK has no runtime
+dependencies and supports Python 3.10–3.13.
+
+Run the server in a **separate environment** using the existing
+[self-hosting instructions](https://github.com/NandhaKishorM/laya#self-hosting-http-server-jev-compatible).
+Installing `laya[serve]` in the application environment would install inference
+dependencies again.
+
+## Predict
+
+```python
+import os
+from laya_client import LayaClient, LayaAPIError, LayaTimeoutError
+
+client = LayaClient(
+ base_url="http://127.0.0.1:8000",
+ api_key=os.environ.get("LAYA_API_KEY"),
+ timeout=120.0,
+)
+questions = {
+ "team": {
+ "type": "choice",
+ "instructions": "Which team should handle this message?",
+ "criteria": {"billing": "Payments and refunds", "technical": "Product support"},
+ },
+}
+try:
+ result = client.predict({"message": "I was charged twice"}, questions)
+ print(result["answers"]["team"]["choice"])
+except LayaAPIError as error:
+ print(error.status_code, error.detail)
+except LayaTimeoutError:
+ print("Stopped waiting; the server may still be processing the request.")
+```
+
+`predict(state, questions, *, model=None, task=None, lang=None, lang_guess=None,
+max_len=None, head_max_len=None, min_confidence=None, timeout=None)` calls
+`POST /v1/systemone`. State is text, a JSON object, or an array; questions use
+the existing [HTTP contract](https://github.com/NandhaKishorM/laya/blob/main/docs/http-api.md).
+Choice, score and noul answers are returned as ordinary dictionaries. Types are
+available in `laya_client.types`.
+
+Controls left as `None` are omitted so server defaults remain in effect.
+In particular, `model` is unset by default. Explicit model names, workflow names,
+language hints and threshold-map keys are resolved by the server. The client
+does not duplicate its model registry or impose its configurable request limits.
+Question extensions such as `option_order` pass through unchanged.
+
+Results retain server extensions and unknown JSON fields. Strict Jev responses
+are accepted without Laya metadata. If a strict server omits `abstention`, this
+does **not** mean a requested threshold passed. Use non-strict responses when
+your application needs that information. Tournament probabilities may contain
+zeros for eliminated options; the reported confidence still belongs to the
+final round and is not recalculated by the client.
+
+`client.health()` calls `GET /health`. Even a server with an API key can return
+`{"status": "ok"}` for an anonymous or incorrectly authenticated probe. Liveness
+does not prove authentication or that checkpoints have been loaded.
+
+## Connection and error behavior
+
+- URLs may include a reverse-proxy prefix, e.g. `https://host.example/laya`.
+ Use ASCII URLs (percent-encode paths and use IDNA hostnames where necessary),
+ without credentials, whitespace, query strings or fragments.
+- The optional API key is a printable ASCII bearer token without whitespace.
+ It is passed explicitly; the SDK does not read environment variables itself.
+- TLS certificate verification and Python's standard proxy settings remain enabled.
+ Redirects are refused: configure the final server URL.
+- Calls block the calling thread. Timeout is in seconds, positive and finite,
+ subject to the platform timeout limit. A per-call `None` inherits the client's
+ 120-second default. This is a blocking I/O timeout, **not** a total request
+ deadline or a server cancellation mechanism.
+- Requests are never automatically retried, including on 503 or timeout.
+- There is no connection pool or open client session to close. Each response is
+ closed before the method returns or raises.
+
+All client errors inherit `LayaError`:
+
+| Exception | Meaning |
+| --- | --- |
+| `LayaValidationError` | Invalid settings or a request rejected locally before sending |
+| `LayaAPIError` | Non-2xx response; `status_code` and `detail` preserve HTTP diagnostics |
+| `LayaConnectionError` | Connection, DNS, TLS or interrupted-response failure |
+| `LayaTimeoutError` | A blocking operation timed out; also a `LayaConnectionError` |
+| `LayaResponseError` | A 2xx response contained invalid JSON or required prediction fields |
+
+HTTP error details can be strings, objects or arrays. Error bodies are read up to
+64 KiB; bodies that cannot be decoded as JSON are returned as text. The SDK does not log requests or credentials;
+applications should handle server diagnostics according to their own logging policy.
+
+The first version exposes `predict` and `health`. It does not implement async,
+batch, local routing, model loading or Python prediction hooks.
+
+## Development and verification
+
+From the repository root, in a build environment:
+
+```bash
+python -m pip install build setuptools wheel
+python -m build --no-isolation sdk/python
+python sdk/python/tests/verify_package.py sdk/python/dist
+```
+
+The verifier builds an isolated client environment, installs the wheel with
+normal dependency resolution and no package index, then runs the standard-library
+tests against that installed package. It also checks the sdist-built wheel.
+
+For live integration, supply a Python with `laya[serve]` dependencies installed:
+
+```bash
+python sdk/python/tests/verify_package.py sdk/python/dist --server-python /path/to/server/python
+```
+
+A tiny offline checkpoint exercises actual HTTP inference. The client process
+has only the SDK installed. This verifies transport and inference parity, not
+pretrained model quality. CI exercises the current server and the 0.4.7 fixture;
+version compatibility is determined by those checks, not by a version handshake.
+
+## Publishing
+
+Package ownership, versioning and PyPI publication are maintained separately
+from the core distribution. Build and inspect the wheel/sdist before release.
+The core release workflow is not changed by this SDK.
diff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.toml
new file mode 100644
index 00000000..5798ed9b
--- /dev/null
+++ b/sdk/python/pyproject.toml
@@ -0,0 +1,30 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "laya-client"
+version = "0.1.0"
+description = "Standard-library Python client for self-hosted Laya"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "Apache-2.0" }
+dependencies = []
+classifiers = [
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
+ "License :: OSI Approved :: Apache Software License",
+]
+
+[project.urls]
+Repository = "https://github.com/NandhaKishorM/laya"
+Issues = "https://github.com/NandhaKishorM/laya/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+laya_client = ["py.typed"]
diff --git a/sdk/python/src/laya_client/__init__.py b/sdk/python/src/laya_client/__init__.py
new file mode 100644
index 00000000..cde18570
--- /dev/null
+++ b/sdk/python/src/laya_client/__init__.py
@@ -0,0 +1,11 @@
+"""A standalone HTTP client; no local model or inference dependencies."""
+from .client import LayaClient
+from .errors import (
+ LayaAPIError, LayaConnectionError, LayaError, LayaResponseError,
+ LayaTimeoutError, LayaValidationError,
+)
+
+__all__ = [
+ "LayaClient", "LayaError", "LayaValidationError", "LayaAPIError",
+ "LayaConnectionError", "LayaTimeoutError", "LayaResponseError",
+]
diff --git a/sdk/python/src/laya_client/_validation.py b/sdk/python/src/laya_client/_validation.py
new file mode 100644
index 00000000..11e5c04c
--- /dev/null
+++ b/sdk/python/src/laya_client/_validation.py
@@ -0,0 +1,116 @@
+"""Transport checks, deliberately separate from the server's domain validation."""
+import math
+import threading
+from typing import Any
+
+from .errors import LayaResponseError, LayaValidationError
+
+
+def finite(value: Any) -> bool:
+ try:
+ return type(value) in (int, float) and math.isfinite(value)
+ except OverflowError:
+ return False
+
+
+def probability(value: Any) -> bool:
+ return finite(value) and 0 <= value <= 1
+
+
+def timeout_value(value: Any) -> float:
+ if not finite(value) or not 0 < value <= threading.TIMEOUT_MAX:
+ raise LayaValidationError("timeout must be positive, finite and within the platform timeout limit")
+ return float(value)
+
+
+def json_value(value: Any, ancestors=None) -> None:
+ if value is None or type(value) in (bool, int):
+ return
+ if isinstance(value, str):
+ value.encode("utf-8") # Reject unpaired surrogates before sending.
+ return
+ if type(value) is float and math.isfinite(value):
+ return
+ if not isinstance(value, (list, dict)):
+ raise LayaValidationError("request must contain only finite JSON values")
+ ancestors = set() if ancestors is None else ancestors
+ if id(value) in ancestors:
+ raise LayaValidationError("request contains a circular reference")
+ ancestors.add(id(value))
+ try:
+ if isinstance(value, dict):
+ for key in value:
+ if not isinstance(key, str):
+ raise LayaValidationError("JSON object keys must be strings")
+ key.encode("utf-8")
+ values = value.values()
+ else:
+ values = value
+ for item in values:
+ json_value(item, ancestors)
+ finally:
+ ancestors.remove(id(value))
+
+
+def questions_value(value: Any) -> None:
+ if not isinstance(value, dict) or not value:
+ raise LayaValidationError("questions must be a nonempty object")
+ for question in value.values():
+ if not isinstance(question, dict) or question.get("type") not in ("choice", "score", "noul"):
+ raise LayaValidationError("each question must have type choice, score or noul")
+ if "instructions" not in question:
+ raise LayaValidationError("each question requires instructions")
+
+
+def expect(condition: bool, field: str) -> None:
+ if not condition:
+ raise LayaResponseError("Invalid Laya response: " + field)
+
+
+def prediction(value: Any, questions: dict) -> None:
+ expect(isinstance(value, dict), "prediction object")
+ expect(isinstance(value.get("model"), str), "model")
+ answers, usage = value.get("answers"), value.get("usage")
+ expect(isinstance(answers, dict), "answers")
+ expect(isinstance(usage, dict), "usage")
+ for key in ("input_tokens", "output_tokens"):
+ expect(type(usage.get(key)) is int and usage[key] >= 0, "usage." + key)
+ if "routing" in value:
+ expect(isinstance(value["routing"], dict), "routing")
+ for qid, question in questions.items():
+ answer = answers.get(qid)
+ field = "answers." + qid
+ expect(isinstance(answer, dict) and answer.get("type") == question["type"], field + ".type")
+ kind = answer["type"]
+ for key in ("confidence", "answer_confidence", "x_jev_confidence", "abstention_threshold"):
+ if key in answer or (key == "confidence" and kind != "noul"):
+ expect(probability(answer.get(key)), field + "." + key)
+ if "low_confidence" in answer:
+ expect(type(answer["low_confidence"]) is bool, field + ".low_confidence")
+ if "abstention" in answer or "abstention_threshold" in answer:
+ # The gate reports a state and the threshold that produced it together.
+ expect(answer.get("abstention") in ("passed", "abstained", "unevaluated"), field + ".abstention")
+ expect(probability(answer.get("abstention_threshold")), field + ".abstention_threshold")
+ for key in ("action", "x_tournament"):
+ if key in answer:
+ expect(isinstance(answer[key], dict), field + "." + key)
+ if kind == "noul":
+ expect(probability(answer.get("noul")), field + ".noul")
+ continue
+ probs = answer.get("probabilities")
+ expect(isinstance(probs, dict) and bool(probs) and all(probability(p) for p in probs.values()),
+ field + ".probabilities")
+ criteria = question.get("criteria")
+ if kind == "choice":
+ expect(isinstance(answer.get("choice"), str) and answer["choice"] in probs, field + ".choice")
+ labels = list(criteria) if isinstance(criteria, dict) else criteria
+ if isinstance(labels, list) and all(isinstance(label, str) for label in labels):
+ expect(answer["choice"] in labels and all(label in probs for label in labels),
+ field + ".choice labels")
+ else:
+ expect(finite(answer.get("score")) and answer["score"] >= 0, field + ".score")
+ expect(isinstance(answer.get("legend"), dict), field + ".legend")
+ if isinstance(criteria, list):
+ expect(answer["score"] <= len(criteria) - 1, field + ".score")
+ expect(all(str(i) in probs and str(i) in answer["legend"] for i in range(len(criteria))),
+ field + ".rubric levels")
diff --git a/sdk/python/src/laya_client/client.py b/sdk/python/src/laya_client/client.py
new file mode 100644
index 00000000..ac80ca1b
--- /dev/null
+++ b/sdk/python/src/laya_client/client.py
@@ -0,0 +1,131 @@
+"""Synchronous requests to the existing self-hosted HTTP API."""
+import http.client
+import json
+import urllib.error
+import urllib.parse
+import urllib.request
+from typing import Any, Optional, cast
+
+from ._validation import expect, json_value, prediction, probability, questions_value, timeout_value
+from .errors import LayaAPIError, LayaConnectionError, LayaResponseError, LayaTimeoutError, LayaValidationError
+from .types import Health, MinConfidence, Prediction, Questions, State
+
+
+class _NoRedirect(urllib.request.HTTPRedirectHandler):
+ def redirect_request(self, req, fp, code, msg, headers, newurl):
+ return None
+
+
+def _invalid_constant(value):
+ raise ValueError("Non-finite JSON number")
+
+
+class LayaClient:
+ """Call a self-hosted Laya server without installing the inference engine.
+
+ timeout is a blocking I/O timeout in seconds, not a total request deadline.
+ Calls are synchronous. No inference request is retried automatically.
+ """
+
+ def __init__(self, *, base_url: str = "http://127.0.0.1:8000",
+ api_key: Optional[str] = None, timeout: float = 120.0):
+ try:
+ if (not isinstance(base_url, str) or any(ord(c) <= 32 or ord(c) == 127 for c in base_url)
+ or "?" in base_url or "#" in base_url):
+ raise ValueError()
+ base_url.encode("ascii")
+ url = urllib.parse.urlsplit(base_url)
+ if (url.scheme not in ("http", "https") or not url.hostname or url.username is not None
+ or url.password is not None or url.query or url.fragment):
+ raise ValueError()
+ url.port # Validate the port even though urllib would otherwise defer it.
+ except (ValueError, UnicodeError):
+ raise LayaValidationError(
+ "base_url must be an ASCII HTTP(S) URL without credentials, whitespace, query or fragment"
+ ) from None
+ if api_key is not None:
+ if not isinstance(api_key, str) or not api_key or any(ord(c) < 33 or ord(c) > 126 for c in api_key):
+ raise LayaValidationError("api_key must be a nonempty token containing printable ASCII without spaces")
+ self._base_url = base_url.rstrip("/")
+ self._api_key = api_key
+ self._timeout = timeout_value(timeout)
+
+ def predict(self, state: State, questions: Questions, *, model: Optional[str] = None,
+ task: Optional[str] = None, lang: Optional[str] = None, lang_guess: Optional[str] = None,
+ max_len: Optional[int] = None, head_max_len: Optional[int] = None,
+ min_confidence: Optional[MinConfidence] = None, timeout: Optional[float] = None) -> Prediction:
+ """Predict choice, score and noul answers; omitted controls inherit server settings."""
+ if not isinstance(state, (str, dict, list)):
+ raise LayaValidationError("state must be text, a JSON object or an array")
+ questions_value(questions)
+ body: dict[str, Any] = {"state": state, "questions": questions}
+ for key, value in (("model", model), ("task", task), ("lang", lang), ("lang_guess", lang_guess)):
+ if value is not None:
+ if not isinstance(value, str) or not value.strip():
+ raise LayaValidationError(key + " must be a nonempty string")
+ body[key] = value
+ for key, value in (("max_len", max_len), ("head_max_len", head_max_len)):
+ if value is not None:
+ if type(value) is not int or value <= 0:
+ raise LayaValidationError(key + " must be a positive integer")
+ body[key] = value
+ if min_confidence is not None:
+ valid_map = (isinstance(min_confidence, dict) and bool(min_confidence)
+ and all(isinstance(k, str) for k in min_confidence)
+ and all(probability(v) for v in min_confidence.values()))
+ if not probability(min_confidence) and not valid_map:
+ raise LayaValidationError("min_confidence must be a threshold in [0, 1] or a nonempty threshold map")
+ body["min_confidence"] = min_confidence
+ result = self._request("/v1/systemone", body, timeout)
+ prediction(result, questions)
+ return cast(Prediction, result)
+
+ def health(self, *, timeout: Optional[float] = None) -> Health:
+ """Read liveness; a minimal response does not prove authentication or model readiness."""
+ result = self._request("/health", None, timeout)
+ expect(isinstance(result, dict) and result.get("status") == "ok", "health")
+ return cast(Health, result)
+
+ def _request(self, path: str, body: Optional[dict], timeout: Optional[float]) -> Any:
+ seconds = self._timeout if timeout is None else timeout_value(timeout)
+ headers = {"Accept": "application/json"}
+ data = None
+ if body is not None:
+ try:
+ json_value(body)
+ data = json.dumps(body, allow_nan=False).encode("utf-8")
+ except (ValueError, TypeError, RecursionError) as exc:
+ raise LayaValidationError("request must contain finite, acyclic JSON values") from exc
+ headers["Content-Type"] = "application/json"
+ if self._api_key is not None:
+ headers["Authorization"] = "Bearer " + self._api_key
+ req = urllib.request.Request(self._base_url + path, data=data, headers=headers,
+ method="GET" if body is None else "POST")
+ try:
+ # A private opener avoids global changes; redirects must not replay a POST or forward its token.
+ with urllib.request.build_opener(_NoRedirect()).open(req, timeout=seconds) as response:
+ raw = response.read()
+ except urllib.error.HTTPError as exc:
+ with exc:
+ try:
+ raw = exc.read(65536)
+ try:
+ payload = json.loads(raw, parse_constant=_invalid_constant)
+ detail = payload.get("detail", payload) if isinstance(payload, dict) else payload
+ except (ValueError, UnicodeError, RecursionError):
+ detail = raw.decode("utf-8", errors="replace")
+ except (OSError, http.client.HTTPException):
+ detail = None
+ raise LayaAPIError(exc.code, detail) from None
+ except urllib.error.URLError as exc:
+ if isinstance(exc.reason, TimeoutError):
+ raise LayaTimeoutError("Laya request timed out; server inference may still finish") from exc
+ raise LayaConnectionError("Could not reach the Laya server") from exc
+ except TimeoutError as exc:
+ raise LayaTimeoutError("Laya request timed out; server inference may still finish") from exc
+ except (OSError, http.client.HTTPException) as exc:
+ raise LayaConnectionError("Laya response could not be read") from exc
+ try:
+ return json.loads(raw, parse_constant=_invalid_constant)
+ except (ValueError, UnicodeError, RecursionError) as exc:
+ raise LayaResponseError("Server returned invalid JSON") from exc
diff --git a/sdk/python/src/laya_client/errors.py b/sdk/python/src/laya_client/errors.py
new file mode 100644
index 00000000..5bb3918c
--- /dev/null
+++ b/sdk/python/src/laya_client/errors.py
@@ -0,0 +1,31 @@
+"""Failures callers can distinguish without importing the inference engine."""
+from typing import Any
+
+
+class LayaError(Exception):
+ """Base class for client errors."""
+
+
+class LayaValidationError(LayaError, ValueError):
+ """An invalid client setting or request, rejected before sending it."""
+
+
+class LayaAPIError(LayaError):
+ """A non-2xx HTTP response; detail preserves the server's JSON value."""
+
+ def __init__(self, status_code: int, detail: Any = None):
+ super().__init__("Laya returned HTTP %d" % status_code)
+ self.status_code = status_code
+ self.detail = detail
+
+
+class LayaConnectionError(LayaError):
+ """The request or response could not be transported."""
+
+
+class LayaTimeoutError(LayaConnectionError):
+ """A blocking I/O operation timed out; server inference may still finish."""
+
+
+class LayaResponseError(LayaError):
+ """A successful HTTP response did not satisfy the prediction contract."""
diff --git a/sdk/python/src/laya_client/py.typed b/sdk/python/src/laya_client/py.typed
new file mode 100644
index 00000000..8b137891
--- /dev/null
+++ b/sdk/python/src/laya_client/py.typed
@@ -0,0 +1 @@
+
diff --git a/sdk/python/src/laya_client/types.py b/sdk/python/src/laya_client/types.py
new file mode 100644
index 00000000..ea802700
--- /dev/null
+++ b/sdk/python/src/laya_client/types.py
@@ -0,0 +1,65 @@
+"""The HTTP payload's required fields and optional Laya extensions.
+
+Responses remain ordinary dictionaries. Unknown JSON fields are preserved.
+"""
+from typing import Dict, List, Literal, TypedDict, Union
+
+JsonValue = Union[None, bool, int, float, str, List["JsonValue"], Dict[str, "JsonValue"]]
+State = Union[str, List[JsonValue], Dict[str, JsonValue]]
+Questions = Dict[str, Dict[str, JsonValue]]
+MinConfidence = Union[float, Dict[str, float]]
+
+
+class _AnswerExtensions(TypedDict, total=False):
+ answer_confidence: float
+ x_jev_confidence: float
+ low_confidence: bool
+ abstention: Literal["passed", "abstained", "unevaluated"]
+ abstention_threshold: float
+ action: Dict[str, JsonValue]
+ x_tournament: Dict[str, JsonValue]
+
+
+class _ConfidenceAnswer(_AnswerExtensions):
+ confidence: float
+
+
+class ChoiceAnswer(_ConfidenceAnswer):
+ type: Literal["choice"]
+ choice: str
+ probabilities: Dict[str, float]
+
+
+class ScoreAnswer(_ConfidenceAnswer):
+ type: Literal["score"]
+ score: float
+ probabilities: Dict[str, float]
+ legend: Dict[str, JsonValue]
+
+
+class _NoulExtensions(_AnswerExtensions, total=False):
+ confidence: float
+
+
+class NoulAnswer(_NoulExtensions):
+ type: Literal["noul"]
+ noul: float
+
+
+class Usage(TypedDict):
+ input_tokens: int
+ output_tokens: int
+
+
+class _PredictionExtensions(TypedDict, total=False):
+ routing: Dict[str, JsonValue]
+
+
+class Prediction(_PredictionExtensions):
+ model: str
+ answers: Dict[str, Union[ChoiceAnswer, ScoreAnswer, NoulAnswer]]
+ usage: Usage
+
+
+class Health(TypedDict):
+ status: Literal["ok"]
diff --git a/sdk/python/tests/_support.py b/sdk/python/tests/_support.py
new file mode 100644
index 00000000..877b2333
--- /dev/null
+++ b/sdk/python/tests/_support.py
@@ -0,0 +1,73 @@
+import copy
+import json
+import threading
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+QUESTIONS = {
+ "team": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": None, "technical": "help"}},
+ "urgency": {"type": "score", "instructions": "Urgency?", "criteria": ["low", "medium", "high"]},
+ "refund": {"type": "noul", "instructions": "Refund?"},
+}
+PREDICTION = {
+ "model": "custom-checkpoint",
+ "answers": {
+ "team": {"type": "choice", "choice": "billing", "probabilities": {"billing": 0.8, "technical": 0.2},
+ "confidence": 0.278},
+ "urgency": {"type": "score", "score": 1.7, "probabilities": {"0": 0.1, "1": 0.1, "2": 0.8},
+ "confidence": 0.418, "legend": {"0": "low", "1": "medium", "2": "high"}},
+ "refund": {"type": "noul", "noul": 0.7},
+ },
+ "usage": {"input_tokens": 123, "output_tokens": 0},
+}
+
+
+class Server:
+ def __init__(self, payload=None, *, raw=None, status=200, headers=None, slow=False, truncated=False):
+ self.payload = copy.deepcopy(PREDICTION if payload is None else payload)
+ self.raw, self.status = raw, status
+ self.headers = headers or {}
+ self.slow, self.truncated = slow, truncated
+ self.requests = []
+ self.release = threading.Event()
+ fixture = self
+
+ class Handler(BaseHTTPRequestHandler):
+ def log_message(self, *args):
+ pass
+
+ def handle_request(self):
+ body = self.rfile.read(int(self.headers.get("Content-Length", 0)))
+ fixture.requests.append((self.command, self.path, dict(self.headers), body))
+ if fixture.slow:
+ fixture.release.wait(5)
+ raw = fixture.raw
+ if raw is None:
+ raw = json.dumps(fixture.payload).encode("utf-8")
+ self.send_response(fixture.status)
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(raw) + (10 if fixture.truncated else 0)))
+ for key, value in fixture.headers.items():
+ self.send_header(key, value)
+ self.end_headers()
+ try:
+ self.wfile.write(raw)
+ except (BrokenPipeError, ConnectionResetError, OSError):
+ pass
+ self.close_connection = True
+
+ do_GET = do_POST = handle_request
+
+ self.httpd = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
+ self.url = "http://127.0.0.1:%d" % self.httpd.server_port
+ self.thread = threading.Thread(target=self.httpd.serve_forever,
+ kwargs={"poll_interval": 0.01}, daemon=True)
+
+ def __enter__(self):
+ self.thread.start()
+ return self
+
+ def __exit__(self, *args):
+ self.release.set()
+ self.httpd.shutdown()
+ self.thread.join(timeout=5)
+ self.httpd.server_close()
diff --git a/sdk/python/tests/integration.py b/sdk/python/tests/integration.py
new file mode 100644
index 00000000..9394aabf
--- /dev/null
+++ b/sdk/python/tests/integration.py
@@ -0,0 +1,142 @@
+"""Run the installed SDK against a separate Python process with real offline inference."""
+import argparse
+import json
+import os
+from pathlib import Path
+import queue
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+
+from laya_client import LayaAPIError, LayaClient, LayaConnectionError
+
+
+def run(server_python, fixture_path, strict=False, legacy=False):
+ with tempfile.TemporaryDirectory(prefix="laya-sdk-live-") as directory:
+ root = Path(directory)
+ command = [str(server_python), "-u", str(fixture_path)]
+ if not legacy:
+ command.append("--tournament")
+ if strict:
+ command.append("--strict")
+ env = dict(os.environ)
+ env.pop("PYTHONPATH", None)
+ env.update(HF_HUB_OFFLINE="1", TRANSFORMERS_OFFLINE="1", LAYA_TOURNAMENT_GROUP="0",
+ LAYA_JEV_STRICT="0", LAYA_PRELOAD="0", LAYA_IDLE_UNLOAD_SECONDS="0")
+ with (root / "stderr.txt").open("w+", encoding="utf-8", errors="replace") as stderr:
+ server = subprocess.Popen(command, cwd=root, env=env, stdout=subprocess.PIPE,
+ stderr=stderr, text=True, encoding="utf-8")
+ lines = queue.Queue()
+
+ def read_lines():
+ for line in server.stdout:
+ if line.startswith("LAYA_TEST_SERVER="):
+ lines.put(json.loads(line.split("=", 1)[1]))
+ return
+ lines.put(None)
+
+ reader = threading.Thread(target=read_lines, daemon=True)
+ reader.start()
+ try:
+ fixture = lines.get(timeout=90)
+ if fixture is None:
+ raise RuntimeError("Server exited before producing its fixture")
+ client = LayaClient(base_url=fixture["baseURL"], api_key="integration-test", timeout=10)
+ deadline = time.monotonic() + 15
+ while True:
+ try:
+ client.health(timeout=0.5)
+ break
+ except LayaConnectionError:
+ if time.monotonic() >= deadline or server.poll() is not None:
+ raise
+ time.sleep(0.05)
+ result = client.predict(fixture["state"], fixture["questions"], model="english")
+ expected = fixture["expected"]
+ assert result["model"] == expected["model"]
+ if strict:
+ fields = {"choice": ("type", "choice", "probabilities", "confidence"),
+ "score": ("type", "score", "probabilities", "confidence", "legend"),
+ "noul": ("type", "noul")}
+ projected = {
+ "model": expected["model"],
+ "answers": {qid: {k: answer[k] for k in fields[answer["type"]]}
+ for qid, answer in expected["answers"].items()},
+ "usage": {k: expected["usage"][k] for k in ("input_tokens", "output_tokens")},
+ }
+ assert result == projected, (result, projected)
+ gated = client.predict(fixture["state"], fixture["questions"], min_confidence=0.99)
+ assert all("abstention" not in a for a in gated["answers"].values())
+ else:
+ for qid, answer in expected["answers"].items():
+ for key, value in answer.items():
+ assert result["answers"][qid][key] == value, (qid, key)
+ for key, value in expected["usage"].items():
+ assert result["usage"][key] == value, key
+ assert result["routing"]["model"] == "english"
+ auto = client.predict(fixture["state"], fixture["questions"])
+ assert auto["routing"]["model"] == "multilingual"
+ gated = client.predict(fixture["state"], fixture["questions"], lang="en",
+ max_len=128, head_max_len=64, min_confidence=0.0)
+ assert gated["routing"]["model"] == "english"
+ assert all(a["abstention"] == "passed" for a in gated["answers"].values())
+
+ if not legacy:
+ labels = {str(i): "Option %d" % i for i in range(5)}
+ answer = client.predict("hello", {"large": {
+ "type": "choice", "instructions": "Pick one", "criteria": labels,
+ }})["answers"]["large"]
+ assert set(answer["probabilities"]) == set(labels)
+ assert answer["choice"] in labels
+ if strict:
+ assert "x_tournament" not in answer
+ else:
+ finalists = answer["x_tournament"]["finalists"]
+ assert 0 < len(finalists) <= 2
+ assert all(p == 0.0 for k, p in answer["probabilities"].items() if k not in finalists)
+
+ anonymous = LayaClient(base_url=fixture["baseURL"], api_key="wrong")
+ assert anonymous.health() == {"status": "ok"}
+ try:
+ anonymous.predict("hello", fixture["questions"])
+ except LayaAPIError as error:
+ assert error.status_code == 401
+ else:
+ raise AssertionError("Unauthenticated prediction succeeded")
+ try:
+ client.predict("x" * 50_001, fixture["questions"])
+ except LayaAPIError as error:
+ assert error.status_code == 413
+ else:
+ raise AssertionError("Server request limit was bypassed")
+ print("PASS real HTTP inference, controls, auth and limits: "
+ + ("legacy fixture" if legacy else "strict" if strict else "extended/tournament"), flush=True)
+ except BaseException:
+ stderr.flush()
+ stderr.seek(0)
+ print(stderr.read(), file=sys.stderr)
+ raise
+ finally:
+ if server.poll() is None:
+ server.terminate()
+ try:
+ server.wait(timeout=5)
+ except subprocess.TimeoutExpired:
+ server.kill()
+ server.wait(timeout=5)
+ server.stdout.close()
+ reader.join(timeout=1)
+
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--server-python", required=True, type=Path)
+ parser.add_argument("--server-fixture", type=Path,
+ default=Path(__file__).resolve().parents[3] / "tests" / "sdk_server_fixture.py")
+ parser.add_argument("--legacy-fixture", action="store_true")
+ args = parser.parse_args()
+ run(args.server_python.resolve(), args.server_fixture.resolve(), legacy=args.legacy_fixture)
+ if not args.legacy_fixture:
+ run(args.server_python.resolve(), args.server_fixture.resolve(), strict=True)
diff --git a/sdk/python/tests/test_api.py b/sdk/python/tests/test_api.py
new file mode 100644
index 00000000..f3f83a7b
--- /dev/null
+++ b/sdk/python/tests/test_api.py
@@ -0,0 +1,64 @@
+import importlib.metadata
+import inspect
+from pathlib import Path
+import sys
+import unittest
+
+import laya_client
+from laya_client import LayaClient, LayaConnectionError, LayaError, LayaTimeoutError
+
+
+class APITests(unittest.TestCase):
+ def test_public_exports(self):
+ self.assertEqual(set(laya_client.__all__), {
+ "LayaClient", "LayaError", "LayaValidationError", "LayaAPIError",
+ "LayaConnectionError", "LayaTimeoutError", "LayaResponseError",
+ })
+ for name in laya_client.__all__:
+ if name.endswith("Error"):
+ self.assertTrue(issubclass(getattr(laya_client, name), LayaError))
+ self.assertTrue(issubclass(LayaTimeoutError, LayaConnectionError))
+
+ def test_predict_parameters(self):
+ parameters = inspect.signature(LayaClient.predict).parameters
+ self.assertEqual(list(parameters), ["self", "state", "questions", "model", "task", "lang",
+ "lang_guess", "max_len", "head_max_len", "min_confidence", "timeout"])
+ for name in list(parameters)[3:]:
+ self.assertEqual(parameters[name].kind, inspect.Parameter.KEYWORD_ONLY)
+ self.assertIsNone(parameters[name].default)
+ self.assertEqual(list(inspect.signature(LayaClient.health).parameters), ["self", "timeout"])
+ self.assertFalse(hasattr(LayaClient, "predict_batch"))
+
+ def test_constructor_defaults(self):
+ parameters = inspect.signature(LayaClient).parameters
+ self.assertEqual({k: p.default for k, p in parameters.items()},
+ {"base_url": "http://127.0.0.1:8000", "api_key": None, "timeout": 120.0})
+ self.assertTrue(all(p.kind == inspect.Parameter.KEYWORD_ONLY for p in parameters.values()))
+
+ def test_types_match_strict_response_requirements(self):
+ from laya_client.types import ChoiceAnswer, NoulAnswer, Prediction, ScoreAnswer
+ self.assertIn("confidence", ChoiceAnswer.__required_keys__)
+ self.assertIn("confidence", ScoreAnswer.__required_keys__)
+ self.assertIn("confidence", NoulAnswer.__optional_keys__)
+ self.assertIn("abstention", ChoiceAnswer.__optional_keys__)
+ self.assertIn("routing", Prediction.__optional_keys__)
+
+ def test_installed_package_has_no_runtime_requirements(self):
+ distribution = importlib.metadata.distribution("laya-client")
+ self.assertFalse(distribution.requires)
+ installed = Path(distribution.locate_file("laya_client/__init__.py")).resolve()
+ self.assertEqual(Path(laya_client.__file__).resolve(), installed)
+ self.assertNotIn("src", installed.parts)
+ self.assertTrue(Path(distribution.locate_file("laya_client/py.typed")).is_file())
+ self.assertTrue(any("LICENSE" in str(p) for p in distribution.files))
+
+ def test_no_inference_or_http_dependencies_in_client_environment(self):
+ names = {d.metadata["Name"].lower().replace("_", "-") for d in importlib.metadata.distributions()}
+ forbidden = {"laya", "torch", "transformers", "numpy", "safetensors",
+ "huggingface-hub", "requests", "httpx"}
+ self.assertFalse(names & forbidden, names & forbidden)
+ self.assertFalse(set(sys.modules) & {"laya", "torch", "transformers", "numpy"})
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/sdk/python/tests/test_client.py b/sdk/python/tests/test_client.py
new file mode 100644
index 00000000..7942167a
--- /dev/null
+++ b/sdk/python/tests/test_client.py
@@ -0,0 +1,278 @@
+import copy
+import io
+import json
+import socket
+import unittest
+import urllib.error
+from unittest.mock import patch
+
+from laya_client import (
+ LayaAPIError, LayaClient, LayaConnectionError, LayaResponseError,
+ LayaTimeoutError, LayaValidationError,
+)
+from _support import PREDICTION, QUESTIONS, Server
+
+
+class ClientTests(unittest.TestCase):
+ def test_predict_preserves_payload_and_server_default(self):
+ state = {"message": "退款 🐍", "items": [None, False, 0, ""], "nested": {"a": 1}}
+ questions = copy.deepcopy(QUESTIONS)
+ questions["team"]["option_order"] = ["technical", "billing"]
+ questions["team"]["future_field"] = {"enabled": True}
+ original = copy.deepcopy((state, questions))
+ with Server() as server:
+ result = LayaClient(base_url=server.url + "/proxy/laya/", api_key="secret").predict(state, questions)
+ method, path, headers, raw = server.requests[0]
+ self.assertEqual((method, path), ("POST", "/proxy/laya/v1/systemone"))
+ self.assertEqual(headers["Authorization"], "Bearer secret")
+ self.assertEqual(headers["Content-Type"], "application/json")
+ self.assertEqual(headers["Accept"], "application/json")
+ self.assertEqual(json.loads(raw), {"state": state, "questions": questions})
+ self.assertEqual(result, PREDICTION)
+ self.assertEqual((state, questions), original)
+
+ def test_controls_and_zero_threshold_are_forwarded(self):
+ controls = dict(model="registered-checkpoint", task="helpdesk", lang="de",
+ lang_guess="de-AT", max_len=128, head_max_len=64, min_confidence=0.0)
+ with Server() as server:
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS, timeout=2, **controls)
+ body = json.loads(server.requests[0][3])
+ self.assertEqual(body, {"state": "hello", "questions": QUESTIONS, **controls})
+ self.assertNotIn("Authorization", server.requests[0][2])
+
+ def test_threshold_names_are_server_owned(self):
+ thresholds = {"default": 0.9, "future:bucket": 0.0}
+ with Server() as server:
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS, min_confidence=thresholds)
+ self.assertEqual(json.loads(server.requests[0][3])["min_confidence"], thresholds)
+
+ def test_shared_references_are_not_cycles(self):
+ shared = {"flag": False}
+ with Server() as server:
+ LayaClient(base_url=server.url).predict([shared, shared], QUESTIONS)
+ self.assertEqual(json.loads(server.requests[0][3])["state"], [shared, shared])
+
+ def test_constructor_does_not_connect(self):
+ with patch("urllib.request.build_opener", side_effect=AssertionError("unexpected connection")):
+ client = LayaClient(api_key="private-token")
+ self.assertNotIn("private-token", repr(client))
+
+ def test_invalid_urls(self):
+ for url in (None, 42, "", "localhost:8000", "file:///tmp/x", "ftp://example.com",
+ "http://", "http://user:pass@localhost", "http://@localhost", "http://localhost?q=1",
+ "http://localhost#x", " http://localhost", "http://localhost/a b",
+ "http://localhost:99999", "http://localhost:abc", "http://[broken", "http://本地",
+ "http://localhost/?", "http://localhost/#", "http://local\x00host"):
+ with self.subTest(url=url), self.assertRaises(LayaValidationError):
+ LayaClient(base_url=url)
+
+ def test_invalid_tokens_do_not_leak(self):
+ for token in ("", 123, " secret", "secret\nheader", "secret\x7f", "密钥"):
+ with self.subTest(token=token), self.assertRaises(LayaValidationError) as failure:
+ LayaClient(api_key=token)
+ self.assertNotIn("secret", str(failure.exception))
+
+ def test_invalid_timeout_before_network(self):
+ for timeout in (0, -1, True, "2", float("nan"), float("inf"), 10 ** 400):
+ with self.subTest(timeout=repr(timeout)[:30]), self.assertRaises(LayaValidationError):
+ LayaClient(timeout=timeout)
+ with Server() as server:
+ client = LayaClient(base_url=server.url)
+ for timeout in (0, -1, True, "2", float("nan"), float("inf")):
+ with self.subTest(timeout=timeout), self.assertRaises(LayaValidationError):
+ client.predict("hello", QUESTIONS, timeout=timeout)
+ self.assertEqual(server.requests, [])
+
+ def test_invalid_state_never_sends(self):
+ cyclic = []
+ cyclic.append(cyclic)
+ deep = []
+ for _ in range(1100):
+ deep = [deep]
+ states = [None, True, 2, 1.5, (), {"x": object()}, {"x": (1, 2)}, {1: "rewritten"},
+ {"x": float("nan")}, {"x": float("inf")}, cyclic, deep, "\ud800"]
+ with Server() as server:
+ client = LayaClient(base_url=server.url)
+ for i, state in enumerate(states):
+ with self.subTest(case=i), self.assertRaises(LayaValidationError):
+ client.predict(state, QUESTIONS)
+ self.assertEqual(server.requests, [])
+
+ def test_invalid_questions_never_send(self):
+ cases = [None, [], {}, {"x": None}, {"x": {"type": "unknown", "instructions": ""}},
+ {"x": {"type": "choice"}}, {42: QUESTIONS["refund"]}]
+ with Server() as server:
+ client = LayaClient(base_url=server.url)
+ for questions in cases:
+ with self.subTest(questions=questions), self.assertRaises(LayaValidationError):
+ client.predict("hello", questions)
+ self.assertEqual(server.requests, [])
+
+ def test_invalid_controls_never_send(self):
+ cases = [{"model": ""}, {"task": 1}, {"lang": False}, {"lang_guess": lambda x: x},
+ {"max_len": True}, {"head_max_len": 0}, {"max_len": 1.5},
+ {"min_confidence": True}, {"min_confidence": -0.1}, {"min_confidence": 1.01},
+ {"min_confidence": {}}, {"min_confidence": {"default": False}},
+ {"min_confidence": {"default": float("nan")}}, {"min_confidence": {1: 0.5}}]
+ with Server() as server:
+ client = LayaClient(base_url=server.url)
+ for controls in cases:
+ with self.subTest(controls=controls), self.assertRaises(LayaValidationError):
+ client.predict("hello", QUESTIONS, **controls)
+ with self.assertRaises(TypeError):
+ client.predict("hello", QUESTIONS, hooks=object())
+ self.assertEqual(server.requests, [])
+
+ def test_health_minimal_and_full(self):
+ for payload in ({"status": "ok"}, {"status": "ok", "loaded": ["new-model"], "device": "cpu",
+ "new_metadata": {"capability": True}}):
+ with self.subTest(payload=payload), Server(payload) as server:
+ self.assertEqual(LayaClient(base_url=server.url).health(), payload)
+ self.assertEqual(server.requests[0][0], "GET")
+ self.assertEqual(server.requests[0][3], b"")
+
+ def test_health_does_not_invent_authentication(self):
+ with Server({"status": "ok"}) as server:
+ result = LayaClient(base_url=server.url, api_key="wrong").health()
+ self.assertEqual(result, {"status": "ok"})
+ self.assertNotIn("authenticated", result)
+
+ def test_strict_response_does_not_invent_abstention(self):
+ with Server() as server:
+ result = LayaClient(base_url=server.url).predict("hello", QUESTIONS, min_confidence=0.99)
+ self.assertNotIn("routing", result)
+ for answer in result["answers"].values():
+ self.assertNotIn("answer_confidence", answer)
+ self.assertNotIn("abstention", answer)
+ self.assertNotIn("low_confidence", answer)
+ self.assertEqual(result["answers"]["refund"], {"type": "noul", "noul": 0.7})
+
+ def test_extensions_and_tournament_values_are_preserved(self):
+ payload = copy.deepcopy(PREDICTION)
+ payload["routing"] = {"model": "future-model", "future": [1, False]}
+ payload["future"] = {"value": None}
+ payload["usage"]["new_measurement"] = 3
+ team = payload["answers"]["team"]
+ team.update(probabilities={"billing": 1.0, "technical": 0.0}, confidence=0.0,
+ answer_confidence=1.0, x_tournament={"finalists": ["billing"], "rounds": []},
+ abstention="passed", abstention_threshold=0.8, new_answer_field=[False])
+ with Server(payload) as server:
+ result = LayaClient(base_url=server.url).predict("hello", QUESTIONS)
+ self.assertEqual(result, payload)
+
+ def test_invalid_responses(self):
+ cases = []
+ for key, value in (("model", None), ("answers", []), ("usage", None), ("routing", [])):
+ p = copy.deepcopy(PREDICTION)
+ p[key] = value
+ cases.append(p)
+ for key, value in (("confidence", True), ("confidence", -1), ("confidence", 1.01),
+ ("probabilities", {}), ("probabilities", {"billing": 1}),
+ ("choice", "missing"), ("type", "score"), ("low_confidence", "true"),
+ ("abstention", "invented"), ("abstention", "passed"),
+ ("abstention_threshold", 0.8), ("action", False)):
+ p = copy.deepcopy(PREDICTION)
+ p["answers"]["team"][key] = value
+ cases.append(p)
+ for key, value in (("score", True), ("score", 3), ("legend", {})):
+ p = copy.deepcopy(PREDICTION)
+ p["answers"]["urgency"][key] = value
+ cases.append(p)
+ p = copy.deepcopy(PREDICTION)
+ p["usage"]["input_tokens"] = True
+ cases.append(p)
+ p = copy.deepcopy(PREDICTION)
+ del p["answers"]["refund"]
+ cases.append(p)
+ for i, payload in enumerate(cases):
+ with self.subTest(case=i), Server(payload) as server, self.assertRaises(LayaResponseError):
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS)
+
+ def test_invalid_json_successes(self):
+ for raw in (b"<html>wrong upstream</html>", b"null", b"[]", b"{}",
+ b'{"x": NaN}', b"\xff", b'{"x": Infinity}'):
+ with self.subTest(raw=raw), Server(raw=raw) as server, self.assertRaises(LayaResponseError):
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS)
+
+ def test_invalid_health(self):
+ for raw in (b"[]", b"null", b'{"status":"starting"}', b'{"loaded":[]}'):
+ with self.subTest(raw=raw), Server(raw=raw) as server, self.assertRaises(LayaResponseError):
+ LayaClient(base_url=server.url).health()
+
+ def test_http_status_and_structured_details(self):
+ for status in (400, 401, 413, 422, 500, 503):
+ detail = [{"loc": ["body", "model"], "msg": "invalid"}]
+ with self.subTest(status=status), Server({"detail": detail}, status=status) as server:
+ with self.assertRaises(LayaAPIError) as failure:
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS)
+ self.assertEqual(failure.exception.status_code, status)
+ self.assertEqual(failure.exception.detail, detail)
+ self.assertEqual(len(server.requests), 1)
+
+ def test_non_json_error_keeps_http_status(self):
+ with Server(raw=b"<html>proxy unavailable</html>", status=503) as server:
+ with self.assertRaises(LayaAPIError) as failure:
+ LayaClient(base_url=server.url).health()
+ self.assertEqual(failure.exception.status_code, 503)
+ self.assertIn("proxy unavailable", failure.exception.detail)
+
+ def test_error_detail_shapes(self):
+ for detail in ("bad request", {"reason": "limit"}, None):
+ with self.subTest(detail=detail), Server({"detail": detail}, status=422) as server:
+ with self.assertRaises(LayaAPIError) as failure:
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS)
+ self.assertEqual(failure.exception.detail, detail)
+
+ def test_deep_error_json_keeps_http_status(self):
+ raw = b"[" * 2000 + b"0" + b"]" * 2000
+ with Server(raw=raw, status=500) as server:
+ with self.assertRaises(LayaAPIError) as failure:
+ LayaClient(base_url=server.url).health()
+ self.assertEqual(failure.exception.status_code, 500)
+
+ def test_redirect_never_forwards_token_or_replays(self):
+ with Server() as other:
+ for status in (301, 302, 303, 307, 308):
+ with self.subTest(status=status), Server(status=status, headers={"Location": other.url}) as server:
+ with self.assertRaises(LayaAPIError) as failure:
+ LayaClient(base_url=server.url, api_key="secret").predict("hello", QUESTIONS)
+ self.assertEqual(failure.exception.status_code, status)
+ self.assertEqual(len(server.requests), 1)
+ self.assertEqual(other.requests, [])
+
+ def test_timeout_does_not_retry(self):
+ with Server(slow=True) as server:
+ with self.assertRaises(LayaTimeoutError):
+ LayaClient(base_url=server.url, timeout=2).predict("hello", QUESTIONS, timeout=0.1)
+ self.assertEqual(len(server.requests), 1)
+
+ def test_connection_refused(self):
+ with socket.socket() as listener:
+ listener.bind(("127.0.0.1", 0))
+ url = "http://127.0.0.1:%d" % listener.getsockname()[1]
+ with self.assertRaises(LayaConnectionError):
+ LayaClient(base_url=url, timeout=0.5).health()
+
+ def test_interrupted_response(self):
+ with Server(truncated=True) as server:
+ with self.assertRaises(LayaConnectionError):
+ LayaClient(base_url=server.url).predict("hello", QUESTIONS)
+
+ def test_response_is_closed_on_decode_error(self):
+ stream = io.BytesIO(b"invalid")
+ with patch("urllib.request.OpenerDirector.open", return_value=stream):
+ with self.assertRaises(LayaResponseError):
+ LayaClient().health()
+ self.assertTrue(stream.closed)
+
+ def test_http_error_stream_is_closed(self):
+ stream = io.BytesIO(b'{"detail":"no"}')
+ error = urllib.error.HTTPError("http://localhost", 401, "Unauthorized", {}, stream)
+ with patch("urllib.request.OpenerDirector.open", side_effect=error):
+ with self.assertRaises(LayaAPIError):
+ LayaClient().health()
+ self.assertTrue(stream.closed)
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/sdk/python/tests/verify_package.py b/sdk/python/tests/verify_package.py
new file mode 100644
index 00000000..0019c004
--- /dev/null
+++ b/sdk/python/tests/verify_package.py
@@ -0,0 +1,88 @@
+"""Verify built distributions in a client environment with no inference dependencies."""
+import argparse
+from email.parser import BytesParser
+import os
+from pathlib import Path
+import subprocess
+import sys
+import tarfile
+import tempfile
+import venv
+import zipfile
+
+
+def command(*args, cwd, env):
+ subprocess.run([str(a) for a in args], cwd=cwd, env=env, check=True)
+
+
+def check_wheel(path):
+ with zipfile.ZipFile(path) as wheel:
+ names = wheel.namelist()
+ metadata = [n for n in names if n.endswith(".dist-info/METADATA")]
+ assert len(metadata) == 1
+ message = BytesParser().parsebytes(wheel.read(metadata[0]))
+ assert message["Name"] == "laya-client"
+ assert not message.get_all("Requires-Dist"), message.get_all("Requires-Dist")
+ assert "laya_client/py.typed" in names
+ assert any("LICENSE" in n for n in names)
+ assert all(n.startswith(("laya_client/", "laya_client-")) for n in names), names
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("dist", type=Path)
+ parser.add_argument("--server-python", type=Path)
+ parser.add_argument("--legacy-server-fixture", type=Path)
+ args = parser.parse_args()
+ tests = Path(__file__).resolve().parent
+ dist = args.dist.resolve()
+ wheels, sdists = list(dist.glob("*.whl")), list(dist.glob("*.tar.gz"))
+ assert len(wheels) == len(sdists) == 1, "Use a dist directory containing exactly one wheel and sdist"
+ check_wheel(wheels[0])
+ env = dict(os.environ)
+ env.pop("PYTHONPATH", None)
+ env["PIP_DISABLE_PIP_VERSION_CHECK"] = "1"
+ with tempfile.TemporaryDirectory(prefix="laya-sdk-package-") as directory:
+ root = Path(directory)
+ environment = root / "client"
+ venv.EnvBuilder(with_pip=True).create(environment)
+ python = environment / ("Scripts/python.exe" if os.name == "nt" else "bin/python")
+ command(python, "-m", "pip", "install", "--no-index", wheels[0], cwd=root, env=env)
+ command(python, "-I", "-m", "unittest", "discover", "-s", tests, "-p", "test_*.py",
+ "-v", cwd=root, env=env)
+ print("PASS wheel installation with normal resolution and no package index", flush=True)
+
+ unpacked = root / "sdist"
+ unpacked.mkdir()
+ with tarfile.open(sdists[0]) as archive:
+ for entry in archive.getmembers():
+ target = (unpacked / entry.name).resolve()
+ assert target.is_relative_to(unpacked.resolve())
+ assert entry.isfile() or entry.isdir(), "Unexpected link or device in sdist"
+ archive.extractall(unpacked)
+ projects = list(unpacked.iterdir())
+ assert len(projects) == 1 and (projects[0] / "pyproject.toml").is_file()
+ rebuilt = root / "rebuilt"
+ command(sys.executable, "-m", "build", "--wheel", "--no-isolation", "--outdir", rebuilt,
+ projects[0], cwd=root, env=env)
+ rebuilt_wheels = list(rebuilt.glob("*.whl"))
+ assert len(rebuilt_wheels) == 1
+ check_wheel(rebuilt_wheels[0])
+ command(python, "-m", "pip", "install", "--no-index", "--force-reinstall", rebuilt_wheels[0],
+ cwd=root, env=env)
+ command(python, "-I", "-m", "unittest", "discover", "-s", tests, "-p", "test_*.py",
+ cwd=root, env=env)
+ print("PASS sdist rebuild and installed-package regression", flush=True)
+ if args.server_python:
+ command(python, "-I", tests / "integration.py", "--server-python", args.server_python.resolve(),
+ cwd=root, env=env)
+ if args.legacy_server_fixture:
+ command(python, "-I", tests / "integration.py", "--server-python", args.server_python.resolve(),
+ "--server-fixture", args.legacy_server_fixture.resolve(), "--legacy-fixture",
+ cwd=root, env=env)
+ elif args.legacy_server_fixture:
+ parser.error("--legacy-server-fixture requires --server-python")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tests/sdk_server_fixture.py b/tests/sdk_server_fixture.py
index 36ac2a98..cc475ec0 100644
--- a/tests/sdk_server_fixture.py
+++ b/tests/sdk_server_fixture.py
@@ -1,4 +1,5 @@
-"""Serve a tiny offline checkpoint for the JavaScript integration test, never for production."""
+"""Serve a tiny offline checkpoint for SDK integration tests, never for production."""
+import argparse
import json
import os
from pathlib import Path
@@ -23,6 +24,10 @@ from laya.serve import create_app # noqa: E402
def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--strict", action="store_true")
+ parser.add_argument("--tournament", action="store_true")
+ args = parser.parse_args()
torch.manual_seed(7)
torch.set_num_threads(1)
with tempfile.TemporaryDirectory(prefix="laya-sdk-test-") as directory:
@@ -60,7 +65,8 @@ def main():
router.attach(name, agent)
os.environ["LAYA_API_KEY"] = "integration-test"
# This fixture exercises the default HTTP contract, including its confidence extension.
- os.environ["LAYA_JEV_STRICT"] = "0"
+ os.environ["LAYA_JEV_STRICT"] = "1" if args.strict else "0"
+ os.environ["LAYA_TOURNAMENT_GROUP"] = "2" if args.tournament else "0"
app = create_app(router)
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index bd72e347..d86d7d1e 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -70,6 +70,9 @@ HOOK_KEYS = {
}
# --------------------------------------------------------------- constructors
+check("HTTP SDK stays outside core exports", "LayaClient" in laya.__all__, False)
+check("core does not import the HTTP SDK", "laya_client" in sys.modules, False)
+
for label, fn in (("Agent.__init__", Agent.__init__), ("load", load),
("Router.__init__", Router.__init__), ("ONNXAgent.__init__", ONNXAgent.__init__)):
for param, default in HOOK_KEYS.items():