Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

Merge pull request #576 from cklxx/feat/compile-cold-start

Nandakishor4196a39b5b35
diff --cc .github/workflows/ci.yml
index ff65c2e8,77463bf2..2cc645ab
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@@ -182,33 -156,9 +182,34 @@@ jobs
tests/test_system_one_lang.py \
tests/test_audit_regressions.py \
tests/test_truncation_direction.py \
+ tests/test_compile.py \
-q
+ # The `test` job above deliberately stays free of `onnx`: this is the lane that
+ # installs it, so the exporter's quantization suite runs for real instead of
+ # skip-guarding into a trivial pass. Weight-free -- it builds tiny synthetic
+ # graphs, no checkpoint download.
+ onnx-export:
+ name: onnx export (quantization, weight-free)
+ runs-on: ubuntu-latest
+ timeout-minutes: 10
+ steps:
+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+ with:
+ python-version: "3.11"
+ cache: pip
+
+ - name: Install
+ run: |
+ python -m pip install --upgrade pip
+ pip install torch --index-url https://download.pytorch.org/whl/cpu
+ pip install -e .
+ pip install onnx onnxruntime
+
+ - name: Quantized-export tests
+ run: python tests/test_onnx_quantize.py
+
# The eval harness needs numpy and laya only, so it stays a separate, weight-free job: metric
# math, dataset parsing, the CLI and the API guard. The real-checkpoint gate is the scheduled
# `evals` workflow, which needs weights and network.
diff --cc laya/agent.py
index 638b70f9,0e325bdf..873be131
--- a/laya/agent.py
+++ b/laya/agent.py
@@@ -234,11 -233,8 +235,13 @@@ class Agent(HookRegistry)
mps_amp_min_rows = MPS_AMP_MIN_ROWS_DEFAULT
# The stock forward is also used by lightweight runtimes built with __new__ in tests.
_fast = None
+ # Scoped CPU-fallback observability: how often _infer's per-request OOM fallback fired
+ # and what the last failure was, so an operator sees a slow lane in /health instead of
+ # discovering it by accident. Class defaults cover instances built without __init__.
+ cpu_fallback_count = 0
+ last_fallback_reason = None
+ # Set when `compile=True` wrapped the model in torch.compile.
+ _compiled = False
def __init__(
self,