Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
Sat Oct 3 18:53:03 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 615.71.09 KMD Version: 615.71.09 CUDA UMD Version: 13.4 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4070 ... On | 00000000:26:00.0 Off | N/A |
| 0% 36C P8 12W / 295W | 738MiB / 16376MiB | 28% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 892 G Hyprland 156MiB |
| 0 N/A N/A 952 G quickshell 38MiB |
| 0 N/A N/A 995 G Xwayland 4MiB |
| 0 N/A N/A 1168 G monocloud 3MiB |
| 0 N/A N/A 3324 C+G /usr/bin/sunshine 215MiB |
| 0 N/A N/A 12883 G /usr/bin/ghostty 100MiB |
| 0 N/A N/A 3906709 C+G ...rack-uuid=3190708988185955192 36MiB |
+-----------------------------------------------------------------------------------------+
torch 2.11.0+cu130 model /home/ckl/.cache/huggingface/hub/models--convaiinnovations--laya/snapshots/55cf4c4ebb4ebe31b2550e8bdf3bd21b99753851
/home/ckl/projects/S/laya-wt/compile-defaults/benchmarks/bench_compile_defaults.py:41: RuntimeWarning: laya: this checkpoint ships invalid temperatures or values outside [0.5, 5]; using choice:11+=0.10058280825614929 -> 0.5. Treat confidence from the affected entries as uncalibrated.
agent = Agent(args.model, device="cuda", compile=True, **kw)
/home/ckl/projects/S/laya/.venv/lib/python3.12/site-packages/torch/_inductor/compile_fx.py:322: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.
warnings.warn(
W1003 18:53:44.221000 4081427 torch/_inductor/utils.py:1731] [0/0_1] Not enough SMs to use max_autotune_gemm mode
/home/ckl/projects/S/laya/.venv/lib/python3.12/site-packages/torch/_inductor/lowering.py:7836: UserWarning:
Online softmax is disabled on the fly since Inductor decides to
split the reduction. Cut an issue to PyTorch if this is an
important use case and you want to speed it up with online
softmax.
warnings.warn(
/home/ckl/projects/S/laya/.venv/lib/python3.12/site-packages/torch/_inductor/lowering.py:7836: UserWarning:
Online softmax is disabled on the fly since Inductor decides to
split the reduction. Cut an issue to PyTorch if this is an
important use case and you want to speed it up with online
softmax.
warnings.warn(
load_s 111.64288430591114 cache /home/ckl/.cache/laya-compile-defaults-verified-20261003/laya/torchinductor
request 1 seconds 0.02286072401329875 graphs 2
request 2 seconds 0.012921674991957843 graphs 2
request 3 seconds 0.009732725913636386 graphs 2
request 4 seconds 0.01357605296652764 graphs 2
request 5 seconds 0.008746225037612021 graphs 2
request 6 seconds 0.023331869044341147 graphs 2
request 7 seconds 0.01587031001690775 graphs 2
request 8 seconds 0.008692333940416574 graphs 2
request 9 seconds 0.0283058590721339 graphs 2
request 10 seconds 0.013749895035289228 graphs 2
request 11 seconds 0.009646686026826501 graphs 2
request 12 seconds 0.012101444997824728 graphs 2
request 13 seconds 0.009533305070362985 graphs 2
request 14 seconds 0.013106117956340313 graphs 2
request 15 seconds 0.008559482987038791 graphs 2
request 16 seconds 0.023173767956905067 graphs 2
request 17 seconds 0.015705847996287048 graphs 2
request 18 seconds 0.008421211037784815 graphs 2
request 19 seconds 0.028346410021185875 graphs 2
request 20 seconds 0.013642733916640282 graphs 2
request 21 seconds 0.009569393936544657 graphs 2
request 22 seconds 0.011785740964114666 graphs 2
request 23 seconds 0.009457602980546653 graphs 2
request 24 seconds 0.01304219604935497 graphs 2
request 25 seconds 0.008564352057874203 graphs 2
request 26 seconds 0.023065467015840113 graphs 2
request 27 seconds 0.015712207998149097 graphs 2
request 28 seconds 0.008489751955494285 graphs 2
request 29 seconds 0.02870765409898013 graphs 2
request 30 seconds 0.016941983019933105 graphs 2
RESULT {"load_s": 111.64288430591114, "requests_s": [0.02286072401329875, 0.012921674991957843, 0.009732725913636386, 0.01357605296652764, 0.008746225037612021, 0.023331869044341147, 0.01587031001690775, 0.008692333940416574, 0.0283058590721339, 0.013749895035289228, 0.009646686026826501, 0.012101444997824728, 0.009533305070362985, 0.013106117956340313, 0.008559482987038791, 0.023173767956905067, 0.015705847996287048, 0.008421211037784815, 0.028346410021185875, 0.013642733916640282, 0.009569393936544657, 0.011785740964114666, 0.009457602980546653, 0.01304219604935497, 0.008564352057874203, 0.023065467015840113, 0.015712207998149097, 0.008489751955494285, 0.02870765409898013, 0.016941983019933105], "allocated_mib": 1616.48291015625, "reserved_mib": 1820.0, "peak_allocated_mib": 1685.626953125, "peak_reserved_mib": 1820.0, "counters": {"stats": {"calls_captured": 6700, "unique_graphs": 2}, "inductor": {"pattern_matcher_count": 786, "pattern_matcher_nodes": 910, "benchmarking.InductorBenchmarker.benchmark_gpu": 105, "pad_mm_bench": 4, "fxgraph_cache_miss": 2, "async_compile_cache_miss": 162, "extern_calls": 384, "async_compile_cache_hit": 81, "triton_bundler_save_kernel": 1048, "triton_bundler_save_static_autotuner": 2, "benchmarking.InductorBenchmarker.benchmark": 97}}}
Sat Oct 3 18:54:55 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 615.71.09 KMD Version: 615.71.09 CUDA UMD Version: 13.4 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4070 ... On | 00000000:26:00.0 Off | N/A |
| 0% 47C P2 158W / 295W | 2823MiB / 16376MiB | 83% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 892 G Hyprland 156MiB |
| 0 N/A N/A 952 G quickshell 38MiB |
| 0 N/A N/A 995 G Xwayland 4MiB |
| 0 N/A N/A 1168 G monocloud 3MiB |
| 0 N/A N/A 3324 C+G /usr/bin/sunshine 215MiB |
| 0 N/A N/A 12883 G /usr/bin/ghostty 100MiB |
| 0 N/A N/A 3906709 C+G ...rack-uuid=3190708988185955192 35MiB |
| 0 N/A N/A 4081427 C ...jects/S/laya/.venv/bin/python 2080MiB |
+-----------------------------------------------------------------------------------------+