Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
Merge pull request #810 from Swatantra-66/fix/concurrent-gpu-oom-fallback
Nandakishor8584c3525960
diff --cc laya/agent.pyindex 1f73a35c,b376706c..4780b379--- a/laya/agent.py+++ b/laya/agent.py@@@ -664,12 -612,12 +713,13 @@@ class Agent(HookRegistry)self.amp_enabled = Trueself.dtype = torch.bfloat16elif self.device.type == "cpu":- if os.environ.get("LAYA_CPU_AMP", "").lower() in ("bf16", "bfloat16"):+ cpu_amp = _cpu_amp_dtype()+ if cpu_amp is not None:self.amp_enabled = True- self.dtype = torch.bfloat16+ self.dtype = cpu_ampself._fast = None+ self._gate = _InferenceGate()# 2. Place on device with graceful fallback to CPU on memory errorfell_back_from = fell_back_why = None