Skip to content

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.

Fix OOM regression in _apply() for quantized models during inference (#13372)

Jun Yamog1de83f91c34a
diff --git a/comfy/ops.py b/comfy/ops.py
index b5cd1d47e..7a9b4b84c 100644
--- a/comfy/ops.py
+++ b/comfy/ops.py
@@ -1151,7 +1151,7 @@ def mixed_precision_ops(quant_config={}, compute_dtype=torch.bfloat16, full_prec
if param is None:
continue
p = fn(param)
- if p.is_inference():
+ if (not torch.is_inference_mode_enabled()) and p.is_inference():
p = p.clone()
self.register_parameter(key, torch.nn.Parameter(p, requires_grad=False))
for key, buf in self._buffers.items():