The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.
fp8 casting is fast on GPUs that support fp8 compute.
comfyanonymous471cd3eace4b
diff --git a/comfy/model_management.py b/comfy/model_management.pyindex 0d5b07305..855e89112 100644--- a/comfy/model_management.py+++ b/comfy/model_management.py@@ -647,6 +647,9 @@ def unet_dtype(device=None, model_params=0, supported_dtypes=[torch.float16, torpassif fp8_dtype is not None:+ if supports_fp8_compute(device): #if fp8 compute is supported the casting is most likely not expensive+ return fp8_dtype+free_model_memory = maximum_vram_for_weights(device)if model_params * 2 > free_model_memory:return fp8_dtype