The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.
comfy_attention configs in model files can now use kitchen sol attention. (#16831)
comfyanonymousb26625f23a88
diff --git a/QUANTIZATION.md b/QUANTIZATION.mdindex a3fd13d14..b91bffe99 100644--- a/QUANTIZATION.md+++ b/QUANTIZATION.md@@ -155,23 +155,43 @@ To create compatible checkpoints, use any quantization tool provided the output### Diffusion attention preferencesA diffusion attention module can have a `<module path>.comfy_attention.config` entry whose-uint8 tensor contains UTF-8 JSON:+uint8 tensor contains UTF-8 JSON. Use a single preference:```json{"attention": "comfy_kitchen_int8"}```+Or an ordered list of preferences, each with its own options:++```json+[+ {"attention": "comfy_kitchen_sol", "tau": 1.0},+ {"attention": "comfy_kitchen_int8"}+]+```+Use the module that performs attention, such as `transformer_blocks.0.attn` for-Qwen Image 2.1 or `blocks.0.attn` for MiniMax H3.--Only `comfy_kitchen_int8` is supported. Invalid targets and other method names-are ignored with a warning during loading, leaving normal attention selection.-Kitchen INT8 support is checked when each preference is loaded-for the primary device; unsupported devices keep normal attention selection.-Explicit attention overrides retain priority.-The `ComfyAttention` child module loads and saves its own metadata through normal-state-dict loading and saving. Preferences do not enable weight-quantization. Text encoder and VAE loaders do not apply these preferences.+Qwen Image 2.1 or `blocks.0.attn` for MiniMax H3. The target must have a+`ComfyAttention` child module.++Only `comfy_kitchen_int8` and `comfy_kitchen_sol` are accepted. ComfyUI selects+the first available preference during loading, checking support for the primary+device. All other method names are skipped with a warning. Unavailable methods+are skipped. If none are available, or the list is empty, normal attention+selection remains in effect. Explicit attention overrides retain priority.++`comfy_kitchen_sol` supports only `tau` as an optional field alongside+`attention`. It defaults to `1.0` and sets the routing threshold; higher values+route fewer blocks exactly. Other SOL options use Comfy Kitchen's defaults.++Bake SOL preferences into modules with compatible inputs: matching FP16/BF16+Q/K/V tensors with a head dimension of 128. ComfyUI trusts the checkpoint's+choice; `low_precision_attention=False` does not disable SOL. Masked calls use+PyTorch attention.++The `ComfyAttention` child module loads and saves its metadata through normal+state-dict loading and saving, preserving the full preference list and options,+including unsupported entries. Preferences do not enable weight quantization.### Weight Quantizationdiff --git a/comfy/ldm/modules/attention.py b/comfy/ldm/modules/attention.pyindex c54415368..d3ea61fa0 100644--- a/comfy/ldm/modules/attention.py+++ b/comfy/ldm/modules/attention.py@@ -84,14 +84,20 @@ class ComfyAttention(nn.Module):self.function = Nonemetadata = state_dict.pop(prefix + "config", None)if metadata is not None:- config = json.loads(metadata.numpy().tobytes())- method = config.get("attention")- if method == "comfy_kitchen_int8":- self.config = config- if COMFY_KITCHEN_INT8_ATTENTION_IS_AVAILABLE and comfy_kitchen.int8_attention_is_available(model_management.get_torch_device()):- self.function = attention_comfy_kitchen_int8- else:- logging.warning(f"Ignoring unsupported attention method {method!r} for {prefix.rstrip('.')}")+ self.config = json.loads(metadata.numpy().tobytes())+ configs = self.config if isinstance(self.config, list) else [self.config]+ for config in configs:+ method = config.get("attention")+ if method == "comfy_kitchen_int8":+ if COMFY_KITCHEN_INT8_ATTENTION_IS_AVAILABLE and comfy_kitchen.int8_attention_is_available(model_management.get_torch_device()):+ self.function = attention_comfy_kitchen_int8+ elif method == "comfy_kitchen_sol":+ if comfy_kitchen.sol_attn_is_available(model_management.get_torch_device()):+ self.function = functools.partial(attention_comfy_kitchen_sol, tau=config.get("tau", 1.0))+ else:+ logging.warning(f"Ignoring unknown attention method {method!r} for {prefix.rstrip('.')}")+ if self.function is not None:+ breaksuper()._load_from_state_dict(state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs)def _save_to_state_dict(self, destination, prefix, keep_vars):@@ -680,6 +686,23 @@ def _attention_comfy_kitchen_int8_containers(q, k, v, heads, mask=None, attn_preattention_comfy_kitchen_int8.container_function = _attention_comfy_kitchen_int8_containers+@wrap_attn+def attention_comfy_kitchen_sol(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, *, tau=1.0, **kwargs):+ if mask is not None:+ return attention_pytorch(q, k, v, heads, mask=mask, attn_precision=attn_precision, skip_reshape=skip_reshape, skip_output_reshape=skip_output_reshape, **kwargs)++ dim_head = q.shape[-1] if skip_reshape else q.shape[-1] // heads+ b = q.shape[0]+ if skip_reshape:+ q, k, v = map(lambda t: t.transpose(1, 2), (q, k, v))+ else:+ q, k, v = _reshape_qkv_to_heads(q, k, v, b, heads, dim_head)+ out = comfy_kitchen.sol_attn(q, k, v, tau=tau, scale=kwargs.get("scale"))+ if skip_output_reshape:+ return out.transpose(1, 2)+ return out.reshape(b, -1, heads * dim_head)++@wrap_attndef attention_sage(q, k, v, heads, mask=None, attn_precision=None, skip_reshape=False, skip_output_reshape=False, **kwargs):if kwargs.get("low_precision_attention", True) is False or (mask is not None and not SAGE_ATTENTION_SUPPORTS_MASK):