PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
Organize test directory structure with logical categorization (#1243)
Copilot0f6dc76e7a89
diff --git a/test/README.md b/test/README.mdindex 9ccca0017..c17170de5 100644--- a/test/README.md+++ b/test/README.md@@ -1,6 +1,6 @@# Unit Test (UT) Guide-This project uses `pytest` for unit testing. All test cases are under the `test/` directory. Below is a simple guide for new users to write and run UTs:+This project uses `pytest` for unit testing. All test cases are under the `test/` directory.## 1. Environment Setup- Recommended Python 3.8 or above.@@ -10,37 +10,160 @@ This project uses `pytest` for unit testing. All test cases are under the `test/pip install pytest```-## 2. Test Structure-- Place your test files in the `test/` directory, and name them starting with `test_`.-- You can refer to existing `test_*.py` files.-- Common fixtures (such as `tiny_opt_model`, `opt_model`, `opt_tokenizer`, `dataloader`) and helper functions (such as `model_infer`) are defined in `confest.py` and `helpers.py` and can be imported directly.-- Example:- ```python- # test_example.py- from ..helpers import model_infer-- def test_model_infer(tiny_opt_model, opt_tokenizer):- result = model_infer(tiny_opt_model, opt_tokenizer, input_text="hello world")- assert result is not None- ```+## 2. Test Directory Structure-## 3. Running Tests-- In the `test/` directory, run:- ```sh- pytest- ```-- You can specify a single file or test case:- ```sh- pytest test_xxx.py- pytest -k "test_func_name"- ```+Tests are organized by hardware backend (`test_cpu/`, `test_cuda/`) and functionality:++- **core/** - Core AutoRound API and quantization workflows+- **quantization/** - Quantization techniques (mixed-bit, MXFP, NVFP4, activation quant)+- **export/** - Model serialization (GGUF, AutoGPTQ, AutoRound format)+- **backends/** - Inference backends (Torch, Marlin, Triton, ExLlamaV2)+- **models/** - Architecture-specific tests (MLLMs, VLMs, MoE, Diffusion)+- **integrations/** - Third-party frameworks (vLLM, SGLang, LLMC, Transformers)+- **schemes/** - Quantization scheme selection and configuration+- **utils/** - Calibration datasets, logging, CLI, model loading+- **advanced/** - Multi-GPU, FP8 input, custom pipelines++## 3. Shared Test Utilities++### conftest.py+Pytest configuration file that:+- Adds parent directory to `sys.path` for easy debugging without installation+- Defines HPU-specific test options (`--mode=compile/lazy`)+- Imports all fixtures from `fixtures.py`++### fixtures.py+Provides reusable pytest fixtures for testing:++**Model Fixtures:**+- `tiny_opt_model_path` - OPT-125M model with 2 layers (session scope)+- `tiny_qwen_model_path` - Qwen-0.6B model with 2 layers+- `tiny_lamini_model_path` - LaMini-GPT-124M with 2 layers+- `tiny_gptj_model_path` - Tiny GPT-J model+- `tiny_phi2_model_path` - Phi-2 model with 2 layers+- `tiny_deepseek_v2_model_path` - DeepSeek-V2-Lite with 2 layers+- `tiny_qwen_moe_model_path` - Qwen-1.5-MoE with 2 layers+- `tiny_qwen_vl_model_path` - Qwen2-VL-2B with 2 layers (vision model)+- `tiny_qwen_2_5_vl_model_path` - Qwen2.5-VL-3B with 2 layers++**Data Fixtures:**+- `dataloader` - Simple calibration dataloader with 4 text samples++All model fixtures:+- Use session scope to avoid reloading models for each test+- Automatically save tiny models to `./tmp/` directory+- Clean up temporary files after test session ends++### helpers.py+Utility functions for testing:++**Model Path Resolution:**+```python+get_model_path(model_name) # Automatically finds local or remote model path+```++**Predefined Model Paths:**+```python+opt_name_or_path # facebook/opt-125m+qwen_name_or_path # Qwen/Qwen3-0.6B+lamini_name_or_path # MBZUAI/LaMini-GPT-124M+qwen_vl_name_or_path # Qwen/Qwen2-VL-2B-Instruct+# ... and more+```++**Model Manipulation:**+```python+get_tiny_model(model_path, num_layers=2) # Create tiny model by slicing layers+save_tiny_model(model_path, save_path) # Save tiny model to disk+```++**Model Inference:**+```python+model_infer(model, tokenizer, input_text) # Run inference and return output+is_model_outputs_similar(out1, out2) # Compare two model outputs+```++**Data Utilities:**+```python+DataLoader() # Simple dataloader for calibration datasets+```++## 4. Writing New Tests++### Basic Example+```python+# test_cpu/quantization/test_new_method.py+import pytest+from auto_round import AutoRound+from ...helpers import opt_name_or_path+++class TestNewQuantMethod:+ def test_quantization(self, tiny_opt_model_path, dataloader):+ """Test new quantization method."""+ autoround = AutoRound(model=tiny_opt_model_path, bits=4, group_size=128, iters=2, dataset=dataloader)+ autoround.quantize()+ assert autoround is not None+```++### Using Helpers and Fixtures+```python+from ...helpers import model_infer, opt_name_or_path, get_model_path+++def test_model_inference(tiny_opt_model_path):+ # Use predefined model path+ model_name = opt_name_or_path++ # Or resolve custom model path+ custom_model = get_model_path("custom/model-name")++ # Run inference using helper+ from transformers import AutoModelForCausalLM, AutoTokenizer++ model = AutoModelForCausalLM.from_pretrained(tiny_opt_model_path)+ tokenizer = AutoTokenizer.from_pretrained(tiny_opt_model_path)+ output = model_infer(model, tokenizer, "Hello world")+```++### Placement Guidelines+- **CPU-specific** → `test_cpu/<category>/`+- **CUDA-specific** → `test_cuda/<category>/`+- **Cross-platform** → Choose most relevant directory+- Import from parent: `from ...helpers import ...`++## 5. Running Tests++```sh+# Run all tests+pytest++# Run specific directory+pytest test_cpu/quantization/++# Run specific file+pytest test_cpu/core/test_autoround.py++# Run specific test+pytest -k "test_layer_config"++# Run with verbose output+pytest -v -s+```-## 4. Debugging Tips-- `confest.py` adds the parent directory to `sys.path`, so you can debug without installing the local package.-- You can directly import project source code in your test cases.+## 6. Hardware-Specific Requirements+- **test_cpu/**: Install `pip install -r test_cpu/requirements.txt`+- **test_cuda/**: Install `pip install -r test_cuda/requirements.txt`+ - VLM: `pip install -r test_cuda/requirements_vlm.txt`+ - Diffusion: `pip install -r test_cuda/requirements_diffusion.txt`+ - LLMC: `pip install -r test_cuda/requirements_llmc.txt`+ - SGLang: `pip install -r test_cuda/requirements_sglang.txt`-## 5. Reference-- Fixtures are defined in `confest.py` and `fixtures.py`-- Helper functions are in `helpers.py`+## 7. Contributing+When adding new tests:+1. Place in appropriate category subdirectory+2. Use existing fixtures and helpers+3. Clean up resources in teardown methods+4. Use descriptive names and docstrings-If you have any questions, feel free to open an issue.+For questions, open an issue.diff --git a/test/test_cpu/advanced/__init__.py b/test/test_cpu/advanced/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/backends/__init__.py b/test/test_cpu/backends/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_torch_backend.py b/test/test_cpu/backends/test_torch_backend.pysimilarity index 98%rename from test/test_cpu/test_torch_backend.pyrename to test/test_cpu/backends/test_torch_backend.pyindex 0be8f76e6..5c70f7e99 100644--- a/test/test_cpu/test_torch_backend.py+++ b/test/test_cpu/backends/test_torch_backend.py@@ -8,7 +8,7 @@ from auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.testing_utils import require_autogptq, require_gptqmodel-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferclass TestAutoRoundTorchBackend:diff --git a/test/test_cpu/core/__init__.py b/test/test_cpu/core/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_autoopt.py b/test/test_cpu/core/test_autoopt.pysimilarity index 100%rename from test/test_cpu/test_autoopt.pyrename to test/test_cpu/core/test_autoopt.pydiff --git a/test/test_cpu/test_autoround.py b/test/test_cpu/core/test_autoround.pysimilarity index 99%rename from test/test_cpu/test_autoround.pyrename to test/test_cpu/core/test_autoround.pyindex aa7aeca5e..4df10efe6 100644--- a/test/test_cpu/test_autoround.py+++ b/test/test_cpu/core/test_autoround.py@@ -9,7 +9,7 @@ from auto_round import AutoRoundfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.utils import get_module-from ..helpers import get_model_path, model_infer, opt_name_or_path, qwen_name_or_path+from ...helpers import get_model_path, model_infer, opt_name_or_path, qwen_name_or_pathclass TestAutoRound:diff --git a/test/test_cpu/test_autoround_acc.py b/test/test_cpu/core/test_autoround_acc.pysimilarity index 98%rename from test/test_cpu/test_autoround_acc.pyrename to test/test_cpu/core/test_autoround_acc.pyindex 876d4a452..4190e3782 100644--- a/test/test_cpu/test_autoround_acc.py+++ b/test/test_cpu/core/test_autoround_acc.py@@ -9,7 +9,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound # pylint: disable=E0401-from ..helpers import gptj_name_or_path+from ...helpers import gptj_name_or_pathclass TestAutoRound:diff --git a/test/test_cpu/test_autoround_export_to_itrex.py b/test/test_cpu/core/test_autoround_export_to_itrex.pysimilarity index 98%rename from test/test_cpu/test_autoround_export_to_itrex.pyrename to test/test_cpu/core/test_autoround_export_to_itrex.pyindex 19f196270..08a4e7753 100644--- a/test/test_cpu/test_autoround_export_to_itrex.py+++ b/test/test_cpu/core/test_autoround_export_to_itrex.py@@ -8,7 +8,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import get_model_path, gptj_name_or_path+from ...helpers import get_model_path, gptj_name_or_pathclass SimpleDataLoader:diff --git a/test/test_cpu/test_init.py b/test/test_cpu/core/test_init.pysimilarity index 100%rename from test/test_cpu/test_init.pyrename to test/test_cpu/core/test_init.pydiff --git a/test/test_cpu/export/__init__.py b/test/test_cpu/export/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_export.py b/test/test_cpu/export/test_export.pysimilarity index 99%rename from test/test_cpu/test_export.pyrename to test/test_cpu/export/test_export.pyindex 4653cd6a0..511fd6287 100644--- a/test/test_cpu/test_export.py+++ b/test/test_cpu/export/test_export.py@@ -7,7 +7,7 @@ from transformers import AutoModelForCausalLM, AutoRoundConfig, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import get_model_path, opt_name_or_path+from ...helpers import get_model_path, opt_name_or_pathdef _get_folder_size(path: str) -> float:diff --git a/test/test_cpu/test_gguf_format.py b/test/test_cpu/export/test_gguf_format.pysimilarity index 99%rename from test/test_cpu/test_gguf_format.pyrename to test/test_cpu/export/test_gguf_format.pyindex 4b89a3a66..5b96716eb 100644--- a/test/test_cpu/test_gguf_format.py+++ b/test/test_cpu/export/test_gguf_format.py@@ -8,7 +8,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import get_model_path, get_tiny_model+from ...helpers import get_model_path, get_tiny_modelclass TestGGUF:diff --git a/test/test_cpu/integrations/__init__.py b/test/test_cpu/integrations/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_llmc_integration.py b/test/test_cpu/integrations/test_llmc_integration.pysimilarity index 100%rename from test/test_cpu/test_llmc_integration.pyrename to test/test_cpu/integrations/test_llmc_integration.pydiff --git a/test/test_cpu/test_llmcompressor.py b/test/test_cpu/integrations/test_llmcompressor.pysimilarity index 98%rename from test/test_cpu/test_llmcompressor.pyrename to test/test_cpu/integrations/test_llmcompressor.pyindex 614701943..c11f12456 100644--- a/test/test_cpu/test_llmcompressor.py+++ b/test/test_cpu/integrations/test_llmcompressor.py@@ -7,7 +7,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import get_model_path, opt_name_or_path+from ...helpers import get_model_path, opt_name_or_pathclass TestLLMC:diff --git a/test/test_cpu/models/__init__.py b/test/test_cpu/models/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_block_names.py b/test/test_cpu/models/test_block_names.pysimilarity index 99%rename from test/test_cpu/test_block_names.pyrename to test/test_cpu/models/test_block_names.pyindex 47c554317..8394e1cc7 100644--- a/test/test_cpu/test_block_names.py+++ b/test/test_cpu/models/test_block_names.py@@ -8,7 +8,7 @@ from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import get_model_path, lamini_name_or_path+from ...helpers import get_model_path, lamini_name_or_path# ================= simple multimodal model =================diff --git a/test/test_cpu/test_conv1d.py b/test/test_cpu/models/test_conv1d.pysimilarity index 95%rename from test/test_cpu/test_conv1d.pyrename to test/test_cpu/models/test_conv1d.pyindex 1997026b3..9e151c3f0 100644--- a/test/test_cpu/test_conv1d.py+++ b/test/test_cpu/models/test_conv1d.py@@ -7,7 +7,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import lamini_name_or_path, model_infer+from ...helpers import lamini_name_or_path, model_inferclass TestQuantizationConv1d:diff --git a/test/test_cpu/test_mllm.py b/test/test_cpu/models/test_mllm.pysimilarity index 99%rename from test/test_cpu/test_mllm.pyrename to test/test_cpu/models/test_mllm.pyindex 3a91be1ca..87e140a26 100644--- a/test/test_cpu/test_mllm.py+++ b/test/test_cpu/models/test_mllm.py@@ -5,7 +5,7 @@ from transformers import AutoProcessor, AutoTokenizer, Qwen2VLForConditionalGenefrom auto_round import AutoRoundMLLM-from ..helpers import get_model_path, opt_name_or_path+from ...helpers import get_model_path, opt_name_or_pathclass FakeDataLoader:diff --git a/test/test_cpu/test_moe_alignment.py b/test/test_cpu/models/test_moe_alignment.pysimilarity index 99%rename from test/test_cpu/test_moe_alignment.pyrename to test/test_cpu/models/test_moe_alignment.pyindex 3e689a7e0..9b6374f70 100644--- a/test/test_cpu/test_moe_alignment.py+++ b/test/test_cpu/models/test_moe_alignment.py@@ -8,7 +8,7 @@ from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRoundfrom auto_round.utils.model import get_module, set_amax_for_all_moe_layers-from ..helpers import get_model_path+from ...helpers import get_model_pathdeepseek_v2_lite_path = get_model_path("deepseek-ai/DeepSeek-V2-Lite-Chat")diff --git a/test/test_cpu/test_moe_model.py b/test/test_cpu/models/test_moe_model.pysimilarity index 99%rename from test/test_cpu/test_moe_model.pyrename to test/test_cpu/models/test_moe_model.pyindex 397e74820..89a6c8b13 100644--- a/test/test_cpu/test_moe_model.py+++ b/test/test_cpu/models/test_moe_model.py@@ -7,7 +7,7 @@ from transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe import Qwen3VLMoeForfrom auto_round import AutoRound-from ..helpers import get_model_path+from ...helpers import get_model_pathgpt_oss_name_or_path = get_model_path("unsloth/gpt-oss-20b-BF16")llama4_name_or_path = get_model_path("meta-llama/Llama-4-Scout-17B-16E-Instruct")diff --git a/test/test_cpu/quantization/__init__.py b/test/test_cpu/quantization/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_act_quantization.py b/test/test_cpu/quantization/test_act_quantization.pysimilarity index 100%rename from test/test_cpu/test_act_quantization.pyrename to test/test_cpu/quantization/test_act_quantization.pydiff --git a/test/test_cpu/test_asym.py b/test/test_cpu/quantization/test_asym.pysimilarity index 98%rename from test/test_cpu/test_asym.pyrename to test/test_cpu/quantization/test_asym.pyindex 32a0151b3..587de1b9c 100644--- a/test/test_cpu/test_asym.py+++ b/test/test_cpu/quantization/test_asym.py@@ -12,7 +12,7 @@ from auto_round import AutoRoundfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.utils import get_module-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferclass LLMDataLoader:diff --git a/test/test_cpu/test_mix_bits.py b/test/test_cpu/quantization/test_mix_bits.pysimilarity index 99%rename from test/test_cpu/test_mix_bits.pyrename to test/test_cpu/quantization/test_mix_bits.pyindex 6cc390637..5db3053cb 100644--- a/test/test_cpu/test_mix_bits.py+++ b/test/test_cpu/quantization/test_mix_bits.py@@ -10,7 +10,7 @@ from transformers import AutoModelForCausalLM, AutoRoundConfig, AutoTokenizerfrom auto_round import AutoRoundfrom auto_round.testing_utils import require_gptqmodel-from ..helpers import opt_name_or_path+from ...helpers import opt_name_or_pathdef _get_folder_size(path: str) -> float:diff --git a/test/test_cpu/test_mx_quant_linear.py b/test/test_cpu/quantization/test_mx_quant_linear.pysimilarity index 100%rename from test/test_cpu/test_mx_quant_linear.pyrename to test/test_cpu/quantization/test_mx_quant_linear.pydiff --git a/test/test_cpu/test_mxfp_nvfp.py b/test/test_cpu/quantization/test_mxfp_nvfp.pysimilarity index 99%rename from test/test_cpu/test_mxfp_nvfp.pyrename to test/test_cpu/quantization/test_mxfp_nvfp.pyindex 8baaf110e..f5044bc73 100644--- a/test/test_cpu/test_mxfp_nvfp.py+++ b/test/test_cpu/quantization/test_mxfp_nvfp.py@@ -7,7 +7,7 @@ from transformers import AutoModelForCausalLM, AutoRoundConfig, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import is_model_outputs_similar+from ...helpers import is_model_outputs_similardef _get_folder_size(path: str) -> float:diff --git a/test/test_cpu/test_mxfp_save_load.py b/test/test_cpu/quantization/test_mxfp_save_load.pysimilarity index 98%rename from test/test_cpu/test_mxfp_save_load.pyrename to test/test_cpu/quantization/test_mxfp_save_load.pyindex 528b0a107..396c47735 100644--- a/test/test_cpu/test_mxfp_save_load.py+++ b/test/test_cpu/quantization/test_mxfp_save_load.py@@ -14,7 +14,7 @@ from auto_round.formats import AutoRoundExportFormatfrom auto_round.inference.backend import MX_TENSOR_DATA_TYPESfrom auto_round.testing_utils import has_module-from ..helpers import get_model_path+from ...helpers import get_model_pathtesting_scheme_name_lst = [AutoRoundExportFormat.MXFP8.value,diff --git a/test/test_cpu/test_nvfp4_quant_linear.py b/test/test_cpu/quantization/test_nvfp4_quant_linear.pysimilarity index 100%rename from test/test_cpu/test_nvfp4_quant_linear.pyrename to test/test_cpu/quantization/test_nvfp4_quant_linear.pydiff --git a/test/test_cpu/schemes/__init__.py b/test/test_cpu/schemes/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_auto_scheme.py b/test/test_cpu/schemes/test_auto_scheme.pysimilarity index 100%rename from test/test_cpu/test_auto_scheme.pyrename to test/test_cpu/schemes/test_auto_scheme.pydiff --git a/test/test_cpu/test_scheme.py b/test/test_cpu/schemes/test_scheme.pysimilarity index 98%rename from test/test_cpu/test_scheme.pyrename to test/test_cpu/schemes/test_scheme.pyindex 8ac1c9b70..f4f0e716b 100644--- a/test/test_cpu/test_scheme.py+++ b/test/test_cpu/schemes/test_scheme.py@@ -5,7 +5,7 @@ import transformersfrom auto_round import AutoRoundfrom auto_round.schemes import QuantizationScheme-from ..helpers import get_model_path, get_tiny_model, opt_name_or_path, qwen_name_or_path+from ...helpers import get_model_path, get_tiny_model, opt_name_or_path, qwen_name_or_pathclass TestAutoRound:diff --git a/test/test_cpu/utils/__init__.py b/test/test_cpu/utils/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cpu/test_alg_ext.py b/test/test_cpu/utils/test_alg_ext.pysimilarity index 96%rename from test/test_cpu/test_alg_ext.pyrename to test/test_cpu/utils/test_alg_ext.pyindex 0bfdfba47..2daf2aada 100644--- a/test/test_cpu/test_alg_ext.py+++ b/test/test_cpu/utils/test_alg_ext.py@@ -1,6 +1,6 @@from auto_round import AutoRound-from ..helpers import qwen_name_or_path+from ...helpers import qwen_name_or_pathclass TestAlgExt:diff --git a/test/test_cpu/test_calib_dataset.py b/test/test_cpu/utils/test_calib_dataset.pysimilarity index 98%rename from test/test_cpu/test_calib_dataset.pyrename to test/test_cpu/utils/test_calib_dataset.pyindex cb276147e..db016f6c7 100644--- a/test/test_cpu/test_calib_dataset.py+++ b/test/test_cpu/utils/test_calib_dataset.py@@ -8,7 +8,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import get_model_path, opt_name_or_path+from ...helpers import get_model_path, opt_name_or_pathclass TestLocalCalibDataset:diff --git a/test/test_cpu/test_cli_usage.py b/test/test_cpu/utils/test_cli_usage.pysimilarity index 98%rename from test/test_cpu/test_cli_usage.pyrename to test/test_cpu/utils/test_cli_usage.pyindex b3aecf2f1..62607c83a 100644--- a/test/test_cpu/test_cli_usage.py+++ b/test/test_cpu/utils/test_cli_usage.py@@ -2,7 +2,7 @@ import osimport shutilimport sys-from ..helpers import get_model_path+from ...helpers import get_model_pathclass TestAutoRoundCmd:diff --git a/test/test_cpu/test_generation.py b/test/test_cpu/utils/test_generation.pysimilarity index 98%rename from test/test_cpu/test_generation.pyrename to test/test_cpu/utils/test_generation.pyindex e1e9dc3f1..6bf7e1020 100644--- a/test/test_cpu/test_generation.py+++ b/test/test_cpu/utils/test_generation.py@@ -7,7 +7,7 @@ from transformers import AutoModelForCausalLM, AutoRoundConfig, AutoTokenizerfrom auto_round import AutoRound-from ..helpers import opt_name_or_path+from ...helpers import opt_name_or_pathclass TestAutoRoundFormatGeneration:diff --git a/test/test_cpu/test_load_awq_gptq.py b/test/test_cpu/utils/test_load_awq_gptq.pysimilarity index 96%rename from test/test_cpu/test_load_awq_gptq.pyrename to test/test_cpu/utils/test_load_awq_gptq.pyindex 6dc295b4e..35686d953 100644--- a/test/test_cpu/test_load_awq_gptq.py+++ b/test/test_cpu/utils/test_load_awq_gptq.py@@ -3,7 +3,7 @@ import shutilimport pytestfrom transformers import AutoModelForCausalLM, AutoRoundConfig, AutoTokenizer-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferclass TestAutoRound:diff --git a/test/test_cpu/test_logger.py b/test/test_cpu/utils/test_logger.pysimilarity index 100%rename from test/test_cpu/test_logger.pyrename to test/test_cpu/utils/test_logger.pydiff --git a/test/test_cpu/test_model_scope.py b/test/test_cpu/utils/test_model_scope.pysimilarity index 97%rename from test/test_cpu/test_model_scope.pyrename to test/test_cpu/utils/test_model_scope.pyindex 7edcab156..b53b73ada 100644--- a/test/test_cpu/test_model_scope.py+++ b/test/test_cpu/utils/test_model_scope.py@@ -7,7 +7,7 @@ import torchfrom auto_round import AutoRound-from ..helpers import get_model_path+from ...helpers import get_model_pathclass TestModelScope:diff --git a/test/test_cpu/test_utils.py b/test/test_cpu/utils/test_utils.pysimilarity index 100%rename from test/test_cpu/test_utils.pyrename to test/test_cpu/utils/test_utils.pydiff --git a/test/test_cpu/test_woq_linear.py b/test/test_cpu/utils/test_woq_linear.pysimilarity index 100%rename from test/test_cpu/test_woq_linear.pyrename to test/test_cpu/utils/test_woq_linear.pydiff --git a/test/test_cuda/advanced/__init__.py b/test/test_cuda/advanced/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_fp8_input.py b/test/test_cuda/advanced/test_fp8_input.pysimilarity index 99%rename from test/test_cuda/test_fp8_input.pyrename to test/test_cuda/advanced/test_fp8_input.pyindex 9e1c1cc3a..9590692a4 100644--- a/test/test_cuda/test_fp8_input.py+++ b/test/test_cuda/advanced/test_fp8_input.py@@ -10,7 +10,7 @@ from auto_round import AutoRoundfrom auto_round.eval.evaluation import simple_evaluatefrom auto_round.utils import llm_load_model-from ..helpers import get_model_path, get_tiny_model+from ...helpers import get_model_path, get_tiny_modelclass TestAutoRound:diff --git a/test/test_cuda/test_multiple_card.py b/test/test_cuda/advanced/test_multiple_card.pysimilarity index 99%rename from test/test_cuda/test_multiple_card.pyrename to test/test_cuda/advanced/test_multiple_card.pyindex e09975a19..ca4dd7cd5 100644--- a/test/test_cuda/test_multiple_card.py+++ b/test/test_cuda/advanced/test_multiple_card.py@@ -10,7 +10,7 @@ from auto_round import AutoRoundfrom auto_round.eval.evaluation import simple_evaluatefrom auto_round.testing_utils import multi_card, require_gptqmodel, require_greater_than_050-from ..helpers import get_model_path, get_tiny_model+from ...helpers import get_model_path, get_tiny_modeldef get_accuracy(data):diff --git a/test/test_cuda/test_multiple_card_calib.py b/test/test_cuda/advanced/test_multiple_card_calib.pysimilarity index 100%rename from test/test_cuda/test_multiple_card_calib.pyrename to test/test_cuda/advanced/test_multiple_card_calib.pydiff --git a/test/test_cuda/backends/__init__.py b/test/test_cuda/backends/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_exllamav2_backend.py b/test/test_cuda/backends/test_exllamav2_backend.pysimilarity index 99%rename from test/test_cuda/test_exllamav2_backend.pyrename to test/test_cuda/backends/test_exllamav2_backend.pyindex d0f5bed53..8d20af99d 100644--- a/test/test_cuda/test_exllamav2_backend.py+++ b/test/test_cuda/backends/test_exllamav2_backend.py@@ -8,7 +8,7 @@ from auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.testing_utils import require_autogptq, require_gptqmodel, require_package_version_ut-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferclass TestAutoRoundexllamaBackend:diff --git a/test/test_cuda/test_marlin_backend.py b/test/test_cuda/backends/test_marlin_backend.pysimilarity index 99%rename from test/test_cuda/test_marlin_backend.pyrename to test/test_cuda/backends/test_marlin_backend.pyindex 8d7594086..3f480c90f 100644--- a/test/test_cuda/test_marlin_backend.py+++ b/test/test_cuda/backends/test_marlin_backend.py@@ -7,7 +7,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate_user_model-from ..helpers import model_infer+from ...helpers import model_inferclass TestAutoRoundMarlinBackend:diff --git a/test/test_cuda/test_torch_backend.py b/test/test_cuda/backends/test_torch_backend.pysimilarity index 98%rename from test/test_cuda/test_torch_backend.pyrename to test/test_cuda/backends/test_torch_backend.pyindex a7eb30552..4594667d9 100644--- a/test/test_cuda/test_torch_backend.py+++ b/test/test_cuda/backends/test_torch_backend.py@@ -8,7 +8,7 @@ from auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.testing_utils import require_autogptq, require_gptqmodel-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferclass TestAutoRoundTorchBackend:diff --git a/test/test_cuda/test_triton_backend.py b/test/test_cuda/backends/test_triton_backend.pysimilarity index 99%rename from test/test_cuda/test_triton_backend.pyrename to test/test_cuda/backends/test_triton_backend.pyindex ac5436f47..f51e8aeba 100644--- a/test/test_cuda/test_triton_backend.py+++ b/test/test_cuda/backends/test_triton_backend.py@@ -8,7 +8,7 @@ from auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.testing_utils import require_greater_than_050-from ..helpers import model_infer+from ...helpers import model_inferclass TestAutoRoundTritonBackend:diff --git a/test/test_cuda/core/__init__.py b/test/test_cuda/core/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_main_func.py b/test/test_cuda/core/test_main_func.pysimilarity index 99%rename from test/test_cuda/test_main_func.pyrename to test/test_cuda/core/test_main_func.pyindex bbd442aca..955096141 100644--- a/test/test_cuda/test_main_func.py+++ b/test/test_cuda/core/test_main_func.py@@ -13,7 +13,7 @@ from auto_round import AutoRound, AutoRoundAdamfrom auto_round.eval.evaluation import simple_evaluatefrom auto_round.testing_utils import require_awq, require_gptqmodel, require_optimum, require_package_version_ut-from ..helpers import get_model_path+from ...helpers import get_model_pathdef get_accuracy(data):diff --git a/test/test_cuda/export/__init__.py b/test/test_cuda/export/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_auto_round_format.py b/test/test_cuda/export/test_auto_round_format.pysimilarity index 99%rename from test/test_cuda/test_auto_round_format.pyrename to test/test_cuda/export/test_auto_round_format.pyindex 821d45fa1..a2753605b 100644--- a/test/test_cuda/test_auto_round_format.py+++ b/test/test_cuda/export/test_auto_round_format.py@@ -16,7 +16,7 @@ from auto_round.testing_utils import (require_package_version_ut,)-from ..helpers import get_model_path, get_tiny_model, model_infer+from ...helpers import get_model_path, get_tiny_model, model_inferclass TestAutoRound:diff --git a/test/test_cuda/test_export.py b/test/test_cuda/export/test_export.pysimilarity index 99%rename from test/test_cuda/test_export.pyrename to test/test_cuda/export/test_export.pyindex c42085317..efd519a51 100644--- a/test/test_cuda/test_export.py+++ b/test/test_cuda/export/test_export.py@@ -9,7 +9,7 @@ from transformers import AutoConfig, AutoModelForCausalLM, AutoRoundConfig, Autofrom auto_round import AutoRoundfrom auto_round.testing_utils import require_awq, require_optimum, require_package_version_ut-from ..helpers import get_model_path, get_tiny_model+from ...helpers import get_model_path, get_tiny_modelclass TestAutoRound:diff --git a/test/test_cuda/test_gguf.py b/test/test_cuda/export/test_gguf.pysimilarity index 99%rename from test/test_cuda/test_gguf.pyrename to test/test_cuda/export/test_gguf.pyindex 395969493..94d3c7461 100644--- a/test/test_cuda/test_gguf.py+++ b/test/test_cuda/export/test_gguf.py@@ -10,7 +10,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRoundfrom auto_round.testing_utils import require_gguf-from ..helpers import get_model_path, get_tiny_model, save_tiny_model+from ...helpers import get_model_path, get_tiny_model, save_tiny_modelclass TestAutoRound:diff --git a/test/test_cuda/integrations/__init__.py b/test/test_cuda/integrations/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/integrations/test_llmc_integration.py b/test/test_cuda/integrations/test_llmc_integration.pynew file mode 120000index 000000000..3422e3cdc--- /dev/null+++ b/test/test_cuda/integrations/test_llmc_integration.py@@ -0,0 +1 @@+../../test_cpu/integrations/test_llmc_integration.py\ No newline at end of filediff --git a/test/test_cuda/test_sglang.py b/test/test_cuda/integrations/test_sglang.pysimilarity index 97%rename from test/test_cuda/test_sglang.pyrename to test/test_cuda/integrations/test_sglang.pyindex 1d8f08052..ac96bed74 100644--- a/test/test_cuda/test_sglang.py+++ b/test/test_cuda/integrations/test_sglang.py@@ -8,7 +8,7 @@ import torchfrom auto_round import AutoRound-from ..helpers import get_model_path, opt_name_or_path+from ...helpers import get_model_path, opt_name_or_pathclass TestAutoRound:diff --git a/test/test_cuda/test_transformers.py b/test/test_cuda/integrations/test_transformers.pysimilarity index 99%rename from test/test_cuda/test_transformers.pyrename to test/test_cuda/integrations/test_transformers.pyindex f37fe94ff..feb2516f7 100644--- a/test/test_cuda/test_transformers.py+++ b/test/test_cuda/integrations/test_transformers.py@@ -27,7 +27,7 @@ from transformers.testing_utils import ()from transformers.utils import is_torch_available-from ..helpers import get_model_path+from ...helpers import get_model_pathif is_torch_available():import torchdiff --git a/test/test_cuda/test_vllm.py b/test/test_cuda/integrations/test_vllm.pysimilarity index 100%rename from test/test_cuda/test_vllm.pyrename to test/test_cuda/integrations/test_vllm.pydiff --git a/test/test_cuda/models/__init__.py b/test/test_cuda/models/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_conv1d.py b/test/test_cuda/models/test_conv1d.pysimilarity index 96%rename from test/test_cuda/test_conv1d.pyrename to test/test_cuda/models/test_conv1d.pyindex 6b955f657..89b82a319 100644--- a/test/test_cuda/test_conv1d.py+++ b/test/test_cuda/models/test_conv1d.py@@ -8,7 +8,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRoundfrom auto_round.testing_utils import require_gptqmodel-from ..helpers import get_model_path, get_tiny_model, model_infer+from ...helpers import get_model_path, get_tiny_model, model_inferclass TestQuantizationConv1d:diff --git a/test/test_cuda/test_diffusion.py b/test/test_cuda/models/test_diffusion.pysimilarity index 100%rename from test/test_cuda/test_diffusion.pyrename to test/test_cuda/models/test_diffusion.pydiff --git a/test/test_cuda/test_get_block_name.py b/test/test_cuda/models/test_get_block_name.pysimilarity index 100%rename from test/test_cuda/test_get_block_name.pyrename to test/test_cuda/models/test_get_block_name.pydiff --git a/test/test_cuda/test_moe_model.py b/test/test_cuda/models/test_moe_model.pysimilarity index 100%rename from test/test_cuda/test_moe_model.pyrename to test/test_cuda/models/test_moe_model.pydiff --git a/test/test_cuda/test_support_vlms.py b/test/test_cuda/models/test_support_vlms.pysimilarity index 100%rename from test/test_cuda/test_support_vlms.pyrename to test/test_cuda/models/test_support_vlms.pydiff --git a/test/test_cuda/test_vlms.py b/test/test_cuda/models/test_vlms.pysimilarity index 100%rename from test/test_cuda/test_vlms.pyrename to test/test_cuda/models/test_vlms.pydiff --git a/test/test_cuda/quantization/__init__.py b/test/test_cuda/quantization/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_2_3bits.py b/test/test_cuda/quantization/test_2_3bits.pysimilarity index 98%rename from test/test_cuda/test_2_3bits.pyrename to test/test_cuda/quantization/test_2_3bits.pyindex 1b305f494..12ed75faa 100644--- a/test/test_cuda/test_2_3bits.py+++ b/test/test_cuda/quantization/test_2_3bits.py@@ -12,7 +12,7 @@ from auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate, simple_evaluate_user_modelfrom auto_round.testing_utils import require_autogptq, require_greater_than_050, require_greater_than_051-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferdef get_accuracy(data):diff --git a/test/test_cuda/test_asym.py b/test/test_cuda/quantization/test_asym.pysimilarity index 99%rename from test/test_cuda/test_asym.pyrename to test/test_cuda/quantization/test_asym.pyindex 1eda6f146..488ef8a73 100644--- a/test/test_cuda/test_asym.py+++ b/test/test_cuda/quantization/test_asym.py@@ -11,7 +11,7 @@ from auto_round import AutoRoundfrom auto_round.eval.evaluation import simple_evaluate_user_modelfrom auto_round.utils import get_module-from ..helpers import model_infer+from ...helpers import model_inferclass LLMDataLoader:diff --git a/test/test_cuda/test_mix_bits.py b/test/test_cuda/quantization/test_mix_bits.pysimilarity index 99%rename from test/test_cuda/test_mix_bits.pyrename to test/test_cuda/quantization/test_mix_bits.pyindex 6988709d5..82da57008 100644--- a/test/test_cuda/test_mix_bits.py+++ b/test/test_cuda/quantization/test_mix_bits.py@@ -14,7 +14,7 @@ from auto_round.testing_utils import (require_package_version_ut,)-from ..helpers import get_model_path+from ...helpers import get_model_pathclass TestAutoRound:diff --git a/test/test_cuda/test_mxfp_and_nvfp_quant.py b/test/test_cuda/quantization/test_mxfp_and_nvfp_quant.pysimilarity index 98%rename from test/test_cuda/test_mxfp_and_nvfp_quant.pyrename to test/test_cuda/quantization/test_mxfp_and_nvfp_quant.pyindex 1abd740c6..d76b556e3 100644--- a/test/test_cuda/test_mxfp_and_nvfp_quant.py+++ b/test/test_cuda/quantization/test_mxfp_and_nvfp_quant.py@@ -12,7 +12,7 @@ from auto_round.export.export_to_autoround import qlinear_fp as ar_qlinear_fpfrom auto_round.formats import AutoRoundExportFormatfrom auto_round.testing_utils import has_module-from ..helpers import get_model_path+from ...helpers import get_model_pathtesting_schemes = [AutoRoundExportFormat.MXFP8.value,diff --git a/test/test_cuda/test_mxfp_nvfp.py b/test/test_cuda/quantization/test_mxfp_nvfp.pysimilarity index 99%rename from test/test_cuda/test_mxfp_nvfp.pyrename to test/test_cuda/quantization/test_mxfp_nvfp.pyindex 41c996b95..922cb9f9b 100644--- a/test/test_cuda/test_mxfp_nvfp.py+++ b/test/test_cuda/quantization/test_mxfp_nvfp.py@@ -9,7 +9,7 @@ from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRoundfrom auto_round.testing_utils import require_awq, require_optimum-from ..helpers import get_model_path, get_tiny_model+from ...helpers import get_model_path, get_tiny_modelclass TestAutoRound:diff --git a/test/test_cuda/test_packing.py b/test/test_cuda/quantization/test_packing.pysimilarity index 100%rename from test/test_cuda/test_packing.pyrename to test/test_cuda/quantization/test_packing.pydiff --git a/test/test_cuda/test_qbits.py b/test/test_cuda/quantization/test_qbits.pysimilarity index 98%rename from test/test_cuda/test_qbits.pyrename to test/test_cuda/quantization/test_qbits.pyindex 37e119b2c..6ebb9d67b 100644--- a/test/test_cuda/test_qbits.py+++ b/test/test_cuda/quantization/test_qbits.py@@ -6,7 +6,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound, AutoRoundConfigfrom auto_round.testing_utils import require_gptqmodel, require_itrex-from ..helpers import get_model_path, model_infer+from ...helpers import get_model_path, model_inferclass TestAutoRound:diff --git a/test/test_cuda/schemes/__init__.py b/test/test_cuda/schemes/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_auto_scheme.py b/test/test_cuda/schemes/test_auto_scheme.pysimilarity index 99%rename from test/test_cuda/test_auto_scheme.pyrename to test/test_cuda/schemes/test_auto_scheme.pyindex 3166e1785..c7aafe8cc 100644--- a/test/test_cuda/test_auto_scheme.py+++ b/test/test_cuda/schemes/test_auto_scheme.py@@ -11,7 +11,7 @@ from auto_round.eval.evaluation import simple_evaluatefrom auto_round.testing_utils import multi_cardfrom auto_round.utils import get_module-from ..helpers import get_model_path, get_tiny_model+from ...helpers import get_model_path, get_tiny_modelclass TestAutoScheme:diff --git a/test/test_cuda/test_scheme.py b/test/test_cuda/schemes/test_scheme.pysimilarity index 99%rename from test/test_cuda/test_scheme.pyrename to test/test_cuda/schemes/test_scheme.pyindex 08156c927..368737946 100644--- a/test/test_cuda/test_scheme.py+++ b/test/test_cuda/schemes/test_scheme.py@@ -5,7 +5,7 @@ import pytestfrom auto_round import AutoRoundfrom auto_round.schemes import QuantizationScheme-from ..helpers import get_model_path+from ...helpers import get_model_pathclass TestAutoRound:diff --git a/test/test_cuda/test_llmc_integration.py b/test/test_cuda/test_llmc_integration.pydeleted file mode 120000index ef860e8d6..000000000--- a/test/test_cuda/test_llmc_integration.py+++ /dev/null@@ -1 +0,0 @@-../test_cpu/test_llmc_integration.py\ No newline at end of filediff --git a/test/test_cuda/utils/__init__.py b/test/test_cuda/utils/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/test/test_cuda/test_alg_ext.py b/test/test_cuda/utils/test_alg_ext.pysimilarity index 98%rename from test/test_cuda/test_alg_ext.pyrename to test/test_cuda/utils/test_alg_ext.pyindex 6cdbc82ab..5ddc02ed4 100644--- a/test/test_cuda/test_alg_ext.py+++ b/test/test_cuda/utils/test_alg_ext.py@@ -8,7 +8,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizerfrom auto_round import AutoRound, AutoRoundConfigfrom auto_round.eval.evaluation import simple_evaluate_user_model-from ..helpers import get_model_path+from ...helpers import get_model_pathclass TestAlgExt:diff --git a/test/test_cuda/test_calib_dataset.py b/test/test_cuda/utils/test_calib_dataset.pysimilarity index 100%rename from test/test_cuda/test_calib_dataset.pyrename to test/test_cuda/utils/test_calib_dataset.pydiff --git a/test/test_cuda/test_customized_data.py b/test/test_cuda/utils/test_customized_data.pysimilarity index 100%rename from test/test_cuda/test_customized_data.pyrename to test/test_cuda/utils/test_customized_data.py