PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
fix typo
wenhuach21c2798a0bc09f
diff --git a/README.md b/README.mdindex 960c7dcec..0f1c4c76a 100644--- a/README.md+++ b/README.md@@ -185,34 +185,6 @@ print(tokenizer.decode(model.generate(**inputs, max_new_tokens=50)[0]))```-## Support List--| Model | Supported |-|--------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-| Intel/neural-chat-7b-v3-3 | [HF-int4-model](https://huggingface.co/Intel/neural-chat-7b-v3-3-int4-inc), [acc](./docs/neural-chat-7b-v3-3-acc.md), [recipe](./examples/language-modeling/scripts/neural-chat-7b-v3-3.sh), [example](./examples/language-modeling/) |-| Intel/neural-chat-7b-v3-1 | [HF-int4-model](https://huggingface.co/Intel/neural-chat-7b-v3-1-int4-inc), [acc](./docs/neural-chat-7b-v3-1-acc.md), [recipe](./examples/language-modeling/scripts/neural-chat-7b-v3-1.sh), [example](./examples/language-modeling/) |-| mistralai/Mistral-7B-v0.1 | [HF-int4-model](https://huggingface.co/Intel/Mistral-7B-v0.1-int4-inc), [acc](./docs/Mistral-7B-v0.1-acc.md), [recipe](./examples/language-modeling/scripts/Mistral-7B-v0.1.sh), [example](./examples/language-modeling/) |-| google/gemma-7b | [HF-int4-model](https://huggingface.co/Intel/gemma-7b-int4-inc) in review, [acc](./docs/gemma-7b-acc.md), [recipe](./examples/language-modeling/scripts/gemma-7b.sh), [example](./examples/language-modeling/) |-| google/gemma-7b-it | [HF-int4-model](https://huggingface.co/Intel/gemma-7b-it-int4-inc) in review, [acc](./docs/gemma-7b-it-acc.md), [recipe](./examples/language-modeling/scripts/gemma-7b-it.sh), [example](./examples/language-modeling/) | |- mistralai/Mixtral-8x7B-Instruct-v0.1 | [HF-int4-model](https://huggingface.co/Intel/Mistral-7B-v0.1-int4-inc) in review, [acc](./docs/Mixtral-8x7B-Instruct-v0.1-acc.md), [recipe](./examples/language-modeling/scripts/Mixtral-8x7B-Instruct-v0.1.sh), [example](./examples/language-modeling/) |-| mistralai/Mixtral-8x7B-v0.1 | [HF-int4-model](https://huggingface.co/Intel/Mixtral-8x7B-v0.1-int4-inc) in review, [acc](./docs/Mixtral-8x7B-v0.1-acc.md), [recipe](./examples/language-modeling/scripts/Mixtral-8x7B-v0.1.sh), [example](./examples/language-modeling/) |-| microsoft/phi-2 | [HF-int4-model](https://huggingface.co/Intel/phi-2-int4-inc) in review, [acc](./docs/phi-2-acc.md), [recipe](./examples/language-modeling/scripts/phi-2.sh), [example](./examples/language-modeling/) |-| Salesforce/codegen25-7b-multi | [example](./examples/code-generation) |-| EleutherAI/gpt-j-6b | [example](./examples/language-modeling/) |-| huggyllama/llama-7b | [example](./examples/language-modeling/) |-| meta-llama/Llama-2-7b-hf | [example](./examples/language-modeling/) |-| facebook/opt-6.7b | [example](./examples/language-modeling/) |-| tiiuae/falcon-7b | [example](./examples/language-modeling/) |-| mosaicml/mpt-7b | [example](./examples/language-modeling/) |-| bigscience/bloom-7b1 | [example](./examples/language-modeling/) |-| baichuan-inc/Baichuan-7B | [example](./examples/language-modeling/) |-| Qwen/Qwen-7B | [example](./examples/language-modeling/) |-| THUDM/chatglm3-6b | [example](./examples/language-modeling/) |-| MBZUAI/LaMini-GPT-124M | [example](./examples/language-modeling/) |-| EleutherAI/gpt-neo-125m | [example](./examples/language-modeling/) |-| databricks/dolly-v2-3b | [example](./examples/language-modeling/) |-| stabilityai/stablelm-base-alpha-3b | [example](./examples/language-modeling/) |-## Comparison with other methodsdiff --git a/auto_round/autoround.py b/auto_round/autoround.pyindex fb492f926..b47271d63 100644--- a/auto_round/autoround.py+++ b/auto_round/autoround.py@@ -491,8 +491,8 @@ class AutoRound(object):dataset_split (str): The split of the dataset to be used (default is "train").use_quant_input (bool): Whether to use quantized input data (default is True).enable_minmax_tuning (bool): Whether to enable min-max tuning (default is True).- lr (float): The learning rate (default is 0.005).- minmax_lr (float): The learning rate for min-max tuning (default is None).+ lr (float): The learning rate (default is None, will be set to 1.0/iters).+ minmax_lr (float): The learning rate for min-max tuning (default is None, will be set to 1.0/iters).low_gpu_mem_usage (bool): Whether to use low GPU memory (default is True).iters (int): Number of iterations (default is 200).seqlen (int): Length of the sequence.@@ -504,7 +504,7 @@ class AutoRound(object):not_use_best_mse (bool): Whether to use mean squared error (default is False).dynamic_max_gap (int): The dynamic maximum gap (default is -1).data_type (str): The data type to be used (default is "int").- scale_dtype (str): The data type of quantization scale to be used (default is "float32")+ scale_dtype (str): The data type of quantization scale to be used (default is "fp32")**kwargs: Additional keyword arguments.Returns: