[WIP]Refine Quantizer 2#2039
Conversation
for more information, see https://pre-commit.ci
…to refine_calib # Conflicts: # auto_round/algorithms/pipeline.py # auto_round/algorithms/quantization/base.py
for more information, see https://pre-commit.ci
|
issue repeated log /home/wenhuach/miniforge3/envs/autoround/bin/python /home/wenhuach/auto-round/auto_round/main.py /models/Qwen3-0.6B --tasks lambada_openai --algs awq,auto-round --seqlen 512 |
|
2 no loss print 2026-07-10 17:02:13 INFO mappings.py L560: AWQ resolved 84 smooth-balance mappings (3 per block × 28 blocks). 3 need to check gradient accumulate step passing correctly 4 refine zero_shot |
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
|
|
||
| @torch.no_grad() | ||
| def quantize_block(self, ctx) -> dict: | ||
| def quantize_block(self, block, fp_inputs, input_others, fp_outputs, q_inputs, block_ctx) -> dict: |
There was a problem hiding this comment.
q_inputs default none
…to refine_calib # Conflicts: # auto_round/algorithms/base.py # auto_round/algorithms/quantization/base.py
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
…to refine_calib
for more information, see https://pre-commit.ci
1 participant
Description
Please briefly describe your main changes, the motivation.
Type of Change
Bug fix
Related Issues
Fixes or relates to #
Checklist Before Submitting
/azp run Unit-Test-CUDA-AutoRound.issue 0 verify algs name
/models/Qwen3-8B --tasks lambada_openai --algs haha --seqlen 512 --iters 1 --gradient_accumulate_steps 2
issue repeated log
/home/wenhuach/miniforge3/envs/autoround/bin/python /home/wenhuach/auto-round/auto_round/main.py /models/Qwen3-0.6B --tasks lambada_openai --algs awq,auto-round --seqlen 512
2026-07-10 17:01:26 INFO main.py L292: start to quantize /models/Qwen3-0.6B
2026-07-10 17:01:26 INFO config.py L53: enable_opt_rtn is turned on, set --disable_opt_rtn for higher speed at the cost of accuracy.
2026-07-10 17:01:26 INFO config.py L53: enable_opt_rtn is turned on, set --disable_opt_rtn for higher speed at the cost of accuracy.
issue no loss print
2026-07-10 17:02:13 INFO mappings.py L560: AWQ resolved 84 smooth-balance mappings (3 per block × 28 blocks).
2026-07-10 17:02:13 INFO mappings.py L423: Using registered AWQ mappings for Qwen3ForCausalLM.
2026-07-10 17:02:13 INFO mappings.py L560: AWQ resolved 84 smooth-balance mappings (3 per block × 28 blocks).
2026-07-10 17:02:13 INFO base.py L201: AWQ: resolved 84 mappings across 28 blocks.
Quantizing model.layers.0: 0%| | 0/28 [00:00<?, ?it/s]2026-07-10 17:02:52 INFO device.py L1450: 'peak_ram': 2.54GB, 'peak_vram': 1.25GB
Quantizing model.layers.1: 4%|▌ | 1/28 [00:39<17:35, 39.10s/it]2026-07-10 17:03:24 INFO device.py L1450: 'peak_ram': 2.57GB, 'peak_vram': 1.37GB
Quantizing model.layers.2: 7%|█▏ | 2/28 [01:10<15:04, 34.80s/it]
refine zero_shot
move attention_mask to quantize_block
refine act calibration hook