We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Thanks for your great work !
I am applying GPTQ quantization to my fine-tuned Qwen2-VL model, and when I using transformers to load the GPTQ model, it shows:
Some weights of the model checkpoint at /home/haige/ckq/Qwen2-VL/train_results/qwen2_vl_full_sft_5_gptq_int4 were not used when initializing Qwen2VLForConditionalGeneration: ['model.layers.0.mlp.down_proj.bias', 'model.layers.0.mlp.gate_proj.bias', 'model.layers.0.mlp.up_proj.bias', 'model.layers.0.self_attn.o_proj.bias', 'model.layers.1.mlp.down_proj.bias', 'model.layers.1.mlp.gate_proj.bias', 'model.layers.1.mlp.up_proj.bias', 'model.layers.1.self_attn.o_proj.bias', 'model.layers.10.mlp.down_proj.bias', 'model.layers.10.mlp.gate_proj.bias', 'model.layers.10.mlp.up_proj.bias', 'model.layers.10.self_attn.o_proj.bias', 'model.layers.11.mlp.down_proj.bias', 'model.layers.11.mlp.gate_proj.bias', 'model.layers.11.mlp.up_proj.bias', 'model.layers.11.self_attn.o_proj.bias', 'model.layers.12.mlp.down_proj.bias', 'model.layers.12.mlp.gate_proj.bias', 'model.layers.12.mlp.up_proj.bias', 'model.layers.12.self_attn.o_proj.bias', 'model.layers.13.mlp.down_proj.bias', 'model.layers.13.mlp.gate_proj.bias', 'model.layers.13.mlp.up_proj.bias', 'model.layers.13.self_attn.o_proj.bias', 'model.layers.14.mlp.down_proj.bias', 'model.layers.14.mlp.gate_proj.bias', 'model.layers.14.mlp.up_proj.bias', 'model.layers.14.self_attn.o_proj.bias', 'model.layers.15.mlp.down_proj.bias', 'model.layers.15.mlp.gate_proj.bias', 'model.layers.15.mlp.up_proj.bias', 'model.layers.15.self_attn.o_proj.bias', 'model.layers.16.mlp.down_proj.bias', 'model.layers.16.mlp.gate_proj.bias', 'model.layers.16.mlp.up_proj.bias', 'model.layers.16.self_attn.o_proj.bias', 'model.layers.17.mlp.down_proj.bias', 'model.layers.17.mlp.gate_proj.bias', 'model.layers.17.mlp.up_proj.bias', 'model.layers.17.self_attn.o_proj.bias', 'model.layers.18.mlp.down_proj.bias', 'model.layers.18.mlp.gate_proj.bias', 'model.layers.18.mlp.up_proj.bias', 'model.layers.18.self_attn.o_proj.bias', 'model.layers.19.mlp.down_proj.bias', 'model.layers.19.mlp.gate_proj.bias', 'model.layers.19.mlp.up_proj.bias', 'model.layers.19.self_attn.o_proj.bias', 'model.layers.2.mlp.down_proj.bias', 'model.layers.2.mlp.gate_proj.bias', 'model.layers.2.mlp.up_proj.bias', 'model.layers.2.self_attn.o_proj.bias', 'model.layers.20.mlp.down_proj.bias', 'model.layers.20.mlp.gate_proj.bias', 'model.layers.20.mlp.up_proj.bias', 'model.layers.20.self_attn.o_proj.bias', 'model.layers.21.mlp.down_proj.bias', 'model.layers.21.mlp.gate_proj.bias', 'model.layers.21.mlp.up_proj.bias', 'model.layers.21.self_attn.o_proj.bias', 'model.layers.22.mlp.down_proj.bias', 'model.layers.22.mlp.gate_proj.bias', 'model.layers.22.mlp.up_proj.bias', 'model.layers.22.self_attn.o_proj.bias', 'model.layers.23.mlp.down_proj.bias', 'model.layers.23.mlp.gate_proj.bias', 'model.layers.23.mlp.up_proj.bias', 'model.layers.23.self_attn.o_proj.bias', 'model.layers.24.mlp.down_proj.bias', 'model.layers.24.mlp.gate_proj.bias', 'model.layers.24.mlp.up_proj.bias', 'model.layers.24.self_attn.o_proj.bias', 'model.layers.25.mlp.down_proj.bias', 'model.layers.25.mlp.gate_proj.bias', 'model.layers.25.mlp.up_proj.bias', 'model.layers.25.self_attn.o_proj.bias', 'model.layers.26.mlp.down_proj.bias', 'model.layers.26.mlp.gate_proj.bias', 'model.layers.26.mlp.up_proj.bias', 'model.layers.26.self_attn.o_proj.bias', 'model.layers.27.mlp.down_proj.bias', 'model.layers.27.mlp.gate_proj.bias', 'model.layers.27.mlp.up_proj.bias', 'model.layers.27.self_attn.o_proj.bias', 'model.layers.3.mlp.down_proj.bias', 'model.layers.3.mlp.gate_proj.bias', 'model.layers.3.mlp.up_proj.bias', 'model.layers.3.self_attn.o_proj.bias', 'model.layers.4.mlp.down_proj.bias', 'model.layers.4.mlp.gate_proj.bias', 'model.layers.4.mlp.up_proj.bias', 'model.layers.4.self_attn.o_proj.bias', 'model.layers.5.mlp.down_proj.bias', 'model.layers.5.mlp.gate_proj.bias', 'model.layers.5.mlp.up_proj.bias', 'model.layers.5.self_attn.o_proj.bias', 'model.layers.6.mlp.down_proj.bias', 'model.layers.6.mlp.gate_proj.bias', 'model.layers.6.mlp.up_proj.bias', 'model.layers.6.self_attn.o_proj.bias', 'model.layers.7.mlp.down_proj.bias', 'model.layers.7.mlp.gate_proj.bias', 'model.layers.7.mlp.up_proj.bias', 'model.layers.7.self_attn.o_proj.bias', 'model.layers.8.mlp.down_proj.bias', 'model.layers.8.mlp.gate_proj.bias', 'model.layers.8.mlp.up_proj.bias', 'model.layers.8.self_attn.o_proj.bias', 'model.layers.9.mlp.down_proj.bias', 'model.layers.9.mlp.gate_proj.bias', 'model.layers.9.mlp.up_proj.bias', 'model.layers.9.self_attn.o_proj.bias'] - This IS expected if you are initializing Qwen2VLForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
Besides, when I using vLLM to load teh GPTQ model, it shows :
[rank0]: Traceback (most recent call last): [rank0]: File "/home/haige/ckq/arm-llm-dev/arm_crl/vlm_arm_api.py", line 15, in <module> [rank0]: from llm_arm.vlm_qwen2vl_vllm import vlm_plan [rank0]: File "/home/haige/ckq/arm-llm-dev/arm_crl/llm_arm/vlm_qwen2vl_vllm.py", line 21, in <module> [rank0]: llm = LLM( [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/entrypoints/llm.py", line 178, in __init__ [rank0]: self.llm_engine = LLMEngine.from_engine_args( [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 557, in from_engine_args [rank0]: engine = cls( [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 324, in __init__ [rank0]: self.model_executor = executor_class( [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 47, in __init__ [rank0]: self._init_executor() [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/executor/gpu_executor.py", line 40, in _init_executor [rank0]: self.driver_worker.load_model() [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/worker/worker.py", line 183, in load_model [rank0]: self.model_runner.load_model() [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 999, in load_model [rank0]: self.model = get_model(model_config=self.model_config, [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 19, in get_model [rank0]: return loader.load_model(model_config=model_config, [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 361, in load_model [rank0]: model.load_weights( [rank0]: File "/home/haige/miniconda3/envs/vllm/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_vl.py", line 1092, in load_weights [rank0]: param = params_dict[name] [rank0]: KeyError: 'model.layers.0.mlp.down_proj.bias'
Could you please provide some helps ?
The text was updated successfully, but these errors were encountered:
kq-chen
No branches or pull requests
Thanks for your great work !
I am applying GPTQ quantization to my fine-tuned Qwen2-VL model, and when I using transformers to load the GPTQ model, it shows:
Besides, when I using vLLM to load teh GPTQ model, it shows :
Could you please provide some helps ?
The text was updated successfully, but these errors were encountered: