You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Start computing and quantizing weights... This may take a while.
Traceback (most recent call last):
File "~/mlc-llm/mlc_llm/build.py", line 42, in main
core.build_model_from_args(parsed_args)
File "~/mlc-llm/mlc_llm/core.py", line 619, in build_model_from_args
new_params = utils.convert_weights(param_manager, params, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/mlc-llm/mlc_llm/utils.py", line 258, in convert_weights
vm["transform_params"]()
File "tvm/_ffi/_cython/./packed_func.pxi", line 332, in tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 263, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 252, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 182, in tvm._ffi._cy3.core.CHECK_CALL
File "~/mambaforge/envs/mlc/lib/python3.11/site-packages/tvm/_ffi/base.py", line 476, in raise_last_ffi_error
raise py_err
File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback
File "~/mlc-llm/mlc_llm/relax_model/param_manager.py", line 558, in get_item
for torch_binname in [
^
File "~/mlc-llm/mlc_llm/relax_model/param_manager.py", line 559, in <listcomp>
self.torch_pname2binname[torch_pname] for torch_pname in torch_pnames
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'model.layers.0.self_attn.q_proj.weight'
The text was updated successfully, but these errors were encountered:
I quantized a custom fine-tuned llama2 70b model like this.
Then I updated
mlc_llm/quantization/__init__.py
like thisWhen I try to compile the model with mlc-llm,
I got this error.
The text was updated successfully, but these errors were encountered: