The ckpt of Quantized OPT model is not be found

[liuxy1103]

We have tried to reproduce the quantization of OPT-30B， but it is still difficult for us. Would you mind releasing the details of the procedure?

[ChenMnZ]

We have released the ckpt of OPT. You can find them at https://huggingface.co/ChenMnZ/OmniQuant/tree/main.

[liuxy1103]

The results by loading the ckpt is not consistent with the reported result in the paper.

[ChenMnZ]

Can you reproduce the results of other models except OPT-30B？

[linloong]

When reproducing evaluation results for OPT-13b-w4a4, I got nan for wikitext2 dataset.

It seems that the checkpoint has also broken now.

And when I try to train by myself, the ppl is very high.

[ChenMnZ]

@linloong Can you provide the training script?

[linloong]

Sure,
Actually, I just use the script you provided in the scripts/opt/opt-13b/w4a4.sh.

CUDA_VISIBLE_DEVICES=0 python main.py \
--model facebook/opt-13b  \
--epochs 20 --output_dir ./log/opt-13b-w4a4 \
--wbits 4 --abits 4 --lwc --let --alpha 0.75

@ChenMnZ