Support for int2/int3 quantization

[ZHITENGLI]

Hi, I would like to inquire if you plan to support INT2/INT3 quantization in the near future, or if it's possible to implement them by modifying some kernels. I'm interested in understanding the effort required to support INT2/INT3 quantization. Additionally, I noticed that AWQ supports INT3 with 128 groups (int3g128), while trt-llm currently only supports INT4. Could you provide more insight into this?

[nv-guomingz]

@Tracin would u please answer this question?

[Tracin]

INT2/INT3 is feasible I think. You can wrote a new plugin for TRT and implement the kernel function.

[ZHITENGLI]

So for int2/int3, we need to rewrite the kernel function instead of reusing the existing int4 framework？