[WIP] speed up CodebookQuantizedTensor inference

[DerekLiu35]

This PR tries to speedup inference for #1195

Currently I've ported code1x16 matrix multiplication kernels from AQLM to torchao

Usage

Demo notebook with preliminary tests

• Preliminary tests show a ~2x speedup for linear layer compared to the fallback implementation.
• Matches AQLM's performance for (1, 8) block sizes
• quantization error of kernel seems much higher than fallback implementation, need to investigate more

ToDo

• Improve performance for block sizes of (1, 1).
• Add triton fall back for other block sizes
• Add tests.
• Update benchmarks

Would appreciate feedback on approach and preliminary results
@jerryzh168 @pawarmanasi07

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1607

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.