quantization flux-dev int8 or fp8 occur oom in L40s

[algorithmconquer]

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 72.00 MiB. GPU 0 has a total capacity of 44.53 GiB of which 70.31 MiB is free. Process 266383 has 44.45 GiB memory in use. Of the allocated memory 43.77 GiB is allocated by PyTorch, and 186.54 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[jingyu-ml]

#96

Could you please try this solution? @algorithmconquer
#96 (comment)