trtllm-bench missing support of moe_ep_size / moe_tp_size.

[juewAtAmazon]

System Info

It appears trtllm-bench is missing support of moe_ep_size / moe_tp_size.

To evaluate MoE model's expert parallelism, e.g., in ref [1], can we get a roadmap update of trtllm-bench's support of MoE parallelisms?

Or please clarify how to get the tooling / process / scripts used in [1] below.

Thanks

1: https://developer.nvidia.com/blog/demystifying-ai-inference-deployments-for-trillion-parameter-large-language-models/

Who can help?

@ncomly-nvidia

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

trtllm-bench --model mistralai/Mixtral-8x22B-v0.1 build --moe_ep_size 4 --pp_size 2 --quantization FP8 --dataset /home/ubuntu/mistral-8x22b.data

Expected behavior

trtllm-bench support MoE parallelisms.

actual behavior

[TensorRT-LLM] TensorRT-LLM version: 0.15.0
Usage: trtllm-bench build [OPTIONS]
Try 'trtllm-bench build --help' for help.

Error: No such option: --moe_ep_size (Possible options: --max_batch_size, --pp_size, --tp_size)

additional notes

n/a