feature request: support different versions of vllm

[Cppowboy]

No description provided.

[vermouth1992]

Hi,
vllm is currently supporting SPMD-based runner. Once it is merged, verl can fully adopt it without pinning vllm version or modifying vllm any longer.

See

• Support torchrun and SPMD-style offline inference vllm-project/vllm#12071
• [core] LLM.collective_rpc interface and RLHF example vllm-project/vllm#12084

[cgq15]

Seems that it's already merged

[vermouth1992]

Seems that it's already merged

Yes. We are currently integrating nightly vllm and will release the next version when vllm releases next version.

[SalmanMohammadi]

Would this mean that verl could be used without Ray, and with torchrun instead?

[vermouth1992]

Would this mean that verl could be used without Ray, and with torchrun instead?

Technically yes. But we wouldn't provide this kind of script. Actually, colocating all models together is equivalent to torchrun except ray-based solution also have a single process controller. Directly using torchrun makes extending new functionalities very difficult, especially when involving concurrent running something.

The programming abstraction of verl is called HybridFlow, which combines single controller and multi-controller. Single controller provides flexibility of controling RLHF flows while multi-controller provides high performance computation.