[Feature Request] Additional max.engine.Model.execute
variants to aid composability
#275
Labels
max.engine.Model.execute
variants to aid composability
#275
What is your request?
Add overloads to
max.engine.Model.execute
to allow taking inputs withVariadicListMem[AnyMemory]
,List[AnyMemory]
andDict[String, AnyMemory]
.What is your motivation for this change?
Currently, due to the lack of support for unpacked arguments (
model.execute(*inputs^)
), there is no way to build abstractions on top of this function from pure Mojo. For instance, I can't write a library where the user provides tensors as input to a wrapper function which loads custom kernels and sets up inference for them if I want to run on the GPU. This prevents a MAX user from writing a "Text" -> "Text" LLM wrapper function optimized for ease of use.This should also enable support for zero-input graphs on the GPU for "free", which is helpful for giving the compiler the full context of computationally expensive pure functions, or for cases where the GPU may be loading data directly from disk in HPC scenarios.
Any other details?
No response
The text was updated successfully, but these errors were encountered: