Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Additional max.engine.Model.execute variants to aid composability #275

Open
owenhilyard opened this issue Dec 21, 2024 · 0 comments
Labels
enhancement New feature or request max-repo

Comments

@owenhilyard
Copy link

What is your request?

Add overloads to max.engine.Model.execute to allow taking inputs with VariadicListMem[AnyMemory], List[AnyMemory] and Dict[String, AnyMemory].

What is your motivation for this change?

Currently, due to the lack of support for unpacked arguments (model.execute(*inputs^)), there is no way to build abstractions on top of this function from pure Mojo. For instance, I can't write a library where the user provides tensors as input to a wrapper function which loads custom kernels and sets up inference for them if I want to run on the GPU. This prevents a MAX user from writing a "Text" -> "Text" LLM wrapper function optimized for ease of use.

This should also enable support for zero-input graphs on the GPU for "free", which is helpful for giving the compiler the full context of computationally expensive pure functions, or for cases where the GPU may be loading data directly from disk in HPC scenarios.

Any other details?

No response

@owenhilyard owenhilyard added the enhancement New feature or request label Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request max-repo
Projects
None yet
Development

No branches or pull requests

2 participants