Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fuse pointwise operations into matmul / convolution operations #371

Open
robertknight opened this issue Sep 21, 2024 · 0 comments
Open

Fuse pointwise operations into matmul / convolution operations #371

robertknight opened this issue Sep 21, 2024 · 0 comments
Labels
performance Issues that affect model inference or loading performance

Comments

@robertknight
Copy link
Owner

robertknight commented Sep 21, 2024

One of the most common optimizations that ML frameworks do is to fuse common pointwise operations such as Relu, Gelu, Silu etc. into preceding convolution / matmul operations. This reduces overhead by applying the pointwise operation while the data is already in the cache. This is not yet implemented in RTen.

@robertknight robertknight added the performance Issues that affect model inference or loading performance label Sep 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Issues that affect model inference or loading performance
Projects
None yet
Development

No branches or pull requests

1 participant