Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Pytorch Backend Support to Ivy Transpiler #28848

Open
YushaArif99 opened this issue Dec 17, 2024 · 1 comment
Open

Add Pytorch Backend Support to Ivy Transpiler #28848

YushaArif99 opened this issue Dec 17, 2024 · 1 comment
Labels
ToDo A ToDo list of tasks Transpiler Anything related to transpiling

Comments

@YushaArif99
Copy link
Contributor

Description:
The current implementation of ivy.transpile supports "jax","numpy" and "tensorflow" as valid target argument. This allows transpiling framework-specific code such as PyTorch functions or classes to target frameworks like TensorFlow, JAX, or NumPy. This task aims to extend the functionality by adding Pytorch as a valid target, enabling code to be transpiled to Pytorch.

For example, after completing this task, we should be able to transpile TensorFlow code using:

ivy.transpile(func, source="tensorflow", target="torch")

Goals:

The main objective is to implement the final stage of the transpilation pipeline:

  1. Lower Ivy IR representation to native Pytorch code.

Once this stage is complete, the rest of the pipeline can be reused to target other frameworks like JAX, PyTorch, or NumPy. The steps would look as follows:

source='tensorflow' → target='tensorflow_frontend'  
source='tensorflow_frontend' → target='ivy'  
source='ivy' → target='torch'  

This mirrors the existing pipeline for eg: PyTorch -> TF:

source='torch' → target='torch_frontend'  
source='torch_frontend' → target='ivy'  
source='ivy' → target='tensorflow'  

Key Tasks:

  1. Add Native Framework-Specific Implementations for Core Transformation Passes:
    • For example, implement the ivy_to_torch_postprocessing_transformer.py for doing postprocessing on the Ivy code when lowering it to Pytorch (eg: mapping ivy dtypes (eg: ivy.float32) to torch dtypes(torch.float32)
    • Use ivy_to_jax_postprocessing_transformer.py as a reference (example here)

2 Define the Transformation Pipeline for Ivy IR to Torch

  • Create a new pipeline in ivy_to_source_translator_config.py to handle the stage source='ivy', target='torch' (example here).
  1. Add Stateful Classes for torch.nn APIs:

    • Add a subclass implementation for torch.nn.Module that exposes the similar interface as that of ivy.Module.
    • Reference the existing implementation for keras.Model or nnx.Module(example 1) (example 2)
    • This allows for sequential lowering for eg:
      keras.Model → (frontend keras.Model) → ivy.Module → nn.Module
      
  2. Understand and Leverage Reusability:

    • Explore reusable components in the existing PyTorch pipeline, especially for AST transformers and configuration management.

Testing:

  • Familiarize yourself with the transpilation flow by exploring transpiler tests
  • Add appropriate tests to validate Pytorch target transpilation at the final stage of the pipeline.

Additional Notes:

  • Keep in mind the modular and extensible design of the transpiler, ensuring that the new implementation integrates smoothly into the existing architecture.
@YushaArif99 YushaArif99 added ToDo A ToDo list of tasks Transpiler Anything related to transpiling labels Dec 17, 2024
@YushaArif99 YushaArif99 added Sub Task a sub task which is stemming from a ToDo list issue and removed Sub Task a sub task which is stemming from a ToDo list issue labels Dec 17, 2024
@YushaArif99 YushaArif99 modified the milestones: #28847, #28846 Dec 17, 2024
@YushaArif99
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ToDo A ToDo list of tasks Transpiler Anything related to transpiling
Projects
None yet
Development

No branches or pull requests

2 participants