Custom driver support for KubeflowV2DagRunner

[jeongukjae]

System information

• TFX Version (you are using): 1.14
• Environment in which you plan to use the feature (e.g., Local
  (Linux/MacOS/Windows), Interactive Notebook, Google Cloud, etc..): Vertex AI
• Are you willing to contribute it (Yes/No): yes

Describe the feature and the current behavior/state.

Currently, KubeflowV2DagRunner does not support custom driver class. It only supports file based example gen driver which is located in

https://github.com/tensorflow/tfx/blob/master/tfx/orchestration/kubeflow/v2/file_based_example_gen/driver.py

Will this change the current API? How?

Not expected to change the API.
But this will change following codes.

• https://github.com/tensorflow/tfx/tree/master/tfx/orchestration/kubeflow/v2/file_based_example_gen
• https://github.com/tensorflow/tfx/blob/master/tfx/orchestration/kubeflow/v2/step_builder.py

Who will benefit with this feature?

Developers who want to leverage the tfx concepts more widely.

Do you have a workaround or are completely blocked by this? :

Blocked by this now.

Name of your Organization (Optional)

Any Other info.

[singhniraj08]

@jeongukjae, Since you mentioned you are willing to contribute for this feature, I will request you to go ahead a create a PR for feature implementation. Thank you for the contributions.

[github-actions]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[jeongukjae]

do not close this