Support branch/condition component

[zhitaoli]

The current data aware TFX pipeline and component does not support conditional execution or branching (consider the PythonBranchOperator in Airflow or Argo's conditional). We should provide some support here.

@ruoyu90 @krazyhaas

[robertlugg]

Hi @zhitaoli, it is possible to filter data based on some criteria. This criteria usually is defined in terms of your data. But, it wouldn't have to be. You could just create a component with a parameter "switch" that makes your decision. This component could output all the data, some of the data, none of the data, or even output some data in one output channel and the other data in a different output channel. Would this type of strategy work for your use case?

[ConverJens]

@zhitaoli Is there an update to this? My main motivation would be to enable/disable Tuner based on a RuntimeParameter in KubeFlow.

[zhitaoli]

@kennethyang404 and @ruoyu90 will work on this. Can you confirm whether branching based on value of a RuntimeParameter is part of the design scope?

[lego0901]

Sorry for the very late response, but this looks viable through tfx.v1.dsl.Cond. Thanks!