Possibility to have subflow fully included in flow (template-like behaviour)

[Ben8t]

Feature description

Problem

With the deprecation of templates, there are some flaws arising with the only use of Subflow:

1. It doubles the number of execution. With templates, tasks were passed directly in the flow, like if they were written directly inside it. Before 1 flow = 1 exec. Now with subflow 1 flow = 2 exec. Making the executions lists overwhelming in the UI.

2. Some users track billing information, with 1 namespace = 1 project. Using subflow here kind of make this billing wrong as one subflow can be used in many namespaces (so there is an overlap sometimes in the executions).

3. Subflow have to be in the same namespace to have access to the same right (service account)

Possible Solutions

• For the execution problem (1. & 2.):
  • can be handle with labels or filtering: bring a flag subflow on the flow, that allow to hide subflow from the list
  • add a included:true property to the io.kestra.core.tasks.flows.Flow :

id: parent
namespace: company.team

tasks:
  - id: call_subflow
    type: io.kestra.plugin.core.flow.Subflow
    namespace: company.team
    flowId: subflow
    inputs:
      user: Rick Astley
      favorite_song: Never Gonna Give You Up
    wait: true
    transmitFailed: true
    included: true 

Without this kind of flag, currently the flow will create two executions: one for the main flow, one for the subflow. The included property let the user to choose if the subflow can be "included" directly in the subflow - merging the old template behaviour into the subflow one; i.e. having only one execution for the thing.

• For the access control (3.):
  • should be handle in this epic https://github.com/kestra-io/kestra-ee/issues/427

[npranav10]

Hi Kestra Team,

I was going to open a similar issue. @Ben8t has perfectly described my thoughts about point 1. Any updates on this feature request ?

[anna-geller]

@npranav10 you can see the estimate when something will be released in the Milestone on the right side below labels, this one is planned for 0.18 atm.

[anna-geller]

this proposal is great! To avoid breaking changes, confusion and too many properties in the Subflow task, we will handle it as a separate task as described here #6518 -- but 100% the same idea still 👍