Support inheritance when parent is a list and child is not a list

[bgoncalv]

Looking at https://tmt.readthedocs.io/en/stable/guide.html#inherit-plans

discover:
    how: fmf
    url: https://github.com/teemtee/tmt
prepare:
    how: ansible
    playbook: ansible/packages.yml
execute:
    how: tmt

/basic:
    summary: Quick set of basic functionality tests
    discover+:
        filter: tier:1

/features:
    summary: Detailed tests for individual features
    discover+:
        filter: tier:2

there are 2 plans that extend the discover with filter option. This works well, as discover is a dict.

discover can also be a list, but in that case the inheritance doesn't work.

discover:
    - how: fmf
      url: https://github.com/project1
    - how: fmf
      url: https://github.com/project2

/basic:
    summary: Quick set of basic functionality tests
    discover+:
        filter: tier:1

/features:
    summary: Detailed tests for individual features
    discover+:
        filter: tier:2

It causes error: MergeError: Key 'discover' in /basic (can only concatenate list (not "dict") to list).

My assumption would be that each entry in the list would be updated with provided dictionary.

[psss]

Supporting this use case sounds like a reasonable extension of the current implementation. Might be a little bit confusing to users but, I guess, if it is well documented, should be ok.

To sum up the proposed approach when parent is a list:

• child is a list ... those would be just concatenated
• child is a dict ... [parent_item.update(child_value) for parent_item in parent] would be done
• child is anything else ... [parent_item + child_value for parent_item in parent] would be performed

Any concerns about this approach? I guess, similar should be done for the - operator as well.

[lukaszachy]

+1 for this, should be very useful

[LecrisUT]

I think this should be better handled in tmt and have fmf be more specific. One way would be to expose a plugin for the +/- operators that the specific user of the fmf tree can expand. Currently this can be circumvented by making the discover+: be a list.

[lukaszachy]

@LecrisUT Is that a necessary complication?

What you mean by "Currently this can be circumvented by making the discover+: be a list."?
There is no way how to add / change attribute of discover phase, is it?

e.g. have two phases and add new attribute to each of them

[LecrisUT]

What you mean by "Currently this can be circumvented by making the discover+: be a list."?
There is no way how to add / change attribute of discover phase, is it?

The following should work (note the extra -). I use something similar for tag

discover:
  - how: fmf
    url: https://github.com/project1
  - how: fmf
    url: https://github.com/project2

/basic:
  summary: Quick set of basic functionality tests
  discover+:
    - filter: tier:1

$ fmf show
/test/basic
discover: [{'how': 'fmf', 'url': 'https://github.com/project1'},
 {'how': 'fmf', 'url': 'https://github.com/project2'},
 {'filter': 'tier:1'}]
summary: Quick set of basic functionality tests

@LecrisUT Is that a necessary complication?

Yes, I want to use fmf in more general context outside of tmt. E.g. I am using it with fmf-jinja for generating input files for HPC academic software. I do want to see fmf progress more so I can push this as a stable and feasible workflow for that community.

This change is currently very tmt specific and would break the generality.

[bgoncalv]

What you mean by "Currently this can be circumvented by making the discover+: be a list."?
There is no way how to add / change attribute of discover phase, is it?

The following should work (note the extra -). I use something similar for tag

discover:
  - how: fmf
    url: https://github.com/project1
  - how: fmf
    url: https://github.com/project2

/basic:
  summary: Quick set of basic functionality tests
  discover+:
    - filter: tier:1

this doesn't work. It doesn't update the current entries, but add a new entry to the list {how: shell, filter: 'tier:1'}

[LecrisUT]

Aah, I misunderstood the situation. You mean the + would add the value to all of the items in the list. It feels very error prone, e.g. when you change the type of tag for example from a value to a list it would have surprising consequences.

What about using a more yaml-like syntax like discover<: (derived from <<: *)

[bgoncalv]

What about using a more yaml-like syntax like discover<: (derived from <<: *)

I personally don't have any syntax preference it is just currently afaik there is no way I can add/update items when it is in a list.

[lukaszachy]

Aah, I misunderstood the situation. You mean the + would add the value to all of the items in the list. It feels very error prone, e.g. when you change the type of tag for example from a value to a list it would have surprising consequences.

Whole merging stuff is very error prone already. If time allows there is a plan to provide command to 'explain what is being done with data...' which could put some light into it if one is in bind.

Currently we have traceback when merging list with dict (or any other type) - Thus any change in this sense will not break an existing valid fmf files. The list with list merging is not going to change - for this reason I think it can be done in fmf directly without any callback/plugins currently.

But having way to customize merge_special might be handy for others, that is for sure. Could you file an RFE for that?

[psss]

I'm here with @lukaszachy. The current behavior is not changed, only the invalid combinations (e.g. list+non-list which are now tracebacking) would be interpreted in a different way. So this would be just extending the current implementation. On the other hand, catching exceptions for unsupported operations can be also a valid use case (not for tmt but there could be other applications for the concept).

So, perhaps, a dedicated operator for this could be a little bit better, to make it obvious at the first sight that something different than just a regular addition is happening? Sounds more in the direction of explicit it better than implicit. And I bet that @happz would vote for this approach as well? Or am I wrong?

[happz]

I'm here with @lukaszachy. The current behavior is not changed, only the invalid combinations (e.g. list+non-list which are now tracebacking) would be interpreted in a different way. So this would be just extending the current implementation. On the other hand, catching exceptions for unsupported operations can be also a valid use case (not for tmt but there could be other applications for the concept).

So, perhaps, a dedicated operator for this could be a little bit better, to make it obvious at the first sight that something different than just a regular addition is happening? Sounds more in the direction of explicit it better than implicit. And I bet that @happz would vote for this approach as well? Or am I wrong?

Actually, I would be fine with using the same operator, in this case :) I might be getting it wrong, but IIUIC, the goal is to enable the list + not a list operation, by adding the not a list as an item to the existing list, correct? If that's so, + (and - too) would work for my taste. I can imagine a warning, for example, "merged list and not a list, just so you're aware", but having an operator for "addition" and an operator for "addition but when operands are not compatible" feels like too many details being thrown users' direction.

It seems similar to + in Python: there's still one operator that raises an error if operands are not compatible, and the solution to such an issue is to coerce one of the operands instead of having a new operator. In our case, the coercion would be implicit, but I'd get a warning. Or not, might be considered as fine enough to go silently if documented and defined as something that fmf does intentionally and not by accident. I would be fine with that too.

[psss]

I might be getting it wrong, but IIUIC, the goal is to enable the list + not a list operation, by adding the not a list as an item to the existing list, correct?

@happz, actually, the proposed implementation is different, see the comment above for details. Basically it aims to cover the use case provided by @bgoncalv, for example adding the same filter field into multiple discover step phases. So there's a bit more magic then just appending a non-list item to a list.

[The-Mule]

I would also appreciate having this implemented, it is very useful for plans that are more or less the same but only differ slightly by the test selection (we have a very clear and specific use case for CI). I just want to double check that I understand it correctly. Assume I have the following plans:

discover:
  - how: fmf
    url: https://github.com/project1
  - how: fmf
    url: https://github.com/project2
    filter: tier:1
  - how: fmf
    url: https://github.com/project3
    tests:
      - first      # this one has tier: 1
      - second     # this one has tier: 2
      - third      # this one has tier: 3

/A:
  discover+:
    - filter: tier:1

/B:
  discover+:
    - filter: tier:2

/C:
  discover+:
     - tests:
        - third

Do I understand correctly that selection in plans A, B, C is applied on all tests found by the main discover? IOW

• A contains tier 1 tests from project1 and project2 and 'first' tests from project3
• B contains tier 2 tests from project1, nothing from project2 and 'second' from project3
• C contains only 'third' test from project3 and nothing else (under assumption that there is no test of that name in project1 and project2)

Is all this compatible with the original request and proposed approach?

[The-Mule]

The reason why this could be pretty useful is that in general you have expect many items in discover phase (mutliple test sources, some of them using filters, some of them using explicit test selection, etc.) and you might easily end up with a discover phase that have ~100 lines of code.

Now you want to run the plan in different configurations, e.g. you need to run the same plan with and without buildroot - obviously you want to add another filter for that but other than that the discover is pretty much the same regardless of buildroot. Add testing in FIPS and from 2 plans you have 4 - again, the only difference between discover phases in all 4 plans is that you want to use a different test selection. With 4 plans we are already talking about significant duplication of discover phases (unless this RFE is implemented).

[LecrisUT]

If you have a list like:

/B:
  discover+:
    - filter: tier:2

Then the current implementation applies and you get:

/B:
  discover:
    - how: fmf
      url: https://github.com/project1
    - how: fmf
      url: https://github.com/project2
      filter: tier:1
    - how: fmf
      url: https://github.com/project3
      tests:
        - first      # this one has tier: 1
        - second     # this one has tier: 2
        - third      # this one has tier: 3
    - filter: tier:2

---

The proposal here kicks in when you make the value a dict (or string)

/B:
  discover+:
    filter: tier:2

Then it would be like the + operator is applied to each individual item there:

/B:
  discover:
    - how: fmf
      url: https://github.com/project1
      filter: tier:2
    - how: fmf
      url: https://github.com/project2
      filter: tier:2
    - how: fmf
      url: https://github.com/project3
      tests:
        - first      # this one has tier: 1
        - second     # this one has tier: 2
        - third      # this one has tier: 3
      filter: tier:2

[The-Mule]

Yes, so it is basically what I want.

I assume that it is already (in the current 'inheritance' implementation) handled that if you have, for instance, a filter already present in the main discover phase and you are adding one more in discover+ they are merged into the list (ie. final result is a conjunction of both) and this applies not just to filter but e.g. to tests too (except that the result is an intersection).

[LecrisUT]

There is no special treatment for filter, tests, etc. All are just yaml fields. There is just some magic on top of it for handling operators, path, inheritance, etc. to make yaml -> fmf.

So you can still add filter+:, tests-: to alter the behavior more granularly. It's the subsequent operator (or lack of) that determines how the fields are treated. Well having the - operator would be a bit more tricky depending on if it should fail if the parent has the item or not, but with the new -~ operator should be able to distinguish between the the 2 cases.

[psss]

I assume that it is already (in the current 'inheritance' implementation) handled that if you have, for instance, a filter already present in the main discover phase and you are adding one more in discover+ they are merged into the list

Just to clarify the filter extension versus overriding the parent value: For the following plan:

discover:
    how: fmf
    filter:
      - tier:1

/extend:
    discover+:
        filter+:
          - tier:2

/override:
    discover+:
        filter:
          - tier:2

You would get this in the data:

> fmf show
/plans/example/extend
discover: {'filter': ['tier:1', 'tier:2'], 'how': 'fmf'}

/plans/example/override
discover: {'filter': ['tier:2'], 'how': 'fmf'}

For the multiple-phase-update case proposed here it should behave similarly. Just note, that the plan is to use a dedicated operator in order to prevent confusion with the normal merging using the +.

Btw, any idea about a suitable character to be used for this? Trying some brainstorm:

• applying addition to multiple items of a list sounds a bit like multiplication, so parent*?
• it is supposed to be an extended version of the + operation, so ++ like applying additiona several times?
• use another modifier after the + character? something like +~, +|, +*

Any better ideas?

[LecrisUT]

Btw, any idea about a suitable character to be used for this? Trying some brainstorm:

Might create more confusion due to lack of support for other operators, but how about field[]+:? Reasoning is that this operator is specific to lists only.

[KwisatzHaderach]

We have a similar need to teemtee/tmt#3198 we need the inverse of this, we have common arguments for discover we want each discover list item to inherit

main.fmf

discover:
    how: fmf
    prune: True
    adjust-tests:
        - duration+: '*3'
          when: hw == slow

plan.fmf

discover+:
    - name: upstream
      url: https://some.url
    - name: downstream
      url: https://other.url

I guess the solution will be same/similar for both versions of this issue, so not creating another one.