Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

!from_yaml seems to break YAML merge key on mappings #198

Open
fortenbt opened this issue Jan 9, 2023 · 2 comments
Open

!from_yaml seems to break YAML merge key on mappings #198

fortenbt opened this issue Jan 9, 2023 · 2 comments

Comments

@fortenbt
Copy link

fortenbt commented Jan 9, 2023

Mappings defined in other YAML files cannot be merged using the YAML merge key. When attempting this, the following error occurs.

$ scuba foo
scuba: Config error: Error loading .scuba.yml: while constructing a mapping
  in ".scuba.yml", line 8, column 13
expected a mapping for merging, but found scalar
  in ".scuba.yml", line 3, column 22

Background

YAML mappings can be merged with with merge key << (https://yaml.org/type/merge.html)

Example
The example in this issue uses the following files: foo.yml, bar.yml, and .scuba.yml

# foo.yml
aliases:
    foo:
        global-vars: &global-vars
            VERSION: "1.0"
        sub-vars: &sub-vars
            SUB_VERSION: "2.0"
        environment:
            <<: [ *global-vars, *sub-vars ]
# bar.yml
variables:
    VERSION: "1.0"

subvars:
    SUB_VERSION: "2.0"
# .scuba.yml (just foo.yml, except replacing the *-vars values with !from_yaml)
aliases:
    foo:
        global-vars: &global-vars
            !from_yaml bar.yml variables
        sub-vars: &sub-vars
            !from_yaml bar.yml subvars
        environment:
            <<: [ *global-vars, *sub-vars ]

PyYAML Output

>>> import yaml
>>> yaml.safe_load(open('foo.yml', 'r').read())
{'aliases': {'foo': {'global-vars': {'VERSION': '1.0'}, 'sub-vars': {'SUB_VERSION': '2.0'}, 'environment': {'SUB_VERSION': '2.0', 'VERSION': '1.0'}}}}

Expected Behavior
The environment key contains the merged variables and subvars mappings from bar.yml:

'environment': {'SUB_VERSION': '2.0', 'VERSION': '1.0'}

General Notes

Didn't look into this too hard, but config.py defines from_yaml and uses construct_scalar, which is probably related.

@fortenbt
Copy link
Author

fortenbt commented Jan 9, 2023

Looked into it a bit more. It appears the node type handed to your from_yaml constructor is already a ScalarNode, when I think it should be a MappingNode. Without understanding how PyYAML really intended on doing this, it probably involves using the Resolver, which is how they tag the << merge key node with 'tag:yaml.org,2002:merge' to begin with.

@haboustak
Copy link
Contributor

A similar issue was reported in pyyaml upstream: yaml/pyyaml#120

In reviewing the code, it seems like a merge key mapping does not construct custom tags. If it did call the !from_yaml constructor, then it would have received a mapping value (at least in your example, but not in every example).

I think this would have to be fixed in pyyaml by adding support and examples to the above issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants