Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot declare params in parent directory #10596

Open
OS-leonardopratesi opened this issue Oct 21, 2024 · 1 comment
Open

Cannot declare params in parent directory #10596

OS-leonardopratesi opened this issue Oct 21, 2024 · 1 comment
Labels
A: pipelines Related to the pipelines feature awaiting response we are waiting for your reply, please respond! :) triage Needs to be triaged

Comments

@OS-leonardopratesi
Copy link

OS-leonardopratesi commented Oct 21, 2024

Bug Report

Hello we are trying to work with DVC in a monorepo configuration where in the same repo we will have multiple project each with his own pipeline inside its own dvc.yaml, this means that when we run an experiment in a specific project it will appear also in the other projects.

To overcome this we are want to have a common param for all experiment that can be then used to filter out the correct experiment group in the DVC VSCode UI.

To do this we need to have a "common_params.yaml" file in the parent directory of all the project and we expect this configuration to work inside the dvc.yaml of each pipeline:

params:´
- ../common_params.yaml:
   - experiment_group

Instead when declaring the param file as in the parent directory it is just ignored.

@shcheklein
Copy link
Member

@OS-leonardopratesi have you considered using https://dvc.org/doc/command-reference/init#--subdir ?

that option would allow you to completely isolate DVC subprojects.

Instead when declaring the param file as in the parent directory it is just ignored.

@OS-leonardopratesi could you clarify please? Do you mean this configuration that you mentioned:

params:
- ../common_params.yaml:
   - experiment_group

or something else?

what do you mean by "ignored"? do you have stages that depend on those params outside? It would be great to have some minimal working example - a simple artificial pipelines with params outside and what command you run, etc.

@shcheklein shcheklein added awaiting response we are waiting for your reply, please respond! :) triage Needs to be triaged A: pipelines Related to the pipelines feature labels Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: pipelines Related to the pipelines feature awaiting response we are waiting for your reply, please respond! :) triage Needs to be triaged
Projects
None yet
Development

No branches or pull requests

2 participants