add ability to specify the select directories/files to be pulled/outputted.

[djschny]

Describe the problem/challenge you have
I'm trying to extract/pull only a select directory out of a image/package.

Describe the solution you'd like
Ability to specify the select directories/files to be pulled/outputted.

Anything else you would like to add:

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help work on this issue.

[joaopapereira]

Hey, thanks for creating this issue, I am sorry for the delay in replying to it.

This sounds like an interesting and very scoped issue. I do have some questions about UX for the command:

1. What should the name of this flag be? Something like --file? Where you can specify the file/folder that you want to download? Or should we have something similar to --file-exclusion that would do the opposite?
2. What should happen if the file/folder does not exist in the image? Should we error out?
3. When using push, we only allow you to exclude/include folders using the folder/file name(no wildcards) is the same behavior ok for pull?

[djschny]

Thanks for considering this and the questions. To the specific questions:

1. What should the name of this flag be? Something like --file? Where you can specify the file/folder that you want to download? Or should we have something similar to --file-exclusion that would do the opposite?

I think whatever you think makes sense in regards to others. --include sounds nice. Given that most of the time people would be specifying a directory avoid file or replacing with path might be appropriate?

2. What should happen if the file/folder does not exist in the image? Should we error out?

This could go either way. If you have a pattern following for similar scenarios I think consistency is important. Otherwise I would probably default to error out and a person could simply append || true to the end of the command if they don't like that behavior.

3. When using push, we only allow you to exclude/include folders using the folder/file name(no wildcards) is the same behavior ok for pull?

I believe so. IMO good design of the tree structure should place all commonly selected items under one directory/area.

[joaopapereira]

I created some acceptance criteria for this story. Please take a look and see if it makes sense.

Scenario 1: 🟢 User specifies does not specify any folder to include

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder -f some-other-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image
And I run tree -L 1 /tmp/some-image
Then I should see

/tmp/some-image
├── some-folder
└── some-other-folder

3 directories, 0 files

Scenario 2: 🟢 User specifies folder they want to download

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder -f some-other-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only some-other-folder
And I run tree -L 1 /tmp/some-image
Then I should see

/tmp/some-image
└── some-other-folder

2 directories, 0 files

Scenario 3: 🟢 User specifies multiple folders to download

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder -f some-other-folder -f yet-another-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only some-other-folder --include-only yet-another-folder
And I run tree -L 1 /tmp/some-image
Then I should see

/tmp/some-image
└── some-other-folder
└── yet-other-folder

3 directories, 0 files

Scenario 4: 🟢 User specifies a subfolder to download

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only some-folder/sub-folder
And I run tree -L 2 /tmp/some-image
Then I should see

/tmp/some-image
└── some-folder
    └── sub-folder

3 directories, 0 files

Scenario 5: 🔴 User specifies a folder not present in the image

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only not-found-folder
Then imgpkg should fail
And I should see the error message

imgpkg: Error: 'not-found-folder' could not be found in the OCI image

Scenario 6: 🔴 User specifies a folder that is present and one that is not present in the image

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only some-folder --include-only not-found-folder
Then imgpkg should fail
And I should see the error message

imgpkg: Error: 'not-found-folder' could not be found in the OCI image

Scenario 7: 🔴 User specifies two folders that do not exist in the image

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only not-found-folder-2 --include-only not-found-folder
Then imgpkg should fail
And I should see the error message

imgpkg: Error: 'not-found-folder-2' could not be found in the OCI image

Scenario 8: 🔴 User provides a wildcard in the --include-only flag

Given I push an OCI image using imgpkg push -i my.registry.io/some-image:some-tag -f some-folder
When I run imgpkg pull -i my.registry.io/some-image:some-tag -o /tmp/my-image --include-only some-*
Then imgpkg should fail
And I should see the error message

imgpkg: Error: Cannot process folder 'some-*' (Wildcards are not supported in this functionality).

Scenario 9: 🔴 User provides the flag --include-only and --recursive

Given I push an OCI image using imgpkg push -b my.registry.io/some-image:some-tag -f some-folder
When I run imgpkg pull -b my.registry.io/some-image:some-tag -o /tmp/my-image --include-only some-folder --recursive
Then imgpkg should fail
And I should see the error message

imgpkg: Error: The flag '--include-only' and '--recursive' cannot be used together.

Notes: Scenarios 1-8 should work the same way when pushing bundles

[djschny]

LGTM

[joaopapereira]

I changed the label to Carvel Accepted, and this issue is ready to be picked up. The maintainers are somewhat backlogged, so I am not 100% sure when we will be able to work on this.
@djschny, is this issue something that you would be interested in contributing?

I am happy to give any help if someone is interested in giving this a go.

[djschny]

@djschny, is this issue something that you would be interested in contributing?

Unfortunately at current juncture I'm tied up with lots of other items so while I could give it a go, it would be a couple weeks and waiting to know direction of things.