Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can we avoid creating empty folders when initializing a repo? #1315

Open
AlphaJack opened this issue Oct 10, 2024 · 4 comments · May be fixed by rustic-rs/rustic_core#334
Open

Can we avoid creating empty folders when initializing a repo? #1315

AlphaJack opened this issue Oct 10, 2024 · 4 comments · May be fixed by rustic-rs/rustic_core#334
Assignees
Labels
S-triage Status: Waiting for a maintainer to triage this issue/PR

Comments

@AlphaJack
Copy link

Is it needed to create all 256 data/ folders when initializing a repo? Or could they also be created when needed?

Borg:

└── 0
    ├── 0
    ├── 1
    ├── 10
    ├── 11
    ├── 12
    ├── 13
    ├── 14
    ├── 15
    ├── 16
    ├── 17
    ├── 18
    ├── 19
    ├── 2
    ├── 20
    ├── 21
    ├── 3
    ├── 4
    ├── 5
    ├── 6
    ├── 7
    ├── 8
    └── 9

2 directories, 22 files

Rustic:

[...]

├── e8
├── e9
├── ea
├── eb
├── ec
│   └── eca42cd695ae8243fa0ab3d3e649bcdbcf603cc0572992c7df3615874ac91620
├── ed
├── ee
├── ef
├── f0
├── f1
├── f2
├── f3
├── f4
├── f5
├── f6
├── f7
├── f8
├── f9
├── fa
├── fb
├── fc
├── fd
├── fe
└── ff

257 directories, 5 files
@github-actions github-actions bot added the S-triage Status: Waiting for a maintainer to triage this issue/PR label Oct 10, 2024
@aawsome
Copy link
Member

aawsome commented Oct 10, 2024

Thanks for opening the issue @AlphaJack.

Can you please be more specific what is your exact problem with having this 256 empty paths?

In fact, yes we could omit creating these folders and just create them on-demand. However, that would loose compatibility with restic which (IIRC) is not working when the paths are not present - as rustic is.

So, we can adapt rustic to be more resilient when those paths are note present (by creating them when they are needed), but if we initialize a new repo without creating these paths we get compatibility problems...

@AlphaJack
Copy link
Author

Thank you for the clarification,
this is not a big issue on my side, more of a question.

I was asking because uploading ~255 empty folders with RClone seemed slow.
If I would tell RClone to ignore uploading empty folders from a repo, once I download my remote backup, I would end up having less folders than the original repo (because I would miss all non-empty folders).

But if Restic cannot handle missing folders, then I guess it's better to leave things as they are, to maintain compatibility.
As long as Rustic works correctly without empty folders, I can set the "don't upload empty folders" option from RClone.

@aawsome
Copy link
Member

aawsome commented Oct 13, 2024

As long as Rustic works correctly without empty folders

Actually I think it won't. But you can open issues for the errors you get when removing empty dirs - I'll fix them in the next release(s)...

@AlphaJack
Copy link
Author

AlphaJack commented Oct 13, 2024

I'm getting this error with most operations (backup, prune, forget), but no further information about which file or directory is involved:

error: opening file failed: `Os { code: 2, kind: NotFound, message: "No such file or directory" }`

or

[ERROR] error backing up /Source/Folder: opening file failed: `Os { code: 2, kind: NotFound, message: "No such file or directory" }`
error: Not all snapshots were generated successfully!

Can we also show the missing file or folder name in the error message?

EDIT: I don't think it's related to missing empty folders, but having the folder in the error message would be really nice!

@simonsan simonsan self-assigned this Oct 14, 2024
simonsan added a commit to rustic-rs/rustic_core that referenced this issue Oct 14, 2024
- Update error messages for file operations in the `LocalBackendErrorKind` enum.
- Refactor the `ReadBackend` and `WriteBackend` implementations in the `LocalBackend` module to handle file opening errors more accurately.
- Add error variants `OpeningFileForPartialReadingFailed` and `OpeningFileForWritingFailed` to provide specific information about file opening failures.
- Create parent directory if it does not exist before opening the file for writing.

Fixes #rustic-rs/rustic#1315

Signed-off-by: simonsan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-triage Status: Waiting for a maintainer to triage this issue/PR
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants