Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine: handling temp files when peer transfers disabled #3964

Conversation

colinthomas-z80
Copy link
Contributor

@colinthomas-z80 colinthomas-z80 commented Oct 30, 2024

Proposed Changes

If peer transfers are disabled change temp file behavior so they are returned on creation like normal files. Taskvine will still manage naming and cleanup after the workflow completes.

Merge Checklist

The following items must be completed before PRs can be merge.
Check these off to verify you have completed all steps.

  • make test Run local tests prior to pushing.
  • make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
  • make lint Run lint on source code prior to pushing.
  • Manual Update Update the manual to reflect user-visible changes.
  • Type Labels Select a github label for the type: bugfix, enhancement, etc.
  • Product Labels Select a github label for the product: TaskVine, Makeflow, etc.
  • PR RTM Mark your PR as ready to merge.

taskvine/src/manager/vine_file.c Outdated Show resolved Hide resolved
@colinthomas-z80
Copy link
Contributor Author

I am thinking about how to add a mechanism to clean up these local temp files when the workflow completes, to make them similar in function to real temp files

@dthain
Copy link
Member

dthain commented Nov 5, 2024

This would seem to give all temp files from the same user the same name.

char *name = string_format("temp-local-%u", uid);

Am I missing something?

@colinthomas-z80
Copy link
Contributor Author

You are right. I was mistaken about uid vs. uuid behavior

@dthain dthain merged commit 43ba8b3 into cooperative-computing-lab:master Nov 5, 2024
10 checks passed
btovar pushed a commit that referenced this pull request Nov 6, 2024
* add temp warning when disabling peer transfers

* without peer transfers, create normal file instead of temps

* unique name for local temps

* unlink local temp when done

* correct uuid generation for temp names
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants