vine: add tag has_free_resources to worker

[JinZhou5042]

Proposed Changes

An improved version suggested by @btovar in #4037

The issue is that even though most allocations work fine in #4035 and tasks run pretty densely, the manager still wastes a lot of time checking tasks when there aren’t any usable cores.

Task scheduling seems to be an expensive part of the main loop, we could save a lot of time by making this more efficient, and then the manager has more time to do other jobs like output retrieving and temp file replication.

The most common issue in check_worker_against_task is that the worker doesn’t have a free core. But it still goes through the whole resource allocation process, and this keeps happening for a bunch of tasks each time, based on the depth set by q->attempt_schedule_depth.

This PR adds a tag for workers to note whether they have any available resources. This allows the checking process to terminate earlier.

Merge Checklist

The following items must be completed before PRs can be merged.
Check these off to verify you have completed all steps.

• make test Run local tests prior to pushing.
• make format Format source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)
• make lint Run lint on source code prior to pushing.
• Manual Update: Update the manual to reflect user-visible changes.
• Type Labels: Select a github label for the type: bugfix, enhancement, etc.
• Product Labels: Select a github label for the product: TaskVine, Makeflow, etc.
• PR RTM: Mark your PR as ready to merge.

[btovar]

Looks good. Are you still testing it?

[JinZhou5042]

Yes, I am going to do some large tests to make sure they work well.