Saving of outputs on VM preemption?

[rmashl]

The pipelines documentation states that files in the outputs directory will be copied to storage when the task is finished. Is it possible to have pipelines do this as well upon preemption of the VM?

[mdmiller53]

Graceful Shutdown: Compute Engine gives you 30 seconds to shut down when you're preempted, letting you save your work in progress for later. At the main doc page it states: '1.Compute Engine sends a preemption notice to the instance in the form of an ACPI G2 Soft Off open_in_new signal. You can use a shutdown script to handle the preemption notice and complete cleanup actions before the instance stops.' so it would depend on if there were anything to save that makes sense and if it could be saved in 30 seconds, if the shutdown script could coordinate or be part of the pipeline. so some big 'ifs' that would need to be worked through. it may be more straight forward to just relaunch the job