You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using the alevin-fry quantitation method. For downstream analysis, I am mainly interested in the final count matrix, e.g.
The content of the af_quant/alevin directory and
The af_quant/quant.json file
Right now, the workflow publishes lots of other files - some of them very large - as well, e.g. the af_map output directory or the af_quant/alevin/map.collated.rad which can be tens of gigabytes in size for large experiments.
It would be great to be able to whittle down the published files to reduce the size of the pipeline's output. (After all, the intermediate files are still available in the working directory.)
For example, I am running nextflow on AWS Batch with an S3 bucket as the publish directory. It takes many times longer to copy the output files to the bucket than to run the actual workflow (because the publishing is not parallelized*.)
*If there is a way to speed this up, I would love to learn!
The text was updated successfully, but these errors were encountered:
Description of feature
I am using the alevin-fry quantitation method. For downstream analysis, I am mainly interested in the final count matrix, e.g.
af_quant/alevin directory
andaf_quant/quant.json
fileRight now, the workflow publishes lots of other files - some of them very large - as well, e.g. the
af_map
output directory or theaf_quant/alevin/map.collated.rad
which can be tens of gigabytes in size for large experiments.It would be great to be able to whittle down the published files to reduce the size of the pipeline's output. (After all, the intermediate files are still available in the working directory.)
For example, I am running nextflow on AWS Batch with an S3 bucket as the publish directory. It takes many times longer to copy the output files to the bucket than to run the actual workflow (because the publishing is not parallelized*.)
*If there is a way to speed this up, I would love to learn!
The text was updated successfully, but these errors were encountered: