Release 0.7 #116

muffato · 2024-10-02T12:19:46Z

The sum of #113, #114, and #117

PR checklist

Use small 1.2G for base nextflow job

Use the org level variable TOWER_COMPUTE_ENV which point to small actually with 1.2G memory.

Use small tower compute env

… the beginning

This will save a lot of space

Support for grid view (and a few other things)

github-actions · 2024-10-02T12:21:27Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 8d2b83f

+| ✅ 128 tests passed       |+
#| ❔  29 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: 0.7.0

❔ Tests ignored:

files_exist - File is ignored: CODE_OF_CONDUCT.md
files_exist - File is ignored: assets/nf-core-blobtoolkit_logo_light.png
files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_light.png
files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_dark.png
files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: conf/igenomes.config
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting_comment.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/email_template.txt
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File ignored due to lint config: assets/nf-core-blobtoolkit_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/blobtoolkit/blobtoolkit/.github/workflows/awstest.yml
template_strings - template_strings
merge_markers - merge_markers

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: lib/WorkflowBlobtoolkit.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-blobtoolkit_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: sanger_test_full.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: sanger_test.yml
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' contains report_section_order
multiqc_config - 'assets/multiqc_config.yml' contains export_plots
multiqc_config - 'assets/multiqc_config.yml' contains report_comment
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.11
Run at 2024-11-04 09:38:03

Indeed, 85.9% of jobs complete within 4 hours, 98% within 47 hours, and 99.7% within 167 hours

muffato · 2024-10-16T09:01:04Z

There will need to be another change in this release. Damon reported that the instructions for setting up the UniProt diamond database don't work.

Misc fixes before release

tkchafin · 2024-10-18T12:30:59Z

I don't think it is immediately necessary, but we might want to manually insert something into the small test dataset that won't be caught in blastx, so the blastn subworkflow will be triggered in the tests?

muffato · 2024-10-18T12:40:09Z

I don't think it is immediately necessary, but we might want to manually insert something into the small test dataset that won't be caught in blastx, so the blastn subworkflow will be triggered in the tests?

That would be very good, yes.
I plan on finishing testing the database setup instructions today. We can discuss next week whether the updated test goes into v0.7 or a next release

Database installation instructions

Trigger blastn subworkflow in test profile

gq1 and others added 27 commits September 13, 2024 15:32

Update sanger_test_full.yml

2584645

Use small 1.2G for base nextflow job

Update sanger_test_full.yml

ea1b05a

Use the org level variable TOWER_COMPUTE_ENV which point to small actually with 1.2G memory.

Merge pull request #113 from sanger-tol/small_tower_compute_env

e0bf8ba

Use small tower compute env

Fixed the comment

d13d1ed

UPDATEBLOBDIR needs to pass the tsv files through certain arguments

2c793aa

Populate as much as possible of the metadata at the beginning

b55f76c

Also generate TSV files

0daf1e2

Example plugging the TSV files into blobtools commands

2cf12fd

Skip BLOBTOOLKIT_METADATA as we generate the yaml file correctly from…

b8b1fa4

… the beginning

Introduced a new samplesheet parameter to track single/paired readsets

41b101f

Populate as much as possible of the metadata at the beginning

80a100b

Version bumps

bea3a77

Updated the documentation

4edafb4

Version bump

904a512

Publish the windowmasker outputs too

51985ad

Convert input BAM/CRAM files to Fasta on the fly

f47dd16

This will save a lot of space

Bumped the version down

43254bc

Pin the version of editorconfig like in the readmapping pipeline

c89b1b9

Exclude sqlite databases from ECLint

152282b

add exclusions for nf-core lint

476b736

Update CITATIONS.md

303b221

prettier linting

27b0a51

The release is going to happen today

4eef0fa

Decreased the runtime requirements for illumina alignments

0d8a5a9

Wording of the changelog

1909646

Merge pull request #114 from sanger-tol/chrom_view

102dbf4

Support for grid view (and a few other things)

Documentation update

8a05dc2

muffato self-assigned this Oct 2, 2024

muffato requested a review from tkchafin October 14, 2024 16:20

muffato added 2 commits October 14, 2024 17:28

Bumped up the underlying BTK version because 4.3.9 had a bug

abe2b76

Updated the blastn runtime requirements to avoid basement jobs

dfb4655

Indeed, 85.9% of jobs complete within 4 hours, 98% within 47 hours, and 99.7% within 167 hours

Merge pull request #117 from sanger-tol/misc_fixes

93bda84

Misc fixes before release

muffato and others added 4 commits October 19, 2024 09:50

Updated database installation instructions

ea6d70a

fix URL not found in uniprot ref ftp path

36fff07

fix ftp path for taxdump

61c0328

Merge pull request #118 from sanger-tol/db_install

b9bcef6

Database installation instructions

tkchafin approved these changes Oct 21, 2024

View reviewed changes

tkchafin added 2 commits October 24, 2024 17:51

Update test.config

2b22f24

Merge pull request #120 from sanger-tol/blastn_test

8d2b83f

Trigger blastn subworkflow in test profile

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 0.7 #116

Release 0.7 #116

muffato commented Oct 2, 2024 •

edited

Loading

github-actions bot commented Oct 2, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

muffato commented Oct 16, 2024

tkchafin commented Oct 18, 2024

muffato commented Oct 18, 2024

Release 0.7 #116

Are you sure you want to change the base?

Release 0.7 #116

Conversation

muffato commented Oct 2, 2024 • edited Loading

PR checklist

github-actions bot commented Oct 2, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

muffato commented Oct 16, 2024

tkchafin commented Oct 18, 2024

muffato commented Oct 18, 2024

muffato commented Oct 2, 2024 •

edited

Loading

github-actions bot commented Oct 2, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️