Redirect CommandLineTool output to default task log file

[francescobianco]

I want collect all generated output by every task into standard Airflow task log file, generally is ./logs/{dag_id}/{task_id}/{run_stamp}/1.log

Furthermore I would get all logs for every task into single file, I want collapse all log into file 1.log and not into 1.log.cwl

[michael-kotliar]

Hi @francescobianco
Sure, it's easy to do. I intentionally patch airflow.cfg to split the logs into two files to keep airflow and cwl logs separately. You can see how I do it here

cwl-airflow/cwl_airflow/components/init/config.py

Line 105 in 9c2c852

["sed", "-i", "-e", "s#^logging_config_class.*#logging_config_class = cwl_airflow.config_templates.airflow_local_settings.DEFAULT_LOGGING_CONFIG#g", args.config],

If you want to to keep both of the logs in the same file, set the default value for logging_config_class in airflow.cfg.

Let me know if you have any other questions.