-
-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy pathWorkflow.yml
918 lines (786 loc) · 32.7 KB
/
Workflow.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
saladVersion: v1.3
$base: "https://w3id.org/cwl/cwl#"
$namespaces:
cwl: "https://w3id.org/cwl/cwl#"
rdfs: "http://www.w3.org/2000/01/rdf-schema#"
$graph:
- name: "WorkflowDoc"
type: documentation
doc:
- |
# Common Workflow Language (CWL) Workflow Description, v1.3.0-dev1
This version:
* https://w3id.org/cwl/v1.3.0-dev1
Latest stable version:
* https://w3id.org/cwl/
- "\n\n"
- {$include: contrib.md}
- "\n\n"
- |
# Abstract
This specification defines the Common Workflow Language (CWL)
Workflow description, a vendor-neutral standard for representing
analysis tasks where a sequence of operations are described
using a directed graph of operations to transform input to
output. CWL is portable across a variety of computing
platforms.
- {$include: intro.md}
- |
## Introduction to CWL Workflow standard v1.3.0-dev1
This specification represents the latest development version from the
CWL group.
## Changelog
* Added a new Loop feature to support iterative workflows. From a technical perspective, this led to the
introduction of a `LoopWorkflowStep`[#LoopWorkflowStep] element to represent iterative subworkflows, a
`LoopInput`[#LoopInput] object to handle termination conditions, and a `LoopOutputMethod`[#LoopOutputMethod]
feature to manage loop output data history.
See also the [CWL Command Line Tool Description, v1.3.0-dev1 changelog](CommandLineTool.html#Changelog).
For other changes since CWL v1.0, see the
[CWL Workflow Description, v1.1 changelog](https://www.commonwl.org/v1.1/Workflow.html#Changelog)
and
[CWL Workflow Description, v1.2 changelog](https://www.commonwl.org/v1.2/Workflow.html#Changelog)
## Purpose
The Common Workflow Language Command Line Tool Description
express workflows for data-intensive science, such as
bioinformatics, physics, astronomy, geoscience, and machine
learning. This specification is intended to define a data and
execution model for Workflows that can be implemented on top of
a variety of computing platforms, ranging from an individual
workstation to cluster, grid, cloud, and high performance
computing systems. Details related to execution of these
workflow not laid out in this specification are open to
interpretation by the computing platform implementing this
specification.
- {$include: concepts.md}
- name: ExpressionToolOutputParameter
type: record
extends: OutputParameter
fields:
- name: type
type:
- CWLType
- OutputRecordSchema
- OutputEnumSchema
- OutputArraySchema
- string
- type: array
items:
- CWLType
- OutputRecordSchema
- OutputEnumSchema
- OutputArraySchema
- string
jsonldPredicate:
"_id": "sld:type"
"_type": "@vocab"
refScope: 2
typeDSL: True
doc: |
Specify valid types of data that may be assigned to this parameter.
Note that this field just acts as a hint, as the outputs of an
ExpressionTool process are always considered valid.
- name: WorkflowInputParameter
type: record
extends: InputParameter
docParent: "#Workflow"
fields:
- name: type
type:
- CWLType
- InputRecordSchema
- InputEnumSchema
- InputArraySchema
- string
- type: array
items:
- CWLType
- InputRecordSchema
- InputEnumSchema
- InputArraySchema
- string
jsonldPredicate:
"_id": "sld:type"
"_type": "@vocab"
refScope: 2
typeDSL: True
doc: |
Specify valid types of data that may be assigned to this parameter.
- name: inputBinding
type: InputBinding?
doc: |
Deprecated. Preserved for v1.0 backwards compatibility. Will be removed in
CWL v2.0. Use `WorkflowInputParameter.loadContents` instead.
jsonldPredicate: "cwl:inputBinding"
- type: record
name: ExpressionTool
extends: Process
specialize:
- specializeFrom: InputParameter
specializeTo: WorkflowInputParameter
- specializeFrom: OutputParameter
specializeTo: ExpressionToolOutputParameter
documentRoot: true
doc: |
An ExpressionTool is a type of Process object that can be run by itself
or as a Workflow step. It executes a pure Javascript expression that has
access to the same input parameters as a workflow. It is meant to be used
sparingly as a way to isolate complex Javascript expressions that need to
operate on input data and produce some result; perhaps just a
rearrangement of the inputs. No Docker software container is required
or allowed.
fields:
- name: class
jsonldPredicate:
"_id": "@type"
"_type": "@vocab"
type:
type: enum
name: ExpressionTool_class
symbols:
- cwl:ExpressionTool
- name: expression
type: Expression
doc: |
The expression to execute. The expression must return a plain
Javascript object which matches the output parameters of the
ExpressionTool.
- name: LinkMergeMethod
type: enum
docParent: "#WorkflowStepInput"
doc: The input link merge method, described in [WorkflowStepInput](#WorkflowStepInput).
symbols:
- merge_nested
- merge_flattened
- name: PickValueMethod
type: enum
docParent: "#WorkflowStepInput"
doc: |
Picking non-null values among inbound data links, described in [WorkflowStepInput](#WorkflowStepInput).
symbols:
- first_non_null
- the_only_non_null
- all_non_null
- name: Sink
type: record
abstract: true
fields:
- name: linkMerge
type: LinkMergeMethod?
jsonldPredicate: "cwl:linkMerge"
default: merge_nested
doc: |
The method to use to merge multiple inbound links into a single array.
If not specified, the default method is "merge_nested".
- name: pickValue
type: [ "null", PickValueMethod ]
jsonldPredicate: "cwl:pickValue"
doc: |
The method to use to choose non-null elements among multiple sources.
- name: InputSink
type: record
extends: Sink
abstract: true
fields:
- name: source
doc: |
Specifies one or more workflow parameters that will provide input to
the underlying step parameter.
jsonldPredicate:
"_id": "cwl:source"
"_type": "@id"
refScope: 2
type:
- string?
- string[]?
- name: OutputSink
type: record
extends: Sink
abstract: true
fields:
- name: outputSource
doc: |
Specifies one or more names of an output from a workflow step (in the form
`step_name/output_name` with a `/` separator`), or a workflow input name,
that supply their value(s) to the output parameter.
the output parameter. It is valid to reference workflow level inputs
here.
jsonldPredicate:
"_id": "cwl:outputSource"
"_type": "@id"
refScope: 1
type:
- string?
- string[]?
- name: WorkflowOutputParameter
type: record
extends: [OutputParameter, OutputSink]
docParent: "#Workflow"
doc: |
Describe an output parameter of a workflow. The parameter must be
connected to one or more parameters defined in the workflow that
will provide the value of the output parameter. It is legal to
connect a WorkflowInputParameter to a WorkflowOutputParameter.
See [WorkflowStepInput](#WorkflowStepInput) for discussion of
`linkMerge` and `pickValue`.
fields:
- name: type
type:
- CWLType
- OutputRecordSchema
- OutputEnumSchema
- OutputArraySchema
- string
- type: array
items:
- CWLType
- OutputRecordSchema
- OutputEnumSchema
- OutputArraySchema
- string
jsonldPredicate:
"_id": "sld:type"
"_type": "@vocab"
refScope: 2
typeDSL: True
doc: |
Specify valid types of data that may be assigned to this parameter.
- type: record
name: WorkflowStepInput
extends: [Identified, InputSink, LoadContents, Labeled]
docParent: "#AbstractWorkflowStep"
doc: |
The input of a workflow step connects an upstream parameter (from the
workflow inputs, or the outputs of other workflows steps) with the input
parameters of the process specified by the `run` field. Only input parameters
declared by the target process will be passed through at runtime to the process
though additional parameters may be specified (for use within `valueFrom`
expressions for instance) - unconnected or unused parameters do not represent an
error condition.
# Input object
A WorkflowStepInput object must contain an `id` field in the form
`#fieldname` or `#prefix/fieldname`. When the `id` field contains a slash
`/` the field name consists of the characters following the final slash
(the prefix portion may contain one or more slashes to indicate scope).
This defines a field of the workflow step input object with the value of
the `source` parameter(s).
# Merging multiple inbound data links
To merge multiple inbound data links,
[MultipleInputFeatureRequirement](#MultipleInputFeatureRequirement) must be specified
in the workflow or workflow step requirements.
If the sink parameter is an array, or named in a [workflow
scatter](#ScatterWorkflowStep) operation, there may be multiple inbound
data links listed in the `source` field. The values from the
input links are merged depending on the method specified in the
`linkMerge` field. If both `linkMerge` and `pickValue` are null
or not specified, and there is more than one element in the
`source` array, the default method is "merge_nested".
If both `linkMerge` and `pickValue` are null or not specified, and
there is only a single element in the `source`, then the input
parameter takes the scalar value from the single input link (it is
*not* wrapped in a single-list).
* **merge_nested**
The input must be an array consisting of exactly one entry for each
input link. If "merge_nested" is specified with a single link, the value
from the link must be wrapped in a single-item list.
* **merge_flattened**
1. The source and sink parameters must be compatible types, or the source
type must be compatible with single element from the "items" type of
the destination array parameter.
2. Source parameters which are arrays are concatenated.
Source parameters which are single element types are appended as
single elements.
# Picking non-null values among inbound data links
If present, `pickValue` specifies how to pick non-null values among inbound data links.
`pickValue` is evaluated
1. Once all source values from upstream step or parameters are available.
2. After `linkMerge`.
3. Before `scatter` or `valueFrom`.
This is specifically intended to be useful in combination with
[conditional execution](#AbstractWorkflowStep), where several upstream
steps may be connected to a single input (`source` is a list), and
skipped steps produce null values.
Static type checkers should check for type consistency after inferring what the type
will be after `pickValue` is applied, just as they do currently for `linkMerge`.
* **first_non_null**
For the first level of a list input, pick the first non-null element. The result is a scalar.
It is an error if there is no non-null element. Examples:
* `[null, x, null, y] -> x`
* `[null, [null], null, y] -> [null]`
* `[null, null, null] -> Runtime Error`
*Intended use case*: If-else pattern where the
value comes either from a conditional step or from a default or
fallback value. The conditional step(s) should be placed first in
the list.
* **the_only_non_null**
For the first level of a list input, pick the single non-null element. The result is a scalar.
It is an error if there is more than one non-null element. Examples:
* `[null, x, null] -> x`
* `[null, x, null, y] -> Runtime Error`
* `[null, [null], null] -> [null]`
* `[null, null, null] -> Runtime Error`
*Intended use case*: Switch type patterns where developer considers
more than one active code path as a workflow error
(possibly indicating an error in writing `when` condition expressions).
* **all_non_null**
For the first level of a list input, pick all non-null values.
The result is a list, which may be empty. Examples:
* `[null, x, null] -> [x]`
* `[x, null, y] -> [x, y]`
* `[null, [x], [null]] -> [[x], [null]]`
* `[null, null, null] -> []`
*Intended use case*: It is valid to have more than one source, but
sources are conditional, so null sources (from skipped steps)
should be filtered out.
fields:
- name: default
type: CWLObjectType?
doc: |
The default value for this parameter to use if either there is no
`source` field, or the value produced by the `source` is `null`. The
default must be applied prior to scattering or evaluating `valueFrom`.
jsonldPredicate:
_id: "sld:default"
_container: "@list"
noLinkCheck: true
- name: valueFrom
type:
- "null"
- string
- Expression
jsonldPredicate: "cwl:valueFrom"
doc: |
To use valueFrom, [StepInputExpressionRequirement](#StepInputExpressionRequirement) must
be specified in the workflow or workflow step requirements.
If `valueFrom` is a constant string value, use this as the value for
this input parameter.
If `valueFrom` is a parameter reference or expression, it must be
evaluated to yield the actual value to be assigned to the input field.
The `self` value in the parameter reference or expression must be
1. `null` if there is no `source` field
2. the value of the parameter(s) specified in the `source` field when this
workflow input parameter **is not** specified in this workflow step's `scatter` field.
3. an element of the parameter specified in the `source` field when this workflow input
parameter **is** specified in this workflow step's `scatter` field.
The value of `inputs` in the parameter reference or expression must be
the input object to the workflow step after assigning the `source`
values, applying `default`, and then scattering. The order of
evaluating `valueFrom` among step input parameters is undefined and the
result of evaluating `valueFrom` on a parameter must not be visible to
evaluation of `valueFrom` on other parameters.
- type: record
name: WorkflowStepOutput
docParent: "#AbstractWorkflowStep"
extends: Identified
doc: |
Associate an output parameter of the underlying process with a workflow
parameter. The workflow parameter (given in the `id` field) be may be used
as a `source` to connect with input parameters of other workflow steps, or
with an output parameter of the process.
A unique identifier for this workflow output parameter. This is
the identifier to use in the `source` field of `WorkflowStepInput`
to connect the output value to downstream parameters.
- name: LoopInput
type: record
extends: [Identified, OutputSink]
fields:
- name: default
type: ["null", File, Directory, Any]
doc: |
The default value for this parameter to use if either there is no
`outputSource` field, or the value produced by the `source` is `null`. The
default must be applied prior to scattering or evaluating `valueFrom`.
jsonldPredicate:
_id: "sld:default"
_container: "@list"
noLinkCheck: true
- name: valueFrom
type:
- "null"
- string
- Expression
jsonldPredicate: "cwl:valueFrom"
doc: |
To use valueFrom, [StepInputExpressionRequirement](#StepInputExpressionRequirement) must
be specified in the workflow or workflow step requirements.
If `valueFrom` is a constant string value, use this as the value for
this input parameter.
If `valueFrom` is a parameter reference or expression, it must be
evaluated to yield the actual value to be assigned to the input field.
The `self` value in the parameter reference or expression must be
`null` if there is no `source` field, or the value of the
parameter(s) specified in the `source` field.
The value of `inputs` in the parameter reference or expression must be
the input object to the previous iteration of the workflow step (or the initial
inputs for the first iteration).
- name: ScatterMethod
type: enum
docParent: "#ScatterWorkflowStep"
doc: The scatter method, as described in [workflow step scatter](#ScatterWorkflowStep).
symbols:
- dotproduct
- nested_crossproduct
- flat_crossproduct
- name: LoopOutputMethod
type: enum
docParent: "#LoopWorkflowStep"
doc: The loop output method, as described in [workflow step loop](#LoopWorkflowStep).
symbols:
- last_iteration
- all_iterations
- name: AbstractWorkflowStep
type: record
extends: [Identified, Labeled, sld:Documented]
abstract: true
docParent: "#Workflow"
doc: |
A workflow step is an executable element of a workflow. It specifies the
underlying process implementation (such as `CommandLineTool` or another
`Workflow`) in the `run` field and connects the input and output parameters
of the underlying process to workflow parameters.
# Conditional execution (Optional)
Conditional execution makes execution of a step conditional on an
expression. A step that is not executed is "skipped". A skipped
step produces `null` for all output parameters.
The `when` field controls conditional execution. This is an
expression that must be evaluated with `inputs` bound to the step
input object (or individual scatter job), and returns a boolean
value. It is an error if this expression returns a value other
than `true` or `false`.
Conditional execution in CWL is an optional feature and is not required
to be implemented by all consumers of CWL documents. An implementation that
does not support conditional executions must return a fatal error when
attempting to execute a workflow that uses conditional constructs the
implementation does not support.
# Subworkflows
To specify a nested workflow as part of a workflow step,
[SubworkflowFeatureRequirement](#SubworkflowFeatureRequirement) must be
specified in the workflow or workflow step requirements.
It is a fatal error if a workflow directly or indirectly invokes itself as
a subworkflow (recursive workflows are not allowed).
fields:
- name: in
type: WorkflowStepInput[]
jsonldPredicate:
_id: "cwl:in"
mapSubject: id
mapPredicate: source
doc: |
Defines the input parameters of the workflow step. The process is ready to
run when all required input parameters are associated with concrete
values. Input parameters include a schema for each parameter which is
used to validate the input object. It may also be used build a user
interface for constructing the input object.
- name: out
type:
- type: array
items: [string, WorkflowStepOutput]
jsonldPredicate:
_id: "cwl:out"
_type: "@id"
identity: true
doc: |
Defines the parameters representing the output of the process. May be
used to generate and/or validate the output object.
- name: requirements
type: ProcessRequirement[]?
jsonldPredicate:
_id: "cwl:requirements"
mapSubject: class
doc: |
Declares requirements that apply to either the runtime environment or the
workflow engine that must be met in order to execute this workflow step. If
an implementation cannot satisfy all requirements, or a requirement is
listed which is not recognized by the implementation, it is a fatal
error and the implementation must not attempt to run the process,
unless overridden at user option.
- name: hints
type: Any[]?
jsonldPredicate:
_id: "cwl:hints"
noLinkCheck: true
mapSubject: class
doc: |
Declares hints applying to either the runtime environment or the
workflow engine that may be helpful in executing this workflow step. It is
not an error if an implementation cannot satisfy all hints, however
the implementation may report a warning.
- name: run
type: [string, Process]
jsonldPredicate:
_id: "cwl:run"
_type: "@id"
subscope: run
doc: |
Specifies the process to run. If `run` is a string, it must be an absolute IRI
or a relative path from the primary document.
- name: when
type:
- "null"
- Expression
jsonldPredicate: "cwl:when"
doc: |
If defined, only run the step when the expression evaluates to
`true`. If `false` the step is skipped. A skipped step
produces a `null` on each output.
- name: WorkflowStep
type: record
extends: AbstractWorkflowStep
docParent: "#Workflow"
- name: ScatterWorkflowStep
type: record
extends: AbstractWorkflowStep
docParent: "#Workflow"
doc: |
To use scatter/gather,
[ScatterFeatureRequirement](#ScatterFeatureRequirement) must be specified
in the workflow or workflow step requirements.
A "scatter" operation specifies that the associated workflow step or
subworkflow should execute separately over a list of input elements. Each
job making up a scatter operation is independent and may be executed
concurrently.
The `scatter` field specifies one or more input parameters which will be
scattered. An input parameter may be listed more than once. The declared
type of each input parameter implicitly becomes an array of items of the
input parameter type. If a parameter is listed more than once, it becomes
a nested array. As a result, upstream parameters which are connected to
scattered parameters must be arrays.
All output parameter types are also implicitly wrapped in arrays. Each job
in the scatter results in an entry in the output array.
If any scattered parameter runtime value is an empty array, all outputs are
set to empty arrays and no work is done for the step, according to
applicable scattering rules.
If `scatter` declares more than one input parameter, `scatterMethod`
describes how to decompose the input into a discrete set of jobs.
* **dotproduct** specifies that each of the input arrays are aligned and one
element taken from each array to construct each job. It is an error
if all input arrays are not the same length.
* **nested_crossproduct** specifies the Cartesian product of the inputs,
producing a job for every combination of the scattered inputs. The
output must be nested arrays for each level of scattering, in the
order that the input arrays are listed in the `scatter` field.
* **flat_crossproduct** specifies the Cartesian product of the inputs,
producing a job for every combination of the scattered inputs. The
output arrays must be flattened to a single level, but otherwise listed in the
order that the input arrays are listed in the `scatter` field.
# Conditional execution (Optional)
The condition is evaluated after `scatter`, using the input object
of each individual scatter job. This means over a set of scatter
jobs, some may be executed and some may be skipped. When the
results are gathered, skipped steps must be `null` in the output
arrays.
fields:
- name: scatter
type:
- string?
- string[]?
jsonldPredicate:
"_id": "cwl:scatter"
"_type": "@id"
"_container": "@list"
refScope: 0
- name: scatterMethod
doc: |
Required if `scatter` is an array of more than one element.
type: ScatterMethod?
jsonldPredicate:
"_id": "cwl:scatterMethod"
"_type": "@vocab"
- name: LoopWorkflowStep
type: record
extends: AbstractWorkflowStep
docParent: "#Workflow"
doc: |
# Iterative execution (Optional)
The `loop` field controls iterative execution. It defines the input
parameters of the loop iterations after the first one (inputs of the
first iteration are the step input parameters, as usual). If no
`loop` rule is specified for a given step `in` field, the initial
value is kept constant among all iterations.
When a `loop` field is present, the `when` field is mandatory. It is
evaluated before each loop iteration and acts as a termination condition:
as soon as the `when` expression evaluates to `false`, the loop terminates
and the step outputs are propagated to the subsequent workflow steps.
The `outputMethod` field describes how to deal with loop outputs after
termination:
* **last_iteration** specifies that only the last computed element for
each output parameter should be propagated to the subsequent steps.
This is the default value.
* **all_iterations** specifies that an array with all output values
computed at the end of each loop iteration should be propagated to
the subsequent steps. Elements in the array must be ordered according
to the loop iterations that produced them.
Iterative execution in CWL is an optional feature and is not required
to be implemented by all consumers of CWL documents. An implementation that
does not support iterative executions must return a fatal error when
attempting to execute a workflow that uses iterative constructs the
implementation does not support.
fields:
- name: loop
doc: |
Defines the input parameters of the loop iterations after the first one
(inputs of the first iteration are the step input parameters). If no
`loop` rule is specified for a given step `in` field, the initial value
is kept constant among all iterations.
type: LoopInput[]?
jsonldPredicate:
_id: "cwl:loop"
mapSubject: id
mapPredicate: outputSource
- name: outputMethod
doc: |
If not specified, the default method is "last_iteration".
type: LoopOutputMethod?
default: last_iteration
jsonldPredicate:
"_id": "cwl:outputMethod"
"_type": "@vocab"
- name: when
type:
- Expression
jsonldPredicate: "cwl:when"
doc: |
Only run the next iteration when the expression evaluates to `true`.
If the first iteration evaluates to `false` the step is skipped.
A skipped step produces a `null` on each output if the `outputMethod`
is set to `last_iteration`, and an empty array if the `outputMethod`
is set to `all_iterations`.
- name: Workflow
type: record
extends: "#Process"
documentRoot: true
specialize:
- specializeFrom: InputParameter
specializeTo: WorkflowInputParameter
- specializeFrom: OutputParameter
specializeTo: WorkflowOutputParameter
doc: |
A workflow describes a set of **steps** and the **dependencies** between
those steps. When a step produces output that will be consumed by a
second step, the first step is a dependency of the second step.
When there is a dependency, the workflow engine must execute the preceding
step and wait for it to successfully produce output before executing the
dependent step. If two steps are defined in the workflow graph that
are not directly or indirectly dependent, these steps are **independent**,
and may execute in any order or execute concurrently. A workflow is
complete when all steps have been executed.
Dependencies between parameters are expressed using the `source`
field on [workflow step input parameters](#WorkflowStepInput) and
`outputSource` field on [workflow output
parameters](#WorkflowOutputParameter).
The `source` field on each workflow step input parameter expresses
the data links that contribute to the value of the step input
parameter (the "sink"). A workflow step can only begin execution
when every data link connected to a step has been fulfilled.
The `outputSource` field on each workflow step input parameter
expresses the data links that contribute to the value of the
workflow output parameter (the "sink"). Workflow execution cannot
complete successfully until every data link connected to an output
parameter has been fulfilled.
## Workflow success and failure
A completed step must result in one of `success`, `temporaryFailure` or
`permanentFailure` states. An implementation may choose to retry a step
execution which resulted in `temporaryFailure`. An implementation may
choose to either continue running other steps of a workflow, or terminate
immediately upon `permanentFailure`.
* If any step of a workflow execution results in `permanentFailure`, then
the workflow status is `permanentFailure`.
* If one or more steps result in `temporaryFailure` and all other steps
complete `success` or are not executed, then the workflow status is
`temporaryFailure`.
* If all workflow steps are executed and complete with `success`, then the
workflow status is `success`.
# Extensions
[ScatterFeatureRequirement](#ScatterFeatureRequirement) and
[SubworkflowFeatureRequirement](#SubworkflowFeatureRequirement) are
available as standard [extensions](#Extensions_and_Metadata) to core
workflow semantics.
fields:
- name: "class"
jsonldPredicate:
"_id": "@type"
"_type": "@vocab"
type:
type: enum
name: Workflow_class
symbols:
- cwl:Workflow
- name: steps
doc: |
The individual steps that make up the workflow. Each step is executed when all of its
input data links are fulfilled. An implementation may choose to execute
the steps in a different order than listed and/or execute steps
concurrently, provided that dependencies between steps are met.
type:
- type: array
items: AbstractWorkflowStep
jsonldPredicate:
mapSubject: id
- type: record
name: SubworkflowFeatureRequirement
extends: ProcessRequirement
doc: |
Indicates that the workflow platform must support nested workflows in
the `run` field of [AbstractWorkflowStep](#AbstractWorkflowStep).
fields:
- name: "class"
type:
type: enum
name: SubworkflowFeatureRequirement_class
symbols:
- cwl:SubworkflowFeatureRequirement
doc: "Always 'SubworkflowFeatureRequirement'"
jsonldPredicate:
"_id": "@type"
"_type": "@vocab"
- name: ScatterFeatureRequirement
type: record
extends: ProcessRequirement
doc: |
Indicates that the workflow platform must support the `scatter` and
`scatterMethod` fields of [ScatterWorkflowStep](#ScatterWorkflowStep).
fields:
- name: "class"
type:
type: enum
name: ScatterFeatureRequirement_class
symbols:
- cwl:ScatterFeatureRequirement
doc: "Always 'ScatterFeatureRequirement'"
jsonldPredicate:
"_id": "@type"
"_type": "@vocab"
- name: MultipleInputFeatureRequirement
type: record
extends: ProcessRequirement
doc: |
Indicates that the workflow platform must support multiple inbound data links
listed in the `source` field of [WorkflowStepInput](#WorkflowStepInput).
fields:
- name: "class"
type:
type: enum
name: MultipleInputFeatureRequirement_class
symbols:
- cwl:MultipleInputFeatureRequirement
doc: "Always 'MultipleInputFeatureRequirement'"
jsonldPredicate:
"_id": "@type"
"_type": "@vocab"
- type: record
name: StepInputExpressionRequirement
extends: ProcessRequirement
doc: |
Indicate that the workflow platform must support the `valueFrom` field
of [WorkflowStepInput](#WorkflowStepInput).
fields:
- name: "class"
type:
type: enum
name: StepInputExpressionRequirement_class
symbols:
- cwl:StepInputExpressionRequirement
doc: "Always 'StepInputExpressionRequirement'"
jsonldPredicate:
"_id": "@type"
"_type": "@vocab"
- {$import: Operation.yml}