Pipeline execution with curl

Examples assume that an LP-ETL instance is running on localhost with default settings.

Pipeline from a file

curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]" localhost:8080/resources/executions

Pipeline from a local instance

curl -i -X POST localhost:8080/resources/executions?pipeline=http://localhost:8080/resources/pipelines/created-1464779048078

Pipeline from a file with user inputs

The content of files data-1.dat and data-2.dat will be available in the input component.

curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]" -F "[email protected]" -F "[email protected]" localhost:8080/resources/executions

Pipeline from a local instance with user inputs

The content of files data-1.dat and data-2.dat will be available in the input component.

curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]" -F "[email protected]" localhost:8080/resources/executions?pipeline=http://localhost:8080/resources/pipelines/created-1464779048078

Unpacked pipeline from a file with user inputs

Pipeline used in this examples (pipeline-unpack.jsonld files) must be in a form suitable for execution (execution-ready). A pipeline can be converted to execution-ready format using Storage API. A execution-ready pipeline can also be obtained from the execution directory (file definition/definition.jsonld).

curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]" -F "[email protected]" -F "[email protected]" localhost:8080/resources/executions?unpacked_pipeline=1

Pipeline from a local instance with user provided configuration

In this case additional configuration is provided in the request body. If you want to provide also an input file please refer to the Pipeline from a local instance with user inputs and configuration example.

curl -X POST -H "Content-Type: application/ld+json" -d '{"@id": "http://localhost/base","@type" : "http://etl.linkedpipes.com/ontology/ExecutionOptions","http://linkedpipes.com/ontology/deleteWorkingData" : true,"http://linkedpipes.com/ontology/saveDebugData" : false, "http://linkedpipes.com/ontology/logPolicy" : { "@id" : "http://linkedpipes.com/ontology/log/DeleteOnSuccess" }}' localhost:8080/resources/executions?pipeline=http://localhost:8080/resources/pipelines/created-1515419801862

Pipeline from a local instance with user inputs and configuration

The content of file data.dat will be available in the input component. The content of options.jsonld will be used to configure the execution.

curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]" -F "[email protected]" localhost:8080/resources/executions?pipeline=http://localhost:8080/resources/pipelines/created-1515419801862

The options.jsonld can be used to set additional execution parameter. An example options.jsonld file may look like this:

{
    "@id": "http://localhost/base",
    "@type" : "http://etl.linkedpipes.com/ontology/ExecutionOptions",
    "http://linkedpipes.com/ontology/deleteWorkingData" : true,
    "http://linkedpipes.com/ontology/saveDebugData" : false,  
    "http://linkedpipes.com/ontology/logPolicy" : { 
    "@id" : "http://linkedpipes.com/ontology/log/DeleteOnSuccess" }
}