Skip to content

Latest commit

 

History

History
15 lines (11 loc) · 1014 Bytes

README.md

File metadata and controls

15 lines (11 loc) · 1014 Bytes

dgsh: The Directed Graph Shell

Build Status

The directed graph shell, dgsh, allows the expressive expression of efficient big data set and streams processing pipelines using existing Unix tools as well as custom-built components. It is a Unix-style shell allowing the specification of pipelines with non-linear scatter-gather operations. These form a directed acyclic process graph, which is typically executed by multiple processor cores, thus increasing the operation's processing throughput.

You can find a complete introduction, reference documentation, and illustrated examples in the suite's web site.

See also, a quick video overview and the associated (open access) paper, Extending Unix pipelines to DAGs, published in the IEEE Transactions on Computers, 66(9):1547–1561, 2017.