Home

Welcome to the flux wiki!

Before we start out building flux, we want to understand whats out there. This will help us in defining the scope of Flipkart Flux and fill the gap wherever we see any

Component	UI Visualisations, Admin Tasks	API/Modelling DAG/StateMachine, DSL/Annotation/Programming Interfaces, Events	Runtime Local/Distributed, Orchestration/ Execution	Integration Configurations, Metrics, RPC support	Deployment Cluster Setup, Discovery	Misc
SWF	Rich	SM can be modelled using programmatic constructs. Notion of 1) activities 2) workflow and 3) asynchronous methods can be used to achieve any workflow constructs.	Distributed + Orchestration	Versions as configurations, Reports, RPC not directly supported	Few AWS configurations to be added to register & start workflows	Still need to explore the UI
Scooby	Create & Edit workflows. (Need to figure if they have admin)	DAGs specified on console. Corresponding code is a camel route	Distributed, Orchestration (using camel) + Execution	Integrations with BigFoot, Varadhi, Spark, Drools	Managed cluster	Presentation
Journey
Activiti	Rich UI (ActivitiExplorer)	State Machine can be constructed using UI or through programming constructs. Uses DSL BPMN.	Distributed, Orchestration + Execution	Can be integrated with Spring, provides History and Audit info	managed cluster	^ supports dbs h2, mysql, oracle, postgres, db2, mssql ^ event listeners, suspend/activation, transactions, timers, retries, history ^ activities, tasks, subprocess, variables, jobs, users, groups ^ script, java, web api, email, mule, camel, java receive (wait for msg) task types
Airflow	Good	A Dag can be constructed using code or UI. Concepts 1.Sensor: Waits for events to happen 2.Remote Execution: Triggers an operation in a remote system. 3Data transfers: Move data from one system to another	stand alone or distributed orchestration + state management		managed cluster	written in python. Dags can be written in python. Uses Mysql as a store for backend. Can be compared to the likes of Azkhaban.
Ceryx	Good	DAG, needs to be programmed, XML input	Distributed (uses akka for communication), orchestration + execution			developed in Scala
Databuilder framework	limited UI which depicts state information & metrics	DAG needs to be programmed	Local, orchestration + execution	has listeners which can be used to plot metrics, to achieve checkpointing and other services	Single node	It's a java library, user has to plug the desired functionality using the listeners
Processing framework