Split kube-burner repository

[rsevilla87]

The quantify of code of the kube-burner repository has grown considerably in the latest months. Part of this growth is thanks to the contributions made to OCP wrapper code and workloads.

Holding the whole OCP wrapper code in this repository is becoming an issue, for several reasons I'm gonna try to summarize along these lines:

• Kube-burner tries to be an agnostic platform performance & scale framework for k8s, holding OpenShift related code in this repository is not something desirable
• Modifying an ocp-wrapper related file triggers all the e2e test suite including k8s tests which are executed with "regular" kube-burner
• Releases: Cutting new releases of kube-burner just because we've made some modifications in a ocp wrapper workload is not a correct approach
• Organization: Having a dedicated repository for the OCP wrapper code simplifies project's organization and makes easier to find issues, updating ocp related things, etc.

My proposal to make this split is to take the OCP wrapper code to a new repository kube-buner-ocp. The code of this new repository will be:

• workloads folder, https://github.com/cloud-bulldozer/kube-burner/tree/master/pkg/workloads, but keeping helpers.go in this repository after some updates to make it platform/wrapper agnostic
• ocp subcommand https://github.com/cloud-bulldozer/kube-burner/blob/master/cmd/kube-burner/ocp.go
• OCP workloads configurations: https://github.com/cloud-bulldozer/kube-burner/tree/master/cmd/kube-burner/ocp-config
• OCP tests: https://github.com/cloud-bulldozer/kube-burner/tree/master/test
• Documentation

An overview of how this repository would look like can be found at https://github.com/rsevilla87/kube-burner-ocp, compiling the code from this repository results in having a new kube-burner-ocp binary with only the ocp wrapper options:

$ ./kube-burner-ocp -h
This kube-burner plugin is meant to be used against OpenShift clusters and serve as a shortcut to trigger well-known workloads

Usage:
  ocp [command]

Available Commands:
  cluster-density-ms             Runs cluster-density-ms workload
  cluster-density-v2             Runs cluster-density-v2 workload
  completion                     Generate the autocompletion script for the specified shell
  crd-scale                      Runs crd-scale workload
  help                           Help about any command
  index                          Runs index sub-command
  networkpolicy-matchexpressions Runs networkpolicy-matchexpressions workload
  networkpolicy-matchlabels      Runs networkpolicy-matchlabels workload
  networkpolicy-multitenant      Runs networkpolicy-multitenant workload
  node-density                   Runs node-density workload
  node-density-cni               Runs node-density-cni workload
  node-density-heavy             Runs node-density-heavy workload
  pvc-density                    Runs pvc-density workload

<truncated>

There're several things we need to consider like:

• kube-burner-ocp will depend on the core kube-burner project, thanks to this, we can cut new kube-burner versions w/o affecting the ocp wrapper, since the ocp wrapper will depend on a fixed kube-burner core tag
• The kube-burner-ocp project must be versioned too, preferably following semantic versioning
• When a change needed by kube-burner-ocp is made into kube-burner core, we will need to cut releases in both kube-burner and kube-burner-ocp to get a binary with that change.

cc: @smalleni @jtaleric @vishnuchalla @dry923 @josecastillolema @paigerube14 @afcollins

[dry923]

Works for me. My only comment would be we need to properly doc and probably add an alert/help notification if a user tries to run kube-burner ocp .... for the time being

[venkataanil]

This is a good idea. kube-burner will focus on implementing a spec (yaml) which has "jobs". It can focus on optimizing k8s requests etc.
On the other hand, ocp wrapper repo (or workloads repo) can have workloads like cluster-density, node-density, web-burner etc..

[smalleni]

+1 on the split. I would also like to advocate for moving both of these repos into a new kube -burner org to manage the growth and traction kube-burner is seeing. That would also help us add additional repositories like those that house the grafonnet code for kube-burner specific dashboards and pave the way to build a wider ecosystem around kube-burner.

[mtodor]

I like this direction. Instead of focusing on OCP wrapper and workload (job) definitions, make kube-burner a tool that runs defined jobs on a Kubernetes cluster and collects the metrics. 👍