CO-103: integration and performance tests runner plugin (#23)

* CO-103: integration and performance tests runner plugin

- [x] gatling sbt plugin based performance test task added
- [x] integration tests are surrounded with docker tasks

* docker version checking plugin is added

* add some todos

* add dump logs plugin

* health check task is added

* small fixes

* scalafmt

* header and scalastyle fixes

* docker lifecycle is now running nearly expected

* scripted tests added for docker-version plugin

* fix build name for docker-version tests

* tests for health plugin

* health check pooling is added

* cleanStop is now running if after failing tests also √

* pooling logic fixed

* revert checkContainersHealth to Task[Unit]

* clean the code base and add in-code docs

* small fix npe

* update readme

* scripted test added for test runner plugin but there is an error in healthcheck task

* small fix in test

* fix recursion problem on healthcheck pool

* improve test runner

* simple performance added to test

* make jenkins happy for now

* improve test scripts

* Added in updateToLatest

* Proposed changes for docker health plugin

* go over the docker health plugin and tests

* remove newline

* dockerhealth scripted test passing now on local with some FIXMEs

* solve testrunner dependency problem

* ensure testrunner is green

* fix scalastyle error

* tests are green on local again

* remove dependency on lower level tests for both perf and int tasks

.gitignore

@@ -1,5 +1,6 @@
 target/
 *.log
+.idea/
 
 # DO NOT ADD ANYTHING TO THIS FILE UNLESS IT IS DIRECTLY
 # RELATED TO THE BUILD.

README.md

@@ -269,6 +269,83 @@ Configuration of this plugin should be avoided in local project SBT build files.
 
 No special tasks are enabled for this plugin.
 
+## `CakeDockerVersionPlugin`: Docker and Docker Compose Version Checking
+
+Plugin requirements: `CakeDockerComposePlugin`
+
+Enabling this plugin in a project provides setting minimum required `docker` and `docker-compose` versions and checking
+capability of either minimum versions are installed in the running environment or not.
+
+### Plugin Configuration
+
+The following configuration settings can be modified in projects that enable this plugin:
+* `minimumDockerVersion` (`(1, 13)` by default) - minimum docker version is `1.13`.
+* `minimumDockerComposeVersion` (`(1, 10)` by default) - minimum docker-compose version is `1.10`.
+
+### SBT Tasks
+
+The following SBT tasks are enabled:
+* `checkDockerVersion` - checks if docker version of machine is above the `minimumDockerVersion`.
+* `checkDockerComposeVersion` - checks if docker-compose version of machine is above the `minimumDockerComposeVersion`. 
+
+
+## `CakeDockerHealthPlugin`: Health Checking of Containers in Docker Compose Scope
+
+Plugin requirements: `CakeDockerComposePlugin`
+
+Enabling this plugin in a project provides dumping the logs and health checking of the containers 
+defined in the `dockerComposeFile` file in the `CakeDockerComposePlugin`. 
+
+### Plugin Configuration
+
+There is no special configuration provided by this plugin.
+
+### SBT Tasks
+
+SBT tasks of that plugin requires a running docker-compose stack.
+The following SBT tasks are enabled:
+* `dumpContainersLogs` - dumping logs of each container in the docker-compose scope.
+* `checkContainersHealth` - checking the health status of each container in the docker-compose scope.
+ All containers who have a health check definition should be healthy. 
+ If a container does not have a health check instruction, it will be ignored.
+ You can check that [post](https://blog.couchbase.com/docker-health-check-keeping-containers-healthy/) for details of docker health check feature.
+
+## `CakeTestRunnerPlugin`: Integration and Performance Test Lifecycle within Docker Stack
+
+Plugin requirements: `CakeDockerComposePlugin`
+
+Enabling this plugin in a project provides the functionality of running integration and performance tests
+within automatically managed docker fleet (stack). This plugin provides resource cleaning in case of failure or success
+of the tests. The lifecycle is consisting of following steps:
+
+* docker and docker-compose version checks
+* docker-compose up
+* health check of containers in docker-compose scope
+* running tests
+* dumping logs of containers
+* docker-compose down
+* docker remove containers in docker-compose scope
+
+
+Integration tests depend on unit tests.
+Performance tests depend on integration tests and using GatlingPlugin and its GatlingIt (integration) tests settings.
+GatlingIt looks for performance tests under `it` directory.
+
+### Plugin Configuration
+
+In order to wait enough for containers to reach to the healthy status,
+the following configuration settings can be modified in projects that enable this plugin:
+
+* `healthCheckRetryCount` (5 by default) - defines how much times plugin needs in total to retry 
+all containers reach to the healthy status
+* `healthCheckIntervalInSeconds` (5 seconds by default) - defines the rechecking interval for containers' health again. 
+
+### SBT Tasks
+
+The following SBT tasks are enabled:
+* `integrationTests` - runs integration tests within all docker operations and steps
+* `performanceTests` - runs performance tests within all docker operations and steps 
+
 ## `CakePublishMavenPlugin`: Artifact Publishing
 
 Plugin requirements: `DynVerPlugin`

build.sbt

@@ -56,6 +56,9 @@ addSbtPlugin(SbtDependencies.packager)
 addSbtPlugin(SbtDependencies.Coursier.sbt)
 addSbtPlugin(SbtDependencies.scoverage)
 addSbtPlugin(SbtDependencies.wartRemover)
+
+addSbtPlugin(SbtDependencies.gatling)
+
 // The following plugin is otherwise known as neo-sbt-scalafmt
 // - see: https://github.com/lucidsoftware/neo-sbt-scalafmt
 addSbtPlugin(SbtDependencies.scalafmt)
@@ -107,19 +110,19 @@ pomExtra := {
   val repository = name.value
 
   <url>https://github.com/{user}/{repository}</url>
-  <scm>
-    <url>
-      https://github.com/{user}/{repository}
-    </url>
-    <connection>
-      https://github.com/{user}/{repository}
-    </connection>
-  </scm>
-  <developers>
-    <developer>
-      <id>{user}</id>
-    </developer>
-  </developers>
+    <scm>
+      <url>
+        https://github.com/{user}/{repository}
+      </url>
+      <connection>
+        https://github.com/{user}/{repository}
+      </connection>
+    </scm>
+    <developers>
+      <developer>
+        <id>{user}</id>
+      </developer>
+    </developers>
 }
 
 scriptedSettings

project/project/CakePlatformDependencies.scala

@@ -101,6 +101,7 @@ object CakePlatformDependencies {
     val buildInfo: ModuleID = "com.eed3si9n" % "sbt-buildinfo" % "0.7.0"
     val digest: ModuleID = "com.typesafe.sbt" % "sbt-digest" % "1.1.3"
     val dynver: ModuleID = "com.dwijnand" % "sbt-dynver" % "1.2.0"
+    val gatling: ModuleID = "io.gatling" % "gatling-sbt" % "2.2.1"
     val git: ModuleID = "com.typesafe.sbt" % "sbt-git" % "0.9.3"
     val gzip: ModuleID = "com.typesafe.sbt" % "sbt-gzip" % "1.0.2"
     val header: ModuleID = "de.heikoseeberger" % "sbt-header" % "2.0.0"

scalastyle-config.xml

@@ -92,7 +92,7 @@
             <parameter name="groups">java,scala,others,organization</parameter>
             <parameter name="group.java">javax?\..*</parameter>
             <parameter name="group.scala">scala\..*</parameter>
-            <parameter name="group.others">.*</parameter>
+            <parameter name="group.others">(?!net\.cakesolutions\.).*</parameter>
             <parameter name="group.organization">net\.cakesolutions\..*</parameter>
         </parameters>
     </check>

src/main/scala/net/cakesolutions/CakeBuildInfoPlugin.scala

@@ -67,13 +67,7 @@ object CakeBuildInfoPlugin extends AutoPlugin {
         "gitRepository" -> gitRepository
       )
     },
-    dockerInfo := {
-      externalBuildTools ++= Seq(
-        "docker --version" ->
-          "`docker` command should be installed and PATH accessible"
-      )
-      Map("buildDockerVersion" -> buildDockerVersion)
-    },
+    dockerInfo := Map("buildDockerVersion" -> buildDockerVersion),
     checkExternalBuildTools := {
       externalBuildTools.value.foreach {
         case (checkCmd, errorMsg) =>

src/main/scala/net/cakesolutions/CakeBuildPlugin.scala

@@ -10,6 +10,7 @@ import java.util.concurrent.atomic.AtomicLong
 import scala.util._
 
 import com.lucidchart.sbt.scalafmt.ScalafmtCorePlugin.autoImport._
+import io.gatling.sbt.GatlingPlugin
 import sbt._
 import sbt.Keys._
 import scoverage.ScoverageKeys._
@@ -147,6 +148,19 @@ object CakeBuildKeys {
             Defaults.testSettings ++ sensibleTestSettings ++ scalafmtSettings
           )
         )
+
+    /**
+      * Enable performance test on top of integration test settings
+      * and GatlingPlugin.
+      *
+      * @return Project with performance test settings and configuration applied
+      */
+    def enablePerformanceTests: Project =
+      p.enableIntegrationTests
+        .enablePlugins(GatlingPlugin)
+        .settings(
+          libraryDependencies ++= CakePlatformKeys.PlatformBundles.gatling
+        )
   }
 
   /**

src/main/scala/net/cakesolutions/CakeDockerComposePlugin.scala

@@ -4,11 +4,11 @@
 package net.cakesolutions
 
 import com.typesafe.sbt.SbtNativePackager._
-import com.typesafe.sbt.packager.Keys._
-import net.cakesolutions.CakeBuildInfoKeys.externalBuildTools
 import sbt._
 import sbt.Keys._
 
+import net.cakesolutions.CakeBuildInfoKeys.externalBuildTools
+
 /**
   * Cake recommended tasks for configuring and using docker-compose within SBT
   * build files (e.g. for use within integration tests, etc.)
@@ -25,7 +25,7 @@ object CakeDockerComposePlugin extends AutoPlugin {
     * When this plugin is enabled, {{autoImport}} defines a wildcard import for
     * set, eval, and .sbt files.
     */
-  val autoImport = CakeDockerComposePluginKeys
+  val autoImport = CakeDockerComposeKeys
   import autoImport._
 
   private val dockerComposeConfigCheckTask: Def.Initialize[Task[Unit]] =
@@ -141,7 +141,7 @@ object CakeDockerComposePlugin extends AutoPlugin {
 /**
   * SBT docker-compose build settings and tasks
   */
-object CakeDockerComposePluginKeys {
+object CakeDockerComposeKeys {
 
   /**
     * Setting key defining the project files to be used by docker-compose

src/main/scala/net/cakesolutions/CakeDockerHealthPlugin.scala

@@ -0,0 +1,80 @@
+// Copyright: 2017 https://github.com/cakesolutions/sbt-cake/graphs
+// License: http://www.apache.org/licenses/LICENSE-2.0
+
+package net.cakesolutions
+
+import sbt._
+import sbt.Keys._
+
+import net.cakesolutions.CakeDockerVersionKeys.{minimumDockerComposeVersion, minimumDockerVersion}
+import net.cakesolutions.internal.Version
+
+/**
+  * Plugin for checking `docker` and `docker-compose` versions on the system
+  * against minimum required versions which have default values in the plugin,
+  * and also configurable with provided setting keys.
+  */
+object CakeDockerHealthPlugin extends AutoPlugin {
+
+  import net.cakesolutions.internal.CakeDockerUtils._
+
+  /**
+    * When this plugin is enabled, {{autoImport}} defines a wildcard import for
+    * set, eval, and .sbt files.
+    */
+  val autoImport = CakeDockerHealthKeys
+  import autoImport._
+
+  /** @see http://www.scala-sbt.org/0.13/api/index.html#sbt.package */
+  override def requires: Plugins =
+    CakeDockerComposePlugin && CakeDockerVersionPlugin
+
+  /** @see http://www.scala-sbt.org/0.13/api/index.html#sbt.package */
+  override def trigger: PluginTrigger = allRequirements
+
+  /** @see http://www.scala-sbt.org/0.13/api/index.html#sbt.package */
+  override val projectSettings = Seq(
+    // Docker 1.12.0 (2016-07-28) introduced HEALTHCHECK
+    // see https://docs.docker.com/release-notes/docker-engine/#1120-2016-07-28
+    minimumDockerVersion :=
+      Version.selectLatest(minimumDockerVersion.value, (1, 12)),
+    // Docker-compose 1.10.0 (2017-01-18) introduced healthcheck
+    // see https://docs.docker.com/release-notes/docker-compose/#1100-2017-01-18
+    minimumDockerComposeVersion :=
+      Version.selectLatest(minimumDockerComposeVersion.value, (1, 10)),
+    dumpContainersLogs := dumpLogs(
+      CakeDockerComposeKeys.dockerComposeFiles.value,
+      file("target")
+    )(streams.value.log),
+    checkContainersHealth := {
+      require(
+        checkHealth(CakeDockerComposeKeys.dockerComposeFiles.value)(
+          streams.value.log
+        ),
+        "All containers should be healthy"
+      )
+    }
+  )
+
+}
+
+/**
+  * Keys that will be auto-imported when this plugin is enabled.
+  */
+object CakeDockerHealthKeys {
+
+  /**
+    * Task that dumps the logs of each container in the
+    * docker-compose scope.
+    */
+  val dumpContainersLogs: TaskKey[Unit] =
+    taskKey[Unit]("Dumps target containers' logs")
+
+  /**
+    * Task that checks the health status of each container in the
+    * docker-compose scope and they should be in healthy state.
+    */
+  val checkContainersHealth: TaskKey[Unit] =
+    taskKey[Unit]("Checks target containers' health")
+
+}

src/main/scala/net/cakesolutions/CakeDockerPlugin.scala

@@ -55,6 +55,12 @@ object CakeDockerPlugin extends AutoPlugin {
 
       Cmd("LABEL", labelArguments.mkString(" "))
     },
+    CakeBuildInfoKeys.externalBuildTools ++= Seq(
+      (
+        "docker --version",
+        "`docker` command should be installed and PATH accessible"
+      )
+    ),
     dockerRemove := dockerRemoveTask.value
   )