Automated Kapacitor task testing using Docker containers

This repository contains code for performing automated checks for testing Kapacitor tasks and variable-files.

Requirements

• Docker daemon (the docker package needs to be installed, and the docker service running)
• Access to pull Dockerhub images and required resources from package repositories during the build process

Flow

1. Docker build (Dockerfile)

2. Docker run  --> run.sh   --> Starting InfluxDB
                            |--> Starting Kapacitor -->  topic_handler_exec.yaml  -->  alert.sh
                            |--> run_tests.sh
                            |--> check_tests.py

Usage

There are four folders that need to be populated (see below) before running the test. To run the test, simply run the following commands in the folder containing the Dockerfile

docker build -t "kapacitor-test" .
docker run "kapacitor-test"

Sample output on success

$ docker build -t "kapacitor-test" . 2>&1 > /tmp/build.log

$ docker run "kapacitor-test"
Starting InfluxDB
Starting Kapactior
Startup done, injecting data
Injecting test data: http_response.metrics.sample
Injecting done, waiting 15s before checking

$ echo $?
0

kapacitor-templates/

This folder should contain all the TICKscript templates that should be tested Files matching the pattern *.* will be copied into /etc/kapacitor/load/templates/ in the container

See https://docs.influxdata.com/kapacitor/v1.5/working/template_tasks/ for more information on these files

kapacitor-var-files/

This folder should contain all the Kapacitor tasks / Variable files that should be tested Files matching the pattern *.* will be copied into /etc/kapacitor/load/tasks/ in the container

See https://docs.influxdata.com/kapacitor/v1.5/working/template_tasks/ for more information on these files

kapacitor-var-tests/checks/

This folder should contain all the alert checks to use for matching alert output Files matching the pattern * will be copied into /checks/ in the container

These files should be in JSON (file ending with .json), with the following format, one or more alerts may be defined in the same or in several files:

{
  "Expected Kapacitor alert ID": "Expected alert level / severity", 
  "Expected Kapacitor alert ID": "Expected alert level / severity", 
  "Expected Kapacitor alert ID": "Expected alert level / severity"
}  

See kapacitor-var-tests/checks/http_response.json.sample for how a sample check might look. The sample assumes that an alarm with level/severity WARNING would be raised with the id mytask:resource=googletest,server=http://google.com,host=google.com.

See https://docs.influxdata.com/kapacitor/v1.5/nodes/alert_node/#id for more information regarding Alert id's and severity/level.

kapacitor-var-tests/testdata/

This folder should contain all the testing data to use for injecting into InfluxDB Files matching the pattern * will be copied into /testdata/ in the container

These files should be in the line-protocol format (file ending with .metrics), one or more metrics may be defined in the same or in several files.

See kapacitor-var-tests/testdata/http_response.metrics.sample for how these files might look like.

See also https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/ for more information regarding the line-protocol.

Files

Dockerfile

This is the Dockerfile used to instruct Docker for the build of the container

run.sh

This file is the entrypoint in the Docker container, and it will execute the most of the magic with a series of other scripts (See Flow)

Exit-codes

• Exit-code 101: Failed to create database in Influx
• Exit-code 102: Kapacitor died after starting
• Exit-code 103: Failed to inject testing data into Influx
• Exit-code 104: A check performed on the events output from Kapacior

topic_handler_exec.yaml

This is a topic handler that will be loaded into Kapacitor, it simply calls a script (alert.sh) for each event Kapacitor triggers

alert.sh

This is a very simple "script", it only takes STDIN and puts it into unique files under /alerts/. These files will be read and checked with check_tests.py

run_tests.sh

A simple script that runs through files containing line-protocol under /testdata/* and attempts to inject it line-by-line into Influx. Exits with code 3 if Influx does not return status 204 on injection.

check_tests.py

The only script written in Python (could be re-written into bash aswell if anyone cares), it loads all the JSON files containing event data (generated by Kapacitor via alert.sh) under /alerts/ and checks it up with the checks under /checks/*.json

Exit-codes

• Exit-code 1: Kapacitor generated an event that does not match any checks
• Exit-code 2: An event did not contain the severity defined in the check
• Exit-code 3: Kapacitor did not generate events for all the defined checks (some checks never triggered)

kapacitor.conf

The configuration file Kapacitor will use in the container.