The benchmark is based on knb
from the k8s-bench-suite.
See PROTOCOL.md
Conditions for a CNI to be benchmarked are :
- it must setup on on-premise bare-metal cluster
- it must be easy to setup (a yaml manifest, a helm chart, or an operator)
- it must not relies on proprietary hardware
All benchmark runs are recorded with record-bench.sh which uses Asciinema :
record-bench.sh starts asciinema recording, and call benchmark.sh who is in charge of :
- setup lab
- setup cni
- run 3 rounds of benchmark, save data to knbdata files
- teardown lab
Please note that benchmark.sh uses setup.sh, the node deployment script that is tailored for our MaaS-based lab environment.
Example :
Results for human being with charts and interpretation are available in an article on Medium.
Work in progress ... Still writing it for now.
You can also check aggregated results on the spreadsheet here : https://docs.google.com/spreadsheets/d/1M8vtxTBN_5Ih36B5lPsZy6w56p85YaCXhxIVFCrtZXc/edit?usp=sharing
Values injected in the spreadsheet are in files results/<cni>.<distrib>-<kernel>/<cni>.<distrib>-<kernel>-run<x>.tsv
Raw results are available in this repository, just check the results directory for *.knbdata
files.
You can generate reports with knb, for example :
knb -fd results/antrea-default.u20.04-hwe/antrea-default.u20.04-hwe-run1.knbdata -o text
# or
knb -fd results/doc-antrea.u18.04-default/doc-antrea.u18.04-default-run1.knbdata -o json
# or
knb -fd results/doc-antrea.u18.04-default/doc-antrea.u18.04-default-run1.knbdata -o yaml
As knbdata
files are just simple tar.gz archives, you can also uncompress the file to see raw containers logs (showing data even before being parsed by knb
)