Elastiflow

Elastiflow is a closed-source network flow collector and enrichment engine

TL;DR

git clone https://github.com/imc-network/elastiflow-helm; cd elastiflow-helm
helm install my-release .

Introduction

This chart was created from the template provided by Bitnami. It bootstraps a Elastiflow deployment on a Kubernetes cluster using Helm

Scaling and limitations

Please note that the current (5.4.1) release of Elastiflow does not have a shared cache. You should not scale the deployment past a single pod unless you are using a proxy mode that supports deterministic hashing (IPVS or eBPF), so that devices that send IPFIX templates only send their flow records to the collector that received the template.

If your flow records are being generated outside the kubernetes cluster, be aware of the effect of SNAT on the records being ingressed. While sFlow stores the collector address in the record itself, IPFIX and NetFlow use the source IP of the UDP packet to determine the collector. Refer to the documentation for your provider for methods of preserving source address if necessary (eBPF, externalTrafficPolicy, etc)

An Ingress template is not included with this chart, as UDP Ingresses are not directly supported by Kubernetes. Consider using an ingress controller that supports custom UDP Ingress like Nginx

Prerequisites

• Kubernetes 1.12+
• Helm 3.1.0

Installing the Chart

To install the chart with the release name my-release:

helm install my-release .

The command deploys Elastiflow on the Kubernetes cluster in the default configuration. The Parameters section lists the parameters that can be configured during installation.

Tip: List all releases using helm list

Uninstalling the Chart

To uninstall/delete the my-release deployment:

helm delete my-release

The command removes all the Kubernetes components associated with the chart and deletes the release.

Parameters

Global parameters

Name	Description	Value
global.imageRegistry	Global Docker image registry	""
global.imagePullSecrets	Global Docker registry secret names as an array	[]
global.storageClass	Global StorageClass for Persistent Volume(s)	""

Common parameters

Name	Description	Value
kubeVersion	Override Kubernetes version	""
nameOverride	String to partially override common.names.fullname	""
fullnameOverride	String to fully override common.names.fullname	""
commonLabels	Labels to add to all deployed objects	{}
commonAnnotations	Annotations to add to all deployed objects	{}
clusterDomain	Kubernetes cluster domain name	cluster.local
extraDeploy	Array of extra objects to deploy with the release	[]

Elastiflow Parameters

Name	Description	Value
config	Elastiflow environment variable config	See docs
image.registry	Elastiflow image registry	docker.io
image.repository	Elastiflow image repository	elastiflow/flow-collector
image.tag	Elastiflow image tag (immutable tags are recommended)	5.4.1
image.pullPolicy	Elastiflow image pull policy	IfNotPresent
image.pullSecrets	Elastiflow image pull secrets	[]
hostNetwork	Enable/disable host network mode in pod	false
replicaCount	Number of Elastiflow replicas to deploy	1
containerPorts.elastiflow	Elastiflow port	9995
customLivenessProbe	Custom livenessProbe that overrides the default one	{}
customReadinessProbe	Custom readinessProbe that overrides the default one	{}
customStartupProbe	Custom startupProbe that overrides the default one	{}
resources.limits	The resources limits for the Elastiflow containers	{}
resources.requests	The requested resources for the Elastiflow containers	{}
existingConfigmap	The name of an existing ConfigMap with your custom configuration for Elastiflow	nil
command	Override default container command (useful when using custom images)	[]
args	Override default container args (useful when using custom images)	[]
hostAliases	Elastiflow pods host aliases	[]
podLabels	Extra labels for Elastiflow pods	{}
podAnnotations	Annotations for Elastiflow pods	{}
podAffinityPreset	Pod affinity preset. Ignored if affinity is set. Allowed values: soft or hard	""
podAntiAffinityPreset	Pod anti-affinity preset. Ignored if affinity is set. Allowed values: soft or hard	soft
nodeAffinityPreset.type	Node affinity preset type. Ignored if affinity is set. Allowed values: soft or hard	""
nodeAffinityPreset.key	Node label key to match. Ignored if affinity is set	""
nodeAffinityPreset.values	Node label values to match. Ignored if affinity is set	[]
affinity	Affinity for Elastiflow pods assignment	{}
nodeSelector	Node labels for Elastiflow pods assignment	{}
tolerations	Tolerations for Elastiflow pods assignment	[]
updateStrategy.type	Elastiflow statefulset strategy type	RollingUpdate
priorityClassName	Elastiflow pods" priorityClassName	""
schedulerName	Name of the k8s scheduler (other than default) for Elastiflow pods	""
lifecycleHooks	for the Elastiflow container(s) to automate configuration before or after startup	{}
extraEnvVars	Array with extra environment variables to add to Elastiflow nodes	[]
extraEnvVarsCM	Name of existing ConfigMap containing extra env vars for Elastiflow nodes	nil
extraEnvVarsSecret	Name of existing Secret containing extra env vars for Elastiflow nodes	nil
extraVolumes	Optionally specify extra list of additional volumes for the Elastiflow pod(s)	[]
extraVolumeMounts	Optionally specify extra list of additional volumeMounts for the Elastiflow container(s)	[]
sidecars	Add additional sidecar containers to the Elastiflow pod(s)	{}
initContainers	Add additional init containers to the Elastiflow pod(s)	{}
service.type	Elastiflow service type	ClusterIP
service.ports.elastiflow	Elastiflow service UDP port	9995
service.nodePorts.http	Node port for HTTP	nil
service.nodePorts.https	Node port for HTTPS	nil
service.clusterIP	Elastiflow service Cluster IP	nil
service.loadBalancerIP	Elastiflow service Load Balancer IP	nil
service.loadBalancerSourceRanges	Elastiflow service Load Balancer sources	[]
service.externalTrafficPolicy	Elastiflow service external traffic policy	Cluster
service.annotations	Additional custom annotations for Elastiflow service	{}
service.extraPorts	Extra ports to expose in Elastiflow service (normally used with the sidecars value)	[]

Other Parameters

Name	Description	Value
rbac.create	Specifies whether RBAC resources should be created	false
serviceAccount.create	Specifies whether a ServiceAccount should be created	false
serviceAccount.name	The name of the ServiceAccount to use.	""

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example,

helm install my-release \
  --set config.EF_FLOW_ACCOUNT_ID=foo \
  --set config.EF_FLOW_ACCOUNT_KEY=bar \
  . 

The above command sets the Elastiflow account ID and license key to foo and bar respectively.

NOTE: Once this chart is deployed, it is not possible to change the application's access credentials, such as usernames or passwords, using Helm. To change these application credentials after deployment, delete any persistent volumes (PVs) used by the chart and re-deploy it, or use the application's built-in administrative tools if available.

Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example,

helm install my-release -f values.yaml .

Tip: You can use the default values.yaml

Configuration and installation details

Rolling VS Immutable tags

It is strongly recommended to use immutable tags in a production environment. This ensures your deployment does not change automatically if the same tag is updated with a different image.

Additional environment variables

In case you want to add extra environment variables (useful for advanced operations like custom init scripts), you can use the extraEnvVars property.

extraEnvVars:
  - name: LOG_LEVEL
    value: error

Alternatively, you can use a ConfigMap or a Secret with the environment variables. To do so, use the extraEnvVarsCM or the extraEnvVarsSecret values.

Sidecars

If additional containers are needed in the same pod as Elastiflow (such as additional metrics or logging exporters), they can be defined using the sidecars parameter. If these sidecars export extra ports, extra port definitions can be added using the service.extraPorts parameter. Learn more about configuring and using sidecar containers.

Pod affinity

This chart allows you to set your custom affinity using the affinity parameter. Find more information about Pod affinity in the kubernetes documentation.

As an alternative, use one of the preset configurations for pod affinity, pod anti-affinity, and node affinity available at the bitnami/common chart. To do so, set the podAffinityPreset, podAntiAffinityPreset, or nodeAffinityPreset parameters.

Troubleshooting

Find more information about how to deal with common errors related to Bitnami's Helm charts in this troubleshooting guide.