Deploy Ezmeral Container Platform on KVM

Please consider this as a work-in-progress. Contributions, recommendations, pull requests are always welcomed. This worked in our demo lab with CentOS 8 single machine deployment. Complex virtual network and clustering is not available yet (please see to-do below).

What & Why

To re-utilize scripts and processes by https://github.com/hpe-container-platform-community/hcp-demo-env-aws-terraform/, enabling a quick deployment opportunity for a demo environemnt for HPE Ezmeral Container Platform.

Pre-requisites

• CentOS/RHEL 7+ (tested on CentOS 8.2 Host)
• libvirt, qemu-kvm, libvirt-client, virt-install
• Python3, openssh, nc, curl, ipcalc, hpecp
• Passwordless sudo

Prepare environment

• KVM & Qemu

sudo dnf install -y qemu-kvm libvirt libvirt-client

• virt-install

sudo dnf install -y virt-install

• python3 & pip3

sudo dnf install -y python3

• ssh-keygen

sudo dnf install -y openssh

• nc

sudo dnf install -y nmap-ncat

• curl

sudo dnf install -y curl

• ipcalc

pip3 install --user ipcalc six

• hpecp

pip3 install --user hpecp

• Edit sudoers file

Collect and customize

git clone https://github.com/hpe-container-platform-community/hcp-demo-env-kvm-bash.git 
cd hcp-demo-env-kvm-bash
vi etc/kvm_config.sh

PROJECT_DIR=this

CENTOS_IMAGE_FILE=path-to-local CentOS-7-x86_64-GenericCloud-2003.qcow2

LOCAL_YUM_REPO=url # Not used now

BEHIND_PROXY=True|False # This defines if we setup env vars for proxy for tools such as yum, wget, curl etc

PROXY_URL=url (if BEHIND_PROXY=True)

LOCALREPO=url to .repo file ! should be replacing LOCAL_YUM_REPO var above

TIMEZONE=your time zone in ?? format ie, "Asia/Dubai"

EPIC_FILENAME="path-to-epic-installer"

EPIC_DL_URL=url # to download EPIC_FILENAME

IMAGE_CATALOG=url # to download EPIC images if you want to use local image repo

CREATE_EIP_GATEWAY=True|False # to enable/disable IP forwarding to gateway # work in progress

OPTIONAL # If you want customization

DOMAIN="ecp.demo"

VIRTUAL_NET_NAME="ecpnet"

NET=10.1.10 # Use this notation x.x.x (skip last dot as it will be added) BRIDGE=virbr10

# Define hosts in a rather strange way

hosts=('controller' 'gw' 'host1' 'host2' 'host3') # hostnames are hard coded (avoid using name gateway as it is resolving to KVM host within VMs) cpus=(16 4 8 8 8)

mems=(65536 32768 65536 65536 65536)

# assign roles (not all roles implemented yet)

# possible roles: controller gateway worker ad rdp mapr1 mapr2

roles=('controller' 'gateway' 'worker' 'worker' 'worker')

# disk sizes (data disk size per host)

disks=(512 0 512 512 512)

Installation

Run

./bin/kvm_create_new_environment.sh

Wait for completion (45 min to 1.5h)

ssh scripts/commands will be copied to ./generated directory. And connectivity information will be displayed as part of script output.

./generated/ssh_controller.sh
./generated/ssh_gw.sh

Open a browser to gateway (if CREATE_EIP_GATEWAY enabled an ip forwarding rule to ports 80/443/8080 will be created for local KVM host)

https://gw.ecp.demo

TODO

• Test with non-root user

• Selectively deploy K8s cluster or EPIC cluster

• Attach to GPU on host

• Public IP via host interface

• Enable RDP host

• Enable external MapR cluster

• Clean up (unneeded variables etc)

• Optimizations (less reboots, less modifications to source scripts etc)

• Enable local YUM repo (use nfs to avoid downloading packages if possible)

• Enable mounted image catalog (nfs to avoid copying catalog images if possible)

Troubleshooting

If you get error for backing disk not accessible "Permission denied", be sure that SE allows permission to all the way up to the backing file /full/path/to/centos.qcow2

Replace the full path: sudo setfacl -m u:qemu:rx /full/path/to/ If file is on NFS share: sudo setsebool virt_use_nfs on