From 6e9e414c4a32514daf452b8372d34712d801a6dc Mon Sep 17 00:00:00 2001 From: Parikshith Date: Wed, 22 Jan 2025 04:29:08 -0500 Subject: [PATCH] e2e: Adding e2e documentation Adding the following in docs/e2e.md: * How to prepare a test config with the clusters * How to run all tests * How to run specific tests * Lists of tests Based on #1477 by Jack Lu Signed-off-by: Parikshith --- docs/e2e.md | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 docs/e2e.md diff --git a/docs/e2e.md b/docs/e2e.md new file mode 100644 index 000000000..39d1a191b --- /dev/null +++ b/docs/e2e.md @@ -0,0 +1,162 @@ + + +# End to End testing + +RamenDR end-to-end (e2e) tests validate various scenarios for regional DR +using predefined workloads and deployment methods. + +## Running End to End tests + +> [!IMPORTANT] +> All commands must be ran from the e2e directory. + +### Preparing a `config.yaml` file + +#### For drenv environment + +If `drenv` was used to configure RDR clusters, easily copy `config.yaml.sample` +and add the clusters kubeconfig paths into `config.yaml` using: + +```sh +cat config.yaml.sample ~/.config/drenv/rdr/config.yaml > config.yaml +``` + +#### For real cluster + +Create a `config.yaml` file by copying the `config.yaml.sample` template: + +```sh +cp config.yaml.sample config.yaml +``` + +Update `config.yaml` by adding kubeconfig paths of the clusters, with format: + +```yaml +Clusters: + c1: + kubeconfigpath: /path/to/kubeconfig/c1 + c2: + kubeconfigpath: /path/to/kubeconfig/c2 + hub: + kubeconfigpath: /path/to/kubeconfig/hub +``` + +### Run all E2E tests + +`run.sh` is a shell script that runs the RamenDR E2E tests for regional DR. + +To run all the tests: + +```sh +./run.sh +``` + +### Run specific E2E tests + +The `-run` option is commonly used when debugging a failing test, developing +a new test, or working on a new deployer or workload. It allows to selectively +execute specific tests by matching full test names using regular expressions, +making it easier to focus on specific scenarios. + +Refer to the [List of tests](#list-of-tests) for a detailed breakdown of +available tests. + +#### Run a single test + +Example: + +```sh +./run.sh -run TestSuites/Exhaustive/subscr-deploy-rbd-busybox +``` + +This command runs the specific test for subscription based RBD busybox application. + +#### Run tests using a specific deployer + +Example: + +```sh +./run.sh -run //appset +``` + +This command runs all tests related to ApplicationSet, covering both RBD and +CephFS PVC based applications. Useful when focusing on a specific deployer. + +#### Run tests using a specific storage + +Example: + +```sh +./run.sh -run //rbd +``` + +This command runs all tests related to RBD PVCs across all deployers. +Ideal for verifying functionality specific to a storage type. + +### List of tests + +The below test list is generated by executing all e2e tests using `./run.sh` +and capturing the output shown at the end of the run. + +```console +--- PASS: TestSuites (0.05s) + --- PASS: TestSuites/Validate (0.05s) + --- PASS: TestSuites/Validate/hub (0.02s) + --- PASS: TestSuites/Validate/c1 (0.01s) + --- PASS: TestSuites/Validate/c2 (0.01s) + --- PASS: TestSuites/Exhaustive (6.11s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox (425.72s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox/Deploy (5.17s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox/Enable (128.64s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox/Failover (165.50s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox/Relocate (90.24s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox/Disable (30.05s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-rbd-busybox/Undeploy (6.10s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox (434.67s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox/Deploy (3.12s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox/Enable (145.74s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox/Failover (98.98s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox/Relocate (138.90s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox/Disable (30.09s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-cephfs-busybox/Undeploy (17.83s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox (496.18s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox/Deploy (3.12s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox/Enable (135.70s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox/Failover (195.01s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox/Relocate (118.72s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox/Disable (25.07s) + --- PASS: TestSuites/Exhaustive/disapp-deploy-rbd-busybox/Undeploy (18.55s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox (525.85s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox/Deploy (5.17s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox/Enable (183.82s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox/Failover (140.29s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox/Relocate (160.37s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox/Disable (30.08s) + --- PASS: TestSuites/Exhaustive/subscr-deploy-cephfs-busybox/Undeploy (6.12s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox (536.09s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox/Deploy (0.25s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox/Enable (134.90s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox/Failover (145.46s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox/Relocate (225.37s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox/Disable (30.07s) + --- PASS: TestSuites/Exhaustive/appset-deploy-cephfs-busybox/Undeploy (0.04s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox (733.34s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox/Deploy (0.26s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox/Enable (100.43s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox/Failover (258.76s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox/Relocate (250.45s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox/Disable (123.42s) + --- PASS: TestSuites/Exhaustive/appset-deploy-rbd-busybox/Undeploy (0.03s) +``` + +### Using multiple config files + +Use this option if you want to maintain multiple configuration files and run +tests using a specific one. Example usage: + +```sh +./run.sh -config my_config.yaml +```