chore: add steps to run upgrade e2e #1218

latin-panda · 2023-10-27T08:29:20Z

Description

Adds steps to run upgrade e2e with some troubleshooting steps.

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

latin-panda · 2023-10-27T08:34:17Z

Hi @lorerod and @tatilepizs can any of you please try these steps and provide feedback? :) Thanks!

lorerod · 2023-11-01T12:41:46Z

content/en/contribute/code/core/fixing-e2e-tests.md

+  - A way to do this is by pushing the branch, let the GitHubActions to run, if all the other e2e as okay, then it will publish the branch.
+  - Check that your branch name is available [here](https://staging.dev.medicmobile.org/_couch/builds_4/_design/builds/_view/releases).
+- Make sure to stop all existing containers
+- Run CHT as usual:


What about couchdb? with what config? I used this.
Maybe it is better to reference dev environment setup

lorerod · 2023-11-01T17:58:08Z

content/en/contribute/code/core/fixing-e2e-tests.md

+```
+Try the following:
+- Make sure to stop all existing containers, because maybe the Nginx port was already allocated and it couldn't start.
+- If you keep getting errors, check that the Nginx ports are available.


I keep getting the error:
Error in hook: StatusCodeError: 404 - "{\"error\":\"not_found\",\"reason\":\"Document is missing attachment\"}\n"
What did you do to fix it?

I ran wdio-local before and that fixed it but I don't think that's the proper way.
@tatilepizs is also getting the same error

@dianabarsan, what other troubleshooting steps can they follow here?

tatilepizs

It is not working for me @latin-panda.

First I tried following the exact same steps that you add, and to able to have a CHT instance up and running I needed a container running. I saw that error that you point, so I stoped the container and tried to run the command without an cht instance running, but I am still seeing the same error.

Images with cht instance up and running, one container

Images without a cht instance.

content/en/contribute/code/core/fixing-e2e-tests.md

tatilepizs · 2023-11-01T18:08:05Z

content/en/contribute/code/core/fixing-e2e-tests.md

+- Make sure your branch has been published and it's available in the market:
+  - A way to do this is by pushing the branch, let the GitHubActions to run, if all the other e2e as okay, then it will publish the branch.
+  - Check that your branch name is available [here](https://staging.dev.medicmobile.org/_couch/builds_4/_design/builds/_view/releases).
+- Make sure to stop all existing containers


What do you mean with this step? Because you need the cht-docker container to be able to run cht-core.
Maybe you could add the link to the CouchDB Setup and say that only that container should be running.

tatilepizs · 2023-11-01T18:09:59Z

content/en/contribute/code/core/fixing-e2e-tests.md

+- Run CHT as usual:
+  - `npm run build-dev-watch`
+  - `npm run dev-api`
+  - `npm run dev-sentinel`


Could you please add the link to run the CHT? It will be easier in case someone needs more help

I think this steps are not needed. npm run upgrade-wdio should set up everything.

tatilepizs · 2023-11-01T20:55:06Z

content/en/contribute/code/core/fixing-e2e-tests.md

+  - `export MARKET_URL_READ=https://staging.dev.medicmobile.org`
+  - `export STAGING_SERVER=_couch/builds`
+  - `export BRANCH=<your branch name>`
+- Run the upgrade e2e tests: `npm run upgrade-wdio`


Could you please add where should that command be run? I assumed it was in the cht-core/ root level, but it would be a good idea to confirm it.

It took some debugging but the correct STAGIG_SERVER for 4.x releases is _couch/builds_4
While for anything previous 4.x the value should be _couch/builds

tatilepizs · 2023-11-13T19:46:43Z

Hi @latin-panda, @dianabarsan, @garethbowen,

Do you have any update about how can we run the upgrade e2e tests?
@lorerod has been trying to fix the upgrade.wdio-spec.js test and I am trying to help, but it is really hard to do it if we cannot run that test locally, we are going blind pushing commits in the CI to see if we have luck 😅

Could someone please help us with steps that we can follow to run the upgrade tests locally?

garethbowen · 2023-11-13T20:36:38Z

@tatilepizs Sorry I'm flat out right now. @dianabarsan is the expert on this test and she's back tomorrow so should be able to help then.

latin-panda · 2023-11-14T11:59:30Z

@tatilepizs @garethbowen
I had a work session with @lorerod and we both figured out a way to run these tests in the local env. It's not the proper way but it works, and enough for now to unblock. I'm going to update the docs shortly, and we can improve them in a second iteration.

…dding_steps_to_run_upgrade_e2e # Conflicts: # content/en/contribute/code/core/fixing-e2e-tests.md

latin-panda · 2023-11-14T15:46:25Z

content/en/contribute/code/core/automated-tests.md

+      - To upgrade from previous versions of CHT use: `export STAGING_SERVER=_couch/builds`.
+    - `export BRANCH=<your branch name>`.
+- Start the containers for e2e tests like this
+  - TBD


To start the container, for now, we need to add this:

@lorerod FYI

@dianabarsan pinging you on this again.
These changes allow the upgrade tests in the local environment to run, but they still fail while in the master's CI the pass. This probably isn't the correct way to run them.

[0-0] AssertionError in "Performing an upgrade.should have an upgrade_log after installing" AssertionError: expected +0 to equal 1 at Context.<anonymous> (/Users/tlepiz/Documents/Medic/Repos/PRs/cht-core/tests/e2e/upgrade/upgrade.wdio-spec.js:82:28) at processTicksAndRejections (node:internal/process/task_queues:95:5) [0-0] Error in "Performing an upgrade.should upgrade to current branch" Error: element ("div*=Deployment complete") still not displayed after 15000ms at async Context.<anonymous> (/Users/tlepiz/Documents/Medic/Repos/PRs/cht-core/tests/e2e/upgrade/upgrade.wdio-spec.js:103:5)

Video from Tatiana

@tatilepizs @lorerod and I are thinking that we're missing some setup. The QA Team wants to learn and document how to run these tests in their local environment to debug whenever needed.

What are we missing here? What is the best way to run them in the local environment?

I went over the upgrade e2e test locally and everything worked. I made no changes to the upgrade e2e test, as suggested above, and those changes should not be necessary.

My experience was:

I needed to stop existing containers that cause port collisions

on slow internet, the tests failed because the images weren't downloaded in sufficient time, this just meant I had to restart the test a bunch of times until the images were downloaded

I had to checkout and run the tests over the branch I wanted to upgrade to. This is mainly necessary to compute the correct version within the test.

@dianabarsan can you please share the exact steps that you are doing to run the tests?
I still need to run npm run wdio-local to have all the containers ready and then run npm run upgrade-wdio and it still fails 😞

You pointed out that maybe on slow internet it will fail, how fast my internet needs to be, maybe my internet is too slow 😅. Is there a way to check if the images started the download process but they didn't finish? Because I ran the test 10 times and it is still failing. How many times do you think that it would require to be run to download all the images successfully?

Screen.Recording.2023-11-17.at.11.32.17.am.mov

Can you change that path locally here: https://github.com/medic/cht-core/blob/b8519be2e415deaf20137c80ce55d9257321cff3/tests/e2e/upgrade/wdio.conf.js#L64 and try again?

DOCKER_CONFIG_PATH: os.homedir(),

Diana!!! 🤯

I just changed it to an empty string (because that is the value of $homeDir) -> DOCKER_CONFIG_PATH: '',

Cool! So happy we got to the bottom of this! Thanks for being patient.
I believe we should just change the code to take this value from an environment variable. I'll try to push a change shortly, and ask you to try the upgrade again (in a few hours) and then we can document the new environment variable.

Perfect!
Thank you so much for all the help!! 🌟

@tatilepizs @dianabarsan it works!! Thank you:
For me it worked changing to DOCKER_CONFIG_PATH: os.homedir(),
I tested manually downloaded the images and without the images.

dianabarsan · 2023-11-17T10:09:20Z

content/en/contribute/code/core/automated-tests.md

+    - `export MARKET_URL_READ=https://staging.dev.medicmobile.org`.
+    - `export STAGING_SERVER=<builds path>`.
+      - To upgrade from CHT v4.x.x use `export STAGING_SERVER=_couch/builds_4`.
+      - To upgrade from previous versions of CHT use: `export STAGING_SERVER=_couch/builds`.


I don't believe you can upgrade from previous 3.x versions, because those do not have docker compose files. We should remove this line.

dianabarsan · 2023-11-17T10:58:54Z

content/en/contribute/code/core/automated-tests.md

+      - To upgrade from previous versions of CHT use: `export STAGING_SERVER=_couch/builds`.
+    - `export BRANCH=<your branch name>`.
+- Start the containers for e2e tests like this
+  - TBD


I went over the upgrade e2e test locally and everything worked. I made no changes to the upgrade e2e test, as suggested above, and those changes should not be necessary.

My experience was:

I needed to stop existing containers that cause port collisions

on slow internet, the tests failed because the images weren't downloaded in sufficient time, this just meant I had to restart the test a bunch of times until the images were downloaded

I had to checkout and run the tests over the branch I wanted to upgrade to. This is mainly necessary to compute the correct version within the test.

lorerod · 2023-12-04T17:09:56Z

@tatilepizs @dianabarsan can you please re-review this PR? Since the issue was fixed in chore(#8727): change docker socket path in upgrade e2e test.

dianabarsan

This is great!

tatilepizs

Thank you @lorerod for updating the steps.

latin-panda added 8 commits August 31, 2023 13:13

Merge branch 'main' of https://github.com/medic/cht-docs

4078f9e

Merge branch 'main' of https://github.com/medic/cht-docs

bdc98df

Merge branch 'main' of https://github.com/medic/cht-docs

55db2ce

Merge branch 'main' of https://github.com/medic/cht-docs

7966c96

Steps

86a721a

Steps

330dacf

Steps

be8b5a3

cleaning

4794fc9

latin-panda requested review from lorerod and tatilepizs October 27, 2023 08:33

lorerod requested changes Nov 1, 2023

View reviewed changes

tatilepizs requested changes Nov 1, 2023

View reviewed changes

latin-panda added 3 commits November 14, 2023 21:26

Merge branch 'main' of https://github.com/medic/cht-docs into chore_a…

409122e

…dding_steps_to_run_upgrade_e2e # Conflicts: # content/en/contribute/code/core/fixing-e2e-tests.md

feedback

7bc862d

style

1485c26

latin-panda commented Nov 14, 2023

View reviewed changes

dianabarsan requested changes Nov 17, 2023

View reviewed changes

Add changes to address feedback

7aa347d

lorerod requested review from dianabarsan and tatilepizs December 4, 2023 17:06

dianabarsan approved these changes Dec 4, 2023

View reviewed changes

tatilepizs approved these changes Dec 4, 2023

View reviewed changes

lorerod self-requested a review December 4, 2023 19:30

lorerod approved these changes Dec 4, 2023

View reviewed changes

Merge branch 'main' into chore_adding_steps_to_run_upgrade_e2e

7ab40b8

lorerod merged commit 70f527f into main Dec 4, 2023
2 checks passed

lorerod deleted the chore_adding_steps_to_run_upgrade_e2e branch December 4, 2023 20:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: add steps to run upgrade e2e #1218

chore: add steps to run upgrade e2e #1218

latin-panda commented Oct 27, 2023

latin-panda commented Oct 27, 2023

lorerod Nov 1, 2023

lorerod Nov 1, 2023

latin-panda Nov 3, 2023 •

edited

Loading

tatilepizs left a comment

tatilepizs Nov 1, 2023

tatilepizs Nov 1, 2023

lorerod Nov 13, 2023

tatilepizs Nov 1, 2023

latin-panda Nov 14, 2023

tatilepizs commented Nov 13, 2023

garethbowen commented Nov 13, 2023

latin-panda commented Nov 14, 2023

latin-panda Nov 14, 2023

latin-panda Nov 14, 2023

latin-panda Nov 17, 2023

dianabarsan Nov 17, 2023

tatilepizs Nov 17, 2023

dianabarsan Nov 23, 2023 •

edited

Loading

tatilepizs Nov 23, 2023

dianabarsan Nov 23, 2023

tatilepizs Nov 23, 2023

lorerod Nov 23, 2023

dianabarsan Nov 17, 2023 •

edited

Loading

dianabarsan Nov 17, 2023

lorerod commented Dec 4, 2023

dianabarsan left a comment

tatilepizs left a comment

chore: add steps to run upgrade e2e #1218

chore: add steps to run upgrade e2e #1218

Conversation

latin-panda commented Oct 27, 2023

Description

License

latin-panda commented Oct 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

latin-panda Nov 3, 2023 • edited Loading

Choose a reason for hiding this comment

tatilepizs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tatilepizs commented Nov 13, 2023

garethbowen commented Nov 13, 2023

latin-panda commented Nov 14, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dianabarsan Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dianabarsan Nov 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lorerod commented Dec 4, 2023

dianabarsan left a comment

Choose a reason for hiding this comment

tatilepizs left a comment

Choose a reason for hiding this comment

latin-panda Nov 3, 2023 •

edited

Loading

dianabarsan Nov 23, 2023 •

edited

Loading

dianabarsan Nov 17, 2023 •

edited

Loading