CLOUDP-272039: Fail when no tests are run

[josvazg]

CLOUDP-272039

Detect and fail when a test suite did not run any tests. Then:

• deletion-protection test was not running because the test suite was removed at CLOUDP-229283: disable ownership detection #1371
• helm-update was not running because of a mistake comparing the major versions.
• helm-update was failing because we had to pass the pull secret name at upgrade time.

All Submissions:

• Have you signed our CLA?

[github-actions]

https://app.codecov.io/github/mongodb/mongodb-atlas-kubernetes/commit/747f5cebc0be2cd01fc5f5c772a71c5d2811c92a

[s-urbaniak]

I am a bit concerned with the repetition of this line across many scripts below.

[josvazg]

The CI should be running the same code as the local dev, but it does not. That is why this fix needs so much repetition.

Fixing the duplicity is a bigger issue than the focus of this PR. It has already been flagged before but not prioritised.

[helderjs]

@josvazg I took a quick look on Ginkgo and we can reduce duplication by adding the following code to the Ginkgo suites we have.

var _ = ReportAfterSuite("Ensure test suite was not empty", func(r Report) {
	Expect(r.SuiteSucceeded && r.PreRunStats.SpecsThatWillRun > 0).To(BeTrue(), "Suite must run at least 1 test in order to be valid")
})

What do you think?

[josvazg]

Uhm, yes I think that would be better.

There is a problem, when running locally we are calling the int tests and int/clusterwide somewhat separately, so we might run a label that only runs on one of them and the other should not complain. There is probably another solution for it, but need to figure that out.

In the meantime, I need to understand the helm-update test failure, which only happens in the CI for some reason.

[josvazg]

Applied this suggestion already.

[helderjs]

Are these debugging bits? if not, could you clarify how it supports the second status check (line 138)?

[josvazg]

yes, I am debugging, please ignore these for the time being.

I will mark as draft and iterate in the helm-update error first, then will try to move to check the issues within ginkgo instead of from a script, as you suggested above.

[josvazg]

Removed the traces, thinking about a different way to debug

[josvazg]

Debugging helm-update e2e test issue.

[josvazg]

After adding traces:
https://github.com/mongodb/mongodb-atlas-kubernetes/actions/runs/10808820530/job/29983049730?pr=1809

  failed to get pods: failed to get deployment: deployments.apps "mongodb-atlas-operator" not found
  failed to get stream: container "manager" in pod "mongodb-atlas-operator-6f76cd65d9-8kbkh" is waiting to start: image can't be pulled

[josvazg]

98b1cab
That worked:
https://github.com/mongodb/mongodb-atlas-kubernetes/actions/runs/10811663472/job/29992054452?pr=1809

[josvazg]

Easier review guide:

ab3c717 Forces failure if tests are skipped. It also fixed the int local script so it invoked clusterwide when using a cluster wide label

2ad192f Removes a non existing test tag deletion-protection that was removed some time ago

9b8d614 Fixes an issue comparing major versions that was skipping the helm-update e2e test.

0504f1f Ensures e2e test helm-update uses the correct pull secret when upgrading the helm chart.

de2f156 Improves the logging for helm-update which was needed to actually debug and understand the failure.

747f5ce Makes the helm upgrade an atomic operation which makes it more reliable and also would dump the error in line if it fails.

[roothorp]

Walked through in zoom together