Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-1728000 TestExecutor and TestSelector for e2e tests #963

Merged
merged 2 commits into from
Oct 17, 2024
Merged

SNOW-1728000 TestExecutor and TestSelector for e2e tests #963

merged 2 commits into from
Oct 17, 2024

Conversation

sfc-gh-mbobowski
Copy link
Contributor

@sfc-gh-mbobowski sfc-gh-mbobowski commented Oct 17, 2024 •
edited
Loading

Overview

SNOW-1728000

test_verify.py was so complicated that adding any new logic there seemed impossible to me. I decided to introduce the following abstractions:

  • TestSelector - which one and only role is to filter tests according to given parameters
  • TestExecutor - which one and only role is to run tests

Overall there is still huge space for improvement but this refactor makes cloud vendor specific filtering much easier.

Pre-review checklist

  • This change should be part of a Behavior Change Release. See go/behavior-change.
  • This change has passed Merge gate tests
  • Snowpipe Changes
  • Snowpipe Streaming Changes
  • This change is TEST-ONLY
  • This change is README/Javadocs only
  • This change is protected by a config parameter <PARAMETER_NAME> eg snowflake.ingestion.method.
    • Yes - Added end to end and Unit Tests.
    • No - Suggest why it is not param protected
  • Is his change protected by parameter <PARAMETER_NAME> on the server side?
    • The parameter/feature is not yet active in production (partial rollout or PrPr, see Changes for Unreleased Features and Fixes).
    • If there is an issue, it can be safely mitigated by turning the parameter off. This is also verified by a test (See go/ppp).

sfc-gh-mbobowski
sfc-gh-mbobowski commented Oct 17, 2024
View reviewed changes
test/test_selector.py
class TestSelector:

def selectTestsToBeRun(self, driver, nameSalt, schemaRegistryAddress, testPlatform, allowedTestsCsv):
test_suites = create_end_to_end_test_suites(driver, nameSalt, schemaRegistryAddress, testPlatform, allowedTestsCsv)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allowedTestCsv filtering will be moved here as well.

sfc-gh-mbobowski
sfc-gh-mbobowski commented Oct 17, 2024
View reviewed changes
test/test_selector.py
return [single_end_to_end_test.test_instance for single_end_to_end_test in test_suites.values() if single_end_to_end_test.run_in_apache == True]
elif testPlatform == "confluent":
return [single_end_to_end_test.test_instance for single_end_to_end_test in test_suites.values() if single_end_to_end_test.run_in_confluent == True]
elif testPlatform == "clean":
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This clean stuff is very wrong but separating it requires a lot of work.

@sfc-gh-mbobowski sfc-gh-mbobowski marked this pull request as ready for review October 17, 2024 14:31
@sfc-gh-mbobowski sfc-gh-mbobowski requested a review from a team as a code owner October 17, 2024 14:31
sfc-gh-achyzy
sfc-gh-achyzy approved these changes Oct 17, 2024
View reviewed changes
Copy link
Contributor

@sfc-gh-achyzy sfc-gh-achyzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sfc-gh-mbobowski sfc-gh-mbobowski merged commit bc8e824 into master Oct 17, 2024
74 of 78 checks passed
@sfc-gh-mbobowski sfc-gh-mbobowski deleted the mbobowski-SNOW--1728000-abstractions branch October 17, 2024 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sfc-gh-achyzy sfc-gh-achyzy sfc-gh-achyzy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sfc-gh-mbobowski @sfc-gh-achyzy