feat: qatool to run webconnectivity lte QA #1394
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We're introducing failure modes that do not exist hence it seems this is not the correct way of moving forward.
I'm doing this mainly to explore whether we could have more robust webconnectivity v0.5 analysis code
Because I am dropping the requests again, we break again the tests with the redirects. I could possibly fix it by putting requests back again but I am not super happy about doing this because that would cause the DSL to do some strange work and I'd honestly rather not do this.
what remains to be done now is to make sure we make green all the tests that are currently skipped we also need to account for differences between the two
then next step is to sort out this mess :-)
(I am thankful there's a ~comprehensive test suite.)
this happens because LTE sucessfully handshakes with the wrong address
Murphy-OrangeMud
pushed a commit
to Murphy-OrangeMud/probe-cli
that referenced
this pull request
Feb 13, 2024
This diff introduces the qatool, originally developed in ooni#1392. This tool allows to run webconnectivitylte netemx-based QA tests and produces the measurement, observations, and analysis files. We can use this tool to run individual tests and inspect their results when they're failing. Additionally, we can use this tool to regenerate the test suite used by the minipipeline. When running the tool, we can select which QA tests to run, whether to rerun netemx and to reprocess. This feature is handy because we don't need to rerun netemx every time and sometimes we may only want to rerun netemx. This work is part of ooni/probe#2634. It helps us to manage and test the minipipeline and webconnectivitylte in an easier way, and it enables further development and testing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This diff introduces the qatool, originally developed in #1392. This tool allows to run webconnectivitylte netemx-based QA tests and produces the measurement, observations, and analysis files.
We can use this tool to run individual tests and inspect their results when they're failing. Additionally, we can use this tool to regenerate the test suite used by the minipipeline.
When running the tool, we can select which QA tests to run, whether to rerun netemx and to reprocess. This feature is handy because we don't need to rerun netemx every time and sometimes we may only want to rerun netemx.
This work is part of ooni/probe#2634. It helps us to manage and test the minipipeline and webconnectivitylte in an easier way, and it enables further development and testing.