Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pytest AssertionErrors more understandable #58

Open
lucmurer opened this issue Jan 19, 2022 · 0 comments
Open

Make pytest AssertionErrors more understandable #58

lucmurer opened this issue Jan 19, 2022 · 0 comments

Comments

@lucmurer
Copy link
Contributor

The pytest details delivered when a test fails are rather programmer-specific and need know-how of the internas of pytest. Depending on how the assertion condition is written, the assertion error details printed out by pytest will rather confuse the NUTS user instead of hinting him to the real problem.

Examples:

  • ARP-Range test: If the ARP table entries count is outside the range, pytest will print
    <min or max> != <actual>
    Expected :<min or max>
    Actual :<actual>
    This needs some interpretation of the NUTS user to comprehend which part of the test (min or max of the range) really failed.
  • ARP-Entries test: When using the assert <element> in <array> notation, pytest will print <element> != <array> as reason in the error message. For sure a single element does not equal the whole array, the hint that this results from a contains-check comes some lines further down, where the respective test function is printed. This confused even me a lot when I first saw this. I am not sure whether this is something pytest could improve, anyhow it highlights the point I want to make with this issue.
    For this specific test I hopefully improved the usefulness of the error message for NUTS users by using assert <array>.count(<element>) >= 1 which will print Expected: 1, Actual: 0 on error. This is probably better understandable from the test context.

Possible Solution:

There is the possibility to change how assertions are rewritten, as described in https://docs.pytest.org/en/6.2.x/assert.html#assertion-introspection-details and the links provided there. This could be a possibility to improve the error message the user gets printed on error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant