Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Snapshot testing framework reports xfail tests as failing. #2282

Closed
rodrigogiraoserrao opened this issue Apr 13, 2023 · 3 comments
Closed

Snapshot testing framework reports xfail tests as failing. #2282

rodrigogiraoserrao opened this issue Apr 13, 2023 · 3 comments
Labels
enhancement New feature or request question Further information is requested

Comments

@rodrigogiraoserrao
Copy link
Contributor

When a snapshot test is marked as xfail, pytest understands the test was expected to fail but the framework still reports a failing test.

What I expected:

  • A yellow notice saying that 1 snapshot x-failed instead of a red notice saying 1 failed.
  • As for the snapshot report, it being generated is probably still useful.

This makes it look like the tests failed:

Screenshot 2023-04-13 at 14 59 29

But pytest is perfectly happy with an xfail:

Screenshot 2023-04-13 at 14 59 45
@rodrigogiraoserrao rodrigogiraoserrao added enhancement New feature or request question Further information is requested labels Apr 13, 2023
@rodrigogiraoserrao
Copy link
Contributor Author

After talking to @darrenburns this might be a bug/design decision of syrupy.

@rodrigogiraoserrao rodrigogiraoserrao mentioned this issue Apr 13, 2023
Closed
rodrigogiraoserrao added a commit that referenced this issue Oct 5, 2023
@rodrigogiraoserrao
Don't use xfail because of #2282.
b5a11ae
rodrigogiraoserrao added a commit that referenced this issue Oct 5, 2023
@rodrigogiraoserrao
Revert "Don't use xfail because of #2282."
5d61678 
This reverts commit b5a11ae.
@willmcgugan
Copy link
Collaborator

Looks like this is fixed upstream.

@github-actions GitHub Actions
Copy link

Don't forget to star the repository!

Follow @textualizeio for Textual updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@willmcgugan @rodrigogiraoserrao