Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky REM test / refactor #2464

Merged
merged 7 commits into from
Aug 15, 2024
Merged

Fix flaky REM test / refactor #2464

merged 7 commits into from
Aug 15, 2024

Conversation

natestemen
Copy link
Member

@natestemen natestemen commented Aug 9, 2024

Description

fixes #2431 by setting a random seed to ensure deterministic test.

I also took the liberty to refactor the function at hand which looked to be more complicated than needed.


cc @FarLab @jordandsullivan @bdg221 since we got this started in the mitiq coding call together.

Copy link

codecov bot commented Aug 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.70%. Comparing base (8e0d5be) to head (65bef0a).
Report is 7 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2464   +/-   ##
=======================================
  Coverage   98.70%   98.70%           
=======================================
  Files          88       88           
  Lines        4083     4091    +8     
=======================================
+ Hits         4030     4038    +8     
  Misses         53       53           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@cosenal cosenal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, first PR in a while on which we don't have to override build failures 😄

abs_tol=0.1,
)
@pytest.mark.parametrize("repeats", range(10))
def test_probability_vector_roundtrip(repeats):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: To make it clear that the actual value of repeats is not actually important in the test body, you can instead use _ as argument.

probability_vector: npt.NDArray[np.float64], samples: int
) -> List[Bitstring]:
probability_vector: Sequence[float], samples: int
) -> list[str]:
"""Generate a number of samples from a probability distribution as
bitstrings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be useful to add an example of input --> output for our future selves.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering this left us a bit confused when first encountering it, I think that's a great idea. Done in f8725eb.

probability_vector: npt.NDArray[np.float64], samples: int
) -> List[Bitstring]:
probability_vector: Sequence[float], samples: int
) -> list[str]:
Copy link
Contributor

@purva-thakre purva-thakre Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious as to what's usually preferred when we specify the return type?

I see you have used list[str] here but you have also used from typing import List before specifying the return type as List[str] on another line.

Searching around for this shows that we do not need to use the typing module for >= Python 3.9. Do we need to add this as a to-do for other modules in mitiq?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we have this exact same discussion with @vprusso in another PR? iirc there we learned that list[str] is the way to go.

Copy link
Contributor

@purva-thakre purva-thakre Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we did. Which is why I was asking if we need to add this to our todos.

We still use things like from typing import List elsewhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to be so proactive. The aliases are deprecated, but they are unlikely to be removed in the foreseeable future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we should not take action until we need to on this. In the meantime lets try to use the builtins instead of typing.List, but it's not a big concern since they both achieve the same goal.

If we do migrate eventually, we will have to import a large majority of the types from collections.abc since many of the types we rely on (e.g. typing.Sequence) are deprecated in favor of coming from collections.abc.

@natestemen natestemen merged commit 6c73a09 into main Aug 15, 2024
18 checks passed
@natestemen natestemen deleted the flaky-rem-tests branch August 15, 2024 13:41
@purva-thakre purva-thakre mentioned this pull request Aug 16, 2024
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky test in rem/tests/test_inverse_confusion_matrix.py
3 participants