Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Negative mutation score improvement #78

Open
mabdi opened this issue Jun 13, 2020 · 1 comment
Open

Negative mutation score improvement #78

mabdi opened this issue Jun 13, 2020 · 1 comment
Labels
bug Something isn't working TODO

Comments

@mabdi
Copy link
Owner

mabdi commented Jun 13, 2020

In the evaluation, some classes show negative improvement.

Technically, it can not happen. Because in the worst case, we don't have any new test method to suggest, and mutation score shouldn't be changed.

TODO: check the reason. (flaky test?)

@mabdi mabdi added bug Something isn't working TODO labels Jun 13, 2020
@mabdi
Copy link
Owner Author

mabdi commented Jun 15, 2020

We see flakyness in generated tests:

(code from: project: zinc class ZnEntityReaderTest)

testReadStreaming_amp_L15
    <madeBySmallAmp>
    | data headers reader entity aCollection aCollection0 |
    aCollection0 := 1 to: 32.
    aCollection := aCollection0 collect: [ :each | (-1 to: 9) atRandom ].
    self should: [ data := aCollection asByteArray ] raise: Error

when at least one -1 presents in the aCollection we will have an error. if no -1 is presented (probability: (10/11)^32 ~= 0.047) we will have an error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working TODO
Projects
None yet
Development

No branches or pull requests

1 participant