[Tech] Workaround for flaky tests

[mgoworko]

Describe your changes

Checklist before merge

• Related issue ID is placed at the beginning of PR title in [brackets] (can be GH issue or Nu Jira issue)
• Code is cleaned from temporary changes and commented out lines
• Parts of the code that are not easy to understand are documented in the code
• Changes are covered by automated tests
• Showcase in dev-application.conf added to demonstrate the feature
• Documentation added or updated
• Added entry in Changelog.md describing the change from the perspective of a public distribution user
• Added MigrationGuide.md entry in the appropriate subcategory if introducing a breaking change
• Verify that PR will be squashed during merge