Should we run the Precompiles tests once weekly?

[PaulRBerg]

One of the most common reasons for git conflicts and CI failure nowadays are the Precompiles tests failing:

https://github.com/sablier-labs/v2-core/blob/36223aeb1867b2fc197978fca19cbd5fcc538914/test/utils/Precompiles.t.sol

@andreivladbrg, what would you say about refactoring the CI workflows to only run the Precompiles tests once weekly, just like the CI Deep workflow?

[PaulRBerg]

any thoughts about this, @andreivladbrg?

[andreivladbrg]

it sounds good, but it solves only half of the problem, git conflicts would remain, which imo are the bigger problem here

[PaulRBerg]

good point

[PaulRBerg]

Any additional thoughts on this, @andreivladbrg, @smol-ninja?

I think that disabling the utils CI checks would minimize the error rates in PRs.

Maybe we could incorporate the utils tests in the daily cron schedule for fork tests.

[andreivladbrg]

so that it is not run on every push/ PR

yes, but don't see how is it related to this "minimize the error rates in PRs"

I propose to run it only once every ~24 hours or so. Not on every push to main

yes, periodically every 24 hours, but on branch main, not for example on staging

[PaulRBerg]

but don't see how is it related to this "minimize the error rates in PRs"

The test-utils job will not run anymore, leading to fewer failed jobs in CI PR reviews.

We will prevent this:

periodically every 24 hours, but on branch main, not for example on staging

Yes.

[andreivladbrg]

We will prevent this:

yeah, but this is not an "error", nvm, wording game

we agree

[PaulRBerg]

wording game

Yes. "error", "failure", whatever.

[smol-ninja]

I am also in favour of removing the test-utils job from every PR and only running on the main branch periodically.

And what about fork tests? For example, when a PR in v2-periphery depends on new changes in v2-core, the fork test will always fail until we deploy the v2-core on the mainnet. So I agree with @andreivladbrg that it should also run only on the main.

[smol-ninja]

Since we all have agreed to move precompile tests alongside the fork tests schedule, should I close this discussion and create an issue out of it?

[PaulRBerg]

Yes

[smol-ninja]

Closing it.

• Run precompiles tests along with fork tests periodically #832
• Run precompiles tests along with fork tests periodically v2-periphery#296

[smol-ninja]

I would suggest that we should bring the fork tests back into ci with less number of runs (~ 20). During PR reviews, it would be helpful to see fork tests passing with the new changes. What's your opinions?

[PaulRBerg]

Agree