[Idea] Correllate test success rate with presence and version of other dependencies

[foucdeg]

First of all, congrats for this great feature. I've had an idea that might take it one step further.

In my experience, I've had (minor or major) version upgrades failing because I needed to upgrade not just one but multiple dependencies at the same time, and sometimes, wait for Package B to release a version compatible with Package A's new version. Figuring that out takes a bit of time.

It seems that with the data you have, you may be able to find correlations between a release's test results and the presence and version of other dependencies, e.g.:

• Upgrading to react v17 from v16 has a 80% test success rate
• Among repos also using react-dom v16, this upgrade has a 20% success rate
• Among repos also using react-dom v17, this upgrade has a 98% success rate
• We conclude that the upgrade to react v17 is successful when it is made along with react-dom v17, and we can help developers upgrade by informing them of that correlation.

This is a fictional example since react is a monorepo, but you probably get the idea.

[rarkins]

Thanks, this is definitely the type of thing we'd aim to support one day.

We support the idea of "monorepo" grouping today, most of the ones we have either must or should be upgraded together at the same time. I think in a future Renovate release we will default to grouping based on source repo instead of dependency name, which will mean less work needed to manually add monorepo definitions.

In my opinion, if a library X can't be updated unless it's updated at the same time as another library Y, there's probably a good chance they should always be updated together.

I am also intrigued by the idea of combining what we can in an automated way with manual/crowd-sourced data too, e.g. via an open source repo or OAuth app. So e.g in a case like you describe, if we can't detect this automatically then the user base is hopefully willing to help each other out and provide manual hints too.

A good way to start may be to display the type of info you suggest on a web page, so e.g. someone browsing it in your example could see the relation between react and react-dom. Then as a second step to try to automate that analysis.

BTW something relevant to this issue is #2

[foucdeg]

So the first version would let us look up statistics to find out whether there is a correlation between a specific upgrade and some other package?

Yes that would be helpful, especially if I can filter on packages that my app uses. And later, since Renovate also has that information, if it detects a high correlation, it could "push" the information to me (e.g. in the PR description).

[foucdeg]

Along with that, It would be great to be able to tell renovate to "close PR until" some other package has made a release.