Added a menu to point to maven test bench project that run a daily check of memory allocation

[pcavezzan]

No description provided.

[slachiewicz]

Interesting, where we can found latest benchmarks for maven 3.8.1 vs 4.0 snapshot?

[jeanbisutti]

@slachiewicz This work from @pcavezzan allows having a daily non-regression check on Maven head heap allocation.

We did not execute the latest benchmarks for Maven 3.8.1 and 4.0 snapshot. It is possible to run them with the instructions given here.

[slachiewicz]

Thx for the effort, but adding a link to the static site will not help here.

We asked here quick-perf/maven-test-bench#18 to maybe add code to our tool to do perf tests and have our reports also - would it be possible to implement?

[jeanbisutti]

@slachiewicz
We asked here quick-perf/maven-test-bench#18 to maybe add code to our tool to do perf tests and have our reports also - would it be possible to implement?
quick-perf/maven-test-bench#18 proposes to "ingest daily Maven memory check to daily Maven dist-tool report", I don't see the mention of performance tests addition. From what I understand from maven-dist-tool Readme and code, it aims at producing a report.

The performance information is automatically generated each day in the following way:

• The Maven head is checkouted and a Maven distribution is built from it
• A specific version of Apache Camel is checkouted, a massive multi module project
• mvn validate is applied on this project, and the heap allocation is checked

Control and reduce heap allocation seem an important matter for Maven project. Indeed, a part of the heap allocation will be garbage collected and the garbage collection activity is succeptible to slow down the build. Also, less heap allocation means that we may execute Maven with a smaller heap size.

Having a daily check on Maven heap allocation may help the Maven development because:

• It would be possible to be comfortable that each new code modification does not lead to a significant increase of heap allocation, so it would not be needed to manually launch a heap bench of a new Maven distribution (for example with Maven 3.8.1 and 4.0 snasphot)
• Having a regular check would ease the identification of the code causing a significant increase of heap allocation

[pcavezzan]

Thank you @slachiewicz for your feedback 😀

Would you prefer to have the tests running alongside with maven-dist-tool ? Like @jeanbisutti said, from what we understood, maven-dist-tool was just about reporting and nothing else.

That's why we were thinking about with @jeanbisutti and @hboutemy to go step by step with this report about daily memory check of maven build:

1. add a simple badge to indicate if check passes or fails (this PR)
2. working on having something more useful for maven development like having a trend chart about the last x daily check.

What do you think about that ?

[hboutemy]

Thx for the effort, but adding a link to the static site will not help here.

It's not a static site: it's the result of the last check against Maven HEAD, and it's failing if the execution hits a threshold.

As it is currently, it's useful to detect if there is a bad move.

But, as a user, I don't only want "build passing" vs "build failing" on HEAD, I also want to see historical results, with the effective memory allocation value that each key past version has attained.

Patrice is working on this = storing values in a S3 bucket to have more that the last success/failure, but a time series of memory allocation values, which in the future can also be shown as a nice graph...

This PR is only the first step, the MVP, of a new type of performance tests: it's not yet the ideal dream we have, but it's what we currently can do in the right direction, while working on next steps