Add var generation benchmark

[swiatekm]

What does this PR do?

Add a benchmark for variable generation from provider mappings. We specifically profile having a lot of Pods from the Kubernetes provider, which is something that has recently become a problem, and which the configuration generation pipeline doesn't deal with very well. I used test data from a real environment where the problems outlined in #5991 and #5835 could be reproduced.

Results on main:

goos: linux
goarch: amd64
pkg: github.com/elastic/elastic-agent/internal/pkg/composable
cpu: 13th Gen Intel(R) Core(TM) i7-13700H
BenchmarkGenerateVars100Pods-20    	      74	  15310045 ns/op	10527199 B/op	  201283 allocs/op
BenchmarkGenerateVars100Pods-20    	      74	  15185648 ns/op	10514404 B/op	  201271 allocs/op
BenchmarkGenerateVars100Pods-20    	      86	  15126049 ns/op	10521262 B/op	  201277 allocs/op
BenchmarkGenerateVars100Pods-20    	      87	  15067870 ns/op	10514019 B/op	  201275 allocs/op
BenchmarkGenerateVars100Pods-20    	      85	  15032633 ns/op	10515568 B/op	  201274 allocs/op
BenchmarkGenerateVars100Pods-20    	      90	  15185170 ns/op	10525087 B/op	  201282 allocs/op
BenchmarkGenerateVars100Pods-20    	      75	  15345854 ns/op	10516840 B/op	  201274 allocs/op
BenchmarkGenerateVars100Pods-20    	      80	  15908398 ns/op	10518576 B/op	  201277 allocs/op
BenchmarkGenerateVars100Pods-20    	      73	  15623320 ns/op	10529545 B/op	  201284 allocs/op
BenchmarkGenerateVars100Pods-20    	      86	  15946078 ns/op	10533571 B/op	  201286 allocs/op
PASS
ok  	github.com/elastic/elastic-agent/internal/pkg/composable	12.832s

Why is it important?

I'd like to make some improvements to both the var generation and the configuration generation in the coordinator, and want a straightforward way of demonstrating their impact.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas

Related issues

• Relates Elastic agent uses too much memory per Pod in k8s #5835
• Relates Memory Consumption Issue with Elastic Agent on Kubernetes with high number of resources #5991

[mergify]

This pull request does not have a backport label. Could you fix it @swiatekm? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

backport-v8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[blakerouse]

Nice!

Really like seeing ways to quantify the improvements.

[pchila]

Looks good.
I imagine we are going to track the results of this benchmark in PRs aimed to improve k8s variable performance but do we have plans to track changes to the benchmark results for the long term ? Are we going to rely on CI to keep track of these results ?

[swiatekm]

Looks good. I imagine we are going to track the results of this benchmark in PRs aimed to improve k8s variable performance but do we have plans to track changes to the benchmark results for the long term ? Are we going to rely on CI to keep track of these results ?

My impression is that this code changes infrequently enough that this wouldn't give us much benefit vs the work necessary, unless we already have a benchmark result tracking solution that would be easy to use. I was mostly hoping that whenever someone makes a change to var generation, they could run this manually.

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[swiatekm]

I thought I could avoid backporting this, but my other changes in #6058 and #6114 depend on it, so I'll have to do it.