Use gitlab's multi-project pipelines to trigger agent build

[iliakur]

What does this PR do?

Before

We curled gitlab API in 2 jobs. The token for those expired, so they broke.

After

We use gitlab's built-in multi-project pipelines.
We can see the status of the agent build in the dd-gitlab/default-pipeline "action". See screenshot below:

Motivation

No need for a separate token, better retries. Clearer Pipeline UI.

Thanks to @BaptisteLalanne for all the support!

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.93%. Comparing base (f3966d5) to head (fe9cdd8).
Report is 13 commits behind head on master.

Additional details and impacted files

Flag	Coverage Δ
activemq	?
cassandra	?
hive	?
hivemq	?
hudi	?
ignite	?
jboss_wildfly	?
kafka	?
presto	?
solr	?

Flags with carried forward coverage won't be shown. Click here to find out more.

[BaptisteLalanne]

👏 praise: ‏
Truly lovely to see all those polling scripts removed !

LGTM on GitLab pipeline definition 💯
Small question on the manually triggered pipeline in case you want to always have it

[BaptisteLalanne]

✅