Experimental: add a proxy to increase CI resiliency to third party services failures

[apostasie]

CI fails regularly because of third-party services transient errors.

Most typically, debian or ubuntu servers, or Docker Hub, returning a 500 during the build phase.

This PR is an experimental proposal to alleviate a bit of the pain with this problem, by adding a local proxy that will retry backends requests on such failures.

The proxy also provisionally does caching for debian and ubuntu domains. Currently, this is not going to do much, although it is good practice IMHO to minimize hammering debian servers.

Besides this PR, we might want to rethink our strategy with building the test image though.
Right now, we build everything once per target.
This is happening in parallel, but this is (obviously) significantly increasing the chances of failures against these services.

Note that this proxy will intercept requests done from the host and from the build phase - not for the tests themselves.

[AkihiroSuda]

Besides this PR, we might want to rethink our strategy with building the test image though.

Can we just use https://docs.docker.com/build/cache/backends/gha/ ?
Then probably no need to set up a proxy

[apostasie]

Besides this PR, we might want to rethink our strategy with building the test image though.

Can we just use https://docs.docker.com/build/cache/backends/gha/ ? Then probably no need to set up a proxy

Will definitely give it a try. I am not optimistic that we will fit in the limitations (especially if we try to mode=max) - but let's see.

[apostasie]

Besides this PR, we might want to rethink our strategy with building the test image though.

Can we just use https://docs.docker.com/build/cache/backends/gha/ ? Then probably no need to set up a proxy

Will definitely give it a try. I am not optimistic that we will fit in the limitations (especially if we try to mode=max) - but let's see.

Notes for later:

• 10G is the limit - cache will get evicted once we hit that
• a PR can use cached entries from the main branch, but not from other PRs
• with mode=max, image cache is about 2G compressed - 2.9G uncompressed - with currently 8 images (rootless, rootful, containerd/ubuntu versions), this will not work - we have to rewrite the image build logic and cache a common target instead