DRAFT: Migrate to tilt and k8s (WIP WIP WIP)

[n1koo]

Up for sharing - not for review or merging yet.

What

Migrate from Docker Compose and Docker Swarm to a Kubernetes based infrastrcture. Also leverages https://tilt.dev/ for local coordination as many of the convenient things to orchestrate local environment(s) doesn't exist in Kubernetes natively.

Tech used

• Kompose to autogenerate k8s templates from docker compose
• Tilt to manage local envs (and probably also the remote/prod ones)
• Helm to template k8s templates for values
• Docker Desktop for local k8s cluster (potentially later evaluating orbstack)

TODO

• Finish off dependencies between builds and deployment in Tiltfile
• Make all envvars that need to be customised actually templated variables for Helm
• Fix all network addresses to k8s ones (currently direct docker compose names)
• Add resource settings and other basic k8s things
• k8s hardening via networkpols etc basics (only basics at this point)
• Fix the builds requiring nginx configs from outside packages
• allow_k8s_contexts doesnt seem to enforce/work?
• Test, test, test

From #6215

[github-actions]

Oops! Looks like you forgot to update the changelog. When updating CHANGELOG.md, please consider the following:

• Changelog is read by country implementors who might not always be familiar with all technical details of OpenCRVS. Keep language high-level, user friendly and avoid technical references to internals.
• Answer "What's new?", "Why was the change made?" and "Why should I care?" for each change.
• If it's a breaking change, include a migration guide answering "What do I need to do to upgrade?".