health: add liveness server #461
Conversation
roobre
force-pushed
the
liveness
branch
3 times, most recently
from
8bbb8f1
to
8727225
Compare
Probes do not work if they are listening on localhost
|
|
||
| {{- if not (include "nriKubernetes.controlPlane.hostNetwork" .) }} | ||
| ports: | ||
| - containerPort: 8999 |
There was a problem hiding this comment.
Would it worth it to make the health port configurable?
There was a problem hiding this comment.
I've started implementing this, but I'm finding UX concerns that are pretty hard to solve. The main one is that, right now, it's easy to expose the containerPort and to disable everything if hostNetwork is on. If we expose the config, that logic gets more complicated, especially if we want to override this setting depending on hostNetwork.
There was a problem hiding this comment.
I'm not a big fan of the hardcoded probe as it is now either... Any ideas, @kang-makes ?
There was a problem hiding this comment.
It should listen to localhost and it is disabled when hostNetwork is enabled so I have no big issues with hardcoding it.
As it could be set in the configuration file we could test in the .Values.common.config has a healthAddress and set it.
There was a problem hiding this comment.
Is there any other already configurable port which may collide if it is set to 8999? If we finally keep it hardcoded, that is the only cause for concern I can think of...
There was a problem hiding this comment.
There is no configurable port in our helm charts. We could configure LOTS of things so we timeboxed and we run out of time.
It is prepared for the port to be variable but right now it is hardcoded in helpers:
There was a problem hiding this comment.
I wanted to have preview but Github seems to be drunk (again)
Ports 8001, 8002, and 8003 are hardcoded right now.
Right now, it is not possible for Kubernetes to know the state of the integration. While the integration currently exits whenever an error occurs, before entering the main loop it will be still waiting for some initialization to complete.
This PR adds a health server that can be used for the application to expose whether it is healthy or not. Currently, the main command of the integration sets the server to Healthy after the first successful scrape.