Skip to content

Latest commit

 

History

History
81 lines (69 loc) · 3.57 KB

notes.md

File metadata and controls

81 lines (69 loc) · 3.57 KB

Some notes while exploring the world of EOS charts

General notes:

  1. The app container should start with the binary to be executed
  • No preparation ==> Use init containers
  • No magic substitutions in config files ==> Use config maps // custom resources
  • Just run the binary or, worst case, supervisord if multiple processes are required
  1. To what extent we want internals to be configurable?
  • Example, should the path of QDB data directory be configurable?
  • This comment is valid for the environment inside the container. Outside, we should be able to use emptyDir, hostPath, persistentVolumeClaim, ...
  1. If stored on hostPath, we should come up with a norm to define the path of:
  • Configuration files (should this rather be configMaps or custom resources?)
  • Data directories (/var/k8s/data/)
  • Log directories (/var/k8s/logs/)

qdb

  1. Removed some old, commented-out, env vars from kuboxed
  2. Created init container to:
quarkdb-create --path /var/quarkdb/node-0
chown -R daemon:daemon /var/quarkdb/node-0
  1. Start the xrtood process for quarkdb when the container starts
  2. /etc/xrd.cf.quarkdb is now a configMap
  3. Logs go to an emptyDir

mq

  1. /etc/xrd.cf.mq is now a configMap
  2. /etc/sysconfig/eos is also a configmap. It is likely it must be shared across MQ, MGM, and FSTs. Check how to do this with an umbrella chart 2b. CORRECTION: /etc/sysconfig/eos are now all env vars so that we do not have to source. The problem of duplication across charts remains.
  3. Does it need to init somehow? . Other things (log dir to emptyDir, clean up, ...) similar to QDB

mgm

  1. /etc/xrd.cf.mgm is now a configMap
  2. /etc/sysconfig/eos are now all env vars so that we do not have to source. The proble m of duplication across charts remains. . Other things (log dir to emptyDir)

umbrella chart

  1. Need to define how to do sequencing across different pods and services
  2. Need to change the way FSTs register and boot. Can this be done by the FSTs themselves? What is required?
  • The eos.keytab should be shared as a secret across all the EOS components

Questions

  1. Who need /etc/sysconfig/eos? All EOS and not QuarkDB?
  2. xrootd binary. Can we have only one instead of /opt/eos/xrootd//bin/xroot || /usr/bin/xrootd? This seems to apply to all the EOS processes but not the xrootd for QuarkDB
  3. What if I want to share a configmap template across several charts?
  4. Should we use a well-defined location inside the container for config files which land there via configmaps? E.g., /var/k8s/config/etc/sysconfig/eos ?
    • Otherwise, I am happy (ugly) mounting over files that already exist

TODO

  1. Check and fix host names
  • They relate to the name generated by Helm
  1. All the configMaps should have values instead of being hard-coded
  • For shared params across charts (see umbrella chart)
  • For params which we would like to change
  • Decide whether to remove or not params which are commented out
  1. Umbrella chart:
  • Set how to choose variables in _helpers.toml
  • Set global values for variables used by multiple charts
  1. Set admin authentication method in EOS for config management (from other container) and FS registration 3b. Move all the other bits and pieces of config to a separate Pod // script
  1. See how to change FS registration so that it can be done by the FST itself
  2. Sequence: Init containers?
  3. Health-checks?
  4. Set eos.keytab as a secret
  • This should also be part of the umbrella chart