The design of xCAT configuration data version control

Taget

1. 3 levels of object abstraction

   • object definition (object definition in xCAT DB)
   • object data (object definition + configuration files referenced in object definition)
   • object entity (object data + object backend dirs/files + relationships with other objects + xCAT generated config files )

   As the first step, the "object definition" and "object data" level will be first covered.

2. all the cluster configuration data in xCAT can be exported/imported as objects in yaml & json format

   • the configuration files should be representation of objects, the schema of objects should be well defined

     • include all KEY information in xCAT DB
     • objects should be loosely coupled
     • suitable for manipulation with RESTful API and representation on GUI

   • the object schema can be configurable

   • object attributes should only contain raw data in DB, without any REGEX translation or group attributes inheritance

   • the configuration files can be used as the inventory file for configuration management tool (Ansible)

3. the config files in "object data" include:

   • node object: postscripts, postbootscripts
   • osimage object: synclists, postscripts, postbootscripts, template, pkglist,otherpkglist, exlist, postinstall, partitionfile

Plan

1. object abstraction
   • figure out the key xCAT objects
   • determine the schemas for the xCAT objects
2. support the mapping between object definitions and tables in xCAT DB
   • ORM: SQLALchemy<==>class for table<= object schema =>class for object
3. determine the format and layout of object persistent files, need to consider
   • json/yaml
   • interaction with configuration management tools(Ansible: https://github.com/chenglch/xcat-play/tree/master/ansible/xcat2)
4. support the persistence of object definitions in yaml/json format
5. support import of persisted object definitions
6. support configurable object schema
7. support the persistency of object data
8. support the "object entity" (TBD)

Progress

1. An summary of current xCAT objects and Tables

there are 72 tables in xCAT

auditlog bootparams boottarget cfgmgt chain deps discoverydata domain eventlog firmware hosts hwinv hypervisor ipmi iscsi kit kitcomponent kitrepo kvm_masterdata kvm_nodedata linuximage litefile litetree mac mic monitoring monsetting mp mpa networks nics nimimage nodegroup nodehm nodelist nodepos noderes nodetype notification openbmc osdistro osdistroupdate osimage passwd pdu pduoutlet performance policy postscripts ppc ppcdirect ppchcp prescripts prodkey rack routes servicenode site statelite storage switch switches taskstate token virtsd vm vmmaster vpd websrv winimage zone zvm

59 of them are abstracted to or referred in xCAT objects

auditlog bootparams boottarget cfgmgt chain domain eventlog firmware hosts hwinv hypervisor ipmi iscsi kit kitcomponent kitrepo linuximage mac mic monitoring mp mpa networks nics nimimage nodegroup nodehm nodelist nodepos noderes nodetype notification openbmc osdistro osdistroupdate osimage pdu pduoutlet policy postscripts ppc ppcdirect ppchcp prescripts prodkey rack routes servicenode site storage switch switches taskstate vm vpd websrv winimage zone zvm

the following 13 tables are not related to any object:

< deps < discoverydata < kvm_masterdata < kvm_nodedata < litefile < litetree < monsetting < passwd < performance < statelite < token < virtsd < vmmaster

2. A draft of xCAT objects available for version control

-auditlog-

boottarget

-eventlog-

firmware

group

kit

kitcomponent

kitrepo

-monitoring-

network(need enhance)

node(need enhance)

-notification-

osdistro

osdistroupdate

osimage(need enhance)

pdu

policy

rack

route

site(need enhance)

-taskstate-

zone

passwd (NEW)

3. some special cases in object abstraction

(1). the "xcatdefaults" entry in postscripts table should be included in site object

(2). the litefile, statelite and litetree tables

[1]. if "image" is osimage groups, the fields should be included as attributes of the osimage object with the osimage group name

[2]. if "image" is ALL, the fields should be included as attributes of the osimage object with the osimage group name "ALL"

[3]. if "image" is an osimage name, the fields should be included in the osimage object

4. investigated the implementation of DB2OBJ

   • plan#1: SQLALchemy+class for table+object schema+class for object

     • pros:

       • no backward compatibility, clean code hierarchy
       • support configurable object schema
       • rich & flexible libraries for DB2OBJ mapping, data persistency in python

   • plan#2: refine XXdef commands to eliminate special case, REGEX translation and attribute inheritance

     • cons:

       • the code logic of object manipulation is TOO complex and mess up, too many special cases and obscure code logic
       • tightly coupled with current xCAT XXdef object ,
       • limited room for xCAT object restructure

5. investigated the layout of the inventory files for Ansible

   • https://github.com/chenglch/xcat-play/blob/master/ansible/xcat2/inventory/inventory.yml.example