Skip to content

Refactor

Jump to bottom
Ondrej Fabry edited this page May 25, 2020 · 22 revisions

This documents contains info related to refactor of cn-infra API.

April 2020

Motivation

The cn-infra code is a monolith with tightly coupled components that are not very usable separately without bringing the whole stack. The term plugin is overused and cluttered across the entire code base without any benefit.

Configuration Loading

  • plugin config usage is not very intuitive and hard to customize
  • real source for config data can be obscure
  • missing support for single configuration file for an app
  • using command-line flags does not play very well with plugin-defined flags

Dependency Injection / Initialization Order

  • lookup of plugin dependencies is black magic
  • default instances are defined as globals and setup even if not needed
  • there is no simple and practical way for overriding the init order

Other Issues

  • APIs for datasync and db/keyval are very confusing (embedded interfaces)
  • exec essentially contains two different implementation for process management
  • package agent has unfitting name and mixed responsibilities
  • some packages with common generic functionality should be part of cn-infra repo

Analysis

Usage

Here is usage summary for cn-infra packages.

vpp-agent

Plugins:

IDX  PLUGIN                TYPE                            PKG PATH                                                
 1   service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
 2   http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 3   logs                  *logmanager.Plugin              go.ligato.io/cn-infra/v2/logging/logmanager             
 4   kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
 5   resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
 6   etcd                  *etcd.Plugin                    go.ligato.io/cn-infra/v2/db/keyval/etcd                 
 7   kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
 8   consul                *consul.Plugin                  go.ligato.io/cn-infra/v2/db/keyval/consul               
 9   kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
10   redis                 *redis.Plugin                   go.ligato.io/cn-infra/v2/db/keyval/redis                
11   kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
12   status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
13   govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
14   netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
15   linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
16   linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
17   kafka                 *kafka.Plugin                   go.ligato.io/cn-infra/v2/messaging/kafka                
18   msgsync               *msgsync.Plugin                 go.ligato.io/cn-infra/v2/datasync/msgsync               
19   vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
20   vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
21   vpp-abfplugin         *abfplugin.ABFPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/abfplugin         
22   vpp-ipsec-plugin      *ipsecplugin.IPSecPlugin        go.ligato.io/vpp-agent/v3/plugins/vpp/ipsecplugin       
23   vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
24   vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
25   vpp-natplugin         *natplugin.NATPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/natplugin         
26   vpp-punt-plugin       *puntplugin.PuntPlugin          go.ligato.io/vpp-agent/v3/plugins/vpp/puntplugin        
27   vpp-stn-plugin        *stnplugin.STNPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/stnplugin         
28   vpp-srplugin          *srplugin.SRPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/srplugin          
29   linux-l3plugin        *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/linux/l3plugin        
30   linux-iptablesplugin  *iptablesplugin.IPTablesPlugin  go.ligato.io/vpp-agent/v3/plugins/linux/iptablesplugin  
31   grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
32   orchestrator          *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
33   configurator          *configurator.Plugin            go.ligato.io/vpp-agent/v3/plugins/configurator          
34   restpapi              *restapi.Plugin                 go.ligato.io/vpp-agent/v3/plugins/restapi               
35   prometheus            *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
36   telemetry             *telemetry.Plugin               go.ligato.io/vpp-agent/v3/plugins/telemetry             
37   probe                 *probe.Plugin                   go.ligato.io/cn-infra/v2/health/probe                   
38   VPPAgent              *app.VPPAgent                   go.ligato.io/vpp-agent/v3/cmd/vpp-agent/app
Complete list of plugins with deps 
IDX  PLUGIN                  TYPE                            PKG PATH                                                
 1   service-label           *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
 2   http                    *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 3   logs                    *logmanager.Plugin              go.ligato.io/cn-infra/v2/logging/logmanager             
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 4   kvscheduler             *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 5   resync                  *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
 6   etcd                    *etcd.Plugin                    go.ligato.io/cn-infra/v2/db/keyval/etcd                 
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
 7   kvdb-etcd-datasync      *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - etcd                  *etcd.Plugin                    go.ligato.io/cn-infra/v2/db/keyval/etcd                 
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
 8   consul                  *consul.Plugin                  go.ligato.io/cn-infra/v2/db/keyval/consul               
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
 9   kvdb-consul-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - consul                *consul.Plugin                  go.ligato.io/cn-infra/v2/db/keyval/consul               
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
10   redis                   *redis.Plugin                   go.ligato.io/cn-infra/v2/db/keyval/redis                
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
11   kvdb-redis-datasync     *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - redis                 *redis.Plugin                   go.ligato.io/cn-infra/v2/db/keyval/redis                
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
12   status-check            *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
13   govpp                   *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
14   netalloc                *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
15   linux-nsplugin          *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
16   linux-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
17   kafka                   *kafka.Plugin                   go.ligato.io/cn-infra/v2/messaging/kafka                
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
18   msgsync                 *msgsync.Plugin                 go.ligato.io/cn-infra/v2/datasync/msgsync               
     - kafka                 *kafka.Plugin                   go.ligato.io/cn-infra/v2/messaging/kafka                
19   vpp-ifplugin            *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - msgsync               *msgsync.Plugin                 go.ligato.io/cn-infra/v2/datasync/msgsync               
20   vpp-aclplugin           *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
21   vpp-abfplugin           *abfplugin.ABFPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/abfplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
22   vpp-ipsec-plugin        *ipsecplugin.IPSecPlugin        go.ligato.io/vpp-agent/v3/plugins/vpp/ipsecplugin       
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
23   vpp-l2plugin            *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
24   vpp-l3plugin            *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
25   vpp-natplugin           *natplugin.NATPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/natplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
26   vpp-punt-plugin         *puntplugin.PuntPlugin          go.ligato.io/vpp-agent/v3/plugins/vpp/puntplugin        
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
27   vpp-stn-plugin          *stnplugin.STNPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/stnplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
28   vpp-srplugin            *srplugin.SRPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/srplugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
29   linux-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/linux/l3plugin        
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
30   linux-iptablesplugin    *iptablesplugin.IPTablesPlugin  go.ligato.io/vpp-agent/v3/plugins/linux/iptablesplugin  
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
31   grpc                    *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
32   orchestrator            *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
     - grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
33   configurator            *configurator.Plugin            go.ligato.io/vpp-agent/v3/plugins/configurator          
     - grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
     - orchestrator          *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
34   restpapi                *restapi.Plugin                 go.ligato.io/vpp-agent/v3/plugins/restapi               
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
35   prometheus              *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
36   telemetry               *telemetry.Plugin               go.ligato.io/vpp-agent/v3/plugins/telemetry             
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - prometheus            *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
     - grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
37   probe                   *probe.Plugin                   go.ligato.io/cn-infra/v2/health/probe                   
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - prometheus            *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
38   VPPAgent                *app.VPPAgent                   go.ligato.io/vpp-agent/v3/cmd/vpp-agent/app             
     - logs                  *logmanager.Plugin              go.ligato.io/cn-infra/v2/logging/logmanager             
     - vpp-abfplugin         *abfplugin.ABFPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/abfplugin         
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - vpp-ipsec-plugin      *ipsecplugin.IPSecPlugin        go.ligato.io/vpp-agent/v3/plugins/vpp/ipsecplugin       
     - vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - vpp-natplugin         *natplugin.NATPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/natplugin         
     - vpp-punt-plugin       *puntplugin.PuntPlugin          go.ligato.io/vpp-agent/v3/plugins/vpp/puntplugin        
     - vpp-stn-plugin        *stnplugin.STNPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/stnplugin         
     - vpp-srplugin          *srplugin.SRPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/srplugin          
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-l3plugin        *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/linux/l3plugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - linux-iptablesplugin  *iptablesplugin.IPTablesPlugin  go.ligato.io/vpp-agent/v3/plugins/linux/iptablesplugin  
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - orchestrator          *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - configurator          *configurator.Plugin            go.ligato.io/vpp-agent/v3/plugins/configurator          
     - restpapi              *restapi.Plugin                 go.ligato.io/vpp-agent/v3/plugins/restapi               
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - telemetry             *telemetry.Plugin               go.ligato.io/vpp-agent/v3/plugins/telemetry             
     - probe                 *probe.Plugin                   go.ligato.io/cn-infra/v2/health/probe
imports 31, depends on 45 
# vpp-agent imports following cn-infra packages (excluding examples/tests)
  1 go.ligato.io/cn-infra/v2/datasync/msgsync
  1 go.ligato.io/cn-infra/v2/db/keyval/consul
  1 go.ligato.io/cn-infra/v2/db/keyval/kvproto
  1 go.ligato.io/cn-infra/v2/db/keyval/redis
  1 go.ligato.io/cn-infra/v2/exec/supervisor
  1 go.ligato.io/cn-infra/v2/health/statuscheck/model/status
  1 go.ligato.io/cn-infra/v2/logging/logmanager
  1 go.ligato.io/cn-infra/v2/messaging/kafka
  1 go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
  2 go.ligato.io/cn-infra/v2/datasync/kvdbsync
  2 go.ligato.io/cn-infra/v2/datasync/syncbase
  2 go.ligato.io/cn-infra/v2/db/keyval/etcd
  2 go.ligato.io/cn-infra/v2/health/probe
  2 go.ligato.io/cn-infra/v2/rpc/prometheus
  2 go.ligato.io/cn-infra/v2/utils/safeclose
  3 go.ligato.io/cn-infra/v2/agent
  3 go.ligato.io/cn-infra/v2/rpc/grpc
  4 go.ligato.io/cn-infra/v2/datasync/resync
  4 go.ligato.io/cn-infra/v2/idxmap/mem
  4 go.ligato.io/cn-infra/v2/rpc/rest
  6 go.ligato.io/cn-infra/v2/datasync
  6 go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
  6 go.ligato.io/cn-infra/v2/db/keyval
  7 go.ligato.io/cn-infra/v2/config
  9 go.ligato.io/cn-infra/v2/logging/logrus
  9 go.ligato.io/cn-infra/v2/servicelabel
  9 go.ligato.io/cn-infra/v2/utils/addrs
 12 go.ligato.io/cn-infra/v2/health/statuscheck
 23 go.ligato.io/cn-infra/v2/idxmap
 23 go.ligato.io/cn-infra/v2/infra
 90 go.ligato.io/cn-infra/v2/logging

# vpp-agent directly imports 31 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/msgsync
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/datasync/syncbase
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/consul
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/db/keyval/kvproto
go.ligato.io/cn-infra/v2/db/keyval/redis
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/health/statuscheck/model/status
go.ligato.io/cn-infra/v2/idxmap
go.ligato.io/cn-infra/v2/idxmap/mem
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/messaging/kafka
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
go.ligato.io/cn-infra/v2/servicelabel
go.ligato.io/cn-infra/v2/utils/addrs
go.ligato.io/cn-infra/v2/utils/safeclose

# vpp-agent has dependency on 45 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/msgsync
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/datasync/syncbase
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/consul
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/db/keyval/kvproto
go.ligato.io/cn-infra/v2/db/keyval/redis
go.ligato.io/cn-infra/v2/exec/processmanager
go.ligato.io/cn-infra/v2/exec/processmanager/status
go.ligato.io/cn-infra/v2/exec/processmanager/template
go.ligato.io/cn-infra/v2/exec/processmanager/template/model/process
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/health/statuscheck/model/status
go.ligato.io/cn-infra/v2/idxmap
go.ligato.io/cn-infra/v2/idxmap/mem
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/logging/measure
go.ligato.io/cn-infra/v2/logging/measure/model/apitrace
go.ligato.io/cn-infra/v2/messaging
go.ligato.io/cn-infra/v2/messaging/kafka
go.ligato.io/cn-infra/v2/messaging/kafka/client
go.ligato.io/cn-infra/v2/messaging/kafka/mux
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/rpc/rest/security
go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
go.ligato.io/cn-infra/v2/servicelabel
go.ligato.io/cn-infra/v2/utils/addrs
go.ligato.io/cn-infra/v2/utils/clienttls
go.ligato.io/cn-infra/v2/utils/once
go.ligato.io/cn-infra/v2/utils/ratelimit
go.ligato.io/cn-infra/v2/utils/redact
go.ligato.io/cn-infra/v2/utils/safeclose

kiknos

imports 20, depends on 36 
# kiknos imports following cn-infra packages
  1 go.ligato.io/cn-infra/v2/config
  1 go.ligato.io/cn-infra/v2/exec/supervisor
  2 go.ligato.io/cn-infra/v2/datasync/resync
  3 go.ligato.io/cn-infra/v2/datasync/kvdbsync
  3 go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
  3 go.ligato.io/cn-infra/v2/db/cryptodata
  3 go.ligato.io/cn-infra/v2/health/probe
  3 go.ligato.io/cn-infra/v2/health/statuscheck
  3 go.ligato.io/cn-infra/v2/logging/logmanager
  4 go.ligato.io/cn-infra/v2/datasync
  4 go.ligato.io/cn-infra/v2/logging/logrus
  5 go.ligato.io/cn-infra/v2/agent
  5 go.ligato.io/cn-infra/v2/db/keyval
  5 go.ligato.io/cn-infra/v2/db/keyval/etcd
  5 go.ligato.io/cn-infra/v2/rpc/prometheus
  6 go.ligato.io/cn-infra/v2/rpc/grpc
  7 go.ligato.io/cn-infra/v2/infra
  7 go.ligato.io/cn-infra/v2/rpc/rest
  9 go.ligato.io/cn-infra/v2/servicelabel
 16 go.ligato.io/cn-infra/v2/logging

# kiknos directly imports 20 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/db/cryptodata
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/servicelabel

# kiknos has dependency on 36 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/datasync/syncbase
go.ligato.io/cn-infra/v2/db/cryptodata
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/db/keyval/kvproto
go.ligato.io/cn-infra/v2/exec/processmanager
go.ligato.io/cn-infra/v2/exec/processmanager/status
go.ligato.io/cn-infra/v2/exec/processmanager/template
go.ligato.io/cn-infra/v2/exec/processmanager/template/model/process
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/health/statuscheck/model/status
go.ligato.io/cn-infra/v2/idxmap
go.ligato.io/cn-infra/v2/idxmap/mem
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/logging/measure
go.ligato.io/cn-infra/v2/logging/measure/model/apitrace
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/rpc/rest/security
go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
go.ligato.io/cn-infra/v2/servicelabel
go.ligato.io/cn-infra/v2/utils/addrs
go.ligato.io/cn-infra/v2/utils/once
go.ligato.io/cn-infra/v2/utils/safeclose

Used commands:

# count of imports for cn-infra packages
go list -f '{{if .Module}}{{if not .Standard}}{{range .Imports}}{{printf "%s\n" .}}{{end}}{{end}}{{end}}' $(go list ./...) | grep cn-infra | sort | uniq -c | sort -h

# list package dependencies (including transitive) for cn-infra
go list ./... | xargs go list -f '{{range .Imports}}{{printf "%s\n" .}}{{end}}' | grep cn-infra | sort | uniq

# list direct imports of cn-infra packages
go list ./... | xargs go list -f '{{range .Imports}}{{printf "%s\n" .}}{{end}}' | grep cn-infra | sort | uniq

Components

Infra

  • config
  • logging
  • runtime/app/cli
  • plugin/service
  • debug/prof
  • metrics/stats
  • version

Plugins

Plugins are components defined by generic API that support pluggable implementations.

  • kvstore plugins: bolt, etcd, consul, filedb, redis, ..
  • sqldb plugins: cassandra, sqlite, ..
  • datasync plugins: kvstore, rpc, .. ???
  • rpc plugins
    • grpc, rest, .. ???
  • exec plugins
    • supervisor, .. ???

Goals

  • Reorganize package locations

    • agent => infra/app or infra/runtime+infra/version?
    • config => infra/cfg?
    • logging => infra/log?
    • servicelabel => infra/service or infra/label?
    • health/probe => infra/health?
    • health/status => infra/health?
    • db/keyval => kvstore or db/kvstore?
    • db/cryptodata => security/crypto?
    • rpc/rest/security => security/rest?
    • utils/redact => security/redact?
    • utils/ratelimit => security/ratelimit?
    • rpc/prometheus => telemetry/prometheus
    • idxmap => utils/idxmap

  • Get rid of magic behaviours

    • plugin dependency lookup should have more explicit behaviour

  • Redefine complex interfaces into simpler ones

    • kvstore package should define cleaner KV store API (no proto or datasync)
    • datasync should use single channel for data (instead of change/resync channels)

  • Look for ways to avoid forcing cn-infra essentials to users

    • Get rid of plugin overuse

  • Decouple packages with mixed responsibilities into smaller pieces

    • agent package should be separated into: app/version/runtime/infra/..?

  • Remove unused code / parts of repo

    • docker directory can be possibly removed
    • utils/safeclose should be removed (promotes unidiomatic code)
    • logging/measure is not used

  • Move common packages from vpp-agent repository

    • pkg/debug => infra/debug
    • pkg/version => infra/version
    • plugins/kvscheduler => processor???

Config

Use cases:

  • read config from file (yaml, json..)
  • unmarshal into defined struct
  • set config defaults
  • merge extra config with current
  • write config to file
  • bind flags to config
  • bind env vars to config
Clone this wiki locally