Skip to content

Refactor

Jump to bottom
Ondrej Fabry edited this page Apr 24, 2020 · 22 revisions

This documents contains info related to refactor of cn-infra API.

April 2020

Motivation

Main Problems

  • package config is hard to use and is missing support for single config file
  • APIs for datasync and db/keyval are very confusing (embedded interfaces)
  • lookup of plugin dependencies is black magic and not very flexible
  • default plugin instances defined as global variable and causing name conflicts

Other Issues

  • term plugin is totally overused and does not really represent its use
  • exec essentially contains two different implementation for process management
  • depends on several unnecessary Go modules (uuid, render..)
  • package agent has unfitting name and mixed responsibilities
  • some packages with common generic functionality are part of vpp-agent repo

Analysis

Usage

Here is usage summary for cn-infra packages.

vpp-agent

Plugins:

IDX  PLUGIN                TYPE                            PKG PATH                                                
 1   service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
 2   http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 3   logs                  *logmanager.Plugin              go.ligato.io/cn-infra/v2/logging/logmanager             
 4   kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
 5   resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
 6   etcd                  *etcd.Plugin                    go.ligato.io/cn-infra/v2/db/keyval/etcd                 
 7   kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
 8   consul                *consul.Plugin                  go.ligato.io/cn-infra/v2/db/keyval/consul               
 9   kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
10   redis                 *redis.Plugin                   go.ligato.io/cn-infra/v2/db/keyval/redis                
11   kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
12   status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
13   govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
14   netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
15   linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
16   linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
17   kafka                 *kafka.Plugin                   go.ligato.io/cn-infra/v2/messaging/kafka                
18   msgsync               *msgsync.Plugin                 go.ligato.io/cn-infra/v2/datasync/msgsync               
19   vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
20   vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
21   vpp-abfplugin         *abfplugin.ABFPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/abfplugin         
22   vpp-ipsec-plugin      *ipsecplugin.IPSecPlugin        go.ligato.io/vpp-agent/v3/plugins/vpp/ipsecplugin       
23   vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
24   vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
25   vpp-natplugin         *natplugin.NATPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/natplugin         
26   vpp-punt-plugin       *puntplugin.PuntPlugin          go.ligato.io/vpp-agent/v3/plugins/vpp/puntplugin        
27   vpp-stn-plugin        *stnplugin.STNPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/stnplugin         
28   vpp-srplugin          *srplugin.SRPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/srplugin          
29   linux-l3plugin        *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/linux/l3plugin        
30   linux-iptablesplugin  *iptablesplugin.IPTablesPlugin  go.ligato.io/vpp-agent/v3/plugins/linux/iptablesplugin  
31   grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
32   orchestrator          *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
33   configurator          *configurator.Plugin            go.ligato.io/vpp-agent/v3/plugins/configurator          
34   restpapi              *restapi.Plugin                 go.ligato.io/vpp-agent/v3/plugins/restapi               
35   prometheus            *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
36   telemetry             *telemetry.Plugin               go.ligato.io/vpp-agent/v3/plugins/telemetry             
37   probe                 *probe.Plugin                   go.ligato.io/cn-infra/v2/health/probe                   
38   VPPAgent              *app.VPPAgent                   go.ligato.io/vpp-agent/v3/cmd/vpp-agent/app
Complete list of plugins with deps 
IDX  PLUGIN                  TYPE                            PKG PATH                                                
 1   service-label           *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
 2   http                    *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 3   logs                    *logmanager.Plugin              go.ligato.io/cn-infra/v2/logging/logmanager             
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 4   kvscheduler             *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
 5   resync                  *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
 6   etcd                    *etcd.Plugin                    go.ligato.io/cn-infra/v2/db/keyval/etcd                 
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
 7   kvdb-etcd-datasync      *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - etcd                  *etcd.Plugin                    go.ligato.io/cn-infra/v2/db/keyval/etcd                 
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
 8   consul                  *consul.Plugin                  go.ligato.io/cn-infra/v2/db/keyval/consul               
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
 9   kvdb-consul-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - consul                *consul.Plugin                  go.ligato.io/cn-infra/v2/db/keyval/consul               
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
10   redis                   *redis.Plugin                   go.ligato.io/cn-infra/v2/db/keyval/redis                
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
11   kvdb-redis-datasync     *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - redis                 *redis.Plugin                   go.ligato.io/cn-infra/v2/db/keyval/redis                
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
12   status-check            *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
13   govpp                   *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - resync                *resync.Plugin                  go.ligato.io/cn-infra/v2/datasync/resync                
14   netalloc                *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
15   linux-nsplugin          *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
16   linux-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
17   kafka                   *kafka.Plugin                   go.ligato.io/cn-infra/v2/messaging/kafka                
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
18   msgsync                 *msgsync.Plugin                 go.ligato.io/cn-infra/v2/datasync/msgsync               
     - kafka                 *kafka.Plugin                   go.ligato.io/cn-infra/v2/messaging/kafka                
19   vpp-ifplugin            *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - msgsync               *msgsync.Plugin                 go.ligato.io/cn-infra/v2/datasync/msgsync               
20   vpp-aclplugin           *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
21   vpp-abfplugin           *abfplugin.ABFPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/abfplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
22   vpp-ipsec-plugin        *ipsecplugin.IPSecPlugin        go.ligato.io/vpp-agent/v3/plugins/vpp/ipsecplugin       
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
23   vpp-l2plugin            *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
24   vpp-l3plugin            *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
25   vpp-natplugin           *natplugin.NATPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/natplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
26   vpp-punt-plugin         *puntplugin.PuntPlugin          go.ligato.io/vpp-agent/v3/plugins/vpp/puntplugin        
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
27   vpp-stn-plugin          *stnplugin.STNPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/stnplugin         
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
28   vpp-srplugin            *srplugin.SRPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/srplugin          
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
29   linux-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/linux/l3plugin        
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
30   linux-iptablesplugin    *iptablesplugin.IPTablesPlugin  go.ligato.io/vpp-agent/v3/plugins/linux/iptablesplugin  
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
31   grpc                    *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
32   orchestrator            *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
     - grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
     - kvscheduler           *kvscheduler.Scheduler          go.ligato.io/vpp-agent/v3/plugins/kvscheduler           
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
33   configurator            *configurator.Plugin            go.ligato.io/vpp-agent/v3/plugins/configurator          
     - grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
     - orchestrator          *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
34   restpapi                *restapi.Plugin                 go.ligato.io/vpp-agent/v3/plugins/restapi               
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
35   prometheus              *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
36   telemetry               *telemetry.Plugin               go.ligato.io/vpp-agent/v3/plugins/telemetry             
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - govpp                 *govppmux.Plugin                go.ligato.io/vpp-agent/v3/plugins/govppmux              
     - prometheus            *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
     - grpc                  *grpc.Plugin                    go.ligato.io/cn-infra/v2/rpc/grpc                       
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
37   probe                   *probe.Plugin                   go.ligato.io/cn-infra/v2/health/probe                   
     - service-label         *servicelabel.Plugin            go.ligato.io/cn-infra/v2/servicelabel                   
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - http                  *rest.Plugin                    go.ligato.io/cn-infra/v2/rpc/rest                       
     - prometheus            *prometheus.Plugin              go.ligato.io/cn-infra/v2/rpc/prometheus                 
38   VPPAgent                *app.VPPAgent                   go.ligato.io/vpp-agent/v3/cmd/vpp-agent/app             
     - logs                  *logmanager.Plugin              go.ligato.io/cn-infra/v2/logging/logmanager             
     - vpp-abfplugin         *abfplugin.ABFPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/abfplugin         
     - vpp-aclplugin         *aclplugin.ACLPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/aclplugin         
     - vpp-ifplugin          *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/ifplugin          
     - vpp-ipsec-plugin      *ipsecplugin.IPSecPlugin        go.ligato.io/vpp-agent/v3/plugins/vpp/ipsecplugin       
     - vpp-l2plugin          *l2plugin.L2Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l2plugin          
     - vpp-l3plugin          *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/vpp/l3plugin          
     - vpp-natplugin         *natplugin.NATPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/natplugin         
     - vpp-punt-plugin       *puntplugin.PuntPlugin          go.ligato.io/vpp-agent/v3/plugins/vpp/puntplugin        
     - vpp-stn-plugin        *stnplugin.STNPlugin            go.ligato.io/vpp-agent/v3/plugins/vpp/stnplugin         
     - vpp-srplugin          *srplugin.SRPlugin              go.ligato.io/vpp-agent/v3/plugins/vpp/srplugin          
     - linux-ifplugin        *ifplugin.IfPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/ifplugin        
     - linux-l3plugin        *l3plugin.L3Plugin              go.ligato.io/vpp-agent/v3/plugins/linux/l3plugin        
     - linux-nsplugin        *nsplugin.NsPlugin              go.ligato.io/vpp-agent/v3/plugins/linux/nsplugin        
     - linux-iptablesplugin  *iptablesplugin.IPTablesPlugin  go.ligato.io/vpp-agent/v3/plugins/linux/iptablesplugin  
     - netalloc              *netalloc.Plugin                go.ligato.io/vpp-agent/v3/plugins/netalloc              
     - orchestrator          *orchestrator.Plugin            go.ligato.io/vpp-agent/v3/plugins/orchestrator          
     - kvdb-etcd-datasync    *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-consul-datasync  *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - kvdb-redis-datasync   *kvdbsync.Plugin                go.ligato.io/cn-infra/v2/datasync/kvdbsync              
     - configurator          *configurator.Plugin            go.ligato.io/vpp-agent/v3/plugins/configurator          
     - restpapi              *restapi.Plugin                 go.ligato.io/vpp-agent/v3/plugins/restapi               
     - status-check          *statuscheck.Plugin             go.ligato.io/cn-infra/v2/health/statuscheck             
     - telemetry             *telemetry.Plugin               go.ligato.io/vpp-agent/v3/plugins/telemetry             
     - probe                 *probe.Plugin                   go.ligato.io/cn-infra/v2/health/probe
imports 31, depends on 45 
# vpp-agent imports following cn-infra packages (excluding examples/tests)
  1 go.ligato.io/cn-infra/v2/datasync/msgsync
  1 go.ligato.io/cn-infra/v2/db/keyval/consul
  1 go.ligato.io/cn-infra/v2/db/keyval/kvproto
  1 go.ligato.io/cn-infra/v2/db/keyval/redis
  1 go.ligato.io/cn-infra/v2/exec/supervisor
  1 go.ligato.io/cn-infra/v2/health/statuscheck/model/status
  1 go.ligato.io/cn-infra/v2/logging/logmanager
  1 go.ligato.io/cn-infra/v2/messaging/kafka
  1 go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
  2 go.ligato.io/cn-infra/v2/datasync/kvdbsync
  2 go.ligato.io/cn-infra/v2/datasync/syncbase
  2 go.ligato.io/cn-infra/v2/db/keyval/etcd
  2 go.ligato.io/cn-infra/v2/health/probe
  2 go.ligato.io/cn-infra/v2/rpc/prometheus
  2 go.ligato.io/cn-infra/v2/utils/safeclose
  3 go.ligato.io/cn-infra/v2/agent
  3 go.ligato.io/cn-infra/v2/rpc/grpc
  4 go.ligato.io/cn-infra/v2/datasync/resync
  4 go.ligato.io/cn-infra/v2/idxmap/mem
  4 go.ligato.io/cn-infra/v2/rpc/rest
  6 go.ligato.io/cn-infra/v2/datasync
  6 go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
  6 go.ligato.io/cn-infra/v2/db/keyval
  7 go.ligato.io/cn-infra/v2/config
  9 go.ligato.io/cn-infra/v2/logging/logrus
  9 go.ligato.io/cn-infra/v2/servicelabel
  9 go.ligato.io/cn-infra/v2/utils/addrs
 12 go.ligato.io/cn-infra/v2/health/statuscheck
 23 go.ligato.io/cn-infra/v2/idxmap
 23 go.ligato.io/cn-infra/v2/infra
 90 go.ligato.io/cn-infra/v2/logging

# vpp-agent directly imports 31 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/msgsync
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/datasync/syncbase
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/consul
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/db/keyval/kvproto
go.ligato.io/cn-infra/v2/db/keyval/redis
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/health/statuscheck/model/status
go.ligato.io/cn-infra/v2/idxmap
go.ligato.io/cn-infra/v2/idxmap/mem
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/messaging/kafka
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
go.ligato.io/cn-infra/v2/servicelabel
go.ligato.io/cn-infra/v2/utils/addrs
go.ligato.io/cn-infra/v2/utils/safeclose

# vpp-agent has dependency on 45 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/msgsync
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/datasync/syncbase
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/consul
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/db/keyval/kvproto
go.ligato.io/cn-infra/v2/db/keyval/redis
go.ligato.io/cn-infra/v2/exec/processmanager
go.ligato.io/cn-infra/v2/exec/processmanager/status
go.ligato.io/cn-infra/v2/exec/processmanager/template
go.ligato.io/cn-infra/v2/exec/processmanager/template/model/process
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/health/statuscheck/model/status
go.ligato.io/cn-infra/v2/idxmap
go.ligato.io/cn-infra/v2/idxmap/mem
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/logging/measure
go.ligato.io/cn-infra/v2/logging/measure/model/apitrace
go.ligato.io/cn-infra/v2/messaging
go.ligato.io/cn-infra/v2/messaging/kafka
go.ligato.io/cn-infra/v2/messaging/kafka/client
go.ligato.io/cn-infra/v2/messaging/kafka/mux
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/rpc/rest/security
go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
go.ligato.io/cn-infra/v2/servicelabel
go.ligato.io/cn-infra/v2/utils/addrs
go.ligato.io/cn-infra/v2/utils/clienttls
go.ligato.io/cn-infra/v2/utils/once
go.ligato.io/cn-infra/v2/utils/ratelimit
go.ligato.io/cn-infra/v2/utils/redact
go.ligato.io/cn-infra/v2/utils/safeclose

kiknos

imports 20, depends on 36 
# kiknos imports following cn-infra packages
  1 go.ligato.io/cn-infra/v2/config
  1 go.ligato.io/cn-infra/v2/exec/supervisor
  2 go.ligato.io/cn-infra/v2/datasync/resync
  3 go.ligato.io/cn-infra/v2/datasync/kvdbsync
  3 go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
  3 go.ligato.io/cn-infra/v2/db/cryptodata
  3 go.ligato.io/cn-infra/v2/health/probe
  3 go.ligato.io/cn-infra/v2/health/statuscheck
  3 go.ligato.io/cn-infra/v2/logging/logmanager
  4 go.ligato.io/cn-infra/v2/datasync
  4 go.ligato.io/cn-infra/v2/logging/logrus
  5 go.ligato.io/cn-infra/v2/agent
  5 go.ligato.io/cn-infra/v2/db/keyval
  5 go.ligato.io/cn-infra/v2/db/keyval/etcd
  5 go.ligato.io/cn-infra/v2/rpc/prometheus
  6 go.ligato.io/cn-infra/v2/rpc/grpc
  7 go.ligato.io/cn-infra/v2/infra
  7 go.ligato.io/cn-infra/v2/rpc/rest
  9 go.ligato.io/cn-infra/v2/servicelabel
 16 go.ligato.io/cn-infra/v2/logging

# kiknos directly imports 20 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/db/cryptodata
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/servicelabel

# kiknos has dependency on 36 packages from cn-infra
go.ligato.io/cn-infra/v2/agent
go.ligato.io/cn-infra/v2/config
go.ligato.io/cn-infra/v2/datasync
go.ligato.io/cn-infra/v2/datasync/kvdbsync
go.ligato.io/cn-infra/v2/datasync/kvdbsync/local
go.ligato.io/cn-infra/v2/datasync/resync
go.ligato.io/cn-infra/v2/datasync/syncbase
go.ligato.io/cn-infra/v2/db/cryptodata
go.ligato.io/cn-infra/v2/db/keyval
go.ligato.io/cn-infra/v2/db/keyval/etcd
go.ligato.io/cn-infra/v2/db/keyval/kvproto
go.ligato.io/cn-infra/v2/exec/processmanager
go.ligato.io/cn-infra/v2/exec/processmanager/status
go.ligato.io/cn-infra/v2/exec/processmanager/template
go.ligato.io/cn-infra/v2/exec/processmanager/template/model/process
go.ligato.io/cn-infra/v2/exec/supervisor
go.ligato.io/cn-infra/v2/health/probe
go.ligato.io/cn-infra/v2/health/statuscheck
go.ligato.io/cn-infra/v2/health/statuscheck/model/status
go.ligato.io/cn-infra/v2/idxmap
go.ligato.io/cn-infra/v2/idxmap/mem
go.ligato.io/cn-infra/v2/infra
go.ligato.io/cn-infra/v2/logging
go.ligato.io/cn-infra/v2/logging/logmanager
go.ligato.io/cn-infra/v2/logging/logrus
go.ligato.io/cn-infra/v2/logging/measure
go.ligato.io/cn-infra/v2/logging/measure/model/apitrace
go.ligato.io/cn-infra/v2/rpc/grpc
go.ligato.io/cn-infra/v2/rpc/prometheus
go.ligato.io/cn-infra/v2/rpc/rest
go.ligato.io/cn-infra/v2/rpc/rest/security
go.ligato.io/cn-infra/v2/rpc/rest/security/model/access-security
go.ligato.io/cn-infra/v2/servicelabel
go.ligato.io/cn-infra/v2/utils/addrs
go.ligato.io/cn-infra/v2/utils/once
go.ligato.io/cn-infra/v2/utils/safeclose

Used commands:

# count of imports for cn-infra packages
go list -f '{{if .Module}}{{if not .Standard}}{{range .Imports}}{{printf "%s\n" .}}{{end}}{{end}}{{end}}' $(go list ./...) | grep cn-infra | sort | uniq -c | sort -h

# list package dependencies (including transitive) for cn-infra
go list ./... | xargs go list -f '{{range .Imports}}{{printf "%s\n" .}}{{end}}' | grep cn-infra | sort | uniq

# list direct imports of cn-infra packages
go list ./... | xargs go list -f '{{range .Imports}}{{printf "%s\n" .}}{{end}}' | grep cn-infra | sort | uniq

Components

Infra

  • config
  • logging
  • runtime/app/cli
  • plugin/service
  • debug/prof
  • metrics/stats
  • version

Plugins

Plugins are components defined by generic API that support pluggable implementations.

  • kvstore plugins: bolt, etcd, consul, filedb, redis, ..
  • sqldb plugins: cassandra, sqlite, ..
  • datasync plugins: kvstore, rpc, .. ???
  • rpc plugins
    • grpc, rest, .. ???
  • exec plugins
    • supervisor, .. ???

Goals

  • Reorganize package locations

    • agent => infra/app or infra/runtime+infra/version?
    • config => infra/cfg?
    • logging => infra/log?
    • servicelabel => infra/service or infra/label?
    • health/probe => infra/health?
    • health/status => infra/health?
    • db/keyval => kvstore or db/kvstore?
    • db/cryptodata => security/crypto?
    • rpc/rest/security => security/rest?
    • utils/redact => security/redact?
    • utils/ratelimit => security/ratelimit?
    • rpc/prometheus => telemetry/prometheus
    • idxmap => utils/idxmap

  • Get rid of magic behaviours

    • plugin dependency lookup should have more explicit behaviour

  • Redefine complex interfaces into simpler ones

    • kvstore package should define cleaner KV store API (no proto or datasync)
    • datasync should use single channel for data (instead of change/resync channels)

  • Look for ways to avoid forcing cn-infra essentials to users

    • Get rid of plugin overuse

  • Decouple packages with mixed responsibilities into smaller pieces

    • agent package should be separated into: app/version/runtime/infra/..?

  • Remove unused code / parts of repo

    • docker directory can be possibly removed
    • utils/safeclose should be removed (promotes unidiomatic code)
    • logging/measure is not used

  • Move common packages from vpp-agent repository

    • pkg/debug => infra/debug
    • pkg/version => infra/version
    • plugins/kvscheduler => processor???

Config

Use cases:

  • read config from file (yaml, json..)
  • unmarshal into defined struct
  • set config defaults
  • merge extra config with current
  • write config to file
  • bind flags to config
  • bind env vars to config
Clone this wiki locally