Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Harvest should load templates from a set of conf directories #2330

Closed
2 tasks done
cgrinds opened this issue Aug 25, 2023 · 2 comments · Fixed by #2329 or #2394
Closed
2 tasks done

Harvest should load templates from a set of conf directories #2330

cgrinds opened this issue Aug 25, 2023 · 2 comments · Fixed by #2329 or #2394
Labels
24.05 feature New feature or request status/done

Comments

@cgrinds
Copy link
Collaborator

cgrinds commented Aug 25, 2023 •
edited
Loading

  • Revise extend documentation to use confPath
  • Docker changes
@cgrinds cgrinds added feature New feature or request 23.11 labels Aug 25, 2023
@cgrinds cgrinds self-assigned this Aug 25, 2023
@cgrinds cgrinds linked a pull request Aug 25, 2023 that will close this issue
feat: Harvest should load templates from a set of conf directories #2329
Merged
2 tasks
@cgrinds
Copy link
Collaborator Author

cgrinds commented Aug 25, 2023

Given:

tree /tmp/foo/
/tmp/foo/
└── zapi
    ├── cdot
    │   └── 9.8.0
    │       ├── aggr.yaml
    │       ├── aggr2.yaml
    │       ├── custom_volume.yaml
    │       └── volume.yaml
    ├── custom.yaml
    ├── default.yaml
    └── limited.yaml
cat /tmp/foo/zapi/custom.yaml 
objects:
  Volume: custom_volume.yaml
cat /tmp/foo/zapi/limited.yaml 
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:              aggr2.yaml
  Node:                   node.yaml
  Volume:                 volume.yaml
cat harvest.yml
Pollers:
  
  fas8060:
    addr: 10.193.48.163
    conf_path: /tmp/foo:conf

  sar:
    addr: 10.193.48.154
    collectors:
      - Zapi:
        - limited.yaml
cat /tmp/foo/zapi/default.yaml
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:                   aggr.yaml
  AggregateEfficiency:         aggr_efficiency.yaml
  CIFSSession:                 cifs_session.yaml
  ClusterPeer:                 clusterpeer.yaml
  Disk:                        disk.yaml
  EmsDestination:              ems_destination.yaml
  LIF:                         lif.yaml
  Lun:                         lun.yaml
#  NetPort:                     netPort.yaml
  Namespace:                   namespace.yaml
  Node:                        node.yaml
  NtpServer:                   ntpserver.yaml
  QosPolicyAdaptive:           qos_policy_adaptive.yaml
  QosPolicyFixed:              qos_policy_fixed.yaml
  QosWorkload:                 qos_workload.yaml
  Qtree:                       qtree.yaml
  Security:                    security.yaml
  SecurityAccount:             security_account.yaml
  SecurityAuditDestination:    security_audit_dest.yaml
  SecurityCert:                security_certificate.yaml
  SecurityLogin:               security_login.yaml
  Sensor:                      sensor.yaml
  Shelf:                       shelf.yaml
  SnapMirror:                  snapmirror.yaml
  SnapshotPolicy:              snapshotpolicy.yaml
  Status:                      status.yaml
  Status_7mode:                status_7.yaml
  Subsystem:                   subsystem.yaml
  Support:                     support.yaml
  SVM:                         svm.yaml
  Volume:                      volume.yaml

sar examples

example 1

go run cmd/poller/poller.go --poller sar --promPort 14001 --confpath conf2:conf3:conf4:/tmp/foo:conf

2023-08-25T12:26:37-04:00 INF poller/poller.go:204 > Init Poller=sar asup=false confPath=conf2:conf3:conf4:/tmp/foo:conf config=harvest.yml daemon=false debug=false homePath= hostname=cgrindst-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=14001 version="harvest version 2.0.2-rc2 (commit HEAD) (build date undefined) darwin/amd64"
2023-08-25T12:26:38-04:00 INF collector/helpers.go:80 > best-fit template Poller=sar collector=Zapi:Volume path=/tmp/foo/zapi/cdot/9.8.0/volume.yaml v=9.13.1
2023-08-25T12:26:38-04:00 INF collector/helpers.go:80 > best-fit template Poller=sar collector=Zapi:Aggregate path=/tmp/foo/zapi/cdot/9.8.0/aggr2.yaml v=9.13.1
2023-08-25T12:26:39-04:00 INF collector/helpers.go:80 > best-fit template Poller=sar collector=Zapi:Node path=conf/zapi/cdot/9.8.0/node.yaml v=9.13.1

Notes:

  • The confpath is conf2:conf3:conf4:/tmp/foo:conf
  • No templates are found in the first three directories (conf2, conf3, conf4) because those directories do not exist
  • Zapi:Volume and Zapi:Aggregate templates are loaded from /tmp/foo/zapi/cdot/9.8.0 since the /tmp/foo directory is listed before conf and those templates exist under /tmp/foo
  • Zapi:Node template is loaded from conf/zapi/cdot/9.8.0/node.yaml since the /tmp/foo directory does not contain a template for node
  • No other objects are attempted because the poller section of harvest.yml uses a collector template named limited.yaml.
  • Since a collector template (limited.yaml) is used, the collector's custom.yaml is not loaded

example 2

go run cmd/poller/poller.go --poller sar --promPort 14001 --confpath conf:/tmp/foo

2023-08-25T12:35:36-04:00 INF poller/poller.go:204 > Init Poller=sar asup=false confPath=conf:/tmp/foo config=harvest.yml daemon=false debug=false homePath= hostname=cgrindst-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=14001 version="harvest version 2.0.2-rc2 (commit HEAD) (build date undefined) darwin/amd64"
2023-08-25T12:35:37-04:00 INF collector/helpers.go:80 > best-fit template Poller=sar collector=Zapi:Volume path=conf/zapi/cdot/9.8.0/volume.yaml v=9.13.1
2023-08-25T12:35:37-04:00 INF collector/helpers.go:80 > best-fit template Poller=sar collector=Zapi:Aggregate path=/tmp/foo/zapi/cdot/9.8.0/aggr2.yaml v=9.13.1
2023-08-25T12:35:38-04:00 INF collector/helpers.go:80 > best-fit template Poller=sar collector=Zapi:Node path=conf/zapi/cdot/9.8.0/node.yaml v=9.13.1

Notes:

  • The confpath is conf:/tmp/foo
  • All templates are loaded from the conf directory

example 3

go run cmd/poller/poller.go --poller sar --promPort 14001

2023-08-25T12:37:49-04:00 INF poller/poller.go:204 > Init Poller=sar asup=false confPath=conf config=harvest.yml daemon=false debug=false homePath= hostname=cgrindst-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=14001 version="harvest version 2.0.2-rc2 (commit HEAD) (build date undefined) darwin/amd64"
2023-08-25T12:37:49-04:00 WRN poller/poller.go:592 > Unable to load template. Poller=sar collector=Zapi confPaths=["conf"] err="template not found on confPath" template=limited.yaml
2023-08-25T12:37:49-04:00 ERR poller/poller.go:273 > Failed to read objects error="no templates loaded for Zapi" Poller=sar collector=Zapi templates=["limited.yaml"]

Notes:

  • The confpath is not set which means it uses the out-of-the-box default, conf
  • The poller fails to load limited.yaml because that file does not exist in conf

fas8060 examples

example 1

go run cmd/poller/poller.go --poller fas8060

2023-08-25T12:46:13-04:00 INF poller/poller.go:204 > Init Poller=fas8060 asup=false confPath=/tmp/foo:conf config=harvest.yml daemon=false debug=false homePath= hostname=cgrindst-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=0 version="harvest version 2.0.2-rc2 (commit HEAD) (build date undefined) darwin/amd64"
2023-08-25T12:46:18-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:Aggregate path=/tmp/foo/zapi/cdot/9.8.0/aggr.yaml v=9.8.0
2023-08-25T12:46:30-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:Volume path=/tmp/foo/zapi/cdot/9.8.0/volume.yaml v=9.8.0
2023-08-25T12:46:30-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:Volume path=/tmp/foo/zapi/cdot/9.8.0/custom_volume.yaml v=9.8.0
2023-08-25T12:45:40-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:LIF path=conf/zapi/cdot/9.8.0/lif.yaml v=9.8.0
...

Notes:

  • The poller's yaml section includes conf_path: /tmp/foo:conf which means the /tmp/foo directory will be searched followed by the conf directory
  • Zapi:Aggregate and Zapi:Volume are read from /tmp/foo/zapi/cdot/ since that path is consulted first and templates exist there.
  • The Zapi:Volume template is merged with the template from /tmp/foo/zapi/cdot/9.8.0/custom_volume.yaml since /tmp/foo/zapi/custom.yaml overrides the Volume template.
  • All other templates are loaded from the conf directory since those templates do not exist in /tmp/foo

example 2

go run cmd/poller/poller.go --poller fas8060 --confpath /tmp/foo

2023-08-25T13:07:33-04:00 INF poller/poller.go:204 > Init Poller=fas8060 asup=false confPath=/tmp/foo config=harvest.yml daemon=false debug=false homePath= hostname=cgrindst-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=0 version="harvest version 2.0.2-rc2 (commit HEAD) (build date undefined) darwin/amd64"
2023-08-25T13:07:36-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:Volume path=/tmp/foo/zapi/cdot/9.8.0/volume.yaml v=9.8.0
2023-08-25T13:07:36-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:Volume path=/tmp/foo/zapi/cdot/9.8.0/custom_volume.yaml v=9.8.0
2023-08-25T13:07:41-04:00 INF collector/helpers.go:80 > best-fit template Poller=fas8060 collector=Zapi:Aggregate path=/tmp/foo/zapi/cdot/9.8.0/aggr.yaml v=9.8.0
2023-08-25T13:07:42-04:00 WRN poller/poller.go:667 > init collector-object error="no best-fit template for lun.yaml on /tmp/foo" Poller=fas8060 collector=Zapi object=Lun
...

Notes:

  • The confpath is /tmp/foo and since command line arguments override harvest.yml values. The confpath will be /tmp/foo
  • Zapi:Aggregate and Zapi:Volume are read from /tmp/foo/zapi/cdot/
  • All other templates fail to load because they do not exist in /tmp/foo

@rahulguptajss rahulguptajss reopened this Sep 4, 2023
@cgrinds cgrinds linked a pull request Oct 3, 2023 that will close this issue
feat: Harvest should load templates from a set of conf directories #2394
Merged
@Hardikl
Copy link
Contributor

Hardikl commented Nov 13, 2023

Tested in release branch 23.11 with commit eeab975

harvest % tree testconf*
testconf1
└── zapi
    ├── cdot
    │   └── 9.8.0
    │       ├── aggr2.yaml
    │       └── volume2.yaml
    └── default.yaml
testconf2
└── zapi
    ├── cdot
    │   └── 9.9.0
    │       └── svm2.yaml
    └── default.yaml
testconf3
└── zapi
    ├── cdot
    │   └── 9.8.0
    │       └── aggr3.yaml
    |       └── node1.yaml
    └── custom.yaml
testconf4
└── zapi
    ├── cdot
    │   └── 9.8.0
    │       ├── aggr4.yaml
    │       └── aggr5.yaml
    ├── custom.yaml
    ├── default.yaml
    └── limited.yaml
testconf5
└── zapi

harvest % cat testconf1/zapi/default.yaml 
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:              aggr2.yaml
  Volume:                 volume2.yaml     

harvest % cat testconf2/zapi/default.yaml
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  SVM:                 svm2.yaml     

harvest % cat testconf3/zapi/custom.yaml 
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:              aggr3.yaml
  Node:                   node1.yaml

testconf4/zapi/default.yaml is same as conf/zapi/default.yaml

harvest % cat testconf4/zapi/custom.yaml 
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:                   aggr4.yaml
  Namespace:                   namespace.yaml
  Node:                        node.yaml
  NtpServer:                   ntpserver.yaml
  QosPolicyAdaptive:           qos_policy_adaptive.yaml
  QosPolicyFixed:              qos_policy_fixed.yaml
  QosWorkload:                 qos_workload.yaml                                                                                                                                                                      

harvest % cat testconf4/zapi/limited.yaml 
collector:          Zapi

# Order here matters!
schedule:
  - data: 3m

objects:
  Aggregate:              aggr5.yaml

Case 1: conf path from harvest poller
harvest.yml file poller detail

  sarzapi: 
    datacenter: zapi
    addr: 10.193.48.154
    password: xxxxxxx
    collectors:
      - Zapi
    conf_path: testconf1:conf

harvest % ./bin/poller -p sarzapi --promPort 13001                                                                          
2023-11-13T16:22:43+05:30 INF poller/poller.go:215 > Init Poller=sarzapi asup=false confPath=testconf1:conf config=harvest.yml configPath=harvest.yml daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=13001 version="harvest version 23.11.1313-v23.08.0 (commit eeab9752) (build date 2023-11-13T13:51:15+0530) darwin/amd64"
2023-11-13T16:22:43+05:30 INF poller/poller.go:239 > started in foreground Poller=sarzapi pid=24983
2023-11-13T16:22:47+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Namespace path=conf/zapi/cdot/9.8.0/namespace.yaml v=9.13.1
2023-11-13T16:22:48+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:NtpServer path=conf/zapi/cdot/9.8.0/ntpserver.yaml v=9.13.1
2023-11-13T16:22:49+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Security path=conf/zapi/cdot/9.8.0/security.yaml v=9.13.1
2023-11-13T16:22:52+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityAccount path=conf/zapi/cdot/9.8.0/security_account.yaml v=9.13.1
2023-11-13T16:22:53+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityCert path=conf/zapi/cdot/9.8.0/security_certificate.yaml v=9.13.1
2023-11-13T16:22:55+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Shelf path=conf/zapi/cdot/9.8.0/shelf.yaml v=9.13.1
--> 2023-11-13T16:22:57+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Volume path=testconf1/zapi/cdot/9.8.0/volume2.yaml v=9.13.1 -->
2023-11-13T16:22:59+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:AggregateEfficiency path=conf/zapi/cdot/9.9.0/aggr_efficiency.yaml v=9.13.1
2023-11-13T16:23:00+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SnapMirror path=conf/zapi/cdot/9.8.0/snapmirror.yaml v=9.13.1
2023-11-13T16:23:03+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Qtree path=conf/zapi/cdot/9.8.0/qtree.yaml v=9.13.1
2023-11-13T16:23:05+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosPolicyAdaptive path=conf/zapi/cdot/9.8.0/qos_policy_adaptive.yaml v=9.13.1
2023-11-13T16:23:06+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:EmsDestination path=conf/zapi/cdot/9.8.0/ems_destination.yaml v=9.13.1
2023-11-13T16:23:07+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityAuditDestination path=conf/zapi/cdot/9.8.0/security_audit_dest.yaml v=9.13.1
2023-11-13T16:23:08+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Subsystem path=conf/zapi/cdot/9.8.0/subsystem.yaml v=9.13.1
2023-11-13T16:23:09+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:CloudTarget path=conf/zapi/cdot/9.10.0/aggr_object_store_config.yaml v=9.13.1
2023-11-13T16:23:10+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:ClusterPeer path=conf/zapi/cdot/9.8.0/clusterpeer.yaml v=9.13.1
2023-11-13T16:23:11+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosWorkload path=conf/zapi/cdot/9.8.0/qos_workload.yaml v=9.13.1
2023-11-13T16:23:12+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Sensor path=conf/zapi/cdot/9.8.0/sensor.yaml v=9.13.1
2023-11-13T16:23:14+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SVM path=conf/zapi/cdot/9.8.0/svm.yaml v=9.13.1
--> 2023-11-13T16:23:16+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=testconf1/zapi/cdot/9.8.0/aggr2.yaml v=9.13.1 -->
2023-11-13T16:23:18+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Node path=conf/zapi/cdot/9.8.0/node.yaml v=9.13.1
2023-11-13T16:23:20+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityLogin path=conf/zapi/cdot/9.8.0/security_login.yaml v=9.13.1
2023-11-13T16:23:21+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SnapshotPolicy path=conf/zapi/cdot/9.8.0/snapshotpolicy.yaml v=9.13.1
2023-11-13T16:23:22+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Status path=conf/zapi/cdot/9.8.0/status.yaml v=9.13.1

Explanation:

  • Aggr and Volume object loaded from testconf1 conf whereas remaining object loaded from conf.

Case 2: limited yaml from harvest poller section with multiple conf path in cli

  sarzapi: 
    datacenter: zapi
    addr: 10.193.48.154
    password: xxxxxxx
    collectors:
      - Zapi:
        - limited.yaml

harvest % ./bin/poller -p sarzapi --promPort 13001 --confpath testconf5:testconf4:conf
2023-11-13T16:33:09+05:30 INF poller/poller.go:215 > Init Poller=sarzapi asup=false confPath=testconf5:testconf4:conf config=harvest.yml configPath=harvest.yml daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=13001 version="harvest version 23.11.1313-v23.08.0 (commit eeab9752) (build date 2023-11-13T13:51:15+0530) darwin/amd64"
2023-11-13T16:33:09+05:30 INF poller/poller.go:239 > started in foreground Poller=sarzapi pid=25374
2023-11-13T16:33:12+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=testconf4/zapi/cdot/9.8.0/aggr5.yaml v=9.13.1
2023-11-13T16:33:13+05:30 INF poller/poller.go:357 > Autosupport scheduled. Poller=sarzapi asupSchedule=24h
2023-11-13T16:33:13+05:30 INF poller/poller.go:366 > poller start-up complete Poller=sarzapi

Explanation:

  • Only Aggr object loaded from testconf4 conf due to limited.yaml applied in harvest.yml.

Case 3: limited yaml from harvest poller section with multiple conf path in cli

harvest % ./bin/poller -p sarzapi --promPort 13001 --confpath testconf1:testconf2:testconf3:testconf5:testconf4:conf
2023-11-13T16:35:09+05:30 INF poller/poller.go:215 > Init Poller=sarzapi asup=false confPath=testconf1:testconf2:testconf3:testconf5:testconf4:conf config=harvest.yml configPath=harvest.yml daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=13001 version="harvest version 23.11.1313-v23.08.0 (commit eeab9752) (build date 2023-11-13T13:51:15+0530) darwin/amd64"
2023-11-13T16:35:09+05:30 INF poller/poller.go:239 > started in foreground Poller=sarzapi pid=25425
2023-11-13T16:35:11+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=testconf4/zapi/cdot/9.8.0/aggr5.yaml v=9.13.1
2023-11-13T16:35:12+05:30 INF poller/poller.go:357 > Autosupport scheduled. Poller=sarzapi asupSchedule=24h
2023-11-13T16:35:12+05:30 INF prometheus/httpd.go:40 > server listen Poller=sarzapi exporter=prometheus1 url=http://:13001/metrics
2023-11-13T16:35:12+05:30 INF poller/poller.go:366 > poller start-up complete Poller=sarzapi

Explanation:

  • Only Aggr object loaded from testconf4 conf due to limited.yaml applied in harvest.yml.

Case 4: normal poller definition from harvest.yml with multiple conf path in cli

  sarzapi: 
    datacenter: zapi
    addr: 10.193.48.154
    password: xxxxxxx
    collectors:
      - Zapi

harvest % ./bin/poller -p sarzapi --promPort 13001 --confpath testconf1:testconf2:testconf3:testconf5:testconf4:conf
2023-11-13T16:36:21+05:30 INF poller/poller.go:215 > Init Poller=sarzapi asup=false confPath=testconf1:testconf2:testconf3:testconf5:testconf4:conf config=harvest.yml configPath=harvest.yml daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=13001 version="harvest version 23.11.1313-v23.08.0 (commit eeab9752) (build date 2023-11-13T13:51:15+0530) darwin/amd64"
2023-11-13T16:36:21+05:30 INF poller/poller.go:239 > started in foreground Poller=sarzapi pid=25460
2023-11-13T16:36:24+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SnapMirror path=conf/zapi/cdot/9.8.0/snapmirror.yaml v=9.13.1
2023-11-13T16:36:26+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:ClusterPeer path=conf/zapi/cdot/9.8.0/clusterpeer.yaml v=9.13.1
--> 2023-11-13T16:36:27+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Node path=conf/zapi/cdot/9.8.0/node.yaml v=9.13.1 -->
--> 2023-11-13T16:36:27+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Node path=testconf3/zapi/cdot/9.8.0/node1.yaml v=9.13.1 --> 
2023-11-13T16:36:28+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosWorkload path=conf/zapi/cdot/9.8.0/qos_workload.yaml v=9.13.1
2023-11-13T16:36:29+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Security path=conf/zapi/cdot/9.8.0/security.yaml v=9.13.1
2023-11-13T16:36:32+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityLogin path=conf/zapi/cdot/9.8.0/security_login.yaml v=9.13.1
2023-11-13T16:36:33+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Sensor path=conf/zapi/cdot/9.8.0/sensor.yaml v=9.13.1
2023-11-13T16:36:35+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SnapshotPolicy path=conf/zapi/cdot/9.8.0/snapshotpolicy.yaml v=9.13.1
2023-11-13T16:36:36+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Status path=conf/zapi/cdot/9.8.0/status.yaml v=9.13.1
2023-11-13T16:36:37+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:CloudTarget path=conf/zapi/cdot/9.10.0/aggr_object_store_config.yaml v=9.13.1
2023-11-13T16:36:38+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityAccount path=conf/zapi/cdot/9.8.0/security_account.yaml v=9.13.1
2023-11-13T16:36:39+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityAuditDestination path=conf/zapi/cdot/9.8.0/security_audit_dest.yaml v=9.13.1
2023-11-13T16:36:40+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Shelf path=conf/zapi/cdot/9.8.0/shelf.yaml v=9.13.1
2023-11-13T16:36:44+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:AggregateEfficiency path=conf/zapi/cdot/9.9.0/aggr_efficiency.yaml v=9.13.1
2023-11-13T16:36:45+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Disk path=conf/zapi/cdot/9.8.0/disk.yaml v=9.13.1
2023-11-13T16:36:46+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Lun path=conf/zapi/cdot/9.8.0/lun.yaml v=9.13.1
2023-11-13T16:36:47+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:NtpServer path=conf/zapi/cdot/9.8.0/ntpserver.yaml v=9.13.1
2023-11-13T16:36:48+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Support path=conf/zapi/cdot/9.8.0/support.yaml v=9.13.1
2023-11-13T16:36:49+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SVM path=conf/zapi/cdot/9.8.0/svm.yaml v=9.13.1
--> 2023-11-13T16:36:52+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Volume path=testconf1/zapi/cdot/9.8.0/volume2.yaml v=9.13.1 -->
2023-11-13T16:36:54+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:LIF path=conf/zapi/cdot/9.8.0/lif.yaml v=9.13.1
2023-11-13T16:36:55+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosPolicyAdaptive path=conf/zapi/cdot/9.8.0/qos_policy_adaptive.yaml v=9.13.1
2023-11-13T16:36:56+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosPolicyFixed path=conf/zapi/cdot/9.8.0/qos_policy_fixed.yaml v=9.13.1
--> 2023-11-13T16:36:57+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=testconf1/zapi/cdot/9.8.0/aggr2.yaml v=9.13.1 --> 
--> 2023-11-13T16:36:57+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=testconf3/zapi/cdot/9.8.0/aggr3.yaml v=9.13.1 -->
2023-11-13T16:36:59+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Namespace path=conf/zapi/cdot/9.8.0/namespace.yaml v=9.13.1
2023-11-13T16:37:01+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityCert path=conf/zapi/cdot/9.8.0/security_certificate.yaml v=9.13.1
2023-11-13T16:37:03+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Subsystem path=conf/zapi/cdot/9.8.0/subsystem.yaml v=9.13.1
2023-11-13T16:37:04+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:EmsDestination path=conf/zapi/cdot/9.8.0/ems_destination.yaml v=9.13.1
2023-11-13T16:37:05+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Qtree path=conf/zapi/cdot/9.8.0/qtree.yaml v=9.13.1
2023-11-13T16:37:06+05:30 INF prometheus/httpd.go:40 > server listen Poller=sarzapi exporter=prometheus1 url=http://:13001/metrics
2023-11-13T16:37:06+05:30 INF poller/poller.go:357 > Autosupport scheduled. Poller=sarzapi asupSchedule=24h
2023-11-13T16:37:06+05:30 INF poller/poller.go:366 > poller start-up complete Poller=sarzapi

Explanation:

  • Aggr and Volume object loaded from testconf1 conf
  • Node object loaded from conf as no relevant file from testconf.
  • Aggr and Node object loaded from testconf3 due to custom.yaml
  • Remaining objects loaded from conf.

Case 5: custom.yaml with conf

harvest % ./bin/poller -p sarzapi --promPort 13001 --confpath testconf3:conf                    
2023-11-13T17:06:29+05:30 INF poller/poller.go:215 > Init Poller=sarzapi asup=false confPath=testconf3:conf config=harvest.yml configPath=harvest.yml daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=13001 version="harvest version 23.11.1313-v23.08.0 (commit eeab9752) (build date 2023-11-13T13:51:15+0530) darwin/amd64"
2023-11-13T17:06:29+05:30 INF poller/poller.go:239 > started in foreground Poller=sarzapi pid=26395
2023-11-13T17:06:31+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:EmsDestination path=conf/zapi/cdot/9.8.0/ems_destination.yaml v=9.13.1
2023-11-13T17:06:32+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Lun path=conf/zapi/cdot/9.8.0/lun.yaml v=9.13.1
2023-11-13T17:06:33+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosPolicyFixed path=conf/zapi/cdot/9.8.0/qos_policy_fixed.yaml v=9.13.1
2023-11-13T17:06:34+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Security path=conf/zapi/cdot/9.8.0/security.yaml v=9.13.1
2023-11-13T17:06:37+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityCert path=conf/zapi/cdot/9.8.0/security_certificate.yaml v=9.13.1
2023-11-13T17:06:39+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Volume path=conf/zapi/cdot/9.8.0/volume.yaml v=9.13.1
2023-11-13T17:06:41+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:NtpServer path=conf/zapi/cdot/9.8.0/ntpserver.yaml v=9.13.1
2023-11-13T17:06:42+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityAccount path=conf/zapi/cdot/9.8.0/security_account.yaml v=9.13.1
2023-11-13T17:06:43+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityAuditDestination path=conf/zapi/cdot/9.8.0/security_audit_dest.yaml v=9.13.1
--> 2023-11-13T17:06:45+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Node path=conf/zapi/cdot/9.8.0/node.yaml v=9.13.1. --> 
--> 2023-11-13T17:06:45+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Node path=testconf3/zapi/cdot/9.8.0/node1.yaml v=9.13.1. --> 
2023-11-13T17:06:46+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosPolicyAdaptive path=conf/zapi/cdot/9.8.0/qos_policy_adaptive.yaml v=9.13.1
2023-11-13T17:06:47+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Qtree path=conf/zapi/cdot/9.8.0/qtree.yaml v=9.13.1
2023-11-13T17:06:49+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityLogin path=conf/zapi/cdot/9.8.0/security_login.yaml v=9.13.1
2023-11-13T17:06:50+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SnapMirror path=conf/zapi/cdot/9.8.0/snapmirror.yaml v=9.13.1
2023-11-13T17:06:52+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Status path=conf/zapi/cdot/9.8.0/status.yaml v=9.13.1
2023-11-13T17:06:53+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Support path=conf/zapi/cdot/9.8.0/support.yaml v=9.13.1
--> 2023-11-13T17:06:54+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=conf/zapi/cdot/9.8.0/aggr.yaml v=9.13.1 --> 
--> 2023-11-13T17:06:54+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Aggregate path=testconf3/zapi/cdot/9.8.0/aggr3.yaml v=9.13.1 --> 
2023-11-13T17:06:57+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:ClusterPeer path=conf/zapi/cdot/9.8.0/clusterpeer.yaml v=9.13.1
2023-11-13T17:06:58+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Disk path=conf/zapi/cdot/9.8.0/disk.yaml v=9.13.1
2023-11-13T17:06:59+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Namespace path=conf/zapi/cdot/9.8.0/namespace.yaml v=9.13.1
2023-11-13T17:07:00+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Subsystem path=conf/zapi/cdot/9.8.0/subsystem.yaml v=9.13.1
2023-11-13T17:07:01+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:AggregateEfficiency path=conf/zapi/cdot/9.9.0/aggr_efficiency.yaml v=9.13.1
2023-11-13T17:07:02+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:CloudTarget path=conf/zapi/cdot/9.10.0/aggr_object_store_config.yaml v=9.13.1
2023-11-13T17:07:03+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:LIF path=conf/zapi/cdot/9.8.0/lif.yaml v=9.13.1
2023-11-13T17:07:04+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Shelf path=conf/zapi/cdot/9.8.0/shelf.yaml v=9.13.1

Explanation:

  • Aggr and Node object loaded from testconf3 due to custom.yaml
  • Remaining objects loaded from conf.

Case 6: without any confpath, default conf would be loaded

harvest % ./bin/poller -p sarzapi --promPort 13001                          
2023-11-13T17:10:43+05:30 INF poller/poller.go:215 > Init Poller=sarzapi asup=false confPath=conf config=harvest.yml configPath=harvest.yml daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=13001 version="harvest version 23.11.1313-v23.08.0 (commit eeab9752) (build date 2023-11-13T13:51:15+0530) darwin/amd64"
2023-11-13T17:10:43+05:30 INF poller/poller.go:239 > started in foreground Poller=sarzapi pid=26505
2023-11-13T17:10:46+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:CloudTarget path=conf/zapi/cdot/9.10.0/aggr_object_store_config.yaml v=9.13.1
2023-11-13T17:10:47+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:QosWorkload path=conf/zapi/cdot/9.8.0/qos_workload.yaml v=9.13.1
2023-11-13T17:10:48+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Qtree path=conf/zapi/cdot/9.8.0/qtree.yaml v=9.13.1
2023-11-13T17:10:50+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:SecurityLogin path=conf/zapi/cdot/9.8.0/security_login.yaml v=9.13.1
2023-11-13T17:10:51+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Sensor path=conf/zapi/cdot/9.8.0/sensor.yaml v=9.13.1
2023-11-13T17:10:53+05:30 INF collector/helpers.go:84 > best-fit template Poller=sarzapi collector=Zapi:Shelf path=conf/zapi/cdot/9.8.0/shelf.yaml v=9.13.1
...

Explanation:

  • It uses the out-of-the-box default conf.

@cgrinds cgrinds added 24.02 and removed 23.11 labels Nov 13, 2023
@cgrinds cgrinds removed the 24.02 label Feb 7, 2024
@cgrinds cgrinds added the 24.05 label Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
24.05 feature New feature or request status/done
Projects
None yet
Milestone
No milestone
3 participants
@cgrinds @rahulguptajss @Hardikl