5.1.1 misses psconfig pscheduler-stats under Ubuntu 20

[szymontrocha]

A system after 5.1.1 upgrade with Ubuntu 20 toolkit. This results in Grafana empty graphs:

# tail -f syslog|grep CRON
Jul  9 07:55:01 psmall-poz1 CRON[450915]: (prometheus) CMD (psconfig pscheduler-stats --format prometheus | sponge /var/lib/prometheus/node-exporter/perfsonar-psconfig-pscheduler-stats.prom)
Jul  9 07:55:01 psmall-poz1 CRON[450916]: (prometheus) CMD (pscheduler metrics --format prometheus | sponge /var/lib/prometheus/node-exporter/perfsonar-pscheduler.prom)
Jul  9 07:55:01 psmall-poz1 CRON[450912]: (CRON) info (No MTA installed, discarding output)

root@psmall-poz1:/var/log# psconfig pscheduler-stats
pscheduler-stats: Unknown command.  Use --help for help.
root@psmall-poz1:/var/log# 

# apt list --installed |grep psconfig

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libperfsonar-psconfig-perl/perfsonar-release,now 5.1.1-1 all [installed,automatic]
perfsonar-psconfig-grafana/perfsonar-release,now 5.1.1-1 all [installed,automatic]
perfsonar-psconfig-pscheduler/perfsonar-release,now 5.1.1-1 all [installed,automatic]
perfsonar-psconfig-publisher/perfsonar-release,now 5.1.1-1 all [installed,automatic]
perfsonar-psconfig-utils/perfsonar-release,now 5.1.1-1 all [installed,automatic]
python3-perfsonar-psconfig/perfsonar-release,now 5.1.1-1 all [installed,automatic]
root@psmall-poz1:/var/log# 

$ pwd
/var/lib/prometheus/node-exporter
$ ls -la
total 12
drwxr-xr-x 2 prometheus prometheus 4096 Jul  9 09:58 .
drwxr-xr-x 3 prometheus prometheus 4096 Feb 20 10:29 ..
-rw-rw-r-- 1 prometheus prometheus 1150 Jul  9 09:58 perfsonar-pscheduler.prom
-rw-rw-r-- 1 prometheus prometheus    0 Jul  9 09:58 perfsonar-psconfig-pscheduler-stats.prom
$ 

[szymontrocha]

After some investigation seems this command changed synatx between 5.0.8 and 5.1:

in 5.0.8 it is psconfig pscheduler-stats:

$ psconfig --help
Usage: psconfig command [ arguments ]

Commands:  (Use 'command --help' for further help.)

    agentctl      pscheduler-migrat pscheduler-transl remote
    agents                pscheduler-stats  publish           translate
    lookup                pscheduler-tasks  published         validate

$

while in 5.1 it is psconfig stats pscheduler:

$ psconfig --help
Usage: psconfig command [ arguments ]

Commands:  (Use 'command --help' for further help.)

    agentctl      pscheduler-tasks  published         stats
    agents                publish           remote            validate

$

[laeti-tia]

Yes, this CLI change was not documented well enough in the release notes.

But your empty Grafana graphs are still a worry.

[szymontrocha]

I think this is not just docs as the CRON command is not invoked properly acording to the log. 5.1 tries to invoke "old" command

[szymontrocha]

here is more info. it seems that cron file contains wrong definition:

# pwd
/etc/cron.d
# cat psconfig-pscheduler-stats
#Ansible: Run psconfig pscheduler-stats for Prometheus
* * * * * prometheus psconfig pscheduler-stats --format prometheus | sponge /var/lib/prometheus/node-exporter/perfsonar-psconfig-pscheduler-stats.prom

I change it in this file to "psconfig stats pscheduler" and it started to work i.e. no error in log and perfsonar-psconfig-pscheduler-stats.prom is filled with data

Seems like upgrade from 5.0.8 didn't properly update the cron file

[laeti-tia]

Hello @szymontrocha I think this cronjob entry is not from perfSONAR itself but from the PMP Ansible playbooks. That's where it needs to be changed.

If I'm not mistaken, the only perfSONAR-related action is to properly document the new behaviour, which will be followed up in perfsonar/docs#258 I think this ticket can be closed. Feel free to open again if there is something I've missed.