rpm install in a syslog-ng environment

[rkomandu]

Environment info

• NooBaa Version: VERSION
• Platform: Kubernetes 1.14.1 | minikube 1.1.1 | OpenShift 4.1 | other: specify

Noobaa 5.17.0-20240921 d/s build

Actual behavior

When we install a noobaa rpm with the default rsyslog enabled in the system, it restarts which is good. However if we have the
syslog-ng enabled in the cluster there is no message in the stdout and the syslog-ng is not restarted. I think we need to have this check in the system

[root@rk522-33 ~]# systemctl stop rsyslog

[root@rk522-33 ~]# systemctl disable rsyslog

check syslog-ng and rsyslog 

[root@rk522-33 ~]# systemctl start syslog-ng
<bunch of messages have come into the stdout/stderr or console>

[root@rk522-33 ~]# systemctl status rsyslog
● rsyslog.service - System Logging Service
   Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:rsyslogd(8)
           https://www.rsyslog.com/doc/

Sep 24 09:53:15 rk522-33 systemd[1]: Stopped System Logging Service.
Sep 24 09:53:15 rk522-33 systemd[1]: Starting System Logging Service...
Sep 24 09:53:15 rk522-33 rsyslogd[60234]: [origin software="rsyslogd" swVersion="8.2102.0-15.el8" x-pid="60234" x-info="https://www.rsyslog.com"] start
Sep 24 09:53:15 rk522-33 systemd[1]: Started System Logging Service.
Sep 24 09:53:15 rk522-33 rsyslogd[60234]: imjournal: journal files changed, reloading...  [v8.2102.0-15.el8 try https://www.rsyslog.com/e/0 ]
Sep 24 10:43:01 rk522-33 rsyslogd[60234]: [origin software="rsyslogd" swVersion="8.2102.0-15.el8" x-pid="60234" x-info="https://www.rsyslog.com"] rsyslogd wa>
Sep 25 07:34:01 rk522-33 systemd[1]: Stopping System Logging Service...
Sep 25 07:34:01 rk522-33 rsyslogd[60234]: [origin software="rsyslogd" swVersion="8.2102.0-15.el8" x-pid="60234" x-info="https://www.rsyslog.com"] exiting on >
Sep 25 07:34:01 rk522-33 systemd[1]: rsyslog.service: Succeeded.
Sep 25 07:34:01 rk522-33 systemd[1]: Stopped System Logging Service.


[root@rk522-33 ~]# systemctl status syslog-ng
● syslog-ng.service - System Logger Daemon
   Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2024-09-25 07:34:34 CEST; 49s ago
     Docs: man:syslog-ng(8)
 Main PID: 2810330 (syslog-ng)
    Tasks: 2 (limit: 49038)
   Memory: 4.1M
   CGroup: /system.slice/syslog-ng.service
           └─2810330 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid

Sep 25 07:34:34 rk522-33 systemd[1]: Starting System Logger Daemon...
Sep 25 07:34:34 rk522-33 systemd[1]: Started System Logger Daemon.

5) Installed the noobaa 5.17.0-20240921 rpm on all the nodes that have the rsyslog and syslog-ng 

[root@rk522-33 ~]# rpm -ivh /root/noobaa-core-5.17.0-20240921.el8.x86_64.rpm  (where syslog-ng is enabled)
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:noobaa-core-5.17.0-20240921.el8  ################################# [100%]
**NOTE: THERE IS NO RESTART OF THE SERVICE HERE (SYSLOG-NG)** 

[root@rk522-31 ~]# rpm -ivh /root/noobaa-core-5.17.0-20240921.el8.x86_64.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:noobaa-core-5.17.0-20240921.el8  ################################# [100%]
Redirecting to /bin/systemctl restart rsyslog.service

Expected behavior

IMO, based on the underlying service (rsyslog, syslog-ng), it should be restarted. Otherwise it should display that please start your logging service etc.

Steps to reproduce

Posted above

More information - Screenshots / Logs / Other output

[rkomandu]

this would also become a problem when we do rpm upgrade in the syslog-ng environment.

[nimrod-becker]

@rkomandu why is that a NooBaa related issue? An easy fix for example would be to restart it in the Scale upgrade process after updating the PRM

[rkomandu]

@nimrod-becker , the way when rpm is installed the underlying service should be started. Given that we are trying to address other logging mechanism (syslog-ng/some other) , atleast there should be a message that restart the respective logging since rsyslog is not enabled in the cluster.

Scale upgrade process doesn't touch these things (just like what was done with rsyslog today) since rpm post script is restarting the daemon

-- Also if customer for efix they can't use the Toolkit upgrade, for them the service has to be restarted.
-- There are manually deployed / upgraded customers who will need to call this explicitly.

There are few different ways of upgrade is what am trying to say.

[nimrod-becker]

For efix, they would get instructions, which can include a comment to restart this.

I am not sure other services when deployed on linux are restarting logging services, that would sound intrusive to me...

[romayalon]

@nimrod-becker @rkomandu @guymguym @naveenpaul1 @tangledbytes
I found inside noobaa.spec file that we restart rsyslog still -

%post
state=$(systemctl show -p ActiveState --value rsyslog)
if [ "${state}" == "active" ]; then
  service rsyslog restart
fi

https://github.com/noobaa/noobaa-core/blob/master/src/deploy/RPM_build/noobaa.spec
should this be removed as well as part of the decoupling between NooBaa and syslog services?

[naveenpaul1]

@romayalon I think we should keep this, because we anyway shipping rsyslog config with RPM, If we remove restart there is no point in keeping rsyslog configuration
For syslog-ng its not the case, complete configuration needs to be done from the customer side and they need to restart after config changes. and we dont know when they will update the config for syslog-ng
@rkomandu

[romayalon]

@madhuthorat @rkomandu due to the fact that you change the default to LOG_TO_SYSLOG_ENABLED=true, I think we should keep the restart, WDYT?