rsyslogd: could not open config file '/var/lib/awx/rsyslog/rsyslog.conf': Permission denied

[jean-christophe-manciot]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that AWX is open source software provided for free and that I might not receive a timely response.
• I am NOT reporting a (potential) security vulnerability. (These should be emailed to [email protected] instead.)

Bug Summary

When starting all awx containers with make docker-compose, the issue rsyslogd: could not open config file '/var/lib/awx/rsyslog/rsyslog.conf': Permission denied pops up once all migrations are done.

AWX version

22.5.0

Select the relevant components

• UI
• UI (tech preview)
• API
• Docs
• Collection
• CLI
• Other

Installation method

docker development environment

Modifications

no

Ansible version

2.10.8

Operating system

Ubuntu 20.04/22.04

Web browser

No response

Steps to reproduce

su - awx
git clone -b 22.5.0 https://github.com/ansible/awx.git
cd awx
git switch -c release_4.4
export RECEPTOR_IMAGE=quay.io/ansible/receptor:latest
make docker-compose-build
make docker-compose &

Expected results

All AWX containers started without issue

Actual results

tools_awx_1       | awx-autoreload stdout | make[1]: Entering directory '/awx_devel'
tools_awx_1       | awx-autoreload stdout | 
tools_awx_1       | awx-rsyslogd stdout | rsyslogd: could not open config file '/var/lib/awx/rsyslog/rsyslog.conf': Permission denied [v8.2102.0-106.el9 try https://www.rsyslog.com/e/2104 ]
tools_awx_1       | awx-rsyslogd stdout | rsyslogd: run failed with error -2104 (see rsyslog.h or try https://www.rsyslog.com/e/2104 to learn what that number means)

Additional information

Full 'make docker-compose' log: could not open config file rsyslog.conf: Permission denied.log

[fosterseth]

what does ls -l show for that file if you exec into the tools_awx_1 container?

[jean-christophe-manciot]

Impossible to perform: tools_awx_1 is terminated a few seconds after being created.

[jean-christophe-manciot]

Is there a requirement regarding glibc version on the host? Should it match centos stream 9 version?

• host: 2.31
• centos stream 9: 2.34

[jean-christophe-manciot]

same issue with 22.6.0.

[jean-christophe-manciot]

Do the owner/group of the file in the git source folder matter?

[stage-1 18/33] ADD tools/ansible/roles/dockerfile/files/rsyslog.conf /var/lib/awx/rsyslog/rsyslog.conf

[Alabate]

I had the same issue on Ubuntu 23.04, it was comming from Ubuntu's apparmor config.

If I run sudo dmesg after the error on the host I get:

[147336.915240] audit: type=1400 audit(1697849807.174:348): apparmor="DENIED" operation="open" class="file" profile="rsyslogd" name="/var/lib/awx/rsyslog/rsyslog.conf" pid=281585 comm="rsyslogd" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

So I disabled the rsyslogd apparmor profile on the host:

sudo ln -s /etc/apparmor.d/usr.sbin.rsyslogd /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.rsyslogd