Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gave up: nginx entered FATAL state, too many start retries too quickly #3258

Open
1 task done
nexcode opened this issue Sep 9, 2023 · 13 comments
Open
1 task done
Labels
bug bw-unified-deploy An Issue related to Bitwarden unified deployment

Comments

@nexcode
Copy link

nexcode commented Sep 9, 2023

Steps To Reproduce

Make a config in which nginx starts on 80 and 443 ports:

version: "3.9"
services:
  bitwarden:
    image: bitwarden/self-host:2023.8.2-beta
    environment:
      BW_PORT_HTTP: 80
      BW_PORT_HTTPS: 443

Expected Result

Normal nginx startup!

Actual Result

bitwarden  | 2023-09-09 12:22:24,452 INFO supervisord started with pid 1
bitwarden  | 2023-09-09 12:22:25,456 INFO spawned: 'identity' with pid 74
bitwarden  | 2023-09-09 12:22:25,463 INFO spawned: 'admin' with pid 75
bitwarden  | 2023-09-09 12:22:25,476 INFO spawned: 'api' with pid 76
bitwarden  | 2023-09-09 12:22:25,485 INFO spawned: 'icons' with pid 77
bitwarden  | 2023-09-09 12:22:25,502 INFO spawned: 'nginx' with pid 78
bitwarden  | 2023-09-09 12:22:25,517 INFO spawned: 'notifications' with pid 79
bitwarden  | 2023-09-09 12:22:26,631 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-09-09 12:22:27,633 INFO spawned: 'nginx' with pid 117
bitwarden  | 2023-09-09 12:22:27,810 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-09-09 12:22:30,263 INFO spawned: 'nginx' with pid 139
bitwarden  | 2023-09-09 12:22:31,346 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-09-09 12:22:34,673 INFO spawned: 'nginx' with pid 152
bitwarden  | 2023-09-09 12:22:35,701 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-09-09 12:22:36,703 INFO gave up: nginx entered FATAL state, too many start retries too quickly

Screenshots or Videos

No response

Additional Context

No response

Githash Version

doesn't work because nginx didn't start

Environment Details

No response

Database Image

No response

Issue-Link

#2480

Issue Tracking Info

  • I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.
@nexcode nexcode added bug bw-unified-deploy An Issue related to Bitwarden unified deployment labels Sep 9, 2023
@sso-bitwarden
Copy link

Hi @nexcode

Thank you for your report. There should be no issue setting up the port to 80 & 443, even though the default is 8080 & 8443. There must be something else causing the error. The usual culprit is usually the SSL certs. You can find out the exact cause by entering the container and check the nginx error logs in /var/log/nginx/error.log.

Please get in touch with our support team for further assistance.

https://bitwarden.com/help/

@nexcode
Copy link
Author

nexcode commented Sep 11, 2023

Hi @sso-bitwarden

The container is built in such a way that nginx does not have the rights to work on ports below 1024. This has nothing to do with ssl or anything like that. You just need to fix this by adding root rights for nginx because that’s how it should be run: #3097

@nexcode
Copy link
Author

nexcode commented Oct 2, 2023

@sso-bitwarden say something? This is a very important note for many users.

@sso-bitwarden
Copy link

Hi @nexcode

Since my colleague Troy in the other issue and I couldn't reproduce this, I'll leave the issue open for the engineering team to take a look. In my installation, I had no issue setting the port to 80 443.

By the way, any reason why you want the Nginx port on 80 & 443? These are just the Nginx listening ports inside the container. Externally, you could always map 80 -> 8080 & 443 -> 8443.

@nexcode
Copy link
Author

nexcode commented Oct 9, 2023

In the environment in which I want to use this, a host network is used, therefore, for convenience, in order not to configure forwarding within the operating system itself, I need to use ports 80 and 443 in the container.

@nexcode
Copy link
Author

nexcode commented Oct 9, 2023

I specified latest version of the image: bitwarden/self-host:2023.9.1-beta

docker compose logs:

bitwarden  | 2023-10-09 11:54:41,743 INFO supervisord started with pid 1
bitwarden  | 2023-10-09 11:54:42,748 INFO spawned: 'identity' with pid 69
bitwarden  | 2023-10-09 11:54:42,754 INFO spawned: 'admin' with pid 70
bitwarden  | 2023-10-09 11:54:42,766 INFO spawned: 'api' with pid 71
bitwarden  | 2023-10-09 11:54:42,782 INFO spawned: 'icons' with pid 72
bitwarden  | 2023-10-09 11:54:42,799 INFO spawned: 'nginx' with pid 73
bitwarden  | 2023-10-09 11:54:42,815 INFO spawned: 'notifications' with pid 74
bitwarden  | 2023-10-09 11:54:43,973 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-10-09 11:54:44,981 INFO spawned: 'nginx' with pid 111
bitwarden  | 2023-10-09 11:54:45,422 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-10-09 11:54:47,665 INFO spawned: 'nginx' with pid 128
bitwarden  | 2023-10-09 11:54:47,796 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-10-09 11:54:51,758 INFO spawned: 'nginx' with pid 147
bitwarden  | 2023-10-09 11:54:52,817 INFO exited: nginx (exit status 1; not expected)
bitwarden  | 2023-10-09 11:54:53,820 INFO gave up: nginx entered FATAL state, too many start retries too quickly

/var/log/nginx/error.log:

2023/10/09 11:54:42 [emerg] 73#73: bind() to 0.0.0.0:80 failed (13: Permission denied)
2023/10/09 11:54:45 [emerg] 111#111: bind() to 0.0.0.0:80 failed (13: Permission denied)
2023/10/09 11:54:47 [emerg] 128#128: bind() to 0.0.0.0:80 failed (13: Permission denied)
2023/10/09 11:54:51 [emerg] 147#147: bind() to 0.0.0.0:80 failed (13: Permission denied)

@nexcode
Copy link
Author

nexcode commented Oct 9, 2023

To fix this you need open entrypoint.sh and replace the last line with:

exec setpriv --reuid=0 --regid=$PGID --init-groups /usr/bin/supervisord

Now, if you still want to run other services from the user bitwarden, you need to add to the supervisor ini files in /etc/supervisor.d/ line user=bitwarden. But not for the nginx.ini file.

Example:

[program:admin]
user=bitwarden

@nexcode
Copy link
Author

nexcode commented Oct 9, 2023

@sso-bitwarden, I hope I have described in as much detail as possible how to fix this :)

@nexcode
Copy link
Author

nexcode commented Oct 9, 2023

nginx is designed to run as root, and when started it starts listening on certain ports and then lowers its privileges. Therefore, it must be run as root user.

@nexcode
Copy link
Author

nexcode commented Oct 19, 2023

@sso-bitwarden, is there any news?

@kamilos956
Copy link

kamilos956 commented Nov 11, 2023

I have the same problem, even after changed entrypoint.sh

2023-11-11 10:16:21,492 INFO exited: nginx (exit status 1; not expected)
2023-11-11 10:16:22,544 INFO spawned: 'nginx' with pid 137
2023-11-11 10:16:22,698 INFO exited: nginx (exit status 1; not expected)
2023-11-11 10:16:24,706 INFO spawned: 'nginx' with pid 140
2023-11-11 10:16:24,721 INFO exited: nginx (exit status 1; not expected)
2023-11-11 10:16:27,730 INFO spawned: 'nginx' with pid 142
2023-11-11 10:16:27,747 INFO exited: nginx (exit status 1; not expected)
2023-11-11 10:16:28,749 INFO gave up: nginx entered FATAL state, too many start retries too quickly
2023-11-11 10:16:36,760 INFO success: identity entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)
2023-11-11 10:16:36,761 INFO success: admin entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)
2023-11-11 10:16:36,761 INFO success: api entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)
2023-11-11 10:16:36,761 INFO success: icons entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)
2023-11-11 10:16:36,762 INFO success: notifications entered RUNNING state, process has stayed up for > than 15 seconds (startsecs)

@nexcode
Copy link
Author

nexcode commented Nov 13, 2023

@kamilos956, you need to look at the file /var/log/nginx/error.log inside the container. Perhaps the port is already in use by someone.

@kamilos956
Copy link

Yeah, right, I had to setup SMTP and it started working properly. Thank you :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug bw-unified-deploy An Issue related to Bitwarden unified deployment
Projects
None yet
Development

No branches or pull requests

3 participants