Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker container doesn't start #279

Open
johannesspohr opened this issue Dec 3, 2024 · 1 comment
Open

Docker container doesn't start #279

johannesspohr opened this issue Dec 3, 2024 · 1 comment

Comments

@johannesspohr
Copy link

After the issue with the SSL connection was fixed (thank you for the quick fix for that), my deployment unfortunately still doesn't come back to life.

I get this on the logs:

2024-12-03T20:26:49.898266396Z [W 2024-12-03 20:26:49.895 LabApp] Failed to instantiate the extension manager pypi. Falling back to read-only manager.
2024-12-03T20:26:49.898286236Z     Traceback (most recent call last):
2024-12-03T20:26:49.898289596Z       File "/app/jupyter/venv/lib/python3.11/site-packages/jupyterlab/labapp.py", line 837, in initialize_handlers
2024-12-03T20:26:49.898292076Z         ext_manager = manager_factory(app_options, listings_config, self)
2024-12-03T20:26:49.898301236Z                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-03T20:26:49.898304516Z       File "/app/jupyter/venv/lib/python3.11/site-packages/jupyterlab/extensions/__init__.py", line 46, in get_pypi_manager
2024-12-03T20:26:49.898320236Z         return PyPIExtensionManager(app_options, ext_options, parent)
2024-12-03T20:26:49.898322676Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-03T20:26:49.898324476Z       File "/app/jupyter/venv/lib/python3.11/site-packages/jupyterlab/extensions/pypi.py", line 134, in __init__
2024-12-03T20:26:49.898326596Z         self._httpx_client = httpx.AsyncClient(proxies=proxies)
2024-12-03T20:26:49.898328436Z                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

which seems to be related to this issue.

The Next application and API seem to run:

2024-12-03T20:26:46.964329557Z   ▲ Next.js 13.5.5
2024-12-03T20:26:46.964361758Z   - Local:        http://localhost:4000
2024-12-03T20:26:46.964364638Z   - Network:      http://0.0.0.0:4000
2024-12-03T20:26:46.964366678Z 
2024-12-03T20:26:46.964368518Z  ✓ Ready in 766ms
...
2024-12-03T20:26:49.962250275Z [I 2024-12-03 20:26:49.962 ServerApp] Jupyter Server 2.12.1 is running at:
2024-12-03T20:26:49.962431876Z [I 2024-12-03 20:26:49.962 ServerApp] http://70ded3768559:8888/?token=...
2024-12-03T20:26:49.962438516Z [I 2024-12-03 20:26:49.962 ServerApp]     http://127.0.0.1:8888/?token=...

After that it waits forever

2024-12-03T20:27:24.042623718Z Waiting for Web and API services to be reachable...
2024-12-03T20:27:26.077350562Z Waiting for Web and API services to be reachable...
2024-12-03T20:27:28.112611728Z Waiting for Web and API services to be reachable...
2024-12-03T20:27:30.142806838Z Waiting for Web and API services to be reachable...
...

It is working when I run it locally, but my coolify deployment has the issue. I tried pulling, redeploying without cache etc. without any success.
@johannesspohr
Copy link
Author

After deploying the latest version the error message is gone, but the container still doesn't start:

2024-12-16T08:36:48.626868663Z 2024-12-16 08:36:48,626 INFO Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
2024-12-16T08:36:48.633015443Z 2024-12-16 08:36:48,632 INFO RPC interface 'supervisor' initialized
2024-12-16T08:36:48.633406324Z 2024-12-16 08:36:48,633 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-12-16T08:36:48.633957686Z 2024-12-16 08:36:48,633 INFO supervisord started with pid 1
2024-12-16T08:36:49.641795304Z 2024-12-16 08:36:49,641 INFO spawned: 'postgres' with pid 8
2024-12-16T08:36:49.651314015Z 2024-12-16 08:36:49,650 INFO spawned: 'init_db' with pid 9
2024-12-16T08:36:49.658355717Z 2024-12-16 08:36:49,658 INFO spawned: 'setup' with pid 10
2024-12-16T08:36:49.685306283Z 2024-12-16 08:36:49,684 INFO spawned: 'jupyter' with pid 12
2024-12-16T08:36:49.694652273Z 2024-12-16 08:36:49,694 INFO spawned: 'api' with pid 14
2024-12-16T08:36:49.719750873Z 2024-12-16 08:36:49,717 INFO spawned: 'web' with pid 15
2024-12-16T08:36:49.736185406Z 2024-12-16 08:36:49,735 INFO spawned: 'nginx' with pid 16
2024-12-16T08:36:49.935329841Z 2024-12-16 08:36:49.927 UTC [8] LOG:  starting PostgreSQL 15.10 (Debian 15.10-0+deb12u1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2024-12-16T08:36:49.935358562Z 2024-12-16 08:36:49.931 UTC [8] LOG:  listening on IPv6 address "::1", port 5432
2024-12-16T08:36:49.935362642Z 2024-12-16 08:36:49.931 UTC [8] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2024-12-16T08:36:49.935959564Z 2024-12-16 08:36:49,935 INFO success: init_db entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-12-16T08:36:49.937924050Z 2024-12-16 08:36:49,937 INFO success: setup entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-12-16T08:36:49.937938650Z 2024-12-16 08:36:49,937 INFO success: jupyter entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-12-16T08:36:49.937942250Z 2024-12-16 08:36:49,937 INFO success: api entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-12-16T08:36:49.937945170Z 2024-12-16 08:36:49,937 INFO success: web entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-12-16T08:36:49.942447024Z 2024-12-16 08:36:49.938 UTC [8] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-12-16T08:36:49.954977584Z 2024-12-16 08:36:49.954 UTC [22] LOG:  database system was interrupted; last known up at 2024-12-16 08:34:02 UTC
2024-12-16T08:36:49.999982088Z INFO:root:Reading config file
2024-12-16T08:36:50.008742796Z INFO:root:Reading config file
2024-12-16T08:36:50.010918683Z INFO:root:Running API
2024-12-16T08:36:50.069525910Z Waiting for Web and API services to be reachable...
2024-12-16T08:36:50.267535182Z 2024-12-16 08:36:50.261 UTC [47] FATAL:  the database system is starting up
2024-12-16T08:36:50.333898754Z Waiting for PostgreSQL to start...
2024-12-16T08:36:50.512071403Z INFO:root:Starting setup
2024-12-16T08:36:50.566349657Z INFO:root:Apps config exists, loading
2024-12-16T08:36:50.566376257Z INFO:root:Setting up postgres
2024-12-16T08:36:50.636875762Z 2024-12-16 08:36:50.625 UTC [56] FATAL:  the database system is starting up
2024-12-16T08:36:50.638578807Z INFO:root:Waiting for postgres to be ready
2024-12-16T08:36:50.638596887Z 2024-12-16 08:36:50,638 INFO success: postgres entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-12-16T08:36:50.850158403Z 2024-12-16 08:36:50,845 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-12-16T08:36:50.850295403Z 2024-12-16 08:36:50.838 UTC [22] LOG:  database system was not properly shut down; automatic recovery in progress
2024-12-16T08:36:50.863917647Z 2024-12-16 08:36:50.863 UTC [22] LOG:  redo starts at 1/AA990170
2024-12-16T08:36:50.880298699Z 2024-12-16 08:36:50.877 UTC [22] LOG:  invalid record length at 1/AA99AD78: wanted 24, got 0
2024-12-16T08:36:50.880323819Z 2024-12-16 08:36:50.877 UTC [22] LOG:  redo done at 1/AA99AD50 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.01 s
2024-12-16T08:36:50.909864913Z 2024-12-16 08:36:50.907 UTC [20] LOG:  checkpoint starting: end-of-recovery immediate wait
2024-12-16T08:36:50.949278439Z 2024-12-16 08:36:50.948 UTC [20] LOG:  checkpoint complete: wrote 9 buffers (0.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.004 s, sync=0.012 s, total=0.043 s; sync files=6, longest=0.005 s, average=0.002 s; distance=43 kB, estimate=43 kB
2024-12-16T08:36:50.956415142Z 2024-12-16 08:36:50.955 UTC [57] FATAL:  the database system is not yet accepting connections
2024-12-16T08:36:50.956442942Z 2024-12-16 08:36:50.955 UTC [57] DETAIL:  Consistent recovery state has not been yet reached.
2024-12-16T08:36:50.969650464Z 2024-12-16 08:36:50.968 UTC [8] LOG:  database system is ready to accept connections
2024-12-16T08:36:50.971878831Z INFO:root:Waiting for postgres to be ready
2024-12-16T08:36:51.340520529Z INFO:root:Postgres is ready
2024-12-16T08:36:51.340546769Z INFO:root:Changing default user password
2024-12-16T08:36:51.417527095Z INFO:root:Password changed
2024-12-16T08:36:51.417549135Z INFO:root:Running migrations
2024-12-16T08:36:51.626340001Z 2024-12-16 08:36:51,626 INFO exited: init_db (exit status 0; expected)
2024-12-16T08:36:52.389339918Z   ▲ Next.js 13.5.5
2024-12-16T08:36:52.389372998Z   - Local:        http://localhost:4000
2024-12-16T08:36:52.389376198Z   - Network:      http://0.0.0.0:4000
2024-12-16T08:36:52.389378198Z 
2024-12-16T08:36:52.389380278Z  ✓ Ready in 546ms
2024-12-16T08:36:52.846556578Z Waiting for Web and API services to be reachable...
2024-12-16T08:36:53.510296457Z [I 2024-12-16 08:36:53.509 ServerApp] Extension package jupyterlab took 0.2141s to import
2024-12-16T08:36:54.618154995Z Prisma schema loaded from packages/database/prisma/schema.prisma
2024-12-16T08:36:54.630578274Z Datasource "db": PostgreSQL database "briefer", schema "public" at "localhost:5432"
2024-12-16T08:36:54.695420802Z [I 2024-12-16 08:36:54.693 ServerApp] jupyter_briefer_extension | extension was successfully linked.
2024-12-16T08:36:54.695451202Z [I 2024-12-16 08:36:54.693 ServerApp] jupyter_lsp | extension was successfully linked.
2024-12-16T08:36:54.725428977Z [I 2024-12-16 08:36:54.721 ServerApp] jupyter_server_terminals | extension was successfully linked.
2024-12-16T08:36:54.733766284Z [I 2024-12-16 08:36:54.733 ServerApp] jupyterlab | extension was successfully linked.
2024-12-16T08:36:54.746337164Z [I 2024-12-16 08:36:54.745 ServerApp] notebook | extension was successfully linked.
2024-12-16T08:36:54.761276372Z 2024-12-16 08:36:54.761 UTC [119] LOG:  could not receive data from client: Connection reset by peer
2024-12-16T08:36:54.801485220Z 
2024-12-16T08:36:54.802286503Z 30 migrations found in prisma/migrations
2024-12-16T08:36:54.802304263Z 
2024-12-16T08:36:54.920675201Z Waiting for Web and API services to be reachable...
2024-12-16T08:36:54.933055440Z 
2024-12-16T08:36:54.933997123Z No pending migrations to apply.
2024-12-16T08:36:55.019191115Z npm notice
2024-12-16T08:36:55.019253396Z npm notice New minor version of npm available! 10.8.2 -> 10.9.2
2024-12-16T08:36:55.019256716Z npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.9.2
2024-12-16T08:36:55.019259036Z npm notice To update run: npm install -g [email protected]
2024-12-16T08:36:55.019261316Z npm notice
2024-12-16T08:36:55.032668638Z INFO:root:Migrations done
2024-12-16T08:36:55.033822682Z INFO:root:Setup finished
2024-12-16T08:36:55.202405380Z (node:27) NOTE: The AWS SDK for JavaScript (v2) is in maintenance mode.
2024-12-16T08:36:55.202433940Z  SDK releases are limited to address critical bug fixes and security issues only.
2024-12-16T08:36:55.202436981Z 
2024-12-16T08:36:55.202439021Z Please migrate your code to use AWS SDK for JavaScript (v3).
2024-12-16T08:36:55.202440941Z For more information, check the blog post at https://a.co/cUPnyil
2024-12-16T08:36:55.202442941Z (Use `node --trace-warnings ...` to show where the warning was created)
2024-12-16T08:36:55.202688621Z 2024-12-16 08:36:55,202 INFO exited: setup (exit status 0; expected)
2024-12-16T08:36:55.267882669Z [08:36:55.262] INFO (27): Server is running on port 3000
2024-12-16T08:36:55.420096316Z [I 2024-12-16 08:36:55.419 ServerApp] notebook_shim | extension was successfully linked.
2024-12-16T08:36:55.444890115Z [08:36:55.442] INFO (27): Creating schedule
2024-12-16T08:36:55.444922475Z     scheduleId: "36012524-d01e-4007-a757-16cf2d83c2db"
2024-12-16T08:36:55.444956875Z     documentId: "3c4d7db3-d2f9-44fb-b9da-63194054a41e"
2024-12-16T08:36:55.444961035Z     cron: "0 1 * * * *"
2024-12-16T08:36:55.444963235Z     module: "schedule"
2024-12-16T08:36:55.464653138Z [I 2024-12-16 08:36:55.464 ServerApp] notebook_shim | extension was successfully loaded.
2024-12-16T08:36:55.465606261Z [I 2024-12-16 08:36:55.464 ServerApp] Jupyter Briefer Extension loaded.
2024-12-16T08:36:55.465625501Z [I 2024-12-16 08:36:55.464 ServerApp] jupyter_briefer_extension | extension was successfully loaded.
2024-12-16T08:36:55.467013545Z [I 2024-12-16 08:36:55.466 ServerApp] jupyter_lsp | extension was successfully loaded.
2024-12-16T08:36:55.467932508Z [I 2024-12-16 08:36:55.467 ServerApp] jupyter_server_terminals | extension was successfully loaded.
2024-12-16T08:36:55.480586109Z [I 2024-12-16 08:36:55.480 LabApp] JupyterLab extension loaded from /app/jupyter/venv/lib/python3.11/site-packages/jupyterlab
2024-12-16T08:36:55.482415915Z [I 2024-12-16 08:36:55.480 LabApp] JupyterLab application directory is /app/jupyter/venv/share/jupyter/lab
2024-12-16T08:36:55.484020040Z [I 2024-12-16 08:36:55.482 LabApp] Extension Manager is 'pypi'.
2024-12-16T08:36:55.508406718Z [08:36:55.503] INFO (27): Updated schedules
2024-12-16T08:36:55.508437198Z     created: 1
2024-12-16T08:36:55.508439758Z     updated: 0
2024-12-16T08:36:55.508441678Z     deleted: 0
2024-12-16T08:36:55.508443438Z     unchanged: 0
2024-12-16T08:36:55.508445238Z     module: "schedule"
2024-12-16T08:36:55.518001708Z [08:36:55.515] ERROR (27): Failed to check if Jupyter is running
2024-12-16T08:36:55.518026508Z     workspaceId: "5dbd050b-aedf-469b-a90d-476f37b35dff"
2024-12-16T08:36:55.518030068Z     err: {
2024-12-16T08:36:55.518031948Z       "type": "TypeError",
2024-12-16T08:36:55.518033908Z       "message": "fetch failed: connect ECONNREFUSED 127.0.0.1:8888",
2024-12-16T08:36:55.518035868Z       "stack":
2024-12-16T08:36:55.518037668Z           TypeError: fetch failed
2024-12-16T08:36:55.518039708Z               at node:internal/deps/undici/undici:12618:11
2024-12-16T08:36:55.518041868Z               at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-12-16T08:36:55.518043828Z           caused by: Error: connect ECONNREFUSED 127.0.0.1:8888
2024-12-16T08:36:55.518045708Z               at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)
2024-12-16T08:36:55.518047588Z     }
2024-12-16T08:36:55.636414006Z [I 2024-12-16 08:36:55.635 ServerApp] jupyterlab | extension was successfully loaded.
2024-12-16T08:36:55.639816737Z [I 2024-12-16 08:36:55.639 ServerApp] notebook | extension was successfully loaded.
2024-12-16T08:36:55.640246138Z [I 2024-12-16 08:36:55.640 ServerApp] Serving notebooks from local directory: /home/jupyteruser
2024-12-16T08:36:55.640290099Z [I 2024-12-16 08:36:55.640 ServerApp] Jupyter Server 2.12.1 is running at:
2024-12-16T08:36:55.640592420Z [I 2024-12-16 08:36:55.640 ServerApp] http://235f0aa4dd6a:8888/?token=...
2024-12-16T08:36:55.640605740Z [I 2024-12-16 08:36:55.640 ServerApp]     http://127.0.0.1:8888/?token=...
2024-12-16T08:36:55.640611300Z [I 2024-12-16 08:36:55.640 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
2024-12-16T08:36:56.127253414Z [I 2024-12-16 08:36:56.127 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
2024-12-16T08:36:56.960644555Z Waiting for Web and API services to be reachable...
2024-12-16T08:36:58.990749837Z Waiting for Web and API services to be reachable...
2024-12-16T08:37:01.024776652Z Waiting for Web and API services to be reachable...
2024-12-16T08:37:03.059218308Z Waiting for Web and API services to be reachable...
2024-12-16T08:37:05.091716957Z Waiting for Web and API services to be reachable...
2024-12-16T08:37:07.125394890Z Waiting for Web and API services to be reachable...
2024-12-16T08:37:09.161828272Z Waiting for Web and API services to be reachable...

is there any way to debug wehich service is having troubles starting?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@johannesspohr