[SCRAPER] - TrueNAS Scale Docker Error

[N20Visuals]

First Check

• I used the GitHub search to find a similar issue and didn't find it.

• I have verified that this issue is not related to the underlying library
  hhyrsev/recipe-scrapers by 1) checking
  the debugger and data is returned, 2)
  verifying that there are errors in the log related to application level code, or
  3) verified that the site provides recipe data, or is otherwise supported by
  hhyrsev/recipe-scrapers

• This issue can be replicated on the demo site (https://demo.mealie.io/)

Please provide 1-5 example URLs that are having errors

https://www.chefkoch.de/rezepte/4228681685379518/One-Pan-Thunfisch-Pasta.html

Please provide your logs for the Mealie container docker logs <container-id> > mealie.logs

2024-10-30 20:09:20.120419+00:00ERROR 2024-10-30T21:09:20 - Exception in ASGI application
2024-10-30 20:09:20.120504+00:00Traceback (most recent call last):
2024-10-30 20:09:20.120537+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
2024-10-30 20:09:20.120566+00:00result = await app( # type: ignore[func-returns-value]
2024-10-30 20:09:20.120685+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
2024-10-30 20:09:20.120717+00:00return await self.app(scope, receive, send)
2024-10-30 20:09:20.120746+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
2024-10-30 20:09:20.120804+00:00await super().call(scope, receive, send)
2024-10-30 20:09:20.120833+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in call
2024-10-30 20:09:20.120859+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.120916+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in call
2024-10-30 20:09:20.120946+00:00raise exc
2024-10-30 20:09:20.120973+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in call
2024-10-30 20:09:20.120999+00:00await self.app(scope, receive, _send)
2024-10-30 20:09:20.121248+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/sessions.py", line 85, in call
2024-10-30 20:09:20.121328+00:00await self.app(scope, receive, send_wrapper)
2024-10-30 20:09:20.121362+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 20, in call
2024-10-30 20:09:20.121434+00:00await responder(scope, receive, send)
2024-10-30 20:09:20.121463+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 39, in call
2024-10-30 20:09:20.121490+00:00await self.app(scope, receive, self.send_with_gzip)
2024-10-30 20:09:20.121517+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in call
2024-10-30 20:09:20.121671+00:00await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-10-30 20:09:20.121730+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.121761+00:00raise exc
2024-10-30 20:09:20.121911+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.121950+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.121980+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 715, in call
2024-10-30 20:09:20.122009+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.122075+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
2024-10-30 20:09:20.122105+00:00await route.handle(scope, receive, send)
2024-10-30 20:09:20.122132+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
2024-10-30 20:09:20.122195+00:00await self.app(scope, receive, send)
2024-10-30 20:09:20.122226+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
2024-10-30 20:09:20.122253+00:00await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-10-30 20:09:20.122280+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.122341+00:00raise exc
2024-10-30 20:09:20.122371+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.122398+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.122425+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
2024-10-30 20:09:20.122486+00:00response = await f(request)
2024-10-30 20:09:20.122515+00:00File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler
2024-10-30 20:09:20.122545+00:00response = await original_route_handler(request)
2024-10-30 20:09:20.122605+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
2024-10-30 20:09:20.122640+00:00raw_response = await run_endpoint_function(
2024-10-30 20:09:20.122666+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
2024-10-30 20:09:20.122694+00:00return await dependant.call(**values)
2024-10-30 20:09:20.122762+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 233, in parse_recipe_url
2024-10-30 20:09:20.122792+00:00return await self._create_recipe_from_web(req)
2024-10-30 20:09:20.122819+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 244, in _create_recipe_from_web
2024-10-30 20:09:20.122879+00:00recipe, extras = await create_from_html(url, self.translator, html)
2024-10-30 20:09:20.122909+00:00File "/app/mealie/services/scraper/scraper.py", line 45, in create_from_html
2024-10-30 20:09:20.122937+00:00new_recipe, extras = await scraper.scrape(url, html)
2024-10-30 20:09:20.122965+00:00File "/app/mealie/services/scraper/recipe_scraper.py", line 41, in scrape
2024-10-30 20:09:20.123035+00:00raw_html = html or await safe_scrape_html(url)
2024-10-30 20:09:20.123065+00:00File "/app/mealie/services/scraper/scraper_strategies.py", line 47, in safe_scrape_html
2024-10-30 20:09:20.123094+00:00async with client.stream(
2024-10-30 20:09:20.123120+00:00File "/usr/local/lib/python3.10/contextlib.py", line 199, in aenter
2024-10-30 20:09:20.123377+00:00return await anext(self.gen)
2024-10-30 20:09:20.123658+00:00ERROR 2024-10-30T21:09:20 - Exception in ASGI application
2024-10-30 20:09:20.123713+00:00Traceback (most recent call last):
2024-10-30 20:09:20.123741+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
2024-10-30 20:09:20.123824+00:00result = await app( # type: ignore[func-returns-value]
2024-10-30 20:09:20.123858+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
2024-10-30 20:09:20.123887+00:00return await self.app(scope, receive, send)
2024-10-30 20:09:20.123956+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
2024-10-30 20:09:20.123987+00:00await super().call(scope, receive, send)
2024-10-30 20:09:20.124014+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in call
2024-10-30 20:09:20.124082+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.124114+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in call
2024-10-30 20:09:20.124142+00:00raise exc
2024-10-30 20:09:20.124168+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in call
2024-10-30 20:09:20.124236+00:00await self.app(scope, receive, _send)
2024-10-30 20:09:20.124266+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/sessions.py", line 85, in call
2024-10-30 20:09:20.124294+00:00await self.app(scope, receive, send_wrapper)
2024-10-30 20:09:20.124320+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 20, in call
2024-10-30 20:09:20.125356+00:00await responder(scope, receive, send)
2024-10-30 20:09:20.125413+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 39, in call
2024-10-30 20:09:20.123419+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1628, in stream
2024-10-30 20:09:20.125594+00:00response = await self.send(
2024-10-30 20:09:20.125651+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send
2024-10-30 20:09:20.125680+00:00response = await self._send_handling_auth(
2024-10-30 20:09:20.125707+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
2024-10-30 20:09:20.126015+00:00response = await self._send_handling_redirects(
2024-10-30 20:09:20.126050+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
2024-10-30 20:09:20.126079+00:00response = await self._send_single_request(request)
2024-10-30 20:09:20.126182+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request
2024-10-30 20:09:20.126218+00:00response = await transport.handle_async_request(request)
2024-10-30 20:09:20.126247+00:00File "/app/mealie/pkgs/safehttp/transport.py", line 64, in handle_async_request
2024-10-30 20:09:20.126463+00:00ip_str = socket.gethostbyname(netloc)
2024-10-30 20:09:20.125444+00:00await self.app(scope, receive, self.send_with_gzip)
2024-10-30 20:09:20.126696+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in call
2024-10-30 20:09:20.126727+00:00await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-10-30 20:09:20.126824+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.126858+00:00raise exc
2024-10-30 20:09:20.126887+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.127126+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.127166+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 715, in call
2024-10-30 20:09:20.127193+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.127219+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
2024-10-30 20:09:20.127288+00:00await route.handle(scope, receive, send)
2024-10-30 20:09:20.127317+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
2024-10-30 20:09:20.127344+00:00await self.app(scope, receive, send)
2024-10-30 20:09:20.127371+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
2024-10-30 20:09:20.127431+00:00await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-10-30 20:09:20.127461+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.127489+00:00raise exc
2024-10-30 20:09:20.127687+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.127857+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.127951+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
2024-10-30 20:09:20.127981+00:00response = await f(request)
2024-10-30 20:09:20.126974+00:00socket.gaierror: [Errno -3] Temporary failure in name resolution
2024-10-30 20:09:20.128298+00:00File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler
2024-10-30 20:09:20.128334+00:00response = await original_route_handler(request)
2024-10-30 20:09:20.128360+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
2024-10-30 20:09:20.128501+00:00raw_response = await run_endpoint_function(
2024-10-30 20:09:20.128539+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
2024-10-30 20:09:20.128568+00:00return await dependant.call(**values)
2024-10-30 20:09:20.128633+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 233, in parse_recipe_url
2024-10-30 20:09:20.128664+00:00return await self._create_recipe_from_web(req)
2024-10-30 20:09:20.128691+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 244, in _create_recipe_from_web
2024-10-30 20:09:20.128717+00:00recipe, extras = await create_from_html(url, self.translator, html)
2024-10-30 20:09:20.128901+00:00File "/app/mealie/services/scraper/scraper.py", line 45, in create_from_html
2024-10-30 20:09:20.128940+00:00new_recipe, extras = await scraper.scrape(url, html)
2024-10-30 20:09:20.128993+00:00File "/app/mealie/services/scraper/recipe_scraper.py", line 41, in scrape
2024-10-30 20:09:20.129087+00:00raw_html = html or await safe_scrape_html(url)
2024-10-30 20:09:20.129119+00:00File "/app/mealie/services/scraper/scraper_strategies.py", line 47, in safe_scrape_html
2024-10-30 20:09:20.129174+00:00async with client.stream(
2024-10-30 20:09:20.129202+00:00File "/usr/local/lib/python3.10/contextlib.py", line 199, in aenter
2024-10-30 20:09:20.129289+00:00return await anext(self.gen)
2024-10-30 20:09:20.129346+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1628, in stream
2024-10-30 20:09:20.129375+00:00response = await self.send(
2024-10-30 20:09:20.129424+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send
2024-10-30 20:09:20.129513+00:00response = await self._send_handling_auth(
2024-10-30 20:09:20.129546+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
2024-10-30 20:09:20.129597+00:00response = await self._send_handling_redirects(
2024-10-30 20:09:20.129625+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
2024-10-30 20:09:20.129811+00:00response = await self._send_single_request(request)
2024-10-30 20:09:20.129934+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request
2024-10-30 20:09:20.129966+00:00response = await transport.handle_async_request(request)
2024-10-30 20:09:20.130080+00:00File "/app/mealie/pkgs/safehttp/transport.py", line 64, in handle_async_request
2024-10-30 20:09:20.130112+00:00ip_str = socket.gethostbyname(netloc)
2024-10-30 20:09:20.130140+00:00socket.gaierror: [Errno -3] Temporary failure in name resolution

Deployment

Docker (Linux)

[N20Visuals]

having ismus scraping the afforementioned recipe, could this be an issing with my internal dns or with the way truenas docker is set up?

[Kuchenpirat]

I can't replicate your issue, and i had never any issues scraping chefkoch.de.
The error stack shows that mealie seems to not even be able to resolve the url, so I recommend checking your internal DNS configuration as you mentioned.
This doesn't seem to be a Mealie issue but rather a configuration problem.