-
Notifications
You must be signed in to change notification settings - Fork 7.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] TypeError: missing a required argument: 'messages' #2088
Comments
Can you try using API instead of UI? It's to verify if it's a Ollama connectivity problem or UI problem |
I am getting an identical error via API calls |
I have been trying to boot PGPT in ollama-api mode, with Ollama running on metal (macOS). I've booted everything using:
After that, I tried running a test with CURL/Postman with the context enabled and I didn't manage to replicate it.
Can you give me more details about the problem? Can you try with ollama? |
Okay, so I think I know the rough area that is causing the problem, I am using a load balancer between 2 Ollama instances using NGINX. The main parts of the config are: upstream ollama_load_balancer {
I am not sure why this is causing the connection not to work, as ingestion works using the same Ollama instances... |
Mmm. Something is wrong with SSE then. Maybe, your proxy has disabled SSE events, buffer or any other related config. I was browsing and I found this post. Can you check it @h14turbo? |
Pre-check
Description
When running the docker instance of privategpt with Ollama, I get an error saying: TypeError: missing a required argument: 'messages'
"Search" mode works, but any mode with the LLM called produces this error. I am using the normal gradio UI§§§
The full traceback is as follows:
private-gpt-ollama-1 | 18:00:31.961 [INFO ] uvicorn.access - 172.18.0.1:62074 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:00:31.980 [INFO ] uvicorn.access - 172.18.0.1:55394 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:00:31.982 [INFO ] uvicorn.access - 172.18.0.1:55394 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | Traceback (most recent call last):
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/queueing.py", line 536, in process_events
private-gpt-ollama-1 | response = await route_utils.call_process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
private-gpt-ollama-1 | output = await app.get_blocks().process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1923, in process_api
private-gpt-ollama-1 | result = await self.call_function(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1520, in call_function
private-gpt-ollama-1 | prediction = await utils.async_iteration(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 768, in asyncgen_wrapper
private-gpt-ollama-1 | response = await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/chat_interface.py", line 652, in _stream_fn
private-gpt-ollama-1 | first_response = await async_iteration(generator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 656, in anext
private-gpt-ollama-1 | return await anyio.to_thread.run_sync(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
private-gpt-ollama-1 | return await get_async_backend().run_sync_in_worker_thread(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
private-gpt-ollama-1 | return await future
private-gpt-ollama-1 | ^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
private-gpt-ollama-1 | result = context.run(func, *args)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 639, in run_sync_iterator_async
private-gpt-ollama-1 | return next(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/ui/ui.py", line 185, in _chat
private-gpt-ollama-1 | query_stream = self._chat_service.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/server/chat/chat_service.py", line 168, in stream_chat
private-gpt-ollama-1 | streaming_response = chat_engine.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 230, in wrapper
private-gpt-ollama-1 | result = func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
private-gpt-ollama-1 | return func(self, *args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/chat_engine/context.py", line 210, in stream_chat
private-gpt-ollama-1 | chat_stream=self._llm.stream_chat(all_messages),
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/components/llm/llm_component.py", line 183, in wrapper
private-gpt-ollama-1 | return func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 221, in wrapper
private-gpt-ollama-1 | bound_args = inspect.signature(func).bind(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3212, in bind
private-gpt-ollama-1 | return self._bind(args, kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3127, in _bind
private-gpt-ollama-1 | raise TypeError(msg) from None
private-gpt-ollama-1 | TypeError: missing a required argument: 'messages'
private-gpt-ollama-1 | 18:01:15.212 [INFO ] uvicorn.access - 172.18.0.1:58670 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:54.097 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:54.099 [INFO ] uvicorn.access - 172.18.0.1:61960 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:54.122 [INFO ] private_gpt.ui.ui - Setting system prompt to:
private-gpt-ollama-1 | 18:03:55.953 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.971 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.972 [INFO ] uvicorn.access - 172.18.0.1:59612 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.987 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.989 [INFO ] uvicorn.access - 172.18.0.1:61960 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:56.960 [INFO ] uvicorn.access - 172.18.0.1:59612 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:30.668 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:30.670 [INFO ] uvicorn.access - 172.18.0.1:59372 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:30.702 [INFO ] private_gpt.ui.ui - Setting system prompt to: You are an AI engine
private-gpt-ollama-1 |
private-gpt-ollama-1 | 18:08:32.171 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.188 [INFO ] uvicorn.access - 172.18.0.1:59382 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.189 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.204 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.207 [INFO ] uvicorn.access - 172.18.0.1:59372 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | Traceback (most recent call last):
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/queueing.py", line 536, in process_events
private-gpt-ollama-1 | response = await route_utils.call_process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
private-gpt-ollama-1 | output = await app.get_blocks().process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1923, in process_api
private-gpt-ollama-1 | result = await self.call_function(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1520, in call_function
private-gpt-ollama-1 | prediction = await utils.async_iteration(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 768, in asyncgen_wrapper
private-gpt-ollama-1 | response = await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/chat_interface.py", line 652, in _stream_fn
private-gpt-ollama-1 | first_response = await async_iteration(generator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 656, in anext
private-gpt-ollama-1 | return await anyio.to_thread.run_sync(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
private-gpt-ollama-1 | return await get_async_backend().run_sync_in_worker_thread(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
private-gpt-ollama-1 | return await future
private-gpt-ollama-1 | ^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
private-gpt-ollama-1 | result = context.run(func, *args)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 639, in run_sync_iterator_async
private-gpt-ollama-1 | return next(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/ui/ui.py", line 185, in _chat
private-gpt-ollama-1 | query_stream = self._chat_service.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/server/chat/chat_service.py", line 168, in stream_chat
private-gpt-ollama-1 | streaming_response = chat_engine.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 230, in wrapper
private-gpt-ollama-1 | result = func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
private-gpt-ollama-1 | return func(self, *args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/chat_engine/context.py", line 210, in stream_chat
private-gpt-ollama-1 | chat_stream=self._llm.stream_chat(all_messages),
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/components/llm/llm_component.py", line 183, in wrapper
private-gpt-ollama-1 | return func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 221, in wrapper
private-gpt-ollama-1 | bound_args = inspect.signature(func).bind(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3212, in bind
private-gpt-ollama-1 | return self._bind(args, kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3127, in _bind
private-gpt-ollama-1 | raise TypeError(msg) from None
private-gpt-ollama-1 | TypeError: missing a required argument: 'messages'
Steps to Reproduce
I build the privategpt package and ran in docker. Modified the model to llama3.1:70b. Ollama is run outside of docker on 11434
Expected Behavior
Generate a response
Actual Behavior
Missing message argument
Environment
Ubuntu 20.04, RTX A6000 ADA
Additional Information
No response
Version
No response
Setup Checklist
NVIDIA GPU Setup Checklist
nvidia-smi
to verify).sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi
)The text was updated successfully, but these errors were encountered: