Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] TypeError: missing a required argument: 'messages' #2088

Open
9 tasks done
h14turbo opened this issue Sep 19, 2024 · 5 comments
Open
9 tasks done

[BUG] TypeError: missing a required argument: 'messages' #2088

h14turbo opened this issue Sep 19, 2024 · 5 comments
Labels
external

Comments

@h14turbo
Copy link

h14turbo commented Sep 19, 2024 •
edited
Loading

Pre-check

  • I have searched the existing issues and none cover this bug.

Description

When running the docker instance of privategpt with Ollama, I get an error saying: TypeError: missing a required argument: 'messages'

"Search" mode works, but any mode with the LLM called produces this error. I am using the normal gradio UI§§§
The full traceback is as follows:

private-gpt-ollama-1 | 18:00:31.961 [INFO ] uvicorn.access - 172.18.0.1:62074 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:00:31.980 [INFO ] uvicorn.access - 172.18.0.1:55394 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:00:31.982 [INFO ] uvicorn.access - 172.18.0.1:55394 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | Traceback (most recent call last):
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/queueing.py", line 536, in process_events
private-gpt-ollama-1 | response = await route_utils.call_process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
private-gpt-ollama-1 | output = await app.get_blocks().process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1923, in process_api
private-gpt-ollama-1 | result = await self.call_function(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1520, in call_function
private-gpt-ollama-1 | prediction = await utils.async_iteration(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 768, in asyncgen_wrapper
private-gpt-ollama-1 | response = await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/chat_interface.py", line 652, in _stream_fn
private-gpt-ollama-1 | first_response = await async_iteration(generator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 656, in anext
private-gpt-ollama-1 | return await anyio.to_thread.run_sync(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
private-gpt-ollama-1 | return await get_async_backend().run_sync_in_worker_thread(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
private-gpt-ollama-1 | return await future
private-gpt-ollama-1 | ^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
private-gpt-ollama-1 | result = context.run(func, *args)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 639, in run_sync_iterator_async
private-gpt-ollama-1 | return next(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/ui/ui.py", line 185, in _chat
private-gpt-ollama-1 | query_stream = self._chat_service.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/server/chat/chat_service.py", line 168, in stream_chat
private-gpt-ollama-1 | streaming_response = chat_engine.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 230, in wrapper
private-gpt-ollama-1 | result = func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
private-gpt-ollama-1 | return func(self, *args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/chat_engine/context.py", line 210, in stream_chat
private-gpt-ollama-1 | chat_stream=self._llm.stream_chat(all_messages),
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/components/llm/llm_component.py", line 183, in wrapper
private-gpt-ollama-1 | return func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 221, in wrapper
private-gpt-ollama-1 | bound_args = inspect.signature(func).bind(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3212, in bind
private-gpt-ollama-1 | return self._bind(args, kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3127, in _bind
private-gpt-ollama-1 | raise TypeError(msg) from None
private-gpt-ollama-1 | TypeError: missing a required argument: 'messages'
private-gpt-ollama-1 | 18:01:15.212 [INFO ] uvicorn.access - 172.18.0.1:58670 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:54.097 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:54.099 [INFO ] uvicorn.access - 172.18.0.1:61960 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:54.122 [INFO ] private_gpt.ui.ui - Setting system prompt to:
private-gpt-ollama-1 | 18:03:55.953 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.971 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.972 [INFO ] uvicorn.access - 172.18.0.1:59612 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.987 [INFO ] uvicorn.access - 172.18.0.1:61960 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:55.989 [INFO ] uvicorn.access - 172.18.0.1:61960 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | 18:03:56.960 [INFO ] uvicorn.access - 172.18.0.1:59612 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:30.668 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:30.670 [INFO ] uvicorn.access - 172.18.0.1:59372 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:30.702 [INFO ] private_gpt.ui.ui - Setting system prompt to: You are an AI engine
private-gpt-ollama-1 |
private-gpt-ollama-1 | 18:08:32.171 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.188 [INFO ] uvicorn.access - 172.18.0.1:59382 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.189 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /run/predict HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.204 [INFO ] uvicorn.access - 172.18.0.1:59372 - "POST /queue/join HTTP/1.1" 200
private-gpt-ollama-1 | 18:08:32.207 [INFO ] uvicorn.access - 172.18.0.1:59372 - "GET /queue/data?session_hash=gjx9zkk6hbu HTTP/1.1" 200
private-gpt-ollama-1 | Traceback (most recent call last):
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/queueing.py", line 536, in process_events
private-gpt-ollama-1 | response = await route_utils.call_process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/route_utils.py", line 276, in call_process_api
private-gpt-ollama-1 | output = await app.get_blocks().process_api(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1923, in process_api
private-gpt-ollama-1 | result = await self.call_function(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/blocks.py", line 1520, in call_function
private-gpt-ollama-1 | prediction = await utils.async_iteration(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 768, in asyncgen_wrapper
private-gpt-ollama-1 | response = await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/chat_interface.py", line 652, in _stream_fn
private-gpt-ollama-1 | first_response = await async_iteration(generator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 663, in async_iteration
private-gpt-ollama-1 | return await iterator.anext()
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 656, in anext
private-gpt-ollama-1 | return await anyio.to_thread.run_sync(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
private-gpt-ollama-1 | return await get_async_backend().run_sync_in_worker_thread(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
private-gpt-ollama-1 | return await future
private-gpt-ollama-1 | ^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
private-gpt-ollama-1 | result = context.run(func, *args)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/gradio/utils.py", line 639, in run_sync_iterator_async
private-gpt-ollama-1 | return next(iterator)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/ui/ui.py", line 185, in _chat
private-gpt-ollama-1 | query_stream = self._chat_service.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/server/chat/chat_service.py", line 168, in stream_chat
private-gpt-ollama-1 | streaming_response = chat_engine.stream_chat(
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 230, in wrapper
private-gpt-ollama-1 | result = func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/callbacks/utils.py", line 41, in wrapper
private-gpt-ollama-1 | return func(self, *args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/chat_engine/context.py", line 210, in stream_chat
private-gpt-ollama-1 | chat_stream=self._llm.stream_chat(all_messages),
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/private_gpt/components/llm/llm_component.py", line 183, in wrapper
private-gpt-ollama-1 | return func(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/home/worker/app/.venv/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 221, in wrapper
private-gpt-ollama-1 | bound_args = inspect.signature(func).bind(*args, **kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3212, in bind
private-gpt-ollama-1 | return self._bind(args, kwargs)
private-gpt-ollama-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
private-gpt-ollama-1 | File "/usr/local/lib/python3.11/inspect.py", line 3127, in _bind
private-gpt-ollama-1 | raise TypeError(msg) from None
private-gpt-ollama-1 | TypeError: missing a required argument: 'messages'

Steps to Reproduce

I build the privategpt package and ran in docker. Modified the model to llama3.1:70b. Ollama is run outside of docker on 11434

Expected Behavior

Generate a response

Actual Behavior

Missing message argument

Environment

Ubuntu 20.04, RTX A6000 ADA

Additional Information

No response

Version

No response

Setup Checklist

  • Confirm that you have followed the installation instructions in the project’s documentation.
  • Check that you are using the latest version of the project.
  • Verify disk space availability for model storage and data processing.
  • Ensure that you have the necessary permissions to run the project.

NVIDIA GPU Setup Checklist

  • Check that the all CUDA dependencies are installed and are compatible with your GPU (refer to CUDA's documentation)
  • Ensure an NVIDIA GPU is installed and recognized by the system (run nvidia-smi to verify).
  • Ensure proper permissions are set for accessing GPU resources.
  • Docker users - Verify that the NVIDIA Container Toolkit is configured correctly (e.g. run sudo docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi)
@h14turbo h14turbo added the bug Something isn't working label Sep 19, 2024
@jaluma
Copy link
Collaborator

jaluma commented Sep 23, 2024

Can you try using API instead of UI? It's to verify if it's a Ollama connectivity problem or UI problem

@h14turbo
Copy link
Author

I am getting an identical error via API calls

@jaluma
Copy link
Collaborator

jaluma commented Sep 26, 2024

I have been trying to boot PGPT in ollama-api mode, with Ollama running on metal (macOS). I've booted everything using:

$ docker compose --profile=ollama-api up --build

After that, I tried running a test with CURL/Postman with the context enabled and I didn't manage to replicate it.

curl --location 'http://localhost:8001/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
  "model": "llama3.1",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "What is deep learning?"
    }
  ],
  "stream": false,
  "max_tokens": 20,
  "use_context": true,
  "context_filter": {
    "docs_ids": ["test"]
  }
}'

Can you give me more details about the problem? Can you try with ollama?

@h14turbo
Copy link
Author

h14turbo commented Oct 9, 2024

Okay, so I think I know the rough area that is causing the problem, I am using a load balancer between 2 Ollama instances using NGINX.

The main parts of the config are:

upstream ollama_load_balancer {
server host.docker.internal:11436;
server host.docker.internal:11435;
}

 }
    location /ollama {
        rewrite ^/ollama(/?)(.*)$ /$2 break;
        proxy_pass http://ollama_load_balancer;

I am not sure why this is causing the connection not to work, as ingestion works using the same Ollama instances...

@jaluma
Copy link
Collaborator

jaluma commented Oct 14, 2024

Mmm. Something is wrong with SSE then. Maybe, your proxy has disabled SSE events, buffer or any other related config. I was browsing and I found this post. Can you check it @h14turbo?
https://stackoverflow.com/questions/13672743/eventsource-server-sent-events-through-nginx

@jaluma jaluma added external and removed bug Something isn't working labels Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
external
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jaluma @h14turbo