Using chat instead of llm will not answer questions which chat can answer itself

[sunlongjian]

#Here's what I did, if i use agent =initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True), it will work fine.

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
chat =ChatOpenAI(temperature=0)
llm =OpenAI(temperature=0)
tools =load_tools(["llm-math", "searx-search"], searx_host="http://localhost:8080", llm=llm)
agent =initialize_agent(tools, chat, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
#agent =initialize_agent(tools, chat, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True) this also will failed
agent.run("how are you today")

Entering new AgentExecutor chain...
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/langchain/agents/chat/output_parser.py", line 21, in parse
action = text.split("```")[1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.9/site-packages/langchain/chains/base.py", line 238, in run
return self(args[0], callbacks=callbacks)[self.output_keys[0]]
File "/usr/local/lib/python3.9/site-packages/langchain/chains/base.py", line 142, in call
raise e
File "/usr/local/lib/python3.9/site-packages/langchain/chains/base.py", line 136, in call
self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.9/site-packages/langchain/agents/agent.py", line 855, in _call
next_step_output = self._take_next_step(
File "/usr/local/lib/python3.9/site-packages/langchain/agents/agent.py", line 723, in _take_next_step
output = self.agent.plan(
File "/usr/local/lib/python3.9/site-packages/langchain/agents/agent.py", line 424, in plan
return self.output_parser.parse(full_output)
File "/usr/local/lib/python3.9/site-packages/langchain/agents/chat/output_parser.py", line 26, in parse
raise OutputParserException(f"Could not parse LLM output: {text}")
langchain.schema.OutputParserException: Could not parse LLM output: I'm an AI language model, so I don't have feelings. Is there anything I can assist you with?

[LouisSilva]

The problem is that with the agents CHAT_ZERO_SHOT_REACT_DESCRIPTION and ZERO_SHOT_REACT_DESCRIPTION, if you were to ask it something that it cannot use a tool to answer, the OutputParser will raise an error because it expects the LLM to use a tool, but when you asked it how it was, it couldn't use its llm-math tool or searx-search tool to answer because it doesn't need to. If you run your code again, but ask it to do a maths problem (so it uses the maths tool), it will work.

If you want an agent that can use its tools and talk to you like a normal chat bot, then use the CHAT_CONVERSATIONAL_REACT_DESCRIPTION agent. The link to the documentation is here. All you will need to do in your code is add a memory object, and change the agent type. I have edited your code to implement these changes:

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
from langchain.memory import ConversationBufferMemory  # Import memory

chat = ChatOpenAI(temperature=0)
llm = OpenAI(temperature=0)
tools = load_tools(["llm-math", "searx-search"], searx_host="http://localhost:8080/", llm=llm)

# Add memory object
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

agent = initialize_agent(tools, chat, agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, verbose=True, memory=memory)

agent.run("how are you today")

[sunlongjian]

Thank you so much it works now, I didn't read docs that deep before. But i still got question about it, since conversional agent is for agent to decide whether it needs to use a tool to answer or not, then why in my case it can directly answer "how are you" when using llm with ZERO_SHOT_REACT_DESCRIPTION agent while using chat can't