Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_iY0nNb7BlKHtLUT3BRmK28r8", 'type': 'invalid_request_error', 'param': 'messages', 'code': None}}

[mrctito]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class State(TypedDict):
    messages: Annotated[list, add_messages]
...
    graph_builder = StateGraph(State)
    graph_builder.add_node("agent", self.call_model)
    graph_builder.add_node("tools", self.tool_node)
    graph_builder.add_edge("agent", "tools")
...

async def call_model(self, state: State):
    messages = state["messages"]
    response = await self.llm_with_tools.ainvoke(state["messages"])  #error here
    return {"messages": [response]}

Description

Anybody knows what error this is?

Error code: 400 - {'error': {'message': "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_iY0nNb7BlKHtLUT3BRmK28r8", 'type': 'invalid_request_error', 'param': 'messages', 'code': None}}

System Info

langchain==0.2.6
langchain-community==0.2.6
langchain-core==0.2.10
langchain-experimental==0.0.62
langchain-openai==0.1.13
langchain-qdrant==0.1.0
langchain-text-splitters==0.2.1

[dosubot[bot]]

To resolve the error indicating that an assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id', you need to ensure that each tool call in the assistant message is followed by a corresponding tool message that responds to the 'tool_call_id'.

Here's an example of how you can structure your messages to include the necessary tool messages:

from langchain_core.messages import AIMessage, ToolMessage

# Example assistant message with tool calls
assistant_message = AIMessage(
    content='',
    tool_calls=[
        {
            'id': 'call_iY0nNb7BlKHtLUT3BRmK28r8',
            'function': {
                'name': 'YourToolFunctionName',
                'arguments': '{"arg1":"value1","arg2":"value2"}'
            },
            'type': 'function'
        }
    ]
)

# Corresponding tool message responding to the tool call
tool_message = ToolMessage(
    role='tool',
    content='{"result":"Your tool result"}',
    name='YourToolFunctionName',
    tool_call_id='call_iY0nNb7BlKHtLUT3BRmK28r8'
)

# Ensure the tool message follows the assistant message
messages = [assistant_message, tool_message]

Additionally, you can use the bind_tools() method to handle converting and binding tools to the model, ensuring that tool schemas are included in its calls to the LLM. For example:

from langchain_core.tools import tool

@tool
def add(a: int, b: int) -> int:
    """Adds a and b."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiplies a and b."""
    return a * b

tools = [add, multiply]

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
llm_with_tools = llm.bind_tools(tools)

When invoking the model, ensure that tool calls are properly handled and tool results are passed back to the model using ToolMessages:

from langchain_core.messages import HumanMessage, ToolMessage

query = "What is 3 * 12? Also, what is 11 + 49?"
messages = [HumanMessage(query)]
ai_msg = llm_with_tools.invoke(messages)
messages.append(ai_msg)
for tool_call in ai_msg.tool_calls:
    selected_tool = {"add": add, "multiply": multiply}[tool_call["name"].lower()]
    tool_output = selected_tool.invoke(tool_call["args"])
    messages.append(ToolMessage(tool_output, tool_call_id=tool_call["id"]))
messages

This ensures that the tool messages are correctly appended and the model can process them accordingly ^{[1][2][3][4][5]}.

To continue talking to Dosu, mention @dosu.

[TomGoodwinCode]

I had this problem because my call_model() function was incorrectly dealing with tool call messages. My prompt template was turning messages with type tool into type human

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", configuration.system_prompt), 
        ("placeholder", "{messages}"),
        ("user", "{input}"),    # <-----  if {input} is type **tool** then it's turned into a 'user' message
    ]
)


model = load_chat_model(configuration.model).bind_tools(TOOLS)

message_value = await prompt.ainvoke(
    {
        "messages": state.messages,
        "input": state.messages[-1].content,  <---- I should never have split off the input state like this
        "title": "London",
        "wiki_title": "London",
        "wiki_extract": "London is the capital city of England.",
        "wiki_description": "London is the capital city of England.",
    },
    config,
)

Obviously this is not correct. I should have been leveraging the ("placeholder", "{messages}") to send all of my messsages. As placeholder maintains the type of the message.

I should have just used placeholder as intended:

prompt = ChatPromptTemplate.from_messages(
    [("system", configuration.system_prompt), ("placeholder", "{messages}")]
)

model = load_chat_model(configuration.model).bind_tools(TOOLS)

message_value = await prompt.ainvoke(
    {
        "messages": state.messages,
        "title": "London",
        "wiki_title": "London",
        "wiki_extract": "London is the capital city of England.",
        "wiki_description": "London is the capital city of England.",
    },
    config,
)

Just thought I would post this incase it gives anyone guidence on solving the same error

[algotechnology]

The worst aspect is that it needs a whole reboot of everything as this error condition (no matter how frequent or infrequent) breaks the whole system. Can this be handled better to recover from this condition (please!). Thanks

[345ishaan]

I am also seeing same error. An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_femmloc3wNKeN3zrEeg5v8zS", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': None

[345ishaan]

I am also seeing same error. An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_femmloc3wNKeN3zrEeg5v8zS", 'type': 'invalid_request_error', 'param': 'messages.[2].role', 'code': None

It seems it happens because the llm outputs wrong format for tool function arguments.

[345ishaan]

I started seeing this when i started making a tools which takes non primitive data types as arguments like Dict[str,str]. Is langraph not well supported in those cases @rlancemartin ?

[sagorbrur]

Hi @algotechnology , @345ishaan

The issue appears to originate from a parallel tool call in my case. The LLM detects two tools, yet I am only passing the output of one. I resolved this by setting parallel_tool_calls=False when binding tools. If you intend to handle parallel tool calls, you will need to retrieve the responses from both functions and pass them to the LLM.

For more please check Openai blog:
https://platform.openai.com/docs/guides/function-calling/parallel-function-calling