Update ChatAnthropic class langchain-anthropic with new function calling features available in Anthropic API
#21871
Replies: 4 comments 10 replies
-
|
@ccurme It looks like you are the most active developer for langchain-anthropic. Any idea about when this will be implemented? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tommasodelorenzo, thanks for the ping on this.
Can you clarify what you mean by vision inputs? If you mean a model calling tools in response to an image input, this should be supported: import base64
from typing import Literal
import httpx
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
@tool
def weather_tool(weather: Literal["sunny", "cloudy", "rainy"]) -> None:
"""Describe the weather"""
pass
image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8")
model = ChatAnthropic(model="claude-3-sonnet-20240229").bind_tools([weather_tool])
message = HumanMessage(
content=[
{"type": "text", "text": "describe the weather in this image"},
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/jpeg",
"data": image_data,
},
},
],
)
response = model.invoke([message])
print(response.tool_calls)If you mean processing images as the result of a tool call, there's an open PR for that that should be merged shortly. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tommasodelorenzo (cc @ccurme) about your first point, are you able to stream tool calls already? I am using langchain-anthropic 0.1.19 and I am not able to do it (I am using AgentExecutor and it just stops when it needs to call a tool). Would be great if you could let me know! Thanks! (: |
Beta Was this translation helpful? Give feedback.
-
|
Streaming tool calls is released: from langchain_anthropic import ChatAnthropic
def magic_function(input: int) -> int:
"""Applies a magic function to an input."""
return input + 2
llm = ChatAnthropic(
model="claude-3-haiku-20240307",
).bind_tools([magic_function])
full = None
for chunk in llm.stream("What is magic_function(3)?"):
full = chunk if full is None else full + chunk
print(chunk.content)print(full.tool_calls)@Thimows the default behavior of |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @ccurme! I am actually already using stream events (astream_events) with ChatOpenAI and everything is working perfectly. However, I want to implement ChatAnthropic as well. Everything seems to work well, however when the agent needs to call a tool the event stream just stops (no errors). Do you know if I am overlooking something? If you could point me in the right direction that would be immensely helpful. Thank you so much (: Example of the behavior (it stops when it needs to call a tool): |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Thimows, this may be a bug related to your tool having no arguments (if I'm reading that correctly). Created an issue to track: #23911 |
Beta Was this translation helpful? Give feedback.
-
|
@ccurme Ah that was it! The tool does not have arguments indeed. When I try a different tool that does have arguments everything works as expected. Hopefully this can be fixed in a future update. Thanks again! (: |
Beta Was this translation helpful? Give feedback.
-
|
It works for me! |
Beta Was this translation helpful? Give feedback.
-
|
Just opened an issue about it #24136 |
Beta Was this translation helpful? Give feedback.
-
|
I have some issues with the current version as well (langchain = "0.2.7", langchain-anthropic = "0.1.20"). I use different models, and when using Anthropic with tool calling, the output of the Is this behaviour expected? I expected that LangChain handles it so that the LLMs are interchangeable. |
Beta Was this translation helpful? Give feedback.
-
|
hi @ccurme, thanks for this excellent package! Do you have any input on this? Since I'm also unable to exchange OpenAI / Anthropic models in my code QuivrHQ/quivr#3485 |
Beta Was this translation helpful? Give feedback.
-
|
This is expected but I appreciate that it's a pain point. The message If you want to stream string tokens out of either OpenAI or Anthropic, I'd suggest using from langchain_anthropic import ChatAnthropic
from langchain_core.output_parsers import StrOutputParser
from langchain_core.tools import tool
@tool
def get_weather(location: str) -> str:
"""Get the weather from a location."""
return "Sunny."
llm = ChatAnthropic(model="claude-3-5-haiku-latest")
llm_with_tools = llm.bind_tools([get_weather])
parser = StrOutputParser()
for chunk in llm_with_tools.stream("What's the weather in San Francisco, CA?"):
print(parser.invoke(chunk), end="|")(You can use this with OpenAI and other providers as well.) Importantly, we do broadly intend for chat models to be interchangeable, so please let me know if this does not solve your problem. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot @ccurme for the swift reply, I'll have a look at the proposed solution and provide some feedback asap |
Beta Was this translation helpful? Give feedback.
-
Checked
Feature request
Update the
ChatAnthropicclass to work with the newest function calling features. Notably streaming, force tool use and vision inputs.Motivation
Streaming and force tool use were a long waited feature for agents based on tools calling with Anthropic models.
Proposal (If applicable)
I tried setting
default_headers = {"anthropic-beta": "tools-2024-05-16"}but it does not seem to work.Beta Was this translation helpful? Give feedback.
All reactions