Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python[patch]: use bind_tools when pulling structured prompt #1380

Closed
wants to merge 2 commits into from
Closed

python[patch]: use bind_tools when pulling structured prompt #1380

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
61ec312
python[patch]: use bind_tools when pulling structured prompt
baskaryan Jan 6, 2025
552cbc5
fmt
baskaryan Jan 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions python/langsmith/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Client for interacting with the LangSmith API.

Check notice on line 1 in python/langsmith/client.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results 

...........
create_5_000_run_trees: Mean +- std dev: 674 ms +- 59 ms
...........
create_10_000_run_trees: Mean +- std dev: 1.37 sec +- 0.08 sec
...........
create_20_000_run_trees: Mean +- std dev: 1.33 sec +- 0.11 sec
...........
dumps_class_nested_py_branch_and_leaf_200x400: Mean +- std dev: 711 us +- 11 us
...........
dumps_class_nested_py_leaf_50x100: Mean +- std dev: 25.3 ms +- 0.2 ms
...........
dumps_class_nested_py_leaf_100x200: Mean +- std dev: 105 ms +- 3 ms
...........
dumps_dataclass_nested_50x100: Mean +- std dev: 25.7 ms +- 0.5 ms
...........
WARNING: the benchmark result may be unstable
* the standard deviation (19.6 ms) is 26% of the mean (75.8 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

dumps_pydantic_nested_50x100: Mean +- std dev: 75.8 ms +- 19.6 ms
...........
dumps_pydanticv1_nested_50x100: Mean +- std dev: 199 ms +- 3 ms

Check notice on line 1 in python/langsmith/client.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main 

+-----------------------------------------------+----------+------------------------+
| Benchmark                                     | main     | changes                |
+===============================================+==========+========================+
| dumps_pydanticv1_nested_50x100                | 222 ms   | 199 ms: 1.12x faster   |
+-----------------------------------------------+----------+------------------------+
| create_5_000_run_trees                        | 739 ms   | 674 ms: 1.10x faster   |
+-----------------------------------------------+----------+------------------------+
| create_20_000_run_trees                       | 1.44 sec | 1.33 sec: 1.08x faster |
+-----------------------------------------------+----------+------------------------+
| create_10_000_run_trees                       | 1.46 sec | 1.37 sec: 1.06x faster |
+-----------------------------------------------+----------+------------------------+
| dumps_class_nested_py_leaf_50x100             | 25.2 ms  | 25.3 ms: 1.00x slower  |
+-----------------------------------------------+----------+------------------------+
| dumps_dataclass_nested_50x100                 | 25.4 ms  | 25.7 ms: 1.01x slower  |
+-----------------------------------------------+----------+------------------------+
| dumps_class_nested_py_leaf_100x200            | 104 ms   | 105 ms: 1.01x slower   |
+-----------------------------------------------+----------+------------------------+
| dumps_class_nested_py_branch_and_leaf_200x400 | 690 us   | 711 us: 1.03x slower   |
+-----------------------------------------------+----------+------------------------+
| dumps_pydantic_nested_50x100                  | 67.6 ms  | 75.8 ms: 1.12x slower  |
+-----------------------------------------------+----------+------------------------+
| Geometric mean                                | (ref)    | 1.02x faster           |
+-----------------------------------------------+----------+------------------------+

Use the client to customize API keys / workspace ocnnections, SSl certs,
etc. for tracing.
Expand Down Expand Up @@ -6435,7 +6435,10 @@
Any: The prompt object in the specified format.
"""
try:
from langchain_core.language_models.base import BaseLanguageModel
from langchain_core.language_models import (
BaseChatModel,
BaseLanguageModel,
)
from langchain_core.load.load import loads
from langchain_core.output_parsers import BaseOutputParser
from langchain_core.prompts import BasePromptTemplate
Expand Down Expand Up @@ -6506,7 +6509,9 @@
rebound_llm = seq.steps[1]
prompt = RunnableSequence(
prompt.first,
rebound_llm.bind(**{**prompt.last.kwargs}),
cast(BaseChatModel, rebound_llm).bind_tools(
**prompt.last.kwargs
),
seq.last,
)
else:
Expand Down
Loading