Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: improve logging and add optional perf logging (#248)
Mostly minor changes: - Remove the use of `Console` from server.py. This cascades into other changes downstream - Add latency logs behind a `LOG_PERFORMANCE` env var: - Add a wrapper for all API calls to measure the latency at API level - Add detailed performance monitoring to the tool_calling_llm core `call()` function - Align the http server (uvicorn) logging pattern with the other logging calls Example of perf metrics for tool_calling_llm: ``` 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(TOTAL) 8167ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(get_all_tools_openai_format) +0ms 0ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(start iteration 0) +0ms 0ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(count tokens) +6ms 6ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(llm.completion) +1831ms 1837ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(pre-tool-calls) +0ms 1837ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(tool completed fetch_finding_by_id) +570ms 2407ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(end iteration 0) +0ms 2407ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(start iteration 1) +0ms 2407ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(count tokens) +7ms 2415ms 2025-01-14 09:41:17.177 INFO tool_calling_llm.call(llm.completion) +5752ms 8167ms ```
- Loading branch information