Skip to content

Commit

Permalink
feat: improve logging and add optional perf logging (#248)
Browse files Browse the repository at this point in the history
Mostly minor changes:

- Remove the use of `Console` from server.py. This cascades into other
changes downstream
- Add latency logs behind a `LOG_PERFORMANCE` env var:
  - Add a wrapper for all API calls to measure the latency at API level
- Add detailed performance monitoring to the tool_calling_llm core
`call()` function
- Align the http server (uvicorn) logging pattern with the other logging
calls

Example of perf metrics for tool_calling_llm:

```
2025-01-14 09:41:17.177 INFO     tool_calling_llm.call(TOTAL) 8167ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(get_all_tools_openai_format) +0ms  0ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(start iteration 0) +0ms  0ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(count tokens) +6ms  6ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(llm.completion) +1831ms  1837ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(pre-tool-calls) +0ms  1837ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(tool completed fetch_finding_by_id) +570ms  2407ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(end iteration 0) +0ms  2407ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(start iteration 1) +0ms  2407ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(count tokens) +7ms  2415ms
2025-01-14 09:41:17.177 INFO     	tool_calling_llm.call(llm.completion) +5752ms  8167ms
```
  • Loading branch information
nherment authored and moshemorad committed Jan 27, 2025
1 parent 1f6c391 commit 1cb3986
Showing 1 changed file with 0 additions and 6 deletions.
6 changes: 0 additions & 6 deletions holmes/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,12 +142,6 @@ def load_from_env(cls):
if val is not None:
kwargs[field_name] = val
kwargs["cluster_name"] = Config.__get_cluster_name()
try:
if os.path.exists(DEFAULT_CONFIG_LOCATION):
logging.info(f"Loading config from file with {len(kwargs)} addional params from env")
return cls.load_from_file(config_file=DEFAULT_CONFIG_LOCATION, **kwargs)
except:
logging.exception("Failed to load config from secret, falling back to loading config from env vars")
return cls(**kwargs)

@staticmethod
Expand Down

0 comments on commit 1cb3986

Please sign in to comment.