SNOW-1805840: Augment telemetry with method_call_count

[sfc-gh-lmukhopadhyay]

1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

   Fixes SNOW-1805840

2. Fill out the following pre-review checklist:

   • I am adding a new automated test(s) to verify correctness of my new code
     • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
   • I am adding new logging messages
   • I am adding a new telemetry message
   • I am adding new credentials
   • I am adding a new dependency
   • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
   • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines

3. Please describe how your code solves the related issue.

Adding method_call_count which is the # of times a pandas API method has been called, and interchange_call_count which is the # of times the __dataframe__ method has been called.

See more info in the interchange protocol design doc here: https://docs.google.com/document/d/1EfqQwejVbF5_36hnOP-ap0t3NaCWmDz62iAcR0PtX20/edit?tab=t.0#heading=h.4uu48icmuq7z

[sfc-gh-mvashishtha]

can we generate the function name with _gen_func_name, as we do for the API call list? https://github.com/snowflakedb/snowpark-python/blob/2247ae5b14ef83c75b5ea6707dd6e53499c197f1/src/snowflake/snowpark/modin/plugin/_internal/telemetry.py#L316C17-L316C31

[sfc-gh-mvashishtha]

LGTM! Thanks

[sfc-gh-joshi]

Can you add a test where 2 different methods are called? Also, how does this interact with query compiler methods that call other query compiler methods?

[sfc-gh-lmukhopadhyay]

I've added a test that calls two different methods on the same QC now.

Trying out align, it returns the telemetry with its own func name for ex
[{'func_name': 'DataFrame.align', 'category': 'snowpark_pandas', 'error_msg': None, 'call_count': 1, 'api_calls': [{'name': 'DataFrame.align'}]}]

Did you have another specific method in mind?

[sfc-gh-joshi]

As an example, SnowflakeQueryCompiler.any with axis=0 calls SnowflakeQueryCompiler._bool_reduce_helper, which then in turn calls SnowflakeQueryCompiler.agg. I was wondering how this would be reflected in telemetry.

[sfc-gh-lmukhopadhyay]

In this case _query_compiler._method_call_counts would include the func_name count 'DataFrame.BasePandasDataset.any' = 1 as well as the other attributes called tracked with telemetry such as 'DataFrame.__repr__' and 'DataFrame.property.iloc_get'.

[sfc-gh-joshi]

I see. Would it be accurate to say that _query_compiler._method_call_counts only tracks methods called on this particular instance of query compiler (the example call chain I gave returns a new query compiler instance every time), and these counts are only used in telemetry for certain frontend methods like dataframe and repr that we specify explicitly?

[sfc-gh-lmukhopadhyay]

Yes exactly the goal is to track call counts on the same query compiler instance, more info is in the design doc here: https://docs.google.com/document/d/1EfqQwejVbF5_36hnOP-ap0t3NaCWmDz62iAcR0PtX20/edit?tab=t.0#heading=h.4uu48icmuq7z

[sfc-gh-helmeleegy]

The PR description is talking about introducing interchange_call_count, which I don't think is accurate -- there is no such variable/field.

[sfc-gh-helmeleegy]

Overall looks good to me. Just left a few nits/questions.

[sfc-gh-helmeleegy]

Is this second call served from some cache or recomputed?

[sfc-gh-lmukhopadhyay]

I believe the telemetry call data is recomputed every time, @sfc-gh-azhan might confirm

[sfc-gh-joshi]

As an example, SnowflakeQueryCompiler.any with axis=0 calls SnowflakeQueryCompiler._bool_reduce_helper, which then in turn calls SnowflakeQueryCompiler.agg. I was wondering how this would be reflected in telemetry.