[Bug]: Vertex Credentials Are Cached

[hakan458]

What happened?

It seems that when using litellm to make calls to vertex_ai models, the vertex credentials are somehow cached within litellm and the ones passed into completion() are not being used on every call

>>> litellm.completion(
...     messages=[{'role': 'user', 'content': "Hey, how's it going?"}], 
...     model='vertex_ai/gemini-1.5-flash', 
...     max_tokens=10, 
...     temperature=0.0, 
...     vertex_credentials='sdsdfsdsdfsdfsddsff'
... )

Exception: Unable to load vertex credentials from environment. Got=sdsdfsdsdfsdfsddsff

So far so good... but then if I use real working credentials, any subsequent calls with anything passed into vertex_credentials still works, when it should not

>>> creds = <my actual google json credentials> 
>>> litellm.completion(
...     messages=[{'role': 'user', 'content': "Hey, how's it going?"}], 
...     model='vertex_ai/gemini-1.5-flash', 
...     max_tokens=10, 
...     temperature=0.0, 
...     vertex_credentials=creds
... )

ModelResponse(id=' ................)  # successful response

>>> litellm.completion(
...     messages=[{'role': 'user', 'content': "Hey, how's it going?"}], 
...     model='vertex_ai/gemini-1.5-flash', 
...     max_tokens=10, 
...     temperature=0.0, 
...     vertex_credentials='sdsdfsdsdfsdfsddsff'
... )

ModelResponse(id=' ................)  # successful response when it should not be

Relevant log output

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

1.47.2

Tried with 1.59.1 as well, same behavior

Twitter / LinkedIn details

No response

[krrishdholakia]

Thanks for the issue @hakan458 i plan to look into this, this week.