Manually setting up Langchain Callback handler with explicit values doesn't configure langfuse client in decorator approach

[FaresKi]

I have a method similar to this:

@observe()
    def _query_llm(
        self, prompt: BasePromptTemplate | Sequence[BaseMessage] | str
    ) -> Any:
        """
        Queries the language model with the given prompt.
        """
        is_test = os.getenv("IS_EVAL", "NONE").lower() == "true"

        if is_test:
            return self.llm.invoke(prompt).content
        else:
            handler = CallbackHandler(public_key=os.getenv("PUBLIC_KEY"), secret_key=os.getenv("SECRET_KEY")) # standard callback
            return self.llm.invoke(
                prompt,
                config={
                    "callbacks": [handler],
                },  # type: ignore
            ).content

When calling the handler, we get this warning:
Langfuse client is disabled since no public_key was provided as a parameter or environment variable 'LANGFUSE_PUBLIC_KEY'

Now I'm looking at the documentation, and I'm not sure whether I should configure the langfuse_context via the langfuse_context.configure() method or whether the callback handler config is enough? and if so, why is the warning message occurring? I've slightly modified your code locally to ensure the public and secret keys are provided:
python3.11/site-packages/langfuse/utils/base_callback_handler.py:55 UserWarning: Public key: pk-lf-xxx
python3.11/site-packages/langfuse/utils/base_callback_handler.py:56 UserWarning: Secret key: sk-lf-xxx

Additional information

I can confirm that if the langfuse context is setup, then you fetch the handler from the context, the warning message disappears.
however if the callback handler is set manually, langfuse is disabled:

langfuse_context.configure(
    public_key=os.getenv("PUBLIC_KEY"),
    secret_key=os.getenv("SECRET_KEY"),
    host=os.getenv("NEXTAUTH_URL"),
)

...

    @observe()
    def _query_llm(
        self, prompt: BasePromptTemplate | Sequence[BaseMessage] | str
    ) -> Any:
        """
        Queries the language model with the given prompt.
        """
        is_test = os.getenv("IS_EVAL", "NONE").lower() == "true"

        if is_test:
            return self.llm.invoke(prompt).content
        else:
            handler = langfuse_context.get_current_langchain_handler()
            langfuse_context.update_current_trace(
                session_id=self.run_id,
                metadata=self.metadata,
            )
            return self.llm.invoke(
                prompt,
                config={
                    "callbacks": [handler],
                },  # type: ignore
            ).content

[dosubot[bot]]

I found a similar discussion regarding the configuration of the Langfuse client when using the CallbackHandler and the observe() decorator ^[1].

In your case, the warning message occurs because the CallbackHandler is not properly configuring the Langfuse client when set manually. The langfuse_context.configure() method is necessary to ensure that the Langfuse client is correctly initialized with the required API keys and other settings. This method should be called at the top of your application to set up the client before any decorated functions are executed.

When you manually set the CallbackHandler without using langfuse_context.configure(), the Langfuse client might not recognize the provided keys, leading to the warning message and the client being disabled. By configuring the langfuse_context and fetching the handler from the context, you ensure that the client is properly set up, and the warning message disappears.

Therefore, it's recommended to use langfuse_context.configure() to set up the Langfuse client, and then use langfuse_context.get_current_langchain_handler() to obtain the handler for your callback configuration. This approach ensures that the client is enabled and properly configured.

To continue talking to Dosu, mention @dosu.

[FaresKi]

@dosu Agreed. In that case it's best to update the documentation here: https://langfuse.com/docs/integrations/langchain/tracing#add-langfuse-to-your-langchain-application

Because there is no mention of configuring the langfuse_context. I could make a PR if it's still the expected behavior @marcklingen

[dosubot[bot]]

Hey @FaresKi! It sounds like you're diving into documentation updates, which is super important. However, I'm just a bot here to help with bugs, technical questions, and engineering knowledge. For documentation changes or PRs, you might want to reach out to the Dosu Team to see if they can assist you further. Keep up the great work!

To continue talking to Dosu, mention @dosu.

[hassiebp]

Hi @FaresKi - the easiest way to make sure all clients are configured correctly is to set the environment variables as described in our docs. That should silence all warnings and deliver events correctly to Langfuse:

LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_HOST="https://cloud.langfuse.com" # 🇪🇺 EU region
# LANGFUSE_HOST="https://us.cloud.langfuse.com" # 🇺🇸 US region

[FaresKi]

Understood and I agree. However the behavior should be the same in both cases. If it's explicitly provided, then the client should be configured & initialized.

[marcklingen]

Did this resolve the issue for you? Thanks for sharing your feedback on the sdk initialization!