Langfuse observation for openai embeddings method

[simrangulati-jktech]

Hi,
I want to trace the OpenAI embeddings method using langfuse, but in Langfuse dashboard, I can't see any cost or token count in the trace generated. While in same function OpenAI().chat.completetions.create function worked fine. Can you pls update if I am missing anything in the code.
Thanks,
Simran

import json
from langfuse.decorators import observe
from langfuse import Langfuse

langfuse = Langfuse()

@observe()
def generate_openai_embeddings(text: str = "", model: str = "text-embedding-ada-002"):
    if text == "":
        return None
    client = OpenAI(api_key=OPENAI_API_KEY)
    text = text.replace("\n", " ")
    
    embeddings = json.dumps(client.embeddings.create(input=[text], model=model).data[0].embedding)
    return embeddings


emdeddings = generate_openai_embeddings(text="Open AI new Embeddings models is great.")

[dosubot[bot]]

Hello @simrangulati-jktech! I'm here to help you with any bugs, questions, or contributions you have. Let's take a look at the issue you're facing with the OpenAI embeddings method using Langfuse.

I found a similar discussion that might be relevant to your issue. In the discussion titled Generation does not come under @observe trace, it was mentioned that you need to get the callback handler from langfuse_context to properly trace the function ^[1].

To trace the OpenAI embeddings method and see the cost or token count in the generated trace, you can follow these steps:

1. Ensure Token and Cost Tracking:

   • Langfuse can automatically infer the cost and token usage for supported models like OpenAI. Ensure you are using the correct model names supported by Langfuse.

2. Use the Callback Handler:

   • You need to get the callback handler from langfuse_context and add it to your OpenAI method call.

Here is an updated version of your code that includes these steps:

import json
from langfuse.decorators import observe, langfuse_context
from langfuse import Langfuse
from openai import OpenAI

langfuse = Langfuse()

@observe()
def generate_openai_embeddings(text: str = "", model: str = "text-embedding-ada-002"):
    if text == "":
        return None
    client = OpenAI(api_key=OPENAI_API_KEY)
    text = text.replace("\n", " ")
    
    # Update trace attributes if needed
    langfuse_context.update_current_trace(
        name="openai-embeddings-trace",
        session_id="user-1234",
        user_id="session-1234",
    )
    
    # Get the Langfuse handler for the current trace
    langfuse_handler = langfuse_context.get_current_langchain_handler()
    
    # Create embeddings with the Langfuse handler as a callback
    embeddings = json.dumps(client.embeddings.create(input=[text], model=model, config={"callbacks": [langfuse_handler]}).data[0].embedding)
    return embeddings

embeddings = generate_openai_embeddings(text="Open AI new Embeddings models is great.")

This code ensures that the OpenAI embeddings method is properly traced, and the cost and token count should be visible in the Langfuse dashboard ^[1].

For more detailed information, you can refer to the Langfuse documentation ^[1].

To continue talking to Dosu, mention @dosu.

[jannikmaierhoefer]

HI @simrangulati-jktech, can you try to trace the OpenAI embeddings method as an observation (@observe(as_type="generation"))? Langfuse tracing currently has 3 distinct observation types (docs): spans, events, generations. Currently embedding jobs can be tracked via generations (add all to input, cost tracking works as expected).

We currently have an open post in our GitHub Ideas board. Feel free to upvote here to get updates: https://github.com/orgs/langfuse/discussions/1021

[gsouf]

@marcklingen

In trace table:

Trace details:

Generation details:

The end of the output object (after the embeddings) is like so:

model: "ada"
usage: {
total_tokens: 103
prompt_tokens: 103
}
object: "list"

Maybe the issue is that the model name is simply "ada"?

That's the observed method in full:

    @observe(as_type="generation")
    async def base_create_embeddings(self, texts):
        res = await self.client.embeddings.create(input = texts, model="text-embedding-ada-002")
        langfuse_context.update_current_observation(
            usage={
                "input": res.usage.prompt_tokens,
                "output": 0,
                "total": res.usage.prompt_tokens,
                "unit": "TOKENS",
            }
        )
        return res

And the openai client is from langfuse.openai import AsyncAzureOpenAI

[marcklingen]

Interesting, the model here does not get logged on the langfuse generation. @hassiebp have you seen this before (Azureopenai)?

[hassiebp]

It looks like the model returned from Azure OpenAI is simply ada rather than the full model name. @gsouf , as a fix could you please try adding the model to the update_current_observation call?

    @observe(as_type="generation")
    async def base_create_embeddings(self, texts):
        res = await self.client.embeddings.create(input = texts, model="text-embedding-ada-002")
        langfuse_context.update_current_observation(
            usage={
                "input": res.usage.prompt_tokens,
                "output": 0,
                "total": res.usage.prompt_tokens,
                "unit": "TOKENS",
            },
            model="text-embedding-ada-002"
        )
        return res

[gsouf]

@hassiebp I can definitely do that, and also just add the price directly. But it would be nice to have it working out of the box!

[hassiebp]

👍🏾 Tracking issue here: #3738