Logging LLM provider request ids as gen_ai attributes

[nirga]

Discussed in #2174

^{Originally posted by dinmukhamedm October 18, 2024}
It would be very useful for debugging purposes (e.g. with OpenAI support) to have a unique identifier to an LLM call span. Has OpenLLMetry considered adding something like gen_ai.request.id attribute?

The biggest challenge I see with this is that they are not unified, and formatted differently across providers and even across endpoints of a single provider (e.g. completions vs assistant). Generally, there is a request-wide unique ID though, and if there is none, the attribute can obviously remain optional.

[dinmukhamedm]

@nirga for this, we'll need to first uncomment https://github.com/traceloop/openllmetry/blob/main/packages/opentelemetry-semantic-conventions-ai/opentelemetry/semconv_ai/__init__.py#L53 and release a new version of opentelemetry.semconv_ai. Could you please do that? I am happy to try and add this to some of the providers/endpoints afterwards.

[nirga]

@dinmukhamedm can you create a PR with that - I'll release the version then on top of your PR

[dinmukhamedm]

I've started a few PRs, so here's the status:

• Alephalpha. N/A
• Anthropic. fix(anthropic): add response id attribute #2543
• Bedrock. fix(bedrock): add response id attribute #2544
• ChromaDB. N/A
• Cohere. fix(cohere): add response id attribute #2545
• Google generativeai. N/A
• Groq. fix(groq): add response id attribute #2546
• Haystack. N/A
• LanceDB. N/A
• Langchain. fix(langchain): add response id (anthropic only) #2548
• Llamaindex. Will be resolved by OpenAI (fix(openai): add response id attribute #2550)
• Marqo. N/A
• Milvus. N/A
• MistralAI. fix(mistral): add response id attribute #2549
• Ollama. N/A
• Openai. fix(openai): add response id attribute #2550
• Pinecone. N/A
• Qdrant. N/A
• Replicate. N/A. It does return the id in the API response, but not in the SDK
• Sagemaker. N/A
• Together. fix(together): add response id attribute #2551
• Transformers. N/A
• VertexAI. N/A
• WatsonX. Rather a TODO, but I thought it wasn't worth for me to figure out how the new package works. The instrumented package is deprecated https://pypi.org/project/ibm-watson-machine-learning/
• Weaviate. N/A. Most responses have IDs according to the API reference, but arguably vector DB ain't GEN_AI.

[dinmukhamedm]

@nirga do you think this work is sufficient to close the issue? WatsonX was a bit difficult to set up, plus, the instrumented library is on deprecation path. I know people from IBM actually have contributed to the library, so if they want they could implement this as well?

[nirga]

@gyliu513 wdyt?