Push new contexts on the stack via ContextManager?

[chrskrchr]

Hi! I'm working with a group of folks to build out a web platform on top of Fastify.js that includes a collection of plugins to handle common functionality like logging, authentication, tracing, etc... We've adopted opentelemetry for tracing, and we have a use case where it would be nice for a plugin to be able to add additional entries to the otel baggage collection and have that extra baggage propagated on all outbound HTTP requests to other services.

The challenge we're running into is that Fastify executes its plugins in a non-nested fashion, which makes it impossible AFAICT for a plugin to do something like api.context.with(newContextWithExtraBaggage, callback) to push a new context on the stack such that it becomes the "active" context for the rest of the request - including both the execution of other plugins that are executed for the request and the route handler that ultimately responds to the request.

I guess my first question: are we being dumb/naive in how we're attempting to go about this? I understand the desire for the context/baggage objects themselves to be immutable, but the inability to activate new contexts appears to prevent any non-nested control flows from decorating the context with additional stuff, e.g. baggage. FWIW, it seems like we may not be the first people to run into this issue:

• OpenTracing shim doesn't change context #2016 (comment)
• How to created nested spans without passing parents around #1963 (comment)

One possible solution would be to allow apps to push/pop contexts on the stack such that they weren't required to use context.with(..., callback) to activate a new context. This would put a burden on the app to properly push/pop the contexts, but maybe that's not such a terrible thing given the additional flexibility?

[rrichardson]

I, too, would like an alternative to context.with() for similar reasons. I have built custom code-gen and middleware around node and grpc, I've had to use context.with() and what is basically a no-op function, just to be able to init an active context and then get its handle.

Note this is precisely a non-answer.

[dyladan]

We are considering adding attach and detach methods to the context manager which change the context for the current async execution instead of using with which creates a new one. Would this work for your usecase?

[maorleger]

Same need here, but for different reasons - neither context.with nor context.bind nor tracer.startActiveSpan would work for our use-case which does not rely on callbacks when creating spans.

Internally within our SDK we rely on explicit context passing in order to parent things correctly; however, that breaks down as soon as the consumer introduces any other instrumentations which are then not nested correctly. You might start a span, call setSpan to set it on a context, but there's no way to hand that context over to other instrumentation libraries which will likely use context.active()

For our use-case it's not always possible to execute all the operations within a callback, thus we need some other way to set the active context for the remainder of the request.

Since our ask is the same, albeit for different reasons, I thought it's better to tack onto this discussion.

cc @lmolkova

[dyladan]

We are considering adding attach and detach methods to the context manager which change the context for the current async execution instead of using with which creates a new one. Would this work for your usecase? In your comment you mention "in our sdk". Have you implemented an alternative otel sdk or do you mean an sdk which happens to use otel?

[maorleger]

I mean an sdk which happens to use otel. I do think attach and detach would work for our usecase. Thanks!

[dyladan]

We are considering adding attach and detach methods to the context manager which change the context for the current async execution instead of using with which creates a new one. Would this work for your usecase?

[lmolkova]

@dyladan, it sounds great! do you have any timeline in mind? Thanks!

[aitskovi]

This would be very useful for us as well. Is there any progress on this API by any chance?

[MannyPamintuanWorkAccount]

Hi @dyladan , following up on other's questions, I was curious if there was any tentative timeline in mind for these methods to be added to the context manager API?

I look forward to hearing back!

[dyladan]

We have a prototype and a potential api PR but changing the context manager is a potentially very impactful change so it needs to be done delicately. I do not have any timeline but it is something we are definitely hoping to add

[MannyPamintuanWorkAccount]

I do not have any timeline but it is something we are definitely hoping to add

Thank you for the update, Daniel! 🤝

changing the context manager is a potentially very impactful change so it needs to be done delicately

Understood, I follow the need to be delicate and intentional with changes to context manager 👍🏽

If there are ways the community/consumers are able to support this, please let us know!

• If you'd like us to trial the potential api updates or provide feedback on thoughts, etc. 🙂

[lmolkova]

a quick update: we decided to use available callback-based APIs, so we don't depend on this feature.