Update streaming processing python

[brnaba-aws]

Issue number:
Adding full streaming support for python.

Summary

Full streaming in python using yield

Changes

route_requesthas now a new parameter, stream_response to either get the Agent's response in chunks (streaming) rather than passing by Callbacks.

async def route_request(self,
                       user_input: str,
                       user_id: str,
                       session_id: str,
                       additional_params: dict[str, str] = {},
                       stream_response: bool | None = False  # wether to stream back the response from the agent
) -> AgentResponse:
        """Route user request to appropriate agent."""

User experience

Please share what the user experience looks like before and after this change

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• I have performed a self-review of this change
• Changes have been tested
• Changes are documented

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[hghandri]

Hi,

do you mean with this approach we don't anymore to use callback agent through "on_llm_new_token" ?

[brnaba-aws]

It might still be needed in some cases. Also, we want to be backward compatible so we would keep it.