Streaming http responses

[maxmcd]

Allow streaming http responses for things like long polling or incrementally streaming a live response to a browser. Ie, letting me implement https://robpike.io/ from val.town :)

I'm not sure this is super important in the grand scheme of things, but would certainly open things up for more creative/fringe http uses.

[saolsen]

I'd like this too, was playing around with SSE but realized responses don't stream so it doesn't work. https://www.val.town/v/saolsen/hono_sse

[saolsen]

Another wild idea I had was building something like https://streamlit.io/ but on top of vals. The way that lib works is the python script executes and streams a bunch of events to the browser (via a websocket) which executes them. I think I could build a similar thing with valtown, and server sent events. Would require streaming responses though.

[tarasglek]

I can't write llm wrappers for https://chatcraft.org properly without this.

[stevekrouse]

Quick update - @maxmcd, the original feature requester, now works at Val Town, and is working to ship this in the coming weeks! We'll keep you all updated

[maxmcd]

Val Town now supports streaming! You can even implement robpike.io: https://www.val.town/v/maxm/robPikeIO

Along with many other fun things:

• https://www.val.town/v/maxm/SSEDemo
• https://www.val.town/v/maxm/streamingResponseExample
• https://www.val.town/v/maxm/valTownChatGPT
• https://www.val.town/v/maxm/openAIStreaming

Let us know if you run into any issue using streaming, or if you need features we don't support!

[simonw]

Bug report: https://docs.val.town/std/openai/ still says that streaming isn't supported.

https://github.com/val-town/val-town-docs/blob/432296e46c1f745c4922c7d61fed0355bf755e04/src/content/docs/std/openai.mdx

[stevekrouse]

Fixed! Thanks