Webhook 'output' event not fired, num_outputs done wrong?

[Pwntus]

Webhook: output

If num_outputs input parameter is > 1 the predict function will yield outputs while they are being generated (like here: https://github.com/anotherjesse/dream-templates/blob/main/controlnet-1.5-canny/predict.py#L313)

I don't seem to receive the output webhook event, although I've set the output webhook event filter in the prediction API-call. Without this webhook event it makes it very hard to logically determine which number in the num_outputs process to update in a client application.

---

Iterating num_outputs

I also suspect iterating over the num_outputs input parameter and running image generation on each of them is not optimal. I've always seen it done by multiplying the prompts by num_outputs (like here: https://github.com/replicate/cog-stable-diffusion/blob/main/predict.py#L108). It appears to be taking num_outputs-times longer using this method.

dream-templates way results in this:

Traditional (multiplying prompts) results in this:

[anotherjesse]

I also suspect iterating over the num_outputs input parameter and running image generation on each of them is not optimal.

I haven't run the numbers, but I don't remember it seeming longer.

The real reason to do this is that it allows you to have a deterministic seed for output 2-N.

Currently if you like the 3rd output when doing the [prompt] * num_outputs, you cannot get back to it, causing frustration.

---

Regarding iterator vs returning at the end, it seems like maybe there is an issue with webhooks & iterators. That might be able to be fixed.

[Pwntus]

Assuming we'll get the webhook event-fixed; Retrieving the 3rd seed is then probably necessary for doing it like this, I'll run some numbers regarding yield vs. non-yield to see if we're sacrificing any speed. I'm adding a yield_output -input parameter that will switch output-strategy, I think if either option has pros/cons the option to switch should be there.

[anotherjesse]

I also want to get AIT supported. Which will require this style of generation. Whether yield or all at once is orthogonal

…

On Sat, Apr 1, 2023 at 7:41 PM Pontus Aurdal ***@***.***> wrote: Assuming we'll get the webhook event-fixed; Retrieving the 3rd seed is then probably necessary for doing it like this, I'll run some numbers regarding yield vs. non-yield to see if we're sacrificing any speed. I'm adding a yield_output -input parameter that will switch output-strategy, I think if either option has pros/cons the option to switch should be there. — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAAG6SFMIYRF4CN6ES6H3W7C4JBANCNFSM6AAAAAAWP2J4UQ> . You are receiving this because you commented.Message ID: ***@***.***>

[Pwntus]

Here are the numbers (4 samples). All at once is negligible faster.

num_outputs = 4 (50 steps)

Yield	At once
35.3s	30.5s
31.5s	31.0s
32.2s	31.0s
33.6s	31.8s

Avg:

33.15s	31.07s

Until webhook output event is functional it isn't orthogonal in my use-case though.