Why do Diffusers schedulers produce lower quality outputs compared to ComfyUI?

[nducthang]

Hi,

I'm encountering an issue when comparing the quality of ComfyUI and Diffusers. I've noticed that the output of Diffusers is consistently lower than ComfyUI in many cases, despite using the same settings and seed. For the base Diffusers, I've utilized: https://github.com/huggingface/diffusers/blob/main/examples/community/lpw_stable_diffusion_xl.py.

Upon closer inspection, I've identified differences in the scheduler/ksampler between the two base codes. I've also observed variations in CLIP Embedding between the two base codes, but in my experiments, this hasn't significantly impacted the output. The main issue seems to lie with the KSampler.

Has anyone else encountered this issue or have any ideas on improving the Scheduler algorithm of Diffusers?

Here are some prompts I've experimented:
Model: RVXL - Size: (896, 1152)
Positive prompt:

female, attractive woman, pretty middle-aged woman, thick hair, (((Caucasian, European, Scandinavian female))), ((hazel eyes, HazelEyed)). (Brunette (Light-Brown-Hair)), ((((long rectangular face, elongated face, oblong face shape, angular chiseled face)), ((wide jaw, big strong chin)))). (((1980s magazine advertisement. Living room. CRT Televesion. 1980s aesthetic. 1980s interior design.))) [object Object] . high quality, dim lighting, soft lighting, sharp focus, f5.6, dslr, High Detail, detailed, ((wide shot))

Negative prompt:

(((male))), (small chin, receding-chin, puffy face), (((Asian, Chinese, Korean, Japanese, Indian, Pakistani, Black, African, Persian, Arab, Middle Eastern, Hispanic, Latino))), (small chin, receding-chin, puffy face), (blurry), (BadDream:1.2), (UnrealisticDream:1.2), ((bad-hands-5)), (strabismus, cross-eyed:1.2), (signature, watermark, name), (worst quality, poor quality, low quality), ((deformed)), (extra limbs), (extra arms), (extra legs), disfigured, malformed, (nude:1.4), (naked:1.4), (nsfw:1.4), (bikini:1.4), (lingerie:1.4), (underwear:1.4), (teen:1.4), (tween:1.4), (teenage:1.4), (kid:1.6), (child:1.6), (topless, shirtless:1.4), (((greyscale))), (cleavage:1.2), (nipples:1.4)

[peki12345]

I have also noticed this issue and feel that the problem lies with sampling. And even some advanced technologies, such as https://github.com/lllyasviel/Omost/blob/731e74922fc6be91171688574d07624f93d3b658/gradio_app.py#L64 give up diffusers sampling.

[asomoza]

Hi, that was the authors choice, doesn't write why though. That code uses a single hard coded sampler which is not a great choice, automatic1111 at least uses the k diffusion repo, which is a choice you can make here too.

Without images we can't help that much here, most of the time is a problem with the users that are doing something that makes the generations worse.

If you're going to use images and show examples we prefer if they aren't human beings so I can really test that prompt here to show some results.

[nducthang]

Hi, I replied in link: #8683 , thks

[christopher-beckham]

@peki12345 @nducthang I'm a bit late to this discussion, I'm only replying as I did have issues in the past with a supposed scheduler in diffusers acting differently to its equivalent in k-diffusers. It can be hard to reconcile those differences since the way schedulers in diffusers are implemented is different to other libs. Because of that I wrote a proof-of-concept repo which "closures" the backbone and makes it possible to pass directly to a k_diffusion sampler. That could be useful either if you want to debug one of those algorithms against an equivalent in diffusers, or if you want to bypass it completely and delegate to external code.

Happy to accept PRs to steer it more towards a library which can be used to directly compare an implementation in diffusers vs k_diffusion.