[Bug]: --gradio-img2img-tool color-sketch breaks "send to img2img" button + weird bug on inpaint tab + Image Browser extension "Send to.." bug

[Nacurutu]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

I have --gradio-img2img-tool color-sketch on set COMMANDLINE_ARGS=

When I press the "send to img2img" button, the image in the img2img tab appears and after 1 second becomes a solid color.

Removing --gradio-img2img-tool color-sketch from set COMMANDLINE_ARGS= makes the send to img2img button works properly and the image appears as it should.

But, If I open an image directly on img2img tab, I can use the "paint tool" without a problem.
It only happens with --gradio-img2img-tool color-sketch and using the send to img2img button.

A second bug (not related with --gradio-img2img-tool color-sketch ARG) occurs on inpaint tab. it doubles the image after I start painting the mask.

The third bug (not related with --gradio-img2img-tool color-sketch ARG):

None of the "send to.." buttons of Image browser extension works. they send the image info but not the image.

Steps to reproduce the problem

Send to img2img button bug:

1. Add --gradio-img2img-tool color-sketch to "set COMMANDLINE_ARGS="

2. Run the WebUI

3. Generate an image on txt2image

4. Click on "send to img2img"

5. Image will break after 1 second.

6. Remove --gradio-img2img-tool color-sketch from "set COMMANDLINE_ARGS="

7. Run the WebUI

8. Generate an image on txt2image

9. Click on "send to img2img"

10. Image appears as it should.

Inpaint tab bug (not related with --gradio-img2img-tool color-sketch ARG):

1. Generate an image on txt2image
2. Click on "send to Inpaint"
3. Image appears as it should.
4. Start Painting a mask.
5. Image gets resized and doubled.

Image browser extension (not related to --gradio-img2img-tool color-sketch ARG):

1. Open "browser Extension" tab
2. Select an image
3. click on any of the "send to..." buttons

What should have happened?

Images appearing as they should on img2img with the --gradio-img2img-tool color-sketch argument active and using the send to img2img button.

Inpaint without a double

Image Browser "send to..." buttons to send images to destination.

Commit where the problem happens

310b71f

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--precision full --no-half --no-half-vae --medvram --opt-split-attention --xformers --opt-channelslast --api --deepdanbooru --gradio-img2img-tool color-sketch --theme=dark --autolaunch

--gradio-img2img-tool color-sketch breaking the send to img2img button

Additional information, context and logs

Google chrome console Log after Using "send to img2img" from txt2img

Sketch.svelte:119 Uncaught DOMException: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The image argument is a canvas element with a width or height of 0.
    at HTMLImageElement.<anonymous> (http://127.0.0.1:7860/assets/index.eee29f42.js:1:8332)
(anonymous) @ Sketch.svelte:119

[pyr0ball]

+1 for your 3rd bug.

send to img2img seems to be broken across the board for me.

Traceback (most recent call last):
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\routes.py", line 284, in run_predict
    output = await app.blocks.process_api(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 982, in process_api
    result = await self.call_function(fn_index, inputs, iterator)
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 824, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\pyr0ball\devl\Stable Diffusion\webui\modules\generation_parameters_copypaste.py", line 113, in send_image_and_dimensions
    img = image_from_url_text(x)
  File "C:\Users\pyr0ball\devl\Stable Diffusion\webui\modules\generation_parameters_copypaste.py", line 60, in image_from_url_text
    image = Image.open(io.BytesIO(filedata))
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\PIL\Image.py", line 3147, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x000001B3EF303010>

[Nacurutu]

This PR fixes the error with the "send to" on the image browser extension:

yfszzx/stable-diffusion-webui-images-browser#51

[kadattack]

Thanks for writing this report. Came here to report the same issue and it saved me time. I also noticed that with mask inpainting img2img, when you press generate it doesn't actually do anything. For some reason it generates a picture with mask brush strokes on it.

[pyr0ball]

@Nacurutu Tried out adding the changes from that PR you linked, but I'm still getting the same error:

Traceback (most recent call last):
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\routes.py", line 284, in run_predict
    output = await app.blocks.process_api(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 982, in process_api
    result = await self.call_function(fn_index, inputs, iterator)
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 824, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\pyr0ball\devl\Stable Diffusion\webui\modules\generation_parameters_copypaste.py", line 113, in send_image_and_dimensions
    img = image_from_url_text(x)
  File "C:\Users\pyr0ball\devl\Stable Diffusion\webui\modules\generation_parameters_copypaste.py", line 60, in image_from_url_text
    image = Image.open(io.BytesIO(filedata))
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\PIL\Image.py", line 3147, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x0000019A613C05E0>

[Gerschel]

@pyr0ball

@Nacurutu Tried out adding the changes from that PR you linked, but I'm still getting the same error:

Traceback (most recent call last):
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\routes.py", line 284, in run_predict
    output = await app.blocks.process_api(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 982, in process_api
    result = await self.call_function(fn_index, inputs, iterator)
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 824, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\pyr0ball\devl\Stable Diffusion\webui\modules\generation_parameters_copypaste.py", line 113, in send_image_and_dimensions
    img = image_from_url_text(x)
  File "C:\Users\pyr0ball\devl\Stable Diffusion\webui\modules\generation_parameters_copypaste.py", line 60, in image_from_url_text
    image = Image.open(io.BytesIO(filedata))
  File "C:\Users\pyr0ball\AppData\Local\Programs\Python\Python310\lib\site-packages\PIL\Image.py", line 3147, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x0000019A613C05E0>

I believe this error is unrelated.
The issue that this bug report suggests that the image goes through fine, but the rules for swapping out the layers (there are severeal image containers at that location, they just hide the ones not in use, and bring up the ones in use) are off.

The issue you have says that the file that was converted into an stream of bytes format (io.BytesIO) is not an image. So either the information wasn't properly converted, or it's sending the wrong file.

Without knowing how you got there, I'd say, with all the updates lately, you are probably using an extension in this situation, and it hasn't been updated.

Detail how you get to the point that causes the error, and make another bug report.

[anapnoe]

put these lines at the end of the style.css
and let me know if it works

/* fix issue with canvas */ #img2img_sketch, #img2img_sketch>.h-60, #img2img_sketch>.h-60>div, #img2img_sketch>.h-60>div>img, #img2maskimg, #inpaint_sketch { object-fit: scale-down !important; width:100% !important; height:auto !important; }

[Gerschel]

@anapnoe Please don't use "!important", just increase the specifity as needed. "!important" is the highest specificity, and the only way to overcome it is to keep using it, and eventually all rules will use it plus needing to be more specific. It is better to just be specific enough at the start, so when other rules come in to play, a user has an oppurtunity to overcome it, reasons can be simple styling or accessibility.

Nacurutu/Dark-Themes-SD-WebUI-Automatic1111#3 (comment)

Scroll down in that comment until you see some css rules. You'll see that the default uses "!important", this is why everyone else is having problems overcoming it. Those rules need to be fixed, then other rules can come in to play, without the need of "!important".

[anapnoe]

@anapnoe Please don't use "!important", just increase the specifity as needed. "!important" is the highest specificity, and the only way to overcome it is to keep using it, and eventually all rules will use it plus needing to be more specific. It is better to just be specific enough at the start, so when other rules come in to play, a user has an oppurtunity to overcome it, reasons can be simple styling or accessibility.

unfortunately when there is a bug and you want to be sure that the inline gradio attributes wont interfere with what you are trying to solve it is necessary (width and height are set inline from gradio app)

[Gerschel]

@anapnoe It isn't a gradio bug. Someone didn't know what they were doing, so they used "!important" in styles.css

[Gerschel]

@anapnoe If the main ui is getting overhauled, it is better to fix these issues, then to build upon the issues; which would make it harder to fix in the future. This is why the ui code is a mess; too many workarounds of other peoples coding styles or people that learned some coding during the lockdown, not knowing the implications and repercussions of what they were doing.

[anapnoe]

you are right i will do a cleanup usually i do it at he end from bottom to top to remove duplicates and fix these issues
it is a mess as you said but we will make it happen

[Gerschel]

@anapnoe It is a challenge. There's just so much. In my opinion, it'll be better to not be concerned about breaking others code, to go with a base standard, let them update to the standards.
When writing some tools, I've come across the issue that I need to accommodate a bunch of bad code, and there is always another curve ball once I've written what I thought was enough rules to cover all edge cases.
What we really need, is a more involved ruleset of standards.
Part of the problem, there is no clear documentation on how to code for a thing, no simple api for tooling. So people hack away, digging through others code to see how they can achieve some goal.
I'm hoping that we get some libraries with a common api. Last night I submitted a pull request for a small js component controller library, that I hope the api is kept simple. #7557
This was a library I used, along with another one I wrote for componentReferences, that allowed you to declare the components you wanted to interact with, in my componente refercences, it would get the type of component from gradio's configs, and build the interaction needed. This allowed just calling set and get, and linking components, that way I was pushing linking well beyond 20,000 components without having to code for them all, it just generated it.
But we need some simple api's, standards, and some rule about documenting code.

[Merkuriij]

I am also having problems with my images seemingly stacking on top of each-other on inpaint. After testing I can confirm this is ONLY happening with Horizontal images. It also doesn't happen with every horizontal image. Seems to be only images that very long. I tested with the same image it is bugging on but vertical, no problems.

[Nacurutu]

@Merkuriij could you please try this one?

[JonyTianming]

I also have the same problem as you. It seems to be a bug, which actually zooms in in the background until it finally splits, and at first it will report an error to me, indicating that the picture already has 280 million pixels, then split, and then I reinstall it again, the problem is the same, but no longer show me the error message, I do not know when this bug can be repaired (if there is a problem with the sketch function in img2img, this will also have a problem)