Update nodes.py so VAEDecode falls back to a tiled method if GPU runs out of memory #5427
+1
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
patched VAEDecode if fails due to lack of VRAM fallback to tiled decode method
|
There is already logic to handle this here: https://github.com/comfyanonymous/ComfyUI/blob/master/comfy/sd.py#L344 Which OS and which pytorch version are you using? |
OS: Windows 10 Pro I have a similar logic as what you linked, but it never works, at least not on my device. |
|
In your patch, can you show what kind of exception occurs when there is no exception handler? |
Here is the full error report when I remove my patch from the latest available version of ComfyUI. ComfyUI Error ReportError Details
Stack TraceSystem Information
Devices(PowerColor 6600xt 8gb will be reported incorrectly with directml. A1111 has this same issue.)
LogsAttached WorkflowPlease make sure that workflow does not contain any sensitive information such as API keys or passwords. Additional Contextnone As you can see, it's a pretty significant error when the AMD GPUs run out of memory. This error is specific to AMD GPUs that do not support ROCm. I have no way of testing if ROCm enabled devices handle or report out-of-memory errors differently. I hope this is enough information to support the need for this change. It's a see here for more information on the OOMException from the modelmanagement.py file ComfyUI/comfy/model_management.py Line 153 in cc9cf6d |
The original code only catches |
revert change to VAEDecode node.
Move catching of RuntimeError and MemoryError to sd.py
Remove unnecessary parameter to decode method
That is what my PR does, catching the RuntimeError as well as any other MemoryError in the decoding process itself. I have modified the code to reflect this, moved it out of the node and into the See screenshots:
|
|
I believe, at this point, my list of potential side-effects or concerns has been reduced to none. |
|
Just checking in to see why this hasn't been merged yet. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
As the title suggests, if the GPU runs out of VRAM during a VAEDecode event, then the entire queued prompt fails. With as many enhancements as ComfyUI has received over the last months, it is all but certain that this failure should no longer exist.
Solution
I have introduced a small change to the VAEDecode node so that it will fallback to a tiled decode process should the GPU run out of memory.
How to test
Run a sufficiently large image generation on a GPU with 8GB of VRAM or less. VAEDecode node will fail due to lack of GPU memory, especially on an AMD GPU that does not support rocm.
Potential side effects or concerns
The VAEDecode node has changed significantly to include batches, which is fine, however the tiled decoding method does not include such support. There is a possibility that the tiled method may fail here. I haven't been able to find any instances where it would fail with simple image generations. In these cases, it may be that the user is simply trying to do too much and should instead switch to a device that has a GPU with more VRAM.