Bug at generate marks

[jyC23333]

Search before asking

• I have searched the Multimodal Maestro issues and found no similar bug report.

Bug

Traceback (most recent call last):
File "/data/megvii/projects/Qwen-VL/scripts/test_maestro.py", line 7, in
marks = generator.generate(image=image)
File "/data/Anaconda/anaconda3/envs/autogpt/lib/python3.10/site-packages/maestro/markers/sam.py", line 44, in generate
return masks_to_marks(masks=masks)
File "/data/Anaconda/anaconda3/envs/autogpt/lib/python3.10/site-packages/maestro/postprocessing/mask.py", line 187, in masks_to_marks
return sv.Detections(
File "", line 8, in init
File "/data/Anaconda/anaconda3/envs/autogpt/lib/python3.10/site-packages/supervision/detection/core.py", line 89, in post_init
_validate_mask(mask=self.mask, n=n)
File "/data/Anaconda/anaconda3/envs/autogpt/lib/python3.10/site-packages/supervision/detection/core.py", line 29, in _validate_mask
raise ValueError("mask must be 3d np.ndarray with (n, H, W) shape")
ValueError: mask must be 3d np.ndarray with (n, H, W) shape

Environment

Ubuntu 20.04
python=3.10.10

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[SkalskiP]

Hi @jyC23333! 👋🏻 Could you share the code you run and image you used? Basically something more so I can understand better what happens?

[jyC23333]

Hi，@SkalskiP ，my code shows below:

import cv2
import maestro

image = cv2.imread('./鲫鱼.png')

generator = maestro.SegmentAnythingMarkGenerator(device='cuda')
marks = generator.generate(image=image)
marks = maestro.refine_marks(marks=marks)

mark_visualizer = maestro.MarkVisualizer()
marked_image = mark_visualizer.visualize(image=image, marks=marks)

something wrong at this line:

marks = generator.generate(image=image)

original image:

[SkalskiP]

Thanks @jyC23333. I'll get back to you!

[SkalskiP]

Hi @jyC23333 👋🏻 I just tested using the latest version of maestro, and everything works.

Take a look at this Colab.

Make sure to use the 0.1.1rc1 version. Please let me know if that helped.

[jyC23333]

@SkalskiP Hi，the bug still exists with the latest version.

This is my cuda info:

And I'm using torch 2.1.0:

[SkalskiP]

Hi @jyC23333, the bug doesn't exist in the Colab I sent you, so it must be environment-dependent. What version of transformers and supervision you have?

[jyC23333]

@SkalskiP Hi，the dependent info is:

[hvaria]

To address the bug, add debug logs around self.pipeline in generate to check data shapes and integrity. Monitor GPU usage to ensure no memory overflow—silent issues may not trigger clear error messages

[SkalskiP]

I'm very sorry, but I'm extremely busy. I'll try to look at it as soon as possible.

[SkalskiP]

The maestro project has pivoted in the direction of a VLM fine-tuning toolkit. As a result, I am closing legacy issues.