Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add widget: remove objects using a probability map #202

Merged
merged 7 commits into from
Mar 4, 2024

Conversation

qin-yu
Copy link
Collaborator

@qin-yu qin-yu commented Feb 20, 2024

A generic pmap-guided object removal widget

Instead of adding an option to each "boundary -> segmentation" algorithm as discussed in #174, I made a separate widget for Napari GUI pipeline for generic usage. This independent design avoids bundled computation within other functions such as GASP, allows easy trial-and-error strategy, accepts flexible input pmaps/images and threshold, and is useful in unexpected use cases.

I came up with three designs and decided to implement only one. I now add this to Napari, and will think about the best way to do it in the other UI.

  1. Method 1: Threshold foreground to a binary mask and multiply with the segmentation pixel-wise ❌
    1. Pros: very fast; false positive parts will be deleted within objects if foreground quality is high
    2. Cons: depending on the quality of foreground segmentation, details may be lost
  2. Method 2: Delete instances based on the centroids' value in foreground ❌
    1. Pros: more educated guesses than 1, preserves details
    2. Cons: merged instances or merged false-and-true instances will be deleted together; not helpful if instances are not star-convex, e.g. C shaped false positives around objects will persist
  3. Method 3: Delete instances that has, say, 90% overlap with the background (I believe it's the best way ✔)
    1. Pros: considering the foreground probability of each pixel in an instance, the algorithm is well-informed
    2. Cons: slightly slower than 1 & 2 but after careful design and simple tests I believe it handles segmentation/superpixels at a scale of 100k instance in minutes

Example Usage

As a standalone widget it not only helps reduce unnecessary computation but also can be flexibly used between different steps. For example, the following image shows how an instance connected to a false positive would be deleted if we simple use GASP followed by method 3:

image

To have a perfect segmentation, users can apply a method 3 foreground filter before and after GASP like this:

Starting the a watershed output:

image

Apply the new filter:

image

Run GASP:

image

Apply my new filter again:

image

@qin-yu
Copy link
Collaborator Author

qin-yu commented Feb 21, 2024

It does work

image

@qin-yu
Copy link
Collaborator Author

qin-yu commented Feb 21, 2024

I think I'm done with the addition of widget, works well:

image

@qin-yu
Copy link
Collaborator Author

qin-yu commented Feb 21, 2024

A problem of only have foreground filter as a post processing:

image

@qin-yu qin-yu added the enhancement New feature or request label Feb 21, 2024
@qin-yu
Copy link
Collaborator Author

qin-yu commented Feb 21, 2024

Screenshots

Raw:

image

Foreground prediction:

image

Boundary P\prediction:

image

Watershed superpixels shown on boundary:

image

Watershed superpixels shown on raw:

image

Filtered superpixels on foreground:

image

Filtered superpixels on raw:

image

GASP ran on filtered superpixels:

image

Filtered GASP:

image

To enable the use of my function on label with more instances (30k in my case), it has to be speed up.
@qin-yu qin-yu marked this pull request as ready for review February 21, 2024 16:46
@qin-yu qin-yu self-assigned this Feb 21, 2024
@qin-yu
Copy link
Collaborator Author

qin-yu commented Feb 21, 2024

Hey @lorenzocerrone and @wolny

I added a widget that uses probability maps to filter objects in another image. It is very flexible but only available in Napari now. I believe the other less interactive interfaces don't need this level of flexibility, and I'll think about how to design the filter into the them later. Please have a look at this PR.

@qin-yu qin-yu changed the title Add foreground/background postprocessing to remove fasle positives Add widget: remove objects using a probability map Feb 21, 2024
@qin-yu qin-yu merged commit 5d2a22c into master Mar 4, 2024
1 check passed
@qin-yu qin-yu deleted the ForegroundRemoveFalsePositive branch March 4, 2024 15:23
@wolny
Copy link
Collaborator

wolny commented Mar 19, 2024

this looks great, nice job @qin-yu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants