multi label graphCut

[Celli119]

I was thinking about starting developing a multi label graphCut (using gridCut library) feature soon. Problem is, I don't know how to manage the GUI part. Do you have any idea how could we manage a user-friendly selection of an arbitrary number of labels in MITK ?
Thanks.

[araex]

I'm not sure i understand what exactly you want to do. What is your starting point here? Do you have a multilabel image and you're looking for a GUI solution to feed it to the GraphCut plugin? In that case, couldn't you just remove the mask selection dropdowns and replace it with one for the multilabel segmentation?

Or do you want to do build something that allows the user to create a multilabel image that can then be passed to the graphCut? I think that could be accomplished with something like this:

• Overview of all labels and a button to add more labels
• A way to select one of them by clicking on it or maybe a key binding to the number next to it?
• Only 1 can be selected at any time
• The selected one is highlighted (green background in my image, might correspond to the visualized label color inside the 2D view).
• Everything you draw inside the multi pane window is assigned to the selected label
• If you want to allow the user to select the color of a label, you can add a color picker to each row

Let me know what you think.

[Celli119]

This will depend if we want to support or not macOS.

• Of course, if we use MITK 2016.11, then we'll use the multilabel image.
• If we want to support all MITK versions, I don't see any other choice but doing what you propose as a second choice.

As the first solution is an easy one, I can start with this one. Then, if you want to build something that allows the user to create a multilabel image, this could be great too for mac users. We could even imagine a conditional build depending on the MITK version.

[Celli119]

This is a preview of the result of the command line multi label gridCut segmentation (available on my forked repo).

The main issues are :

• I don't know how to set a directed graph and I'm not sure it's even possible (I've asked gridCut team today).
• We will have to adapt the mesher to generate a mesh from multilabel images
• Should we do an independant plugin? I guess this would depend if you're ready to build something that allows the user to create a multilabel image. In that case, we can imagine the binary graphCut would be called if there are only 2 labels, otherwise the multi-label plugin would be used.

[araex]

Awesome! How about this:

1. In the graph cut plugin, we add a radio button to the mask selection with 2 Options: Binary / Multilabel. Depending on which is selected, the drop downs for multilabel or binary mask selection will be shown. This way, we can integrate it even if we do not have the multilabel segmentation support (yet). I used a similar kind of radioboxes in ch.zhaw.volumemesh for the tetgen / CGAL mesher selection - the available options will be swapped out depending on the selected mesher.

2. We can start by adding multilabel image support to ch.zhaw.voxel2mesh. This way, the user can extract a mesh for each label manually and then feed it into the volume mesher seperately. Adding that shouldn't be too much trouble. The existing filter already uses a vtkImageThreshold internally (from 1 - 255) to find the segmentation. We can make that configurable. Given that, we apply the filter for each label inside the multilabel image.

[Celli119]

Sounds like good ideas, do you think you would have time to do it ?

[araex]

I'll see what i can do.

What intensity values are used in the resulting multilabel grid cut segmentation? Voxel value 0 for the first label, increment by 1 for each additional label? Highest intensity value in the segmentation would thus be number of labels - 1?

[Celli119]

No, the background (not labeled voxels) gets 0 value, then the 1st label is 1 and so on.
The number of values is the highest intensity value.

[Besler]

Great work @Celli119! I've been following along here because I'm interested in using the GraphCut method with a sheetness measure to annotate a large number of images.

I was able to compile your command line tools. I got similar results on my data to what you showed above. I have the same issue as in the pelvis above. I'm trying with some sheetness measures now.

What is the state of the project? Have most people dropped it for now? Do you think pursuing a graphical interface is worthwhile?

[Celli119]

Actually, I'm not comfortable with GUIs, I've tried to see how to write the ideas of @araex but I realized I would need a lot of time to do it. This is the reason why I've done a dirty GUI here which does the job. It requires a special version of MITK (as for the current MITK-GEM) you can find here.
First, build MITK, second when configuring MITK-GEM set superbuild at false and then fill the appropriate variable with the MITK build path.

[Besler]

Okay, that's great. Thanks @Celli119!

I was thinking of implementing GraphCut as it's own ITK module and then submoduling it with git. There are issues with the daviddoria method (when you try and use non-standard boundary or regional terms). You're probably pretty busy, so I do not want to bog you down. But, I do want to give you credit. If you would like, you could copy the GraphCut code to this repository with a commit. I would then make my modifications there. Or you could copy it to your own repository. I don't really care, I just don't want to start from scratch.

[ypauchard]

Could this be used to split the GraphCut3D folder of MITK-GEM into a new repository, keeping the history?

[Besler]

@ypauchard, you're brilliant! Okay, I'll give it a try sometime here.

[Besler]

Thanks for that @ypauchard. I think I have added it now... It is just in a separate repository. Sadly, I don't know if I'll ever be able to remove the stank of GPL.

[Celli119]

I've finally got some time to finish the multilabel/binary user interface. Of course this will only work with MITK-2016.11. Let me know about the way you want to merge this stuff.
This is what it looks like:

[Besler]

@Celli119 That is awesome, thank you for doing this! I will give it a try this weekend. It will help me out a lot.

[Besler]

Hey @Celli119! I just compiled branch uiInerface on both a Mac and CentOS machine. Two minor comments:

• In file CMakeLists.txt of directory Plugins/ch.zhaw.graphcut I believe you need to change MODULE_DEPENDS MitkQtWidgetsExt to MODULE_DEPENDS MitkQtWidgetsExt MitkMultilabel to mitk_create_plugin. Same with ch.zhaw.voxel2mesh.
• Your use of std::make_unique requires c++14 (or c++11 on some compilers). No other part of the superbuild requires c++14. I would use the "old" technique of new/delete. You use this in file Plugins/ch.zhaw.graphcut/src/internal/lib/GraphCut3D/ImageMultiLabelGridCutFilter.hxx. I know, it sucks, but compatibility is great. In fact, my linux box does not support make_unique in c++11! c++14 causes the build to fail all over.

If you have a better solution for both I would be happy to hear!

Finally, is it possible to have bi-directional multi-label graph cuts? I've only looked at the multi label gridcut API once, but I believe you create a table of neighbourhood values for each label. Intuitively, you could have 'n-directional cuts'. I guess this isn't a simple UI implementation though.

On Mac I compiled with CXX flags -Wno-deprecated-declarations -std=c++11. After modification to use new/delete, I do not think I needed c++11. I just don't want to recompile the whole bloody thing :)

[Besler]

Sadly, I cannot select Multilabel segmentation for the multilabel graphcut or a foreground or background segmentation for the binary graphcut segmentation. I will test on a linux box (working remotely) tomorrow.

Maybe I built something wrong and this is a common issue?

Edit:
Nope, this was a 'me' thing. MITK-GEM only recognizes certain types of image data (unknown which type) as possible segmentations.

[Celli119]

Hello @Besler ,
Thanks for your comments:

• You're right, I didn't think about module dependencies since I've always been compiling with MITK workbench 2011, I'll change that.
• I encountered the same issue as you did for the c++14 feature but only on one of my linux machine. To fix it, I've just added a set (CMAKE_CXX_STANDARD 14) in the CmakeLists on in the SuperBuild, I don't remember. The std::make_unique was something @araex advised me, then I'd like to keep it that way.
• I've been looking for a while to set up an oriented multilabel graph but it does not seem possible. I've even asked gridCut team about it but I didn't have any answer.
• I explicitely wanted the user can only select valid types of image data, this prevents from wrong uses.

[Besler]

Sorry, I realize I interjected myself a little strong there.

That all makes sense then. The GridCut team doesn't respond much it seems... I hear they're considering making GridCut a module on GitHub so you can submodule it in Git.

Anyways, I will try set (CMAKE_CXX_STANDARD 14) and see how things work out.

[Besler]

How does MITK-GEM establish that an image can be used for multilabel graph cut? I have image data that is of type uint8. I have 5 labels. Label 0 is 'unlabelled' data. Label 1 is 'background'. the remaining labels are {pelvis, femur} x {left, right}.

In your implementation @Celli119, is there anyway to load such an image data into MITK-GEM and apply multilabel graph cut?

[Celli119]

Did you convert your image into segmentation ? (right click on it in the data manager -> convert to segmentation) 2017-09-18 23:47 GMT+02:00 Bryce Besler <[email protected]>:

…

How does MITK-GEM establish that an image can be used for multilabel graph cut? I have image data that is of type uint8. I have 5 labels. Label 0 is 'unlabelled' data. Label 1 is 'background'. the remaining labels are {pelvis, femur} x {left, right}. In your implementation @Celli119 <https://github.com/celli119>, is there anyway to load such an image data into MITK-GEM and apply multilabel graph cut? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#37 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJnA8l4tizL11WstUEmrLRpYKibzGWOFks5sjuT8gaJpZM4M1gED> .

[Besler]

Okay, now I see. I don't have that option on right click. I'm using MITK-GEM 2017.5.0 after the superbuild. I am going to guess the difference between how you and I compiled (SuperBuild vs your linking/building separately) is making a difference in the MITK version. I'm probably wrong on that front though.