Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

for your consideration: support other formats #6

Open
fedorov opened this issue Sep 2, 2016 · 33 comments
Open

for your consideration: support other formats #6

fedorov opened this issue Sep 2, 2016 · 33 comments

Comments

@fedorov
Copy link

fedorov commented Sep 2, 2016

It is not very trivial to convert .mnc to other formats. There are many questions about going from mnc to other formats if one googles (and I did), but there are no binary tools for conversion. Perhaps the reason is that MINC software has LGPL license Anyway, I've just been through the exercise to help a colleague of mine with this task, and it took much longer than I thought, so I decided to spend this extra bit of time to document and report to you.

I think your atlas might be more useable to the community if you provided labels in formats other than mnc. I attach the file I converted for my colleague so you can use it for your purposes.

The recipe I used to convert:

  1. have a build of 3D Slicer handy
  2. In the ITKv4-build directory in Slicer super-build change the variable Module_ITKIOMINC to on
  3. Run make from the Slicer superbuild directory - this will rebuild ITK and SimpleITK
  4. The MINC reader will still not be available in the Slicer "Add data" dialog to load MINC files, but you can do it via the python console: Ctrl+3 will show the python console, then you can do the following
>>> import SimpleITK as sitk
>>> i=sitk.ReadImage("your_file.mnc")
>>> sitk.WriteImage(i,"your_compressed_file.nrrd",True)

Resulting file: thalamus_subdivisions.nrrd.zip

Hopefully this is helpful to you, or other people who would have the same issue.

@fedorov
Copy link
Author

fedorov commented Sep 2, 2016

followup for completeness:

I opened the data into my scene, but it doesn’t seem to line up with the colin27 images or the surface rendering of the thalamus.

Looks like perhaps the A/P directions are flipped?

and a bit later

It lines up after reflection in y

ps I found this opening about MINC rather ironic: "The MINC file format and toolbox was originally conceived, written and released by Peter Neelin in 1992 due to the frustrations of dealing with multiple file formats from varying scanners and research groups." The result feels like exactly the opposite of the original intent ...

@dzenanz
Copy link

dzenanz commented Sep 2, 2016

Neither first nor last: https://xkcd.com/927/

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

The core of the minc-tools contains the program mnc2nii which produces NIFTI files. It is available for
Ubuntu: http://packages.ubuntu.com/search?keywords=minc-tools&searchon=names&suite=all&section=all
Debian: https://packages.debian.org/search?keywords=minc-tools&searchon=names&exact=1&suite=all&section=all

The full superset of minc-toolkit is available at https://bic-mni.github.io/ for a variety of platforms, as both source and binary.

Furthermore, MINC is also supported by ITK and ANTs via the superbuild (ANTsX/ANTs#198), so the ConvertImage tool can convert to/from NIFTI/MINC.

The MINC format has a documented standard https://en.wikibooks.org/wiki/MINC/SoftwareDevelopment/MINC2.0_File_Format_Reference
and a reference open-source reader for that standard:
https://github.com/BIC-MNI/libminc

Unfortunately, I cannot say the same for other tools. For example, which NIFTI file do I support? SPM2? SPM5? SPM12? FSL? Freesurfer? Which do I find the standards documents?

These files were created and are used with the official BIC release of colin27 available here http://www.bic.mni.mcgill.ca/ServicesAtlases/Colin27.

@fedorov
Copy link
Author

fedorov commented Sep 5, 2016

Neither I nor my collaborator had an Ubuntu or debian Linux to try the packages you suggested.

FYI your windows or Mac packages do not include the tool you mentioned.

NIFTI and NRRD work out of the box with the binary of Slicer I can download for any of the three platforms.

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

Since libminc is integrated into ITK, any ITK tool (such as Slicer or ITKSnap) can enable MINC support through an adjustment of their build system and allowance of MINC files in their UI, perhaps asking for that there would be a good place to look

https://github.com/Slicer

https://groups.google.com/forum/#!forum/itksnap-dev

@fedorov
Copy link
Author

fedorov commented Sep 5, 2016

Thanks for pointing to the Mac binaries. Hope this will help someone who runs into this issue next time. I don't have a current need for this atlas or MINC. I was just helping a colleague.

About the support of this format, as I mentioned, MINC libraries are covered by LGPL license (according to Bill Lorensen), incompatible with 3D Slicer. It cannot be included.

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

The main difference between the GPL and the LGPL is that the latter allows the work to be linked with (in the case of a library, 'used by') a non-(L)GPLed program, regardless of whether it is free software or proprietary software.[1]

https://en.wikipedia.org/wiki/GNU_Lesser_General_Public_License#Differences_from_the_GPL

Looks A-OK to me. It will just be linked against.

@dzenanz
Copy link

dzenanz commented Sep 5, 2016

LGPL is problematic for static library route.

Many people (myself included) prefer to compile libraries as static to avoid the problem of DLL hell during software development. Especially important if you have a dozen different versions of the same library (as I have of ITK and VTK).

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

On a side note, should you be attempting to use 3DSlicer and these atlases to segment other files, we provide a pipeline that supports NIFTI as an input, to do automatic segmentation of scans
https://github.com/CobraLab/antsRegistration-MAGeT

Based on ANTs tools + the MAGeT algorithm.

@gdevenyi
Copy link
Member

gdevenyi commented Sep 5, 2016

One more way to get all the MINC tools in one place I forgot to mention:

I build an automated VM based on ubuntu 16.04 available here https://github.com/cobralab/MINC-VM

@gdevenyi
Copy link
Member

gdevenyi commented Sep 6, 2016

After a followup with the libminc developers, turns out most of the MINC tools are released under a permissive MIT-like licence:
https://github.com/BIC-MNI/libminc/blob/master/COPYING

@fedorov
Copy link
Author

fedorov commented Sep 6, 2016

@gdevenyi, again, to cite the thread I mentioned earlier, according to Bill Lorensen @lorensen, who has a lot of expertise in this area:

MINC has an LGPL license which is not compatible with the ITK apache license.
This is why we make it an optional build.

So there is a confusion on either MINC or ITK side.

If you want your format have better support in tools like ITK and 3D Slicer, you might want to follow up with the respective groups and invest effort into improving that support. I have neither resources nor need to work on this.

The purpose of submitting this issue was to help other users of your atlas, so that my time spent investigating the conversion issues can benefit someone else, and to raise your awareness about the limitations of this format support. You take it from there as you see appropriate.

HTH

@lassoan
Copy link

lassoan commented Sep 12, 2016

Great discussion! We've been planning enabling MINC IO support in Slicer build (http://www.na-mic.org/Bug/view.php?id=4085) but we constantly run into build problems on Windows. If you can help in fixing those then I'd be happy to assist with getting it into Slicer.

@lassoan
Copy link

lassoan commented Sep 12, 2016

For example, if I enable Module_ITKIOMINC CMake option in Slicer's ITK build then I get this error (Windows, VS2013):

1>------ Build started: Project: ZERO_CHECK, Configuration: Debug x64 ------
...
2>------ Build started: Project: itkvcl, Configuration: Debug x64 ------
3>------ Build started: Project: itkv3p_netlib, Configuration: Debug x64 ------
4>------ Build started: Project: itknetlib, Configuration: Debug x64 ------
5>------ Build started: Project: itksys, Configuration: Debug x64 ------
6>------ Build started: Project: itkdouble-conversion, Configuration: Debug x64 ------
7>------ Build started: Project: gdcmCommon, Configuration: Debug x64 ------
8>------ Build started: Project: ITKEXPAT, Configuration: Debug x64 ------
9>------ Build started: Project: ITK_bld_internal_H5make_libsettings, Configuration: Debug x64 ------
...
49>------ Build started: Project: itkminc2, Configuration: Debug x64 ------
49>  Building Custom Rule C:/D/S4D/ITKv4/Modules/ThirdParty/MINC/src/libminc/CMakeLists.txt
49>  CMake does not need to re-run because C:\D\S4D\ITKv4-build\Modules\ThirdParty\MINC\src\libminc\CMakeFiles\generate.stamp is up-to-date.
49>  convert.c
49>  datatype.c
49>  dimension.c
49>C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(515): error C2375: 'rint' : redefinition; different linkage
49>          c:\d\s4d\itkv4\modules\thirdparty\minc\src\libminc\libsrc2\minc2_private.h(209) : see declaration of 'rint'
49>  free.c
...

@gdevenyi
Copy link
Member

Thanks @lassoan,

I'll notify the MINC maintainers of the attempts and problems you're running into.

@gdevenyi
Copy link
Member

@lassoan For your bug, doing a quick dig through the MINC sources, rint used to be not supplied so it had to be supplied.

You can attempt to copy out the relevant code to see if the compile continues.

https://github.com/BIC-MNI/libminc/blob/develop/libsrc2/minc2_private.h#L209-L215

@lassoan
Copy link

lassoan commented Sep 13, 2016

After commenting out rint in header and c files, to compilation continues and I get other errors.

@lassoan
Copy link

lassoan commented Sep 13, 2016

2>C:\D\S4D\ITKv4\Modules\ThirdParty\MINC\src\libminc\libsrc2\hyper.c(52): error C2084: function 'double copysign(double,double)' already has a body
2> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(483) : see previous definition of 'copysign'
2>C:\D\S4D\ITKv4\Modules\ThirdParty\MINC\src\libminc\libsrc2\hyper.c(65): error C2084: function 'double round(double)' already has a body
2> C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\math.h(516) : see previous definition of 'round'

commenting out copysign and round implementation makes the build continue...

@lassoan
Copy link

lassoan commented Sep 13, 2016

I get this error then:

1>------ Build started: Project: itkhdf5, Configuration: Debug x64 ------
1> itkhdf5.vcxproj -> C:\D\S4D\ITKv4-build\lib\Debug\itkhdf5-4.10.lib
2>------ Build started: Project: itkminc2, Configuration: Debug x64 ------
3>------ Skipped Build: Project: ITKHeaderTests, Configuration: Debug x64 ------
3>Project not selected to build for this solution configuration
2> minc2_error.c
2> minc_config.c
2> Generating Code...
2> itkminc2.vcxproj -> C:\D\S4D\ITKv4-build\lib\Debug\itkminc2-4.10.lib
4>------ Build started: Project: ITKIOMINC, Configuration: Debug x64 ------
4> Creating library C:/D/S4D/ITKv4-build/lib/Debug/ITKIOMINC-4.10.lib and object C:/D/S4D/ITKv4-build/lib/Debug/ITKIOMINC-4.10.exp
4>itkminc2-4.10.lib(minc_config.obj) : error LNK2019: unresolved external symbol strncasecmp referenced in function miread_cfg
4>C:\D\S4D\ITKv4-build\bin\Debug\ITKIOMINC-4.10.dll : fatal error LNK1120: 1 unresolved externals

Replacing strncasecmp by _strnicmp in minc_config.c makes the build complete!

@lassoan
Copy link

lassoan commented Sep 13, 2016

If you have a patch then let me know and I can test it.

You would need to set up a nightly Windows build or convince some ITK build machine developers to enable MINC so that you can see when something breaks.

@lassoan
Copy link

lassoan commented Sep 13, 2016

I've just noticed the link to BIC-MNI/libminc#75. If you can get MINC enabled on ITK by default that will make MINC support widely available. Many ITK-based applications will just start to recognize MINC file formats without requiring any change from app developers (in Slicer it requires a few minor changes that I'll be happy to make). It'll also take care of quality control, as errors will show up on standard ITK dashboards.

@fedorov
Copy link
Author

fedorov commented Sep 19, 2016

A request from my collaborator, 2 weeks after I helped him with some of the atlases ... just to remind you - this is real!

I am really sorry to ask this of you, but would you be able to save [the attached .mnc files] to .nrrd also?

@vfonov
Copy link

vfonov commented Oct 27, 2016

there is itk_convert binary include inside minc-toolkit_v2 - you can run that to convert minc to nrrd.

@gdevenyi
Copy link
Member

gdevenyi commented Mar 3, 2017

Hi,

There is now a lightly quality-controlled (aka it looks okay to me) NIFTI version of files available for download at:
http://cobralab.net/files/atlases-nifti.zip
http://cobralab.net/files/brains_t1_nifti.tar.bz2

@vfonov
Copy link

vfonov commented Mar 3, 2017

so, what is it for?

@gdevenyi
Copy link
Member

gdevenyi commented Mar 3, 2017

@vfonov discussion above requesting NIFTI versions of the CoBrALab atlases

@fedorov
Copy link
Author

fedorov commented Mar 3, 2017

There is now a lightly quality-controlled (aka it looks okay to me) NIFTI version of files available for download at

It would make sense to me to note this fact in the README

@gdevenyi
Copy link
Member

gdevenyi commented Mar 3, 2017

@fedorov for now, these are unofficial so they're not going to go there yet.

@fedorov
Copy link
Author

fedorov commented Mar 3, 2017

This means that chances of them being found are approaching nil. Either way is fine with me.

Have you ever tried to put yourself into the shoes of a researcher who has no clue and no interest to learn about the peculiarities of imaging formats, has very little time, has no patience, and just wants to use your atlas. Let's say, they just want to see it overlayed on the structural image. They come across your repo here: https://github.com/CobraLab/atlases. Did you ever consider they might be confused how to do that?

@lassoan
Copy link

lassoan commented Mar 5, 2017

I've downloaded the files and they all load correctly into 3D Slicer, so it's already a great step towards better compatibility. I'm not an expert in neuroimaging, but the segmentations look beautiful.

On Windows, extracting .tar.bz2 archives requires installation of an application. If you choose to make the images available in .zip format as well then that could be browsed/extracted by the built-in file manager without installing any additional software.

I'm not sure if atlases-nifti.zip is complete, as it only contains 6 subdirectories and in colin27-subcortical there are no labels. There is also a "bin" directory that might be left there accidentally, containing some temporary file and python scripts.

It might be useful to save the atlas in format that is optimized for use in specific applications (3D Slicer, ITK-Snap, MITK, ...). If you were interested, I can help in creating an atlas that can be easily loaded and reviewed in 3D Slicer, even upload it into the 3D Slicer Data Store so that users can download it with a single click (similarly to the SPL brain atlas - http://www.spl.harvard.edu/publications/item/view/2037).

Another thing that you might be interested in is joint smoothing of the segmentations. In 3D Slicer we have a smoothing method that is designed to smooth segmentations, smoothing all labels at once, smoothing boundaries while respecting interfaces (mesh remains watertight).

Segmentation of brain2 cerebellum loaded into 3D Slicer:
2017-03-05-original

![Result after joint smoothing with a factor of 0.3:
2017-03-05-smoothed-0 3

@gdevenyi
Copy link
Member

gdevenyi commented Mar 6, 2017

@lassoan Thank you for the constructive feedback.

There indeed are missing files, which I will get to fixing up and uploading. Thank you also for the note re: .tar.bz2, as a linux-only shop I usually consider .zip to be harder to handle than other formats. I have now uploaded a zipped version (http://cobralab.net/files/brains_t1_nifti.zip) and I have updated for the missing files (http://cobralab.net/files/atlases-nifti.zip). The extra directories are there because this zip file was (supposed) to just be a copy of this repository with nifti instead of mnc, so I just kept everything else.

As for the slicer files, we do not currently use slicer nor does anyone I know, so I really have no clue how to produce such files. I would be interested in some help to produce a slicer version.

As for smoothing, this operation is not in line with our intent for these atlases, which are all intended as a anatomically-informed manually segmented ground-truth and are a result of the exact segmentation protocol. Users are of course welcome to do with them as they wish, but we will keep our versions as produced by our anatomists.

thewtex pushed a commit to thewtex/ITK that referenced this issue Apr 29, 2017
Run the UpdateFromUpstream.sh script to extract upstream MINC
using the following shell commands.

$ git archive --prefix=upstream-minc/ 3d79acf3 -- 
./ChangeLog
./volume_io
./libsrc
./libsrc/minc_compat.h
./libcommon/minc_config.h
./libcommon/minc_config.c
./libsrc/minc_format_convert.c
./libsrc/value_conversion.c
./libsrc/hdf_convenience.c
./libcommon/read_file_names.c
./libsrc/minc_simple.c
./libcommon/time_stamp.h
./libsrc/hdf_convenience.h
./libsrc/type_limits.h
./libcommon/read_file_names.h
./libsrc/minc_varlists.h
./libcommon/restructure.h
./libsrc/nd_loop.h
./libcommon/restructure.c
./libsrc/minc_simple.h
./libsrc/strdup.c
./libsrc/minc_useful.h
./libsrc/minc_convenience.c
./libsrc/minc_basic.h
./libsrc/minc_compat.c
./libcommon/time_stamp.c
./libsrc/voxel_loop.h
./libsrc/minc_routines.h
./libsrc/minc_private.h
./libsrc/netcdf_convenience.c
./libsrc/voxel_loop.c
./libsrc/dim_conversion.c
./libsrc/minc_format_convert.h
./libsrc/nd_loop.c
./libcommon/ParseArgv.h
./libcommon/minc_error.c
./libcommon/minc_error.h
./libsrc/minc_structures.h
./libsrc/minc.h
./libsrc/image_conversion.c
./libcommon/ParseArgv.c
./libcommon/minc_common_defs.h
./COPYING
./UseLIBMINC.cmake.in
./NEWS
./AUTHORS
./libsrc2/m2util.c
./libsrc2/minc2_defs.h
./libsrc2/minc2_api.h
./libcommon/minc2_error.h
./libsrc2/grpattr.c
./libsrc2/hyper.c
./libsrc2/minc_compat2.h
./libcommon/minc2_error.c
./libsrc2/record.c
./libsrc2/volume.c
./libsrc2/datatype.c
./libsrc2/volprops.c
./libsrc2/valid.c
./libsrc2/convert.c
./libsrc2/free.c
./libsrc2/dimension.c
./libsrc2/label.c
./libsrc2/minc2.h
./libsrc2/minc2_private.h
./libsrc2/slice.c
./libsrc2/minc2_structs.h
./CMakeLists.txt
./nifti
./LIBMINCConfig.cmake.in
./README.release
./INSTALL
./config.h.cmake
./README
 | tar x
$ git shortlog --perl-regexp --author='^((?!Kitware Robot).*)$' --no-merges --abbrev=8 --format='%h %s' 8632513e..3d79acf3

Chris Hammill (1):
      06f3e87b Unsatisfying solution to issue InsightSoftwareConsortium#71

Robert D. Vincent (34):
      5b483ac0 Fix problem discovered by Jerome Redoute - voxel_loop programs do not properly remove the loop dimension if it contains a 'xxxx-width' variable.
      59678daa Minor fixes for MINC2.0 API. Implement alignment, properly record dimorder for irregular dimension variables, quiet some errors.
      f0126d35 Hopefully useful version of minc_get_world_transform() and new function minc_transform_to_world().
      3775a0a9 Propagate read errors such that voxel_loop() now returns an integer exit code.
      2e33bca0 Propagate read errors through volume_io as well.
      5ac1aca8 Fix warning.
      3ca34259 Correctly place attributes intended for the image variable (e.g. 'complete').
      25792953 Avoid repetitive code while correcting placement of attributes within the HDF5 hierarchy.
      17aa090d Fix BIC-MNI/minc-tools#45 - add quotes around argv items that contain spaces.
      e7101d7a Make sure variable is initialized.
      d99dd016 Fix InsightSoftwareConsortium#74, don't assume hid_t === int since this is no longer true in HDF5 1.10.
      e7df7ce1 Minor fixes so that miclose() return value is correct.
      2f6b922f Fix issues with proper ranging of data in NIfTI files.
      6fbea42f Fix memory leaks to keep LeakSanitizer happy.
      eb7cf239 Add multidim array tests.
      1f4a4c5a Fix warnings.
      3f0c0389 Fix BIC-MNI/Display#63, Ignore MGZ/freesurfer offsets so that volumes align with their surfaces.
      00ac284f Fix warnings for unused variables (in an unused test).
      3b2df807 Major fix to NIfTI reading for images with non-standard transforms.
      f01e9be2 Fix problem if scl_slope is zero.
      063fce99 Correctly read NIfTI files with a fifth dimension.
      1e1b300d Attempt to improve speed of NIfTI loader.
      f3f881fe Fix bug in nifti reader.
      433acc15 Implement some additional special cases for volume interpolation.
      85733672 Basic reader for NRRD format.
      14cee37d Fix minor memory leak.
      baced81b Add libcommon to LIBMINC_INCLUDE_DIRS_CONFIG so that minctools will build.
      931df645 Fix InsightSoftwareConsortium#78, correct documentation for miget_dimension_sampling_flag().
      c3841475 Fix use-after-free error.
      3d5a2277 Fix range calculation for small NIfTI files.
      62a6a849 Fix InsightSoftwareConsortium#80, memory leaks in hyperslab reading.
      b2fdfff7 testing for leaks.
      4d68e862 Free memory at end of test.
      3daa90b4 Fix issue when copying a non-allocated volume.

Vladimir S. FONOV (38):
      2c9fe645 Removing allocation  debug code, it is superseded by address sanitizer or valgrind
      d3ab23b5 Added proper flag to MIcomplete when minc file is successfully closed
      39d86771 Replaced C++ comment to C comment
      6ef58fe9 Added option to work with integer labels (avoid inter-slice scaling)
      9c27bde6 Disabled writing of complete flag temporarely
      5df49651 WIP: adding labels to volumeio
      89a82cf9 WIP: modifying volume_io to work properly with labels
      466cdede WIP: adding possibility to debug minc2 volume io interface
      bbf80974 Added labels support in volume_io and a new environment configuration variable MINC_PREFER_V2_API to force volume_io to use MINC2 API directly, mostly for debugging purposes
      3764357a Finalizing MINC2 api choise support in volume_io
      0bd33a2e Fixed segfault that creeped in after the previous commit
      46261240 Tweaking voxel_loop
      745d75a9 Removed FindHDF5 and bumped cmake version requirements, addresses InsightSoftwareConsortium#71
      d641c4fd Updating HDF5 variables, fixing libnifti internal build dependencies
      b45312d9 Disabled allocation debugging in volume_io - superceded by valgrind and asan
      21125c4b Downgraded cmake requirements to satisfy travis-ci
      44b8d4b3 Fixed memory leak in minc_long_attr tests
      92ba4fa2 Fixing function forgotten in d99dd01632dbcb62870870557b333142b9117e9a , related to InsightSoftwareConsortium#74
      eb7eb487 Fixing memory leak, addresses InsightSoftwareConsortium#75
      70a27484 Added description of environment variables
      96793536 Added explicit test for rint, addresses InsightSoftwareConsortium#77
      f134ca9c Updated brew control sum for HDF5
      eecefd5d Refactoring minc2 internals
      34b08c7e Trying to make travis-build happy
      d9d98771 Fixed dimorder variable contents for image-min and image-max
      54169226 Made some tests actually perform something meaningless and report errors if things don't work
      dbf3e037 Minor change: fixed comments
      dd93215a Refactring error handling in volume_io
      f864dd22 Fixing failed build when MINC1 support is disabled
      bb7ce2db Refactoring common minc1&minc2 code
      a74eef51 Refactoring minc error reporting
      cba6f09b Bumped package patch version
      b1cd7f50 Updated comment for restructure
      d8cdab34 Hopefully fixes compilation issue on Windows, addresses CoBrALab/atlases#6
      bd17b1f2 Replaced c++ style comment with c
      a44cf8de Added a hack to address BIC-MNI/minc-tools#59
      f8f16142 Fixed volume_io error reporting routine, adresses BIC-MNI/Register#16
      3d79acf3 Fixing issue from Slicer/Slicer#716: moved code from libsrc into libcommon, removed unused varable

Change-Id: I8d3238e3bf830b6c4df9a64df35567e01268a030
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants