support for ubuntu 16 on OH master

[mauricefallon]

support for the OpenHumanoids/DRC build of Director in ubuntu 16.06

• these changes are against the RL/director master (as of yesterday)

• core modifications: use the same versions of qt (4), vtk-pre-compiled (7.1) and pythonqt, PCL (1.8) and PointCloudLibraryPlugin using in the superbuild.

• uses the 'oh fork' of drake and ddDrakeModelOH.cpp - which is 1 year old.

• Key changes are in: src/vtk/DRCFilters/CMakeLists.txt

• minor API changes in C++ code.

Remaining issues are that
(1) map server has to be disabled because it fails to compile against into a Python library
(2) octomap has a related issue
(3) various calls to setInput() still need to change to setInputData() in python segmentation code.
(4) the 'Task Panel' pop up doesnt work. (Pose utility does work)

Proof:
https://www.dropbox.com/s/pjoje19c2a63uys/2017-08-director-16-04.mp4?dl=0

[mauricefallon]

@patmarion I have been mostly successful in upgrading OH's fork of director to vtk-7.1 as mentioned here: issue on RL/director repo

I used RL/director's master and made the minimal modifications. Could you help me with a few of the remaining changes?

(Note: when I forked director to RobotPerception I broke the connection in github.)
FYI: @wxmerkt and @VladimirIvan

[patmarion]

Sure, I'm happy to help. I haven't made an openhumanoids build in a while though. What branch of openhumanoids are you building with? Or, feel free to copy and paste and compiler errors (here or email) and I'll try to offer suggestions. If there are any changes you'd like to move into director/master I can help with that too.

I am leaving for a camping trip tomorrow though and won't be back until Tuesday!

[mauricefallon]

Probably easier if I just copy in the compile error here. This is the only compile error I haven't been able to resolve. It is triggered when I enable:
-DUSE_DRC_MAPS:BOOL=OFF

I guess the problem is in wrapping that library for Python:

[ 12%] Building CXX object src/vtk/DRCFilters/CMakeFiles/vtkDRCFiltersPythonD.dir/vtkLCMGLPropPython.cxx.o
[ 13%] Building CXX object src/vtk/DRCFilters/CMakeFiles/vtkDRCFiltersPythonD.dir/vtkCollectionsPython.cxx.o
[ 13%] Building CXX object src/vtk/DRCFilters/CMakeFiles/vtkDRCFiltersPythonD.dir/vtkDRCFiltersPythonInitImpl.cxx.o
make[4]: *** No rule to make target '/usr/lib/x86_64-linux-gnu/libpython2.7.soPythonD', needed by 'lib/libvtkDRCFiltersPythonD.so'. Stop.
make[4]: Leaving directory '/home/mfallon/rpg-navigation/software/director/distro/pods/drc/pod-build'
CMakeFiles/Makefile2:1050: recipe for target 'src/vtk/DRCFilters/CMakeFiles/vtkDRCFiltersPythonD.dir/all' failed
make[3]: *** [src/vtk/DRCFilters/CMakeFiles/vtkDRCFiltersPythonD.dir/all] Error 2
make[3]: Leaving directory '/home/mfallon/rpg-navigation/software/director/distro/pods/drc/pod-build'
Makefile:127: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/home/mfallon/rpg-navigation/software/director/distro/pods/drc/pod-build'
Makefile:26: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/home/mfallon/rpg-navigation/software/director/distro/pods/drc'
Makefile:16: recipe for target 'drc' failed
make: *** [drc] Error 2

The only change I made to cmake was for src/vtk/DRCFilters/CMakeLists.txt:
https://github.com/robotperception/director/pull/7/files#diff-10f186afafb41fd1b9f9f831b97e18e6
I wouldn't be confident I did this correctly.

Any ideas?

[patmarion]

Can you edit src/vtk/DRCFilters/CMakeLists.txt and go to the bottom, line 210, right before the last block of lines. Try adding:

message("VTK_LIBRARIES is:  ${VTK_LIBRARIES}")

set(VTK_LIBRARIES
  vtkWrappingPythonCore
  vtkGUISupportQt
  vtkInteractionWidgets
  vtkRenderingAnnotation
  vtkRenderingFreeType
  vtkInteractionStyle
  vtkIOXML
  vtkIOImage
  vtkIOGeometry)

My guess is that when USE_DRC_MAPS is enabled, and it causes a find_package(PCL) I wonder if PCL is setting the VTK_LIBRARIES variable to include other non-vtk libraries.

[patmarion]

Then when you run make or cmake it will print out the value of VTK_LIBRARIES, and I'd like to see what it is. Then I am re-setting VTK_LIBRARIES which might fix the problem.

[mauricefallon]

It complied successfully when I removed all the libraries in the variable by setting: set(VTK_LIBRARIES)

(Note that vtkWrappingPythonCore and vtkGIISupportQt could not be found)
But libvtkWrappingPython27Core-7.1.so and libvtkGUISupportQt-7.1.so do exist

VTK_LIBRARIES is: vtkIOPLY;vtkCommonCore;vtksys;vtkCommonDataModel;vtkCommonMath;vtkCommonMisc;vtkCommonSystem;vtkCommonTransforms;vtkIOCore;vtkCommonExecutionModel;vtkzlib;vtklibxml2;vtkChartsCore;vtkCommonColor;vtkFiltersGeneral;vtkCommonComputationalGeometry;vtkFiltersCore;vtkInfovisCore;vtkFiltersExtraction;vtkFiltersStatistics;vtkImagingFourier;vtkImagingCore;vtkalglib;vtkRenderingContext2D;vtkRenderingCore;vtkFiltersGeometry;vtkFiltersSources;vtkRenderingFreeType;vtkfreetype;vtkFiltersHybrid;vtkImagingSources;vtkFiltersImaging;vtkImagingGeneral;vtkFiltersAMR;vtkIOXML;vtkIOXMLParser;vtkexpat;vtkParallelCore;vtkIOLegacy;vtkFiltersFlowPaths;vtkIOSQL;vtksqlite;vtkImagingMorphological;vtkIOExport;vtkIOImage;vtkDICOMParser;vtkmetaio;vtkjpeg;vtkpng;vtktiff;vtkRenderingGL2PS;vtkRenderingContextOpenGL;vtkRenderingOpenGL;vtkImagingHybrid;vtkgl2ps;vtkFiltersHyperTree;vtkFiltersVerdict;verdict;vtkIOAMR;vtkhdf5_hl;vtkhdf5;vtkInteractionImage;vtkImagingColor;vtkInteractionStyle;vtkInteractionWidgets;vtkFiltersModeling;vtkRenderingAnnotation;vtkRenderingVolume;vtkRenderingImage;vtkFiltersPython;/usr/lib/x86_64-linux-gnu/libpython2.7.so;vtkWrappingPythonCore;vtkWrappingTools;vtkGUISupportQt;vtkFiltersPoints;vtkGeovisCore;vtkInfovisLayout;vtkViewsCore;vtkproj4;vtkFiltersGeneric;vtkIOMINC;vtkNetCDF;vtkNetCDF_cxx;vtkViewsInfovis;vtkRenderingLabel;vtkFiltersSMP;vtkDomainsChemistry;vtkIOParallel;vtkFiltersParallel;vtkIOGeometry;vtkIONetCDF;vtkexoIIc;vtkjsoncpp;vtkFiltersSelection;vtkoggtheora;vtkFiltersParallelImaging;vtkIOMovie;vtkIOInfovis;vtkIOTecplotTable;vtkIOEnSight;vtkIOParallelXML;vtkIOExodus;vtkIOLSDyna;vtkViewsContext2D;vtkRenderingVolumeOpenGL;vtkRenderingLOD;vtkFiltersTexture;vtkImagingMath;vtkIOVideo;vtkImagingStencil;vtkFiltersProgrammable;vtkRenderingLIC;vtkIOImport;vtkImagingStatistics

[gtinchev]

@patmarion Could you please specify what the VTK_LIBRARIES should output?
Currently I have only the following ones:

VTK_LIBRARIES is: vtkInteractionWidgets;vtkInteractionStyle;vtkRenderingFreeType;vtkRenderingCore;vtkRenderingOpenGL;vtkFiltersExtraction;vtkFiltersCore;vtkCommonCore

Director compiles, but segfaults. (that's without Maps' Server)

[patmarion]

Earlier in src/app/CMakeLists.txt this code is called which initializes the VTK_LIBRARIES variable:

find_package(VTK REQUIRED COMPONENTS
  vtkWrappingPythonCore
  vtkGUISupportQt
  vtkInteractionWidgets
  vtkRenderingAnnotation
  vtkRenderingFreeType
  vtkInteractionStyle
  vtkIOXML
  vtkIOImage
  vtkIOGeometry
  )
include(${VTK_USE_FILE})

I suspect that later, when find_package(PCL REQUIRED) is called, the pcl cmake script overwrites the vtk variable. It's possible that duplicating the find_package(VTK) afterward will reset things correctly.

If director segfaults, I guess it could be because there are multiple versions of vtk installed. So director compiles and links against vtk7.1, but at runtime the python code import vtk finds the system installed vtk5. One way to avoid this is to prepend the PYTHONPATH environment variable with the location of the vtk7 python files (using something like prefix/lib/python2.7/dist-packages). Then try running: python -c "import vtk; print vtk.__file__" and make sure the correct location is dispalyed. Another workaround is to apt-get remove the system vtk5.

[gtinchev]

@patmarion I did have VTK6.2 installed systemwide. I have removed it since, and trying to compile once again.

I have manually unpacked (i.e. ctrl+c and ctrl+v) the precompiled vtk7.1 into software/build:
http://patmarion.com/bottles/vtk7.1-qt4.8-python2.7-ubuntu16.04.tar.gz

And I'm trying to get director to use that, however, the following packages cannot be found from that compiled version of VTK 7.1:

/usr/bin/ld: cannot find -lvtkIOODBC
/usr/bin/ld: cannot find -lvtkRenderingParallelLIC
/usr/bin/ld: cannot find -lvtkParallelMPI
/usr/bin/ld: cannot find -lvtkIOParallelLSDyna
/usr/bin/ld: cannot find -lvtkParallelMPI4Py
/usr/bin/ld: cannot find -lvtkPythonInterpreter
/usr/bin/ld: cannot find -lvtkTestingGenericBridge
/usr/bin/ld: cannot find -lvtkIOGDAL
/usr/bin/ld: cannot find -lvtkInfovisBoostGraphAlgorithms
/usr/bin/ld: cannot find -lvtkIOVPIC
/usr/bin/ld: cannot find -lvtkIOParallelExodus
/usr/bin/ld: cannot find -lvtkRenderingParallel
/usr/bin/ld: cannot find -lvtkRenderingQt
/usr/bin/ld: cannot find -lvtkIOFFMPEG
/usr/bin/ld: cannot find -lvtkGUISupportQtOpenGL
/usr/bin/ld: cannot find -lvtkTestingIOSQL
/usr/bin/ld: cannot find -lvtkWrappingJava
/usr/bin/ld: cannot find -lvtkFiltersParallelMPI
/usr/bin/ld: cannot find -lvtkTestingRendering
/usr/bin/ld: cannot find -lvtkIOXdmf2
/usr/bin/ld: cannot find -lvtkIOPostgreSQL
/usr/bin/ld: cannot find -lvtkIOParallelNetCDF
/usr/bin/ld: cannot find -lvtkRenderingFreeTypeFontConfig
/usr/bin/ld: cannot find -lvtkGUISupportQtWebkit
/usr/bin/ld: cannot find -lvtkViewsQt
/usr/bin/ld: cannot find -lvtkRenderingExternal
/usr/bin/ld: cannot find -lvtkIOMPIImage
/usr/bin/ld: cannot find -lvtkFiltersReebGraph
/usr/bin/ld: cannot find -lvtkRenderingMatplotlib
/usr/bin/ld: cannot find -lvtkIOGeoJSON
/usr/bin/ld: cannot find -lvtkRenderingFreeTypeOpenGL
/usr/bin/ld: cannot find -lvtkIOMPIParallel
/usr/bin/ld: cannot find -lvtkFiltersParallelFlowPaths
/usr/bin/ld: cannot find -lvtkViewsGeovis
/usr/bin/ld: cannot find -lvtkLocalExample
/usr/bin/ld: cannot find -lvtkIOMySQL
/usr/bin/ld: cannot find -lvtkFiltersParallelStatistics
/usr/bin/ld: cannot find -lvtkFiltersParallelGeometry
/usr/bin/ld: cannot find -lvtkGUISupportQtSQL

as well as:

/usr/bin/ld: cannot find -lvtkftgl
/usr/bin/ld: cannot find -lvtkPythonInterpreter
/usr/bin/ld: cannot find -lvtkFiltersReebGraph
/usr/bin/ld: cannot find -lvtkLocalExample
/usr/bin/ld: cannot find -lvtkxdmf2

When checking the where python's vtk is pointing at (python -c "import vtk; print vtk.__file__") I get:

/home/gtinchev/rpg-navigation/software/build/lib/python2.7/site-packages/vtk/__init__.pyc

Which seems correct.

Do you have any suggestions where to get the aforementioned packages, if they're not included in the precompiled version of VTK7.1?

[patmarion]

What is printing that error message? Could you post this in the context of the build log? Also can you post the CmakeCache.txt file?

[gtinchev]

I'm trying to build director using make drc -j6
Build log: https://pastebin.com/XwPB5L98
CMakeCache.txt: https://pastebin.com/iXNf1dWD
src/vtk/DRCFilters/CMakeLists.txt: https://pastebin.com/RUbffpDU
distro/pods/drc/Makefile: https://pastebin.com/PqPSFPNW

Thanks in advance for your help!

[patmarion]

I think the problem is PCL. You'll need to make sure that PCL is compiled with the same version of VTK 7.1.

PCL_DIR:PATH=/home/gtinchev/rpg-navigation/software/build/share/pcl-1.8

If you just rm the pcl build and re-run it then it might just work. You can always help ensure it finds the correct version of VTK by setting this environment variable: export CMAKE_PREFIX_PATH=/home/gtinchev/rpg-navigation/software/build/share:$CMAKE_PREFIX_PATH but it might not be necessary.

[gtinchev]

You were correct, the PCL was build under the old version of VTK (6.2). I'm currently re-building PCL w/ VTK7.1 and will try again.
One thing I've noticed is that PCL is currently linking to Qt5 (system installed) - is that going to cause problems for director?

[patmarion]

It might cause an issue, but it might be ok. But you can apt-get install both qt4 and qt5 on ubuntu, they do not conflict at all. Then you can build pcl with the system version of qt4, which is the same version that the packaged vtk7.1 uses. Another alternative is to compile vtk from source using the system qt5, if you really want to use qt5.