Convex Hull Value Iteration

Unofficial implementation of the algorithm proposed in the article "Learning All Optimal Policies with Multiple Criteria" by Barrett and Narayanan (Proceedings of the International Conference on Machine Learning, 2008). This implementation has been used in the experimental evaluation of "Guaranteeing Alignment with Value Systems by Means of Multi-Objective Reinforcement Learning" by Rodriguez-Soto et al. (under review in Artificial Intelligence, 2024).

Download

Clone this repository with --recursive to clone all submodules recursively.

Required Packages

• CMake
• A version of g++ supporting C++20
• OpenMP for C++ parallelism
• Gperftools for profiling
• pip

Compile Qhull with PIC and Optimizations

1. Navigate to the chvi/qhull subdirectory
2. Adjust the desired optimization options in the CMakeList.txt file. Levels -O3 and above can potentially affect the output of Qhull for some degenerate instances.
3. Build and install Qhull with cmake ., make and sudo make install

Build and Install the Cython Module

1. Download required packages with pip install -r requirements.txt
2. Build the Cython module and package it in a Python Wheel with ./build.sh
3. Install the Python Wheel with ./install.sh

Build the Native C++ Version

1. Navigate to the chvi subdirectory
2. Compile the C++ native version with ./build.sh