Tuning Interface revision

[crtrott]

My scatch of how I think this could look after the changes. Note how Tools::VariableValue is now only used in places where I actually semantically have to associate values with some id in the tool.

#include <Kokkos_Core.hpp>
#include <Kokkos_Macros.hpp>
#include <cmath>
#include <array>
#include <mpi.h>
#include <chrono>
#include <thread>
#include <iostream>
#include <vector>
constexpr const int num_iterations = 1000;
constexpr const float penalty = 0.1f;
void sleep_difference(int actual, int guess){
  std::this_thread::sleep_for(std::chrono::milliseconds(int(penalty * std::abs(guess-actual))));
}

// This is a simple tuning example, in which we have a function that takes two values and
// sleeps for their difference. The tool is given one value, and asked to guess the other
//
// The correct answer is to guess the same value the tool was given in the context

int main(int argc, char* argv[]) {
  MPI_Init(&argc, &argv);
  Kokkos::initialize(argc, argv);
  // *Here we want to declare a context variable, something a tool might tune using
  //

  Kokkos::Tools::VariableInfo context_info;
  // ** now we want to let the tool know about the semantics of that variable
  // *** it's an integer
  context_info.type          = kokkos_value_integer;
  // *** it's 'interval', roughly meaning that subtracting two values makes sense
  context_info.category      = kokkos_value_interval;
  // *** and the candidate values are in a range, not a set 
  context_info.valueQuantity = kokkos_value_range;
  Kokkos::Tools::SetOrRange value_range;

  // ** this is just the earlier "range" construction
  // ** the last two values are bools representing whether 
  // ** the range is open (endpoints not included) or closed (endpoints included)
  // ** Why would this need the context_variable_id?
  value_range.range = Kokkos::Tools::ValueRange{
      /*context_variable_id,*/ int64_t(0), int64_t(num_iterations),
      int64_t(1),         false,        false};
  // ** here we actually declare it to the tool
  size_t context_variable_id = Kokkos::Tools::declareVariableType("target_value",
                                         context_info, value_range);


   // ** its semantics exactly mirror the tuning variable, it's an
  // ** integer interval value
  Kokkos::Tools::VariableInfo tuning_variable_info;
  tuning_variable_info.category      = kokkos_value_interval;
  tuning_variable_info.type          = kokkos_value_integer;

  // ** Here I'm setting the candidate values to be from a set for two reasons
  // ** 1) It shows the other side of this interface
  // ** 2) ... the prototype tool doesn't support guessing from ranges yet
  tuning_variable_info.valueQuantity = kokkos_value_set;

  std::vector<int64_t> candidate_value_array { 0, 100, …, 900 };

  Kokkos::Tools::SetOrRange candidate_values;
  candidate_values.set = Kokkos::Tools::ValueSet{/*tuning_value_id,*/ 10,
                                                  candidate_value_array.data()};
  size_t tuning_value_id = Kokkos::Tools::declareTuningVariable("tuned_choice", 
                                        tuning_variable_info, candidate_values);

  {
    // * declaring a VariableValue which can hold the results
    // *   of the tuning tool
    Kokkos::Tools::VariableValue tuned_choice;
    tuned_choice.id = tuning_value_id;
    // ** Note that the default value must not crash the program
    tuned_choice.value.int_value = 0;

    // * Looping multiple times so the tool can converge
    for(int attempt =0;attempt<120;++attempt){
    for (int work_intensity = 0; work_intensity < num_iterations;
         work_intensity+=200) {
      std::cout << "Attempting a run with value "<<work_intensity<<std::endl;
      size_t contextId = Kokkos::Tools::getNewContextId();

      // *Here we tell the tool the value of the context variable
      Kokkos::Tools::VariableValue context_value = Kokkos::Tools::make_variable_value(
              context_variable_id, int64_t(work_intensity));
      Kokkos::Tools::setInputValues(
          contextId, 1, 
          &context_value);
            
      // *Now we ask the tool to give us the value it thinks will perform best
      Kokkos::Tools::getOutputValues(contextId, 1, &tuned_choice); 

      // *Call the function with those two values
      sleep_difference(work_intensity, tuned_choice.value.int_value);

      // *This call tells the tool the context is over, so it can
      // *take measurements
      Kokkos::Tools::endContext(contextId);
    }
    }
  }
  Kokkos::finalize();
  MPI_Finalize();
}

[vlkale]

I am noting that the tuning/autotuning interface ought to also consider multiple objectives, which are potentially ranked/weighted.

When we talk about autotuning in the area High-Performance Computing, we often assume an application programmer wants to find the best performing parameters to maximize performance (through metrics such as FLOPS or GB/s), which is single objective. In reality, finding the best tunedChoice for, e.g., maximizing performance and minimizing energy usage (two objectives) is often useful as well, as suggested in Github Issue #92. We could generalize this further to multiple objectives. For example, we autotune the parameters for performance, energy efficiency and numerical accuracy. It may also be useful to identify which objective carries most weight (quantifying the weights) or just rank them, so that (consider the 3-objective example) numerical accuracy is the first priority objective for tuning, the performance is the second priority objective and energy efficiency is the third priority objective.

[vlkale]

Question of interest to ANL and UOregon: What do values of tuning variables mean for Kokkos-specific parameters? Right now, they are opaque and normalized values between 0.0 and 1.0. Can they be actual values of the parameters, e.g., team size of 16?

@khuck