Different device selection for the same platform like GPU.

[yinrun]

Hi all. I am working on a server with multiple accelerator devices such as GPUs. When I as creating several sycl queues, I want to assign each queue to the specific GPU device I assigned. In cuda, we can use cudaSetDevice to choose the specific device , in dpc++ runtime, the selection seems to be made in device_selector::select_device() and it will select a default gpu device, how can I bind different queue to different accelerators respectively?

[AlexeySachkov]

Hi @yinrun,

When I as creating several sycl queues, I want to assign each queue to the specific GPU device I assigned.

This is for sure possible, just take a look at constructors available for sycl::queue: you explicitly pass device which interest you or pass device_selector to choose it during queue creation.

in dpc++ runtime, the selection seems to be made in device_selector::select_device() and it will select a default gpu device

That is because you are likely using default selector - it indeed contains some hardcoded logic which favors GPUs. However, there are other selectors available according to the spec and you can even create your own:

int prefer_my_vendor(const sycl::device & d) {
  // Return 1 if the vendor name is "MyVendor" or 0 else.
  // 0 does not prevent another device to be picked as a second choice
  return d.get_info<info::device::vendor>() == "MyVendor";
}

// Get the preferred device or another one if not available
sycl::device preferred_device { prefer_my_vendor };

The example is copied from the spec section I linked above.

Let me know if you other questions

[AlexeySachkov]

and they override operator(), I am not sure where I can set different devices.

That is correct, device selector is a callable object which is called for every available device computing the score for it. Device with the highest score is returned to you.

Therefore, you have a few options to explore:

1. Create your own device selector in form of a functor (struct with operator()) which remembers which device was selected the first time to avoid selecting it for the second time. The idea is to select device for the first time, store selected device into device selector and select device for the second time.
2. use device::get_devices() to get list of devices and manually filter them selecting whichever you want
3. Another option for device selector: if you know some unique properties which are different for the first and second device, then you can just prepare two device selectors that look for those properties - it could be a vendor name, device name, driver version, etc.

[alexbatashev]

@yinrun Alexey is probably referring to platform::get_devices() You can do something like:

#include <algorithm>
...

sycl::platform Platform{...}; // somehow select your platform, e.g. with device selector
auto Devices = Platform.get_devices();
std::vector<sycl::queue> Queues;
std::transform(Devices.begin(), Devices.end(), std::back_inserter(Queues), [](const sycl::device &D) {
  return sycl::queue{D};
});

This will give you one queue per each device in the platform.

Also there's a FilterSelector.

[yinrun]

Hi @alexbatashev @AlexeySachkov , Platform.get_devices() is a very good idea. Howerver, in my situation, I have two seperate progress to use different devices and all the devices are totally the same(3rd option maybe not suitable). for the 1st option, do you mean I need something like a global id in method such as int gpu_selector::operator()(const device &dev) const', this method is used in select_device()` method and I don't know how to pass a parameter like device_id to it.

[alexbatashev]

If you somehow can identify your processes, you can go for filter_selector, which accepts strings in format cuda:gpu:0. Just replace 0 with gpu id. It is guaranteed to be stable across runs (but not across different machines, obviously).

[yinrun]

Thanks @alexbatashev I think this is exactly what I want. I will try it.