Made MPI calls in exchangeStateVectors non-blocking

[jjacobx]

The function exchangeStateVectors was modified to take advantage of non-blocking MPI communication.

Currently, QuEST uses blocking communication calls to exchange the statevector in chunks of 2GB. On large systems, which feature high memory and a fast interconnect, this caused a noticeable slowdown. By changing the communication to non-blocking, up to 10% speedup was achieved on ARCHER2, as described in the following paper: https://arxiv.org/abs/2308.07402.

Ideally the request array would not be allocated and freed in this function, however, placing it elsewhere would require introducing MPI headers in areas of the code in which they are not currently required, for example in QuESTEnv.

[TysonRayJones]

Hi there,

Thanks for an excellent contribution! Absolutely kicking myself that the chunks were ever sent serially.
I'm running the unit tests on some beefy distributed hardware, and have confirmed the code works for few chunks.

I think the requests malloc is fine. The number of messages depends on the Qureg size, and even the extreme case of a 1 TB RAM node exchanging a RAM-filling statevector, requests has only ~2000 elements.

Do you have a preferred name to be referred to as a QuEST contributor? (examples here)

[jjacobx]

Hi Tyson, thank you for accepting my contribution! You can refer to me as Jakub Adamski.