Add end on device support to Loadgen

[nvitramble]

Adds optional callback argument to QuerySamplesComplete() which will allow the user to perform a accelerator->host copy after timing the query response time and before logging the response data for accuracy mode.

Discussed in mlcommons/inference_policies#213.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[nvpohanh]

@guschmue This looks like an infra issue:

[2021-06-14 22:13:36] [build-stderr] Traceback (most recent call last):
[2021-06-14 22:13:36] [build-stderr]   File "setup.py", line 29, in <module>
[2021-06-14 22:13:36] [build-stderr]     from version_generator import generate_loadgen_version_definitions
[2021-06-14 22:13:36] [build-stderr]   File "/opt/src/loadgen/version_generator.py", line 25, in <module>
[2021-06-14 22:13:36] [build-stderr]     from absl import app
[2021-06-14 22:13:36] [build-stderr] ImportError: No module named absl

[guschmue]

I can take a look at absl, probably some dependency missing in setup.py

[guschmue]

@nvpohanh - good to with you to merge this PR?

[guschmue]

PR with the fix for the absl dependency is here #952

[nvpohanh]

Looks good to me. Thanks for implementing this!

[nvpohanh]

@guschmue we can merge this