From b99b539a1da5ee044eaa756203e5b74216a4b796 Mon Sep 17 00:00:00 2001 From: Arjun Suresh Date: Wed, 15 Jan 2025 00:50:55 +0000 Subject: [PATCH 1/2] Create TEST01 audit.conf for rgat --- compliance/nvidia/TEST01/rgat/audit.conf | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 compliance/nvidia/TEST01/rgat/audit.conf diff --git a/compliance/nvidia/TEST01/rgat/audit.conf b/compliance/nvidia/TEST01/rgat/audit.conf new file mode 100644 index 000000000..c861e2a3d --- /dev/null +++ b/compliance/nvidia/TEST01/rgat/audit.conf @@ -0,0 +1,9 @@ +# The format of this config file is 'key = value'. +# The key has the format 'model.scenario.key'. Value is mostly int64_t. +# Model maybe '*' as wildcard. In that case the value applies to all models. +# All times are in milli seconds + +# mode dictionary (0 = submission, 1 = accuracy, 2 = performance, 3 = find peak perf) +*.*.mode = 2 +*.*.accuracy_log_rng_seed = 720381539243781796 +*.*.accuracy_log_sampling_target = 4096 From 65705f2b2ad5991b02a17bbe05c954c914d34c21 Mon Sep 17 00:00:00 2001 From: Arjun Suresh Date: Wed, 15 Jan 2025 16:03:16 +0000 Subject: [PATCH 2/2] Update benchmark-checklist.md | RGAT checklist completed --- graph/R-GAT/benchmark-checklist.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/graph/R-GAT/benchmark-checklist.md b/graph/R-GAT/benchmark-checklist.md index f83c816cb..1d0a8d379 100644 --- a/graph/R-GAT/benchmark-checklist.md +++ b/graph/R-GAT/benchmark-checklist.md @@ -48,11 +48,11 @@ Documented whether **Equal Issue Mode** is applicable in --- #### **9. Reference Implementation Dataset Coverage** -- [ ] Reference implementation successfully processes the entire validation dataset during: - - [ ] Performance runs - - [ ] Accuracy runs - - [ ] Compliance runs -- [ ] Valid log files passing the submission checker are generated for all runs. +- [X] Reference implementation successfully processes the entire validation dataset during: + - [X] Performance runs + - [X] Accuracy runs + - [X] Compliance runs +- [X] Valid log files passing the submission checker are generated for all runs - [link](https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.0/tree/main/closed/MLCommons/results/mlc-server-reference-gpu-pytorch_v2.4.0-cu124/rgat/offline/performance/run_1). --- @@ -81,7 +81,8 @@ Documented whether **Equal Issue Mode** is applicable in --- #### **14. Sample Log Files** -- [ ] Include sample logs for all the applicable scenario runs: - - [ ] `mlperf_log_summary.txt` - - [ ] `mlperf_log_detail.txt` -- [ ] Ensure sample logs successfully pass the submission checker and applicable compliance runs. +- [X] Include sample logs for all the applicable scenario runs: + - [X] Offline + - [X] [`mlperf_log_summary.txt`](https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.0/blob/main/closed/MLCommons/results/mlc-server-reference-gpu-pytorch_v2.4.0-cu124/rgat/offline/performance/run_1/mlperf_log_summary.txt) + - [X] [`mlperf_log_detail.txt`](https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.0/blob/main/closed/MLCommons/results/mlc-server-reference-gpu-pytorch_v2.4.0-cu124/rgat/offline/performance/run_1/mlperf_log_detail.txt) +- [X] Ensure sample logs successfully pass the submission checker and applicable compliance runs. [Link](https://htmlpreview.github.io/?https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.0/blob/refs/heads/auto-update/closed/MLCommons/results/mlc-server-reference-gpu-pytorch_v2.4.0-cu124/summary.html)