R-GAT Offline Test failed with file not found in the dataset directory

[dineshchitlangia]

Discussed in #2041

^{Originally posted by dineshchitlangia January 16, 2025}
I setup R-GAT following the README and downloaded the full dataset.
Verified the dataset directory is 2.2TB as expected.

What am I missing?

(gnn) $:/mlperf/inference/graph/R-GAT$ python3 main.py --dataset igbh-dgl --dataset-path igbh/ --profile rgat-dgl-full --model-path $MODEL_PATH --device cpu --dtype fp32 --scenario Offline

(gnn) $:/mlperf/inference/graph/R-GAT$ INFO:main:Namespace(dataset='igbh-dgl', dataset_path='igbh/', in_memory=False, layout='COO', profile='rgat-dgl-full', scenario='Offline', max_batchsize=1, threads=1, accuracy=False, find_peak_performance=False, backend='dgl', model_name='rgat', output='output', qps=None, model_path='/mlperf/inference/graph/R-GAT/model/', dtype='fp32', device='cpu', user_conf='user.conf', audit_conf='audit.config', time=None, count=None, debug=False, performance_sample_count=5000, max_latency=None, samples_per_query=8)
/mlperf/inference/graph/R-GAT/dgl_utilities/feature_fetching.py:231: UserWarning: The given NumPy array is not writable, and PyTorch does              not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to              protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this prog             ram. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:206.)
  return edge, torch.from_numpy(
Traceback (most recent call last):
  File "/mlperf/inference/graph/R-GAT/main.py", line 510, in <module>
    main()
  File "/mlperf/inference/graph/R-GAT/main.py", line 363, in main
    ds = dataset_class(
  File "/mlperf/inference/graph/R-GAT/dgl_utilities/feature_fetching.py", line 131, in __init__
    self.igbh_dataset = IGBHeteroGraphStructure(
  File "/mlperf/inference/graph/R-GAT/dgl_utilities/feature_fetching.py", line 203, in __init__
    self.edge_dict = self.load_edge_dict()
  File "/mlperf/inference/graph/R-GAT/dgl_utilities/feature_fetching.py", line 237, in load_edge_dict
    loaded_edges = {
  File "/mlperf/inference/graph/R-GAT/dgl_utilities/feature_fetching.py", line 237, in <dictcomp>
    loaded_edges = {
  File "/home/amd/miniconda3/envs/gnn/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/home/amd/miniconda3/envs/gnn/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/home/amd/miniconda3/envs/gnn/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/home/amd/miniconda3/envs/gnn/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/amd/miniconda3/envs/gnn/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/mlperf/inference/graph/R-GAT/dgl_utilities/feature_fetching.py", line 232, in load_edge
    np.load(osp.join(parent_path, edge, "edge_index.npy"), mmap_mode=mmap))
  File "/home/amd/miniconda3/envs/gnn/lib/python3.10/site-packages/numpy/lib/npyio.py", line 427, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: 'igbh/full/processed/paper__written_by__author/edge_index.npy'

Contents of dataset

(gnn)$:/mlperf/inference/graph/R-GAT$ ls -R igbh/full/processed/
igbh/full/processed/:
author  paper  paper__cites__paper  train_idx.pt  val_idx.pt

igbh/full/processed/author:
author_id_index_mapping.npy  node_feat.npy

igbh/full/processed/paper:
node_feat.npy  node_label_19.npy  node_label_2K.npy  paper_id_index_mapping.npy

igbh/full/processed/paper__cites__paper:
edge_index.npy

On investigating the stack trace further:

inference/graph/R-GAT/dgl_utilities/feature_fetching.py

Line 219 in ef1bd3b

edges = [

expects

        edges = [
            "paper__cites__paper",
            "paper__written_by__author",
            "author__affiliated_to__institute",
            "paper__topic__fos"]

But it seems the dataset does not have other edges except "paper__cites__paper"

[mrmhodak]

@arjunsuresh @ashwin @pgmpablo157321 - Can you help?

I do not know github handle of Uday.

[arjunsuresh]

@dineshchitlangia This is the contents of igbh dataset we have.

ls /cm-mount/data/common/anandhu/igbh/full/processed/
author                            checksums_md5.txt  conference  institute  paper                paper__published__journal  paper__venue__conference   train_idx.pt
author__affiliated_to__institute  computemd5sum.sh   fos         journal    paper__cites__paper  paper__topic__fos          paper__written_by__author  val_idx.pt

Checksums

cat /cm-mount/data/common/anandhu/igbh/full/processed/checksums_md5.txt
Processing folder: author/
58c15aab7dae03bbd57e6a4ac5e61bd9  author//author_id_index_mapping.npy
2ec2512b554088381c04ec013e893c8d  author//node_feat.npy
f1daf0d944478aca9ae4d9e3763e04ce  author//wget-log
Processing folder: author__affiliated_to__institute/
e35dba208f81e0987207f78787c75711  author__affiliated_to__institute//edge_index.npy
Processing folder: conference/
0bf7c555d8c697b31b6af6c4cb6b6612  conference//conference_id_index_mapping.npy
898ff529b8cf972261fedd50df6377f8  conference//node_feat.npy
Processing folder: fos/
0f0cfde619361cde35d3be9f201d081a  fos//fos_id_index_mapping.npy
3ef3df19e2475c387fec10bac82773df  fos//node_feat.npy
Processing folder: institute/
03fb45eafb7bd35875ef4c7cd2a299a9  institute//institute_id_index_mapping.npy
12eaeced22d17b4e97d4b4742331c819  institute//node_feat.npy
Processing folder: journal/
b630c20852b76d17a5c9c37b39176f69  journal//journal_id_index_mapping.npy
49d51b554b3004f10bee19d1c7f9b416  journal//node_feat.npy
Processing folder: paper/
71058b9ac8011bafa1c5467504452d13  paper//node_feat.npy
be6fda45566e679bdb05ebea98ad16d4  paper//node_label_19.npy
6eccab9a14f92f42be5b367c39002031  paper//node_label_2K.npy
f70dd642a4f7e41d926c91c8c054fc4c  paper//paper_id_index_mapping.npy
Processing folder: paper__cites__paper/
f4897f53636c04a9c66f6063ec635c16  paper__cites__paper//edge_index.npy
Processing folder: paper__published__journal/
38505e83bde8e5cf94ae0a85afa60e13  paper__published__journal//edge_index.npy
Processing folder: paper__topic__fos/
427fb350a248ee6eaa8c21cde942fda4  paper__topic__fos//edge_index.npy
Processing folder: paper__venue__conference/
541b8d43cd93579305cfb71961e10a7d  paper__venue__conference//edge_index.npy
Processing folder: paper__written_by__author/
df39fe44bbcec93a640400e6d81ffcb5  paper__written_by__author//edge_index.npy

[dineshchitlangia]

@arjunsuresh ok, let me give another try and download dataset again. I will report back.