Shape mismatch when training

[martinarroyo]

Steps taken:

• Cloned main branch today.
• Downloaded ECSSD and MSRA10K.
• Set up repository with PyTorch 1.8.
• Launched bash train_eval.sh.

Right on the first training step, the following error is raised:

 File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
    fn(i, *args)
  File "/aot-benchmark/tools/train.py", line 18, in main_worker
    trainer.sequential_training()
  File "./networks/managers/trainer.py", line 473, in sequential_training
    use_prev_prob=use_prev_prob)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./networks/engines/aot_engine.py", line 52, in forward
    self.add_reference_frame(frame_step=0, obj_nums=obj_nums)
  File "./networks/engines/aot_engine.py", line 239, in add_reference_frame
    size_2d=self.enc_size_2d)
  File "./networks/models/aot.py", line 105, in LSTT_forward
    pos_emb, size_2d)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./networks/layers/transformer.py", line 113, in forward
    size_2d=size_2d)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./networks/layers/transformer.py", line 352, in forward
    tgt3 = self.short_term_attn(local_Q, local_K, local_V)[0]
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./networks/layers/attention.py", line 525, in forward
    output = agg_value + agg_bias
RuntimeError: The size of tensor a (32) must match the size of tensor b (256) at non-singleton dimension 3

I did not alter any of the configuration values. I noticed that #2 had the same issue, but no solution was provided there. Any help would be much appreciated!

Thanks in advance

[94kiki]

I also have this problem.

[yoxu515]

Hello, sorry for the error. This bug exits when the "pytorch correlation" is not correctly installed, similar in issue #45. I recommend you to install it and the code also runs faster with it.

[94kiki]

Thank you very much!

[martinarroyo]

Thanks for the clarification. I am now testing this and will report back. In the meanwhile, I updated the README to explain this situation to future users.