Question about advantage function computation

[pruksmhc]

In this line:

instructGOOSE/instruct_goose/trainer.py

Line 49 in 5b9ac3f

for t in reversed(range(gen_len)):

What is the purpose of iterating through in the 0th dimension, given the tensor is of size '1 b' (so it will only iterate once).

[xrsrke]

Hi @pruksmhc. Sorry for the delayed response. The compute_advantage_and_return function treats all the text in a batch as an episode. Therefore, technically, the batch size is not one.

[pruksmhc]

Thanks for the response!
However, in this line

instructGOOSE/instruct_goose/trainer.py

Line 39 in 5b9ac3f

rewards = rearrange(rewards, 'b -> 1 b')

, you cast the tensor to size 1 in dimension 0, and gen_len is len(rewards), which is 1.