Memory leak problem

[cycaoyang]

Hello, I am a beginner of reinforcement learning. When I ran your code, I noticed that the server's memory utilization skyrocketed, causing the final workout to stop midway through. So I have a question: Is there a memory leak in your code?

[godka]

Hi Caoyang,

We started reproducing the problem you reported. However, we didn't observe a memory leak -- the memory utilization is kept around 380MB, detailed are shown as follows:

Here are the raw logs, including memory and tensorboard logs

events.out.tfevents.1681531853.zip

Feel free to let us know if you have any questions.

Best,
Tianchi