Record gradients every epoch

[takuseno]

@hasan-yaman Hi, I really like the new feature you added. Now, I'd like to push this forward to make it a default metric. In this PR, I dropped gradient_logging_steps. Instead, the logger saves the metrics every epoch. Does this still work for your use case?

[codecov]

Codecov Report

Attention: Patch coverage is 58.49057% with 22 lines in your changes missing coverage. Please review.

Project coverage is 91.01%. Comparing base (7951f3d) to head (8e6ec36).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
d3rlpy/logging/file_adapter.py	38.09%	13 Missing ⚠️
d3rlpy/logging/tensorboard_adapter.py	44.44%	5 Missing ⚠️
d3rlpy/logging/wandb_adapter.py	25.00%	3 Missing ⚠️
d3rlpy/logging/utils.py	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #425      +/-   ##
==========================================
- Coverage   91.06%   91.01%   -0.06%     
==========================================
  Files         118      118              
  Lines        8353     8360       +7     
==========================================
+ Hits         7607     7609       +2     
- Misses        746      751       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hasan-yaman]

Hi @takuseno

Yes, it still works for my case.

This metric creates many artifacts, that's why I didn't set it as a default metric.

Now, adding this as a default metric I think we should document it. By the way, I think metric is self explanatory when using it with Tensorboard or WandB, but not for FileAdapter. Maybe we can add a header to CSV files?

[takuseno]

Maybe we can add a header to CSV files?

That's a good idea. I have updated this PR to include the header and additionally record standard deviation. I'll go ahead merge this. Thanks for the feedback! I think this number of artifacts are still acceptable.