Version 0.3.5

[muditbhargava66]

Features

• Simple and easy-to-use gradient regularization technique
• Compatible with various optimizers and learning rate schedulers
• Supports per-parameter drop rates for fine-grained control
• Implements drop rate schedulers for dynamic regularization
• Provides an option to apply "full" update drop for further regularization
• Utilizes mixed-precision training for improved performance and memory efficiency (CUDA devices only)
• Cross-platform compatibility: Works seamlessly on macOS, Windows, and Linux

What's New in Version 0.3.5?

• Added support for the Lion optimizer in the ViT experiments
• Implemented gradient clipping to prevent gradient explosion and improve training stability
• Enhanced data augmentation techniques for better model generalization
• Improved error handling and user interruption handling during training
• Updated test suite to cover various aspects of DropGrad, including initialization, optimization step, drop rate scheduling, and saving of loss values
• Code refactoring and documentation enhancements for better readability and maintainability

[dingo-actual]

Will review this weekend when I have bandwidth. Thanks for the contribution!

[muditbhargava66]

Are there any updates or changes for this pull request? Please let me know so I can update my local repository.

[dingo-actual]

Sorry about the delay. I haven't been feeling that great this weekend. I'll try to look over the PR before I go to bed.