Tutorial Introduction to PyTorch

In this repository, we present a basic tutorial for PyTorch. We provide basic examples of how to apply automatic differentiation. Additionally, using torch.nn, we demonstrate how to solve regression problems and how to program a neural network to solve multi-classification problems (using datasets.make_moons and datasets.make_circles from sklearn as well as the classic MNIST dataset). The considered architectures range from multilayer perceptrons (MLPs) (from 1-layer to deep neural networks) to residual networks (ResNets). The table of contents is as follows:

Contents

1. Theoretical Framework

   • 1.1. Optimization Problem
   • 1.2. Classical Models
   • 1.3. Optimization Algorithms in Deep Learning

2. Programming

   • 2.1. Basic Operations in PyTorch
   • 2.2. Linear Regression
   • 2.3. One-Layer Neural Network for Classification (Exercise)
   • 2.4. Three-Layer Neural Network (MNIST)
   • 2.5. ResNet for Classification

The final example, in particular, allows us to observe how mini-batch gradient descent can overcome saddle points to achieve data classification. We also show how the "dynamic" of a residual neural network evolves, as shown in the animation below.

Any comments or suggestions are welcome :)!