🔥 PyTorch Workshop

A comprehensive PyTorch workshop covering the fundamentals and advanced techniques of deep learning.

📋 Table of Contents

1. Introduction to Tensors
   Learn about PyTorch tensors, the foundational data structure, and how to manipulate them.
2. Gradient and Autograd
   Understand gradients, automatic differentiation, and how PyTorch handles backpropagation with autograd.
3. Perceptron and Adaline
   Explore the basics of the simplest neural network model (perceptron) and Adaptive Linear Neuron (Adaline).
4. Linear Regression
   Implement linear regression using PyTorch, including model training and prediction.
5. Activation Functions
   Study different activation functions (ReLU, Sigmoid, Tanh, ...) and their roles in neural networks.
6. Loss Functions
   Dive into common loss functions used in neural networks, including MSE, Cross-Entropy, and others.
7. Logistic Regression
   Learn how to build a logistic regression model for binary classification tasks.
8. Working with Datasets
   Understand how to work with datasets in PyTorch using torch.utils.data.Dataset and torch.utils.data.DataLoader.
9. Multi-Layer Perceptron
   Implement and explore multi-layer perceptron (MLP) for more complex tasks.
10. Custom Classes in PyTorch
    Learn how to define custom models, dataset, loss function, ... using PyTorch's class-based approach.
11. Radial Basis Function Networks
    Implement and explore Radial Basis Function (RBF) networks and how they differ from traditional neural networks.
12. Transforms in PyTorch
    Learn to apply transforms like data augmentation on datasets using torchvision.transforms.v2.
13. Convolutional Neural Networks
    Explore concepts around convolutional neural networks (CNNs).
14. Normalizations
    Understand normalization techniques such as Batch Normalization and Layer Normalization.
15. Feature Extraction
    Learn how to extract features from pre-trained models for downstream tasks.
16. Transfer Learning
    Apply transfer learning by using pre-trained models for a new tasks.
17. Fine-Tuning Models
    Understand how to fine-tune models by updating specific layers while freezing others.
18. Save and Load Checkpoints
    Learn how to save and load model checkpoints to resume training or for inference.

📦 Installing Dependencies

You can install all dependencies listed in requirements.txt using pip.

pip install -r requirements.txt

Note: This project was developed using Python v3.12.3. If you encounter issues running the dependencies or code, consider using this specific Python version.

🛠️ Usage

• Open the root folder with VS Code
  • Windows/Linux: Ctrl + K followed by Ctrl + O
  • macOS: Cmd + K followed by Cmd + O
• Open .ipynb files using Jupyter extension integrated with VS Code
• Allow Visual Studio Code to install any recommended dependencies for working with Jupyter Notebooks.
• Jupyter is integrated with both VS Code & Google Colab

🔍 Find Me

Any mistakes, suggestions, or contributions? Feel free to reach out to me at:

• 📍linktr.ee/mr_pylin

I look forward to connecting with you!

📋 Prerequisites

• Programming Fundamentals
  • Proficiency in Python (data types, control structures, functions, etc.).
  • Experience with libraries like NumPy, Pandas and Matplotlib.
• Mathematics for Machine Learning
  • Linear Algebra: Vectors, matrices, matrix operations.
    • Linear Algebra Review and Reference by Zico Kolter
    • Notes on Linear Algebra by Peter J. Cameron
    • MATH 233 - Linear Algebra I Lecture Notes by Cesar O. Aguilar
  • Calculus: Derivatives, gradients, partial derivatives, chain rule (for backpropagation).
    • Lecture notes on advanced gradient descent by Cl´ement W. Royer
    • MATH 221 – CALCULUS LECTURE NOTES VERSION 2.0 by Sigurd Angenent
    • Calculus by Gilbert Strang
  • Probability & Statistics: Probability distributions, mean/variance, etc.
    • MATH1024: Introduction to Probability and Statistics by Sujit Sahu

🔗 Usefull Links

• PyTorch:
  • Source Code
    • Over 3000 contributers are currently working on PyTorch.
    • Link: github.com/pytorch/pytorch
  • Website
    • The official website for PyTorch, offering comprehensive documentation, tutorials, and resources for deep learning and machine learning with PyTorch.
    • Link: pytorch.org
  • Pytorch Documentations
    • Detailed and comprehensive documentation for all PyTorch features and functionalities, including tutorials and guides to help you get started and master PyTorch.
    • Link: pytorch.org/docs/stable/index.html
  • TorchVision Documentations:
    • The torchvision package [part of the PyTorch] consists of popular datasets, model architectures, and common image transformations for computer vision.
    • Link: pytorch.org/vision/stable/index.html
  • TorchAudio Documentation:
    • The torchaudio package [part of the PyTorch] consists of audio I/O and signal processing functionalities, enabling efficient loading, transforming, and manipulating audio.
    • Link: pytorch.org/audio/stable/index.html
• NumPy:
  • Website
    • The official website for NumPy, providing information, tutorials, and resources for the NumPy library
    • Link: numpy.org
  • Documentation
    • Comprehensive guide and reference for all functionalities and features of the NumPy library
    • Link: numpy.org/doc
• MatPlotLib:
  • A comprehensive library for creating static, animated, and interactive visualizations in Python
  • Link: matplotlib.org
• Pandas:
  • A powerful, open-source data analysis and manipulation library for Python
  • Pandas is built on top of NumPy
  • Link: pandas.pydata.org/