keras2torch Study

1. 스터디 소개

Keras documentaion에 올라온 코드를 Pytorch로 코드 이전하는 스터디 입니다.

keras documentaion examples

Original github address: https://github.com/keras-team/keras-io

참여자: 강형원, 김수빈, 김정섭, 정용기, 허재혁, 고유경

시작 일자: 2021.03.04(목)

---

Members

Hyeongwon	Subin	Jeongsub	Yonggi	Jaehyuk	Yookyung
Github	Github	Github	Github	Github	Github

---

목록

Computer Vision

• Image segmentation with a U-Net-like architecture
• 3D image classification from CT scans
• Semi-supervision and domain adaptation with AdaMatch
• Classification using Attention-based Deep Multiple Instance Learning (MIL).
• Convolutional autoencoder for image denoising
• Barlow Twins for Contrastive SSL
• Image Classification using BigTransfer (BiT)
• OCR model for reading Captchas
• Compact Convolutional Transformers
• Consistency training with supervision
• Next-Frame Video Prediction with Convolutional
• Image classification with ConvMixer
• CutMix data augmentation for image classification
• Multiclass semantic segmentation using DeepLabV3+
• Monocular depth estimation
• Image classification with EANet (External Attention Transformer)
• FixRes: Fixing train-test resolution discrepancy
• Grad-CAM class activation visualization
• Gradient Centralization for Better Training Performance
• Handwriting recognition
• Image Captioning
• Image classification via fine-tuning with EfficientNet
• Image classification with Vision Transformer
• Model interpretability with Integrated Gradients
• Involutional neural networks
• Keypoint Detection with Transfer Learning
• Knowledge Distillation
• Learning to Resize in Computer Vision
• Masked image modeling with Autoencoders
• Metric learning for image similarity search
• Low-light image enhancement using MIRNet
• MixUp augmentation for image classification
• Image classification with modern MLP models
• MobileViT: A mobile-friendly Transformer-based model for image classification
• Near-duplicate image search
• 3D volumetric rendering with NeRF
• Self-supervised contrastive learning with NNCLR
• Augmenting convnets with aggregated attention
• Image classification with Perceiver
• Point cloud classification with PointNet
• Point cloud segmentation with PointNet
• RandAugment for Image Classification for Improved Robustness
• Few-Shot learning with Reptile
• Object Detection with RetinaNet
• Semantic Image Clustering
• Semi-supervised image classification using contrastive pretraining with SimCLR
• Image similarity estimation using a Siamese Network with a contrastive loss
• Image similarity estimation using a Siamese Network with a triplet loss
• Self-supervised contrastive learning with SimSiam
• Image Super-Resolution using an Efficient Sub-Pixel CNN
• Supervised Contrastive Learning
• Image classification with Swin Transformers
• Learning to tokenize in Vision Transformers
• Video Classification with Transformers + Video Vision Transformer
• Visualizing what convnets learn
• Train a Vision Transformer on small datasets
• Zero-DCE for low-light image enhancement

Natural Language Processing

• Review Classification using Active Learning
• Sequence to sequence learning for performing number addition
• Bidirectional LSTM on IMDB
• Character-level recurrent sequence-to-sequence model
• End-to-end Masked Language Modeling with BERT
• Large-scale multi-label text classification
• Multimodal entailment
• Named Entity Recognition using Transformers
• English-to-Spanish translation with a sequence-to-sequence Transformer
• Natural language image search with a Dual Encoder
• Using pre-trained word embeddings
• Question Answering with Hugging Face Transformers
• Semantic Similarity with BERT
• Text classification with Switch Transformer
• Text classification with Transformer
• Text Extraction with BERT
• Text Generation using FNet

Extra

• TorchText introduction

Structured Data

• Classification with Gated Residual and Variable Selection Networks
• Collaborative Filtering for Movie Recommendations
• Classification with Neural Decision Forests
• Imbalanced classification: credit card fraud detection
• A Transformer-based recommendation system
• Structured data learning with TabTransformer
• Structured data learning with Wide, Deep, and Cross networks

Timeseries

• Timeseries anomaly detection using an Autoencoder
• Timeseries classification with a Transformer model
• Traffic forecasting using graph neural networks and LSTM
• Timeseries forecasting for weather prediction

Audio Data

• Automatic Speech Recognition using CTC
• MelGAN-based spectrogram inversion using feature matching
• Speaker Recognition
• Automatic Speech Recognition with Transformer

Generative Deep Learning

• Variational AutoEncoder
• DCGAN to generate face images
• WGAN-GP overriding Model.train_step
• Neural style transfer
• Deep Dream
• Neural Style Transfer with AdaIN
• Conditional GAN
• CycleGAN
• Data-efficient GANs with Adaptive Discriminator Augmentation
• GauGAN for conditional image generation
• Character-level text generation with LSTM
• PixelCNN
• Density estimation using Real NVP
• Face image generation with StyleGAN
• Text generation with a miniature GPT
• Vector-Quantized Variational Autoencoders
• WGAN-GP with R-GCN for the generation of small molecular graphs

Extra

• Distributions_TFP_Pyro
• Non-linear Independent Component Estimation (NICE)

Reinforcement Learning

• Actor Critic Method
• Deep Deterministic Policy Gradient (DDPG)
• Deep Q-Learning for Atari Breakout
• Proximal Policy Optimization

Graph Data

• Graph attention networks for node classification
• Node Classification with Graph Neural Networks
• Message-passing neural network for molecular property prediction
• Graph representation learning with node2vec

Adversarial Attacks

• Fast Gradient Sign Method
• Projected Gradient Descent

Pytorch Accelerator

• Automatic Mixed Precision
• Gradient Accumulation
• Distributed Data Parallel