techzzt/codelab

DSBA code study

Table of Contents

• CodeLab
• Computer Vision
• Natural Language Processing
• Tabular Data
• Time-Series
• Reinforcement Learning
• Audio Data
• Multi-modality
• Extra
• Pytorch Accelerator
• Members

CodeLab

DSBA code study (2021.03.04 ~ )

Computer Vision

Classification

• Model Soup [Jaehyuk Heo]
• Point cloud classification with PointNet [Hyeongwon Kang]
• Involutional neural networks [Subin Kim]
• Image classification with Vision Transformer [Jaehyuk Heo]
• Video Classification with Transformers + Video Vision Transformer [Hyeongwon Kang]

Self-Supervised Learning

• Semi-supervised image classification using contrastive pretraining with SimCLR [Subin Kim]
• Self-supervised contrastive learning with SimSiam [Jaehyuk Heo]
• Supervised Contrastive Learning [Subin Kim]

Image Denoising

• Convolutional autoencoder for image denoising [Jeongseob Kim]

Segmentation

• Point cloud segmentation with PointNet [Hyeongwon Kang]
• Image segmentation with a U-Net-like architecture [Jeongseob Kim]

Object Detection

• Object Detection with RetinaNet [Jaehyuk Heo]

Knowledge Distillation

• Knowledge Distillation [Jaehyuk Heo]

Retrieval

• Metric learning for image similarity search [Jaehyuk Heo]
• Image similarity estimation using a Siamese Network with a triplet loss [Yonggi Jeong]

OCR

• OCR model for reading Captchas [Subin Kim]

Augmentation

• RandAugment for Image Classification for Improved Robustness [Yonggi Jeong]
• CutMix data augmentation for image classification [Jaehyuk Heo]

Clustering

• Semantic Image Clustering [Yonggi Jeong]

Depth Estimation

• Monocular depth estimation [Hyeongwon Kang]

Attribution Methods

• Grad-CAM class activation visualization [Jaehyuk Heo]
• Model interpretability with Integrated Gradients [Jaehyuk Heo]
• Visualizing what convnets learn [Jaehyuk Heo]

Optimizer

• Gradient Centralization for Better Training Performance [Jaehyuk Heo]

Adepter

• Finetuning ViT with LoRA [Jaehyuk Heo]

Generative Models

• Variational AutoEncoder [Jaehyuk Heo]
• DCGAN to generate face images [Hyeongwon Kang]
• Neural style transfer [Subin Kim]
• Deep Dream [Jaehyuk Heo]
• Conditional GAN [Yonggi Jeong]
• CycleGAN [Yonggi Jeong]
• PixelCNN [Jeongseob Kim]
• Density estimation using Real NVP [Jeongseob Kim]
• Non-linear Independent Component Estimation (NICE) [Jeongseob Kim]
• Diffusion generative model(Tutorials) [Jeongseob Kim]
• Diffusion generative model(Examples - Swiss-roll, MNIST, F-MNIST, CELEBA) [Jeongseob Kim]
• Score based generative model(Tutorials) [Jeongseob Kim]

Adversarial Attacks

• Fast Gradient Sign Method [Jaehyuk Heo]
• Projected Gradient Descent [Jaehyuk Heo]

Adversarial Detection

• Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domain [Jaehyuk Heo]

Anomaly Detection

• PatchCore: Towards Total Recall in Industrial Anomaly Detection [Jaehyuk Heo]
• MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities [Jaehyuk Heo]

Natural Language Processing

Classification

• Text classification with Switch Transformer [Subin Kim]
• Text classification with Transformer [Yookyung Kho]
• Bidirectional LSTM on IMDB [Jeongseob Kim]

Generation

• Text generation with a miniature GPT [Subin Kim]
• Sequence to sequence learning for performing number addition [Yookyung Kho]
• Character-level recurrent sequence-to-sequence model [Jeongseob Kim]
• English-to-Spanish translation with a sequence-to-sequence Transformer [Yookyung Kho]

Question Answering

• Question Answering with Hugging Face Transformers [Yookyung Kho]
• Text Extraction with BERT [Jaehyuk Heo]

Pretrained Language Model

• End-to-end Masked Language Modeling with BERT [Subin Kim]

Named Entity Recognition

• Named Entity Recognition using Transformers [Subin Kim]

Natural Language Inference

• Semantic Similarity with BERT [Jaehyuk Heo]

Table MRC

• Table Pre-training with TapasForMaskedLM [Yookyung Kho]

Tutorial

• TorchText introduction [Jeongseob Kim]

Tabular Data

Classification

• Classification with Gated Residual and Variable Selection Networks [Hyeongwon Kang]
• Structured data learning with TabTransformer [Hyeongwon Kang]

Recommendation

• Collaborative Filtering for Movie Recommendations [Hyeongwon Kang]
• A Transformer-based recommendation system [Hyeongwon Kang]

Anomaly Detection

• Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection [Sunwoo Kim]

Time-Series

Anomaly Detection

• Timeseries anomaly detection using an Autoencoder [Hyeongwon Kang]

Classification

• Timeseries classification with a Transformer model [Hyeongwon Kang]

Forecasting

• Timeseries forecasting for weather prediction [Hyeongwon Kang]

Reinforcement Learning

• Actor Critic Method [Hyeongwon Kang]
• Deep Deterministic Policy Gradient (DDPG) [Hyeongwon Kang]
• Deep Q-Learning for Atari Breakout [Hyeongwon Kang]
• Proximal Policy Optimization [Hyeongwon Kang]

Audio Data

Recognition

• Speaker Recognition [Subin Kim]

Multi-modality

Vision-Langauge

• Multimodal entailment [Yookyung Kho]
• Natural language image search with a Dual Encoder [Subin Kim]

Extra

• Distributions_TFP_Pyro [Jeongseob Kim]

Pytorch Accelerator

• Huggingface Accelerator [Jaehyuk Heo]
• Automatic Mixed Precision [Jaehyuk Heo]
• Gradient Accumulation [Jaehyuk Heo]
• Distributed Data Parallel [Jaehyuk Heo]

Members

Imhun (2022.8 ~ )	Jinwoo (2022.8 ~ )	SeongHee (2022.8 ~ )	SaeRan (2022.7 ~ )
Github	Github	Github	Github

Sunwoo (2022.5 ~ )	Jaehyuk (2021.3 ~ )	Jeongsub (2021.3 ~ 2022.12)	Suzie (2022.5 ~ 2022.11)
Github	Github	Github	Github

Yookyung (2021.10 ~ 2022.7)	Yonggi (2021.10 ~ 2022.3)	Hyeongwon (2021.3 ~ 2022.3)	Subin (2021.3 ~ 2022.2)
Github	Github	Github	Github