Autoencoders

Mitesh M. Khapra

1. Lecture PDF
2. Introduction to Autoencoders (Video)
3. Link between PCA and Autoencoders (Video)
4. Regularization in autoencoders (Motivation) (Video)
5. Denoising Autoencoders (Video)
6. Sparse Autoencoders (Video)
7. Contractive autoencoders (Video)

Hugo Larochelle

1. Autoencoder – definition (Video)
2. Autoencoder – loss function (Video)
3. Example (Video)
4. Linear Autoencoder (Video)
5. Autoencoder – undercomplete vs. overcomplete hidden layer (Video)
6. Autoencoder – denoising autoencoder (Video)
7. Autoencoder – contractive autoencoder (Video)

Papers

1. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion – Pascal Vincent et al. – PDF
2. A Connection Between Score Matching and Denoising Autoencoders – Pascal Vincent – PDF
3. Sparse autoencoder – CS294A Lecture notes – Andrew Ng – PDF
4. Contractive Auto-Encoders: Explicit Invariance During Feature – PDF
5. Auto-Association by Multilayer Perceptrons and Singular Value Decomposition by Hervé Bourlard and Yves Kamp – PDF
6. Deep Learning using Robust Interdependent Codes by Hugo Larochelle, Dumitru Erhan and Pascal Vincent – PDF
7. Semi-supervised Learning of Compact Document Representations with Deep Networks by Marc’Aurelio Ranzato and Martin Szummer – PDF – Video
8. Large-Scale Learning of Embeddings with Reconstruction Sampling by Yann Dauphin, Xavier Glorot and Yoshua Bengio – PDF – Video
9. On Nonparametric Guidance for Learning Autoencoder Representations by Jasper Snoek, Ryan Adams and Hugo Larochelle – PDF
10. Gradient-based learning of higher-order image features by Roland Memisevic – PDF
11. Manifolds

Books

1. Autoencoders – Chapter – 14 – Ian Goodfellow et al. – Link

Examples / Exercises

1. Linear Autoencoder (Pytorch, MNIST Handwritten Digits) – Link
2. Implementing Deep Autoencoder in PyTorch (Pytorch, Fashion MNIST) – Link
3. Denoising Autoencoder (Pytorch, MNIST Handwritten Digits) – Link
4. Denoising Autoencoder (Pytorch, Fashion MNIST) – Link
5. Denoising Text Image Documents using Autoencoders – Link1 – Link2
6. Convolutional Autoencoder (Pytorch, MNIST Handwritten Digits) – Link
7. Convolutional Autoencoder (Pytorch, CIFAR10) – Link
8. Sparse Autoencoders using L1 Regularization with PyTorch – Link
9. Auto-encoder – Collaborative Filtering – Predicting the Rating a User would give a Movie – Link1, Link2, Link3

PPTs

1. Autoencoders by CloudxLab – Link