Deep Learning on the Edge

  1. How do we Optimize Deep Learning models for edge devices?
  2. Training a single AI model can emit as much carbon as five cars in their lifetimes
  1. Parameter Removal: Identify the trainable layers in the model and reduce the number of parameters!
    1. Weight Pruning: Specific connections between neurons are removed. In practice this removal means replacing the weight by zero. Will this removal not affect the knowledge learned by the model? Do this removal until it does not decrease the accuracy much (acceptable level).
    2. Optimal Brain Damage: Yann LeCun et. al [5] [6] proposed the following:
      1. Train a network.
      2. Estimate the importance of each weight by watching how the loss would change upon perturbing the weight. Smaller change means less importance. (This importance is called the saliency.)
      3. Remove the weights with low importance.
      4. Go back to Step 1. and retrain the network, permanently fixing the removed weights to zero.
    3. Lottery Ticket Hypothesis (Published as a conference paper at ICLR 2019)
    4. SynFlow (Published in NeurIPS 2020)
    5. Pruning in PyTorch tutorial
  1. Structured Vs. Unstructured Pruning

Important Links

  1. Pruning Convolutional Neural Networks for Resource Efficient Inference by Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, Jan Kautz, published in Published as a conference paper at ICLR 2017
  2. https://github.com/jacobgil/pytorch-pruning
  3. https://jacobgil.github.io/deeplearning/pruning-deep-learning
  4. https://www.deeplite.ai/
  5. Optimal Brain Damage
  6. How to compress a neural network