L0 Norm, L1 Norm, L2 Norm & L-Infinity Norm
Data vs. Model Parallelism
Dictionary learning for sparse representation – Link1
Entropy: It is the measure of the disorder / uncertainty / randomness in the information being processed. Our goal normally in machine learning is normally to reduce this. Mathematical formula of Entropy (sometime also represented as H):
Curse of dimensionality: Curse of dimensionality phenomena occurs in many domains inculding machine learning. The common theme of these problems is that when the dimensionality increases, the volume of the space increases so fast that the available data become sparse. This sparsity is problematic for any method that requires statistical significance. In order to obtain a statistically sound and reliable result, the amount of data needed to support the result often grows exponentially with the dimensionality.