Real-world high dimensional data (such as images) lie on low-dimensional manifolds embedded in the high-dimensional space.
Chapter 14.6 Deep Learning Book
Like many other machine learning algorithms, autoencoders exploit the idea that data concentrates around a low-dimentional manifold or a small set of such manifolds. Some machine learning algorithms exploit this ideal only insofar as they learn a function that behaves correctly on the manifold but they may have unusual behaviour if given an input that is off the manifold. Autoencoders take this idea further and aim to learn the structure of the manifold.
Geometric Understanding of Deep Learning
Deep learning learns the manifold and the probability distribution on it.
Baloo’s song [The Jungle Book]
Look for the bare necessities
The simple bare necessities
Forget about your worries and your strife
I mean the bare necessities
Old Mother Nature’s recipes
That bring the bare necessities of life