Morten Hjorth-Jensen
42 lines
1.9 KiB
Plaintext
42 lines
1.9 KiB
Plaintext
TITLE: Data Analysis and Machine Learning: Autoencoders
|
||
AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
|
||
DATE: today
|
||
|
||
!split
|
||
===== Tod do =====
|
||
|
||
* make example link with pca
|
||
* develop examples and add more text
|
||
* excellent website http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/
|
||
|
||
|
||
!split
|
||
===== Autoencoders: Overarching view =====
|
||
|
||
Autoencoders are artificial neural networks capable of learning
|
||
efficient representations of the input data (these representations are called codings) without
|
||
any supervision (i.e., the training set is unlabeled). These codings
|
||
typically have a much lower dimensionality than the input data, making
|
||
autoencoders useful for dimensionality reduction.
|
||
|
||
More importantly, autoencoders act as powerful feature detectors, and
|
||
they can be used for unsupervised pretraining of deep neural networks.
|
||
|
||
Lastly, they are capable of randomly generating new data that looks
|
||
very similar to the training data; this is called a generative
|
||
model. For example, you could train an autoencoder on pictures of
|
||
faces, and it would then be able to generate new faces. Surprisingly,
|
||
autoencoders work by simply learning to copy their inputs to their
|
||
outputs. This may sound like a trivial task, but we will see that
|
||
constraining the network in various ways can make it rather
|
||
difficult. For example, you can limit the size of the internal
|
||
representation, or you can add noise to the inputs and train the
|
||
network to recover the original inputs. These constraints prevent the
|
||
autoencoder from trivially copying the inputs directly to the outputs,
|
||
which forces it to learn efficient ways of representing the data. In
|
||
short, the codings are byproducts of the autoencoder’s attempt to
|
||
learn the identity function under some constraints.
|
||
|
||
!split
|
||
===== Simple examples of Autoencoders =====
|