mhjensen
121 lines
3.8 KiB
HTML
121 lines
3.8 KiB
HTML
<!--
|
|
Automatically generated HTML file from DocOnce source
|
|
(https://github.com/hplgit/doconce/)
|
|
-->
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
|
<meta name="description" content="Data Analysis and Machine Learning: Autoencoders">
|
|
|
|
<title>Data Analysis and Machine Learning: Autoencoders</title>
|
|
|
|
|
|
<style type="text/css">
|
|
/* bloodish style */
|
|
|
|
body {
|
|
font-family: Helvetica, Verdana, Arial, Sans-serif;
|
|
color: #404040;
|
|
background: #ffffff;
|
|
}
|
|
h1 { font-size: 1.8em; color: #8A0808; }
|
|
h2 { font-size: 1.6em; color: #8A0808; }
|
|
h3 { font-size: 1.4em; color: #8A0808; }
|
|
h4 { color: #8A0808; }
|
|
a { color: #8A0808; text-decoration:none; }
|
|
tt { font-family: "Courier New", Courier; }
|
|
/* pre style removed because it will interfer with pygments */
|
|
p { text-indent: 0px; }
|
|
hr { border: 0; width: 80%; border-bottom: 1px solid #aaa}
|
|
p.caption { width: 80%; font-style: normal; text-align: left; }
|
|
hr.figure { border: 0; width: 80%; border-bottom: 1px solid #aaa}
|
|
|
|
div { text-align: justify; text-justify: inter-word; }
|
|
</style>
|
|
|
|
|
|
</head>
|
|
|
|
<!-- tocinfo
|
|
{'highest level': 2,
|
|
'sections': [('Autoencoders: Overarching view', 2, None, '___sec0'),
|
|
('Simple examples of Autoencoders', 2, None, '___sec1')]}
|
|
end of tocinfo -->
|
|
|
|
<body>
|
|
|
|
|
|
<!-- ------------------- main content ---------------------- -->
|
|
|
|
|
|
|
|
<center><h1>Data Analysis and Machine Learning: Autoencoders</h1></center> <!-- document title -->
|
|
|
|
<p>
|
|
<!-- author(s): Morten Hjorth-Jensen -->
|
|
|
|
<center>
|
|
<b>Morten Hjorth-Jensen</b> [1, 2]
|
|
</center>
|
|
|
|
<p>
|
|
<!-- institution(s) -->
|
|
|
|
<center>[1] <b>Department of Physics, University of Oslo</b></center>
|
|
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
|
<br>
|
|
<p>
|
|
<center><h4>Jan 1, 2019</h4></center> <!-- date -->
|
|
<br>
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec0">Autoencoders: Overarching view </h2>
|
|
|
|
<p>
|
|
Autoencoders are artificial neural networks capable of learning
|
|
efficient representations of the input data (these representations are called codings) without
|
|
any supervision (i.e., the training set is unlabeled). These codings
|
|
typically have a much lower dimensionality than the input data, making
|
|
autoencoders useful for dimensionality reduction.
|
|
|
|
<p>
|
|
More importantly, autoencoders act as powerful feature detectors, and
|
|
they can be used for unsupervised pretraining of deep neural networks.
|
|
|
|
<p>
|
|
Lastly, they are capable of randomly generating new data that looks
|
|
very similar to the training data; this is called a generative
|
|
model. For example, you could train an autoencoder on pictures of
|
|
faces, and it would then be able to generate new faces. Surprisingly,
|
|
autoencoders work by simply learning to copy their inputs to their
|
|
outputs. This may sound like a trivial task, but we will see that
|
|
constraining the network in various ways can make it rather
|
|
difficult. For example, you can limit the size of the internal
|
|
representation, or you can add noise to the inputs and train the
|
|
network to recover the original inputs. These constraints prevent the
|
|
autoencoder from trivially copying the inputs directly to the outputs,
|
|
which forces it to learn efficient ways of representing the data. In
|
|
short, the codings are byproducts of the autoencoder’s attempt to
|
|
learn the identity function under some constraints.
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec1">Simple examples of Autoencoders </h2>
|
|
|
|
<!-- ------------------- end of main content --------------- -->
|
|
|
|
|
|
<center style="font-size:80%">
|
|
<!-- copyright --> © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
|
|
</center>
|
|
|
|
|
|
</body>
|
|
</html>
|
|
|
|
|