lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added new section

Browse Source
This commit is contained in:
mhjensen
2019-01-09 17:38:02 -05:00
parent 48ede08249
commit 623f928cf1
11 changed files with 211 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/Recurrent/html/Recurrent.html
+40 -8
View File
@@ -43,11 +43,28 @@ div { text-align: justify; text-justify: inter-word; }
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
'___sec0')]}
'___sec0'),
('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "AMS" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- ------------------- main content ---------------------- -->
@@ -69,7 +86,7 @@ end of tocinfo -->
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 29, 2018</h4></center> <!-- date -->
<center><h4>Jan 8, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -77,16 +94,15 @@ end of tocinfo -->
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
<p>
We have mostly looked at feedforward neural networks, where the
Till now our focus has been, including convolutional neural networks as well,
on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
<p>
A recurrent neural network (RNN) looks very much like a feedforward
neural network, except it also has connections pointing
backward. Let&#8217;s look at the simplest possible RNN, composed of just
one neuron receiving inputs, producing an output, and sending that
output back to itself.
neural network, except that it also has connections pointing
backward.
<p>
RNNs are used to analyze time series data such as stock prices, and
@@ -98,11 +114,27 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec1">Set up of an RNN </h2>
<p>
The figure here displays a simple example of an RNN, with inputs \( x_t \)
at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
offers an intutitive way of understanding these networks. In addition
to the inputs \( x_t \), the layer at a time \( t \) receives also as input
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
<p>
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
<!-- ------------------- end of main content --------------- -->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>