lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added new section

Browse Source
This commit is contained in:
mhjensen
2019-01-09 17:38:02 -05:00
parent 48ede08249
commit 623f928cf1
11 changed files with 211 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/Recurrent/html/._Recurrent-bs000.html
+22 -3
View File
@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
'___sec0')]}
'___sec0'),
('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
@@ -67,6 +84,7 @@ end of tocinfo -->
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Recurrent-bs001.html#___sec0" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="._Recurrent-bs002.html#___sec1" style="font-size: 80%;">Set up of an RNN</a></li>
</ul>
</li>
@@ -101,7 +119,7 @@ end of tocinfo -->
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 29, 2018</h4></center> <!-- date -->
<center><h4>Jan 8, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -116,6 +134,7 @@ end of tocinfo -->
<ul class="pagination">
<li class="active"><a href="._Recurrent-bs000.html">1</a></li>
<li><a href="._Recurrent-bs001.html">2</a></li>
<li><a href="._Recurrent-bs002.html">3</a></li>
<li><a href="._Recurrent-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
@@ -133,7 +152,7 @@ end of tocinfo -->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/Recurrent/html/._Recurrent-bs001.html
+26 -6
View File
@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
'___sec0')]}
'___sec0'),
('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
@@ -67,6 +84,7 @@ end of tocinfo -->
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="._Recurrent-bs002.html#___sec1" style="font-size: 80%;">Set up of an RNN</a></li>
</ul>
</li>
@@ -85,16 +103,15 @@ end of tocinfo -->
<h2 id="___sec0" class="anchor">Recurrent neural networks: Overarching view </h2>
<p>
We have mostly looked at feedforward neural networks, where the
Till now our focus has been, including convolutional neural networks as well,
on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
<p>
A recurrent neural network (RNN) looks very much like a feedforward
neural network, except it also has connections pointing
backward. Let&#8217;s look at the simplest possible RNN, composed of just
one neuron receiving inputs, producing an output, and sending that
output back to itself.
neural network, except that it also has connections pointing
backward.
<p>
RNNs are used to analyze time series data such as stock prices, and
@@ -106,12 +123,15 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Recurrent-bs000.html">&laquo;</a></li>
<li><a href="._Recurrent-bs000.html">1</a></li>
<li class="active"><a href="._Recurrent-bs001.html">2</a></li>
<li><a href="._Recurrent-bs002.html">3</a></li>
<li><a href="._Recurrent-bs002.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/Recurrent/html/Recurrent-bs.html
+22 -3
View File
@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
'___sec0')]}
'___sec0'),
('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
@@ -67,6 +84,7 @@ end of tocinfo -->
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Recurrent-bs001.html#___sec0" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="._Recurrent-bs002.html#___sec1" style="font-size: 80%;">Set up of an RNN</a></li>
</ul>
</li>
@@ -101,7 +119,7 @@ end of tocinfo -->
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 29, 2018</h4></center> <!-- date -->
<center><h4>Jan 8, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -116,6 +134,7 @@ end of tocinfo -->
<ul class="pagination">
<li class="active"><a href="._Recurrent-bs000.html">1</a></li>
<li><a href="._Recurrent-bs001.html">2</a></li>
<li><a href="._Recurrent-bs002.html">3</a></li>
<li><a href="._Recurrent-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
@@ -133,7 +152,7 @@ end of tocinfo -->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/Recurrent/html/Recurrent-reveal.html
+38 -7
View File
@@ -107,6 +107,22 @@ td.padding {
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
@@ -132,12 +148,12 @@ td.padding {
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Dec 29, 2018</h4></center> <!-- date -->
<center><h4>Jan 8, 2019</h4></center> <!-- date -->
<br>
<p>
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
</section>
@@ -146,16 +162,15 @@ td.padding {
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
<p>
We have mostly looked at feedforward neural networks, where the
Till now our focus has been, including convolutional neural networks as well,
on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
<p>
A recurrent neural network (RNN) looks very much like a feedforward
neural network, except it also has connections pointing
backward. Let&#8217;s look at the simplest possible RNN, composed of just
one neuron receiving inputs, producing an output, and sending that
output back to itself.
neural network, except that it also has connections pointing
backward.
<p>
RNNs are used to analyze time series data such as stock prices, and
@@ -169,6 +184,22 @@ systems such as automatic translation and speech-to-text.
</section>
<section>
<h2 id="___sec1">Set up of an RNN </h2>
<p>
The figure here displays a simple example of an RNN, with inputs \( x_t \)
at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
offers an intutitive way of understanding these networks. In addition
to the inputs \( x_t \), the layer at a time \( t \) receives also as input
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
<p>
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
</section>
</div> <!-- class="slides" -->
</div> <!-- class="reveal" -->
doc/pub/Recurrent/html/Recurrent-solarized.html
+40 -8
View File
@@ -38,11 +38,28 @@ div { text-align: justify; text-justify: inter-word; }
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
'___sec0')]}
'___sec0'),
('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "AMS" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- ------------------- main content ---------------------- -->
@@ -64,7 +81,7 @@ end of tocinfo -->
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 29, 2018</h4></center> <!-- date -->
<center><h4>Jan 8, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -72,16 +89,15 @@ end of tocinfo -->
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
<p>
We have mostly looked at feedforward neural networks, where the
Till now our focus has been, including convolutional neural networks as well,
on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
<p>
A recurrent neural network (RNN) looks very much like a feedforward
neural network, except it also has connections pointing
backward. Let&#8217;s look at the simplest possible RNN, composed of just
one neuron receiving inputs, producing an output, and sending that
output back to itself.
neural network, except that it also has connections pointing
backward.
<p>
RNNs are used to analyze time series data such as stock prices, and
@@ -93,11 +109,27 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec1">Set up of an RNN </h2>
<p>
The figure here displays a simple example of an RNN, with inputs \( x_t \)
at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
offers an intutitive way of understanding these networks. In addition
to the inputs \( x_t \), the layer at a time \( t \) receives also as input
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
<p>
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
<!-- ------------------- end of main content --------------- -->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/Recurrent/html/Recurrent.html
+40 -8
View File
@@ -43,11 +43,28 @@ div { text-align: justify; text-justify: inter-word; }
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
'___sec0')]}
'___sec0'),
('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "AMS" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- ------------------- main content ---------------------- -->
@@ -69,7 +86,7 @@ end of tocinfo -->
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 29, 2018</h4></center> <!-- date -->
<center><h4>Jan 8, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -77,16 +94,15 @@ end of tocinfo -->
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
<p>
We have mostly looked at feedforward neural networks, where the
Till now our focus has been, including convolutional neural networks as well,
on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
<p>
A recurrent neural network (RNN) looks very much like a feedforward
neural network, except it also has connections pointing
backward. Let&#8217;s look at the simplest possible RNN, composed of just
one neuron receiving inputs, producing an output, and sending that
output back to itself.
neural network, except that it also has connections pointing
backward.
<p>
RNNs are used to analyze time series data such as stock prices, and
@@ -98,11 +114,27 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec1">Set up of an RNN </h2>
<p>
The figure here displays a simple example of an RNN, with inputs \( x_t \)
at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
offers an intutitive way of understanding these networks. In addition
to the inputs \( x_t \), the layer at a time \( t \) receives also as input
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
<p>
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
<!-- ------------------- end of main content --------------- -->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/Recurrent/ipynb/Recurrent.ipynb
+19 -8
View File
@@ -10,24 +10,23 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Dec 29, 2018**\n",
"Date: **Jan 8, 2019**\n",
"\n",
"Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
"\n",
"\n",
"\n",
"## Recurrent neural networks: Overarching view\n",
"\n",
"We have mostly looked at feedforward neural networks, where the\n",
"Till now our focus has been, including convolutional neural networks as well, \n",
"on feedforward neural networks. The output or the\n",
"activations flow only in one direction, from the input layer to the\n",
"output layer.\n",
"\n",
"A recurrent neural network (RNN) looks very much like a feedforward\n",
"neural network, except it also has connections pointing\n",
"backward. Let’s look at the simplest possible RNN, composed of just\n",
"one neuron receiving inputs, producing an output, and sending that\n",
"output back to itself.\n",
"neural network, except that it also has connections pointing\n",
"backward. \n",
"\n",
"RNNs are used to analyze time series data such as stock prices, and\n",
"tell you when to buy or sell. In autonomous driving systems, they can\n",
@@ -36,7 +35,19 @@
"fixed-sized inputs like all the nets we have discussed so far. For\n",
"example, they can take sentences, documents, or audio samples as\n",
"input, making them extremely useful for natural language processing\n",
"systems such as automatic translation and speech-to-text."
"systems such as automatic translation and speech-to-text.\n",
"\n",
"\n",
"## Set up of an RNN\n",
"\n",
"The figure here displays a simple example of an RNN, with inputs $x_t$\n",
"at a given time $t$ and outputs $y_t$. Introducing time as a variable\n",
"offers an intutitive way of understanding these networks. In addition\n",
"to the inputs $x_t$, the layer at a time $t$ receives also as input\n",
"the output from the previous layer $t-1$, that is $y_{t1}$.\n",
"\n",
"This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link\n",
"the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN."
]
}
],
doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/Recurrent/pdf/Recurrent-minted.pdf
BIN
View File
Binary file not shown.
doc/src/Splines/Splines.do.txt
+3
View File
@@ -3,6 +3,9 @@ AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of
DATE: today
!split
===== Optimization problems, why? =====
!split
===== Optimization, the central part of any Machine Learning algortithm =====
doc/src/Statistics/Statistics.do.txt
+1
View File
@@ -3,6 +3,7 @@ AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of
DATE: today
!split
===== Domains and probabilities =====
!bblock