rnn update
This commit is contained in:
mhjensen
@@ -127,7 +127,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Dec 16, 2019</h4></center> <!-- date -->
|
||||
<center><h4>Dec 17, 2019</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
|
||||
|
||||
@@ -111,10 +111,9 @@ MathJax.Hub.Config({
|
||||
<h2 id="___sec0" class="anchor">Recurrent neural networks: Overarching view </h2>
|
||||
|
||||
<p>
|
||||
Till now our focus has been, including convolutional neural networks as well,
|
||||
on feedforward neural networks. The output or the
|
||||
activations flow only in one direction, from the input layer to the
|
||||
output layer.
|
||||
Till now our focus has been, including convolutional neural networks
|
||||
as well, on feedforward neural networks. The output or the activations
|
||||
flow only in one direction, from the input layer to the output layer.
|
||||
|
||||
<p>
|
||||
A recurrent neural network (RNN) looks very much like a feedforward
|
||||
@@ -132,7 +131,7 @@ input, making them extremely useful for natural language processing
|
||||
systems such as automatic translation and speech-to-text.
|
||||
|
||||
<p>
|
||||
<b>The text here is under development</b>. Planned finished Jan 2020.
|
||||
<b>The text here is under development</b>. Planned finished mid Jan 2020.
|
||||
|
||||
<p>
|
||||
<p>
|
||||
|
||||
@@ -118,11 +118,10 @@ to the inputs \( x_t \), the layer at a time \( t \) receives also as input
|
||||
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
|
||||
|
||||
<p>
|
||||
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
|
||||
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
|
||||
|
||||
<p>
|
||||
Add figures and bring up equations.
|
||||
This means also that we need to have weights that link both the inputs
|
||||
\( x_t \) to the outputs \( y_t \) as well as weights that link the output
|
||||
from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an
|
||||
example of a simple RNN.
|
||||
|
||||
<p>
|
||||
<p>
|
||||
|
||||
@@ -127,7 +127,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Dec 16, 2019</h4></center> <!-- date -->
|
||||
<center><h4>Dec 17, 2019</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
|
||||
|
||||
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p> <br>
|
||||
<center><h4>Dec 16, 2019</h4></center> <!-- date -->
|
||||
<center><h4>Dec 17, 2019</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
|
||||
@@ -162,10 +162,9 @@ MathJax.Hub.Config({
|
||||
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
|
||||
|
||||
<p>
|
||||
Till now our focus has been, including convolutional neural networks as well,
|
||||
on feedforward neural networks. The output or the
|
||||
activations flow only in one direction, from the input layer to the
|
||||
output layer.
|
||||
Till now our focus has been, including convolutional neural networks
|
||||
as well, on feedforward neural networks. The output or the activations
|
||||
flow only in one direction, from the input layer to the output layer.
|
||||
|
||||
<p>
|
||||
A recurrent neural network (RNN) looks very much like a feedforward
|
||||
@@ -183,7 +182,7 @@ input, making them extremely useful for natural language processing
|
||||
systems such as automatic translation and speech-to-text.
|
||||
|
||||
<p>
|
||||
<b>The text here is under development</b>. Planned finished Jan 2020.
|
||||
<b>The text here is under development</b>. Planned finished mid Jan 2020.
|
||||
</section>
|
||||
|
||||
|
||||
@@ -198,11 +197,10 @@ to the inputs \( x_t \), the layer at a time \( t \) receives also as input
|
||||
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
|
||||
|
||||
<p>
|
||||
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
|
||||
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
|
||||
|
||||
<p>
|
||||
Add figures and bring up equations.
|
||||
This means also that we need to have weights that link both the inputs
|
||||
\( x_t \) to the outputs \( y_t \) as well as weights that link the output
|
||||
from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an
|
||||
example of a simple RNN.
|
||||
</section>
|
||||
|
||||
|
||||
@@ -210,7 +208,15 @@ Add figures and bring up equations.
|
||||
<h2 id="___sec2">Solving differential equations and eigenvalue problems with RNNs </h2>
|
||||
|
||||
<p>
|
||||
Have example with eigenvalues solvers as well.
|
||||
In our discussions of ordinary differential equations and partial
|
||||
differential equations using neural networks. Here we will discuss how
|
||||
we can solve say ordinary differential equations and eigenvalue
|
||||
problems using RNNs. Eigenvalue problems can be solved using RNNs by
|
||||
rewriting such a problems as a non-linear differential equation.
|
||||
|
||||
<p>
|
||||
Instead of starting with a well-known ordinary differential equation,
|
||||
we start directly with an eigenvaule problem.
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
@@ -87,7 +87,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Dec 16, 2019</h4></center> <!-- date -->
|
||||
<center><h4>Dec 17, 2019</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -95,10 +95,9 @@ MathJax.Hub.Config({
|
||||
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
|
||||
|
||||
<p>
|
||||
Till now our focus has been, including convolutional neural networks as well,
|
||||
on feedforward neural networks. The output or the
|
||||
activations flow only in one direction, from the input layer to the
|
||||
output layer.
|
||||
Till now our focus has been, including convolutional neural networks
|
||||
as well, on feedforward neural networks. The output or the activations
|
||||
flow only in one direction, from the input layer to the output layer.
|
||||
|
||||
<p>
|
||||
A recurrent neural network (RNN) looks very much like a feedforward
|
||||
@@ -116,7 +115,7 @@ input, making them extremely useful for natural language processing
|
||||
systems such as automatic translation and speech-to-text.
|
||||
|
||||
<p>
|
||||
<b>The text here is under development</b>. Planned finished Jan 2020.
|
||||
<b>The text here is under development</b>. Planned finished mid Jan 2020.
|
||||
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -131,11 +130,10 @@ to the inputs \( x_t \), the layer at a time \( t \) receives also as input
|
||||
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
|
||||
|
||||
<p>
|
||||
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
|
||||
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
|
||||
|
||||
<p>
|
||||
Add figures and bring up equations.
|
||||
This means also that we need to have weights that link both the inputs
|
||||
\( x_t \) to the outputs \( y_t \) as well as weights that link the output
|
||||
from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an
|
||||
example of a simple RNN.
|
||||
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -143,7 +141,15 @@ Add figures and bring up equations.
|
||||
<h2 id="___sec2">Solving differential equations and eigenvalue problems with RNNs </h2>
|
||||
|
||||
<p>
|
||||
Have example with eigenvalues solvers as well.
|
||||
In our discussions of ordinary differential equations and partial
|
||||
differential equations using neural networks. Here we will discuss how
|
||||
we can solve say ordinary differential equations and eigenvalue
|
||||
problems using RNNs. Eigenvalue problems can be solved using RNNs by
|
||||
rewriting such a problems as a non-linear differential equation.
|
||||
|
||||
<p>
|
||||
Instead of starting with a well-known ordinary differential equation,
|
||||
we start directly with an eigenvaule problem.
|
||||
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
|
||||
@@ -92,7 +92,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Dec 16, 2019</h4></center> <!-- date -->
|
||||
<center><h4>Dec 17, 2019</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -100,10 +100,9 @@ MathJax.Hub.Config({
|
||||
<h2 id="___sec0">Recurrent neural networks: Overarching view </h2>
|
||||
|
||||
<p>
|
||||
Till now our focus has been, including convolutional neural networks as well,
|
||||
on feedforward neural networks. The output or the
|
||||
activations flow only in one direction, from the input layer to the
|
||||
output layer.
|
||||
Till now our focus has been, including convolutional neural networks
|
||||
as well, on feedforward neural networks. The output or the activations
|
||||
flow only in one direction, from the input layer to the output layer.
|
||||
|
||||
<p>
|
||||
A recurrent neural network (RNN) looks very much like a feedforward
|
||||
@@ -121,7 +120,7 @@ input, making them extremely useful for natural language processing
|
||||
systems such as automatic translation and speech-to-text.
|
||||
|
||||
<p>
|
||||
<b>The text here is under development</b>. Planned finished Jan 2020.
|
||||
<b>The text here is under development</b>. Planned finished mid Jan 2020.
|
||||
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -136,11 +135,10 @@ to the inputs \( x_t \), the layer at a time \( t \) receives also as input
|
||||
the output from the previous layer \( t-1 \), that is \( y_{t1} \).
|
||||
|
||||
<p>
|
||||
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
|
||||
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
|
||||
|
||||
<p>
|
||||
Add figures and bring up equations.
|
||||
This means also that we need to have weights that link both the inputs
|
||||
\( x_t \) to the outputs \( y_t \) as well as weights that link the output
|
||||
from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an
|
||||
example of a simple RNN.
|
||||
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -148,7 +146,15 @@ Add figures and bring up equations.
|
||||
<h2 id="___sec2">Solving differential equations and eigenvalue problems with RNNs </h2>
|
||||
|
||||
<p>
|
||||
Have example with eigenvalues solvers as well.
|
||||
In our discussions of ordinary differential equations and partial
|
||||
differential equations using neural networks. Here we will discuss how
|
||||
we can solve say ordinary differential equations and eigenvalue
|
||||
problems using RNNs. Eigenvalue problems can be solved using RNNs by
|
||||
rewriting such a problems as a non-linear differential equation.
|
||||
|
||||
<p>
|
||||
Instead of starting with a well-known ordinary differential equation,
|
||||
we start directly with an eigenvaule problem.
|
||||
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
"<!-- Author: --> \n",
|
||||
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
|
||||
"\n",
|
||||
"Date: **Dec 16, 2019**\n",
|
||||
"Date: **Dec 17, 2019**\n",
|
||||
"\n",
|
||||
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
|
||||
"\n",
|
||||
@@ -19,10 +19,9 @@
|
||||
"\n",
|
||||
"## Recurrent neural networks: Overarching view\n",
|
||||
"\n",
|
||||
"Till now our focus has been, including convolutional neural networks as well, \n",
|
||||
"on feedforward neural networks. The output or the\n",
|
||||
"activations flow only in one direction, from the input layer to the\n",
|
||||
"output layer.\n",
|
||||
"Till now our focus has been, including convolutional neural networks\n",
|
||||
"as well, on feedforward neural networks. The output or the activations\n",
|
||||
"flow only in one direction, from the input layer to the output layer.\n",
|
||||
"\n",
|
||||
"A recurrent neural network (RNN) looks very much like a feedforward\n",
|
||||
"neural network, except that it also has connections pointing\n",
|
||||
@@ -37,7 +36,7 @@
|
||||
"input, making them extremely useful for natural language processing\n",
|
||||
"systems such as automatic translation and speech-to-text.\n",
|
||||
"\n",
|
||||
"**The text here is under development**. Planned finished Jan 2020.\n",
|
||||
"**The text here is under development**. Planned finished mid Jan 2020.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Set up of an RNN\n",
|
||||
@@ -48,17 +47,26 @@
|
||||
"to the inputs $x_t$, the layer at a time $t$ receives also as input\n",
|
||||
"the output from the previous layer $t-1$, that is $y_{t1}$.\n",
|
||||
"\n",
|
||||
"This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link\n",
|
||||
"the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Add figures and bring up equations. \n",
|
||||
"This means also that we need to have weights that link both the inputs\n",
|
||||
"$x_t$ to the outputs $y_t$ as well as weights that link the output\n",
|
||||
"from the previous time $y_{t-1}$ and $y_t$. The figure here shows an\n",
|
||||
"example of a simple RNN.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Solving differential equations and eigenvalue problems with RNNs\n",
|
||||
"\n",
|
||||
"Have example with eigenvalues solvers as well.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"In our discussions of ordinary differential equations and partial\n",
|
||||
"differential equations using neural networks. Here we will discuss how\n",
|
||||
"we can solve say ordinary differential equations and eigenvalue\n",
|
||||
"problems using RNNs. Eigenvalue problems can be solved using RNNs by\n",
|
||||
"rewriting such a problems as a non-linear differential equation.\n",
|
||||
"\n",
|
||||
"Instead of starting with a well-known ordinary differential equation,\n",
|
||||
"we start directly with an eigenvaule problem.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Long-Short Time Memory\n",
|
||||
|
||||
Binary file not shown.
Binary file not shown.
@@ -6,10 +6,9 @@ DATE: today
|
||||
!split
|
||||
===== Recurrent neural networks: Overarching view =====
|
||||
|
||||
Till now our focus has been, including convolutional neural networks as well,
|
||||
on feedforward neural networks. The output or the
|
||||
activations flow only in one direction, from the input layer to the
|
||||
output layer.
|
||||
Till now our focus has been, including convolutional neural networks
|
||||
as well, on feedforward neural networks. The output or the activations
|
||||
flow only in one direction, from the input layer to the output layer.
|
||||
|
||||
A recurrent neural network (RNN) looks very much like a feedforward
|
||||
neural network, except that it also has connections pointing
|
||||
@@ -24,7 +23,7 @@ example, they can take sentences, documents, or audio samples as
|
||||
input, making them extremely useful for natural language processing
|
||||
systems such as automatic translation and speech-to-text.
|
||||
|
||||
_The text here is under development_. Planned finished Jan 2020.
|
||||
_The text here is under development_. Planned finished mid Jan 2020.
|
||||
|
||||
|
||||
!split
|
||||
@@ -36,18 +35,27 @@ offers an intutitive way of understanding these networks. In addition
|
||||
to the inputs $x_t$, the layer at a time $t$ receives also as input
|
||||
the output from the previous layer $t-1$, that is $y_{t1}$.
|
||||
|
||||
This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link
|
||||
the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN.
|
||||
|
||||
|
||||
Add figures and bring up equations.
|
||||
This means also that we need to have weights that link both the inputs
|
||||
$x_t$ to the outputs $y_t$ as well as weights that link the output
|
||||
from the previous time $y_{t-1}$ and $y_t$. The figure here shows an
|
||||
example of a simple RNN.
|
||||
|
||||
|
||||
|
||||
!split
|
||||
===== Solving differential equations and eigenvalue problems with RNNs =====
|
||||
|
||||
Have example with eigenvalues solvers as well.
|
||||
|
||||
|
||||
In our discussions of ordinary differential equations and partial
|
||||
differential equations using neural networks. Here we will discuss how
|
||||
we can solve say ordinary differential equations and eigenvalue
|
||||
problems using RNNs. Eigenvalue problems can be solved using RNNs by
|
||||
rewriting such a problems as a non-linear differential equation.
|
||||
|
||||
Instead of starting with a well-known ordinary differential equation,
|
||||
we start directly with an eigenvaule problem.
|
||||
|
||||
|
||||
|
||||
!split
|
||||
|
||||
Reference in New Issue
Block a user