-
@@ -135,6 +143,8 @@ MathJax.Hub.Config({
+The text here is under development. Planned finished Jan 2020. +
@@ -131,6 +142,8 @@ systems such as automatic translation and speech-to-text.
+Add figures and bring up equations. + +
-
@@ -135,6 +143,8 @@ MathJax.Hub.Config({
-
@@ -181,6 +181,9 @@ fixed-sized inputs like all the nets we have discussed so far. For example, they can take sentences, documents, or audio samples as input, making them extremely useful for natural language processing systems such as automatic translation and speech-to-text. + +
+The text here is under development. Planned finished Jan 2020. @@ -197,6 +200,25 @@ the output from the previous layer \( t-1 \), that is \( y_{t1} \).
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN. + +
+Add figures and bring up equations.
+
+
+
+
+Have example with eigenvalues solvers as well.
+
+Discussions about dynamic unrolling through time. discuss memory cells, input and output
Solving differential equations and eigenvalue problems with RNNs
+
+Long-Short Time Memory
+
+
-
@@ -109,6 +115,9 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
+
+The text here is under development. Planned finished Jan 2020. +
@@ -125,6 +134,25 @@ the output from the previous layer \( t-1 \), that is \( y_{t1} \).
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
+
+Add figures and bring up equations. + +
+
+
+
+Have example with eigenvalues solvers as well. + +
+
+
+
+Discussions about dynamic unrolling through time. discuss memory cells, input and output + diff --git a/doc/pub/Recurrent/html/Recurrent.html b/doc/pub/Recurrent/html/Recurrent.html index 55e404e12..2f9b30def 100644 --- a/doc/pub/Recurrent/html/Recurrent.html +++ b/doc/pub/Recurrent/html/Recurrent.html @@ -44,7 +44,13 @@ div { text-align: justify; text-justify: inter-word; } 2, None, '___sec0'), - ('Set up of an RNN', 2, None, '___sec1')]} + ('Set up of an RNN', 2, None, '___sec1'), + ('Solving differential equations and eigenvalue problems with ' + 'RNNs', + 2, + None, + '___sec2'), + ('Long-Short Time Memory', 2, None, '___sec3')]} end of tocinfo -->
@@ -86,7 +92,7 @@ MathJax.Hub.Config({-
@@ -114,6 +120,9 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
+
+The text here is under development. Planned finished Jan 2020. +
@@ -130,6 +139,25 @@ the output from the previous layer \( t-1 \), that is \( y_{t1} \).
This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
+
+Add figures and bring up equations. + +
+
+
+
+Have example with eigenvalues solvers as well. + +
+
+
+
+Discussions about dynamic unrolling through time. discuss memory cells, input and output + diff --git a/doc/pub/Recurrent/ipynb/Recurrent.ipynb b/doc/pub/Recurrent/ipynb/Recurrent.ipynb index 56545c516..8e583fc12 100644 --- a/doc/pub/Recurrent/ipynb/Recurrent.ipynb +++ b/doc/pub/Recurrent/ipynb/Recurrent.ipynb @@ -10,7 +10,7 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Jan 8, 2019**\n", + "Date: **Dec 16, 2019**\n", "\n", "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", @@ -37,6 +37,8 @@ "input, making them extremely useful for natural language processing\n", "systems such as automatic translation and speech-to-text.\n", "\n", + "**The text here is under development**. Planned finished Jan 2020.\n", + "\n", "\n", "## Set up of an RNN\n", "\n", @@ -47,7 +49,21 @@ "the output from the previous layer $t-1$, that is $y_{t1}$.\n", "\n", "This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link\n", - "the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN." + "the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN.\n", + "\n", + "\n", + "Add figures and bring up equations. \n", + "\n", + "\n", + "\n", + "## Solving differential equations and eigenvalue problems with RNNs\n", + "\n", + "Have example with eigenvalues solvers as well.\n", + "\n", + "\n", + "## Long-Short Time Memory\n", + "\n", + "Discussions about dynamic unrolling through time. discuss memory cells, input and output" ] } ], diff --git a/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz b/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz index 33f7542bf..7a09d6f0f 100644 Binary files a/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz and b/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz differ diff --git a/doc/pub/Recurrent/pdf/Recurrent-minted.pdf b/doc/pub/Recurrent/pdf/Recurrent-minted.pdf index 7ab225033..6a7bf9437 100644 Binary files a/doc/pub/Recurrent/pdf/Recurrent-minted.pdf and b/doc/pub/Recurrent/pdf/Recurrent-minted.pdf differ diff --git a/doc/src/Recurrent/Recurrent.do.txt b/doc/src/Recurrent/Recurrent.do.txt index 9a1e83dba..ae8d5c6b2 100644 --- a/doc/src/Recurrent/Recurrent.do.txt +++ b/doc/src/Recurrent/Recurrent.do.txt @@ -24,6 +24,8 @@ example, they can take sentences, documents, or audio samples as input, making them extremely useful for natural language processing systems such as automatic translation and speech-to-text. +_The text here is under development_. Planned finished Jan 2020. + !split ===== Set up of an RNN ===== @@ -38,8 +40,17 @@ This means also that we need to have weights that link both the inputs $x_t$ to the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN. -Add figures and bring up equations. Have example with eigenvalues solvers as well. -Discussions about dynamic unrolling through time. discuss memory cells, input and output +Add figures and bring up equations. -Add about LSTM cell. -Should add autoencoders? + + +!split +===== Solving differential equations and eigenvalue problems with RNNs ===== + +Have example with eigenvalues solvers as well. + + +!split +===== Long-Short Time Memory ===== + +Discussions about dynamic unrolling through time. discuss memory cells, input and output diff --git a/doc/src/Recurrent/Recurrent.do.txt~ b/doc/src/Recurrent/Recurrent.do.txt~ deleted file mode 100644 index 2d054e465..000000000 --- a/doc/src/Recurrent/Recurrent.do.txt~ +++ /dev/null @@ -1,41 +0,0 @@ -TITLE: Data Analysis and Machine Learning: Recurrent neural networks -AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -DATE: today - - -!split -===== Recurrent neural networks: Overarching view ===== - -Till now our focus has been, including convolutional neural networks as well, -on feedforward neural networks. The output or the -activations flow only in one direction, from the input layer to the -output layer. - -A recurrent neural network (RNN) looks very much like a feedforward -neural network, except that it also has connections pointing -backward. - -RNNs are used to analyze time series data such as stock prices, and -tell you when to buy or sell. In autonomous driving systems, they can -anticipate car trajectories and help avoid accidents. More generally, -they can work on sequences of arbitrary lengths, rather than on -fixed-sized inputs like all the nets we have discussed so far. For -example, they can take sentences, documents, or audio samples as -input, making them extremely useful for natural language processing -systems such as automatic translation and speech-to-text. - - -!split -===== Set up of an RNN ===== - -The figure here displays a simple example of an RNN, with inputs $x_t$ -at a given time $t$ and outputs $y_t$. Introducing time as a variable -offers an intutitive way of understanding these networks. In addition -to the inputs $x_t$, the layer at a time $t$ receives also as input -the output from the previous layer $t-1$, that is $y_{t1}$. - -This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link -the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN. - - -Add figures and bring up equations. Have example with eigenvalues solvers