@@ -67,6 +84,7 @@ end of tocinfo -->
Contents
@@ -101,7 +119,7 @@ end of tocinfo -->
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
-
Dec 29, 2018
+
Jan 8, 2019
@@ -116,6 +134,7 @@ end of tocinfo -->
@@ -133,7 +152,7 @@ end of tocinfo -->
- © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+ © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/Recurrent/html/._Recurrent-bs001.html b/doc/pub/Recurrent/html/._Recurrent-bs001.html
index 090fd5c9e..e41f16735 100644
--- a/doc/pub/Recurrent/html/._Recurrent-bs001.html
+++ b/doc/pub/Recurrent/html/._Recurrent-bs001.html
@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
- '___sec0')]}
+ '___sec0'),
+ ('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
+
+
+
+
+
+
+
@@ -67,6 +84,7 @@ end of tocinfo -->
Contents
@@ -85,16 +103,15 @@ end of tocinfo -->
Recurrent neural networks: Overarching view
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well,
+on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let’s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.
RNNs are used to analyze time series data such as stock prices, and
@@ -106,12 +123,15 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
+
diff --git a/doc/pub/Recurrent/html/Recurrent-bs.html b/doc/pub/Recurrent/html/Recurrent-bs.html
index 6df563e32..12f45e139 100644
--- a/doc/pub/Recurrent/html/Recurrent-bs.html
+++ b/doc/pub/Recurrent/html/Recurrent-bs.html
@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
- '___sec0')]}
+ '___sec0'),
+ ('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
+
+
+
+
+
+
+
@@ -67,6 +84,7 @@ end of tocinfo -->
Contents
@@ -101,7 +119,7 @@ end of tocinfo -->
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
-
Dec 29, 2018
+
Jan 8, 2019
@@ -116,6 +134,7 @@ end of tocinfo -->
@@ -133,7 +152,7 @@ end of tocinfo -->
- © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+ © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/Recurrent/html/Recurrent-reveal.html b/doc/pub/Recurrent/html/Recurrent-reveal.html
index 750c6bf39..a073f8b6a 100644
--- a/doc/pub/Recurrent/html/Recurrent-reveal.html
+++ b/doc/pub/Recurrent/html/Recurrent-reveal.html
@@ -107,6 +107,22 @@ td.padding {
+
+
+
+
+
+
+
@@ -132,12 +148,12 @@ td.padding {
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
-
Dec 29, 2018
+
Jan 8, 2019
- © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+ © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
@@ -146,16 +162,15 @@ td.padding {
Recurrent neural networks: Overarching view
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well,
+on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let’s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.
RNNs are used to analyze time series data such as stock prices, and
@@ -169,6 +184,22 @@ systems such as automatic translation and speech-to-text.
+
+Set up of an RNN
+
+
+The figure here displays a simple example of an RNN, with inputs \( x_t \)
+at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
+offers an intutitive way of understanding these networks. In addition
+to the inputs \( x_t \), the layer at a time \( t \) receives also as input
+the output from the previous layer \( t-1 \), that is \( y_{t1} \).
+
+
+This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
+the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
+
+
+
diff --git a/doc/pub/Recurrent/html/Recurrent-solarized.html b/doc/pub/Recurrent/html/Recurrent-solarized.html
index 7c8fb6b99..ca9871a5c 100644
--- a/doc/pub/Recurrent/html/Recurrent-solarized.html
+++ b/doc/pub/Recurrent/html/Recurrent-solarized.html
@@ -38,11 +38,28 @@ div { text-align: justify; text-justify: inter-word; }
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
- '___sec0')]}
+ '___sec0'),
+ ('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
+
+
+
+
+
+
+
@@ -64,7 +81,7 @@ end of tocinfo -->
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
-
Dec 29, 2018
+
Jan 8, 2019
@@ -72,16 +89,15 @@ end of tocinfo -->
Recurrent neural networks: Overarching view
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well,
+on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let’s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.
RNNs are used to analyze time series data such as stock prices, and
@@ -93,11 +109,27 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
+
+
+
+
Set up of an RNN
+
+
+The figure here displays a simple example of an RNN, with inputs \( x_t \)
+at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
+offers an intutitive way of understanding these networks. In addition
+to the inputs \( x_t \), the layer at a time \( t \) receives also as input
+the output from the previous layer \( t-1 \), that is \( y_{t1} \).
+
+
+This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
+the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
+
- © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+ © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/Recurrent/html/Recurrent.html b/doc/pub/Recurrent/html/Recurrent.html
index f7b594356..55e404e12 100644
--- a/doc/pub/Recurrent/html/Recurrent.html
+++ b/doc/pub/Recurrent/html/Recurrent.html
@@ -43,11 +43,28 @@ div { text-align: justify; text-justify: inter-word; }
'sections': [('Recurrent neural networks: Overarching view',
2,
None,
- '___sec0')]}
+ '___sec0'),
+ ('Set up of an RNN', 2, None, '___sec1')]}
end of tocinfo -->
+
+
+
+
+
+
+
@@ -69,7 +86,7 @@ end of tocinfo -->
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
-
Dec 29, 2018
+
Jan 8, 2019
@@ -77,16 +94,15 @@ end of tocinfo -->
Recurrent neural networks: Overarching view
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well,
+on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let’s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.
RNNs are used to analyze time series data such as stock prices, and
@@ -98,11 +114,27 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
+
+
+
+
Set up of an RNN
+
+
+The figure here displays a simple example of an RNN, with inputs \( x_t \)
+at a given time \( t \) and outputs \( y_t \). Introducing time as a variable
+offers an intutitive way of understanding these networks. In addition
+to the inputs \( x_t \), the layer at a time \( t \) receives also as input
+the output from the previous layer \( t-1 \), that is \( y_{t1} \).
+
+
+This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
+the output from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an example of a simple RNN.
+
- © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+ © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/Recurrent/ipynb/Recurrent.ipynb b/doc/pub/Recurrent/ipynb/Recurrent.ipynb
index 55b7ea533..56545c516 100644
--- a/doc/pub/Recurrent/ipynb/Recurrent.ipynb
+++ b/doc/pub/Recurrent/ipynb/Recurrent.ipynb
@@ -10,24 +10,23 @@
" \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
- "Date: **Dec 29, 2018**\n",
+ "Date: **Jan 8, 2019**\n",
"\n",
- "Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
+ "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
"\n",
"\n",
"\n",
"## Recurrent neural networks: Overarching view\n",
"\n",
- "We have mostly looked at feedforward neural networks, where the\n",
+ "Till now our focus has been, including convolutional neural networks as well, \n",
+ "on feedforward neural networks. The output or the\n",
"activations flow only in one direction, from the input layer to the\n",
"output layer.\n",
"\n",
"A recurrent neural network (RNN) looks very much like a feedforward\n",
- "neural network, except it also has connections pointing\n",
- "backward. Let’s look at the simplest possible RNN, composed of just\n",
- "one neuron receiving inputs, producing an output, and sending that\n",
- "output back to itself.\n",
+ "neural network, except that it also has connections pointing\n",
+ "backward. \n",
"\n",
"RNNs are used to analyze time series data such as stock prices, and\n",
"tell you when to buy or sell. In autonomous driving systems, they can\n",
@@ -36,7 +35,19 @@
"fixed-sized inputs like all the nets we have discussed so far. For\n",
"example, they can take sentences, documents, or audio samples as\n",
"input, making them extremely useful for natural language processing\n",
- "systems such as automatic translation and speech-to-text."
+ "systems such as automatic translation and speech-to-text.\n",
+ "\n",
+ "\n",
+ "## Set up of an RNN\n",
+ "\n",
+ "The figure here displays a simple example of an RNN, with inputs $x_t$\n",
+ "at a given time $t$ and outputs $y_t$. Introducing time as a variable\n",
+ "offers an intutitive way of understanding these networks. In addition\n",
+ "to the inputs $x_t$, the layer at a time $t$ receives also as input\n",
+ "the output from the previous layer $t-1$, that is $y_{t1}$.\n",
+ "\n",
+ "This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link\n",
+ "the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN."
]
}
],
diff --git a/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz b/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz
index fb10e07f9..33f7542bf 100644
Binary files a/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz and b/doc/pub/Recurrent/ipynb/ipynb-Recurrent-src.tar.gz differ
diff --git a/doc/pub/Recurrent/pdf/Recurrent-minted.pdf b/doc/pub/Recurrent/pdf/Recurrent-minted.pdf
index d9d7782f6..7ab225033 100644
Binary files a/doc/pub/Recurrent/pdf/Recurrent-minted.pdf and b/doc/pub/Recurrent/pdf/Recurrent-minted.pdf differ
diff --git a/doc/src/Splines/Splines.do.txt b/doc/src/Splines/Splines.do.txt
index 57a5d4059..b6ad185d5 100644
--- a/doc/src/Splines/Splines.do.txt
+++ b/doc/src/Splines/Splines.do.txt
@@ -3,6 +3,9 @@ AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of
DATE: today
+!split
+===== Optimization problems, why? =====
+
!split
===== Optimization, the central part of any Machine Learning algortithm =====
diff --git a/doc/src/Statistics/Statistics.do.txt b/doc/src/Statistics/Statistics.do.txt
index f282d896e..7d14ac390 100644
--- a/doc/src/Statistics/Statistics.do.txt
+++ b/doc/src/Statistics/Statistics.do.txt
@@ -3,6 +3,7 @@ AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of
DATE: today
+
!split
===== Domains and probabilities =====
!bblock