From 562baf92e84161152cf250d2d881c554fa0d0005 Mon Sep 17 00:00:00 2001 From: mhjensen Date: Sat, 5 Jan 2019 20:51:58 -0500 Subject: [PATCH] dill --- doc/src/Recurrent/Recurrent.do.txt | 38 ++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 doc/src/Recurrent/Recurrent.do.txt diff --git a/doc/src/Recurrent/Recurrent.do.txt b/doc/src/Recurrent/Recurrent.do.txt new file mode 100644 index 000000000..e37213cb8 --- /dev/null +++ b/doc/src/Recurrent/Recurrent.do.txt @@ -0,0 +1,38 @@ +TITLE: Data Analysis and Machine Learning: Recurrent neural networks +AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University +DATE: today + + +!split +===== Recurrent neural networks: Overarching view ===== + +Till now our focus has been, including convolutional neural networks as well, +on feedforward neural networks. The output or the +activations flow only in one direction, from the input layer to the +output layer. + +A recurrent neural network (RNN) looks very much like a feedforward +neural network, except that it also has connections pointing +backward. + +RNNs are used to analyze time series data such as stock prices, and +tell you when to buy or sell. In autonomous driving systems, they can +anticipate car trajectories and help avoid accidents. More generally, +they can work on sequences of arbitrary lengths, rather than on +fixed-sized inputs like all the nets we have discussed so far. For +example, they can take sentences, documents, or audio samples as +input, making them extremely useful for natural language processing +systems such as automatic translation and speech-to-text. + + +!split +===== Set up of an RNN ===== + +The figure here displays a simple example of an RNN, with inputs $x_t$ +at a given time $t$ and outputs $y_t$. Introducing time as a variable +offers an intutitive way of understanding these networks. In addition +to the inputs $x_t$, the layer at a time $t$ receives also as input +the output from the previous layer $t-1$, that is $y_{t1}$. + +This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link +the output from the previous time $y_{t-1}$ and $y_t$.