diff --git a/doc/src/Recurrent/Recurrent.do.txt b/doc/src/Recurrent/Recurrent.do.txt index 9cf4a4e49..e37213cb8 100644 --- a/doc/src/Recurrent/Recurrent.do.txt +++ b/doc/src/Recurrent/Recurrent.do.txt @@ -6,15 +6,14 @@ DATE: today !split ===== Recurrent neural networks: Overarching view ===== -We have mostly looked at feedforward neural networks, where the +Till now our focus has been, including convolutional neural networks as well, +on feedforward neural networks. The output or the activations flow only in one direction, from the input layer to the output layer. A recurrent neural network (RNN) looks very much like a feedforward -neural network, except it also has connections pointing -backward. Let’s look at the simplest possible RNN, composed of just -one neuron receiving inputs, producing an output, and sending that -output back to itself. +neural network, except that it also has connections pointing +backward. RNNs are used to analyze time series data such as stock prices, and tell you when to buy or sell. In autonomous driving systems, they can @@ -28,3 +27,12 @@ systems such as automatic translation and speech-to-text. !split ===== Set up of an RNN ===== + +The figure here displays a simple example of an RNN, with inputs $x_t$ +at a given time $t$ and outputs $y_t$. Introducing time as a variable +offers an intutitive way of understanding these networks. In addition +to the inputs $x_t$, the layer at a time $t$ receives also as input +the output from the previous layer $t-1$, that is $y_{t1}$. + +This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link +the output from the previous time $y_{t-1}$ and $y_t$.