rnn update

This commit is contained in:
mhjensen
2019-12-17 16:29:09 +01:00
parent aa177fbcba
commit 8c4db31d18
11 changed files with 103 additions and 71 deletions
+19 -11
View File
@@ -6,10 +6,9 @@ DATE: today
!split
===== Recurrent neural networks: Overarching view =====
Till now our focus has been, including convolutional neural networks as well,
on feedforward neural networks. The output or the
activations flow only in one direction, from the input layer to the
output layer.
Till now our focus has been, including convolutional neural networks
as well, on feedforward neural networks. The output or the activations
flow only in one direction, from the input layer to the output layer.
A recurrent neural network (RNN) looks very much like a feedforward
neural network, except that it also has connections pointing
@@ -24,7 +23,7 @@ example, they can take sentences, documents, or audio samples as
input, making them extremely useful for natural language processing
systems such as automatic translation and speech-to-text.
_The text here is under development_. Planned finished Jan 2020.
_The text here is under development_. Planned finished mid Jan 2020.
!split
@@ -36,18 +35,27 @@ offers an intutitive way of understanding these networks. In addition
to the inputs $x_t$, the layer at a time $t$ receives also as input
the output from the previous layer $t-1$, that is $y_{t1}$.
This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link
the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN.
Add figures and bring up equations.
This means also that we need to have weights that link both the inputs
$x_t$ to the outputs $y_t$ as well as weights that link the output
from the previous time $y_{t-1}$ and $y_t$. The figure here shows an
example of a simple RNN.
!split
===== Solving differential equations and eigenvalue problems with RNNs =====
Have example with eigenvalues solvers as well.
In our discussions of ordinary differential equations and partial
differential equations using neural networks. Here we will discuss how
we can solve say ordinary differential equations and eigenvalue
problems using RNNs. Eigenvalue problems can be solved using RNNs by
rewriting such a problems as a non-linear differential equation.
Instead of starting with a well-known ordinary differential equation,
we start directly with an eigenvaule problem.
!split