rnn update
This commit is contained in:
@@ -6,10 +6,9 @@ DATE: today
|
||||
!split
|
||||
===== Recurrent neural networks: Overarching view =====
|
||||
|
||||
Till now our focus has been, including convolutional neural networks as well,
|
||||
on feedforward neural networks. The output or the
|
||||
activations flow only in one direction, from the input layer to the
|
||||
output layer.
|
||||
Till now our focus has been, including convolutional neural networks
|
||||
as well, on feedforward neural networks. The output or the activations
|
||||
flow only in one direction, from the input layer to the output layer.
|
||||
|
||||
A recurrent neural network (RNN) looks very much like a feedforward
|
||||
neural network, except that it also has connections pointing
|
||||
@@ -24,7 +23,7 @@ example, they can take sentences, documents, or audio samples as
|
||||
input, making them extremely useful for natural language processing
|
||||
systems such as automatic translation and speech-to-text.
|
||||
|
||||
_The text here is under development_. Planned finished Jan 2020.
|
||||
_The text here is under development_. Planned finished mid Jan 2020.
|
||||
|
||||
|
||||
!split
|
||||
@@ -36,18 +35,27 @@ offers an intutitive way of understanding these networks. In addition
|
||||
to the inputs $x_t$, the layer at a time $t$ receives also as input
|
||||
the output from the previous layer $t-1$, that is $y_{t1}$.
|
||||
|
||||
This means also that we need to have weights that link both the inputs $x_t$ to the outputs $y_t$ as well as weights that link
|
||||
the output from the previous time $y_{t-1}$ and $y_t$. The figure here shows an example of a simple RNN.
|
||||
|
||||
|
||||
Add figures and bring up equations.
|
||||
This means also that we need to have weights that link both the inputs
|
||||
$x_t$ to the outputs $y_t$ as well as weights that link the output
|
||||
from the previous time $y_{t-1}$ and $y_t$. The figure here shows an
|
||||
example of a simple RNN.
|
||||
|
||||
|
||||
|
||||
!split
|
||||
===== Solving differential equations and eigenvalue problems with RNNs =====
|
||||
|
||||
Have example with eigenvalues solvers as well.
|
||||
|
||||
|
||||
In our discussions of ordinary differential equations and partial
|
||||
differential equations using neural networks. Here we will discuss how
|
||||
we can solve say ordinary differential equations and eigenvalue
|
||||
problems using RNNs. Eigenvalue problems can be solved using RNNs by
|
||||
rewriting such a problems as a non-linear differential equation.
|
||||
|
||||
Instead of starting with a well-known ordinary differential equation,
|
||||
we start directly with an eigenvaule problem.
|
||||
|
||||
|
||||
|
||||
!split
|
||||
|
||||
Reference in New Issue
Block a user