diff --git a/doc/pub/week42/html/._week42-bs000.html b/doc/pub/week42/html/._week42-bs000.html index 8db429738..80d4c12ae 100644 --- a/doc/pub/week42/html/._week42-bs000.html +++ b/doc/pub/week42/html/._week42-bs000.html @@ -7,9 +7,9 @@ Automatically generated HTML file from DocOnce source - + -Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks and Autoencoders +Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks @@ -88,16 +88,8 @@ Automatically generated HTML file from DocOnce source 2, None, '___sec26'), - ('A simple example', 2, None, '___sec27'), - ('Set up of an RNN', 2, None, '___sec28'), - ('Solving differential equations and eigenvalue problems with ' - 'RNNs', - 2, - None, - '___sec29'), - ('Long-Short Time Memory', 2, None, '___sec30'), - ('Autoencoders: Overarching view', 2, None, '___sec31'), - ('Simple examples of Autoencoders', 2, None, '___sec32')]} + ('Set up of an RNN', 2, None, '___sec27'), + ('A simple example', 2, None, '___sec28')]} end of tocinfo --> @@ -127,7 +119,7 @@ MathJax.Hub.Config({ - Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks and Autoencoders + Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks -

-









- -

Set up of an RNN

- -

-The figure here displays a simple example of an RNN, with inputs \( x_t \) -at a given time \( t \) and outputs \( y_t \). Introducing time as a variable -offers an intutitive way of understanding these networks. In addition -to the inputs \( x_t \), the layer at a time \( t \) receives also as input -the output from the previous layer \( t-1 \), that is \( y_{t1} \). - -

-This means also that we need to have weights that link both the inputs -\( x_t \) to the outputs \( y_t \) as well as weights that link the output -from the previous time \( y_{t-1} \) and \( y_t \). The figure here shows an -example of a simple RNN. - -

-More material will be added here. - -

-









- -

Solving differential equations and eigenvalue problems with RNNs

- -

-In our discussions of ordinary differential equations and partial -differential equations using neural networks. Here we will discuss how -we can solve say ordinary differential equations and eigenvalue -problems using RNNs. Eigenvalue problems can be solved using RNNs by -rewriting such a problems as a non-linear differential equation. - -

-Instead of starting with a well-known ordinary differential equation, -we start directly with an eigenvaule problem. - -

-









- -

Long-Short Time Memory

- -

-Discussions about dynamic unrolling through time. discuss memory cells, input and output - -

-









- -

Autoencoders: Overarching view

- -

-Autoencoders are artificial neural networks capable of learning -efficient representations of the input data (these representations are called codings) without -any supervision (i.e., the training set is unlabeled). These codings -typically have a much lower dimensionality than the input data, making -autoencoders useful for dimensionality reduction. - -

-More importantly, autoencoders act as powerful feature detectors, and -they can be used for unsupervised pretraining of deep neural networks. - -

-Lastly, they are capable of randomly generating new data that looks -very similar to the training data; this is called a generative -model. For example, you could train an autoencoder on pictures of -faces, and it would then be able to generate new faces. Surprisingly, -autoencoders work by simply learning to copy their inputs to their -outputs. This may sound like a trivial task, but we will see that -constraining the network in various ways can make it rather -difficult. For example, you can limit the size of the internal -representation, or you can add noise to the inputs and train the -network to recover the original inputs. These constraints prevent the -autoencoder from trivially copying the inputs directly to the outputs, -which forces it to learn efficient ways of representing the data. In -short, the codings are byproducts of the autoencoder’s attempt to -learn the identity function under some constraints. - -

-









- -

Simple examples of Autoencoders

-

diff --git a/doc/pub/week42/html/week42.html b/doc/pub/week42/html/week42.html index 2d7df1280..ff3687cb1 100644 --- a/doc/pub/week42/html/week42.html +++ b/doc/pub/week42/html/week42.html @@ -7,9 +7,9 @@ Automatically generated HTML file from DocOnce source - + -Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks and Autoencoders +Week 42 Convolutional (CNN) and Recurrent (RNN) Neural Networks