added new section

2019-01-09 17:38:02 -05:00
parent 48ede08249
commit 623f928cf1
11 changed files with 211 additions and 43 deletions
@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
 'sections': [('Recurrent neural networks: Overarching view',
               2,
               None,
-               '___sec0')]}
+               '___sec0'),
+              ('Set up of an RNN', 2, None, '___sec1')]}
 end of tocinfo -->

 <body>

+
+
+<script type="text/x-mathjax-config">
+MathJax.Hub.Config({
+  TeX: {
+     equationNumbers: {  autoNumber: "none"  },
+     extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
+  }
+});
+</script>
+<script type="text/javascript" async
+ src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
+</script>
+
+
+
    
 <!-- Bootstrap navigation bar -->
 <div class="navbar navbar-default navbar-fixed-top">
@@ -67,6 +84,7 @@ end of tocinfo -->
        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
        <ul class="dropdown-menu">
     <!-- navigation toc: --> <li><a href="._Recurrent-bs001.html#___sec0" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
+     <!-- navigation toc: --> <li><a href="._Recurrent-bs002.html#___sec1" style="font-size: 80%;">Set up of an RNN</a></li>

        </ul>
      </li>
@@ -101,7 +119,7 @@ end of tocinfo -->
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2018</h4></center> <!-- date -->
+<center><h4>Jan 8, 2019</h4></center> <!-- date -->
 <br>
 <p>

@@ -116,6 +134,7 @@ end of tocinfo -->
 <ul class="pagination">
  <li class="active"><a href="._Recurrent-bs000.html">1</a></li>
  <li><a href="._Recurrent-bs001.html">2</a></li>
+  <li><a href="._Recurrent-bs002.html">3</a></li>
  <li><a href="._Recurrent-bs001.html">&raquo;</a></li>
 </ul>
 <!-- ------------------- end of main content --------------- -->
@@ -133,7 +152,7 @@ end of tocinfo -->


 <center style="font-size:80%">
-<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
 </center>


@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
 'sections': [('Recurrent neural networks: Overarching view',
               2,
               None,
-               '___sec0')]}
+               '___sec0'),
+              ('Set up of an RNN', 2, None, '___sec1')]}
 end of tocinfo -->

 <body>

+
+
+<script type="text/x-mathjax-config">
+MathJax.Hub.Config({
+  TeX: {
+     equationNumbers: {  autoNumber: "none"  },
+     extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
+  }
+});
+</script>
+<script type="text/javascript" async
+ src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
+</script>
+
+
+
    
 <!-- Bootstrap navigation bar -->
 <div class="navbar navbar-default navbar-fixed-top">
@@ -67,6 +84,7 @@ end of tocinfo -->
        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
        <ul class="dropdown-menu">
     <!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
+     <!-- navigation toc: --> <li><a href="._Recurrent-bs002.html#___sec1" style="font-size: 80%;">Set up of an RNN</a></li>

        </ul>
      </li>
@@ -85,16 +103,15 @@ end of tocinfo -->
 <h2 id="___sec0" class="anchor">Recurrent neural networks: Overarching view </h2>

 <p>
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well, 
+on feedforward neural networks. The output or the
 activations flow only in one direction, from the input layer to the
 output layer.

 <p>
 A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let&#8217;s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.

 <p>
 RNNs are used to analyze time series data such as stock prices, and
@@ -106,12 +123,15 @@ example, they can take sentences, documents, or audio samples as
 input, making them extremely useful for natural language processing
 systems such as automatic translation and speech-to-text.

+<p>
 <p>
 <!-- navigation buttons at the bottom of the page -->
 <ul class="pagination">
 <li><a href="._Recurrent-bs000.html">&laquo;</a></li>
  <li><a href="._Recurrent-bs000.html">1</a></li>
  <li class="active"><a href="._Recurrent-bs001.html">2</a></li>
+  <li><a href="._Recurrent-bs002.html">3</a></li>
+  <li><a href="._Recurrent-bs002.html">&raquo;</a></li>
 </ul>
 <!-- ------------------- end of main content --------------- -->

@@ -44,11 +44,28 @@ Automatically generated HTML file from DocOnce source
 'sections': [('Recurrent neural networks: Overarching view',
               2,
               None,
-               '___sec0')]}
+               '___sec0'),
+              ('Set up of an RNN', 2, None, '___sec1')]}
 end of tocinfo -->

 <body>

+
+
+<script type="text/x-mathjax-config">
+MathJax.Hub.Config({
+  TeX: {
+     equationNumbers: {  autoNumber: "none"  },
+     extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
+  }
+});
+</script>
+<script type="text/javascript" async
+ src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
+</script>
+
+
+
    
 <!-- Bootstrap navigation bar -->
 <div class="navbar navbar-default navbar-fixed-top">
@@ -67,6 +84,7 @@ end of tocinfo -->
        <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
        <ul class="dropdown-menu">
     <!-- navigation toc: --> <li><a href="._Recurrent-bs001.html#___sec0" style="font-size: 80%;">Recurrent neural networks: Overarching view</a></li>
+     <!-- navigation toc: --> <li><a href="._Recurrent-bs002.html#___sec1" style="font-size: 80%;">Set up of an RNN</a></li>

        </ul>
      </li>
@@ -101,7 +119,7 @@ end of tocinfo -->
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2018</h4></center> <!-- date -->
+<center><h4>Jan 8, 2019</h4></center> <!-- date -->
 <br>
 <p>

@@ -116,6 +134,7 @@ end of tocinfo -->
 <ul class="pagination">
  <li class="active"><a href="._Recurrent-bs000.html">1</a></li>
  <li><a href="._Recurrent-bs001.html">2</a></li>
+  <li><a href="._Recurrent-bs002.html">3</a></li>
  <li><a href="._Recurrent-bs001.html">&raquo;</a></li>
 </ul>
 <!-- ------------------- end of main content --------------- -->
@@ -133,7 +152,7 @@ end of tocinfo -->


 <center style="font-size:80%">
-<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
 </center>


@@ -107,6 +107,22 @@ td.padding {



+
+
+<script type="text/x-mathjax-config">
+MathJax.Hub.Config({
+  TeX: {
+     equationNumbers: {  autoNumber: "none"  },
+     extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
+  }
+});
+</script>
+<script type="text/javascript" async
+ src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
+</script>
+
+
+
    


@@ -132,12 +148,12 @@ td.padding {
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>&nbsp;<br>
-<center><h4>Dec 29, 2018</h4></center> <!-- date -->
+<center><h4>Jan 8, 2019</h4></center> <!-- date -->
 <br>
 <p>

 <center style="font-size:80%">
-<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
 </center>
 </section>

@@ -146,16 +162,15 @@ td.padding {
 <h2 id="___sec0">Recurrent neural networks: Overarching view </h2>

 <p>
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well, 
+on feedforward neural networks. The output or the
 activations flow only in one direction, from the input layer to the
 output layer.

 <p>
 A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let&#8217;s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.

 <p>
 RNNs are used to analyze time series data such as stock prices, and
@@ -169,6 +184,22 @@ systems such as automatic translation and speech-to-text.
 </section>


+<section>
+<h2 id="___sec1">Set up of an RNN </h2>
+
+<p>
+The figure here displays a simple example of an RNN, with inputs \( x_t \)
+at a given time \( t \) and outputs \( y_t \).  Introducing time as a variable
+offers an intutitive way of understanding these networks. In addition
+to the inputs \( x_t \), the layer at a time \( t \) receives also as input
+the output from the previous layer \( t-1 \), that is \( y_{t1} \).
+
+<p>
+This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
+the output from the previous time \( y_{t-1} \) and \( y_t \).   The figure here shows an example of a simple RNN.
+</section>
+
+

 </div> <!-- class="slides" -->
 </div> <!-- class="reveal" -->
@@ -38,11 +38,28 @@ div { text-align: justify; text-justify: inter-word; }
 'sections': [('Recurrent neural networks: Overarching view',
               2,
               None,
-               '___sec0')]}
+               '___sec0'),
+              ('Set up of an RNN', 2, None, '___sec1')]}
 end of tocinfo -->

 <body>

+
+
+<script type="text/x-mathjax-config">
+MathJax.Hub.Config({
+  TeX: {
+     equationNumbers: {  autoNumber: "AMS"  },
+     extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
+  }
+});
+</script>
+<script type="text/javascript" async
+ src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
+</script>
+
+
+
    
 <!-- ------------------- main content ---------------------- -->

@@ -64,7 +81,7 @@ end of tocinfo -->
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2018</h4></center> <!-- date -->
+<center><h4>Jan 8, 2019</h4></center> <!-- date -->
 <br>
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -72,16 +89,15 @@ end of tocinfo -->
 <h2 id="___sec0">Recurrent neural networks: Overarching view </h2>

 <p>
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well, 
+on feedforward neural networks. The output or the
 activations flow only in one direction, from the input layer to the
 output layer.

 <p>
 A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let&#8217;s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.

 <p>
 RNNs are used to analyze time series data such as stock prices, and
@@ -93,11 +109,27 @@ example, they can take sentences, documents, or audio samples as
 input, making them extremely useful for natural language processing
 systems such as automatic translation and speech-to-text.

+<p>
+<!-- !split --><br><br><br><br><br><br><br><br><br><br>
+
+<h2 id="___sec1">Set up of an RNN </h2>
+
+<p>
+The figure here displays a simple example of an RNN, with inputs \( x_t \)
+at a given time \( t \) and outputs \( y_t \).  Introducing time as a variable
+offers an intutitive way of understanding these networks. In addition
+to the inputs \( x_t \), the layer at a time \( t \) receives also as input
+the output from the previous layer \( t-1 \), that is \( y_{t1} \).
+
+<p>
+This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
+the output from the previous time \( y_{t-1} \) and \( y_t \).   The figure here shows an example of a simple RNN.
+
 <!-- ------------------- end of main content --------------- -->


 <center style="font-size:80%">
-<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
 </center>


@@ -43,11 +43,28 @@ div { text-align: justify; text-justify: inter-word; }
 'sections': [('Recurrent neural networks: Overarching view',
               2,
               None,
-               '___sec0')]}
+               '___sec0'),
+              ('Set up of an RNN', 2, None, '___sec1')]}
 end of tocinfo -->

 <body>

+
+
+<script type="text/x-mathjax-config">
+MathJax.Hub.Config({
+  TeX: {
+     equationNumbers: {  autoNumber: "AMS"  },
+     extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
+  }
+});
+</script>
+<script type="text/javascript" async
+ src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
+</script>
+
+
+
    
 <!-- ------------------- main content ---------------------- -->

@@ -69,7 +86,7 @@ end of tocinfo -->
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2018</h4></center> <!-- date -->
+<center><h4>Jan 8, 2019</h4></center> <!-- date -->
 <br>
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -77,16 +94,15 @@ end of tocinfo -->
 <h2 id="___sec0">Recurrent neural networks: Overarching view </h2>

 <p>
-We have mostly looked at feedforward neural networks, where the
+Till now our focus has been, including convolutional neural networks as well, 
+on feedforward neural networks. The output or the
 activations flow only in one direction, from the input layer to the
 output layer.

 <p>
 A recurrent neural network (RNN) looks very much like a feedforward
-neural network, except it also has connections pointing
-backward. Let&#8217;s look at the simplest possible RNN, composed of just
-one neuron receiving inputs, producing an output, and sending that
-output back to itself.
+neural network, except that it also has connections pointing
+backward.

 <p>
 RNNs are used to analyze time series data such as stock prices, and
@@ -98,11 +114,27 @@ example, they can take sentences, documents, or audio samples as
 input, making them extremely useful for natural language processing
 systems such as automatic translation and speech-to-text.

+<p>
+<!-- !split --><br><br><br><br><br><br><br><br><br><br>
+
+<h2 id="___sec1">Set up of an RNN </h2>
+
+<p>
+The figure here displays a simple example of an RNN, with inputs \( x_t \)
+at a given time \( t \) and outputs \( y_t \).  Introducing time as a variable
+offers an intutitive way of understanding these networks. In addition
+to the inputs \( x_t \), the layer at a time \( t \) receives also as input
+the output from the previous layer \( t-1 \), that is \( y_{t1} \).
+
+<p>
+This means also that we need to have weights that link both the inputs \( x_t \) to the outputs \( y_t \) as well as weights that link
+the output from the previous time \( y_{t-1} \) and \( y_t \).   The figure here shows an example of a simple RNN.
+
 <!-- ------------------- end of main content --------------- -->


 <center style="font-size:80%">
-<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
+<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
 </center>