update with refs

2022-09-27 14:48:07 +02:00
parent 4e7bc1767a
commit b5a9d3dcd0
5 changed files with 22 additions and 13 deletions
@@ -191,7 +191,10 @@ doconce format html week39.do.txt --html_style=bootstrap --pygments_html_style=d
               None,
               'second-moment-of-the-gradient'),
              ('RMS prop', 2, None, 'rms-prop'),
-              ('ADAM optimizer', 2, None, 'adam-optimizer'),
+              ('"ADAM optimizer":"https://arxiv.org/abs/1412.6980"',
+               2,
+               None,
+               'adam-optimizer-https-arxiv-org-abs-1412-6980'),
              ('Practical tips', 2, None, 'practical-tips'),
              ('Automatic differentiation',
               2,
@@ -325,7 +328,7 @@ MathJax.Hub.Config({
     <!-- navigation toc: --> <li><a href="._week39-bs059.html#momentum-parameter" style="font-size: 80%;">Momentum parameter</a></li>
     <!-- navigation toc: --> <li><a href="._week39-bs060.html#second-moment-of-the-gradient" style="font-size: 80%;">Second moment of the gradient</a></li>
     <!-- navigation toc: --> <li><a href="._week39-bs061.html#rms-prop" style="font-size: 80%;">RMS prop</a></li>
-     <!-- navigation toc: --> <li><a href="._week39-bs062.html#adam-optimizer" style="font-size: 80%;">ADAM optimizer</a></li>
+     <!-- navigation toc: --> <li><a href="._week39-bs062.html#adam-optimizer-https-arxiv-org-abs-1412-6980" style="font-size: 80%;">"ADAM optimizer":"https://arxiv.org/abs/1412.6980"</a></li>
     <!-- navigation toc: --> <li><a href="._week39-bs063.html#practical-tips" style="font-size: 80%;">Practical tips</a></li>
     <!-- navigation toc: --> <li><a href="._week39-bs064.html#automatic-differentiation" style="font-size: 80%;">Automatic differentiation</a></li>
     <!-- navigation toc: --> <li><a href="._week39-bs065.html#using-autograd" style="font-size: 80%;">Using autograd</a></li>
@@ -2155,7 +2155,7 @@ Hessians.
 <p>Recently, a number of methods have been introduced that accomplish
 this by tracking not only the gradient, but also the second moment of
 the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and
-ADAM.
+<a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>.
 </p>
 </section>

@@ -2192,9 +2192,9 @@ learning rate for flat directions.
 </section>

 <section>
-<h2 id="adam-optimizer">ADAM optimizer </h2>
+<h2 id="adam-optimizer-https-arxiv-org-abs-1412-6980"><a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM optimizer</a> </h2>

-<p>A related algorithm is the ADAM optimizer. In ADAM, we keep a running
+<p>A related algorithm is the ADAM optimizer. In <a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running
 average of both the first and second moment of the gradient and use
 this information to adaptively change the learning rate for different
 parameters. In addition to keeping a running average of the first and
@@ -218,7 +218,10 @@ div.toc p,a {
               None,
               'second-moment-of-the-gradient'),
              ('RMS prop', 2, None, 'rms-prop'),
-              ('ADAM optimizer', 2, None, 'adam-optimizer'),
+              ('"ADAM optimizer":"https://arxiv.org/abs/1412.6980"',
+               2,
+               None,
+               'adam-optimizer-https-arxiv-org-abs-1412-6980'),
              ('Practical tips', 2, None, 'practical-tips'),
              ('Automatic differentiation',
               2,
@@ -2075,7 +2078,7 @@ Hessians.
 <p>Recently, a number of methods have been introduced that accomplish
 this by tracking not only the gradient, but also the second moment of
 the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and
-ADAM.
+<a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>.
 </p>

 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -2108,9 +2111,9 @@ learning rate for flat directions.
 </p>

 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="adam-optimizer">ADAM optimizer </h2>
+<h2 id="adam-optimizer-https-arxiv-org-abs-1412-6980"><a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM optimizer</a> </h2>

-<p>A related algorithm is the ADAM optimizer. In ADAM, we keep a running
+<p>A related algorithm is the ADAM optimizer. In <a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running
 average of both the first and second moment of the gradient and use
 this information to adaptively change the learning rate for different
 parameters. In addition to keeping a running average of the first and
@@ -295,7 +295,10 @@ div.toc p,a {
               None,
               'second-moment-of-the-gradient'),
              ('RMS prop', 2, None, 'rms-prop'),
-              ('ADAM optimizer', 2, None, 'adam-optimizer'),
+              ('"ADAM optimizer":"https://arxiv.org/abs/1412.6980"',
+               2,
+               None,
+               'adam-optimizer-https-arxiv-org-abs-1412-6980'),
              ('Practical tips', 2, None, 'practical-tips'),
              ('Automatic differentiation',
               2,
@@ -2152,7 +2155,7 @@ Hessians.
 <p>Recently, a number of methods have been introduced that accomplish
 this by tracking not only the gradient, but also the second moment of
 the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and
-ADAM.
+<a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>.
 </p>

 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -2185,9 +2188,9 @@ learning rate for flat directions.
 </p>

 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
-<h2 id="adam-optimizer">ADAM optimizer </h2>
+<h2 id="adam-optimizer-https-arxiv-org-abs-1412-6980"><a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM optimizer</a> </h2>

-<p>A related algorithm is the ADAM optimizer. In ADAM, we keep a running
+<p>A related algorithm is the ADAM optimizer. In <a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running
 average of both the first and second moment of the gradient and use
 this information to adaptively change the learning rate for different
 parameters. In addition to keeping a running average of the first and