diff --git a/doc/pub/week39/html/week39-bs.html b/doc/pub/week39/html/week39-bs.html index fc49a1729..e853e960d 100644 --- a/doc/pub/week39/html/week39-bs.html +++ b/doc/pub/week39/html/week39-bs.html @@ -191,7 +191,10 @@ doconce format html week39.do.txt --html_style=bootstrap --pygments_html_style=d None, 'second-moment-of-the-gradient'), ('RMS prop', 2, None, 'rms-prop'), - ('ADAM optimizer', 2, None, 'adam-optimizer'), + ('"ADAM optimizer":"https://arxiv.org/abs/1412.6980"', + 2, + None, + 'adam-optimizer-https-arxiv-org-abs-1412-6980'), ('Practical tips', 2, None, 'practical-tips'), ('Automatic differentiation', 2, @@ -325,7 +328,7 @@ MathJax.Hub.Config({
Recently, a number of methods have been introduced that accomplish this by tracking not only the gradient, but also the second moment of the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and -ADAM. +ADAM.
@@ -2192,9 +2192,9 @@ learning rate for flat directions.A related algorithm is the ADAM optimizer. In ADAM, we keep a running +
A related algorithm is the ADAM optimizer. In ADAM, we keep a running average of both the first and second moment of the gradient and use this information to adaptively change the learning rate for different parameters. In addition to keeping a running average of the first and diff --git a/doc/pub/week39/html/week39-solarized.html b/doc/pub/week39/html/week39-solarized.html index 6e235d7fd..cf8a6af8b 100644 --- a/doc/pub/week39/html/week39-solarized.html +++ b/doc/pub/week39/html/week39-solarized.html @@ -218,7 +218,10 @@ div.toc p,a { None, 'second-moment-of-the-gradient'), ('RMS prop', 2, None, 'rms-prop'), - ('ADAM optimizer', 2, None, 'adam-optimizer'), + ('"ADAM optimizer":"https://arxiv.org/abs/1412.6980"', + 2, + None, + 'adam-optimizer-https-arxiv-org-abs-1412-6980'), ('Practical tips', 2, None, 'practical-tips'), ('Automatic differentiation', 2, @@ -2075,7 +2078,7 @@ Hessians.
Recently, a number of methods have been introduced that accomplish this by tracking not only the gradient, but also the second moment of the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and -ADAM. +ADAM.
A related algorithm is the ADAM optimizer. In ADAM, we keep a running +
A related algorithm is the ADAM optimizer. In ADAM, we keep a running average of both the first and second moment of the gradient and use this information to adaptively change the learning rate for different parameters. In addition to keeping a running average of the first and diff --git a/doc/pub/week39/html/week39.html b/doc/pub/week39/html/week39.html index 530f28186..1e49d0904 100644 --- a/doc/pub/week39/html/week39.html +++ b/doc/pub/week39/html/week39.html @@ -295,7 +295,10 @@ div.toc p,a { None, 'second-moment-of-the-gradient'), ('RMS prop', 2, None, 'rms-prop'), - ('ADAM optimizer', 2, None, 'adam-optimizer'), + ('"ADAM optimizer":"https://arxiv.org/abs/1412.6980"', + 2, + None, + 'adam-optimizer-https-arxiv-org-abs-1412-6980'), ('Practical tips', 2, None, 'practical-tips'), ('Automatic differentiation', 2, @@ -2152,7 +2155,7 @@ Hessians.
Recently, a number of methods have been introduced that accomplish this by tracking not only the gradient, but also the second moment of the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and -ADAM. +ADAM.
A related algorithm is the ADAM optimizer. In ADAM, we keep a running +
A related algorithm is the ADAM optimizer. In ADAM, we keep a running average of both the first and second moment of the gradient and use this information to adaptively change the learning rate for different parameters. In addition to keeping a running average of the first and diff --git a/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz b/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz index a9eadf037..81b0905a3 100644 Binary files a/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz and b/doc/pub/week39/ipynb/ipynb-week39-src.tar.gz differ