From 4e7bc1767a66c0f73517635e44dfe0a6ed8a3207 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Tue, 27 Sep 2022 11:47:39 +0200 Subject: [PATCH] Update week39.do.txt --- doc/src/week39/week39.do.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/src/week39/week39.do.txt b/doc/src/week39/week39.do.txt index 6a9ed5815..1329d9d67 100644 --- a/doc/src/week39/week39.do.txt +++ b/doc/src/week39/week39.do.txt @@ -1508,7 +1508,7 @@ Hessians. Recently, a number of methods have been introduced that accomplish this by tracking not only the gradient, but also the second moment of the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and -ADAM. +"ADAM":"https://arxiv.org/abs/1412.6980". !split ===== RMS prop ===== @@ -1538,9 +1538,9 @@ learning rate for flat directions. !split -===== ADAM optimizer ===== +===== "ADAM optimizer":"https://arxiv.org/abs/1412.6980" ===== -A related algorithm is the ADAM optimizer. In ADAM, we keep a running +A related algorithm is the ADAM optimizer. In "ADAM":"https://arxiv.org/abs/1412.6980", we keep a running average of both the first and second moment of the gradient and use this information to adaptively change the learning rate for different parameters. In addition to keeping a running average of the first and