diff --git a/doc/src/week39/week39.do.txt b/doc/src/week39/week39.do.txt index 6a9ed5815..1329d9d67 100644 --- a/doc/src/week39/week39.do.txt +++ b/doc/src/week39/week39.do.txt @@ -1508,7 +1508,7 @@ Hessians. Recently, a number of methods have been introduced that accomplish this by tracking not only the gradient, but also the second moment of the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and -ADAM. +"ADAM":"https://arxiv.org/abs/1412.6980". !split ===== RMS prop ===== @@ -1538,9 +1538,9 @@ learning rate for flat directions. !split -===== ADAM optimizer ===== +===== "ADAM optimizer":"https://arxiv.org/abs/1412.6980" ===== -A related algorithm is the ADAM optimizer. In ADAM, we keep a running +A related algorithm is the ADAM optimizer. In "ADAM":"https://arxiv.org/abs/1412.6980", we keep a running average of both the first and second moment of the gradient and use this information to adaptively change the learning rate for different parameters. In addition to keeping a running average of the first and