Update week39.do.txt
This commit is contained in:
@@ -1508,7 +1508,7 @@ Hessians.
|
||||
Recently, a number of methods have been introduced that accomplish
|
||||
this by tracking not only the gradient, but also the second moment of
|
||||
the gradient. These methods include AdaGrad, AdaDelta, Root Mean Squared Propagation (RMS-Prop), and
|
||||
ADAM.
|
||||
"ADAM":"https://arxiv.org/abs/1412.6980".
|
||||
|
||||
!split
|
||||
===== RMS prop =====
|
||||
@@ -1538,9 +1538,9 @@ learning rate for flat directions.
|
||||
|
||||
|
||||
!split
|
||||
===== ADAM optimizer =====
|
||||
===== "ADAM optimizer":"https://arxiv.org/abs/1412.6980" =====
|
||||
|
||||
A related algorithm is the ADAM optimizer. In ADAM, we keep a running
|
||||
A related algorithm is the ADAM optimizer. In "ADAM":"https://arxiv.org/abs/1412.6980", we keep a running
|
||||
average of both the first and second moment of the gradient and use
|
||||
this information to adaptively change the learning rate for different
|
||||
parameters. In addition to keeping a running average of the first and
|
||||
|
||||
Reference in New Issue
Block a user