update
This commit is contained in:
Morten Hjorth-Jensen
@@ -203,6 +203,10 @@ doconce format html week39.do.txt --html_style=bootstrap --pygments_html_style=d
|
||||
2,
|
||||
None,
|
||||
'adam-optimizer-https-arxiv-org-abs-1412-6980'),
|
||||
('Algorithms and codes for Adagrad, RMSprop and Adam',
|
||||
2,
|
||||
None,
|
||||
'algorithms-and-codes-for-adagrad-rmsprop-and-adam'),
|
||||
('Practical tips', 2, None, 'practical-tips'),
|
||||
('Automatic differentiation',
|
||||
2,
|
||||
@@ -342,7 +346,7 @@ MathJax.Hub.Config({
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs042.html#program-example-for-gradient-descent-with-ridge-regression" style="font-size: 80%;">Program example for gradient descent with Ridge Regression</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs043.html#using-gradient-descent-methods-limitations" style="font-size: 80%;">Using gradient descent methods, limitations</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs044.html#improving-gradient-descent-with-momentum" style="font-size: 80%;">Improving gradient descent with momentum</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs081.html#same-code-but-now-with-momentum-gradient-descent" style="font-size: 80%;">Same code but now with momentum gradient descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs082.html#same-code-but-now-with-momentum-gradient-descent" style="font-size: 80%;">Same code but now with momentum gradient descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs046.html#overview-video-on-stochastic-gradient-descent" style="font-size: 80%;">Overview video on Stochastic Gradient Descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs047.html#batches-and-mini-batches" style="font-size: 80%;">Batches and mini-batches</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs048.html#stochastic-gradient-descent-sgd" style="font-size: 80%;">Stochastic Gradient Descent (SGD)</a></li>
|
||||
@@ -362,28 +366,29 @@ MathJax.Hub.Config({
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs062.html#second-moment-of-the-gradient" style="font-size: 80%;">Second moment of the gradient</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs063.html#rms-prop" style="font-size: 80%;">RMS prop</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs064.html#adam-optimizer-https-arxiv-org-abs-1412-6980" style="font-size: 80%;">"ADAM optimizer":"https://arxiv.org/abs/1412.6980"</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs065.html#practical-tips" style="font-size: 80%;">Practical tips</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs066.html#automatic-differentiation" style="font-size: 80%;">Automatic differentiation</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs067.html#using-autograd" style="font-size: 80%;">Using autograd</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs068.html#autograd-with-more-complicated-functions" style="font-size: 80%;">Autograd with more complicated functions</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs069.html#more-complicated-functions-using-the-elements-of-their-arguments-directly" style="font-size: 80%;">More complicated functions using the elements of their arguments directly</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs070.html#functions-using-mathematical-functions-from-numpy" style="font-size: 80%;">Functions using mathematical functions from Numpy</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs071.html#more-autograd" style="font-size: 80%;">More autograd</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs072.html#and-with-loops" style="font-size: 80%;">And with loops</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs073.html#using-recursion" style="font-size: 80%;">Using recursion</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs074.html#unsupported-functions" style="font-size: 80%;">Unsupported functions</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs075.html#the-syntax-a-dot-b-when-finding-the-dot-product" style="font-size: 80%;">The syntax a.dot(b) when finding the dot product</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs076.html#recommended-to-avoid" style="font-size: 80%;">Recommended to avoid</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs077.html#using-autograd-with-ols" style="font-size: 80%;">Using Autograd with OLS</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs081.html#same-code-but-now-with-momentum-gradient-descent" style="font-size: 80%;">Same code but now with momentum gradient descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs079.html#but-noen-of-these-can-compete-with-newton-s-method" style="font-size: 80%;">But noen of these can compete with Newton's method</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs080.html#including-stochastic-gradient-descent-with-autograd" style="font-size: 80%;">Including Stochastic Gradient Descent with Autograd</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs081.html#same-code-but-now-with-momentum-gradient-descent" style="font-size: 80%;">Same code but now with momentum gradient descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs082.html#similar-second-order-function-now-problem-but-now-with-adagrad" style="font-size: 80%;">Similar (second order function now) problem but now with AdaGrad</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs083.html#rmsprop-for-adaptive-learning-rate-with-stochastic-gradient-descent" style="font-size: 80%;">RMSprop for adaptive learning rate with Stochastic Gradient Descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs084.html#and-logistic-regression" style="font-size: 80%;">And Logistic Regression</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs085.html#introducing-jax-https-jax-readthedocs-io-en-latest" style="font-size: 80%;">Introducing "JAX":"https://jax.readthedocs.io/en/latest/"</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs086.html#weekend-challenge" style="font-size: 80%;">Weekend challenge</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs065.html#algorithms-and-codes-for-adagrad-rmsprop-and-adam" style="font-size: 80%;">Algorithms and codes for Adagrad, RMSprop and Adam</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs066.html#practical-tips" style="font-size: 80%;">Practical tips</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs067.html#automatic-differentiation" style="font-size: 80%;">Automatic differentiation</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs068.html#using-autograd" style="font-size: 80%;">Using autograd</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs069.html#autograd-with-more-complicated-functions" style="font-size: 80%;">Autograd with more complicated functions</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs070.html#more-complicated-functions-using-the-elements-of-their-arguments-directly" style="font-size: 80%;">More complicated functions using the elements of their arguments directly</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs071.html#functions-using-mathematical-functions-from-numpy" style="font-size: 80%;">Functions using mathematical functions from Numpy</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs072.html#more-autograd" style="font-size: 80%;">More autograd</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs073.html#and-with-loops" style="font-size: 80%;">And with loops</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs074.html#using-recursion" style="font-size: 80%;">Using recursion</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs075.html#unsupported-functions" style="font-size: 80%;">Unsupported functions</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs076.html#the-syntax-a-dot-b-when-finding-the-dot-product" style="font-size: 80%;">The syntax a.dot(b) when finding the dot product</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs077.html#recommended-to-avoid" style="font-size: 80%;">Recommended to avoid</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs078.html#using-autograd-with-ols" style="font-size: 80%;">Using Autograd with OLS</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs082.html#same-code-but-now-with-momentum-gradient-descent" style="font-size: 80%;">Same code but now with momentum gradient descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs080.html#but-noen-of-these-can-compete-with-newton-s-method" style="font-size: 80%;">But noen of these can compete with Newton's method</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs081.html#including-stochastic-gradient-descent-with-autograd" style="font-size: 80%;">Including Stochastic Gradient Descent with Autograd</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs082.html#same-code-but-now-with-momentum-gradient-descent" style="font-size: 80%;">Same code but now with momentum gradient descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs083.html#similar-second-order-function-now-problem-but-now-with-adagrad" style="font-size: 80%;">Similar (second order function now) problem but now with AdaGrad</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs084.html#rmsprop-for-adaptive-learning-rate-with-stochastic-gradient-descent" style="font-size: 80%;">RMSprop for adaptive learning rate with Stochastic Gradient Descent</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs085.html#and-logistic-regression" style="font-size: 80%;">And Logistic Regression</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs086.html#introducing-jax-https-jax-readthedocs-io-en-latest" style="font-size: 80%;">Introducing "JAX":"https://jax.readthedocs.io/en/latest/"</a></li>
|
||||
<!-- navigation toc: --> <li><a href="._week39-bs087.html#weekend-challenge" style="font-size: 80%;">Weekend challenge</a></li>
|
||||
|
||||
</ul>
|
||||
</li>
|
||||
@@ -438,7 +443,7 @@ MathJax.Hub.Config({
|
||||
<li><a href="._week39-bs008.html">9</a></li>
|
||||
<li><a href="._week39-bs009.html">10</a></li>
|
||||
<li><a href="">...</a></li>
|
||||
<li><a href="._week39-bs086.html">87</a></li>
|
||||
<li><a href="._week39-bs087.html">88</a></li>
|
||||
<li><a href="._week39-bs001.html">»</a></li>
|
||||
</ul>
|
||||
<!-- ------------------- end of main content --------------- -->
|
||||
|
||||
@@ -2370,10 +2370,17 @@ learning rate for flat directions.
|
||||
<section>
|
||||
<h2 id="adam-optimizer-https-arxiv-org-abs-1412-6980"><a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM optimizer</a> </h2>
|
||||
|
||||
<p>A related algorithm is the ADAM optimizer. In <a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running
|
||||
average of both the first and second moment of the gradient and use
|
||||
this information to adaptively change the learning rate for different
|
||||
parameters. In addition to keeping a running average of the first and
|
||||
<p>A related algorithm is the ADAM optimizer. In
|
||||
<a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running average of
|
||||
both the first and second moment of the gradient and use this
|
||||
information to adaptively change the learning rate for different
|
||||
parameters. The method isefficient when working with large
|
||||
problems involving lots data and/or parameters. It is a combination of the
|
||||
gradient descent with momentum algorithm and the RMSprop algorithm
|
||||
discussed above.
|
||||
</p>
|
||||
|
||||
<p>In addition to keeping a running average of the first and
|
||||
second moments of the gradient
|
||||
(i.e. \( \mathbf{m}_t=\mathbb{E}[\mathbf{g}_t] \) and
|
||||
\( \mathbf{s}_t=\mathbb{E}[\mathbf{g}^2_t] \), respectively), ADAM
|
||||
@@ -2419,6 +2426,14 @@ $$
|
||||
<p> <br>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2 id="algorithms-and-codes-for-adagrad-rmsprop-and-adam">Algorithms and codes for Adagrad, RMSprop and Adam </h2>
|
||||
|
||||
<p>The algorithms we have implemented are well described in the text by <a href="https://www.deeplearningbook.org/contents/optimization.html" target="_blank">Goodfellow, Bengio and Courville, chapter 8</a>.</p>
|
||||
|
||||
<p>The codes which implement these algorithms are discussed after our presentation of automatic differentiation.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2 id="practical-tips">Practical tips </h2>
|
||||
|
||||
|
||||
@@ -230,6 +230,10 @@ div.toc p,a {
|
||||
2,
|
||||
None,
|
||||
'adam-optimizer-https-arxiv-org-abs-1412-6980'),
|
||||
('Algorithms and codes for Adagrad, RMSprop and Adam',
|
||||
2,
|
||||
None,
|
||||
'algorithms-and-codes-for-adagrad-rmsprop-and-adam'),
|
||||
('Practical tips', 2, None, 'practical-tips'),
|
||||
('Automatic differentiation',
|
||||
2,
|
||||
@@ -2318,10 +2322,17 @@ learning rate for flat directions.
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="adam-optimizer-https-arxiv-org-abs-1412-6980"><a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM optimizer</a> </h2>
|
||||
|
||||
<p>A related algorithm is the ADAM optimizer. In <a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running
|
||||
average of both the first and second moment of the gradient and use
|
||||
this information to adaptively change the learning rate for different
|
||||
parameters. In addition to keeping a running average of the first and
|
||||
<p>A related algorithm is the ADAM optimizer. In
|
||||
<a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running average of
|
||||
both the first and second moment of the gradient and use this
|
||||
information to adaptively change the learning rate for different
|
||||
parameters. The method isefficient when working with large
|
||||
problems involving lots data and/or parameters. It is a combination of the
|
||||
gradient descent with momentum algorithm and the RMSprop algorithm
|
||||
discussed above.
|
||||
</p>
|
||||
|
||||
<p>In addition to keeping a running average of the first and
|
||||
second moments of the gradient
|
||||
(i.e. \( \mathbf{m}_t=\mathbb{E}[\mathbf{g}_t] \) and
|
||||
\( \mathbf{s}_t=\mathbb{E}[\mathbf{g}^2_t] \), respectively), ADAM
|
||||
@@ -2363,6 +2374,13 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="algorithms-and-codes-for-adagrad-rmsprop-and-adam">Algorithms and codes for Adagrad, RMSprop and Adam </h2>
|
||||
|
||||
<p>The algorithms we have implemented are well described in the text by <a href="https://www.deeplearningbook.org/contents/optimization.html" target="_blank">Goodfellow, Bengio and Courville, chapter 8</a>.</p>
|
||||
|
||||
<p>The codes which implement these algorithms are discussed after our presentation of automatic differentiation.</p>
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="practical-tips">Practical tips </h2>
|
||||
|
||||
|
||||
@@ -307,6 +307,10 @@ div.toc p,a {
|
||||
2,
|
||||
None,
|
||||
'adam-optimizer-https-arxiv-org-abs-1412-6980'),
|
||||
('Algorithms and codes for Adagrad, RMSprop and Adam',
|
||||
2,
|
||||
None,
|
||||
'algorithms-and-codes-for-adagrad-rmsprop-and-adam'),
|
||||
('Practical tips', 2, None, 'practical-tips'),
|
||||
('Automatic differentiation',
|
||||
2,
|
||||
@@ -2395,10 +2399,17 @@ learning rate for flat directions.
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="adam-optimizer-https-arxiv-org-abs-1412-6980"><a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM optimizer</a> </h2>
|
||||
|
||||
<p>A related algorithm is the ADAM optimizer. In <a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running
|
||||
average of both the first and second moment of the gradient and use
|
||||
this information to adaptively change the learning rate for different
|
||||
parameters. In addition to keeping a running average of the first and
|
||||
<p>A related algorithm is the ADAM optimizer. In
|
||||
<a href="https://arxiv.org/abs/1412.6980" target="_blank">ADAM</a>, we keep a running average of
|
||||
both the first and second moment of the gradient and use this
|
||||
information to adaptively change the learning rate for different
|
||||
parameters. The method isefficient when working with large
|
||||
problems involving lots data and/or parameters. It is a combination of the
|
||||
gradient descent with momentum algorithm and the RMSprop algorithm
|
||||
discussed above.
|
||||
</p>
|
||||
|
||||
<p>In addition to keeping a running average of the first and
|
||||
second moments of the gradient
|
||||
(i.e. \( \mathbf{m}_t=\mathbb{E}[\mathbf{g}_t] \) and
|
||||
\( \mathbf{s}_t=\mathbb{E}[\mathbf{g}^2_t] \), respectively), ADAM
|
||||
@@ -2440,6 +2451,13 @@ $$
|
||||
$$
|
||||
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="algorithms-and-codes-for-adagrad-rmsprop-and-adam">Algorithms and codes for Adagrad, RMSprop and Adam </h2>
|
||||
|
||||
<p>The algorithms we have implemented are well described in the text by <a href="https://www.deeplearningbook.org/contents/optimization.html" target="_blank">Goodfellow, Bengio and Courville, chapter 8</a>.</p>
|
||||
|
||||
<p>The codes which implement these algorithms are discussed after our presentation of automatic differentiation.</p>
|
||||
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
<h2 id="practical-tips">Practical tips </h2>
|
||||
|
||||
|
||||
Binary file not shown.
+305
-285
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,4 @@
|
||||
# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent
|
||||
# Using Autograd to calculate gradients using Adam and Stochastic Gradient descent
|
||||
# OLS example
|
||||
from random import random, seed
|
||||
import numpy as np
|
||||
@@ -32,19 +32,30 @@ theta = np.random.randn(3,1)
|
||||
|
||||
# Value for learning rate
|
||||
eta = 0.01
|
||||
rho1 = 0.9
|
||||
rho2 = 0.99
|
||||
# Including AdaGrad parameter to avoid possible division by zero
|
||||
delta = 1e-8
|
||||
for epoch in range(n_epochs):
|
||||
Giter = np.zeros(shape=(3,3))
|
||||
t = 0
|
||||
s = np.zeros(shape=(3,1))
|
||||
for i in range(m):
|
||||
random_index = M*np.random.randint(m)
|
||||
xi = X[random_index:random_index+M]
|
||||
yi = y[random_index:random_index+M]
|
||||
gradients = (1.0/M)*training_gradient(yi, xi, theta)
|
||||
t += 1
|
||||
Previous = Giter
|
||||
Giter +=gradients @ gradients.T
|
||||
Ginverse = np.c_[eta/(delta+np.sqrt(np.diagonal(Giter)))]
|
||||
update = np.multiply(Ginverse,gradients)
|
||||
theta -= update
|
||||
print("theta from own AdaGrad")
|
||||
Gnew = (rho2*Previous+(1-rho2)*Giter)
|
||||
Gnew = Gnew#/(1.0-rho2*t)
|
||||
Ginverse = np.c_[eta/(delta+np.sqrt(np.diagonal(Gnew)))]
|
||||
s += rho1*s+(1-rho1)*gradients
|
||||
# snew = s/(1.0-rho1*t)
|
||||
theta -= Ginverse.T @ s
|
||||
print("theta from own Adam")
|
||||
print(theta)
|
||||
|
||||
|
||||
from math import sqrt
|
||||
@@ -1,4 +1,4 @@
|
||||
# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent
|
||||
# Using Autograd to calculate gradients using RMSprop and Stochastic Gradient descent
|
||||
# OLS example
|
||||
from random import random, seed
|
||||
import numpy as np
|
||||
@@ -48,5 +48,5 @@ for epoch in range(n_epochs):
|
||||
Ginverse = np.c_[eta/(delta+np.sqrt(np.diagonal(Gnew)))]
|
||||
update = np.multiply(Ginverse,gradients)
|
||||
theta -= update
|
||||
print("theta from own AdaGrad")
|
||||
print("theta from own RMSprop")
|
||||
print(theta)
|
||||
@@ -1676,10 +1676,16 @@ learning rate for flat directions.
|
||||
!split
|
||||
===== "ADAM optimizer":"https://arxiv.org/abs/1412.6980" =====
|
||||
|
||||
A related algorithm is the ADAM optimizer. In "ADAM":"https://arxiv.org/abs/1412.6980", we keep a running
|
||||
average of both the first and second moment of the gradient and use
|
||||
this information to adaptively change the learning rate for different
|
||||
parameters. In addition to keeping a running average of the first and
|
||||
A related algorithm is the ADAM optimizer. In
|
||||
"ADAM":"https://arxiv.org/abs/1412.6980", we keep a running average of
|
||||
both the first and second moment of the gradient and use this
|
||||
information to adaptively change the learning rate for different
|
||||
parameters. The method isefficient when working with large
|
||||
problems involving lots data and/or parameters. It is a combination of the
|
||||
gradient descent with momentum algorithm and the RMSprop algorithm
|
||||
discussed above.
|
||||
|
||||
In addition to keeping a running average of the first and
|
||||
second moments of the gradient
|
||||
(i.e. $\mathbf{m}_t=\mathbb{E}[\mathbf{g}_t]$ and
|
||||
$\mathbf{s}_t=\mathbb{E}[\mathbf{g}^2_t]$, respectively), ADAM
|
||||
@@ -1717,7 +1723,12 @@ update rule for this parameter is given by
|
||||
\]
|
||||
!et
|
||||
|
||||
!split
|
||||
===== Algorithms and codes for Adagrad, RMSprop and Adam =====
|
||||
|
||||
The algorithms we have implemented are well described in the text by "Goodfellow, Bengio and Courville, chapter 8":"https://www.deeplearningbook.org/contents/optimization.html".
|
||||
|
||||
The codes which implement these algorithms are discussed after our presentation of automatic differentiation.
|
||||
|
||||
|
||||
!split
|
||||
|
||||
Reference in New Issue
Block a user