additional typos
This commit is contained in:
@@ -3436,24 +3436,24 @@ def cost_function(P, x):
|
||||
The idea of the gradient descent algorithm is to update parameters in
|
||||
direction where the cost function decreases goes to a minimum.
|
||||
|
||||
In general, the update of some parameters $\vec \omega$ given a cost
|
||||
function defined by some weights $\vec \omega$, $c(x, \vec \omega)$,
|
||||
In general, the update of some parameters $\hat{\omega}$ given a cost
|
||||
function defined by some weights $\hat{\omega}$, $c(x, \hat{\omega})$,
|
||||
goes as follows:
|
||||
|
||||
!bt
|
||||
\[
|
||||
\vec \omega_{\mathrm{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega),
|
||||
\hat{\omega}_{\mathrm{new} } = \hat{\omega} - \lambda \nabla_{\hat{\omega}} c(x, \hat{\omega}),
|
||||
\]
|
||||
!et
|
||||
|
||||
for a number of iterations or until $ \big|\big| \vec
|
||||
\omega_{\mathrm{new} } - \vec \omega \big|\big|$ is smaller than some
|
||||
for a number of iterations or until $\big|\big| \hat{\omega}_{\mathrm{new} } - \hat{\omega} \big|\big|$
|
||||
is smaller than some
|
||||
given tolerance.
|
||||
|
||||
The value of $\lambda$ decides how large steps the algorithm must take
|
||||
in the direction of $ \nabla_{\vec \omega} c(x, \vec \omega)$. The
|
||||
notatation $\nabla_{\vec \omega}$ denotes the gradient with respect to
|
||||
the elements in $\vec \omega$.
|
||||
in the direction of $ \nabla_{\hat{\omega}} c(x, \hat{\omega})$. The
|
||||
notatation $\nabla_{\hat{\omega}}$ denotes the gradient with respect to
|
||||
the elements in $\hat{\omega}$.
|
||||
|
||||
|
||||
!split
|
||||
@@ -3536,8 +3536,7 @@ The feedforward step is similar to as for the neural netowork, but now consideri
|
||||
The $i$-th neuron at layer $l$ recieves the result
|
||||
$\hat{x}_j^{(l-1),\mathrm{hidden} }$ from the $j$-th neuron at layer
|
||||
$l-1$. The $i$-th neuron at layer $l$ weights all of the elements in
|
||||
$\hat{x}_j^{(l-1),\mathrm{hidden} }$ with a weight vector $\vec
|
||||
w_{i,j}^{(l), \ \mathrm{hidden} }$ with as many weigths as there are
|
||||
$\hat{x}_j^{(l-1),\mathrm{hidden} }$ with a weight vector $w_{i,j}^{(l), \ \mathrm{hidden}}$ with as many weigths as there are
|
||||
elements in$\hat{x}_j^{(l-1),\mathrm{hidden} }$, and adds a bias
|
||||
$b_i^{(l), \ \mathrm{hidden} }$:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user