Solving the equations

We can now use the Newton-Raphson method or different variants of the gradient descent family (from plain gradient descent to various stochastic gradient descent approaches) to solve the equations

$$ b \leftarrow b +\eta \frac{\partial C}{\partial b}, $$

and

$$ \boldsymbol{w} \leftarrow \boldsymbol{w} +\eta \frac{\partial C}{\partial \boldsymbol{w}}, $$

where \( \eta \) is our by now well-known learning rate.