The idea of the gradient descent algorithm is to update parameters in direction where the cost function decreases goes to a minimum.
In general, the update of some parameters \( \hat{\omega} \) given a cost function defined by some weights \( \hat{\omega} \), \( c(x, \hat{\omega}) \), goes as follows: $$ \hat{\omega}_{\mathrm{new} } = \hat{\omega} - \lambda \nabla_{\hat{\omega}} c(x, \hat{\omega}), $$
for a number of iterations or until \( \big|\big| \hat{\omega}_{\mathrm{new} } - \hat{\omega} \big|\big| \) is smaller than some given tolerance.
The value of \( \lambda \) decides how large steps the algorithm must take in the direction of $ \nabla_{\hat{\omega}} c(x, \hat{\omega})$. The notatation \( \nabla_{\hat{\omega}} \) denotes the gradient with respect to the elements in \( \hat{\omega} \).