The idea of the gradient descent algorithm is to update parameters in direction where the cost function decreases goes to a minimum.
In general, the update of some parameters \( \vec \omega \) given a cost function defined by some weights \( \vec \omega \), \( c(x, \vec \omega) \), goes as follows: $$ \vec \omega_{\text{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), $$
for a number of iterations or until $ \big|\big| \vec \omega_{\text{new} } - \vec \omega \big|\big|$ is smaller than some given tolerance.
The value of \( \lambda \) decides how large steps the algorithm must take in the direction of $ \nabla_{\vec \omega} c(x, \vec \omega)$. The notatation \( \nabla_{\vec \omega} \) denotes the gradient with respect to the elements in \( \vec \omega \).