No account yet ?

Building neural networks, we compute gradients with the backpropagation algorithm.

These gradients are used to perform the parameter updates.

The default way of doing this is to update gradient along the negative gradient direction i.e. using the gradient descent optimizer.

SGD

Momentum update

Nesterov momentum

Learning Rate Decay

Adagrad

RMSprop

Adam

Source : https://kharshit.github.io/blog/2018/08/17/optimizers