No account yet ?

In machine learning, the difference between the predicted output and the actual output is used to tune the parameters of the algorithm.

This prediction error, called loss, is a crucial part of designing a good model because it evaluates the performance of the model.

For accurate predictions, this loss must be minimized.

In neural networks, this is done using gradient descent.

There are many types of loss functions.

L2 loss (Mean-squared error)

Mean squared error is the most common used loss function for regression loss.

It minimizes the squared difference between the predicted value and the actual value.

The residual sum of squares is defined as :

L1 loss (Mean-absolute error)

L1 loss minimized the sum of absolute error between the predicted value and the actual value.

Hinge loss

The hinge loss is used for classification problems e.g. in case of suppport vector machines.

It is defined as :

Cross-entropy (log loss)

The cross-entropy loss is also used in case of classification problems for estimating the accuracy of model whose output is a probability, p, which lies between 0 and 1.

In case of binary classification, it can be written as :

For multi-class classification problems, it is :

Source : https://kharshit.github.io/blog/2018/08/24/loss-functions