Huber Error| Loss Functions
We already know that Mean Square Error(MSE) is greater for learning the outliers in the dataset, on the other hand, Mean Absolute Error(MAE) is good to ignore the outliers. But in some cases, the data which looks like outliers not bothered and also those points should not get high priority. Here where Huber Loss comes in.
Huber Loss = Combination of both MSE and MAE
Huber loss is both MSE and MAE means it is quadratic(MSE) when the error is small else MAE. Here delta is the hyperparameter to define the range for MAE and MSE which can be iterative to make sure the correct delta value.
From the equation, We find that when the error is less than delta, the error is quadratic else it is absolute. Below sample code shows the implementation of the Huber loss. Let’s understand this with example.
The image shows the example data I am using to calculate the Huber loss using Linear Regression. As we see in the image, Most of the Y values are +/- 5 to its X value approximately.
I used this small script to find the Huber loss for the sample dataset we have. 2.71 is the loss I got. I gave delta=4 because the difference between most of the inputs is below 5. Now I am adding data to the set having a higher difference between X and Y.
I have added the last three values in the image which deliberately says outliers. Noe lets calculate the Huber loss. It is 3.15. Even after adding some big outliers, Huber loss not tilted much. Still, we can say it stays neutral for all range of values.
When to use HuberLoss:
As said earlier that Huber loss has both MAE and MSE. So when we think higher weightage should not be given to outliers, go for Huber. And one thing we have to define is the delta value. Maybe some iterations needed with the respective algorithm used to find the correct delta value.
That’s all about the Huber Loss for now. Thanks.
