Class-Balanced Loss Based on Effective Number of Samples, Cui, Jia, Lin, Song, Belongie; 2019 - Summary
author: ishank-arora
score: 9 / 10

What is the core idea?

With large-scale data comes the problem of having only a few classes account for most of the data, while most other classes being under-represented.

Therefore, it is critical to address the problem of tailed data distribution.

In this paper, the authors design a re-weighting scheme that uses the effective number of samples for each class to re-balance the loss.

The key idea is to associate each sample with a small neighboring region instead of a single point.

How is it realized (technically)?

The effective number of samples is defined simply as:

\[E_n = \frac{(1-\beta^n)}{(1-\beta)}\]

where \(n\) is the number of samples and \(\beta \in [0,1)]\) is a hyperparameter.

Consider the data sampling process as a simplified version of random covering

The idea of the authors is to capture the diminishing marginal benfits by using more data points of a class.

CNNs are trained with heavy data augmentations (random cropping, re-scaling, horizontal flipping)

Class-Balanced Loss

Addresses the problem of training imbalanced data by introducing a weighting factor inversely proportional to \(E_n\).

The following loss functions are described in the paper.

How well does the paper perform?