author: mmcinnestaylor
score: 9 / 10

Core Idea

Variants

The authors propose a tweaked version of batch normalization, which they call Mean-Only Batch Normalization. They then combine this normalization procedure with weight normalization. The neuron activations are computed using:
\(t = \textbf{w}\cdot\textbf{x},\qquad\tilde{t}=t-\mu[t]+b,\qquad y=\phi(\tilde{t})\)

Performance

Supervised Classification - CIFAR-10

TL;DR