Towards Open Set Deep Networks, Bendale, Boult; 2015 - Summary
author: ywen666
score: 9 / 10

Motivation

Problem It is easy to generate images that humans would never classify as a particular object class, yet networks classify such images high confidence as that given class. The closed set nature of deep networks forces them to choose from one of the known classes leading to such artifacts. However, recognition in the real world is open set, i.e. the recognition system should reject unknown/unseen classes at test time.

The SoftMax layer is a significant component of the problem because of its closed nature. The authors propose an alternative, OpenMax, which extends SoftMax layer by enabling it to predict an unknown class.

OpenMax

Design: “open space risk” should be measured in feature space, rather than in pixel space. The key question: is there a feature space, ideally a layer in the deep network, where these adversarial images are far away from training examples?

Experiments

The authors evaluate their models on ImageNet 2012. During the testing phase, we test the system with all the 1000 categories from ILSVRC 2012 validation set, fooling categories and previously unseen categories. The previously unseen categories are selected from ILSVRC 2010. It has been noted that approximately 360 categories from ILSVRC 2010 were discarded and not used in ILSVRC 2012. The final test set consists of 50K closed set images from ILSVRC 2012, 15K open set images (from the 360 distinct categories from ILSVRC 2010) and 15K fooling images (with 15 images each per ILSVRC 2012 categories).