Coding 3: Multi-class linear classifier

In this exercise, we will implement a multinomial logistic regression classifier for the CIFAR-10 dataset. The data and setup for the model is similar to the previous coding assignment, but much of the new training code will be different.

Multinomial linear classifier

In [0]:
%matplotlib inline
import matplotlib.pyplot as plt
import torch
import torchvision
import torchvision.transforms as transforms
import time

Function to load CIFAR-10.

In [0]:
def fetch_dataloader(transform=None, batch_size=-1, is_train=True):
    """
    Loads data from disk and returns a data_loader.
    A DataLoader is similar to a list of (image, label) tuples.
    You do not need to fully understand this code to do this assignment, we're happy to explain though.
    """
    data = torchvision.datasets.CIFAR10(root='./data',
                                            train=is_train, download=True, transform=transform)
    batch = len(data) if batch_size is -1 else batch_size
    loader = torch.utils.data.DataLoader(data, batch_size=batch,
                                              shuffle=True, num_workers=4)
      
    return loader

Fetch and preprocess data.

In [0]:
classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

transform = transforms.Compose(
            [transforms.ToTensor(),
             transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

loader_trn = fetch_dataloader(transform, is_train=True)
x_trn, y_trn = iter(loader_trn).next()
print(x_trn.shape, y_trn.shape)

Tensorboard for visualizing the training process.

In [0]:
import torch.utils.tensorboard as tb
%load_ext tensorboard
import tempfile
log_dir = tempfile.mkdtemp()
%tensorboard --logdir {log_dir} --reload_interval 1

Implement the model here.

In [0]:
from torch.nn.parameter import Parameter

class LinearClassifier(torch.nn.Module):
    def __init__(self, input_dim, output_dim):
        """
        Implement this function.
        Initialize your model.
        Hint: define the layers needed for the linear model.
        """ 
        pass

    
    def forward(self, x):
        """
        Implement this function.
        Calculate the classification score.
        Input: 
          x (float tensor N x 3072): input flattened images
        Output:
          y (float tensor N x 10): scores for each class
        """
        pass
    
    
    def predict(self, image):
      """
      Implement this function.
      Predict the class label of the image.
      Input:
        image (float tensor 3 x 32 x 32): the test image
      Output:
        label (long): the class label in range 0 ~ 9
      """
      pass

    
def train_model(model, x_trn, y_trn, learning_rate=0.1, num_iterations=1000):
    """
    Implement this function.
    Train the model. e.g., find the optimal parameter of the linear model.
    Input:
      model (torch.nn.Module): the model to train
      x_trn (float tensor N x 3 x 32 x 32): training images
      y_trn (long tensor N): training labels in range 0 ~ 9
      (optional) learning_rate: hyperprameter of training
      (optional) num_iterations: computational budget of training
    Output:
      None. The train parameters should be stored in model
    """
    # use `logger.add_scalar('loss', loss, global_step=iteration)` to add tensorboard log
    logger = tb.SummaryWriter(log_dir + '/{}'.format(time.strftime('%Y-%m-%d-%H-%M')))
    pass    

Initialize an instance of LinearClassifier and train

In [0]:
model = LinearClassifier(3 * 32 * 32, 10)
num_train_samples = 1000
train_model(model, x_trn[:num_train_samples], y_trn[:num_train_samples])

Validate the trained model

In [0]:
num_val_samples = 100
num_correct = 0
for i in range(num_val_samples):
    pred = model.predict(x_trn[num_train_samples + i])
    print('Prediction: {}, label: {}'.format(pred, y_trn[num_train_samples + i]))
    num_correct += (pred == y_trn[num_train_samples + i])
print('Accuracy: {}'.format(1. * num_correct / num_val_samples))