CS 395T - Deep learning seminar - Fall 2016

meets TTh 2:00 - 3:30pm in WAG 308

instructor Philipp Krähenbühl
email philkr (at) utexas.edu
office GDC 4.824
office hours by appointment (send email)

TA Huihuang Zheng
email huihuang@utexas.edu
office GDC 6.802
TA hours We 9:30-10:30am at TA station desk 2

Please use piazza for assignment questions.

Prerequisites

Class overview

This class covers advanced topics in deep learning, ranging from optimization to computer vision, computer graphics and unsupervised feature learning, and touches on deep language models, as well as deep learning for games. This is meant to be a very interactive class for upper level students (MS or PhD). For every class we read two recent research papers (most no older than two years), which we will discuss in class.

Goals of the class

After this class you should be able to

Grading

To map percentages to letter grades we use the following python script

def grade(p):
  from math import floor
  if p < 50: return 'F'
  v = (100-p) * 4 / (50 + 1e-5)
  return chr(ord('A')+int(v)) + ['+','','','-'][int((v-floor(v))*4)]

Schedule

Date Topic Papers Presenters Notes and due dates
Aug 25 Administrative and intro (Linear models)   Philipp slides  
Aug 30 Gradient based optimization Large-scale machine learning with stochastic gradient descent, Bottou 2010 Philipp slides  
Sep 1 Deep networks and backpropagation Deep learning, LeCun, Bengio and Hinton 2015 Philipp slides paper selection Th Sep 1, 6am email TA
Sep 06 Dropout and batch normalization Dropout: A Simple Way to Prevent Neural Networks from Overfitting, Srivastava etal. 2014

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, Ioffe etal. 2015
Dongguang You

Philipp
project 1 out
Sep 08 Advanced optimization and initialization Adam: A Method for Stochastic Optimization, Kingma and Ba 2015

Data-dependent Initializations of Convolutional Neural Networks, Krähenbühl etal. 2016
Rongting Zhang

Philipp
 
Sep 13 Convolutional Networks for image classification Gradient-based learning applied to document recognition, LeCun etal. 1998

ImageNet Classification with Deep Convolutional Neural Networks, Krizhevsky etal. 2012
intro

Xiaoxia Wu

Yinan Zhao
 
Sep 15 Convolutional Networks for object detection Rich feature hierarchies for accurate object detection and semantic segmentation, Girshick etal. 2014

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, Ren etal. 2015
intro

Mit Shah

Zhuode Liu
 
Sep 20 Convolutional networks for pixel-wise prediction Fully Convolutional Networks for Semantic Segmentation, Long etal 2015

Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs, Chen etal. 2015
intro

Nayan Singhal

Louis Ly
Project 1 QA
Sep 22 Advanced deep network architectures Very Deep Convolutional Networks for Large-Scale Image Recognition, Simonyan etal. 2015

Deep Residual Learning for Image Recognition, He etal. 2016
intro

Mit Shah

Yanyao Shen
 
Sep 27 Visualizing deep networks Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, Simonyan etal. 2014

Inverting visual representations with convolutional networks, Dosovitskiy 2016
intro

Zhenpei Yang (2nd)

Ruohan Gao
 
Sep 29 Image manipulation Understanding Deep Image Representations by Inverting Them, Mahendran etal. 2015

A Neural Algorithm of Artistic Style, Gatys 2015
intro

Thomas Czerniawski (2nd)

Soumyajit Gupta
 
Oct 04 Project 1 presentations - Yearbook 10 min per team Mit Shah, Nayan Singha

Jeremy Hintz

Akanksha Saran, Reymundo Gutierrez

Aishwarya Padmakumar, Ashish Bora, Amir Gholaminejad

Julian Sia

Zain Admani

Tushar Nagarajan
Project 1 due 6am
Oct 06 Project 1 presentations - Yearbook/Geolocation 10 min per team Thomas Czerniawski, Bob de Vries, Xiaoxia Wu

Zhuode Liu, Yanyao Shen, Yingying Wu

Long Ly, Soumyajit Gupta, Alex Braylan

Ruohan Gao, Yinan Zhao, Zhenpei Yang

Rongting Zhang

Dongguang You
Project 2 out
Oct 11 Stereo, Flow Computing the Stereo Matching Cost with a Convolutional Neural Network, Zbontar 2015

EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow, Revaud etal 2015
intro

Zain Admani (2nd)

Huihuang Zheng
 
Oct 13 Monocular depth and normal estimation Depth Map Prediction from a Single Image using a Multi-Scale Deep Network, Eigen etal. 2014

Designing Deep Networks for Surface Normal Estimation, Wang etal. 2015
intro

Tushar Nagarajan

Akanksha Saran
 
Oct 18 Image generation Auto-Encoding Variational Bayes, Klingma etal. 2014

Generative Adversarial Nets, Goodfellow etal. 2014
intro

Ashish Bora

Zain Admani
 
Oct 20 Recurrent drawing models Pixel Recurrent Neural Networks, Oord etal 2016 intro

Ashish Bora (2nd)
Project 2 flash presentations (2min)
Oct 25 Auto-encoders Extracting and Composing Robust Features with Denoising Autoencoders, Vincent etal 2008

Context Encoders: Feature Learning by Inpainting, Pathak etal. 2016
intro

Yanyao Shen

Ruohan Gao
 
Oct 27 Self-supervision Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles, Noroozi etal 2016

Colorful Image Colorization, Zhang etal. 2016
intro

Jeremy Hintz

Thomas Czerniawski
 
Nov 01 Recurrent language models Sequence to sequence learning with neural networks, Sutskever etal. 2014

Learning longer memory in recurrent neural networks, Mikolov etal. 2014
intro

Alex Braylan

Bob de Vries
 
Nov 03 Image and language From Captions to Visual Concepts and Back, Rang etal. 2015

Learning to Compose Neural Networks for Question Answering, Andreas etal. 2016
intro

Jeremy Hintz (2nd)

Aishwarya Padmakumar
 
Nov 08 Atari games Playing Atari with Deep Reinforcement Learning, Mhin etal. 2013

Human-level control through deep reinforcement learning, Mhin etal. 2015
intro

Zhenpei Yang

Dongguang You (2nd)
 
Nov 10 Alpha GO Mastering the game of Go with deep neural networks and tree search, Silver etal. 2016 intro

Tushar Nagarajan (2nd)

Project 2 - QA
 
Nov 15 Data Collection ImageNet: A Large-Scale Hierarchical Image Database, Deng etal. 2009

Microsoft COCO: Common Objects in Context, Lin etal. 2014
intro

Julian Sia

Alex Gutierrez
 
Nov 17 Alternative data collection FlowNet: Learning Optical Flow with Convolutional Networks, Fischer etal. 2015

Playing for Data: Ground Truth from Computer Games, Richter etal. 2016
intro

Yingying Wu

Bob de Vries
 
Nov 22 Fresh off the press Learning to act by predicting the future, Dosovitskiy etal. 2016

Neural machine translation in linear time, Kalchbrenner etal. 2016
intro

Nayan Singhal
 
Nov 24 No Class Thanksgiving holidays    
Nov 29 Final project presentations 10 min including questions Ruohan Gao, Zhenpei Yang, Yinan Zhao

Dongguang You, Xiaoxia Wu, Rongting Zhang

Yingying Wu

Tushar Nagarajan, Mit Shah, Nayan Singhal

Julian Sia

Yanyao Shen

Ashish Bora, Aishwarya Padmakumar, Akanksha Saran
Project 2 due 6am
Dec 01 Final project presentations 10 min including questions Alex Braylan, Soumyajit Gupta, Louis Long

Zhuode Liu

Jeremy Hintz

Zain Admani, Reymundo Gutierrez

Bob De Vries

Thomas Czerniawski
 

Paper assignment

All papers are assigned by a Deep Networks based on student preferences. If you don’t like your paper, please blame deep learning, tensorflow or your preference list (instead of the instructor). The code for the assignment problem can be found here: html ipynb.

Expected workload

Estimates of required effort to pass the class are:

General tips

Tips for reading/reviewing a paper

Tips for presentations

Notes

Syllabus subject to change.