CS 395T - Deep learning seminar - Fall 2017

meets TTh 2:00 - 3:30pm in GSB 2.122

instructor Philipp Krähenbühl
email philkr (at) utexas.edu
office hours by appointment (send email)

TA Nayan Singhal
email nayans (at) cs.utexas.edu
office TA station desk 4, GDC 1st floor
TA hours Wed 4:00-5:00 p.m.

Please use canvas for assignment questions.

Prerequisites

Class overview

This class covers advanced topics in deep learning, ranging from optimization to computer vision, computer graphics and unsupervised feature learning, and touches on deep language models, as well as deep learning for games. This is meant to be a very interactive class for upper level students (MS or PhD). For every class we read two recent research papers (most no older than two years), which we will discuss in class.

Goals of the class

After this class you should be able to

Grading

To map percentages to letter grades we use the following python script

def grade(p):
  from math import floor
  if p < 60: return 'F'
  v = (100-p) * 4 / (40 + 1e-5)
  return chr(ord('A')+int(v)) + ['+','','','-'][int((v-floor(v))*4)]

Schedule

Date Topic Papers Presenters Notes and due dates
Aug 31 Administrative and intro (Linear models)   Philipp  
Sep 5 Gradient based optimization Large-scale machine learning with stochastic gradient descent, Bottou 2010 Philipp  
Sep 7 Deep networks and backpropagation Deep learning, LeCun, Bengio and Hinton 2015

Efficient BackProp, LeCun etal. 1998
Philipp paper selection Th Sep 7, 1am email TA
Sep 12 Dropout and batch normalization Dropout: A Simple Way to Prevent Neural Networks from Overfitting, Srivastava etal. 2014

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, Ioffe etal. 2015
Michael A Griffin
Tongliang Liao

Jialin Wu
Chao-Yuan Wu
project 1 out
Sep 14 Advanced optimization and initialization Adam: A Method for Stochastic Optimization, Kingma and Ba 2015

Data-dependent Initializations of Convolutional Neural Networks, Krähenbühl etal. 2016
Philipp
Erik Lindgren

Santhosh Ramakrishnan
Yunzhi Shi
 
Sep 19 Convolutional Networks for image classification Gradient-based learning applied to document recognition, LeCun etal. 1998 (sec 4+ optional)

ImageNet Classification with Deep Convolutional Neural Networks, Krizhevsky etal. 2012
Keivaun Waugh
Philipp

Chia-Wen Cheng
Farzan Memarian
 
Sep 21 Convolutional Networks for object detection and pixel-wise prediction Rich feature hierarchies for accurate object detection and semantic segmentation, Girshick etal. 2014

Fully Convolutional Networks for Semantic Segmentation, Long etal 2015
Pandian Raju
Jialin Wu

Chandana Amanchi
Amin Anvari
 
Sep 26 Advanced deep network architectures Deep Residual Learning for Image Recognition, He etal. 2016

Densely Connected Convolutional Networks
Chao-Yuan Wu
Brady Zhou

David Gros
Pandian Raju
Project 1 QA
Sep 28 Visualizing deep networks Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, Simonyan etal. 2014

Inverting visual representations with convolutional networks, Dosovitskiy 2016
Farzan Memarian
Santhosh Ramakrishnan

Sheng Cao
Haris Krijestorac
 
Oct 3 Image generation Auto-Encoding Variational Bayes, Klingma etal. 2014

Generative Adversarial Nets, Goodfellow etal. 2014
Darshan Thaker
Zhan Shi

Amin Anvari
Heng-Lu Chang
 
Oct 5 Image manipulation Context Encoders: Feature Learning by Inpainting, Pathak etal. 2016

A Neural Algorithm of Artistic Style, Gatys 2015
Brady Zhou
Paul Choi

Zhan Shi
David Gros
 
Oct 10 Project 1 presentations - Yearbook/Geolocation 8 min per team Taewan Kim
Praful Gupta
Erik Lindgren

Chandana Amanchi
Pandian Raju

Chao-Yuan Wu
Chia-Wen Cheng
Heng-Lu Chang

Diego Garcia-Olano
Farzan Memarian
Amin Anvari

Zhan Shi
Jialin Wu
Qi Lei

Shailee Jain
Vivek Khetan
Chin Wei Yeap

Haris Krijestorac

Tongliang Liao
Project 1 due 1am
Oct 12 Project 1 presentations - Yearbook/Geolocation 8 min per team Keivaun Waugh
Paul Choi
Brady Zhou

Darshan Thaker

Yashwant Marathe
Anikesh Kamath
Guillaume Dardelet

Sheng Cao
Yunzhi Shi
David Gros

Venkata Sailesh Sanampudi
Santhosh Kumar Ramakrishnan
Yasumasa Onoe

Yuchen Cui

Michael Griffin

Oussama Kanawati
Project 2 out
Oct 17 Image translation Colorful Image Colorization, Zhang etal. 2016

Photographic Image Synthesis with Cascaded Refinement Networks, Chen etal. 2017
Guillaume Dardelet
Shailee Jain

Paul Choi
Ousama Kanawati
 
Oct 19 Image manipulation Generative Visual Manipulation on the Natural Image Manifold, Zhu etal. 2016 Yuchen Cui
Michael Griffin
Project 2 flash presentations (2min)
Oct 24 Stereo, Flow Computing the Stereo Matching Cost with a Convolutional Neural Network, Zbontar 2015

FlowNet: Learning Optical Flow with Convolutional Networks, Fischer etal. 2015
Tongliang Liao
Guillaume Dardelet

Chin Wei Yeap
Taewan Kim
 
Oct 26 Monocular depth and normal estimation Depth Map Prediction from a Single Image using a Multi-Scale Deep Network, Eigen etal. 2014

Designing Deep Networks for Surface Normal Estimation, Wang etal. 2015
Taewan Kim
Praful Gupta

Praful Gupta
Vivek Khetan
 
Oct 31 Recurrent models Generating Sequences With Recurrent Neural Networks, Graves 2013

Pixel Recurrent Neural Networks, Oord etal 2016
Anikesh Kamath
Darshan Thaker

Erik Lindgren
Chia-Wen Cheng
 
Nov 2 Language models Sequence to sequence learning with neural networks, Sutskever etal. 2014

Neural machine translation in linear time, Kalchbrenner etal. 2016
Heng-Lu Chang
Chin Wei Yeap

Diego Olano
Yasumasa Onoe
 
Nov 7 Image and language From Captions to Visual Concepts and Back, Rang etal. 2015

Learning to Compose Neural Networks for Question Answering, Andreas etal. 2016
Shailee Jain
Chandana Amanchi

Yasumasa Onoe
Anikesh Kamath
 
Nov 9 Atari games Playing Atari with Deep Reinforcement Learning, Mhin etal. 2013

Human-level control through deep reinforcement learning, Mhin etal. 2015
Venkata Sailesh Sanampud
Yashwant Marathe

Vivek Khetan
Diego Olano
 
Nov 14 Alpha GO Mastering the game of Go with deep neural networks and tree search, Silver etal. 2016 Yashwant Marathe
Sheng Cao
Project 2 - QA
Nov 16 Not RL Learning to act by predicting the future, Dosovitskiy etal. 2016

Evolution Strategies as a Scalable Alternative to Reinforcement Learning, Salimans etal. 2017
Ousama Kanawati
Venkata Sailesh Sanampudi

Qi Lei
Yuchen Cui
 
Nov 21 Data Collection ImageNet: A Large-Scale Hierarchical Image Database, Deng etal. 2009

Microsoft COCO: Common Objects in Context, Lin etal. 2014
Yunzhi Shi
Keivaun Waugh

Haris Krijestorac
Qi Lei
 
Nov 23 No Class Thanksgiving holidays    
Nov 28 Fresh off the press TBD Pandian Raju
Chin Wei Yeap

Chandana Amanchi
Yuchen Cui
 
Nov 30 Fresh off the press TBD Darshan Thaker
Erik Lindgren

Praful Gupta
Brady Zhou
 
Dec 5 Final project presentations 8 min including questions   Project 2 due 1am
Dec 7 Final project presentations 8 min including questions    

Paper assignment

All papers are assigned by a Deep Networks based on student preferences. If you don’t like your paper, please blame deep learning, tensorflow or your preference list (instead of the instructor). The code for the assignment problem will be make available.

Expected workload

Estimates of required effort to pass the class are:

General tips

Tips for reading/reviewing a paper

Tips for presentations

Notes

Syllabus subject to change.