CS 395T - Deep learning seminar - Fall 2016
meets TTh 2:00 - 3:30pm in WAG 308
TA Huihuang Zheng
office GDC 6.802
TA hours We 9:30-10:30am at TA station desk 2
Please use piazza for assignment questions.
- 391L - Intro Machine learning (or equivalent)
- 311 or 311H - Discrete math for computer science (or equivalent)
- proficiency in Python, high level C++ understanding
- All projects are in Python with Caffe, Tensorflow or Theano as recommended deep learning backends. It is also recommended to familiarize yourself with numpy, scipy, scikit-learn and matplotlib as additional libraries.
- NOTE: It is possible to use other languages, but the course staff cannot provide support.
- Basic deep learning background
- You should be familiar with at least one deep learning package (Caffe, Tensorflow, Torch, Matconvnet, …). You should have trained at least one deep network with one of these packages. I’d encourage the use of Caffe, Tensorflow or Theano for projects (if you want help from the course staff), but it is not required.
- The class reads (rates and reviews) two recent research papers per class
- Each paper is presented by one student (two per class), and discussed by entire class.
- Two projects in teams up to 3 students (teams can change between projects)
- Auditing allowed if there is space (no homework or presentation, but participation required)
This class covers advanced topics in deep learning, ranging from optimization to computer vision, computer graphics and unsupervised feature learning, and touches on deep language models, as well as deep learning for games. This is meant to be a very interactive class for upper level students (MS or PhD). For every class we read two recent research papers (most no older than two years), which we will discuss in class.
Goals of the class
After this class you should be able to
- Read, rate and review deep learning papers
- Create and give an interesting presentation on in deep learning
- Devise and execute a research project in deep learning (at the level of a top tier workshop publication: CVPR, ICCV, ICML, NIPS, ACL, SIGGRAPH or equivalent)
- 30% paper presentation
- 30% project 1 (10% presentation, 20% project)
- 40% project 2 (10% presentation, 30% project)
- (optional) 12.5% volunteering for second presentation
To map percentages to letter grades we use the following python script
def grade(p): from math import floor if p < 50: return 'F' v = (100-p) * 4 / (50 + 1e-5) return chr(ord('A')+int(v)) + ['+','','','-'][int((v-floor(v))*4)]
All papers are assigned by a Deep Networks based on student preferences. If you don’t like your paper, please blame deep learning, tensorflow or your preference list (instead of the instructor). The code for the assignment problem can be found here: html ipynb.
Estimates of required effort to pass the class are:
- 2-4 hours per week reading / reviewing papers
- 7 hours per semester (1/2 hour per week) preparing paper presentations
- 2-10 hours per week of programming
- Start the projects early
- most deep neural networks take 1 day to train in a GPU
- let us know early if you don’t have GPU access (first or second week)
- read your assigned papers early and prepare the slides early
- a bad presentation will waste 30 min of your fellow students lives
- you have the option to get feedback on your slides ahead of time from the instructor
Tips for reading/reviewing a paper
- Just reading the paper is not sufficient
- Do more than merely summarizing the paper
- No paper is trivial
- Question any decision and claim made by the authors
- It is the authors responsibility to convince you that their approach works better than prior (or simpler) alternatives
- If a claim is not backed by experiments or a citation (or backed by a wrong citation), you may assume it’s wrong
- Think about how this fits with other peoples findings
- Is there a larger theme across a series of papers?
- Does it contradict other paper you know?
- Use colored markers
- Mark important things in one color
- Mark things you disagree with (you think are wrong) in a different color
Tips for presentations
- Have a story
- Motivate well
- Provide context
- Make the presentation visual
- Use at least one picture/figure/graph per slide
- Walls of bulleted text are unacceptable
- Most mathematical concepts can first be expressed in a figure
- Feel free to make things interactive
- as long as it fits into the time budget
- Show your slides to the instructor a week before the presentation
Syllabus subject to change.