CS 342 - Neural networks - Fall 2017
meets
TTh 10am-11am GDC 5.302
Fr 1pm-2pm / 2pm-3pm CBA 4.326
instructor Philipp Krähenbühl ( philkr (at) utexas.edu )
office hours Tue 11am-12pm GDC 4.824
Please only visit during office hours or with appointment (or if I didn’t reply your email after the third try)
TA Thomas Crosley ( crosleythomas (at) utexas.edu )
TA hours Thursdays 11:30am-12:30pm, Desk 1 at the UT CS TA Stations
We will use canvas for questions and homework.
Schedule
class section
| Date | Topic | Slides | Corresponding book chapters (optional see below) | Notes and due dates |
|---|---|---|---|---|
| Aug 31 | Introduction | intro | Background: Ch 1-5 | |
| Sep 1 | Installing tensorflow and building the first compute graph | HW1 out | ||
| Sep 5 | Single layer networks | slides visualizations | ||
| Sep 7 | Stochastic gradient descent | slides | 5.9 | HW1 due 11:59pm solution |
| Sep 8 | Logistic regression and MLP | Tutorial | HW2 out | |
| Sep 12 | Multi-layer networks and back propagation | slides | Ch 6 | |
| Sep 14 | Loss functions | slides | HW2 due 11:59pm solution | |
| Sep 15 | Regression vs classification | Section Practice Section Master | HW3 out | |
| Sep 19 | Activation functions + Initialization | slides | Ch 6 | |
| Sep 21 | Convolutions + Pooling | slides | 9.1-9.3 | HW3 due 11:59pm |
| Sep 22 | ConvNets | HW4 out | ||
| Sep 26 | Early stopping, weight regularization, dropout | 7.1-7.3, 7.8, 7.12 | ||
| Sep 28 | Augmentation, batch normalization, parameter sharing, ensembles | 7.4, 7.9, 7.11 | HW4 due 11:59pm | |
| Sep 29 | Regularization and normalization | HW5 out | ||
| Oct 3 | Popular Architectures I: LeNet, AlexNet, VGG | |||
| Oct 5 | Popular Architectures II: GoogLeNet, ResNet, DenseNet | HW5 due 11:59pm | ||
| Oct 6 | ResNets | HW6 out | ||
| Oct 10 | Object detection | |||
| Oct 12 | Segmentation | HW6 due 11:59pm | ||
| Oct 13 | Fully convolutional networks | HW7 out | ||
| Oct 17 | Visualization and understanding | |||
| Oct 19 | Generative models | 20.10, 20.12, 20.13 | HW7 due 11:59pm | |
| Oct 20 | Upconvolution and deep rendering | HW8 out | ||
| Oct 24 | Recurrent Network I: RNN, LSTM, GRU | 10.1-10.11 | ||
| Oct 26 | Recurrent Network II: ConvNets for recurrent tasks | HW8 due 11:59pm | ||
| Oct 27 | Future prediction | HW9 out | ||
| Oct 31 | Deep Reinforcement Learning I: Q-learning and policy gradients | |||
| Nov 2 | Deep Reinforcement Learning II: Actor critic | HW9 due 11:59pm | ||
| Nov 3 | Acting through future prediction | HW10 out | ||
| Nov 7 | Deep Reinforcement Learning III: Direct future prediction | |||
| Nov 9 | Deep Reinforcement Learning IV: Gradient free optimization | HW10 due 11:59pm | ||
| Nov 10 | Gradient free optimization | HW11 out | ||
| Nov 14 | Embedding learning | |||
| Nov 16 | Style transfer | HW11 due 11:59pm | ||
| Nov 17 | Driving in SuperTuxKart | Final project out | ||
| Nov 21 | Language + best results of HW11 | |||
| Nov 23 | no class | |||
| Nov 24 | no class | |||
| Nov 28 | Adversarial examples | 7.13 | ||
| Nov 30 | Negative results | |||
| Dec 1 | Final project Q/A | |||
| Dec 5 | Final project flash presentations | Final project due 11:59pm | ||
| Dec 7 | Final project competition | |||
| Dec 8 | Nagging TA for better grade |
Prerequisites
- 311 or 311H - Discrete math for computer science (or equivalent)
- 343 or 363D - Artificial Intelligence or Statistical Learning and Data Mining
- proficiency in Python
- Bonus: numpy, scipy and matplotlib
Class overview
- The class meets three times a week, twice is a classroom setting, once in a lab setting
- In the classroom setting you’ll learn the theoretical and mathematical background on neural networks and deep learning
- In the lab you’ll learn to use tensorflow and implement the techniques learned in class
Goals of the class
After this class you should be able to
- Implement and train neural networks in tensorflow
- Have a basic understanding of the inner workings of neural networks
- Know several types of neural networks, including convolutional and recurrent neural networks
Grading
- 60% Homework
- 40% Final project
There will be a homework assignment every week, but we will keep them small and manageable. Most of the homework will be graded automatically, you’ll have access to a partial grader while you’re working on the assignment. We will use the same grader, but different test data for your final grade. You’ll submit your homework through canvas.
Late policy:
- 1 day: -25%
- 2 days: -50%
- 3+ days: -100%
Expected workload
Estimates of required effort to pass the class are:
- 3 hours per week: attend class
- 3 hours per week: background reading
- 3 hours per week: coding homework
Course material
The course should be self contained, but if you need additional reading material consult Deep learning, Goodfellow, Bengio and Courville 2016 www.deeplearningbook.org.
Notes
Syllabus subject to change.