Visa version
Visa
< föregående
|
nästa >
Jämför
< föregående
|
nästa >
Lectures
Here is a provisional list of the course's lecture titles and content. The order in which I present the material may change as may the content. I will make the slides for each lecture available (probably just) before I give the lecture.
Lecture 1
Title: The Deep Learning Revolution
Topics covered: Review of the impact deep networks have had in the application fields of speech, computer vision and NLP. Review of the course's syllabus, Review of the course's assignments, project, written exam and assessment.
Slides: Lecture1_2by2.pdf
Lecture 2
Title: The basic training regime of modern networks (mini-batch gradient descent optimization applied to "Loss + Regularization" cost functions)
Topics covered: Binary SVM classifiers as an unconstrained optimization problem, supervised learning = minimizing loss + regularization, Gradient Descent, SGD, mini-batch optimization, Variants of SGD, multi-class classification with one layer networks, Different loss-functions, Gradient comp
utations for these networks, Chain Rule, Back-propagation
Slides:
Lecture 3
Title: Specification & Training: n-layer fully connected neural networks
Topics covered: Two layer fully connected neural networks, Introduction of non-linearity with activation functions, Back-propagation for n-layer networks, Pros & cons of different activation functions, Trouble with vanishing gradients to training deep networks, Batch normalization
Slides:
Lecture 4
Title: From fully connected networks to Convolutional Networks (ConvNets)
Topics covered: Backprop with Batch Normalization, Trouble of overfitting, Logistics of training:
Training, Validation \& Test sets, Cross-validation, Limitations of fully connected neural networks, Introducing Convolutional Networks
Slides:
Lecture 5
Title: ConvNets Ctd
Topics covered: max-pooling layers, history of ConvNets, Backprop for convolutional and max-pooling layers, viewing the fully connected layers as convolutional layers, Initialization of the parameters, Regularization, Batch normalization again, Evaluation of the models - ensembles
Slides:
Lecture 6
Title: ConvNets in Practice
Topics covered: Specification of architecture (big or many small filters, wide or deep), Training ConvNets in practice, Transfer learning, Distributed training, CPU/GPU bottlenecks
Slides:
Lecture 7
Title: Modern deep ConvNets + Variety of Applications
Topics covered: Review of the modern top performing deep ConvNets - VGG, Inception, ResNet, Object detection with ConvNets, Speech Recognition with ConvNets, Encode - Decode ConvNets (estimating quantities for every pixel)
Slides:
Lecture 8
Title: Recurrent Neural Networks
Topics covered: Coping with sequential data of variable length, Recurrent neural networks, GRUS, LSTMS, Using these for video analysis, RNNs for text synthesis
Slides:
Lecture 9
Title: Incorporating Attention Mechanisms in Deep Networks
Topics covered: Including attention mechanism into neural networks, Describe applications Machine translation, Generating text descriptions of images, Q&A systems
Slides:
Lecture 10
Title: Incorporating Explicit Memory Mechanisms in Deep Networks or Reinforcement Learning
Topics covered:
Slides:
Lecture 11
Title: Generative Adversarial networks (guest lecture)
Topics covered: The hot and very exciting area of generative learning via generative adversarial networks.
Slides: