IL2230 Hardware Architectures for Deep Learning 7.5 credits

Hårdvaruarkitekturer för djupinlärning

The course focuses on specialized hardware implementation architectures of deep learning algorithms for resource-constrained dependable embedded systems. It also discusses system architectures exploiting deep leaming.

Show course information based on the chosen semester and course offering:

Offering and execution

No offering selected

Select the semester and course offering above to get information from the correct course syllabus and course offering.

Course information

Content and learning outcomes

Course contents *

The course consists of two modules. Module I introduces basic knowledge in machine learning and algorithms for deep learning Module II focuses on specialised hardware implementation architectures for deep learning algorithms and new brain-like computer system architectures. Apart from presenting relevant informative knowledge, the course contains laboratory and project assignments to create understanding of the related algorithms applied to deal with real problems and to contrast and evaluate alternative implementation architectures, in term of performance, cost, and reliability.

Module I: Algorithms for deep learning

Module I introduces basic machine learning algorithms, basic neural network algorithms and algorithms for deep learning. Among a number of machine learning algorithms, this module introduces the algorithms for linear regression, polynomial regression, logistic regression that are fundamental and most relevant for neural networks. For neural networks we consider perceptrons, multi-layer-perceptrons and in particular the back-propagation algorithm. After presenting traditional statistical learning machine learning and neural networks this module further examplifies deep learning algorithms, specifically Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN).

Module II: Architecture specialization for deep learning

Module II examines specialised hardware based implementation architectures for deep learning algorithms. From a broad spectrum of potential hardware architectures the design alternatives, such as GPGPU:s, domain specific processors, FPGA/ASIC-based accelerators are presented, together with their advantages and disadvantages. In particular limitations and design alternatives for using deep learning algorithms in embedded resource constrained systems will be discussed. Furthermore this module will discuss new architectures in deep learning for computer system design such as brain-like computer system architectures. A case study with analysis, evaluation and application of a deep learing architectures will be carried out.

Intended learning outcomes *

After passing the course, the student should be able to

  • describe and explain basic neural networks and deep learning algorithms and their relations
  • explain and justify the hardware design space for deep learning algorithms
  • choose and apply an appropriate deep learning algorithm, to solve real problems with artificial intelligence in embedded systems
  • analyse and evaluate hardware implementation alternatives for deep learning algorithms
  • suggest and justify an implementation architecture for applications with deep learning in embedded resource constrained systems
  • discuss and comment new hardware implementation architectures for deep learning and new brain-like computer system architectures that utilise new devices and new concepts

in order to

  • understand the necessity, importance, and potential of accelerating deep learning algorithms with low power consumption through specialized hardware architecture
  • discuss, suggest and evaluate specialised hardware architectures to implement deep learning algorithms and utilise deep learning concepts in resource constrained reliable systems.

Course Disposition

No information inserted

Literature and preparations

Specific prerequisites *

  • Basic digital technology equivalent to IE1204 Digital Design.
  • Basic knowledge of microprocessor design and instruction execution equivalent to IS1200 Computer Hardware Engineering..
  • Basic knowledge in computer software and programming skills equivalent to DD1345 Introductory Programming and computer science.
  • Knowledge of hardware techniques equivalent to IS2202 Computer System Architecture or IL2225 Hardware design in ASIC and FPGA for embedded systems or IL2236 Architectures for embedded many-core systems.

Recommended prerequisites

No information inserted

Equipment

Student is expected to use an own portable computer during laboratory sessions.

Literature

Information about the course literature will be announced in the course memo.

Examination and completion

Grading scale *

A, B, C, D, E, FX, F

Examination *

  • LAB1 - Laboratory work, 3.0 credits, Grading scale: P, F
  • TEN1 - Written exam, 4.5 credits, Grading scale: A, B, C, D, E, FX, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

The examiner decides, in consultation with KTH's coordinator for disabilities (Funka), about possible adapted examination for students with documented, permanent disabilities. The examiner may permit other examination format for re-examination of individual students.

Opportunity to complete the requirements via supplementary examination

No information inserted

Opportunity to raise an approved grade via renewed examination

No information inserted

Examiner

Zhonghai Lu

Further information

Course web

Further information about the course can be found on the Course web at the link below. Information on the Course web will later be moved to this site.

Course web IL2230

Offered by

EECS/Electrical Engineering

Main field of study *

Computer Science and Engineering, Electrical Engineering

Education cycle *

Second cycle

Add-on studies

No information inserted

Contact

Zhonghai Lu

Ethical approach *

  • All members of a group are responsible for the group's work.
  • In any assessment, every student shall honestly disclose any help received and sources used.
  • In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.

Supplementary information

In this course, the EECS code of honor applies, see: http://www.kth.se/en/eecs/utbildning/hederskodex.