Network functions. Virtualisation. Kernel bypass technologies (e.g. DPDK) for networks with over 100 gigabits per second. Offloading to Smart Network Interface Cards (SmartNIC). Fast networking with little or no CPU intervention using remote direct memory access (RDMA). Network aspects of machine learning inference using graphics processing units (GPUs). Load estimation and load balancing Request for dispatch and scheduling. Efficient, large-scale machine learning inference. Inference with large language models (LLM).
IK2221 Networked Systems for Machine Learning 7.5 credits
Machine learning inference (model serving) is becoming vital for every aspect of the society. Unfortunately, inference can have very tight latency bounds and requires unsustainable amounts of resources. This course teaches students the most recent techniques for effectively serving machine learning workloads with datacenters.
Choose semester and course offering
Choose semester and course offering to see current information and more about the course, such as course syllabus, study period, and application information.
Application
For course offering
Spring 2025 Start 17 Mar 2025 programme students
Application code
60061
Headings with content from the Course syllabus IK2221 (Spring 2025–) are denoted with an asterisk ( )
Content and learning outcomes
Course contents
Intended learning outcomes
After passing the course, the student should be able to:
- describe Network Functions Virtualization components and how they work together
- configure an example of a service using virtualisation of network functions (NFV)
- in detail explain the important advantages of remote direct memory access (RDMA) and how it operates
- analyse methods for performing network I/O directly to/from graphics processors (GPU)
- in detail, explain methods to improve inference latency
- describe the role of the load balancing for servers
- give examples of and describe current research problems with serving machine learning workloads in data centers
Literature and preparations
Specific prerequisites
Knowledge in advanced Internet technique, 7.5 higher education credits, equivalent completed course IK2215. Knowledge and skills in programming in C++, Java or Python, 6 higher education credits, equivalent completed course DD1310-DD1319/DD1331/DD1337/DD100N/ID1018.
Recommended prerequisites
No information inserted
Equipment
No information inserted
Literature
No information inserted
Examination and completion
If the course is discontinued, students may request to be examined during the following two academic years.
Grading scale
A, B, C, D, E, FX, F
Examination
- PRO1 - Project assignments, 2.5 credits, grading scale: A, B, C, D, E, FX, F
- SEM1 - Paper summaries, 2.5 credits, grading scale: A, B, C, D, E, FX, F
- TEN1 - Written exam, 2.5 credits, grading scale: A, B, C, D, E, FX, F
Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.
The examiner may apply another examination format when re-examining individual students.
Opportunity to complete the requirements via supplementary examination
No information inserted
Opportunity to raise an approved grade via renewed examination
No information inserted
Examiner
No information inserted
Ethical approach
- All members of a group are responsible for the group's work.
- In any assessment, every student shall honestly disclose any help received and sources used.
- In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.
Further information
Course room in Canvas
Registered students find further information about the implementation of the course in the course room in Canvas. A link to the course room can be found under the tab Studies in the Personal menu at the start of the course.
Offered by
Main field of study
Computer Science and Engineering
Education cycle
Second cycle
Add-on studies
No information inserted