CS410
Parallel and Distributed Computing
Faculty
Ivan Puzyrevskiy
Tech Lead at Yandex
Course length
Duration
Total hours
Credits
Language
Course type
Fee for single course
Fee for degree students
Skills you’ll learn
Overview
Modern computing is all about distribution. Big Data, Machine Learning, Systems – all these new and fancy subjects require us to leverage the power of many available cores and devices. This course aims to provide students with background knowledge in parallel programming and concurrent and distributed computing. Students will learn how to structure, write, run and debug multithreaded applications in Java. More importantly, they will also learn how to reason about the code running in parallel to ensure that the desired safety and liveness properties are present in a scalable way without sacrificing performance. As a teaser, by the end of the course students are introduced to distributed computing.
The course builds on the classical model of computation with a single processor and memory, and iteratively introduces one more bit of complexity at once. First, the course explores the effects of introducing several processors into the computational model while keeping a shared memory. Then, the course explores the effects of replacing the shared memory model with the message-passing model (as in distributed computing). If time permits, the effects of failures on the computational model will be discussed.
On the practical side, the course includes a lot of labs and programming exercises to ensure enough hands-on experience. Performance matters are covered as well, as students will be delivered content on basic performance analysis capabilities.
Learning highlights
- Write a multithreaded application in Java
- Structure the multi-threaded code in a composable and scalable manner
- Understand and formulate the safety and the liveness properties
- Use synchronisation primitives to ensure the correct behaviour of an algorithm
- Identify problems that hinder the performance of a parallel algorithm
- Transform existing programmes to leverage the available parallelism
- Argument when the algorithm will not benefit from the available parallelism
Course outline
15 classes
Class 1
Introduction to the Course
Course syllabus, logistics, grading. Introduction to the concurrency and parallelism. Hardware support. Real-world environments that exhibit parallelism
Class 2
Multithreaded Programming in Java (1)
Threads, interrupts and concurrency effects
Class 3
Multithreaded Programming in Java (2)
Locks, Conditional Variables and synchronization issues
Class 4
Basic Patterns and Techniques
Data/work-parallelism; load balancing and distribution; synchronisation and communication
Class 5
Higher-level Abstractions
Executors and Futures
Class 6
Performance of the Multithreaded Programs
Common pitfalls; tooling to investigate performance issues; basics of systems performance
Class 7
Practical Session & Laboratories
Class 8
Foundations of the Shared Memory Computing
The model; causality; linearizability; lock-free & wait-free computing; fairness; safety and liveness properties
Class 9
Locks & Registers (1)
Different kinds of locks; Different properties of the R/W operations; equivalence and expressiveness
Class 10
Locks & Registers (2)
Different kinds of locks; Different properties of the R/W operations; equivalence and expressiveness
Class 11
Consensus
The definition of consensus; implementation; universality of the consensus
Class 12
Impossibility Results
Consensus numbers; FLP theorem; implications for the real-world applications
Class 13
Practical Session & Laboratories
Class 14
Introduction to the Distributed Computing
The model; happens-before relation; clocks and ordering; communication; registers and consensus in the distributed setting
Class 15
Failures and their Effects
Failure models; Paxos algorithm and its importance; FLP in the distributed setting; fault tolerance in practice
Course materials
Books
Prerequisites
Fluency in Java programming
Basics of discrete math & logic
Methodology
The course consists of four blocks: (1) practical matters of parallel programming in Java, (2) shared-memory computing (practice), (3) shared-memory computing (theory), (4) distributed computing.
The course has a strong focus on the first three blocks that span 12-14 sessions (depending on the class performance). There are 3 to 4 laboratory classes during the course to ensure enough hands-on experience for the students. Besides the labs, there are 1-3 programming assignments per week and 1-2 quizzes per week. Each session has a lecture part and a seminar part, which is used either for demonstrations, for laboratories, or for exercises, depending on the topic.
Grading
Currently Ivan is a technical lead in Yandex, where he supervises and leads the development of Yandex.Travel, an online travel service. Prior to that, he was working for 6+ years on the data storage and processing infrastructure that powers all the services in the company and scales over thousands of machines.
Ivan teaches at Higher School of Economics, where he leads the distributed systems seminar. He also contributed significantly towards the distributed systems programme for the bachelors. Also, Ivan has been teaching at Yandex School of Data Analysis since 2011.
See full profileApply for this course
Parallel and Distributed Computing
by Ivan Puzyrevskiy
Total hours
45 Hours
Dates
May 03 - May 21, 2021
Fee for single course
€1500
Fee for degree students
€750
How to secure your spot
Complete the form below to kickstart your application
Schedule your Harbour.Space interview
If successful, get ready to join us on campus
FAQ
Will I receive a certificate after completion?
Yes. Upon completion of the course, you will receive a certificate signed by the director of the program your course belonged to.
Do I need a visa?
This depends on your case. Please check with the Spanish or Thai consulate in your country of residence about visa requirements. We will do our part to provide you with the necessary documents, such as the Certificate of Enrollment.
Can I get a discount?
Yes. The easiest way to enroll in a course at a discounted price is to register for multiple courses. Registering for multiple courses will reduce the cost per individual course. Please ask the Admissions Office for more information about the other kinds of discounts we offer and what you can do to receive one.