The course gives an introduction to basic concepts of parallel and distributed programming. Practical issues for programming multicore systems, for distributed computing on clusters and for massive parallelism using graphics processing units are also discussed.
The lecture is managed via the learning platform Moodle. All documents and further information can be found in the Moodle course Parallele und verteilte Systeme.
Lecture
Introduction
Multi-core programming
- Parallel programming, multi-core programming, OpenMP
- Communication between threads, synchronization techniques
- Parallel sections, data dependencies, mutexes
Distributed systems
- Efficient parallelization, distributed systems, interprocess communication, MPI
- Send and receive, buffered, (non)blocking, (a)synchronous, (non)local
- Collective communication, broadcasting, scatter and gather, reduction operators, derived data types, communicators and groups
Massively parallel programming
- Data parallelism on GPUs, GPGPU, OpenCL, model views
- Device information, read kernel, parameter passing, timing, compiler messages, watchdog timers, multidimensional grids, memory optimization
- Parallel reduction, memory organization, splitting kernel calls, vector data types, additional library features, hybrid parallel systems
Outlook C++23, summary
Modeling and verification
- Petri nets, mathematical representation, modeling distributed systems, formal analysis, special cases, generalization
- (Linear) temporal logic, limits of efficient parallelization
Exercise
The course is managed using the learning platform Moodle. All documents and further information can be found in the Moodle course Parallele und verteilte Systeme WiSe 2026/27.
The self-enrollment for this Moodle course begins in Oktober,
and the first exercise class takes place on 23th of October (Friday) 2026.
