Parallel and Distributed Systems

The course gives an introduction to basic concepts of parallel and distributed programming. Practical issues for programming multicore systems, for distributed computing on clusters and for massive parallelism using graphics processing units are also discussed.

The lecture is managed via the learning platform Moodle. All documents and further information can be found in the Moodle course Parallele und verteilte Systeme.

Please notice: The materials for our lectures and exercises are only available through the network of the Bauhaus-Universität Weimar.

Lecture

Introduction

  1. Organization, fundamentals, architectures (slides, print)

Multi-core programming

  1. Parallel programming, multi-core programming, OpenMP
  2. Communication between threads, synchronization techniques
  3. Parallel sections, data dependencies, mutexes

Distributed systems

  1. Efficient parallelization, distributed systems, interprocess communication, MPI
  2. Send and receive, buffered, (non)blocking, (a)synchronous, (non)local
  3. Collective communication, broadcasting, scatter and gather, reduction operators, derived data types, communicators and groups

Massively parallel programming

  1. Data parallelism on GPUs, GPGPU, OpenCL, model views
  2. Device information, read kernel, parameter passing, timing, compiler messages, watchdog timers, multidimensional grids, memory optimization
  3. Parallel reduction, memory organization, splitting kernel calls, vector data types, additional library features, hybrid parallel systems
  4. Outlook C++23, summary

Modeling and verification

  1. Petri nets, mathematical representation, modeling distributed systems, formal analysis, special cases, generalization
  2. (Linear) temporal logic, limits of efficient parallelization

Exercise

The course is managed using the learning platform Moodle. All documents and further information can be found in the Moodle course Parallele und verteilte Systeme WiSe 2023. 

The self-enrollment for this Moodle course begins in October,
and the first exercise class takes place on 20th of October 2023.

Exam

Written exam

  • Date: February 16, 2024, at 9:30 am in seminar room 2.16 (Schwanseestraße 143)
  • Auxiliary resources: none

Preparation material