Parallelisme avancé

This course mainly focus on distributed-memory parallel framework, where each processor (core, CPU, GPU, IoT device) has its own memory space which can not be shared with others.

This course mainly focus on distributed-memory parallel framework, where each processor (core, CPU, GPU, IoT device) has its own memory space which can not be shared with others. This parallel framework is popular as it corresponds to many real application scenarios such as the cluster, the sensor network and IoT.

The course will cover the fundamentals of distributed algorithms, including complexity analysis, typical consensus algorithms, and their guarantees. Students will learn to use parallel programming tools to implement a wide range of parallelization strategies for applications in machine learning (ML) as well as numerical and non-numerical algorithms. In addition to studying and analyzing advanced parallel mechanisms, lab sessions will be provided to familiarize students with GPU cluster usage and the PyTorch machine learning framework.

More details on the skills to be acquired and the evaluation methods can be found on this website.