OpenMP and MPI

Parallel computing is use of multiple resources to solve a computational problem. The prime reason for using parallel computing is the need for speed. Dividing a problem and then distributing it on different processors definitely gives a boost towards solution. The motivation to use parallel computing is to reduce wall clock time and to make solutions to large problems available in feasible time. Making efficient use of local and remote processors, overcoming memory considerations, concurrency are additional reasons for "why parallel".

Of various parallel Programming model, the ones of particular interest in this post are: Thread Model and Message Passing Model.

Thread Model

• A single process has  multiple execution paths.
• Associated with shared memory architectures.
• POSIX threads and OpenMP are implementation of threads.

 

Talking about OpenMP – Open specifications for Multi Processing

It is based on existence of multiple threads in shared memory architecture. Single process consists of multiple threads. The Programming model is explicit and hence programmer has full control over the parallelism. It is an API to implement multi-threading, shared memory parallelism. Data sharing is implicit due to shared memory architecture. It follows a dynamic parallelism, meaning new threads can be spawned dynamically.

Message Passing Model – MPI – Message Passing Interface

• MPI is specification targeted towards distributed memory architectures.
• It consists of set of processes having their own memory space.
• Communication is explicit and is done by sending messages using send and receive.
• Programming model is explicit and programmer has to identify parallelism.
• It does not have dynamic parallelism, i.e the number of processes created is static and new ones cannot be created at runtime.

Illustration demonstrating the difference.