Distributed Systems

MESSAGE
PASSING

Explaining process communication in distributed computing environments.

Presented By Group Members

Aurangzeb
Aurangzeb
Ahmed Ameen
Ahmed Ameen
Rehan
Rehan
Muneed Iqbal
Muneed Iqbal

Islamia University Bahawalpur

Bahawalnagar Campus

CS DEPT SEMESTER 6th (B)
Introduction

Message passing is a communication method used in parallel and distributed computing. In this approach, a program is divided into multiple independent processes.

Independent Processes

Each process has its own memory and cannot directly access the memory of other processes.

Data Exchange

Processes communicate by sending and receiving messages, improving safety, scalability, and reliability.

Concept Visualized

Process A Memory A Process B Memory B Message
Message Passing Model

The message passing model defines how independent processes interact in a system without shared memory. Each process executes separately and communicates explicitly through messages.

Reduces race conditions & data inconsistency

Architecture View

MESSAGE INTERCONNECT Proc 1 Memory Proc 2 Memory Proc 3 Memory

This model is commonly used in distributed systems and high-performance computing.

Communication Types

Communication in the message passing model is classified based on how processes exchange data.

Point-to-Point Communication

Involves direct interaction between two processes.

ONE-TO-ONE

Direct Transfer

One process sends a message to exactly one receiving process. This method is used when precise and direct communication is required between tasks.

P1 P2

Collective Communication

Involves multiple processes participating in communication together.

One-to-Many

BROADCAST

A single process sends the same message to multiple processes. Commonly used to distribute data or instructions.

P1 P2 P3 P4
P1 P2 P3 P4

Many-to-One

GATHER

Multiple processes send data to a single process. Often used to collect results from different tasks.

MPI Basics

Message Passing Interface

MPI is a standardized specification for implementing message passing. It is not a programming language but a set of rules and functions.

C C++ Fortran Python

Portable • Efficient • Scalable

It provides mechanisms to initialize communication, send and receive messages, and terminate execution.

Initialize

Start communication environment

Transfer

Send and receive messages

Terminate

End execution gracefully

MPI Layer Abstraction

MPI P1 MPI P2 MPI P3 MPI LIBRARY (Communication Layer) PHYSICAL NETWORK INTERCONNECT
History & Versions of MPI

Before MPI, many incompatible message passing systems existed. MPI was introduced in the early 1990s to create a unified standard.

MPI-1

The Foundation

Basic point-to-point and collective communication

MPI-2

Expansion

Added dynamic processes and file input/output

MPI-3

Performance

Improved performance and memory handling

MPI-4

Modern Era

Enhanced scalability and modern hardware support