- •Contents
- •List of Figures
- •List of Tables
- •Acknowledgments
- •Introduction to MPI
- •Overview and Goals
- •Background of MPI-1.0
- •Background of MPI-1.1, MPI-1.2, and MPI-2.0
- •Background of MPI-1.3 and MPI-2.1
- •Background of MPI-2.2
- •Who Should Use This Standard?
- •What Platforms Are Targets For Implementation?
- •What Is Included In The Standard?
- •What Is Not Included In The Standard?
- •Organization of this Document
- •MPI Terms and Conventions
- •Document Notation
- •Naming Conventions
- •Semantic Terms
- •Data Types
- •Opaque Objects
- •Array Arguments
- •State
- •Named Constants
- •Choice
- •Addresses
- •Language Binding
- •Deprecated Names and Functions
- •Fortran Binding Issues
- •C Binding Issues
- •C++ Binding Issues
- •Functions and Macros
- •Processes
- •Error Handling
- •Implementation Issues
- •Independence of Basic Runtime Routines
- •Interaction with Signals
- •Examples
- •Point-to-Point Communication
- •Introduction
- •Blocking Send and Receive Operations
- •Blocking Send
- •Message Data
- •Message Envelope
- •Blocking Receive
- •Return Status
- •Passing MPI_STATUS_IGNORE for Status
- •Data Type Matching and Data Conversion
- •Type Matching Rules
- •Type MPI_CHARACTER
- •Data Conversion
- •Communication Modes
- •Semantics of Point-to-Point Communication
- •Buffer Allocation and Usage
- •Nonblocking Communication
- •Communication Request Objects
- •Communication Initiation
- •Communication Completion
- •Semantics of Nonblocking Communications
- •Multiple Completions
- •Non-destructive Test of status
- •Probe and Cancel
- •Persistent Communication Requests
- •Send-Receive
- •Null Processes
- •Datatypes
- •Derived Datatypes
- •Type Constructors with Explicit Addresses
- •Datatype Constructors
- •Subarray Datatype Constructor
- •Distributed Array Datatype Constructor
- •Address and Size Functions
- •Lower-Bound and Upper-Bound Markers
- •Extent and Bounds of Datatypes
- •True Extent of Datatypes
- •Commit and Free
- •Duplicating a Datatype
- •Use of General Datatypes in Communication
- •Correct Use of Addresses
- •Decoding a Datatype
- •Examples
- •Pack and Unpack
- •Canonical MPI_PACK and MPI_UNPACK
- •Collective Communication
- •Introduction and Overview
- •Communicator Argument
- •Applying Collective Operations to Intercommunicators
- •Barrier Synchronization
- •Broadcast
- •Example using MPI_BCAST
- •Gather
- •Examples using MPI_GATHER, MPI_GATHERV
- •Scatter
- •Examples using MPI_SCATTER, MPI_SCATTERV
- •Example using MPI_ALLGATHER
- •All-to-All Scatter/Gather
- •Global Reduction Operations
- •Reduce
- •Signed Characters and Reductions
- •MINLOC and MAXLOC
- •All-Reduce
- •Process-local reduction
- •Reduce-Scatter
- •MPI_REDUCE_SCATTER_BLOCK
- •MPI_REDUCE_SCATTER
- •Scan
- •Inclusive Scan
- •Exclusive Scan
- •Example using MPI_SCAN
- •Correctness
- •Introduction
- •Features Needed to Support Libraries
- •MPI's Support for Libraries
- •Basic Concepts
- •Groups
- •Contexts
- •Intra-Communicators
- •Group Management
- •Group Accessors
- •Group Constructors
- •Group Destructors
- •Communicator Management
- •Communicator Accessors
- •Communicator Constructors
- •Communicator Destructors
- •Motivating Examples
- •Current Practice #1
- •Current Practice #2
- •(Approximate) Current Practice #3
- •Example #4
- •Library Example #1
- •Library Example #2
- •Inter-Communication
- •Inter-communicator Accessors
- •Inter-communicator Operations
- •Inter-Communication Examples
- •Caching
- •Functionality
- •Communicators
- •Windows
- •Datatypes
- •Error Class for Invalid Keyval
- •Attributes Example
- •Naming Objects
- •Formalizing the Loosely Synchronous Model
- •Basic Statements
- •Models of Execution
- •Static communicator allocation
- •Dynamic communicator allocation
- •The General case
- •Process Topologies
- •Introduction
- •Virtual Topologies
- •Embedding in MPI
- •Overview of the Functions
- •Topology Constructors
- •Cartesian Constructor
- •Cartesian Convenience Function: MPI_DIMS_CREATE
- •General (Graph) Constructor
- •Distributed (Graph) Constructor
- •Topology Inquiry Functions
- •Cartesian Shift Coordinates
- •Partitioning of Cartesian structures
- •Low-Level Topology Functions
- •An Application Example
- •MPI Environmental Management
- •Implementation Information
- •Version Inquiries
- •Environmental Inquiries
- •Tag Values
- •Host Rank
- •IO Rank
- •Clock Synchronization
- •Memory Allocation
- •Error Handling
- •Error Handlers for Communicators
- •Error Handlers for Windows
- •Error Handlers for Files
- •Freeing Errorhandlers and Retrieving Error Strings
- •Error Codes and Classes
- •Error Classes, Error Codes, and Error Handlers
- •Timers and Synchronization
- •Startup
- •Allowing User Functions at Process Termination
- •Determining Whether MPI Has Finished
- •Portable MPI Process Startup
- •The Info Object
- •Process Creation and Management
- •Introduction
- •The Dynamic Process Model
- •Starting Processes
- •The Runtime Environment
- •Process Manager Interface
- •Processes in MPI
- •Starting Processes and Establishing Communication
- •Reserved Keys
- •Spawn Example
- •Manager-worker Example, Using MPI_COMM_SPAWN.
- •Establishing Communication
- •Names, Addresses, Ports, and All That
- •Server Routines
- •Client Routines
- •Name Publishing
- •Reserved Key Values
- •Client/Server Examples
- •Ocean/Atmosphere - Relies on Name Publishing
- •Simple Client-Server Example.
- •Other Functionality
- •Universe Size
- •Singleton MPI_INIT
- •MPI_APPNUM
- •Releasing Connections
- •Another Way to Establish MPI Communication
- •One-Sided Communications
- •Introduction
- •Initialization
- •Window Creation
- •Window Attributes
- •Communication Calls
- •Examples
- •Accumulate Functions
- •Synchronization Calls
- •Fence
- •General Active Target Synchronization
- •Lock
- •Assertions
- •Examples
- •Error Handling
- •Error Handlers
- •Error Classes
- •Semantics and Correctness
- •Atomicity
- •Progress
- •Registers and Compiler Optimizations
- •External Interfaces
- •Introduction
- •Generalized Requests
- •Examples
- •Associating Information with Status
- •MPI and Threads
- •General
- •Initialization
- •Introduction
- •File Manipulation
- •Opening a File
- •Closing a File
- •Deleting a File
- •Resizing a File
- •Preallocating Space for a File
- •Querying the Size of a File
- •Querying File Parameters
- •File Info
- •Reserved File Hints
- •File Views
- •Data Access
- •Data Access Routines
- •Positioning
- •Synchronism
- •Coordination
- •Data Access Conventions
- •Data Access with Individual File Pointers
- •Data Access with Shared File Pointers
- •Noncollective Operations
- •Collective Operations
- •Seek
- •Split Collective Data Access Routines
- •File Interoperability
- •Datatypes for File Interoperability
- •Extent Callback
- •Datarep Conversion Functions
- •Matching Data Representations
- •Consistency and Semantics
- •File Consistency
- •Random Access vs. Sequential Files
- •Progress
- •Collective File Operations
- •Type Matching
- •Logical vs. Physical File Layout
- •File Size
- •Examples
- •Asynchronous I/O
- •I/O Error Handling
- •I/O Error Classes
- •Examples
- •Subarray Filetype Constructor
- •Requirements
- •Discussion
- •Logic of the Design
- •Examples
- •MPI Library Implementation
- •Systems with Weak Symbols
- •Systems Without Weak Symbols
- •Complications
- •Multiple Counting
- •Linker Oddities
- •Multiple Levels of Interception
- •Deprecated Functions
- •Deprecated since MPI-2.0
- •Deprecated since MPI-2.2
- •Language Bindings
- •Overview
- •Design
- •C++ Classes for MPI
- •Class Member Functions for MPI
- •Semantics
- •C++ Datatypes
- •Communicators
- •Exceptions
- •Mixed-Language Operability
- •Problems With Fortran Bindings for MPI
- •Problems Due to Strong Typing
- •Problems Due to Data Copying and Sequence Association
- •Special Constants
- •Fortran 90 Derived Types
- •A Problem with Register Optimization
- •Basic Fortran Support
- •Extended Fortran Support
- •The mpi Module
- •No Type Mismatch Problems for Subroutines with Choice Arguments
- •Additional Support for Fortran Numeric Intrinsic Types
- •Language Interoperability
- •Introduction
- •Assumptions
- •Initialization
- •Transfer of Handles
- •Status
- •MPI Opaque Objects
- •Datatypes
- •Callback Functions
- •Error Handlers
- •Reduce Operations
- •Addresses
- •Attributes
- •Extra State
- •Constants
- •Interlanguage Communication
- •Language Bindings Summary
- •Groups, Contexts, Communicators, and Caching Fortran Bindings
- •External Interfaces C++ Bindings
- •Change-Log
- •Bibliography
- •Examples Index
- •MPI Declarations Index
- •MPI Function Index
3.9. PERSISTENT COMMUNICATION REQUESTS |
69 |
Returns ag = false, otherwise. If a receive operation might be canceled then one should call MPI_TEST_CANCELLED rst, to check whether the operation was canceled, before checking on the other elds of the return status.
Advice to users. Cancel can be an expensive operation that should be used only exceptionally. (End of advice to users.)
Advice to implementors. If a send operation uses an \eager" protocol (data is transferred to the receiver before a matching receive is posted), then the cancellation of this send may require communication with the intended receiver in order to free allocated bu ers. On some systems this may require an interrupt to the intended receiver. Note that, while communication may be needed to implement MPI_CANCEL, this is still a local operation, since its completion does not depend on the code executed by other processes. If processing is required on another process, this should be transparent to the application (hence the need for an interrupt and an interrupt handler). (End of advice to implementors.)
3.9 Persistent Communication Requests
Often a communication with the same argument list is repeatedly executed within the inner loop of a parallel computation. In such a situation, it may be possible to optimize the communication by binding the list of communication arguments to a persistent communication request once and, then, repeatedly using the request to initiate and complete messages. The persistent request thus created can be thought of as a communication port or a \half-channel." It does not provide the full functionality of a conventional channel, since there is no binding of the send port to the receive port. This construct allows reduction of the overhead for communication between the process and communication controller, but not of the overhead for communication between one communication controller and another. It is not necessary that messages sent with a persistent request be received by a receive operation using a persistent request, or vice versa.
A persistent communication request is created using one of the ve following calls. These calls involve no communication.
MPI_SEND_INIT(buf, count, datatype, dest, tag, comm, request)
IN |
buf |
initial address of send bu er (choice) |
IN |
count |
number of elements sent (non-negative integer) |
IN |
datatype |
type of each element (handle) |
IN |
dest |
rank of destination (integer) |
IN |
tag |
message tag (integer) |
IN |
comm |
communicator (handle) |
OUT |
request |
communication request (handle) |
int MPI_Send_init(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
70 |
CHAPTER 3. POINT-TO-POINT COMMUNICATION |
1MPI_SEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR)
2<type> BUF(*)
3INTEGER REQUEST, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR
4
5fMPI::Prequest MPI::Comm::Send_init(const void* buf, int count, const
6
7
MPI::Datatype& datatype, int dest, int tag) const (binding deprecated, see Section 15.2) g
8Creates a persistent communication request for a standard mode send operation, and
9binds to it all the arguments of a send operation.
10 |
|
|
|
|
11 |
|
|
|
|
12 |
MPI_BSEND_INIT(buf, count, datatype, dest, tag, comm, request) |
|||
13 |
IN |
buf |
initial address of send bu er (choice) |
|
|
||||
14 |
|
count |
number of elements sent (non-negative integer) |
|
15 |
IN |
|||
|
|
|
||
16 |
IN |
datatype |
type of each element (handle) |
|
17 |
IN |
dest |
rank of destination (integer) |
|
|
||||
18 |
IN |
tag |
message tag (integer) |
|
19 |
||||
|
|
|
||
20 |
IN |
comm |
communicator (handle) |
|
21 |
OUT |
request |
communication request (handle) |
|
|
||||
22
23
int MPI_Bsend_init(void* buf, int count, MPI_Datatype datatype, int dest,
24
int tag, MPI_Comm comm, MPI_Request *request)
25
26MPI_BSEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR)
27<type> BUF(*)
28INTEGER REQUEST, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR
29
fMPI::Prequest MPI::Comm::Bsend_init(const void* buf, int count, const
30
MPI::Datatype& datatype, int dest, int tag) const (binding
31
deprecated, see Section 15.2) g
32
33
34
35
Creates a persistent communication request for a bu ered mode send.
MPI_SSEND_INIT(buf, count, datatype, dest, tag, comm, request)
36
37 |
IN |
buf |
initial address of send bu er (choice) |
|
|
|
|
38 |
IN |
count |
number of elements sent (non-negative integer) |
|
|||
39 |
|
|
type of each element (handle) |
40 |
IN |
datatype |
|
|
|
|
|
41 |
IN |
dest |
rank of destination (integer) |
42 |
IN |
tag |
message tag (integer) |
|
|||
43 |
|
|
communicator (handle) |
44 |
IN |
comm |
|
|
|
|
|
45 |
OUT |
request |
communication request (handle) |
46 |
|
|
|
47 |
int MPI_Ssend_init(void* buf, int count, MPI_Datatype datatype, int dest, |
|
|
48 |
int tag, MPI_Comm comm, MPI_Request *request) |
|
3.9. PERSISTENT COMMUNICATION REQUESTS |
71 |
MPI_SSEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR) <type> BUF(*)
INTEGER COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR
fMPI::Prequest MPI::Comm::Ssend_init(const void* buf, int count, const MPI::Datatype& datatype, int dest, int tag) const (binding deprecated, see Section 15.2) g
Creates a persistent communication object for a synchronous mode send operation.
MPI_RSEND_INIT(buf, count, datatype, dest, tag, comm, request)
IN |
buf |
initial address of send bu er (choice) |
IN |
count |
number of elements sent (non-negative integer) |
IN |
datatype |
type of each element (handle) |
IN |
dest |
rank of destination (integer) |
IN |
tag |
message tag (integer) |
IN |
comm |
communicator (handle) |
OUT |
request |
communication request (handle) |
int MPI_Rsend_init(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request)
MPI_RSEND_INIT(BUF, COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR) <type> BUF(*)
INTEGER COUNT, DATATYPE, DEST, TAG, COMM, REQUEST, IERROR
fMPI::Prequest MPI::Comm::Rsend_init(const void* buf, int count, const MPI::Datatype& datatype, int dest, int tag) const (binding deprecated, see Section 15.2) g
Creates a persistent communication object for a ready mode send operation.
MPI_RECV_INIT(buf, count, datatype, source, tag, comm, request)
OUT |
buf |
initial address of receive bu er (choice) |
IN |
count |
number of elements received (non-negative integer) |
IN |
datatype |
type of each element (handle) |
IN |
source |
rank of source or MPI_ANY_SOURCE (integer) |
IN |
tag |
message tag or MPI_ANY_TAG (integer) |
IN |
comm |
communicator (handle) |
OUT |
request |
communication request (handle) |
int MPI_Recv_init(void* buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Request *request)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
72 |
CHAPTER 3. POINT-TO-POINT COMMUNICATION |
1MPI_RECV_INIT(BUF, COUNT, DATATYPE, SOURCE, TAG, COMM, REQUEST, IERROR)
2<type> BUF(*)
3INTEGER COUNT, DATATYPE, SOURCE, TAG, COMM, REQUEST, IERROR
4
5fMPI::Prequest MPI::Comm::Recv_init(void* buf, int count, const
6
7
MPI::Datatype& datatype, int source, int tag) const (binding deprecated, see Section 15.2) g
8Creates a persistent communication request for a receive operation. The argument buf
9is marked as OUT because the user gives permission to write on the receive bu er by passing
10the argument to MPI_RECV_INIT.
11A persistent communication request is inactive after it was created | no active com-
12munication is attached to the request.
13A communication (send or receive) that uses a persistent request is initiated by the
14function MPI_START.
15
16
17 MPI_START(request)
18
19
20
21
INOUT request |
communication request (handle) |
int MPI_Start(MPI_Request *request)
22MPI_START(REQUEST, IERROR)
23INTEGER REQUEST, IERROR
24
fvoid MPI::Prequest::Start() (binding deprecated, see Section 15.2) g
25
26The argument, request, is a handle returned by one of the previous ve calls. The
27associated request should be inactive. The request becomes active once the call is made.
28If the request is for a send with ready mode, then a matching receive should be posted
29before the call is made. The communication bu er should not be modi ed after the call,
30and until the operation completes.
31The call is local, with similar semantics to the nonblocking communication operations
32described in Section 3.7. That is, a call to MPI_START with a request created by
33MPI_SEND_INIT starts a communication in the same manner as a call to MPI_ISEND; a
34call to MPI_START with a request created by MPI_BSEND_INIT starts a communication
35in the same manner as a call to MPI_IBSEND; and so on.
36
37
38
39
40
41
42
MPI_STARTALL(count, array_of_requests)
IN |
count |
list length (non-negative integer) |
INOUT |
array_of_requests |
array of requests (array of handle) |
int MPI_Startall(int count, MPI_Request *array_of_requests)
43
44MPI_STARTALL(COUNT, ARRAY_OF_REQUESTS, IERROR)
45INTEGER COUNT, ARRAY_OF_REQUESTS(*), IERROR
46 |
fstatic void MPI::Prequest::Startall(int count, |
47 |
48 |
MPI::Prequest array_of_requests[]) (binding deprecated, see |
|