- •Contents
- •List of Figures
- •List of Tables
- •Acknowledgments
- •Introduction to MPI
- •Overview and Goals
- •Background of MPI-1.0
- •Background of MPI-1.1, MPI-1.2, and MPI-2.0
- •Background of MPI-1.3 and MPI-2.1
- •Background of MPI-2.2
- •Who Should Use This Standard?
- •What Platforms Are Targets For Implementation?
- •What Is Included In The Standard?
- •What Is Not Included In The Standard?
- •Organization of this Document
- •MPI Terms and Conventions
- •Document Notation
- •Naming Conventions
- •Semantic Terms
- •Data Types
- •Opaque Objects
- •Array Arguments
- •State
- •Named Constants
- •Choice
- •Addresses
- •Language Binding
- •Deprecated Names and Functions
- •Fortran Binding Issues
- •C Binding Issues
- •C++ Binding Issues
- •Functions and Macros
- •Processes
- •Error Handling
- •Implementation Issues
- •Independence of Basic Runtime Routines
- •Interaction with Signals
- •Examples
- •Point-to-Point Communication
- •Introduction
- •Blocking Send and Receive Operations
- •Blocking Send
- •Message Data
- •Message Envelope
- •Blocking Receive
- •Return Status
- •Passing MPI_STATUS_IGNORE for Status
- •Data Type Matching and Data Conversion
- •Type Matching Rules
- •Type MPI_CHARACTER
- •Data Conversion
- •Communication Modes
- •Semantics of Point-to-Point Communication
- •Buffer Allocation and Usage
- •Nonblocking Communication
- •Communication Request Objects
- •Communication Initiation
- •Communication Completion
- •Semantics of Nonblocking Communications
- •Multiple Completions
- •Non-destructive Test of status
- •Probe and Cancel
- •Persistent Communication Requests
- •Send-Receive
- •Null Processes
- •Datatypes
- •Derived Datatypes
- •Type Constructors with Explicit Addresses
- •Datatype Constructors
- •Subarray Datatype Constructor
- •Distributed Array Datatype Constructor
- •Address and Size Functions
- •Lower-Bound and Upper-Bound Markers
- •Extent and Bounds of Datatypes
- •True Extent of Datatypes
- •Commit and Free
- •Duplicating a Datatype
- •Use of General Datatypes in Communication
- •Correct Use of Addresses
- •Decoding a Datatype
- •Examples
- •Pack and Unpack
- •Canonical MPI_PACK and MPI_UNPACK
- •Collective Communication
- •Introduction and Overview
- •Communicator Argument
- •Applying Collective Operations to Intercommunicators
- •Barrier Synchronization
- •Broadcast
- •Example using MPI_BCAST
- •Gather
- •Examples using MPI_GATHER, MPI_GATHERV
- •Scatter
- •Examples using MPI_SCATTER, MPI_SCATTERV
- •Example using MPI_ALLGATHER
- •All-to-All Scatter/Gather
- •Global Reduction Operations
- •Reduce
- •Signed Characters and Reductions
- •MINLOC and MAXLOC
- •All-Reduce
- •Process-local reduction
- •Reduce-Scatter
- •MPI_REDUCE_SCATTER_BLOCK
- •MPI_REDUCE_SCATTER
- •Scan
- •Inclusive Scan
- •Exclusive Scan
- •Example using MPI_SCAN
- •Correctness
- •Introduction
- •Features Needed to Support Libraries
- •MPI's Support for Libraries
- •Basic Concepts
- •Groups
- •Contexts
- •Intra-Communicators
- •Group Management
- •Group Accessors
- •Group Constructors
- •Group Destructors
- •Communicator Management
- •Communicator Accessors
- •Communicator Constructors
- •Communicator Destructors
- •Motivating Examples
- •Current Practice #1
- •Current Practice #2
- •(Approximate) Current Practice #3
- •Example #4
- •Library Example #1
- •Library Example #2
- •Inter-Communication
- •Inter-communicator Accessors
- •Inter-communicator Operations
- •Inter-Communication Examples
- •Caching
- •Functionality
- •Communicators
- •Windows
- •Datatypes
- •Error Class for Invalid Keyval
- •Attributes Example
- •Naming Objects
- •Formalizing the Loosely Synchronous Model
- •Basic Statements
- •Models of Execution
- •Static communicator allocation
- •Dynamic communicator allocation
- •The General case
- •Process Topologies
- •Introduction
- •Virtual Topologies
- •Embedding in MPI
- •Overview of the Functions
- •Topology Constructors
- •Cartesian Constructor
- •Cartesian Convenience Function: MPI_DIMS_CREATE
- •General (Graph) Constructor
- •Distributed (Graph) Constructor
- •Topology Inquiry Functions
- •Cartesian Shift Coordinates
- •Partitioning of Cartesian structures
- •Low-Level Topology Functions
- •An Application Example
- •MPI Environmental Management
- •Implementation Information
- •Version Inquiries
- •Environmental Inquiries
- •Tag Values
- •Host Rank
- •IO Rank
- •Clock Synchronization
- •Memory Allocation
- •Error Handling
- •Error Handlers for Communicators
- •Error Handlers for Windows
- •Error Handlers for Files
- •Freeing Errorhandlers and Retrieving Error Strings
- •Error Codes and Classes
- •Error Classes, Error Codes, and Error Handlers
- •Timers and Synchronization
- •Startup
- •Allowing User Functions at Process Termination
- •Determining Whether MPI Has Finished
- •Portable MPI Process Startup
- •The Info Object
- •Process Creation and Management
- •Introduction
- •The Dynamic Process Model
- •Starting Processes
- •The Runtime Environment
- •Process Manager Interface
- •Processes in MPI
- •Starting Processes and Establishing Communication
- •Reserved Keys
- •Spawn Example
- •Manager-worker Example, Using MPI_COMM_SPAWN.
- •Establishing Communication
- •Names, Addresses, Ports, and All That
- •Server Routines
- •Client Routines
- •Name Publishing
- •Reserved Key Values
- •Client/Server Examples
- •Ocean/Atmosphere - Relies on Name Publishing
- •Simple Client-Server Example.
- •Other Functionality
- •Universe Size
- •Singleton MPI_INIT
- •MPI_APPNUM
- •Releasing Connections
- •Another Way to Establish MPI Communication
- •One-Sided Communications
- •Introduction
- •Initialization
- •Window Creation
- •Window Attributes
- •Communication Calls
- •Examples
- •Accumulate Functions
- •Synchronization Calls
- •Fence
- •General Active Target Synchronization
- •Lock
- •Assertions
- •Examples
- •Error Handling
- •Error Handlers
- •Error Classes
- •Semantics and Correctness
- •Atomicity
- •Progress
- •Registers and Compiler Optimizations
- •External Interfaces
- •Introduction
- •Generalized Requests
- •Examples
- •Associating Information with Status
- •MPI and Threads
- •General
- •Initialization
- •Introduction
- •File Manipulation
- •Opening a File
- •Closing a File
- •Deleting a File
- •Resizing a File
- •Preallocating Space for a File
- •Querying the Size of a File
- •Querying File Parameters
- •File Info
- •Reserved File Hints
- •File Views
- •Data Access
- •Data Access Routines
- •Positioning
- •Synchronism
- •Coordination
- •Data Access Conventions
- •Data Access with Individual File Pointers
- •Data Access with Shared File Pointers
- •Noncollective Operations
- •Collective Operations
- •Seek
- •Split Collective Data Access Routines
- •File Interoperability
- •Datatypes for File Interoperability
- •Extent Callback
- •Datarep Conversion Functions
- •Matching Data Representations
- •Consistency and Semantics
- •File Consistency
- •Random Access vs. Sequential Files
- •Progress
- •Collective File Operations
- •Type Matching
- •Logical vs. Physical File Layout
- •File Size
- •Examples
- •Asynchronous I/O
- •I/O Error Handling
- •I/O Error Classes
- •Examples
- •Subarray Filetype Constructor
- •Requirements
- •Discussion
- •Logic of the Design
- •Examples
- •MPI Library Implementation
- •Systems with Weak Symbols
- •Systems Without Weak Symbols
- •Complications
- •Multiple Counting
- •Linker Oddities
- •Multiple Levels of Interception
- •Deprecated Functions
- •Deprecated since MPI-2.0
- •Deprecated since MPI-2.2
- •Language Bindings
- •Overview
- •Design
- •C++ Classes for MPI
- •Class Member Functions for MPI
- •Semantics
- •C++ Datatypes
- •Communicators
- •Exceptions
- •Mixed-Language Operability
- •Problems With Fortran Bindings for MPI
- •Problems Due to Strong Typing
- •Problems Due to Data Copying and Sequence Association
- •Special Constants
- •Fortran 90 Derived Types
- •A Problem with Register Optimization
- •Basic Fortran Support
- •Extended Fortran Support
- •The mpi Module
- •No Type Mismatch Problems for Subroutines with Choice Arguments
- •Additional Support for Fortran Numeric Intrinsic Types
- •Language Interoperability
- •Introduction
- •Assumptions
- •Initialization
- •Transfer of Handles
- •Status
- •MPI Opaque Objects
- •Datatypes
- •Callback Functions
- •Error Handlers
- •Reduce Operations
- •Addresses
- •Attributes
- •Extra State
- •Constants
- •Interlanguage Communication
- •Language Bindings Summary
- •Groups, Contexts, Communicators, and Caching Fortran Bindings
- •External Interfaces C++ Bindings
- •Change-Log
- •Bibliography
- •Examples Index
- •MPI Declarations Index
- •MPI Function Index
1
2
3
276 |
CHAPTER 8. MPI ENVIRONMENTAL MANAGEMENT |
...
CALL MPI_FREE_MEM(A, IERR) ! memory is freed
4Since standard Fortran does not support (C-like) pointers, this code is not Fortran 77
5or Fortran 90 code. Some compilers (in particular, at the time of writing, g77 and Fortran
6
7
compilers for Intel) do not support this code.
8Example 8.2 Same example, in C
9 |
float |
(* f)[100][100] ; |
|
||
10 |
/* no memory is allocated */ |
|
|
||
11 |
MPI_Alloc_mem(sizeof(float)*100*100, MPI_INFO_NULL, &f); |
|
|
||
12 |
/* memory allocated */ |
|
|
||
13 |
... |
|
|
|
|
14 |
(*f)[5][3] = 2.71; |
|
|
||
15 |
... |
|
|
|
|
16 |
MPI_Free_mem(f); |
|
|
||
17 |
|
|
18 |
|
|
19 |
8.3 |
Error Handling |
20 |
|
|
21An MPI implementation cannot or may choose not to handle some errors that occur during
22MPI calls. These can include errors that generate exceptions or traps, such as oating point
23errors or access violations. The set of errors that are handled by MPI is implementation-
24dependent. Each such error generates an MPI exception.
25The above text takes precedence over any text on error handling within this document.
26Speci cally, text that states that errors will be handled should be read as may be handled.
27A user can associate error handlers to three types of objects: communicators, windows,
28and les. The speci ed error handling routine will be used for any MPI exception that occurs
29during a call to MPI for the respective object. MPI calls that are not related to any objects
30are considered to be attached to the communicator MPI_COMM_WORLD. The attachment
31of error handlers to objects is purely local: di erent processes may attach di erent error
32handlers to corresponding objects.
33Several prede ned error handlers are available in MPI:
34
35
36
37
MPI_ERRORS_ARE_FATAL The handler, when called, causes the program to abort on all executing processes. This has the same e ect as if MPI_ABORT was called by the process that invoked the handler.
38MPI_ERRORS_RETURN The handler has no e ect other than returning the error code to
39the user.
40
Implementations may provide additional prede ned error handlers and programmers
41
can code their own error handlers.
42
The error handler MPI_ERRORS_ARE_FATAL is associated by default with MPI_COMM-
43
_WORLD after initialization. Thus, if the user chooses not to control error handling, every
44
error that MPI handles is treated as fatal. Since (almost) all MPI calls return an error code,
45
a user may choose to handle errors in its main code, by testing the return code of MPI calls
46
and executing a suitable recovery code when the call was not successful. In this case, the
47
error handler MPI_ERRORS_RETURN will be used. Usually it is more convenient and more
48
8.3. ERROR HANDLING |
277 |
e cient not to test for errors after each MPI call, and have such error handled by a non trivial MPI error handler.
After an error is detected, the state of MPI is unde ned. That is, using a user-de ned error handler, or MPI_ERRORS_RETURN, does not necessarily allow the user to continue to use MPI after an error is detected. The purpose of these error handlers is to allow a user to issue user-de ned error messages and to take actions unrelated to MPI (such as ushing I/O bu ers) before a program exits. An MPI implementation is free to allow MPI to continue after an error but is not required to do so.
1
2
3
4
5
6
7
8
9
Advice to implementors. A good quality implementation will, to the greatest possible extent, circumscribe the impact of an error, so that normal processing can continue after an error handler was invoked. The implementation documentation will provide information on the possible e ect of each class of errors. (End of advice to implementors.)
10
11
12
13
14
15
An MPI error handler is an opaque object, which is accessed by a handle. MPI calls are |
16 |
provided to create new error handlers, to associate error handlers with objects, and to test |
17 |
which error handler is associated with an object. C and C++ have distinct typedefs for |
18 |
user de ned error handling callback functions that accept communicator, le, and window |
19 |
arguments. In Fortran there are three user routines. |
20 |
An error handler object is created by a call to MPI_XXX_CREATE_ERRHANDLER(function, 21
errhandler), where XXX is, respectively, COMM, WIN, or FILE. |
22 |
An error handler is attached to a communicator, window, or le by a call to |
23 |
MPI_XXX_SET_ERRHANDLER. The error handler must be either a prede ned error han- |
24 |
dler, or an error handler that was created by a call to MPI_XXX_CREATE_ERRHANDLER, |
25 |
with matching XXX. The prede ned error handlers MPI_ERRORS_RETURN and |
26 |
MPI_ERRORS_ARE_FATAL can be attached to communicators, windows, and les. In C++, |
27 |
the prede ned error handler MPI::ERRORS_THROW_EXCEPTIONS can also be attached to |
28 |
communicators, windows, and les. |
29 |
The error handler currently associated with a communicator, window, or le can be |
30 |
retrieved by a call to MPI_XXX_GET_ERRHANDLER. |
31 |
The MPI function MPI_ERRHANDLER_FREE can be used to free an error handler that |
32 |
was created by a call to MPI_XXX_CREATE_ERRHANDLER. |
33 |
MPI_fCOMM,WIN,FILEg_GET_ERRHANDLER behave as if a new error handler object |
34 |
is created. That is, once the error handler is no longer needed, MPI_ERRHANDLER_FREE |
35 |
should be called with the error handler returned from MPI_ERRHANDLER_GET or |
36 |
MPI_fCOMM,WIN,FILEg_GET_ERRHANDLER to mark the error handler for deallocation. |
37 |
This provides behavior similar to that of MPI_COMM_GROUP and MPI_GROUP_FREE. |
38 |
|
39 |
Advice to implementors. High-quality implementation should raise an error when an error handler that was created by a call to MPI_XXX_CREATE_ERRHANDLER is attached to an object of the wrong type with a call to MPI_YYY_SET_ERRHANDLER. To do so, it is necessary to maintain, with each error handler, information on the typedef of the associated user function. (End of advice to implementors.)
40
41
42
43
44
45
The syntax for these calls is given below.
46
47
48
278 |
CHAPTER 8. MPI ENVIRONMENTAL MANAGEMENT |
1
2
3
4
5
6
7
8
8.3.1 Error Handlers for Communicators
MPI_COMM_CREATE_ERRHANDLER(function, errhandler)
IN |
function |
user de ned error handling procedure (function) |
OUT |
errhandler |
MPI error handler (handle) |
9int MPI_Comm_create_errhandler(MPI_Comm_errhandler_function *function,
10
11
MPI_Errhandler *errhandler)
12MPI_COMM_CREATE_ERRHANDLER(FUNCTION, ERRHANDLER, IERROR)
13EXTERNAL FUNCTION
14INTEGER ERRHANDLER, IERROR
15
fstatic MPI::Errhandler
16
17
18
MPI::Comm::Create_errhandler(MPI::Comm::Errhandler_function* function) (binding deprecated, see Section 15.2) g
19Creates an error handler that can be attached to communicators. This function is
20identical to MPI_ERRHANDLER_CREATE, whose use is deprecated.
21The user routine should be, in C, a function of type MPI_Comm_errhandler_function, which
22is de ned as
23typedef void MPI_Comm_errhandler_function(MPI_Comm *, int *, ...);
24
The rst argument is the communicator in use. The second is the error code to be
25
returned by the MPI routine that raised the error. If the routine would have returned
26
MPI_ERR_IN_STATUS, it is the error code returned in the status for the request that caused
27
the error handler to be invoked. The remaining arguments are \stdargs" arguments whose
28
number and meaning is implementation-dependent. An implementation should clearly doc-
29
ument these arguments. Addresses are used so that the handler may be written in Fortran.
30
This typedef replaces MPI_Handler_function, whose use is deprecated.
31
In Fortran, the user routine should be of the form:
32
SUBROUTINE COMM_ERRHANDLER_FUNCTION(COMM, ERROR_CODE)
33
INTEGER COMM, ERROR_CODE
34
35In C++, the user routine should be of the form:
36ftypedef void MPI::Comm::Errhandler_function(MPI::Comm &, int *, ...);
37
38
39
(binding deprecated, see Section 15.2) g
Rationale. The variable argument list is provided because it provides an ISO-
40standard hook for providing additional information to the error handler; without this
41hook, ISO C prohibits additional arguments. (End of rationale.)
42
43Advice to users. A newly created communicator inherits the error handler that
44is associated with the \parent" communicator. In particular, the user can specify
45a \global" error handler for all communicators by associating this handler with the
46communicator MPI_COMM_WORLD immediately after initialization. (End of advice to
47users.)
48
8.3. ERROR HANDLING |
279 |
|
MPI_COMM_SET_ERRHANDLER(comm, errhandler) |
||
INOUT |
comm |
communicator (handle) |
IN |
errhandler |
new error handler for communicator (handle) |
int MPI_Comm_set_errhandler(MPI_Comm comm, MPI_Errhandler errhandler)
MPI_COMM_SET_ERRHANDLER(COMM, ERRHANDLER, IERROR)
INTEGER COMM, ERRHANDLER, IERROR
fvoid MPI::Comm::Set_errhandler(const MPI::Errhandler& errhandler) (binding deprecated, see Section 15.2) g
Attaches a new error handler to a communicator. The error handler must be either a prede ned error handler, or an error handler created by a call to
MPI_COMM_CREATE_ERRHANDLER. This call is identical to MPI_ERRHANDLER_SET, whose use is deprecated.
MPI_COMM_GET_ERRHANDLER(comm, errhandler)
IN |
comm |
communicator (handle) |
OUT |
errhandler |
error handler currently associated with communicator |
|
|
(handle) |
int MPI_Comm_get_errhandler(MPI_Comm comm, MPI_Errhandler *errhandler)
MPI_COMM_GET_ERRHANDLER(COMM, ERRHANDLER, IERROR)
INTEGER COMM, ERRHANDLER, IERROR
fMPI::Errhandler MPI::Comm::Get_errhandler() const (binding deprecated, see Section 15.2) g
Retrieves the error handler currently associated with a communicator. This call is identical to MPI_ERRHANDLER_GET, whose use is deprecated.
Example: A library function may register at its entry point the current error handler for a communicator, set its own private error handler for this communicator, and restore before exiting the previous error handler.
8.3.2 Error Handlers for Windows
MPI_WIN_CREATE_ERRHANDLER(function, errhandler)
IN |
function |
user de ned error handling procedure (function) |
OUT |
errhandler |
MPI error handler (handle) |
int MPI_Win_create_errhandler(MPI_Win_errhandler_function *function, MPI_Errhandler *errhandler)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
MPI_WIN_CREATE_ERRHANDLER(FUNCTION, ERRHANDLER, IERROR)
48
1
2
3
4
5
6
280 |
CHAPTER 8. MPI ENVIRONMENTAL MANAGEMENT |
EXTERNAL FUNCTION
INTEGER ERRHANDLER, IERROR
fstatic MPI::Errhandler MPI::Win::Create_errhandler(MPI::Win::Errhandler_function* function) (binding deprecated, see Section 15.2) g
7Creates an error handler that can be attached to a window object. The user routine
8should be, in C, a function of type MPI_Win_errhandler_function which is de ned as
9typedef void MPI_Win_errhandler_function(MPI_Win *, int *, ...);
10
11The rst argument is the window in use, the second is the error code to be returned.
12In Fortran, the user routine should be of the form:
13SUBROUTINE WIN_ERRHANDLER_FUNCTION(WIN, ERROR_CODE)
14INTEGER WIN, ERROR_CODE
15In C++, the user routine should be of the form:
16ftypedef void MPI::Win::Errhandler_function(MPI::Win &, int *, ...);
17 |
|
(binding deprecated, see Section 15.2) g |
||
18 |
|
|||
19 |
|
|
|
|
20 |
MPI_WIN_SET_ERRHANDLER(win, errhandler) |
|||
21 |
||||
|
|
|
||
22 |
INOUT |
win |
window (handle) |
|
23 |
IN |
errhandler |
new error handler for window (handle) |
|
|
||||
24
25
int MPI_Win_set_errhandler(MPI_Win win, MPI_Errhandler errhandler)
26
27MPI_WIN_SET_ERRHANDLER(WIN, ERRHANDLER, IERROR)
28INTEGER WIN, ERRHANDLER, IERROR
38 |
IN |
win |
window (handle) |
39 |
OUT |
errhandler |
error handler currently associated with window (han- |
|
|||
40 |
|
|
dle) |
|
|
|