- •Contents
- •List of Figures
- •List of Tables
- •Acknowledgments
- •Introduction to MPI
- •Overview and Goals
- •Background of MPI-1.0
- •Background of MPI-1.1, MPI-1.2, and MPI-2.0
- •Background of MPI-1.3 and MPI-2.1
- •Background of MPI-2.2
- •Who Should Use This Standard?
- •What Platforms Are Targets For Implementation?
- •What Is Included In The Standard?
- •What Is Not Included In The Standard?
- •Organization of this Document
- •MPI Terms and Conventions
- •Document Notation
- •Naming Conventions
- •Semantic Terms
- •Data Types
- •Opaque Objects
- •Array Arguments
- •State
- •Named Constants
- •Choice
- •Addresses
- •Language Binding
- •Deprecated Names and Functions
- •Fortran Binding Issues
- •C Binding Issues
- •C++ Binding Issues
- •Functions and Macros
- •Processes
- •Error Handling
- •Implementation Issues
- •Independence of Basic Runtime Routines
- •Interaction with Signals
- •Examples
- •Point-to-Point Communication
- •Introduction
- •Blocking Send and Receive Operations
- •Blocking Send
- •Message Data
- •Message Envelope
- •Blocking Receive
- •Return Status
- •Passing MPI_STATUS_IGNORE for Status
- •Data Type Matching and Data Conversion
- •Type Matching Rules
- •Type MPI_CHARACTER
- •Data Conversion
- •Communication Modes
- •Semantics of Point-to-Point Communication
- •Buffer Allocation and Usage
- •Nonblocking Communication
- •Communication Request Objects
- •Communication Initiation
- •Communication Completion
- •Semantics of Nonblocking Communications
- •Multiple Completions
- •Non-destructive Test of status
- •Probe and Cancel
- •Persistent Communication Requests
- •Send-Receive
- •Null Processes
- •Datatypes
- •Derived Datatypes
- •Type Constructors with Explicit Addresses
- •Datatype Constructors
- •Subarray Datatype Constructor
- •Distributed Array Datatype Constructor
- •Address and Size Functions
- •Lower-Bound and Upper-Bound Markers
- •Extent and Bounds of Datatypes
- •True Extent of Datatypes
- •Commit and Free
- •Duplicating a Datatype
- •Use of General Datatypes in Communication
- •Correct Use of Addresses
- •Decoding a Datatype
- •Examples
- •Pack and Unpack
- •Canonical MPI_PACK and MPI_UNPACK
- •Collective Communication
- •Introduction and Overview
- •Communicator Argument
- •Applying Collective Operations to Intercommunicators
- •Barrier Synchronization
- •Broadcast
- •Example using MPI_BCAST
- •Gather
- •Examples using MPI_GATHER, MPI_GATHERV
- •Scatter
- •Examples using MPI_SCATTER, MPI_SCATTERV
- •Example using MPI_ALLGATHER
- •All-to-All Scatter/Gather
- •Global Reduction Operations
- •Reduce
- •Signed Characters and Reductions
- •MINLOC and MAXLOC
- •All-Reduce
- •Process-local reduction
- •Reduce-Scatter
- •MPI_REDUCE_SCATTER_BLOCK
- •MPI_REDUCE_SCATTER
- •Scan
- •Inclusive Scan
- •Exclusive Scan
- •Example using MPI_SCAN
- •Correctness
- •Introduction
- •Features Needed to Support Libraries
- •MPI's Support for Libraries
- •Basic Concepts
- •Groups
- •Contexts
- •Intra-Communicators
- •Group Management
- •Group Accessors
- •Group Constructors
- •Group Destructors
- •Communicator Management
- •Communicator Accessors
- •Communicator Constructors
- •Communicator Destructors
- •Motivating Examples
- •Current Practice #1
- •Current Practice #2
- •(Approximate) Current Practice #3
- •Example #4
- •Library Example #1
- •Library Example #2
- •Inter-Communication
- •Inter-communicator Accessors
- •Inter-communicator Operations
- •Inter-Communication Examples
- •Caching
- •Functionality
- •Communicators
- •Windows
- •Datatypes
- •Error Class for Invalid Keyval
- •Attributes Example
- •Naming Objects
- •Formalizing the Loosely Synchronous Model
- •Basic Statements
- •Models of Execution
- •Static communicator allocation
- •Dynamic communicator allocation
- •The General case
- •Process Topologies
- •Introduction
- •Virtual Topologies
- •Embedding in MPI
- •Overview of the Functions
- •Topology Constructors
- •Cartesian Constructor
- •Cartesian Convenience Function: MPI_DIMS_CREATE
- •General (Graph) Constructor
- •Distributed (Graph) Constructor
- •Topology Inquiry Functions
- •Cartesian Shift Coordinates
- •Partitioning of Cartesian structures
- •Low-Level Topology Functions
- •An Application Example
- •MPI Environmental Management
- •Implementation Information
- •Version Inquiries
- •Environmental Inquiries
- •Tag Values
- •Host Rank
- •IO Rank
- •Clock Synchronization
- •Memory Allocation
- •Error Handling
- •Error Handlers for Communicators
- •Error Handlers for Windows
- •Error Handlers for Files
- •Freeing Errorhandlers and Retrieving Error Strings
- •Error Codes and Classes
- •Error Classes, Error Codes, and Error Handlers
- •Timers and Synchronization
- •Startup
- •Allowing User Functions at Process Termination
- •Determining Whether MPI Has Finished
- •Portable MPI Process Startup
- •The Info Object
- •Process Creation and Management
- •Introduction
- •The Dynamic Process Model
- •Starting Processes
- •The Runtime Environment
- •Process Manager Interface
- •Processes in MPI
- •Starting Processes and Establishing Communication
- •Reserved Keys
- •Spawn Example
- •Manager-worker Example, Using MPI_COMM_SPAWN.
- •Establishing Communication
- •Names, Addresses, Ports, and All That
- •Server Routines
- •Client Routines
- •Name Publishing
- •Reserved Key Values
- •Client/Server Examples
- •Ocean/Atmosphere - Relies on Name Publishing
- •Simple Client-Server Example.
- •Other Functionality
- •Universe Size
- •Singleton MPI_INIT
- •MPI_APPNUM
- •Releasing Connections
- •Another Way to Establish MPI Communication
- •One-Sided Communications
- •Introduction
- •Initialization
- •Window Creation
- •Window Attributes
- •Communication Calls
- •Examples
- •Accumulate Functions
- •Synchronization Calls
- •Fence
- •General Active Target Synchronization
- •Lock
- •Assertions
- •Examples
- •Error Handling
- •Error Handlers
- •Error Classes
- •Semantics and Correctness
- •Atomicity
- •Progress
- •Registers and Compiler Optimizations
- •External Interfaces
- •Introduction
- •Generalized Requests
- •Examples
- •Associating Information with Status
- •MPI and Threads
- •General
- •Initialization
- •Introduction
- •File Manipulation
- •Opening a File
- •Closing a File
- •Deleting a File
- •Resizing a File
- •Preallocating Space for a File
- •Querying the Size of a File
- •Querying File Parameters
- •File Info
- •Reserved File Hints
- •File Views
- •Data Access
- •Data Access Routines
- •Positioning
- •Synchronism
- •Coordination
- •Data Access Conventions
- •Data Access with Individual File Pointers
- •Data Access with Shared File Pointers
- •Noncollective Operations
- •Collective Operations
- •Seek
- •Split Collective Data Access Routines
- •File Interoperability
- •Datatypes for File Interoperability
- •Extent Callback
- •Datarep Conversion Functions
- •Matching Data Representations
- •Consistency and Semantics
- •File Consistency
- •Random Access vs. Sequential Files
- •Progress
- •Collective File Operations
- •Type Matching
- •Logical vs. Physical File Layout
- •File Size
- •Examples
- •Asynchronous I/O
- •I/O Error Handling
- •I/O Error Classes
- •Examples
- •Subarray Filetype Constructor
- •Requirements
- •Discussion
- •Logic of the Design
- •Examples
- •MPI Library Implementation
- •Systems with Weak Symbols
- •Systems Without Weak Symbols
- •Complications
- •Multiple Counting
- •Linker Oddities
- •Multiple Levels of Interception
- •Deprecated Functions
- •Deprecated since MPI-2.0
- •Deprecated since MPI-2.2
- •Language Bindings
- •Overview
- •Design
- •C++ Classes for MPI
- •Class Member Functions for MPI
- •Semantics
- •C++ Datatypes
- •Communicators
- •Exceptions
- •Mixed-Language Operability
- •Problems With Fortran Bindings for MPI
- •Problems Due to Strong Typing
- •Problems Due to Data Copying and Sequence Association
- •Special Constants
- •Fortran 90 Derived Types
- •A Problem with Register Optimization
- •Basic Fortran Support
- •Extended Fortran Support
- •The mpi Module
- •No Type Mismatch Problems for Subroutines with Choice Arguments
- •Additional Support for Fortran Numeric Intrinsic Types
- •Language Interoperability
- •Introduction
- •Assumptions
- •Initialization
- •Transfer of Handles
- •Status
- •MPI Opaque Objects
- •Datatypes
- •Callback Functions
- •Error Handlers
- •Reduce Operations
- •Addresses
- •Attributes
- •Extra State
- •Constants
- •Interlanguage Communication
- •Language Bindings Summary
- •Groups, Contexts, Communicators, and Caching Fortran Bindings
- •External Interfaces C++ Bindings
- •Change-Log
- •Bibliography
- •Examples Index
- •MPI Declarations Index
- •MPI Function Index
List of Figures
5.1Collective comminucations, an overview . . . . . . . . . . . . . . . . . . . . 132
5.2Intercommunicator allgather . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.3Intercommunicator reduce-scatter . . . . . . . . . . . . . . . . . . . . . . . . 136
5.4Gather example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.5Gatherv example with strides . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.6 |
Gatherv example, 2-dimensional . . . . . . . . . . . . . . . . . . . . . . . . |
145 |
5.7 |
Gatherv example, 2-dimensional, subarrays with di erent sizes . . . . . . . |
146 |
5.8Gatherv example, 2-dimensional, subarrays with di erent sizes and strides . 147
5.9Scatter example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.10Scatterv example with strides . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.11 |
Scatterv example with di erent strides and counts . . . . . . . . . . . . . . |
154 |
5.12 |
Race conditions with point-to-point and collective communications . . . . . |
185 |
6.1Intercommunicator create using MPI_COMM_CREATE . . . . . . . . . . . . 204
6.2Intercommunicator constructionwith MPI_COMM_SPLIT . . . . . . . . . . . 207
6.3Three-group pipeline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.4Three-group ring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.1Set-up of process structure for two-dimensional parallel Poisson solver. . . . 270
11.1 Active target communication . . . . . . . . . . . . . . . . . . . . . . . . . . 349
11.2Active target communication, with weak synchronization . . . . . . . . . . . 350
11.3Passive target communication . . . . . . . . . . . . . . . . . . . . . . . . . . 351
11.4Active target communication with several processes . . . . . . . . . . . . . . 355
11.5 Schematic description of window . . . . . . . . . . . . . . . . . . . . . . . . 364
11.6Symmetric communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
11.7Deadlock situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
11.8No deadlock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
13.1 |
Etypes and letypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
390 |
13.2 |
Partitioning a le among parallel processes . . . . . . . . . . . . . . . . . . |
390 |
13.3 |
Displacements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
402 |
13.4 |
Example array le layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
450 |
13.5 |
Example local array letype for process 1 . . . . . . . . . . . . . . . . . . . |
451 |
xv
1
2
3
4
5
6
7
8
9
10
11
12
List of Tables
2.1 Deprecated constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . |
17 |
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
3.1 |
Prede ned MPI datatypes corresponding to Fortran datatypes . . . . . . . |
27 |
3.2 |
Prede ned MPI datatypes corresponding to C datatypes . . . . . . . . . . |
28 |
3.3 |
Prede ned MPI datatypes corresponding to both C and Fortran datatypes |
29 |
4.1 |
combiner values returned from MPI_TYPE_GET_ENVELOPE . . . . . . . . |
106 |
6.1MPI_COMM_* Function Behavior (in Inter-Communication Mode) . . . . . 218
8.1Error classes (Part 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
8.2Error classes (Part 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
11.1 Error classes in one-sided communication routines . . . . . . . . . . . . . . 363
13.1Data access routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
13.2\external32" sizes of prede ned datatypes . . . . . . . . . . . . . . . . . . . 433
13.3 I/O Error Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448
2816.1 C++ names for the MPI C and C++ prede ned datatypes . . . . . . . . . . 472
2916.2 C++ names for the MPI Fortran prede ned datatypes . . . . . . . . . . . . 472
3016.3 C++ names for other MPI datatypes . . . . . . . . . . . . . . . . . . . . . . 473
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
xvi