- •Contents
- •List of Figures
- •List of Tables
- •Acknowledgments
- •Introduction to MPI
- •Overview and Goals
- •Background of MPI-1.0
- •Background of MPI-1.1, MPI-1.2, and MPI-2.0
- •Background of MPI-1.3 and MPI-2.1
- •Background of MPI-2.2
- •Who Should Use This Standard?
- •What Platforms Are Targets For Implementation?
- •What Is Included In The Standard?
- •What Is Not Included In The Standard?
- •Organization of this Document
- •MPI Terms and Conventions
- •Document Notation
- •Naming Conventions
- •Semantic Terms
- •Data Types
- •Opaque Objects
- •Array Arguments
- •State
- •Named Constants
- •Choice
- •Addresses
- •Language Binding
- •Deprecated Names and Functions
- •Fortran Binding Issues
- •C Binding Issues
- •C++ Binding Issues
- •Functions and Macros
- •Processes
- •Error Handling
- •Implementation Issues
- •Independence of Basic Runtime Routines
- •Interaction with Signals
- •Examples
- •Point-to-Point Communication
- •Introduction
- •Blocking Send and Receive Operations
- •Blocking Send
- •Message Data
- •Message Envelope
- •Blocking Receive
- •Return Status
- •Passing MPI_STATUS_IGNORE for Status
- •Data Type Matching and Data Conversion
- •Type Matching Rules
- •Type MPI_CHARACTER
- •Data Conversion
- •Communication Modes
- •Semantics of Point-to-Point Communication
- •Buffer Allocation and Usage
- •Nonblocking Communication
- •Communication Request Objects
- •Communication Initiation
- •Communication Completion
- •Semantics of Nonblocking Communications
- •Multiple Completions
- •Non-destructive Test of status
- •Probe and Cancel
- •Persistent Communication Requests
- •Send-Receive
- •Null Processes
- •Datatypes
- •Derived Datatypes
- •Type Constructors with Explicit Addresses
- •Datatype Constructors
- •Subarray Datatype Constructor
- •Distributed Array Datatype Constructor
- •Address and Size Functions
- •Lower-Bound and Upper-Bound Markers
- •Extent and Bounds of Datatypes
- •True Extent of Datatypes
- •Commit and Free
- •Duplicating a Datatype
- •Use of General Datatypes in Communication
- •Correct Use of Addresses
- •Decoding a Datatype
- •Examples
- •Pack and Unpack
- •Canonical MPI_PACK and MPI_UNPACK
- •Collective Communication
- •Introduction and Overview
- •Communicator Argument
- •Applying Collective Operations to Intercommunicators
- •Barrier Synchronization
- •Broadcast
- •Example using MPI_BCAST
- •Gather
- •Examples using MPI_GATHER, MPI_GATHERV
- •Scatter
- •Examples using MPI_SCATTER, MPI_SCATTERV
- •Example using MPI_ALLGATHER
- •All-to-All Scatter/Gather
- •Global Reduction Operations
- •Reduce
- •Signed Characters and Reductions
- •MINLOC and MAXLOC
- •All-Reduce
- •Process-local reduction
- •Reduce-Scatter
- •MPI_REDUCE_SCATTER_BLOCK
- •MPI_REDUCE_SCATTER
- •Scan
- •Inclusive Scan
- •Exclusive Scan
- •Example using MPI_SCAN
- •Correctness
- •Introduction
- •Features Needed to Support Libraries
- •MPI's Support for Libraries
- •Basic Concepts
- •Groups
- •Contexts
- •Intra-Communicators
- •Group Management
- •Group Accessors
- •Group Constructors
- •Group Destructors
- •Communicator Management
- •Communicator Accessors
- •Communicator Constructors
- •Communicator Destructors
- •Motivating Examples
- •Current Practice #1
- •Current Practice #2
- •(Approximate) Current Practice #3
- •Example #4
- •Library Example #1
- •Library Example #2
- •Inter-Communication
- •Inter-communicator Accessors
- •Inter-communicator Operations
- •Inter-Communication Examples
- •Caching
- •Functionality
- •Communicators
- •Windows
- •Datatypes
- •Error Class for Invalid Keyval
- •Attributes Example
- •Naming Objects
- •Formalizing the Loosely Synchronous Model
- •Basic Statements
- •Models of Execution
- •Static communicator allocation
- •Dynamic communicator allocation
- •The General case
- •Process Topologies
- •Introduction
- •Virtual Topologies
- •Embedding in MPI
- •Overview of the Functions
- •Topology Constructors
- •Cartesian Constructor
- •Cartesian Convenience Function: MPI_DIMS_CREATE
- •General (Graph) Constructor
- •Distributed (Graph) Constructor
- •Topology Inquiry Functions
- •Cartesian Shift Coordinates
- •Partitioning of Cartesian structures
- •Low-Level Topology Functions
- •An Application Example
- •MPI Environmental Management
- •Implementation Information
- •Version Inquiries
- •Environmental Inquiries
- •Tag Values
- •Host Rank
- •IO Rank
- •Clock Synchronization
- •Memory Allocation
- •Error Handling
- •Error Handlers for Communicators
- •Error Handlers for Windows
- •Error Handlers for Files
- •Freeing Errorhandlers and Retrieving Error Strings
- •Error Codes and Classes
- •Error Classes, Error Codes, and Error Handlers
- •Timers and Synchronization
- •Startup
- •Allowing User Functions at Process Termination
- •Determining Whether MPI Has Finished
- •Portable MPI Process Startup
- •The Info Object
- •Process Creation and Management
- •Introduction
- •The Dynamic Process Model
- •Starting Processes
- •The Runtime Environment
- •Process Manager Interface
- •Processes in MPI
- •Starting Processes and Establishing Communication
- •Reserved Keys
- •Spawn Example
- •Manager-worker Example, Using MPI_COMM_SPAWN.
- •Establishing Communication
- •Names, Addresses, Ports, and All That
- •Server Routines
- •Client Routines
- •Name Publishing
- •Reserved Key Values
- •Client/Server Examples
- •Ocean/Atmosphere - Relies on Name Publishing
- •Simple Client-Server Example.
- •Other Functionality
- •Universe Size
- •Singleton MPI_INIT
- •MPI_APPNUM
- •Releasing Connections
- •Another Way to Establish MPI Communication
- •One-Sided Communications
- •Introduction
- •Initialization
- •Window Creation
- •Window Attributes
- •Communication Calls
- •Examples
- •Accumulate Functions
- •Synchronization Calls
- •Fence
- •General Active Target Synchronization
- •Lock
- •Assertions
- •Examples
- •Error Handling
- •Error Handlers
- •Error Classes
- •Semantics and Correctness
- •Atomicity
- •Progress
- •Registers and Compiler Optimizations
- •External Interfaces
- •Introduction
- •Generalized Requests
- •Examples
- •Associating Information with Status
- •MPI and Threads
- •General
- •Initialization
- •Introduction
- •File Manipulation
- •Opening a File
- •Closing a File
- •Deleting a File
- •Resizing a File
- •Preallocating Space for a File
- •Querying the Size of a File
- •Querying File Parameters
- •File Info
- •Reserved File Hints
- •File Views
- •Data Access
- •Data Access Routines
- •Positioning
- •Synchronism
- •Coordination
- •Data Access Conventions
- •Data Access with Individual File Pointers
- •Data Access with Shared File Pointers
- •Noncollective Operations
- •Collective Operations
- •Seek
- •Split Collective Data Access Routines
- •File Interoperability
- •Datatypes for File Interoperability
- •Extent Callback
- •Datarep Conversion Functions
- •Matching Data Representations
- •Consistency and Semantics
- •File Consistency
- •Random Access vs. Sequential Files
- •Progress
- •Collective File Operations
- •Type Matching
- •Logical vs. Physical File Layout
- •File Size
- •Examples
- •Asynchronous I/O
- •I/O Error Handling
- •I/O Error Classes
- •Examples
- •Subarray Filetype Constructor
- •Requirements
- •Discussion
- •Logic of the Design
- •Examples
- •MPI Library Implementation
- •Systems with Weak Symbols
- •Systems Without Weak Symbols
- •Complications
- •Multiple Counting
- •Linker Oddities
- •Multiple Levels of Interception
- •Deprecated Functions
- •Deprecated since MPI-2.0
- •Deprecated since MPI-2.2
- •Language Bindings
- •Overview
- •Design
- •C++ Classes for MPI
- •Class Member Functions for MPI
- •Semantics
- •C++ Datatypes
- •Communicators
- •Exceptions
- •Mixed-Language Operability
- •Problems With Fortran Bindings for MPI
- •Problems Due to Strong Typing
- •Problems Due to Data Copying and Sequence Association
- •Special Constants
- •Fortran 90 Derived Types
- •A Problem with Register Optimization
- •Basic Fortran Support
- •Extended Fortran Support
- •The mpi Module
- •No Type Mismatch Problems for Subroutines with Choice Arguments
- •Additional Support for Fortran Numeric Intrinsic Types
- •Language Interoperability
- •Introduction
- •Assumptions
- •Initialization
- •Transfer of Handles
- •Status
- •MPI Opaque Objects
- •Datatypes
- •Callback Functions
- •Error Handlers
- •Reduce Operations
- •Addresses
- •Attributes
- •Extra State
- •Constants
- •Interlanguage Communication
- •Language Bindings Summary
- •Groups, Contexts, Communicators, and Caching Fortran Bindings
- •External Interfaces C++ Bindings
- •Change-Log
- •Bibliography
- •Examples Index
- •MPI Declarations Index
- •MPI Function Index
1
2
318 |
CHAPTER 10. PROCESS CREATION AND MANAGEMENT |
/*
* Parallel code here.
3* The manager is represented as the process with rank 0 in (the remote
4* group of) the parent communicator. If the workers need to communicate
5* among themselves, they can use MPI_COMM_WORLD.
6*/
7
8MPI_Finalize();
9return 0;
10 }
11
12
13
14
15 10.4 Establishing Communication
16
This section provides functions that establish communication between two sets of MPI
17
processes that do not share a communicator.
18
Some situations in which these functions are useful are:
19
20
21
1. Two parts of an application that are started independently need to communicate.
22
23
24
25
2.A visualization tool wants to attach to a running process.
3.A server wants to accept connections from multiple clients. Both clients and server may be parallel programs.
26In each of these situations, MPI must establish communication channels where none existed
27before, and there is no parent/child relationship. The routines described in this section
28establish communication between the two sets of processes by creating an MPI intercom-
29municator, where the two groups of the intercommunicator are the original sets of processes.
30Establishing contact between two groups of processes that do not share an existing
31communicator is a collective but asymmetric process. One group of processes indicates its
32willingness to accept connections from other groups of processes. We will call this group
33the (parallel) server, even if this is not a client/server type of application. The other group
34connects to the server; we will call it the client.
35
36Advice to users. While the names client and server are used throughout this section,
37MPI does not guarantee the traditional robustness of client server systems. The func-
38tionality described in this section is intended to allow two cooperating parts of the
39same application to communicate with one another. For instance, a client that gets a
40segmentation fault and dies, or one that doesn't participate in a collective operation
41may cause a server to crash or hang. (End of advice to users.)
42
43 |
10.4.1 Names, Addresses, Ports, and All That |
|
44
45Almost all of the complexity in MPI client/server routines addresses the question \how
46does the client nd out how to contact the server?" The di culty, of course, is that there
47is no existing communication channel between them, yet they must somehow agree on a
48rendezvous point where they will establish communication.
10.4. ESTABLISHING COMMUNICATION |
319 |
Agreeing on a rendezvous point always involves a third party. The third party may itself provide the rendezvous point or may communicate rendezvous information from server to client. Complicating matters might be the fact that a client doesn't really care what server it contacts, only that it be able to get in touch with one that can handle its request.
Ideally, MPI can accommodate a wide variety of run-time systems while retaining the ability to write simple portable code. The following should be compatible with MPI:
The server resides at a well-known internet address host:port.
The server prints out an address to the terminal, the user gives this address to the client program.
The server places the address information on a nameserver, where it can be retrieved with an agreed-upon name.
The server to which the client connects is actually a broker, acting as a middleman between the client and the real server.
MPI does not require a nameserver, so not all implementations will be able to support all of the above scenarios. However, MPI provides an optional nameserver interface, and is compatible with external name servers.
A port_name is a system-supplied string that encodes a low-level network address at which a server can be contacted. Typically this is an IP address and a port number, but an implementation is free to use any protocol. The server establishes a port_name with the MPI_OPEN_PORT routine. It accepts a connection to a given port with MPI_COMM_ACCEPT. A client uses port_name to connect to the server.
By itself, the port_name mechanism is completely portable, but it may be clumsy to use because of the necessity to communicate port_name to the client. It would be more convenient if a server could specify that it be known by an application-supplied service_name so that the client could connect to that service_name without knowing the port_name.
An MPI implementation may allow the server to publish a (port_name, service_name) pair with MPI_PUBLISH_NAME and the client to retrieve the port name from the service name with MPI_LOOKUP_NAME. This allows three levels of portability, with increasing levels of functionality.
1.Applications that do not rely on the ability to publish names are the most portable. Typically the port_name must be transferred \by hand" from server to client.
2.Applications that use the MPI_PUBLISH_NAME mechanism are completely portable among implementations that provide this service. To be portable among all implementations, these applications should have a fall-back mechanism that can be used when names are not published.
3.Applications may ignore MPI's name publishing functionality and use their own mechanism (possibly system-supplied) to publish names. This allows arbitrary exibility but is not portable.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
1
2
3
4
5
6
7
8
320 |
CHAPTER 10. PROCESS CREATION AND MANAGEMENT |
10.4.2 Server Routines
A server makes itself available with two routines. First it must call MPI_OPEN_PORT to establish a port at which it may be contacted. Secondly it must call MPI_COMM_ACCEPT to accept connections from clients.
MPI_OPEN_PORT(info, port_name)
9
10
11
12
13
14
IN |
info |
implementation-speci c information on how to estab- |
|
|
lish an address (handle) |
OUT |
port_name |
newly established port (string) |
int MPI_Open_port(MPI_Info info, char *port_name)
15MPI_OPEN_PORT(INFO, PORT_NAME, IERROR)
16CHARACTER*(*) PORT_NAME
17INTEGER INFO, IERROR
18fvoid MPI::Open_port(const MPI::Info& info, char* port_name) (binding
19
20
deprecated, see Section 15.2) g
21This function establishes a network address, encoded in the port_name string, at which
22the server will be able to accept connections from clients. port_name is supplied by the
23system, possibly using information in the info argument.
24MPI copies a system-supplied port name into port_name. port_name identi es the newly
25opened port and can be used by a client to contact the server. The maximum size string
26that may be supplied by the system is MPI_MAX_PORT_NAME.
27
28
29
Advice to users. The system copies the port name into port_name. The application must pass a bu er of su cient size to hold this value. (End of advice to users.)
30port_name is essentially a network address. It is unique within the communication
31universe to which it belongs (determined by the implementation), and may be used by any
32client within that communication universe. For instance, if it is an internet (host:port)
33address, it will be unique on the internet. If it is a low level switch address on an IBM SP,
34it will be unique to that SP.
35
36Advice to implementors. These examples are not meant to constrain implementa-
37tions. A port_name could, for instance, contain a user name or the name of a batch
38job, as long as it is unique within some well-de ned communication domain. The
39larger the communication domain, the more useful MPI's client/server functionality
40will be. (End of advice to implementors.)
41
The precise form of the address is implementation-de ned. For instance, an internet address
42
may be a host name or IP address, or anything that the implementation can decode into
43
an IP address. A port name may be reused after it is freed with MPI_CLOSE_PORT and
44
released by the system.
45
46Advice to implementors. Since the user may type in port_name by hand, it is useful
47to choose a form that is easily readable and does not have embedded spaces. (End of
48advice to implementors.)
10.4. ESTABLISHING COMMUNICATION |
321 |
info may be used to tell the implementation how to establish the address. It may, and usually will, be MPI_INFO_NULL in order to get the implementation defaults.
MPI_CLOSE_PORT(port_name)
IN |
port_name |
a port (string) |
int MPI_Close_port(char *port_name)
MPI_CLOSE_PORT(PORT_NAME, IERROR)
CHARACTER*(*) PORT_NAME
INTEGER IERROR
fvoid MPI::Close_port(const char* port_name) (binding deprecated, see Section 15.2) g
This function releases the network address represented by port_name.
MPI_COMM_ACCEPT(port_name, info, root, comm, newcomm)
IN |
port_name |
port name (string, used only on root) |
IN |
info |
implementation-dependent information (handle, used |
|
|
only on root) |
IN |
root |
rank in comm of root node (integer) |
IN |
comm |
intracommunicator over which call is collective (han- |
|
|
dle) |
OUT |
newcomm |
intercommunicator with client as remote group (han- |
|
|
dle) |
int MPI_Comm_accept(char *port_name, MPI_Info info, int root, MPI_Comm comm, MPI_Comm *newcomm)
MPI_COMM_ACCEPT(PORT_NAME, INFO, ROOT, COMM, NEWCOMM, IERROR) CHARACTER*(*) PORT_NAME
INTEGER INFO, ROOT, COMM, NEWCOMM, IERROR
fMPI::Intercomm MPI::Intracomm::Accept(const char* port_name,
const MPI::Info& info, int root) const (binding deprecated, see Section 15.2) g
MPI_COMM_ACCEPT establishes communication with a client. It is collective over the calling communicator. It returns an intercommunicator that allows communication with the client.
The port_name must have been established through a call to MPI_OPEN_PORT.
info is a implementation-de ned string that may allow ne control over the ACCEPT
call.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48