1The type signature associated with sendcounts[j], sendtypes[j] at process i must be equal

2to the type signature associated with recvcounts[i], recvtypes[i] at process j. This implies

3that the amount of data sent must be equal to the amount of data received, pairwise between

4every pair of processes. Distinct type maps between sender and receiver are still allowed.

5The outcome is as if each process sent a message to every other process with

6

7MPI_Send(sendbuf + sdispls[i]; sendcounts[i]; sendtypes[i]; i; :::);

12All arguments on all processes are signi cant. The argument comm must describe the

13same communicator on all processes.

14Like for MPI_ALLTOALLV, the \in place" option for intracommunicators is speci ed by

15passing MPI_IN_PLACE to the argument sendbuf at all processes. In such a case, sendcounts,

16sdispls and sendtypes are ignored. The data to be sent is taken from the recvbuf and replaced

17by the received data. Data sent and received must have the same type map as speci ed

18by the recvcounts and recvtypes arrays, and is taken from the locations of the receive bu er

19speci ed by rdispls.

20If comm is an intercommunicator, then the outcome is as if each process in group A

21sends a message to each process in group B, and vice versa. The j-th send bu er of process

22i in group A should be consistent with the i-th receive bu er of process j in group B, and

23vice versa.

24

25Rationale. The MPI_ALLTOALLW function generalizes several MPI functions by care-

26fully selecting the input arguments. For example, by making all but one process have

27sendcounts[i] = 0, this achieves an MPI_SCATTERW function. (End of rationale.)

28

31The functions in this section perform a global reduce operation (for example sum, maximum,

32and logical and) across all members of a group. The reduction operation can be either one of

33a prede ned list of operations, or a user-de ned operation. The global reduction functions

34come in several avors: a reduce that returns the result of the reduction to one member of a

35group, an all-reduce that returns this result to all members of a group, and two scan (parallel

36pre x) operations. In addition, a reduce-scatter operation combines the functionality of a

37reduce and of a scatter operation.

38

39

40

41

42

43

44

45

46

47

48

48

1This may change the result of the reduction for operations that are not strictly associative

4Advice to implementors. It is strongly recommended that MPI_REDUCE be imple-

5mented so that the same result be obtained whenever the function is applied on the

6same arguments, appearing in the same order. Note that this may prevent optimiza-

7tions that take advantage of the physical location of processors. (End of advice to

8implementors.)

9

10Advice to users. Some applications may not be able to ignore the non-associative na-

11ture of oating-point operations or may use user-de ned operations (see Section 5.9.5)

12that require a special reduction order and cannot be treated as associative. Such

13applications should enforce the order of evaluation explicitly. For example, in the

14case of operations that require a strict left-to-right (or right-to-left) evaluation or-

16MPI_GATHER), applying the reduction operation in the desired order (e.g., with

17MPI_REDUCE_LOCAL), and if needed, broadcast or scatter the result to the other

18processes (e.g., with MPI_BCAST). (End of advice to users.)

19

The datatype argument of MPI_REDUCE must be compatible with op. Prede ned op-

20

erators work only with the MPI types listed in Section 5.9.2 and Section 5.9.4. Furthermore,

21

the datatype and op given for prede ned operators must be the same on all processes.

22

Note that it is possible for users to supply di erent user-de ned operations to

23

MPI_REDUCE in each process. MPI does not de ne which operations are used on which

24

operands in this case. User-de ned operators may operate on general, derived datatypes.

25

In this case, each argument that the reduce operation is applied to is one element described

26

by such a datatype, which may contain several basic values. This is further explained in

27

Section 5.9.5.

28

29Advice to users. Users should make no assumptions about how MPI_REDUCE is

30implemented. It is safest to ensure that the same function is passed to MPI_REDUCE

31by each process. (End of advice to users.)

32

Overlapping datatypes are permitted in \send" bu ers. Overlapping datatypes in \re-

33

ceive" bu ers are erroneous and may give unpredictable results.

34

The \in place" option for intracommunicators is speci ed by passing the value

35

MPI_IN_PLACE to the argument sendbuf at the root. In such a case, the input data is taken

36

at the root from the receive bu er, where it will be replaced by the output data.

37

If comm is an intercommunicator, then the call involves all processes in the intercom-

38

municator, but with one group (group A) de ning the root process. All processes in the

39

other group (group B) pass the same value in argument root, which is the rank of the root

40

in group A. The root passes the value MPI_ROOT in root. All other processes in group A

41

pass the value MPI_PROC_NULL in root. Only send bu er arguments are signi cant in group

42

B and only receive bu er arguments are signi cant at the root.

43

44

5.9.2 Prede ned Reduction Operations

45

46The following prede ned operations are supplied for MPI_REDUCE and related functions

47MPI_ALLREDUCE, MPI_REDUCE_SCATTER, MPI_SCAN, and MPI_EXSCAN. These oper-

48ations are invoked by placing the following in op.

26REAL sum

27INTEGER m, comm, i, ierr

35! global sum

36CALL MPI_REDUCE(sum, c, 1, MPI_REAL, MPI_SUM, 0, comm, ierr)

37RETURN

38

39Example 5.16 A routine that computes the product of a vector and an array that are

40distributed across a group of processes and returns the answer at node zero.

45REAL sum(n)

46INTEGER n, comm, i, j, ierr

47