exarl.network.mpi_comm

Module Contents

Classes

ExaMPI

This class is built to replacement the pickling of mpi4py.

Attributes

threads

recv_mprobe

exarl.network.mpi_comm.threads = False
exarl.network.mpi_comm.recv_mprobe = False
class exarl.network.mpi_comm.ExaMPI(comm=MPI.COMM_WORLD, procs_per_env=1, run_length=False)

Bases: exarl.base.comm_base.ExaComm

This class is built to replacement the pickling of mpi4py. The idea is that only numpy arrays will be sent to the underlying mpi4py calls thus avoiding pickle.

Variables

  • mpi (MPI) – MPI class used to access comm, sizes, and rank

  • comm (MPI.comm) – The underlying communicator

  • size (int) – Number of processes in the communicator

  • rank (int) – Rank of the current process

  • run_length (int) – Flag indicating to use run length encoding

  • buffers (map) – These are preallocated buffers for sending and recieving to avoid memory thrashing

Parameters

  • comm (MPI Comm, optional) – The base MPI comm to split into sub-comms. If set to None will default to MPI.COMM_WORLD

  • procs_per_env (int, optional) – Number of processes per environment (sub-comm)

  • num_learners (int, optional) – Number of learners (multi-learner)

mpi
np_type_converter(self, the_type)

Gets the appropriate numpy type based on non-numpy types

Parameters

the_type (type) – Type to convert

mpi_type_converter(self, the_type)

Gets the appropriate MPI type based on numpy types

Parameters

the_type (type) – Type to convert

encode_type(self, the_type)

Encodes type to int value for sending

Parameters

the_type (type) – Type to encode

decode_type(self, the_type, cast=True)

Decodes int to type for receiving, and casts input value as type.

Parameters

  • the_type (int) – Int to decode

  • cast (bool, optional) – Indicates if we should return type(false) or a value of type

list_like(self, data, prep=True)

Returns if data is like a list and if not converts to list

Parameters

  • data (any) – Data to check if is list/convert

  • prep (bool, optional) – Converts data to list and returns as second value if true

is_float(self, data)

Returns if data a float, np.float32, or np.float64

Parameters

data (any) – Data to check if is float

get_flat_size(self, data)

Gets the flatten size of data

Parameters

data (any) – Data to check

encode_int_list(self, data, buff=None, level=0)

This converts a list of ints into a flat list while encoding the shape. This is done by flattening and adding 2 to each number.

Parameters

  • data (list of ints) – Data to encode

  • buff (list, optional) – Used internally for recursive call. Builds up return list.

  • level (int, optional) – Used internally for recursive call. Indicates level of nesting.

decode_int_list(self, buff, data=None, index=0, level=0)

Reshapes a flat list of ints to list of lists. 0 and 1 indicate start and end of a list. All other values have 2 subtracted from them.

Parameters

  • buff (flat list of ints) – Buffer of data to decode

  • data (list of list of ints, optional) – Used in recursive call to build up return data list

  • index (int, optional) – Current position in decoding buffer

  • level (int, optional) – Used internally for recursive call. Indicates level of nesting.

encode_list_format(self, data, buff=None, np_arrays=None, level=0)

This call will result in the encoding the type information of a list of lists. Also returns a list of np.array shapes for special np.array fast path.

Parameters

  • data (any) – The data to encode

  • buff (list, optional) – Used internally for recursive call. Buffer will hold the typing information.

  • np_arrays (np.array, optional) – Used internally for recursive call. Array will hold flat data.

  • level (int, optional) – Indicates level of nesting.

decode_list_format(self, buff, data=None, is_np=False, np_arrays=None, np_index=0, index=0)

This call decodes the type and shape the originally encoded data. np_arrays is for a np.array fast path.

Parameters

  • buff (list) – Buffer will hold the typing information. Buffer can be supplied for memory reuse.

  • data (list) – This is the data that will be return. It is also used for internal recursive call.

  • is_np (bool) – Fast path for np.array

  • np_arrays (np.array, optional) – This is the data that is to be decoded.

  • np_index (int, optional) – Used internally for recursive call. Indicates level of nesting.

  • index (int, optional) – Depth of the recursion.

run_length_encode(self, data)

This implements runlength encoding of ints. Runlegth encoding takes duplicate values and reduces it to two, the number and the number of repeating values.

Parameters

data (list of ints) – This is the data that is to be encoded

run_length_decode(self, data)

This implements runlength encoding of ints. Runlegth encoding takes duplicate values and reduces it to two, the number and the number of repeating values. This only supports ints so the data must be cast if coming from float buffer

Parameters

data (list of ints) – This is the data that is to be encoded

buffer(self, data_type, data_count, exact=False)

Allocates a buffer from a pool of buffers.

Parameters

  • data_type (type) – Type of buffer to get

  • data_count (int) – Size of buffer

  • exact (bool) – Does buffer need to be exact size

delete_buffers(self)

Deallocate all buffers and call the garbage collection

marshall(self, data, buff, data_type, data_count=None, index=0, first=True)

This does the marshalling of the data into a buffer This should be called last before sending or right before receiving

Parameters

  • data (list) – Data to mashall

  • buff (list) – Memory to use for mashalled data

  • data_type (type) – Type of data

  • data_count (int, optional) – Size of data

  • index (int, optional) – Used to keep track of what value is being mashalled within data

  • first (bool, optional) – Used to keep track in recursion

demarshall(self, data, buff, data_count=None, index=0, first=True)

This demarshalls the data from a marshall call. This should be called immediatly after a receive. The type is maintained by the data object. Be careful to make sure that the types are the same across send/recv. If not the underlying buffer type could mismatch and mashalling will fail.

Parameters

  • data (list) – data to demarshall

  • buff (list) – Size of buffer

  • data_cout (int, optional) – Used for flat list fast path

  • index (int) – Keeps track of which element we are demarshalling

  • first (bool) – Used for recursive calls to track top level call

prep_data(self, data, copy=True, default_buffer_type=np.int64)

This function allocates the buffer to send/recv. Will optionally copy data into the buffer.

Parameters

  • data (list) – data to prep

  • copy (bool, optional) – Inidicate if we should marshall data

  • default_buffer_type (type, optional) – Buffer type

send_with_type(self, data, dest, default_buffer_type=np.int64)

This will send messages if we do not know what “types” are in the message. First it will find the data format and send it along with the size/buffer type of the data. Next we send the actuall data. Can use compression to decrease message size.

Parameters

  • data (list) – data to send

  • dest (int) – Rank to send to

  • default_buffer_type (type, optional) – Buffer type

send(self, data, dest, pack=False, default_buffer_type=np.int64)

Point-to-point communication between ranks. Send must have matching recv.

Parameters

  • data (any) – Data to be sent

  • dest (int) – Rank within comm where data will be sent.

  • pack (int, optional) – If we do not know the type on both sides of the send/recv

recv_with_type(self, source, default_buffer_type=np.int64)

The receives messagse when we don’t know the type ahead of time. This follows the procedures outline in send_with_type.

Parameters

  • source (int) – Sending rank

  • default_buffer_type (type, optional) – Buffer type

recv(self, data, source=MPI.ANY_SOURCE, default_buffer_type=np.int64)

Point-to-point communication between ranks. Send must have matching send.

Parameters

  • data (any) – Not used

  • dest (int) – Rank within comm where data will be sent. Must have matching recv.

  • source (int, optional) – Rank to recieve data from. Default allows data from any source.

  • default_buffer_type (type, optional) – Buffer type

bcast(self, data, root)

Broadcasts data from the root to all other processes in comm.

Parameters

  • data (any) – Data to be broadcast

  • root (int) – Indicate which process data comes from

barrier(self)

Block synchronization for the comm.

reduce(self, arg, op, root)

Data is joined from all processes in comm by doing op. Result is placed on root.

Parameters

  • arg (any) – Data to reduce

  • op (str) – Supports sum, max, and min reductions

  • root (int) – Rank the result will end on

allreduce(self, arg, op=MPI.LAND)

Data is joined from all processes in comm by doing op. Data is put on all processes in comm.

Parameters

  • arg (any) – Data to reduce

  • op (MPI op, optional) – Operation to perform

time(self)

Returns MPI wall clock time

split(self, procs_per_env)

This splits the comm into agent, environment, and learner comms. Returns three simple sub-comms

Parameters

  • procs_per_env (int) – Number of processes per environment comm

  • num_learners (int) – Number of processes per learner comm

raw(self)

Returns raw MPI comm

printBufSize(self)

Prints size of internal buffers.