LoadManager master implementation, which is responsible for distributing work and synchronizing data. More...

#include <LoadManager.hpp>

Inheritance diagram for HMP::LoadManagerMaster:

Collaboration diagram for HMP::LoadManagerMaster:

Public Member Functions
virtual void	calculate (const StackIdentifier *stackIds, const int size) override
	Calculate a list of stacks, where the stack identifiers are provided in list form. More...

void	printRuntimeStatistics () const override
	Print runtime statistics, including information on the efficiency of LoadManager instances running on different MPI ranks.

Public Member Functions inherited from HMP::LoadManager
virtual	~LoadManager ()
	Destroy the LoadManager object.

template<class StackT >
StackIdentifier	addMasterStackExplicit (StackT *const data, const int size, const std::function< StackT(int)> &calculator, const int recommendedChunkSizeMultiple=1, const int recommendedChunksPerRank=10, const bool autoBroadcast=false)
	Create an explicit data stack and attach it to the LoadManager. More...

template<class StackT >
StackIdentifier	addMasterStackImplicit (StackT *const data, const int size, const std::function< void(int)> &calculator, const int typeMultiplicity=1, const int recommendedChunkSizeMultiple=1, const int recommendedChunksPerRank=10, const bool autoBroadcast=false)
	Create an implicit data stack and attach it to the LoadManager. More...

template<class StackT >
StackIdentifier	addSlaveStack (StackT *const data, const int size, const StackIdentifier master, const int typeMultiplicity=1)
	Create a slave data stack and attach it to the LoadManager. More...

template<class StackT >
StackIdentifier	addPassiveStack (StackT *const data, const int size)
	Create a passive data stack and attach it to the LoadManager. More...

void	calculate (const std::initializer_list< StackIdentifier > &stackIds)
	Calculate a list of stacks, where the stack identifiers are provided in initializer list form. More...

void	calculate (const StackIdentifier stackId)
	Calculate a single stack. More...

void	calculateAll ()
	Calculate all stacks.

void	broadcast (const StackIdentifier *stackIds, const int size)
	Broadcast a list of stacks, where the stack identifiers are provided in list form. More...

void	broadcast (const std::initializer_list< StackIdentifier > &stackIds)
	Broadcast a list of stacks, where the stack identifiers are provided in initializer list form. More...

void	broadcast (const StackIdentifier stackId)
	Broadcast a single stack. More...

void	broadcastAll ()
	Broadcast all stacks.

Protected Member Functions
	LoadManagerMaster (const int serverRank, const MPI_Comm communicator)
	Construct a new LoadManagerMaster object. More...

	~LoadManagerMaster ()
	Destroy the LoadManagerMaster object.

virtual StackIdentifier	_registerStack (DataStackBase *stack) override
	Register a DataStackBase with the LoadManager. By registering the stack, the LoadManager assumes responsibility to synchronize data between MPI ranks as necessary. More...

void	_runLocalClient ()
	Worker loop to run calculators locally.

void	_initChunkSpawner (const StackIdentifier *stackIds, const int size)
	Prepare the chunk spawner, i.e. the routine responsible for breaking down workload into smaller chunsk, for the execution of calculators on the specified list of stacks. More...

void	_initChunkSpawner (const std::initializer_list< StackIdentifier > &stackIds)
	Prepare the chunk spawner, i.e. the routine responsible for breaking down workload into smaller chunsk, for the execution of calculators on the specified initializer list of stacks. More...

Chunk	_spawnChunk (const int rank)
	The chunk spawner routine, which is responsible for breaking down the overall workload of calculator executions into smaller chunks of workload. The algorithm to spawn a chunk and determine its workload is as follows: More...

void	_issueChunk (const int rank)
	Send a workload chunk to a specified MPI rank. More...

void	_despawnChunk (const int rank)
	Mark a workload chunk as completed. Relevant only for runtime statistics information. More...

Protected Member Functions inherited from HMP::LoadManager
	LoadManager (const int serverRank, const MPI_Comm communicator)
	Construct a new Load Manager object. More...

void	_calculateChunk (const Chunk &chunk)
	Calculate the workload defined by a specific chunk. More...

Protected Attributes
float	_totalCalculationTime
	Accumulated time in milliseconds which has been spent on calculate() calls over the lifetime of the LoadManager instance.

std::vector< float > *	_totalComputeTime
	_totalComputeTime[rank][stack] is the accumulated time in milliseconds which MPI rank `rank` spent computing on `stack`.

std::vector< int > *	_currentCalculationWorkDone
	_currentCalculationWorkDone[rank][stack] is the number of calculations which have been performed by MPI rank `rank` on `stack` in the current calculate() call.

std::vector< float > *	_currentCalculationTime
	_currentCalculationTime[rank][stack] is the time in milliseconds spent by MPI rank `rank` until returning chunk result for `stack` in the current calculate() call.

boost::posix_time::ptime *	_currentCalculationChunkSpawntime
	_currentCalculationChunkSpawntime[rank] specifies time at which the most recent chunk has been issued to MPI rank `rank` in the current calculate() call.

Chunk *	_currentCalculationChunkSpawned
	_currentCalculationChunkSpawned[rank] stores the most recent chunk generated for MPI rank `rank` in the current calculate() call.

std::vector< float >	_currentCalculationComputeTimeBuffer
	_currentCalculationComputeTimeBuffer[rank*_stacks.size()+stack] is a buffer for the time in milliseconds spent on computing `stack` in the current calculate() call.

std::vector< bool >	_currentCalculationStackMask
	_currentCalculationStackMask[stack] specifies whether `stack` should be computed in the current calculate() call.

std::vector< int >	_currentCalculationStackProgress
	_currentCalculationStackProgress[stack] specifies the current progress (pointer to the next unissued value) which has already been issued for computation in the current calculate() call.

std::mutex	_currentCalculationChunkSpawnerLock
	Lock to synchronize chunk spawning for remote calculations and for local worker threads.

MPI_Request *	_pendingRequests
	MPI request objects associated with the return values for workload chunks that have been issued.

Protected Attributes inherited from HMP::LoadManager
std::vector< DataStackBase * >	_stacks
	List of all registered stacks.

int	_serverRank
	Designated MPI master rank.

int	_rank
	MPI rank of the current LoadManager instance.

int	_commSize
	MPI communicator size.

MPI_Comm	_communicator
	MPI communicator to operate on.

Friends
LoadManager *	newLoadManager (const int serverRank, const MPI_Comm communicator)
	Create a new LoadManager instance. If the current MPI rank is the designated master rank, returns a LoadManagerMaster instance, otherwise returns a LoadManagerSlave instance. More...

Detailed Description

LoadManager master implementation, which is responsible for distributing work and synchronizing data.

See also: LoadManager

Constructor & Destructor Documentation

◆ LoadManagerMaster()

HMP::LoadManagerMaster::LoadManagerMaster	(	const int	serverRank,
		const MPI_Comm	communicator
	)

inlineprotected

Construct a new LoadManagerMaster object.

Parameters

serverRank	The MPI rank to take on the master role.
communicator	The MPI communicator used for communication.

Member Function Documentation

◆ _despawnChunk()

void HMP::LoadManagerMaster::_despawnChunk ( const int rank )

inlineprotected

Mark a workload chunk as completed. Relevant only for runtime statistics information.

Parameters

rank	MPI rank which had completed the workload.

◆ _initChunkSpawner() [1/2]

void HMP::LoadManagerMaster::_initChunkSpawner	(	const StackIdentifier *	stackIds,
		const int	size
	)

inlineprotected

Prepare the chunk spawner, i.e. the routine responsible for breaking down workload into smaller chunsk, for the execution of calculators on the specified list of stacks.

Parameters

stackIds	Pointer to the first StackIdentifier to be calculated.
size	Number of StackIdentifiers to be calculated.

◆ _initChunkSpawner() [2/2]

void HMP::LoadManagerMaster::_initChunkSpawner ( const std::initializer_list< StackIdentifier > & stackIds )

inlineprotected

Prepare the chunk spawner, i.e. the routine responsible for breaking down workload into smaller chunsk, for the execution of calculators on the specified initializer list of stacks.

Parameters

stackIds Initializer list of the stacks to be calculated.

◆ _issueChunk()

void HMP::LoadManagerMaster::_issueChunk ( const int rank )

inlineprotected

Send a workload chunk to a specified MPI rank.

Parameters

rank	Receiver's MPI rank.

◆ _registerStack()

virtual StackIdentifier HMP::LoadManagerMaster::_registerStack ( DataStackBase * stack )

inlineoverrideprotectedvirtual

Register a DataStackBase with the LoadManager. By registering the stack, the LoadManager assumes responsibility to synchronize data between MPI ranks as necessary.

Parameters

stack The stack to be registered.

Returns: StackIdentifier Unique identifier of the newly registered stack; Used e.g. for LoadManager::calculate calls.

Reimplemented from HMP::LoadManager.

◆ _spawnChunk()

Chunk HMP::LoadManagerMaster::_spawnChunk ( const int rank )

inlineprotected

The chunk spawner routine, which is responsible for breaking down the overall workload of calculator executions into smaller chunks of workload. The algorithm to spawn a chunk and determine its workload is as follows:

Choose a stack that has not finished computing.
Determine the maximum chunk size according to the stack's recommended number of chunks per rank.
Round up that number to be a multiple of the recommended chunk size.
Determine chunk size according to relative computing power of the different ranks, based on performance on previous chunks.
Clip chunk size to minimum of 100ms expected return time and maximum as determined before.

Parameters

rank	MPI rank for which the workload chunk is being requested.

Returns: Chunk Definition of the workload.

◆ calculate()

virtual void HMP::LoadManagerMaster::calculate	(	const StackIdentifier *	stackIds,
		const int	size
	)

inlineoverridevirtual

Calculate a list of stacks, where the stack identifiers are provided in list form.

Parameters

stackIds	Pointer to the first StackIdentifier.
size	Number of stacks.

Implements HMP::LoadManager.

Friends And Related Function Documentation

◆ newLoadManager

LoadManager* newLoadManager	(	const int	serverRank = `0`,
		const MPI_Comm	communicator = `MPI_COMM_WORLD`
	)

friend

Create a new LoadManager instance. If the current MPI rank is the designated master rank, returns a LoadManagerMaster instance, otherwise returns a LoadManagerSlave instance.

Parameters

serverRank	MPI rank to assume the master role.
communicator	The MPI communicator to operate on.

Returns: LoadManager* Newly generated LoadManager instance.

The documentation for this class was generated from the following file:

/home/runner/work/SpinParser/SpinParser/src/lib/LoadManager.hpp

Public Member Functions

Protected Member Functions

Protected Attributes

Friends

Detailed Description

Constructor & Destructor Documentation

◆ LoadManagerMaster()

Member Function Documentation

◆ _despawnChunk()

◆ _initChunkSpawner() [1/2]

◆ _initChunkSpawner() [2/2]

◆ _issueChunk()

◆ _registerStack()

◆ _spawnChunk()

◆ calculate()

Friends And Related Function Documentation

◆ newLoadManager