mapper - Parallel processing implementations

BaseMapper

MPIMapper

MPMapper

SerialMapper

can_pickle

Returns True if problem can be pickled.

nice

setpriority

Set The Priority of a Windows Process.

using_mpi

Parallel and serial mapper implementations.

The API is a bit crufty since interprocess communication has evolved from the original implementation. And the names are misleading.

Usage:

Mapper.start_worker(problem)
mapper = Mapper.start_mapper(problem, None, cpus)
result = mapper(points)
...
mapper = Mapper.start_mapper(problem, None, cpus)
result = mapper(points)
Mapper.stop_mapper()
class bumps.mapper.BaseMapper[source]

Bases: object

has_problem = False
static start_mapper(problem, modelargs=None, cpus=0)[source]

Called with the problem on a new fit.

static start_worker(problem)[source]

Called with the problem to initialize the worker

static stop_mapper(mapper=None)[source]
class bumps.mapper.MPIMapper[source]

Bases: BaseMapper

has_problem = True

For MPIMapper only the worker is initialized with the fit problem.

static start_mapper(problem, modelargs=None, cpus=0)[source]

Called with the problem on a new fit.

static start_worker(problem)[source]

Start the worker process.

For the main process this does nothing and returns immediately. The worker processes never return.

Each worker sits in a loop waiting for the next batch of points for the problem, or for the next problem. Set t problem is set to None, then exit the process and never

static stop_mapper(mapper=None)[source]
class bumps.mapper.MPMapper[source]

Bases: BaseMapper

has_problem = False
manager = None
pool = None
problem = None
problem_id = 0
shared_pickled_problem = None
static start_mapper(problem, modelargs=None, cpus=0)[source]

Called with the problem on a new fit.

static start_worker(problem)[source]

Called with the problem to initialize the worker

static stop_mapper(mapper=None)[source]
class bumps.mapper.SerialMapper[source]

Bases: BaseMapper

has_problem = False
static start_mapper(problem, modelargs=None, cpus=0)[source]

Called with the problem on a new fit.

static start_worker(problem)[source]

Called with the problem to initialize the worker

static stop_mapper(mapper=None)[source]
bumps.mapper.can_pickle(problem, check=False)[source]

Returns True if problem can be pickled.

If this method returns False then MPMapper cannot be used and SerialMapper should be used instead.

If check is True then call nllf() on the duplicated object as a “smoke test” to verify that the function will run after copying. This is not foolproof. For example, access to a database may work in the duplicated object because the connection is open and available in the current process, but it will fail when trying to run on a remote machine.

bumps.mapper.nice()[source]
bumps.mapper.setpriority(pid=None, priority=1)[source]

Set The Priority of a Windows Process. Priority is a value between 0-5 where 2 is normal priority and 5 is maximum. Default sets the priority of the current python process but can take any valid process ID.

bumps.mapper.using_mpi()[source]