Single-machine, multi-process integration
This MR adds the MultiprocIntegrator
class for distributing the work over a set of processes on the local machine.
To simplify task distribution, each worker (process) handles one full dataset (finer granularity would involve IPC).
To do
-
Refactor the classes to use a base class DistributedIntegrator
-
Ensure that distributed integration (over SLURM) still works -
Test MultiprocIntegrator
-
Automatic resources handling: -
Determine automatically the best affinity mask -
Pick the best opencl platform
-
-
Plug MultiprocIntegrator
to the CLI/configuration file -
Clean-up the test/debug code fragments (eg. os.environ
,is_power9
, etc)
Edited by Pierre Paleo