Multi-Threaded Software
How to request a multi-threaded slot and how to ensure your software only uses the CPU cores it has been allocated
Running multi-threaded programs can cause significant problems with cluster scheduling software if the clustering software is not made aware of the multiple threads (your job is allocated one slot but actually consumes many more, often ALL the CPUs, overloading the machine).
We support the running of shared memory multi-threaded software only (e.g. OpenMP, multi-threaded MKL, OpenBLAS etc).
To submit an OpenMP job, use the -s (or --parallelenv) option to fsl_sub. For example:
fsl_sub -s 2 <command or script>
2 being the number of threads you wish to allocate to your jobs.
The task running on the queue will be able to determine how many slots it has by querying the environment variable pointed to by FSLSUB_NSLOTS. For example in BASH the number of slots is equal to ${!FSLSUB_NSLOTS}.
In Python you would be able to get this figure with the following code:
import os slots = os.environ[os.environ['FSLSUB_NSLOTS']]
Within MATLAB you can control the number of slots with:
n = getenv("FSLSUB_NSLOTS"); LASTN = maxNumCompThreads(n);
To be able to provide these threads the cluster software needs to reserve slots on compute nodes, this may lead to significant wait times whilst sufficient slots become available on a single device.