The GPFS fileserver of MOGON provides a file system which is exclusively reserved for ATLAS users. The maximum total I/O bandwidth of this file system is about 8000 MB/s. Until the GPFS file system has been optimized for typical ATLAS ROOT jobs this value is set to a lower value in order to prevent oversubscription of the provided bandwidth, which would result in an unnecessarily large Wall time of the job. When submitting a job, the user must specify the expected bandwidth.
If the user needs an I/O bandwidth of e.g. 10MB/s, the
SBATCH statements must provide the following parameter:
#SBATCH -L atlasio:10
Jobs will only start running, if there is enough I/O bandwidth available. The amount of available bandwidth can be checked via:
$ scontrol show license
Here, the first number gives the total bandwidth (in MB/s), which is available. The second gives the currently reserved bandwidth. Hence, jobs requesting more bandwidth than currently available will have to wait. (The result also mentions the hosts managing this bandwidth in the following columns.)
1. If the rusage parameter is omitted by a user, slurm will automatically assume a bandwith of 10 MB/s.
2. For jobs that will not do a reasonable amount (sum of all files < 100MByte) of I/O, the user should specify the parameter with 0 MB/s, otherwise - like mentioned before - 10MB/s will be assumed.