User Tools

Site Tools


This is an old revision of the document!

I/O bandwidth reservation

The GPFS fileserver of MOGON provides a file system which is exclusively reserved for ATLAS users. The maximum total I/O bandwidth of this file system is about 8000 MB/s. Until the GPFS file system has been optimized for typical ATLAS ROOT jobs this value is set to a lower value in order to prevent oversubscription of the provided bandwidth, which would result in an unnecessarily large Wall time of the job. When submitting a job, the user must specify the expected bandwidth. This is done via a so called rusage parameter.

If the user needs an I/O bandwidth of e.g. 10MB/s, the bsub command must provide the following parameter:

$ bsub -R "rusage[atlasio=10]" ...

Jobs will only start running, if there is enough I/O bandwidth available. The amount of available bandwidth can be checked via:

$ bhosts -s | grep atlasio

Here, the first number gives the total bandwidth (in MB/s), which is available. The second gives the currently reserved bandwidth. Hence, jobs requesting more bandwidth than currently available will have to wait. (The result also mentions the hosts managing this bandwidth in the following columns.)


1. If the rusage parameter is omitted by a user, LSF will automatically add the parameter -R “rusage[atlasio=10]”.

2. For jobs that will not do a reasonable amount (sum of all files < 100MByte) of I/O, the user should specify the parameter with 0 MB/s -R “rusage[atlasio=0]” otherwise - like mentioned before - 10MB/s will be assumed.

3. For multiprocessor jobs the value posted with atlasio is taken into account per core. e.g. a 2 core job will reserve 2 times the atlasio value:

$ bsub -R "rusage[atlasio=10]" -n 2 ...

This job will reserve 20 MB/s

io_reservation.1461835695.txt.gz · Last modified: 2016/04/28 11:28 by schlarbm