User Tools

Site Tools


io_reservation

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
io_reservation [2015/05/19 13:18]
meesters [I/O bandwidth reservation]
io_reservation [2017/06/30 08:35]
meesters
Line 1: Line 1:
 ====== I/O bandwidth reservation ====== ====== I/O bandwidth reservation ======
  
-The GPFS fileserver of MOGON provides a file system which is exclusively reserved for ATLAS users. The maximum total I/O bandwidth of this file system is about 8000 MB/s. Until the GPFS file system has been optimized for typical ATLAS ROOT jobs this value is set to a lower value in order to prevent oversubscription of the provided bandwidth, which would result in an unnecessarily large Wall time of the job. When submitting a job, the user must specify the expected bandwidth. This is done via a so called **//​rusage//​** parameter.+The GPFS fileserver of MOGON provides a file system which is exclusively reserved for ATLAS users. The maximum total I/O bandwidth of this file system is about 8000 MB/s. Until the GPFS file system has been optimized for typical ATLAS ROOT jobs this value is set to a lower value in order to prevent oversubscription of the provided bandwidth, which would result in an unnecessarily large Wall time of the job. When submitting a job, the user must specify the expected bandwidth.
  
-If the user needs an I/O bandwidth of e.g. 10MB/s, the bsub command ​must provide the following parameter:+If the user needs an I/O bandwidth of e.g. 10MB/s, the ''​SBATCH''​ statements ​must provide the following parameter:
  
 <code bash> <code bash>
-$ bsub -R "​rusage[atlasio=10]" ...+#​SBATCH ​-atlasio:10
 </​code>​ </​code>​
  
Line 12: Line 12:
  
 <code bash> <code bash>
-bhosts -s | grep atlasio+scontrol show license
 </​code>​ </​code>​
  
Line 21: Line 21:
 ==== PLEASE NOTE ==== ==== PLEASE NOTE ====
  
-1. If the rusage parameter is omitted by a user, LSF will automatically ​add the parameter -R "​rusage[atlasio=10]"​. +1. If the rusage parameter is omitted by a user, slurm will automatically ​assume ​bandwith ​of 10 MB/s.
- +
-2. For jobs that will not do reasonable amount (sum of all files < 100MByte) of I/O, the user should specify the parameter with 0 MB/s -R "​rusage[atlasio=0]"​ otherwise - like mentioned before - 10MB/s will be assumed. +
- +
-3. For multiprocessor jobs the value posted with //atlasio// is taken into account per core. e.g. a 2 core job will reserve 2 times the //atlasio// value: +
- +
-<code bash> +
-$ bsub -R "​rusage[atlasio=10]"​ -n 2 ... +
-</​code>​ +
- +
-This job will reserve 20 MB/s+
  
 +2. For jobs that will not do a reasonable amount (sum of all files < 100MByte) of I/O, the user should specify the parameter with 0 MB/s, otherwise - like mentioned before - 10MB/s will be assumed.
  
io_reservation.txt · Last modified: 2017/06/30 08:35 by meesters