start:working_on_mogon:io_odds_and_ends:slurm_localscratch

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Last revision Both sides next revision
start:working_on_mogon:io_odds_and_ends:slurm_localscratch [2021/06/10 17:21]
meesters
start:working_on_mogon:io_odds_and_ends:slurm_localscratch [2021/06/10 17:27]
meesters
Line 1: Line 1:
 ====== Local Scratch Space ====== ====== Local Scratch Space ======
  
-On every node, there is local scratch space available to your running jobs that you should use if required by your jobs IO-pattern.+On every node, there is local scratch space available to your running jobs.
 Every job can therefore use a directory called ''/localscratch/${SLURM_JOB_ID}/'' on the local disk. If a job array starts then this directory also called ''/localscratch/${SLURM_JOB_ID}/'', where the variable ''SLURM_ARRAY_TASK_ID'' is an index of a subjob in the job array and unrelated to ''$SLURM_JOB_ID'' Every job can therefore use a directory called ''/localscratch/${SLURM_JOB_ID}/'' on the local disk. If a job array starts then this directory also called ''/localscratch/${SLURM_JOB_ID}/'', where the variable ''SLURM_ARRAY_TASK_ID'' is an index of a subjob in the job array and unrelated to ''$SLURM_JOB_ID''
 +
 +<callout type="info" icon="true" title="When to use Local Scratch">
 +If your job(s) in question are merely reading and writing big files in a linear mode, there is no requirement to use a local scratch or a ramdisk. However, these are scenarios, where using the local scratch might be beneficial:
 +  * if your job produces many temporary files
 +  * if your job reads a file or set of files in a directory repeatedly during run time (for multiple threads or concurrent jobs mean a random access pattern to the global file system, which is a true performance killer)
 +</callout>
  
 <callout type="info" icon="true"> <callout type="info" icon="true">
  • start/working_on_mogon/io_odds_and_ends/slurm_localscratch.txt
  • Last modified: 2022/06/20 18:05
  • by meesters