start:working_on_mogon:io_odds_and_ends:slurm_localscratch

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
start:working_on_mogon:io_odds_and_ends:slurm_localscratch [2021/06/10 17:21]
meesters
start:working_on_mogon:io_odds_and_ends:slurm_localscratch [2021/06/10 17:27] (current)
meesters
Line 1: Line 1:
 ====== Local Scratch Space ====== ====== Local Scratch Space ======
  
-On every node, there is local scratch space available to your running jobs that you should use if required by your jobs IO-pattern.+On every node, there is local scratch space available to your running jobs.
 Every job can therefore use a directory called ''/localscratch/${SLURM_JOB_ID}/'' on the local disk. If a job array starts then this directory also called ''/localscratch/${SLURM_JOB_ID}/'', where the variable ''SLURM_ARRAY_TASK_ID'' is an index of a subjob in the job array and unrelated to ''$SLURM_JOB_ID'' Every job can therefore use a directory called ''/localscratch/${SLURM_JOB_ID}/'' on the local disk. If a job array starts then this directory also called ''/localscratch/${SLURM_JOB_ID}/'', where the variable ''SLURM_ARRAY_TASK_ID'' is an index of a subjob in the job array and unrelated to ''$SLURM_JOB_ID''
 +
 +<callout type="info" icon="true" title="When to use Local Scratch">
 +If your job(s) in question are merely reading and writing big files in a linear mode, there is no requirement to use a local scratch or a ramdisk. However, these are scenarios, where using the local scratch might be beneficial:
 +  * if your job produces many temporary files
 +  * if your job reads a file or set of files in a directory repeatedly during run time (for multiple threads or concurrent jobs mean a random access pattern to the global file system, which is a true performance killer)
 +</callout>
  
 <callout type="info" icon="true"> <callout type="info" icon="true">
  • start/working_on_mogon/io_odds_and_ends/slurm_localscratch.txt
  • Last modified: 2021/06/10 17:27
  • by meesters