User Tools

Site Tools


job_arrays

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Last revision Both sides next revision
job_arrays [2017/10/09 08:38]
meesters [Multiprog for Uneven Arrays]
job_arrays [2018/03/12 21:32]
meesters [Job Arrays in Slurm]
Line 55: Line 55:
 The ''​%A_%a''​ construct in the output and error file names is used to generate unique output and error files based on the master job ID (''​%A''​) and the array-tasks ID (''​%a''​). In this fashion, each array-tasks will be able to write to its own output and error file. The ''​%A_%a''​ construct in the output and error file names is used to generate unique output and error files based on the master job ID (''​%A''​) and the array-tasks ID (''​%a''​). In this fashion, each array-tasks will be able to write to its own output and error file.
  
 +==== Limiting the number of concurrent jobs of an array ====
 +
 +It is possible to limit the number of concurrently executed jobs of an array, e.g. to minimize I/O overhead within one approach, with this syntax:
 +
 +<code bash>
 +#SBATCH --array=1-1000%50
 +</​code>​
 +
 +where a limit of 50 concurrent jobs would be in place.
 ===== Multiprog for "​Uneven"​ Arrays ===== ===== Multiprog for "​Uneven"​ Arrays =====
  
 The ''​%%--multi-prog%%''​ option in ''​srun''​ allows you to assign each parallel task in your job with a different option. More information can be found at [[node_local_scheduling|our wiki page on node-local scheduling]]. The ''​%%--multi-prog%%''​ option in ''​srun''​ allows you to assign each parallel task in your job with a different option. More information can be found at [[node_local_scheduling|our wiki page on node-local scheduling]].
job_arrays.txt · Last modified: 2019/03/27 11:55 by meesters