In addition to this wiki entry, there is a man page available:

$ man spank-jgu-ramdisk

Ramdisk Reservation

Especially for I/O intensive jobs, which issue many system calls, the local disk as well as our GPFS fileserver can be a bottleneck. Staging of files to a local RAM disk (or ramdisk) can be a solution for these jobs.

In order to create a ramdisk for your job you must specify an (additional) sbatch statement, where the size of the ramdisk is stated in the usual kilobytes, megabytes or gigabytes, abbreviated as M, G or T.

The ramdisk is created in the jobdir directory /localscratch/${SLURM_JOB_ID}/ramdisk

For this example we assume submission like $ sbatch <jobscript> and a script like:

#SBATCH -J ramdisk_example
#SBATCH -t 5 # we request 5 minutes, pro forma
#SBATCH --ramdisk=100G   # the actual 
#SBATCH -p <partition name>   # do not forget to set the partition
#SBATCH -A <your association> # and your association
# the stage in step
cp <file in parallel file system> $RAMDISK/.

The specification can be given on the command line, too:

$ sbatch --ramdisk=100M

Unit Considerations

  • Omitting the unit will result in an error informing you about this requirement to supply a unit.
  • A unit like T for Terabyte is to be used with caution as other big values with G: The current implementation will not check, whether or not the physical RAM is actually provided by the selected nodes.

The reserved disk size is stored in the environment variable SLURM_SPANK_JOB_RAMDISK. This can be used in job scripts and will hold the reserved memory value in units of megabytes.

Multi Node Jobs

For jobs using multiple nodes multiple ramdisks are created. To facilitate the stage in phase, you can use the sbcast command:

# let $FILE be a file on the parallel file system, then:
sbcast $FILE $RAMDISK/$(basename $FILE)
# should work, as sbcast requires the full path name.

Now, all nodes can access the file at the same path.

  • start/working_on_mogon/io_odds_and_ends/ramdisk.txt
  • Last modified: 2020/04/16 11:49
  • by jrutte02