Documentation
In addition to this wiki entry, there is a man page available:
$ man spank-jgu-ramdisk
Ramdisk Reservation
Especially for I/O intensive jobs, which issue many system calls, the local disk as well as our GPFS fileserver can be a bottleneck. Staging of files to a local RAM disk (or ramdisk) can be a solution for these jobs.
In order to create a ramdisk for your job you must specify an (additional) sbatch statement, where the size of the ramdisk is stated in the usual kilobytes, megabytes or gigabytes, abbreviated as M, G or T.
The ramdisk is created in the jobdir directory /localscratch/${SLURM_JOB_ID}/ramdisk
For this example we assume submission like $ sbatch <jobscript> and a script like:
#!/bin/bash #SBATCH -J ramdisk_example #SBATCH -t 5 # we request 5 minutes, pro forma #SBATCH --ramdisk=100G # the actual #SBATCH -p <partition name> # do not forget to set the partition #SBATCH -A <your association> # and your association JOBDIR=/localscratch/$SLURM_JOB_ID RAMDISK=$JOBDIR/ramdisk # the stage in step cp <file in parallel file system> $RAMDISK/.
The specification can be given on the command line, too:
$ sbatch --ramdisk=100M
Unit Considerations
- Omitting the unit will result in an error informing you about this requirement to supply a unit.
- A unit like
Tfor Terabyte is to be used with caution as other big values withG: The current implementation will not check, whether or not the physical RAM is actually provided by the selected nodes.
The reserved disk size is stored in the environment variable SLURM_SPANK_JOB_RAMDISK. This can be used in job scripts and will hold the reserved memory value in units of megabytes.
Multi Node Jobs
For jobs using multiple nodes multiple ramdisks are created. To facilitate the stage in phase, you can use the sbcast command:
# let $FILE be a file on the parallel file system, then: sbcast $FILE $RAMDISK/$(basename $FILE) # should work, as sbcast requires the full path name.
Now, all nodes can access the file at the same path.