In addition to this wiki entry, there is a man page available:
$ man spank-jgu-ramdisk
Especially for I/O intensive jobs, which issue many system calls, the local disk as well as our GPFS fileserver can be a bottleneck. Staging of files to a local RAM disk (or ramdisk) can be a solution for these jobs.
In order to create a ramdisk for your job you must specify an (additional) sbatch statement, where the size of the ramdisk is stated in the usual kilobytes, megabytes or gigabytes, abbreviated as
The ramdisk is created in the jobdir directory
For this example we assume submission like
$ sbatch <jobscript> and a script like:
#!/bin/bash #SBATCH -J ramdisk_example #SBATCH -t 5 # we request 5 minutes, pro forma #SBATCH --ramdisk=100G # the actual #SBATCH -p <partition name> # do not forget to set the partition #SBATCH -A <your association> # and your association JOBDIR=/localscratch/$SLURM_JOB_ID RAMDISK=$JOBDIR/ramdisk # the stage in step cp <file in parallel file system> $RAMDISK/.
The specification can be given on the command line, too:
$ sbatch --ramdisk=100M
Tfor Terabyte is to be used with caution as other big values with
G: The current implementation will not check, whether or not the physical RAM is actually provided by the selected nodes.
The reserved disk size is stored in the environment variable
SLURM_SPANK_JOB_RAMDISK. This can be used in job scripts and will hold the reserved memory value in units of megabytes.
For jobs using multiple nodes multiple ramdisks are created. To facilitate the stage in phase, you can use the
# let $FILE be a file on the parallel file system, then: sbcast $FILE $RAMDISK/$(basename $FILE) # should work, as sbcast requires the full path name.
Now, all nodes can access the file at the same path.