lsf_gpu

This is an old revision of the document!


GPU Queues

The titan-Queues (titanshort/long) currently include hosts i0001-i0009, while the gpu-Queues (gpushort/long) include the hosts g0001-g0009. The titan-hosts carry 4 GeForce GTX TITAN, hence a usage request up to cuda=4 can be selected (see below). In contrast the GeForce GTX 480 is installed on the gpu-hosts (for the gpushort/long queues). Finally, for the tesla-Queues (teslashort/long) 4 Tesla K20m cards are installed.

The following link gives an overview on the Compute Nodes. To associate hosts with queues, type

$ bqueues -l <queuename> | grep HOSTS

pick the resulting short name from the output and with

$ bhosts <nodename>

you will get the relevant hosts.

The max. runtime is analogous to the other short/long-Queues.

GPU Usage

To use a GPU you have to explicitly reserve it as a resource in the bsub call:

$ bsub -n 1 -R 'rusage[cuda=1]' -q gpulong ./my_program

The code or application to be carried out needs to

  1. be an executable script or program.
  2. carry a shebang.

While this is true for LSF in general, it is imposed for the GPU-resource requests.

In the previous example 1 CPU is requested (-n 1) and 1 GPU (cuda=1). The actual number of GPUs available depends on the queue you are using.

If supported by the queue, you can request multiple GPUs like

$ bsub -n 1 -R 'rusage[cuda=4]' -q titanshort ./my_program

Be sure to add a sufficient time estimate with -W. Also, multiple CPUs can be requested with the usual ptile option.

In order to use multiples nodes, you have to request entire nodes and entire GPU sets, e.g.

$ bsub -q titanshort -n 2 -R 'span[ptile=1]' -R 'affinity[core(16)]' -R 'rusage[cuda=4]

In this example 2 entire titannodes will be used (also the CPU set).

Your job script / job command has to export the environment of your job. mpirun implementations do have an option for this (see your mpirun man page).

  • lsf_gpu.1434487669.txt.gz
  • Last modified: 2015/06/16 22:47
  • by meesters