Quick start - Debugging MPI programs on Mogon

This is a greatly shortened version of how to use our debug wrapper selective-debug.
If you aren't familiar with the basics of using a debugger like gdb, you should follow our Tutorial: Debugging MPI programs at Mogon.

This is the procedure for debugging one node from the amount of nodes you've chosen for running your program.
If you want to debug multiple nodes, you should read the section “Debugging multiple nodes at once” of our tutorial.
In this tutorial you can also find information about how you can use the other debugging utilities ltrace and strace.

  1. Make the libraries available which are needed for compilation and execution, e.g.
    $ module purge 
    $ module add gcc/4.7.0 
    $ module add mpi/openmpi/1.6.1/gcc_4.7.0
  2. Modify your program by including the “selective-debug1) wrapper and use the BREAKPOINT_AND_SLEEP() macro to halt where appropriate.
    #include “selective-debug.h”
    if(my_rank == <rank on which the problem exists>) {
    	BREAKPOINT_AND_SLEEP(10); //delay of 10 seconds, allowing gdb to start up and halt the program
  3. Compile your program with the switch -g 2), e.g.
    $ mpicc <your program> -g -o <executable program>
  4. Enqueue your job to LSF by adding your program as a parameter of “selective-debug”. E.g.
    bsub -I -n <amount of processes> -q short -a openmpi mpirun ./selective-debug <your program>

In case “selective-debug” is not available in your $PATH environment, you can view and download the current version here.
Generates debugging information. This information is stored in the object file and copied from there to the final executable file by the linker, where it can be read by the debugger. You must use the -g switch if you plan on using the debugger.
  • debug_quickstart.txt
  • Last modified: 2013/10/02 10:44
  • by ulges