debugging:valgrind

Valgrind

Valgrind is - to quote its web site - “an instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.”

In particular Valgrind is frequently used to:

  • detect memory problems (e.g. leaks and cache misses)
  • detect problems related to branch misses

While the program suite is able to profile parallel programs (e.g. those with threading by OpenMP or even multiprocessing with MPI) it does not run programs in parallel. Threads for instance are serialized. This results in an fairly big overhead. If you want to analyze MPI-Programs, you may want to build Valgrind for the respective program yourself – you can ask the HPC-team for support1).

On Mogon, valgrind is compiled on the a*-nodes and available as the module debugger/Valgrind/<version>-<toolchain>.

Valgrind's documentation is pretty good and straight forward, we recommend the quick start tutorial as well as the actual user manual.

When submitting a job for profiling consider the run time overhead or else the profiling might be partial, skewed and the output abbreviated: The run time overhead of valgrind is substantial (sometimes » 10x), therefore the test case should be small enough to run in reasonable time, yet big enough to yield meaningful results. This might require a trial and error approach for first set-ups.

Remember to compile with -g -O0 (gcc, apply similar options for other compilers). -O1 might also work, but line numbers will not be accurate any more.

Imaging you submit a job like:

 srun -p short -A <your account> -n 1 --mem 1800M -t 300 valgrind [other valgrind args] --tool=callgrind <path/to/cmd> [args]

where cmd is the program to profile with its optional arguments (args).

This will produce an output file (e.g. callgrind.out.44289, with 44289 being the PID of the process) which you can further analyze on the command line:

callgrind_annotate --auto=yes callgrind.out.44289 |less

Or – on login nodes – with kcachegrind:

kcachegrind callgrind.out.44289 &

giving results like (displaying a function with obvious potential for improvements).


1)
It might be simpler to try Allinea, which is installed on Mogon, too.
  • debugging/valgrind.txt
  • Last modified: 2019/05/22 10:10
  • by meesters