User Tools

Site Tools


development

Development on JGU HPC Systems

Introduction

The JGU HPC group encourages to develop on its HPC systems: Only if tested on an HPC system, scalability can be tested1). For this, we provide a number of tools from compilers to debugging tools.

Collaborating with the HPC Group

We are open for two kinds of collaborations with regard to development:

  • Code Review in a systematic fashion as part of supervising a student's work (please approach us, beforehand!).
  • As a genuine collaboration with either a grant proposal in mind or after writing one together.

Getting Compute Resources

Development is / can be cumbersome, particularly, if you would have to wait for test resources. Hence, we introduced the devel partitions on both clusters for short and not-to-big jobs. More information on this partition2) can be found in our partition description.

IDEs

Monitoring

Our Monitoring Page describes tools to monitor the cluster nodes (Ganglia) and shell based tools to monitor compute tasks with regard to CPU usage, memory, open files, I/O statistics.

Source Code Management

Source Code Management is an integral part of collaborative development and essential for reproducible research and for supervising students.

The ZDV3) offers a gitlab service, which allows to keep your work, your student's or other's work confidential and to publish it at any desired time. The ZDV offers courses on regular intervals, please see our enrollment page; please approach us, if additional courses are required.

Parallelization Libraries and Libraries to support parallel Work

In order to accomplish parallel applications of high efficiency, a number of libraries are provided:

Parallel File I/O

  • HDF5 under modules of the name data/HDF5.
  • netCDF under modules of the name data/netCDF

MPI

MPI (the Message Passing Interface) is the most common parallelization library on HPC systems. We think this merits a dedicated wiki entry, here.

Performance Optimization Tools

Performance Analysis Tools are crucial to find bottlenecks and to remedy performance issues.

Debugging Tools

Without Debugging Tools no greater software project can be productive. We provide a number of tools at the given link.

Scripting Languages

On HPC systems all scripting languages should serve as a glue (e.g. wrappers, pipe-lining code or similar) for compiled programs, only. The following pages contain some hints on performance, respectively performance considerations.

Here, we list a number of scripting languages and some considerations / hints for usage on our HPC systems:

1)
You can never be save on the parallel efficiency of a given application, as it may differ from system to system.
2)
as it might change
3)
not the HPC Group
development.txt · Last modified: 2019/10/18 10:51 by meesters