Parallelization

General description

The world of supercomputers and large clusters is characterized by the rapid developments in available hardware and software. The speed of the supercomputers and, the amount of memory and disk space are growing exponentionally. At first sight, one could think that users of supercomputers are able to tackle larger and larger projects every year.

Alas, this is not always the case. The growth of supercomputer power is to a large extent possible by architectural changes in the hardware which necessitate special programming paradigms. At the HPCV department of SARA we have experience with a number of supercomputer architectures and we learned how to program them efficiently by assisting users with optimizing their programs.

Adapting a computer program to a parallel supercomputer

Parallel computer architectures are common in the world of supercomputers. It appears that adapting or building computer programs for these parallel architectures requires special skills. Normally, users don't have the time to acquire these skills and cannot benefit from the developments in supercomputer technology. Therefore, SARA decided to acquire the necessary skills for parallel programming. This initiative resulted in a number of parallelization projects, most of which were funded by NCF.

A typical parallelization project

In general, the following steps are taken when SARA parallelizes a computer program:

• Talk with the owner/writer of the program. The purpose of this is to get an understanding how the program is structured, so that opportunities for parallelization can be analyzed.
• Decide upon the parallelization method. The choice is dependent on the program at hand and on the computer architecture, not necessarily one of SARA's supercomputers.
• Optimize the serial version of the program. The techniques used for optimizing serial programs are well-known and quite general usable. In practice, we see improvements in this fase ranging from 20 % to a factor of 20.
• Parallelize the program mostly using OpenMP or MPI.
• Run benchmarks and perform scaling experiments. Produce report.

Parallelization projects

We have reports about our parallelization projects available.

More info

Contact: hic@sara.nl