Compiling Jobs on Triton

Porting Existing Programs to Triton

If you have an existing serial or MPI-based parallel application program already running on a distributed-memory platform:

Copy your application source files to the Triton Resource, to your $HOME directory or to the /phase1 area associated with your user account:

/phase1/<username> where <username> is your Triton login name

In the coming weeks, /phase1 will be superceded by /oasis. For a short time, both systems will be available.

Note that the /phase1 area is not backed up and files stored on this system may be lost or destroyed without recourse to restore them. Long-term file storage should be maintained in your $HOME directory or on HPSS.

Compiling Serial and MPI Programs

Compiling MPI Programs

MPI source code should be recompiled for the Triton system with the following default compiler commands:

• mpicc [options] file.c (C and C++) myrinet/mx switch and the Portland Compiler
• mpif77 [options] file.f (Fortran 77 source code) myrinet/mx switch and the Portland Compiler
• mpif90 [options] file.f (free format code/dynamic memory allocation/object oriented Fortran source code) myrinet/mx switch and the Portland Compiler

Other MPI stack/compiler combinations may be obtained by choosing the appropriate modules. The choices include:

Myrinet

• Portland Group compilers + myrinet/mx
• Intel compilers + myrinet/mx
• Gnu compilers + myrinet/mx

Gigabit Ethernet

• Portland Group compilers + gigabit ethernet
• Intel compilers + gigabit ethernet
• Gnu compilers + gigabit ethernet

Compiling Serial Programs

Serial source code should be recompiled for the Triton system with the following compiler commands:

Portland Group Compilers

• pgcc [options] file.c (C and C++)
• pgf77 [options] file.f (fixed form Fortran source code)
• pgf90 [options] file.f90 (free format Fortran source code)

Intel Compilers

• icc [options] file.c (C and C++)
• ifort [options] file.f (fixed form Fortran source code)
• ifort [options] file.f90 (free format Fortran source code)

Gnu Compilers

• gcc [options] file.c (C and C++)
• g77 [options] file.f (fixed form Fortran source code)
• gfortran [options] file.f90 (free format Fortran source code)

Compatibility Options for Fortran

• -Mcppruns the Fortran preprocessor on source files prior to compilation
• -Dname[=value]specifies name as a definition to use with conditional compilation directives or the Fortran preprocessor (-Mcpp)
• -silent/-w/-Minform=severesuppress messages about use of non-standard Fortran
• -byteswapioswap bytes from big-endian to little-endian or vice-versa for unformatted files
• -i8set size of INTEGER and LOGICAL variables to 8 bytes
• -i4set size of INTEGER and LOGICAL variables to 4 bytes
• -i2set size of INTEGER variables to 2 bytes
• -r8treat REAL and CMPLX types as REAL*8 and DCMPLX
• -Msavesave all local variables between calls (static allocation)

Detecting Programming Errors

• -g produce symbolic debug information in object file (implies -O0); required for debugging with DDT
• -C/-Mboundsarray bounds checking
• -kTrap=(option, option...)specifies behavior on floating point exceptions (used only for main program):
  • dnorm trap on denormalized (very, very small) operands
  • divz trap on divide by zero
  • fp trap on floating point exceptions
  • inexact trap on inexact result
  • inv trap on invalid operands
  • none (default) disables all traps
  • ovf trap on floating point overflow
  • unf trap on floating point underflow
• -traceback add debug information

Optimization

Optimization levels of the Portland Group compilers are:

• -O0 No optimization
• -O1 (default) Task scheduling within extended basic blocks is performed. Some register allocation; no global optimizations.
• -O2 all level 1 optimizations and global scalar optimizations (optimization over all blocks)
• -O3/O4 all level 1 and 2 optimizations as well as more agressive optimizations
• -Mflushz flush very, very small values to zero.

Alternatively, the Portland Compiler User Guide recommends the use of the -fast flag. This option is host dependent and typically has the following effects:

• set the optimization level at -O2
• unroll loops (-Munroll=c:1)
• do not generate code to set up a stack frame pointer for every function (-Mnoframe)
• enable loop redundancy elimination

Numerical Libraries

The Portland Group compilers come with the Optimized ACML library (LAPACK/BLAS/FFT).

To link:

pg90/pgf77 myprog.f -llapack -lblas

Intel has developed Math Kernel Library (MKL) which contains many linear algebra, FFT and other useful numerical routines.

• Basic linear algebra subprograms (BLAS) with additional sparse routines
• Fast Fourier Transforms (FFT) in 1 and 2 dimensions, complex and real
• The linear algebra package, LAPACK
• A C interface to BLAS
• Vector Math Library (VML)
• Vector Statistical Library (VSL)
• Multi-dimensional Discrete Fourier Transforms (DFTs)

Documentation is available in HTML and PDF formats in ${MKL_ROOT}/../Documentation.

To link the MKL libraries

To link the MKL libraries, please refer to the Intel MKL Link Line Advisor Web page. This tool accepts inputs for several variables based on your environment and automatically generates a link line for you.

When using the output generated by this site, substitute the Triton path of the Intel MKL for the value $MKLPATH in the generated script. That value is ${MKL_ROOT}/lib/em64t.

All third-party applications can be found in /opt, or view the complete description of Triton software packages on the Triton Software page.