• Print
• View
• History
• Login

Developing, compiling etc

This page provides information on issues specific to developing for the UJ cluster. For more general information on developing in Linux please see the pages on Programming in the PSI wiki.

Compiling your source code

The UIs have the compilers of the GNU Compiler Collection - gcc, g++, g77 and gfortran. Other compilers could be made available on request, please contact the System Administrator.

The Worker Nodes have AMD Opteron CPUs and run Scientific Linux 4.7 64-bit (x86_64); the UIs instead run Scientific Linux CERN 4.6 32-bit, for maximum compatibility with the Grid software stack. Because of this, you cannot compile in 64-bit mode on the UIs. You may still compile on your own PC with SL 4 64-bit, and transfer the compiled program to the cluster, to be run on the WNs.

It is strongly recommended that you turn on the compiler optimisations for CPUs. The following seem to give good results on floating-point intensive code: -O3 -march=nocona -msse3 -mfpmath=sse -fomit-frame-pointer -finline-functions . If your code is well tested and does not rely on IEEE standard treatment of Floating Point errors (NANs and INFinities), you may see an improvement adding -ffast-math -funsafe-math-optimizations -ffinite-math-only. See Performance section below for details.

Libraries and compilers

With few exceptions, only standard libraries are installed on the Worker Nodes. Upon request, software available as RPM install packages from the main RPM repositories may be installed by the sysadmins, if it does not risk to conflict with Grid requirements. Non-packaged or conflicting software may be installed by the user himself either in his own directory or, if of interest to other users, in a shared directory, to be agreed with the sysadmins.

If you use Grid tools and expect to run your code also on remote sites, you should not expect any software to be installed there, unless dictated by the Virtual Organisation.

The compilers available on the cluster are the standard GCC suite. Other compilers may be considered, if motivated and funded. Please note that the Intel compilers are only free for personal use and they must not be installed on the cluster UI without a paid license.

Developing and testing your code outside of the cluster

For many reasons it may not be most practical to develop your code directly on the cluster UI servers. No Graphical User Interface is provided, only basic debugging tools are available, the capacity of the UI is limited and must be shared fairly with other users, and finally, without administrative access you cannot use OProfile.

If you are a Windows user, you can choose to:

• Install Scientific Linux 4 on your own PC
• Use the Scientific Linux 4 LiveDVD, which can be run without installing in a dedicated hard disk partition.
• Install Scientific Linux 4 in a Virtual Machine. VirtualBox, http://www.vmware.com/products/player/?, VMWare Server 2 are free alternatives for virtualisation; the preference of your SysAdmin, for desktop use, goes to VirtualBox. VMW Player is easy to use, but requires a pre-made VM image; VMW Server is more complex and does not have a bit performance advantage over VB.

If you are a Mac OsX user, you probably don't need to get Linux, because much computational code will compile and run with minimal changes on both. If you find you must, VirtualBox is probably the best alternative, if you don't want to spend for VMWare Fusion. Parallels Desktop is to be avoided, as its integration with Linux is much worse.

If you are a user of a different version of Linux (Scientific Linux 5, Ubuntu, SuSE...), it's unlikely that you will need to do anything special to run your code on the cluster. You may have to adapt your code to the specific version of compilers used in SL4 (ggc=v3.4, gcc4=v4.1.2). In the worst case, you may compile your code with static libraries and copy the executable to the UI.

Performance Testing the PBS system with Blankenbecler simulations, running from glite-ui.

# #install compilers on UIs
# yum install gcc4-gfortran gcc-g77 gcc-c++
# #install libraries on WNs 
# shm yum -y install libgfortran libf2c libstdc++

Compiler options for the cluster

The above code was compiled on glite-ui (SLC4.6, 32bit) in 32bit mode, and ran on both glite-ui and WNs, to choose the best optimisation options.

With g77, I don't see any significant difference between -march=nocona and -march=opteron when running either on glite-ui (2GHz Intel Xeon E5405, under VMWare) or on the WNs (AMD Opteron, native). On gfortran instead there is a dramatic difference for the worse with -march=opteron. For comparison, GCC 4.3/gfortran on a MacBook (2GHz Intel Core Duo [Yonah], OsX 10.5.6)

Please note that GCC 3.4 does not support -march=core2, and GCC 4.1 does not support -march=opteron-sse3 or -march=barcelona. We would need to install GCC 4.2 to have the best optimisation for the CPUs of the WNs.