UJ Research Cluster | User / Torque, the Batch System

The UJ Cluster uses Torque, a derivative of OpenPBS, for the job queue management; it uses MAUI as job scheduler.

Quick start for beginners

First of all, you must login on one of the UIs (glite-ui or on osg-ui) using ssh. SSH is available from the terminal window in Linux, Os X and other Unixes; from a Windows PC you may use

If you have your own program or other data files that you need to copy to the cluster, you can use scp, sftp or rsync. On OsX, a good GUI client is CyberDuck (OpenSource), which also allows direct editing of remote files.

Depending on users' requests and security considerations, we may consider allowing direct SMB/CIFS (Windows file sharing) access to certain filesystems of the cluster, from within the UJ campus.

Compiling your source code

Once you have copied your source code and data to the UI, you will need to compile the sources to produce an executable. The UIs have the compilers of the GNU Compiler Collection - gcc, g++, g77 and gfortran, versions 3.4 and 4.1. Other compilers could be made available on request, please contact the System Administrator.
See the Developing page for more details.

Submit a job from the command line

[user@osg-ui ~]$ echo "/bin/hostname" | qsub -q batch

The output will appear in the directory from which the command was issued, as STDIN.o# (the stdout, Fortran UNIT=6) and STDIN.e# (the stderr, Fortran UNIT=0) where the # is the number of the job, which is printed out when you run qsub. Any serious error in the execution of the job will be sent by mail. Please note that at the moment mail to users is disabled. If many of your jobs fail, please contact the system administrator.

To give a different name to the job, instead of STDIN, use qsub -N myFancyJobName

Please note that the jobs are started on a different machine (a Worker Node), in the home directory of the same user. If you want your job to start with a different current directory, you could use

[user@osg-ui ~]$ echo "cd $PWD;./myprogram" | qsub -q batch

The home directories are shared between the UI and the WNs via NFS, so files produced under the home directory will be immediately visible on the UI and all the WNs. Because of this, you must be careful, if you submit more than one job to the queue, that they don't write to the same files.

If you care about performance, you should also avoid writing large temporary files to the NFS areas like home directories; you can instead use the /tmp area which will be local to the single WN.

Which jobs are running ?

 qstat
 qstat -a 
 qstat -f

Which other queues are available ?

  qmgr -c "print server"

Not so quick start...

Unfortunately the on-line documentation for Torque is detailed but not very user-friendly. If you find a good tutorial, please inform the sysadmin so that he can add it here.