Installation Instructions for ATLAS Condor/PBS PANDA Job Monitoring

Below are the instructions for install PANDA job monitoring for ATLAS on your cluster. You will need to jump to either the PBS or Condor section for the details.

Prerequisites

You will need to have the PANDA/OSG environment installed as well as:

Web-server (Apache)
Batch system specific software:
- Condor: Need condor_q command, with condor_q -global able to show queue contents for your cluster
- PBS: Need qstat command, with qstat able to show queue contents for your cluster
Perl with modules POSIX, SDBM_File, Fcntl (all should be standard)
The unix commmands awk, grep, tail, ln
The Globus grid-cert-info command

Condor Details

Select a location which can be "served" by your web browser. Download the make_condor_mon.pl script from http://gate01.aglt2.org/Monitoring/make_condor_mon.pl and store it in this selected directory as <your_directory>/make_condor_mon.pl . I choose to store the scripts in the same directory where I put the output html file (served by Apache). You can store it elsewhere, just "point" to the correct location in your crontab entry (see below).

Edit the perl script to insert your local values for:

Binary locations: condor_q, awk, tail, grep
Status page output location (/var/www/html/Monitoring)
URL corresponding to status page output location (/Monitoring)

Create a subdirectory call CLog (/var/www/html/Monitoring/CLog), where we will store the softlinks to the Condor job logfiles.
Test the script "by hand" to insure it creates the output page:

perl make_condor_mon.pl

This should produce condor_jobs.html in the specified output location. Make sure you can view it on your web browser.

PBS Details

NOTE: My PBS monitor is somewhat out of date!
Select a location which can be "served" by your web browser. Download my current make_pbs_mon.pl script from http://gate01.aglt2.org/Monitoring/make_pbs_mon.pl and store it in this selected directory as <your_directory>make_pbs_mon.pl. I choose to store the scripts in the same directory where I put the output html file (served by Apache). You can store it elsewhere, just "point" to the correct location in your crontab entry (see below).

Edit the perl script to insert your local values for:

Binary locations: qstat, awk, tail, grep
Status page output location (/var/www/html/Monitoring)
URL corresponding to status page output location (/Monitoring)

Create a subdirectory call PLog (/var/www/html/Monitoring/PLog), where we will store the softlinks to the PBS job logfiles.
Test the script "by hand" to insure it creates the output page:

perl make_pbs_mon.pl

This should produce qstat_jobs.html (or whatever you set the variable for the output file to) in the specified output location. Make sure you can view it on your web browser.

Install Into cron

Once you can make the appropriate output file "by hand", you should install it into the crontab of a user capable or running the script and producing output into your selected directory. For example, on the Michigan cluster I choose to install the monitoring scripts into root's crontab as follows:

su (change to root)
export EDITOR="emacs -nw"
crontab -e
Add a line: */5 * * * * /usr/local/bin/perl /var/www/html/Monitoring/make_<x>_mon.pl
The <x> is either condor or pbs and your local perl and install directory may be different.

Verify that you web page(s) update every 5 minutes.

That's it...you should be successfully monitoring your Condor/PBS queues for DC2.
Let the list (and Horst) know about your URL.

Contact Info