Recent Changes - Search:


edit SideBar

Install Introduction

1.  Guiding requirements, ideas and design


  • An HPC cluster for general use
  • Focus first on High Energy Physics work for ATLAS
  • Support Open Science Grid
  • Also support EGEE gLite


  • A head node that concentrates all services and connects to the external world
  • computing nodes (Worker Nodes or WNs in Grid speak) on a dedicated network
  • OSG and gLite CEs cannot be on the same host, so we use Virtual Machines
  • also gLite CE and UI cannot be the same host
  • for symmetry, also separate OSG CE and UI
  • the services that are not related to a specific Grid (like PBS, NFS etc) are ran on the non-virtual machine instead.
  • we also keep a couple of VMs for WNs, to use all the CPU power of the head node.

Operating System

  • gLite 3.1 requires Scientific Linux 4. Prefers 32bit on CE and UI
  • OSG can run on SL4, 64bit is fine
  • the bare metal requires 64bit OS to use the 16GB of RAM

so we run SL4.7 x86_64 on the bare metal (head and WNs) and SLC4.6 i386 on the CE and UI VMs.


  • both Xen (used by UJ IT dep) or VZ (used on Wits head node) could be nice.
  • VZ may be trickier because of 64/32 bit issues (gLite less tested on 64bit)
  • but we have ready-made WMware VM images of SLC4 from Gilda, ready for installing gLite CE and UI, so we start with VMWare Server.
  • the VMWare web-based GUI does not work on OsX, or on SL4
  • it's not very well documented or publicised, but one can do almost everything without using the VMWare GUI


  • users should only have interactive login access to the UIs, not the CEs, the WNs or the head
  • ssh root login only on internal WNs
  • use iptables to restrict access


  • try to centralise configuration as much as possible
  • but we are in an hurry, and I'm doing this setup remotely, so skip netbooting
  • skip Oscar too, not time to learn that one as well
  • NIS helps for users and few other things


  • we need to get all the mail sent from all the systems, as it contains logs, error reports etc
  • do we need any outgoing mail ?

2.  Hardware and services

  • gridvm head node
    • 8 CPUs: 4 Xeon E5405 2GHz (quad-core)
    • RAM: 16GB
    • Disk: 900GB by 6 (or7?) SAS HDs on RAID5
    • Net:
      • eth0 on UJ network and outgoing (
      • eth1 on cluster switch (
    • OS: SL4.7 x86_64
    • services: NFSv4, NIS, PBS/Torque, Postfix SMTP
    • VMWare virtual hosts:
      • osg-ce
      • osg-ui
      • glite-ce
      • glite-ui
  • wn001-wn007 worker nodes
    • 8 CPUs: 4 AMD Opteron 2350 2GHz (Barcelona)
    • RAM: 16GB
    • Disk: 2x150GB SATA
    • OS: SL4.7 x86_64
Edit - History - Print - Recent Changes - Search
Page last modified on April 11, 2009, at 07:11 PM