• Print
• View
• History
• Login

Space, Power and Cooling Requirements

These notes are from July 2006. The may not be actually relevant now.

The space requirements are extremely modest - in fact, we could even host all the 500 CPUs of a full-scale ATLAS Tier-2 in as little as 2 standard racks.
But I (Sergio) started to think about power and cooling requirements, and here the figures are pretty impressive instead. I've found some interesting considerations on this thread: http://www.beowulf.org/archive/2005-June/013072.html What they are talking about are pretty large clusters, but the basic concepts would apply also to a small cluster in a small room.

Each 14-node , 7U height Blade Chassis (like IBM's http://www-03.ibm.com/servers/eserver/bladecenter/chassis/) is rated at 2000W. The management/file-server unit and its disk array will be at least another 500W.
Therefore a 3000VA, 3-U height rackmount UPS (like APC's http://www.apcc.com/products/family/index.cfm?id=164) is needed for powerline protection - pretty much necessary given the frequency of powerfailures here in Jo'burg. Many computational clusters use no UPS for computational nodes, but in general Grids use the computational nodes for storage too [must check more carefully the documentation of ATLAS to see if that's the case] so it might be necessary to protect all the nodes, at least to guarantee a clean shutdown.

Cooling for 2500W also must be taken into account - not huge, but it's already more than one standard aircon. And if it fails, in a small room the temperature can rise quickly, and the damage to the equipment would be terrible. Now, if you scale up these figures not to the full node, but just a full rack (2 cores*14 blades/chassis*5 chassis/rack = 140 CPUs ), you have ~11KW of power to protect and to dissipate. Definitely non trivial...

Looking at these things, I would say that I would feel much better if the node was placed in a proper "computer room", where these matters have been addressed already. I've anyway written the piece for requesting "normal" space - see below.

We propose an initial configuration consisting of 10 computational units and one service/management unit.

Each computational units will host one dual-core CPU (AMD Opteron 175), either in a 1-U rackmount case [like Sun Fire X2100 or IBM e326], or in a Blade case [like IBM LS20]. The service/management unit will be a single- or dual-CPU system in a 1-U or 2-U rackmount case, with an external RAID storage in a 3-U rackmount case.

The expected power requirements for the initial configuration are in the order of 2000-3000 Watts, with protection by Uninterruptible Power Supply for at least 3000VA. Adequate cooling power must be available to ensure proper functioning and to avoid reducing the lifespan of the equipment.

The complete initial configuration can be accomodated in one standard 42-Units EIA 19" rack, 100cm x 60cm base, 206cm height. Some space is required to access both front and back sides of the rack, so we should consider approximately 3 m2 of floor space for the rack. Sufficient space for one medium size desk (approx. 5 m2) should be made available next to the rack, for setup and regular on-site maintenance. Altogether only a relatively small room is necessary, in the order of 10-15 m2; a larger room might ease the requirements on cooling and ventilation, but it cannot be used as office space because of the high level of noise that will be produced by the equipment and by cooling and ventilation systems.