The AVIDD-B Cluster

Hardware Configuration

AVIDD-B is a configured cluster of IBM eSeries, rack-mounted server-class system, located on the Bloomington campus of Indiana University.

Cluster Nodes

Compute (96 per cluster, eSeries x335):

  • 2 x 2.4 GHz Pentium 4 Xeons, 512K L2 cache (Hyperthreading disabled)
  • 2.5 GB 266 MHz ECC DDR SDRAM
  • ServerWorks GC-LE chip set
  • 400 MHz front side bus
  • 2 x 18 GB Ultra320 SCSI Disk, RAID1 (13 GB of local scratch)
  • 2 x 1GigE Broadcom 5703
  • 2 x PCI-X 64bit/100MHz PCI
  • 1 x Myricom M3F-PCI64C-2 (Myrinet interconnect)

User (3 on AVIDD-B eSeries x345):

  • 2 x 2.4 GHz Pentium 4 Xeons, 512K L2 cache (Hyperthreading disabled)
  • 2.5 GB 266 MHz ECC DDR SDRAM
  • ServerWorks GC-LE chip set
  • 400 MHz front side bus
  • 2 x 36 GB Ultra320 SCSI Disk, RAID1 (27 GB of local scratch)
  • 3 x 1GigE Intel PRO/1000
  • 4 x PCI-X 64bit/100MHz PCI
  • 1 x Myricom M3F-PCI64C-2 (Myrinet interconnect)

Storage (4 per cluster, eSeries x345):

  • 2 x 2.4 GHz Pentium 4 Xeons, 512K L2 cache (Hyperthreading disabled)
  • 4 GB 266 MHz ECC DDR SDRAM
  • ServerWorks GC-LE chip set
  • 400 MHz front side bus
  • 2 x 36 GB Ultra320 SCSI Disk, RAID1
  • 2 x 1GigE Intel PRO/1000
  • 4 x PCI-X 64bit/100MHz PCI
  • 1 x Myricom M3F-PCI64C-2 (Myrinet interconnect)

Networking

There are two major interconnects on the AVIDD-B cluster: a Myrinet M3-E128 switch provides 2+2Gb/s bandwidth with latency on the order of 7 microseconds; a Force10 E600 switch provides Gigabit Ethernet. The Myrinet switches are internal to the cluster, while the Force10 switches are bridged via two trunked 10 Gigabit/s lines running 80km between Bloomington and Indianapolis.

Storage

The storage nodes provide access, via Gigabit Ethernet on AVIDD-B to IBM's General Parallel File System (GPFS), which is distributed over approximately 60 36GB Fiber Channel disks. AVIDD-B has a 1.6TB filesystem accessible from all respective compute and user nodes. GPFS write performance has been benchmarked at 64MB/s writes and 170MB/s reads, using 32 processes, MPI-IO, and a 32GB file.

Home directories are available via NFSv3 over Gigabit Ethernet.