User login
Qlustar: Install and enjoy!

Successful update of a Lustre installation

Qlustar's Lustre/HA architecture passed an important test at the end of last year when a large existing Lustre cluster installed and maintained by Q-Leap Networks was successfully upgraded from Lustre version 1.8.7 to 2.1.3.

The system is part of a multi-cluster Lustre infrastructure at the Steinbuch Centre for Computing (SCC), a department of the Karlsruhe Institute of Technology. At the time of installation in 2008 it was one of the largest Lustre deployments in Germany. This was already the 3rd major update for the system, bringing it forward from an initial 1.6.x version to the current 2.1.3. During the years, Qlustar has proven to provide the optimal setup/infrastructure even for such complex installations.

Hardware/Software configuration details are as follows:

  • Two Lustre filesystems:
    • 80TB net for home directories
    • 320TB net for scratch space
  • 2 Headnodes configured in a High-Availablity (HA) configuration
    • Two shared, external RAID systems partitioned and mirrored.
    • Serve two Lustre Metadata targets (MDT).
    • Act as Qlustar control/boot machines for the remaining storage nodes.
  • 20 Storage nodes (OSS) configured as HA pairs
    • Six shared, external RAID systems per pair partitioned and used for six RAID6 arrays.
    • Serve six Lustre Object Storage targets (OST) per pair.
    • Boot a Qlustar image over the network.
  • 4 nodes configured as HA pairs for NFS and CIFS reexport (also booting a Qlustar image)
  • Openais/corosync/pacemaker HA software stack.
  • Infiniband DDR network.
  • 980 connected Lustre clients (different HPC Clusters).

Additional information and interesting material about the SCC installations as well as Lustre in general are published at the SCC Lustre page .

glqxz9283 sfy39587stf02 mnesdcuix8
sfy39587stf03
sfy39587p08