User login
Qlustar: Install and enjoy!

Qlustar 9.0.0 - Release Notes

Qlustar 9.0.0 is a new major release and the first one based on Ubuntu/Trusty (14.04). It includes QluMan 2 (Qlustar's management framework) with a large number of new features and improvements (see below), BeeGFS as a new parallel filesystem option and many major component updates. Highlights are the brand-new Slurm 14.11.2 and OpenMPI 1.8.4 releases, as well as Lustre 2.6.

Qlustar 9 is based on Ubuntu 14.04.1 and Debian 7.5. It includes all security fixes and other package updates published before December 24th 2014. Available security updates relevant to Qlustar 9, that have appeared after this date, will be announced on the Qlustar website and in the Qlustar security news-letter.

Please note that this release does not support updates from Qlustar 8.1 installations yet. Update support will be added in a forthcoming maintenance release.

New Features

  • QluMan 2
    • Remote Execution Engine (RXengine):
      • Powerful command editor allows to define custom commands with precise control about the command itself (also using various substitution fields), about the list of the nodes on which to execute (via assignable Host Filters) and about the QluMan users that will be allowed to launch the command.
    • Host Filter Module:
      • Strongly improved to allow the dynamic generation of host lists based on various node properties. The filters can be used in custom RXengine commands to dynamically set the hosts where a command should be executed.
    • Slurm configurator:
      • Slurm (the Qlustar default workload manager) can now conveniently and consistently be configured using its QluMan module. It allows to set general Slurm options in text format and additionally to define partitions and node groups together with their properties. The latter can the be assigned to hosts resulting in consistent configuration files.
    • Graphical Node Monitoring:
      • LED indicators in the cluster tree view now show the state of nodes and indicate whether they are in a flawless state.

New components

  • BeeGFS
    For the first time Qlustar includes BeeGFS as a fully integrated and supported option to be used as a super-fast parallel filesystem. This is the result of our recently started cooperation with ThinkParQ. Note that Qlustar is the only Cluster OS/Linux distribution with out-of-the-box support for BeeGFS.

Major component updates

  • Slurm 14.11.2
    • Qlustar 9 features the Slurm 14.11 series with the current version being 14.11.2.
    • Qlustar's default configuration now makes full use of cgroups support. This makes sure that jobs are constrained to their allocated amount of RAM and CPUs/cores.
    • We also implemented clean PMI2 support which is now the default MPI launching method.
  • OpenMPI 1.8.4 feature release
    • Qlustar 9 features the OpenMPI 1.8 series with the current version being 1.8.4.
    • Qlustar's OpenMPI packages now provide CUDA support as well as PMI2 launching being used as the default method under Slurm.
    • OpenMPI 1.8.4 brings a significant performance boost for intra-node communication with its new Vader mechanism.
  • Lustre 2.6
    • ZFS is now the only supported back-end filesystem in Qlustar Lustre installations. Starting from this release, ldiskfs is not supported anymore.
    • Instead of the in-kernel staging client, we've reverted to use the out-of-kernel Lustre (2.6) client again.
    • Qlustar now provides a DKMS package for the Lustre client kernel modules. This can be used to provide Lustre access on non-Qlustar installations.
  • Qlustar installer
    • Allows to select BeeGFS.
    • Uses GPT partition now by default.
    • Now provided as an ISO image to be burned onto a DVD or a USB flash drive.

Other Package Version Updates

  • hwloc
    1.8 → 1.10     --     Changes

General changes/improvements

  • No more swap devices by default
    In a new default Qlustar installation, no swap devices are created per default anymore. This change was made to improve a) performance and b) stability. In particular, when using ZFS, swap devices could bring down performance significantly and in some circumstances lead to kernel dead-locks. Dedicated HPC/Cluster machine usually don't benefit from having swap devices. If you require swap, you'd have to set it up manually.
  • KVM virtual machines use virtio
    Significant performance improvements have been achieved for virtual FE and demo nodes, by using new virtio methods for network and disk devices.
  • New console utilities for virtual machines
    To provide convenient access to the console of virtual FE and demo nodes two helper aliases (console-fe-vm and console-demo-vms) for root are now defined on the head-node.

Critical Bug Fixes

  • Kernel dead-locks with ZFS
    In some circumstances, nodes using ZFS could hang. See the following upstream issue. This is fixed with the upgrade to ZFS/SPL 0.6.3+1.1-ql.2.

Known issues

  • Grub throws the error message Error: diskfilter writes are not supported and hangs for a couple of seconds. This is a bug in the Ubuntu 14.04 package but harmless. See this upstream bug report.


  • A detailed log of changes in the image modules can be found in the directories
    As an example, in the directory
    you will find a summary changelog in core.changelog, a complete list of packages with version numbers entering the current core module in core.packages.version.gz, a complete changelog of the core modules package versions in core.packages.version.gz and finally a complete log of changed files in core.contents.changelog.gz.
  • As usual, individual changelogs of packages can be found on an installed head-node in the directories
    /usr/share/doc/<package name>.
glqxz9283 sfy39587stf02 mnesdcuix8