A new alternative to parallel file systems or software defined storage for archiving is Quobyte. Here we will outline current design highlights to build a high performance cluster utilizing Quobyte. Why Quobyte? The value in Quobyte is that it's a storage system of a new generation. Quobyte offers a solution with unified storage, simplified operations, [...]
On-Prem to Cloud Deployment of Elastifile Cloud Bursting? Looking to extend HPC with Bright and Elastifile? Here are some points of consideration when using Bright Cluster Manager Portal and Elastifile as the extended file system for the datacenter to a cloud like Amazon Web Services (AWS) or Azure. Elastifile can be deployed as bare metal, [...]
Looking to upgrade your Bright Cluster Manager to 8.0? Depending on your existing cluster components, you will need to update your OS and scheduler as well. Here is a guideline on how to deploy a Bright/RHEL7/Slurm upgrade. Prepare environment for a Bright/RHEL7/Slurm upgrade. Inspect system state, queues and health. Backup OS, applications and settings. Prepare [...]
Statistical genomics require rapid rehashing of a central, consistent dataset against an ever-evolving set of variables to strike correlations. This central consistent dataset is often fairly small and serves as a reference to seek variability between genomes. Every sequenced genome is the product of an extensive computational process that transforms hundreds of gigabytes of raw sequencer output [...]
The purpose of burst buffers is to absorb bulk data produced by applications a hundred times faster than what the parallel file system can absorb while draining the data to the PFS on the back end. Burst buffers sit between the HPC application and the parallel file system as an intermediate high speed layer of [...]
With LSF, as an end user, you have some tools and methods to inspect jobs while they are running. An example would be using the bpeek command. bpeek shows you the standard output and standard error output produced by an unfinished job at the moment in time it is invoked. Three reasons why you would run this command: It is useful in monitoring the progress of [...]
OpenStack is an open-source software platform for cloud computing, mostly deployed as an infrastructure-as-a-service (IaaS) within cloud providers. The OpenStack software platform consists of interrelated components controlling hardware pools of processing, storage, and networking resources throughout a data center. OpenStack has become a powerful contender for building private clouds to automate: Compute Storage, backup and [...]
The drumbeat of an SSD/Flash/NVMe replacing disk sounds like the old tape replacement by disk in years past. It appears that as tape did not disappear completely from most data centers, disk will have a place in the data center ecosystem for years to come. However, the density and cost of chip-based storage media is [...]