The drumbeat of an SSD/Flash/NVMe replacing disk sounds like the old tape replacement by disk in years past. It appears that as tape did not disappear completely from most data centers, disk will have a place in the data center ecosystem for years to come. However, the density and cost of chip-based storage media is becoming increasingly more attractive for Information Technology executives.
Adaption of SSD/Flash in the HPC ecosystem have been for very specific use cases:
- file system metadata in large GPFS or Lustre environments,
- smaller (relative to overall capacity) landing or scratch areas, and
- within virtualization environments.
One of the challenges that SSD/Flash faces in the HPC space is the relatively large amounts of “hot data”. Causing a problem because providing a large capacity storage subsystem front ended by a small flash cannot perform as the network throughput between the storage subsystem the storage controller and the target host becomes a bottleneck. This limitation is not as visible in Enterprise environments as the relative amount of “hot” data is usually measured as a single digit percentage of the overall capacity, hence a huge increase in IO and latency is seen over a disk-based subsystem.
Further, storage subsystem controllers are often saturated at a fraction of the overall potential of the aggregate IO of the system. Think of it this way, we can put 24 SAS-based SSDs into a single 2U chassis, however, the most throughput we can get out of the PCIe backplane is 5GB/s, far less than 24 SSDs can provide.
A string of new technologies that incorporate a scaleout/scaleup methodology intend to revolutionize flash, specifically, the NVMe market. With NVMe devices approaching 10TB in raw capacity and the price steadily declining, vendors are taking advantage of an all NVMe storage environment using a network mesh to tie NVMe devices in commodity servers together and present to targets as block storage.
Excelero, a leader in this new NVMe technology, is a Software-Defined Storage NVMesh that enables customers to design Server SAN infrastructures for the most demanding enterprise and cloud-scale applications, leveraging standard servers and multiple tiers of flash. The primary benefit of NVMesh is that it enables true converged infrastructures by logically disaggregating storage from compute.
In a recent design, Data In Science Technologies (DST) has architected one such NVMe software defined block storage solution as a general purpose high performance SAN for a research institution using Penguin servers, Mellanox 40GbE cards and a pair of Mellanox 40GbE switch environment running Broadcom OS. Applications such as GPFS and a high-performance Linux-based virtualization environments can take advantage of the same storage resources across the environment without having to purchase purpose-built siloed storage.