The purpose of burst buffers is to absorb bulk data produced by applications a hundred times faster than what the parallel file system can absorb while draining the data to the PFS on the back end. Burst buffers sit between the HPC application and the parallel file system as an intermediate high speed layer of storage.
A burst buffer consists of a combination of rapidly accessed persistent memory with its own processing power. The persistent memory is positioned between a set of processors with their non-persistent memory counterparts and a chunk of symmetric multi-processor compute through direct PCIe high bandwidth links. The back end of the burst buffer is comprised of slower large capacity storage systems.
In other words: Would you like to add a hyper-fast storage tier between the compute nodes and your parallel file system? Then you have come to the right place.
A little more detail on the specifics. The burst buffer’s purpose is to allow applications running on an SMP’s fast processor cores to perceive that the application data – data first residing in the SMP’s local and volatile memory – will be quickly saved on some persistent media. As far as the application is concerned, its data, once written into the burst buffer, had become persistent with a very low latency; the application did not need to wait long to learn that its data had been saved. If the power goes off after the completion of such a write, the data is assumed to be available for subsequent use.
Where would I use a Burst Buffer?