Move Over Lustre & Spectrum Scale – Here Comes BeeGFS?
Is BeeGFS – the parallel file system with European roots – on a path to compete with Lustre and Spectrum Scale worldwide? Frank Herold’s unsurprising answer is yes. Herold is CEO of ThinkParQ, the company created in 2014 to commercialize BeeGFS. You may remember that BeeGFS got its start as an in-house project (2005) at the Fraunhofer Institute for Industrial Mathematics (ITWM) and was called the Fraunhofer Gesellschaft File System (FhGFS). It was later spun out as BeeGFS under ThinkParQ’s control although a fair amount of development still occurs in collaboration with ITWM.
Leaving the history aside for a moment, Herold was busy at SC18 this month making the case that BeeGFS’s mature technical strength combined with a solidifying channel landscape make it a worthy alternative (rival) to Lustre and IBM Spectrum Scale. In making the case, Herold highlighted a recent win with partner Dell EMC to deploy BeeGFS at CSIRO (Commonwealth Scientific and Industrial Research Organization) in Australia.
“We started going more international a year and a half ago. It’s interesting that the number of international projects are not high, but from a project size perspective those projects are quite significant. As of today, combining North America and Asia Pacific, roughly 45 percent of our revenue is from those regions,” Herold told HPCwire. “We have a very significant deal with our partner and Dell EMC at CSIRO. It’s a 2-Petabyte, all NVMe, storage solution built for AI. This is in deployment right now.”
That’s a big win and potential proof point. ThinkParQ doesn’t publicly report revenues so it’s hard to accurately judge market traction and because BeeGFS is open source it is difficult to know how many organizations are using it or what it is being used for. Nevertheless, the timing of ThinkParQ’s push may be spot on.
Lustre’s future has seemed hazy lately. It is very strong in supercomputing centers but not so much in commercial HPC. Intel purchased Lustre provider Whamcloud in 2012 intending to build up a Lustre business, but exited the business in 2017. This past June Intel sold its Lustre assets to DDN, which, as a leading HPC storage provider with several Lustre appliances, may become a better steward for Lustre. (see HPCwire articles, Intel Open Sources All Lustre Work, Brent Gorda Exits, DDN Acquires Lustre File System Capability from Intel.)
In 2015 IBM rebranded its popular General Parallel File System (GPFS) as IBM Spectrum Scale. GPFS has also been a strong player in HPC and especially strong in high end enterprise computing.
Practically speaking, no parallel file system dominates commercial HPC noted analyst Addison Snell, CEO, Intersect360 Research, “BeeGFS has grown in popularity, particularly in Europe. However, no parallel file system is yet to be broadly adopted across commercial HPC segments, which constitute the majority of HPC usage.”
One observer directly involved in HPC storage technology selection and deployment agrees interest in BeeGFS is rising.
“With the current transition and uncertainty present both for Lustre and Spectrum Scale offerings in HPC, it is no surprise that this year there has been more buzz (forgive the pun) around BeeGFS than before,” said Aaron Gardner, director of technology, BioTeam, a research computing consultancy specializing in life sciences. “HPC has long been searching for a distributed parallel file system alternative. We’ve watched BeeGFS evolve over the past few years into a potential contender.” (diagram of architecture below.)
For the complete story on sister publication HPCwire, please go to: https://bit.ly/2zGpJ9B
