HPC in the Datacenter: Together or Apart?
High performance computers do not necessarily require their own specially designed datacenters, yet even when design, cooling, or other concessions are necessary, benefits far outweigh any upheaval.
"There is a lot of misinformation out there that prevents folks from acting. It costs too much. It takes up lots of space. It needs its own climate-controlled room. It requires hundreds of computers," said Tim Lynch, CEO of HPC vendor Psychsoftpc, in an interview. "All pretty much false. A working cluster can be made with as little as five machines and a gig switch. That doesn't take up much room at all and the cost is not prohibitive."
A growing number of today's newer advanced scale computing systems – some of which come from vendors ensconced across the server spectrum – are designed for traditional datacenters, executives said. By focusing on cooling and power consumption during development, vendors are closing the gap between traditional and high performance computers, they noted.
Industries relying on digital innovation or digital manufacturing for success see HPC as critical infrastructure since they support the strategic and competitive capabilities that differentiate an organization, Per Nyberg, senior director of worldwide business development at Cray, told EnterpriseTech. Whether or not they require a separate datacenter, therefore, becomes a less important factor.
The Design Difference?
HPC environments require a differently designed facility from the ground up, said Cortney Thompson, chief technology officer at datacenter service provider Green House Data and a member of the National Science Foundation International's Committee on Environmental Server Standards.
Cortney Thompson, Green House Data
"Compared to a normal datacenter, they have different power densities, availability, and distribution. We're seeing up to and above 90 kW per cabinet in HPC environments, and these facilities are going to need room to grow and add additional megawatts," Thompson told EnterpriseTech. "Legacy systems often don't need as much power, and have different distribution specifications."
But that's not necessarily the case. In addition to operating HPC systems as distinct, separate environments, organizations do combine HPC and enterprise datacenter facilities, said Mike Thomas, senior vice president at Environmental Systems Design. ESD – whose clients include Tier 1 research institutions (including half of the Big Ten), national labs, and Fortune 500 businesses – uses a total cost of ownership methodology to align IT, research and development, and facilities stakeholders and programs, right-size and scale datacenter infrastructure, develop a cost-model strategy, and plot a datacenter roadmap for future growth, he said. This tactic leverages – and often combines – the strengths of HPC and traditional enterprise datacenter systems, Thomas noted.
"Enterprise environments are primarily about ensuring uptime, availability, and redundancy through N+1 or 2N design, whereas many HPC systems require only a graceful shutdown and, therefore, limited infrastructure redundancy," he said. "On the other hand, power and cooling for HPC racks far outpaces the 2-8 kW per cabinet seen in most enterprise environments."
Mike Thomas, ESD
HPC systems use more power and, therefore, need more cooling, said Thompson. Depending on their design, some HPC datacenters could require other cooling technologies compared with traditional IT datacenters, he said.
"Higher power leads to higher temperatures, and while we operate at a higher temp on the server floor than we used to, you still might need to think about liquid cooling, free cooling, versus chilled loop," said Thompson.
It's important for datacenter managers or designers to conduct capacity surveys of their existing spaces, agreed Thomas. They also should do computational fluid dynamics (CFD) modeling of what-if scenarios to make certain they don't under- or over-size the power and cooling infrastructure and achieve a low power-usage-effectiveness (PUE), he added.
"Finally, even the network design is going to be different," Thompson said. "Bursting is common for big data, meaning fabric architecture is easier to scale with less client-server traffic."
But even if an organization has to revisit its datacenter design, the payoff can be worthwhile and long-lived, HPC advocates argue. Enterprises want clear-cut business advantages, viewing them as far more important than typical IT metrics, said Thomas.
"In business case and planning dialogs with clients with HPC environments, invariably it comes down to their ability to improve competitive advantage, enhance R&D effectiveness and ROI, and improve recruiting and retention: In other words, not uptime," he said.
