Hybrid Approach Emerges for AI Workloads
The discussion surrounding enterprise adoption of AI technologies is shifting from if to when, and what mix of IT infrastructure will be required to deploy and scale machine learning and other workloads.
HPC vendors that have been making inroads in the enterprise market view AI as another opportunity. Among them is supercomputer leader Cray Inc. (NASDAQ: CRAY) which released an industry survey earlier this month that claims a growing shift toward HPC platforms to handle AI workloads. Much of the AI-dedicated hardware is on-premise as early adopters hustle to keep pace with growing computing and storage demands, the vendor survey found.
Few would doubt Cray’s assertion that HPC will play a leading role in enterprise adoption of AI. How that infrastructure is configured and whether, as Cray asserts, much of it will remain on-premise in the form of supercomputers and HPC clusters remains to be seen.
The Cray survey of more than 300 IT administrators notes that nearly 40 percent are using in-house systems to crunch AI workloads. Among the priorities are better performance and what the survey calls “data locality [and] security.”
What’s more, the survey found that 65 percent of respondents with AI applications plan to expand in-house infrastructure to meet performance requirements.
Those findings run counter to recent efforts by cloud partners to make AI workload processing more widely available. For example, Amazon Web Services (NASDAQ: AMZN) and GPU leader Nvidia (NASDAQ: NVDA) are teaming to make the latest graphics accelerators available via the public cloud for AI workloads such as machine learning inference. AWS announced last week it was offering new G4 cloud instances that combine Nvidia T4 Tensor Core GPUs, second-generation Intel Xeon Scalable processors along with increased network bandwidth and local storage to handle enterprise machine learning applications.
Hence, it appears a hybrid approach to running enterprise AI workloads is emerging, a trend the Cray survey acknowledges. While a majority of respondents are staying in-house with AI workloads, more than 50 percent said they are currently running AI workloads on public cloud services. That number could increase as major vendors offer more cloud instances that include graphics acceleration and other AI-oriented infrastructure.
Nevertheless, the Cray survey found that 36 percent of those polled are seeking to move at least some of the cloud workloads to on-premise infrastructure, much of it presumably powered by HPC systems. Cray’s takeaway is that more IT managers will seek to keep critical AI applications in-house running on exascale-class systems such as its Shasta platform.
“The use of cloud-based solutions for AI has garnered much public attention,” the enterprise AI survey notes. Still, it concludes, “cloud is not the only choice for enterprise AI workflows. A large percentage of organizations are utilizing on-premises infrastructure for their AI projects and many see the need to expand these on-premises systems in order to keep up with growing performance demands.”
The Cray AI survey is available here.
