Why Google-Style Infrastructure Is In Your Future 

One of the founding themes of EnterpriseTech is that the hardware and software approaches employed by hyperscale datacenter operators, which squeeze more work out of machines with less energy than commercial-grade hardware and software, will become prevalent in the coming years.

The analysts at Gartner agree, and they have said so in a recent report called Web-Scale Singularity Means Goodbye to Conventional IT Wisdom. This is by no means a new idea, but it is one that Gartner now feels a bit more comfortable about prognosticating upon and therefore is willing to put some numbers on it.

We call it the hyperscale datacenter, Gartner calls it Web-centric IT, but it all comes to the same thing. The idea is to take the same hacker approach that made Linux a viable commercial option and apply it to every aspect of the hardware and software stack in the datacenter – including the datacenter itself.

Hyperscale datacenters run distributed applications, much as traditional supercomputers do, and have failover provisions and disaster recovery built into those applications rather than being a manual process that is subject to human error. Data is often stored in triplicates in geographically dispersed sites and this can be done by using the cheapest storage arrays in the world: An X86 server with a bunch of cheap and fat SATA drives hanging off it, running some storage serving code.

The barriers between application development and system operations have similarly been broken down at these hyperscale datacenters, with the people responsible for coding the applications being responsible for installing, tweaking, and updating them in production. Because of the vast scale of their datacenters, which have hundreds of thousands to more than 1 million machines, hyperscale data center operators tend to have homegrown workload and systems management tools that can span this vast infrastructure and make it behave itself. Collectively, this new way of creating code and managing its deployment is called DevOps in the lingo.

In many cases, as readers of EnterpriseTech know full well, companies either design their own servers and storage to fit a particular workload. In some cases the larger hyperscale datacenter operators are even able to compel that chip makers such as Intel and AMD create custom processors – running at higher speeds or in hotter environments than commercially available variants, or sometimes having special instructions to accelerate their workloads that only they can gain access to – for their workloads. The tailoring does not stop at the processor or the server or storage. Google has custom switches and routers today much as it has its own servers and storage for years, and now Facebook and its Open Compute Project friends are fostering an open networking movement to create whitebox switches that can run changeable and hackable network operating systems and add network functions atop a virtualization layer inside the switch. You move out from the core infrastructure to rack designs mated to specific workloads, and then unique power and cooling mechanisms that take advantage of the local environment to either generate power or cool the systems – or both.

"Large cloud services providers such as Amazon, Google, Facebook, etc., are reinventing the way in which IT services can be delivered," explains Cameron Haight, a research vice president at Gartner. "Their capabilities go beyond scale in terms of sheer size to also include scale as it pertains to speed and agility. If enterprises want to keep pace, then they need to emulate the architectures, processes and practices of these exemplary cloud providers."

This is precisely what many enterprises will do, within the constraints of their budgets and the physical location of their datacenters and customers. (Big banks have to be near financial centers – there is no getting around that, because latency is money in that world.)

In fact, says Haight, by 2017 – and that is only three years away – this Web-scale IT approach pioneered by the hyperscale datacenter operators will be found in 50 percent of global enterprises. Not Mom and Pop shops with a server tucked under the desk or in a closet, not midrange companies with a half rack of machines, but large enterprises that operate on a global basis. Think of it as the top 4,000 or 5,000 companies in the world. (That is our number, not Gartner's.)

This ramp jibes somewhat with what we have heard from hyperscale server maker Quanta Computer. A month ago, Mike Yang, general manager of the Quanta QCT subsidiary that makes and sells server, storage, and rack infrastructure to hyperscale companies, told EnterpriseTech that about 20 percent of server shipments in 2013 were going into these hyperscale shops. Yang is in a position to know because Quanta is by far the dominant original design manufacturer (ODM) in the market right now. Yang estimated that within the next three years, these hyperscale customers would account for something on the order of 40 percent of server shipments, which is a stunning ramp and one that has to be making the tier one server makers nervous. This represents a tremendous shift in IT buying and in architecture, and it is coming to storage and networking, too.

By the way, Gartner's rival, IDC, doesn't think the hyperscale server ramp will be that steep. As we previously reported, IDC's box counters think it will take five years, not three, to hit a point where the hyperscale datacenter operators will account for 40 percent of server shipments.

The way that flash card maker Fusion-io carves up the world, there are around 200 or so hyperscale datacenter operators, about 4,000 large enterprises, and another 100,000 or so small and midrange enterprises that are targets for its various flash acceleration cards. There are, of course, millions of corporations and tens of thousands of government agencies worldwide, but most do not operate on anything approaching what we would call scale. We bring this up merely to put a number on the hyperscale operators and the enterprises who are going to start behaving like them if Gartner is right.

By the way, it is our opinion that Gartner is right in concept, if not in timing. It will take a long time for some large enterprises to make the architectural shift that Gartner is talking about. For one thing, storage tends to be on a three to four year cycle at most large enterprises and servers can be anywhere from three to five years, depending on the nature of the business. Those cycles are shortening somewhat, but the rules of depreciation are pretty strict and companies don't buy hardware because they like that new server smell, but because they expect machinery to do work for an extended period of time to pay for itself.

Equally importantly, the new supply chain of suppliers, which includes companies like Quanta QCT, WiWynn, Hyve Solutions, AMAX, Penguin Computing, and Avnet among an expanding list, will have to mature, whether they are peddling custom gear or Open Compute designs. Goldman Sachs and Fidelity Investments are two big financial services companies that have worked out some of the manufacturing and liability issues and are getting ready to deploy their first OCP machines in their datacenters. Others will no doubt follow.

But the limiting factor for the adoption of a new IT architecture is always software. Retooling applications takes time, money, and immense effort, which is why such moves usually happen with new applications. In this case, private clouds for application development and for virtual desktop infrastructure are the obvious first places to go, followed by clouds for actually running both old and new applications.

"Historically, enterprise IT has been focused on managing risk – particularly for companies that reside in regulated industries," explains Haight. "However, the major DevOps underpinnings, such as automation, are enabling these same enterprises to realize they can be fast and 'safe.' Embracing risk is not as risky as it sounds with a DevOps mindset. Having the architecture of the application being more resilient in the first place enables IT operations teams to implement and support leaner and more agile processes that might otherwise be viewed as inappropriate for conservatively minded organizations."

Another interesting tidbit from Gartner: Haight predicts that by 2020, a mere six years from now, 25 percent of CIOs at global enterprises will have experience in corporate Web-scale IT projects, up from 5 percent last year. If application architectures change as much as we believe they can given cheap and flexible iron, widely adopted application frameworks that support a wide variety of languages, and a culture that is used to the DevOps approach and its increasingly popular tools, then Haight's projections could turn out to be conservative.

The fun bit to consider is what the rest of the world's companies will be doing. It is a safe bet that many will simply throw in the towel and run their businesses from the public cloud, further fueling the growth and power of these hyperscale providers.