Enterprises Cooking Up Infrastructure With Chef 

The DevOps culture that started out in hyperscale datacenters is coming to the enterprise, and it will no doubt shake up the way that systems and their applications are deployed and maintained. Chef Software may have started out as an open source project to help make it easier to manage machines and code at hyperscale, but the same benefits accrue to those in enterprises with smaller scale infrastructure that often has more complexity buried in that infrastructure.

Tools like Chef are not just allowing enterprises to manage systems more efficiently, their use is a reflection that the barriers between application development and system administration are breaking down, and not because of any anathema between the two camps in the datacenter but simply because the artificial wall between development and production environments that has persisted for five decades in the datacenter makes it harder for IT departments to adapt applications to changing conditions in business in an agile fashion.

Chef and Puppet are the two darlings of the so-called DevOps movement. Chef is hosting its user and partner conference this week in San Francisco, and wanted to brag a bit about how enterprises were taking to this movement.

“The concept of web-scale IT is really hitting the enterprise front and center this year,” Ken Cheney, vice president of business development at Chef Software, explains to EnterpriseTech. “As the demands of the digital economy are driving pretty broad and widespread changes that are sweeping through enterprises.”

At this point, more than 90 percent of the Fortune 500 and more than 80 percent of the Global 2000 are using either open source Chef, the open source version of the configuration management tool, or Chef Enterprise, which as the name suggests is the variant of the tool with some extra goodies and a support contract to go along with it to make it appealing to enterprise customers who do not have as deep of a technical bench as IT vendors and hyperscale datacenter operators that are by and large not only self-supporting, but are significant contributors to the Chef community itself.

It is hard to find enterprise contributors to the Linux kernel or the OpenStack cloud controller; both projects are dominated by individual contributors or those sponsored by IT vendors to do the coding they do. Not so in the Chef community, which has 289 of its more than 2,000 contributors coming from real corporations like Nike and Lockheed Martin. In fact, in the past year, more than 79 new corporate contributors have joined the Chef development community and are helping to steer where Chef is going in the future. The reason why corporate involvement is so high, and will likely go higher, is obvious. Chef is a level of abstraction above the hardware and systems software and is not down in the guts of these things. It is easier to contribute, and more importantly, this is the level where enterprises probably know what they need more than does a software engineer deep down in some hairy C++ code. Everything in Chef, from configuration recipes to system cookbooks, is coded in Ruby.

The use of Chef is exploding as it takes off in the enterprise. Between April 2013 and March 2013, there were over 5 million downloads of Chef. And the revenue growth at Chef Software is being driven by enterprise adoption. Cheney says that in 2013, sales were up 188 percent, which is only driven by the Chef Enterprise licenses and support contracts since the open source  edition of Chef is free. (The open source Chef has tens of thousands of registered users.) About 30 percent of those revenues last year came from hyperscale Web companies like Facebook and Yahoo, who deploy both the open source and enterprise editions of the tool in those particular cases. But the other 70 percent is driven by regular enterprises like GE Capital and Target, just to name two who have come forward and said they are using Chef tools to manage their infrastructure. As you might expect, the revenues driven by enterprises is growing a lot faster than among the Web and emerging technology customers.

It is hard to say which is the cause and which is the effect, but what can be said is that the role of the developer is on the rise at large IT shops, much as it has always been front and center at the hyperscale Web properties that have always been largely software development organizations.

“If you think about DevOps and the whole concept of automation that treats infrastructure like code, it shows how empowered developers are today and what a driver they are for the economy,” says Cheney. To try to quantify that, Chef Software commissioned a survey of developers to get a sense of what they think about their roles at their companies. “What GE Capital and Target are talking about this week at ChefCon is how they went from pushing out code with a few changes a year to making multiple changes per day.”

The idea with DevOps is to mimic what happens at the big Web properties, who are constantly tweaking and changing their code and rolling out several minor updates daily. That is not to say that they don’t do massive updates to parts of their software infrastructure – they do. But change becomes normal and they create processes to manage it, and importantly, tools to help them do it better. Or, in the case of Chef, such hyperscale companies participate in the existing community around a tool and steer it so it addresses their needs as well as those of smaller organizations.

According to the survey, the vast majority (93 percent) of the more than 1,000 software developers located in the United States contacted by Chef feel empowered to experiment at their companies to test out new ideas and new code, and 94 believe they will be “a revolutionary influence” in their organizations. About 69 percent of developers said that they believed their jobs were recession proof, and 84 percent believe that they are paid what they are worth; more than half believe that they will be millionaires at some point in their career. About 89 percent of them said that their company leaders saw them as being essential, but 79 percent of them said they they felt that the pursuit of shirt-term profits by their employers was holding back investments for the future ­– no doubt investments in new software were top of mind here.

At ChefCon this week, Chef Software is also rolling out some tweaks the Chef Enterprise 11, which has been around since last summer. One of the interesting new features is called Chef Metal and it is a framework to manage a cluster of machines the same way that a single machine is managed. Chef Metal 0.2 is currently in a alpha release state, so it is not meant for production. It can be used to orchestrate the provisioning of LXC Linux containers on bare metal clusters; it can also be used to deploy clusters of machines on the Amazon Web Services, DigitalOcean, and Vagrant public clouds. Chef Metal is expected to be production grade sometime in the third quarter of this year.

The Chef DK, short for developer kit, is being rolled out for the first time as well and includes a number of open source add-ons for Chef, including Berkshelf, Chef Spec, Food Critic, and Test Kitchen. The core Chef server is also being updated with Chef Actions, which is akin to a newsfeed for everything that is happening inside of a Chef Server, to use a social media metaphor. And finally, Chef Software is open sourcing the code behind its community site in a project called Supermarket. The idea is to let coders help evolve the community itself, as embodied in its code, and to allow other software developer communities to adopt it and adapt it for their own needs.

Chef Enterprise comes in a freebie version that can manage up to five server nodes and does not have a support contract. Three different packages, called Launch, Standard, and Premium, come with 20, 50, or 100 nodes covered at a cost of $6 per node per month. Volume discounts above these levels apply, says Cheney. Role-based access control, high availability clustering, LDAP and Active Directory authentication, and an enhanced management console and reporting features are added to turn open source Chef into Chef Enterprise.

Chef Software, formerly known as Opscode, closed a $32 million round of venture funding last December, giving it total funding of $63 million to date. Rival Puppet Labs, which does some of the same system and application configuration automation work, has raised $45.5 million in several rounds of funding, notably $30 million from server virtualization juggernaut VMware in January 2013. Facebook is probably Chef Software’s biggest and most famous user of Chef Enterprise, and the social network uses the open source Chef edition in production as well just to make sure that it has a handle on what the differences are between the two and gives back to the community.