Docker Acquires Team Behind Apache Aurora
Application container leader Docker has acquired the stealthy startup behind an open source project designed to keep applications and services up and running after a system failure.
San Francisco-based Docker said Thursday (March 3) it is acquiring Conductant Inc., launched last year by former Google and Twitter engineers. Conductant co-founder Bill Farmer also created the Apache Aurora project to run applications on shared pools of machines to avoid service interruptions.
Terms of the acquisition were not disclosed. The deal follows Docker's January acquisition of Unikernel Systems, a developer of lightweight operating systems used to deliver distributed applications.
Aurora is an extension of the Apaches Mesos clustering platform designed for production workloads. In a blog post, Docker CTO Solomon Hykes said Aurora could be integrated with Docker's native clustering capability called Swarm for used as a "large-scale web operations stack."
Farmer's team at Twitter built Aurora to help automate the social media giant's massive server farms. The approach allows a handful of operations engineers to keep applications and services up and running.
Twitter shared its work on Aurora with the Apache Software Foundation in 2013. Farmer said in a separate blog post that the goal of the Aurora project was to leverage Mesos to run "Twitter-scale workloads on large clusters." Twitter currently uses Aurora to manage and scale its micro-services.
Apache Aurora founder Bill Farmer.
Conductant was formed last year to expand access to the Aurora tools for deploying and scaling micro-services, Farmer added.
The work also was informed by Farmer's experience with Borg, an orchestration tool used by Google to manage massive compute clusters while making them accessible to systems and services developers. At Twitter, Farmer combined that background with Mesos to build a "high-level orchestration system that would solve for Twitter the problems Borg solved for Google."
Farmer variously describes Aurora as a "factory" designed to ensure that applications and services keep running indefinitely in the event of a machine failure. When a failure occurs, the automated system reschedules those jobs onto "healthy" machines.
Docker said Farmer and Conductant co-founder David Chung along with other key engineers from the startup would join its infrastructure development team. The deal illustrates Docker's growing emphasis on what CTO Hykes called "operations-driven development" as the container movement transitions to operating on large-scale cloud platforms.
Docker Swarm is intended as a "standard base layer" for scaling applications ranging from databases and scientific computing to big data pipelines. "A stack combining Docker Swarm and Aurora could democratize the battle-tested Twitter operations model," Hykes asserted.
Hykes also provided some metrics on the expanding use of its Linux containers in production settings. He said Docker Cloud is currently deploying containers in "dozens of datacenters." Meanwhile, the company announced in February that users have pulled more than 2 billon images off of Docker Hub, its platform for accessing container images.
