CoreOS Releases Building Block For Distributed Systems 

Hyperscale Linux operating system specialist CoreOS said it is releasing its latest open source component for sharing and managing configuration data and other functions used in distributed systems.

San Francisco-based CoreOS announced its first stable release of etcd, or "etc distributed," an open-source distributed key value store that provides the backbone of CoreOS clusters and the etcd clients that run on each machine in a cluster. “Our goal with etcd has been to make building and using distributed systems easier,” CoreOS CTO Brandon Philips said Wednesday (January 28) in announcing the release.

Etcd is described as a core component of CoreOS software that facilitates safe automatic updates while coordinates work being scheduled to hosts. It is also used to set up overlay networking for containers.

Philips added in an interview that a key objective of the etcd project was to remove single points of failure in developing distributed systems. Proponents argue that using etcd-based applications ensure that they will continue to work even when individual servers fail.

For example, automatic updates provided by etcd that are akin to a browser update mean critical configuration and other data can be retained.

The features can also be used for dynamic load balancing and other traffic management tasks. "This is a component that will help drive forward various projects to make infrastructure stronger and highly-available," Philips said, adding that the goal is to extend server availability from thousands to millions of clients.

Philips said more than 150 developers contributed to its etcd 2.0 release, including feedback from Google, which is using etcd in its Kubernetes project. Other projects leveraging etcd include Apache Mesos and Mesosphere DCOS. Much of the focus was on making etcd run faster while using less memory in order to scale the distributed system component, he added.

CoreOS said etcd is currently being used in more than 500 other distributed systems projects. Other current users include Pivotal Software in its Cloud Foundry, the open source cloud computing platform-as-a-service.

Onsi Fakhouri, a Pivotal engineering manager, said in a testimonial that it evaluated other persistent stores but concluded that etcd's HTTP API and support for Go, the language in which it is written, prompted Pivotal's decision to run with the open source key store.

Indeed, the fact that etcd is written in Go is cited as a key reason for backing the CoreOS approach since the language has strong cross-platform support and a large open source community backing it.

CoreOS said its etcd project is licensed under Apache 2.0 and is being shepherded with the help of a community of open source developers. Internally, etcd uses the Raft consensus algorithm to provide fault tolerance and the distributed replication of information, the company said.

Philips characterized the open-source community's support of the project as "exponential, long-tail contributions." While etcd is not a CoreOS commercial product, "it is one of the components [of distributed systems] that needs to exist."