Facebook Opens Up Tools To Scale Memcached
When a back-end database can't serve up bits of data fast enough to deal with the hit load on the web servers that sit in front of it, the common thing to do is to put some kind of caching software between the database and the web servers. In the open source world and increasingly among enterprises, the software most commonly chosen for this job is Memcached. Social media giant Facebook has come up with its own extensions to Memcached and has done the altruistic thing and opened them up for others to use.
Facebook Engineering, the hardware and software development arm of the company, hosted its fifth @Scale Conference in San Francisco, and the opening up of the mcrouter software that extends Memcached was the big news of the show. Facebook developed mcrouter for its own personal needs to make Memcached more malleable and scalable as it sits between the company's MySQL database clusters and its web servers. As the name suggests, Memcached is a program for storing elements of a web page in the main memory of servers that are linked together and kept in synch as the web pages are updated in the production databases that are the foundation of the pages. Hitting the cache servers for stored elements is much quicker than pulling them out of the database, which allows for lower latency and higher throughput on web applications.
In a blog post, Facebook software engineers Anton Likhtarov, Rajesh Nishtala, and Ryan McElroy announced the opening up of the McRouter code, which is technically a Memcached protocol router. The software is mostly written in C++ with some libraries written in C and protocol parsing written in Ragel; the McRouter stack also uses the Folly and fbthrift libraries developed by Facebook. The latter is an improved version of the Facebook tool for automatically generating the client and server sides of remote procedure calls that was improved and open sourced earlier this year.
The Instagram picture hosting site adopted mcrouter as well in the wake of being acquired by Facebook and deployed it on the Amazon Web Services cloud before Instagram's infrastructure was brought in-house to run in Facebook's own datacenters in Oregon, North Carolina, and Sweden. Facebook says that mcrouter handles traffic across thousands of cache servers in dozens of clusters in its datacenters, and during peak loads handles close to 5 billion requests per second. News aggregation site Reddit has also deployed mcrouter in its infrastructure for a limited beta test. By opening up the mcrouter code under a BSD license, Facebook is hoping others can benefit from the work it has done to help Memcached scale better. The company is no doubt also interested in fomenting a community of developers to help make mcrouter better as they put it into production, improvements that Facebook itself will be able to benefit from in the future.
By the way, two and a half years ago, Twitter open sourced its own proxy server for Memcached and Redis NoSQL data stores, called Twemproxy, which allows web front ends and Memcached cache server pools to scale independently; it looks like mcrouter is a more sophisticated bit of code.
Any Memcached server or any other caching program that supports the Memcached protocols can be hooked into the mcrouter software. The software allows for Memcached servers, what are called destinations in mcrouter, to be pooled so multiple clients can come in on a single mcrouter instance and yet share outgoing connections from the Memcached servers. When a working dataset gets bigger than a single Memcached server instance, mcrouter creates a consistent hashing algorithm and shards the data and then spreads it across multiple Memcached servers. The mcrouter software also knows how to segment different datasets stored in the memory cache and split them into separate pools, and it can handle the updating of data on replicated pools that are created to give extra throughput for specific data. The mcrouter software also knows how to create backups of Memcached datasets and route requests to the backups in the event that a primary cache server in the cluster fails. There is a pool-level failover feature that allows for a local cluster of Memcached machines to fall back to a neighboring pool of cache servers in the event of a network failure, and groups of mcrouters can maintain consistency across Memcached clusters by broadcasting deletes of data globally across the routers and therefore their downstream Memcached servers.
Given all of this functionality, it would be interesting to see mcrouter, which has a BSD license, rolled into the actual Memcached project, which has a BSD-New license.
In addition to the opening up of the McRouter software, Facebook Engineering also said that it was teaming up with Box, Dropbox, Github, Google, Khan Academy, Stripe, Square, Twitter, and Walmart Labs to create a new open source collaboration organization called TODO, which is short for "talk openly, develop openly." The partners are being vague for the moment about precisely what TODO is, except to say that they are open source software advocates that "have come together to help solve the problem of utilizing and releasing open source software." The idea is to have common tooling for creating, sharing, testing, and consuming open source software and to also share best practices relating to open source code.
