Amadeus Travel Goes Up Tempo with CouchbaseDB
In the near future, if you search for a flight online, it’s likely that your query will run against Couchbase’s NoSQL database. That’s because Amadeus, the Spanish company that processes many of the world’s flight, hotel, and cruise ship reservation transactions, is in the process of implementing a NoSQL database from Couchbase to streamline the big task of serving real-time airline flight lookups.
Amadeus may not be a household name, but it’s the B2B brains behind some of the most heavily visited travel websites on the Web. The company provides soup-to-nuts travel-related services for 110 airlines, 207 tour operators, 110,000 hotel chains, 30 rental car agencies, and 50 cruise ship lines. In addition to displaying real-time availability for seats on airplanes, cars, and ships, and rooms in hotels, it will complete transactions for its customers, whether they’re handled over the Web or processed through a bricks and mortar travel agency.
As you might imagine, Amadeus has fairly large IT systems set up to handle this massive global travel workload. The company maintains its own server infrastructure, which includes thousands of Linux-based X86 servers that are housed in a large data center near Munich, Germany, with a disaster recovery site about 100 miles away.
Recently, Amadeus initiated a major overhaul of its airline flight availability and booking system with the goal of making it easier to expand to meet an expected ten-fold increase in flight lookups coming in over the Web. The system is composed of three parts. The first is the flight availability system that maintains current flight availability information for 97 percent of the world's bookable flights.
Flight availability information from that source system is fed into two separate systems. The first one is used to serve searches for seat availability, and is based on a relational MySQL database and Memcached caches. The second is used to process purchase transactions, and is built on Oracle’s eponymous database configured in an active-active RAC cluster configuration.
Amadeus implemented the Memcached system on a MySQL database in 2008, and it has served the company well over the years, according to Dietmar Fauser, vice president of R&D for Amadeus IT Group. The system that Amadeus implemented involves a basic data model that uses simple select statements against a primary key to retrieve a block of information – a setup very similar to the key/value stores that are so popular today. “We started doing this well before the NoSQL term got coined,” he says.
Dietmar Fauser, vice president of R&D for Amadeus IT Group
This simple architecture has done well to support the sheer scale of transactions at Amadeus. Currently, the Memcached system handles 2 million physical GETs per second, and 400,000 writes per second. The GETs represent flight availability queries coming in from all over the world, while the writes represent changes of the availability or pricing of seats on flights.
At any given time, the company’s flight lookup system maintains about 350,000 active connections to users across the world. When you add all that activity up at the end of the day, it averages about 8 billion GET transactions, for a total of 3.7 million bookings. With such a massive global system, latency is a big deal, and Amadeus strives to return data to users within two seconds of hitting the search button. To deliver such responsiveness at the end of the wire, the latency in Amadeus’ server-to-server communications is in the sub-millisecond range.
In-memory systems, such as Memcached, are known for delivering extremely quick responses to queries against very large data sets. And the Memcached system has worked well for Amadeus up to this point. But the setup is not perfect, and maintaining it can be a difficult task that involves making application-level changes whenever an upgrade is required.
Capacity upgrades are the big problems. Because of the architecture of Memcached, anytime Amadeus added another node, it required the entire server topology to be changed. That is a major ordeal. “It happens unfrequently, like once a month. But each time, it's a significant effort,” Fauser says. “The application has been built to manage this, but it's a manual task to create the new configurations, and it’s an operational task to update the configuration on thousands of nodes. And it's something we don't want to do.”
Considering that Amadeus expects its GET transactions to increase to 20 to 30 million per second in the next few years – thanks largely to Web traffic driven by meta search engines like Expedia and Kayak – it's understandable why the company wants to get off the Memcached train and simplify its operations.
By adopting a NoSQL database, it could maintain its current key/value store architecture, while pushing all the technical complexity for handling data distribution (which is currently baked into the application) down into the database level where it belongs. This would be very good for Amadeus, which strives to maintain as homogenous an IT setup as possible for the sake of simplicity.
“We hope to be able to manage this transparently at the database layer, and not at the application layer anymore,” Fauser says. “You alleviate the application from the topological knowledge, which is always good from a software design point of view. You want to have applications that are not aware of the underlying topology.”
Amadeus researched various NoSQL databases in the market, and selected CouchbaseDB for several reasons. Considering the sheer number of NoSQL database vendors currently in the market and the level of competition out there, this was a significant win for Couchbase’s business. “We believe that Couchbase is one of the survivors,” Fauser says, hinting at industry consolidation that is widely expected to occur in the NoSQL database market over the next few years.
On a technical level, Fauser cited two features in CouchbaseDB that swung the needle in its direction. This includes datacenter data replication and automated data sharding features, which he called “awesome,” and the capability to write a large amount of data into persistent storage. Currently, the Memcached system keeps all data in memory, but Amadeus would like to write the data into persistent storage to minimize administrative effort. “We want to go a step further by keeping the data not volatile, like in a memory cache, but keeping the data persistent in the cache, which will allow us to restart machines without losing the data,” Fauser says.
The plan is to replace the Memcached setup and the MySQL database it runs on with CouchbaseDB. The company will maintain Oracle databases for formal ACID transactions. “For the time being, we have no plans to get rid of Oracle,” Fauser says. “We operate many, many systems, and I believe that relational databases will be around for a long while. If you have very heavy index activity, for example, or you need full consistency and recoverability at any place, I believe solutions like Oracle are very good and will be around for a while.”
The company is currently testing CouchbaseDB, and Fauser says it is on pace to make the switch by the second half of 2014. If things work well, the company could expand its use of CouchbaseDB to other aspects of its business, such as car rental, cruise ship, or hotel room booking, Fauser says.
