On-Chip Networking May Survive Calxeda Shutdown 

Just after EnterpriseTech went on hiatus for the holiday, ARM server chip pioneer Calxeda closed its doors. This was something of a shock to those who have been championing ARM-based processors as an alternative to X86 server chips from Intel and Advanced Micro Devices. Despite the shutdown, Calxeda's technology could be picked up by any number of interested parties, depending on what price Calxeda's investors try to get for it.

Any current ARM server chip maker could be interested in acquiring the assets of Calxeda, as could any server maker or hyperscale datacenter operator with ARM aspirations of their own. The Fleet Services fabric interconnect that Calxeda developed is particularly valuable and can be dangerous falling into enemy hands.

The Fleet Services interconnect is a distributed Layer 2 switch that is put on each ECX-1000 and EXC-2000 processor, and it currently scales to 4,096 server nodes without the use of a single top-of-rack switch. The plan was to push the Fleet Services fabric up to over 100,000 nodes in a single cluster. The fabric can be set up in 2D torus, mesh, butterfly tree, and fat tree network configurations, all managed from software. The Fleet Services fabric has 80 Gb/sec of aggregate bandwidth per chip, with multiple 10 Gb/sec Ethernet links coming off the die to interconnect server nodes on a single card, multiple cards in an enclosure, multiple enclosures in a rack, and multiple racks in a data center. Some of the Ethernet links are used to allow users to get to applications running on clusters, of course.

If Barry Evans, the company's co-founder, does manage to reorganize Calxeda with its investors, as he is currently trying to do, it is reasonable to expect the company to emerge to peddle this interconnect technology in a more generic way. Perhaps Calxeda can license the Fleet Services interconnect to other X86, Power, MIPS, and ARM chip makers, much as ARM Holdings licenses the ARM core designs to companies like Calxeda.

Big plans for little cores

Calxeda had a very long-range roadmap for ARM server chips and interconnects to link them together into clusters. But perhaps it jumped the gun a little ahead of the software stack, and perhaps developing 64-bit ARM chips took longer than Calxeda expected. The upshot is that the company's backers decided at a meeting on December 16 to not put any more capital into the firm. Their reasons for not continuing to invest in Calxeda were not explained.

But the effect was immediate and most of the 125 employees at the company were laid off. Karl Freund, who was vice president of marketing at Calxeda, told EnterpriseTech that the company would support the server makers who have agreed to use its "Midway" ECX-2000 processors. These Midway chips, which have 32-bit ARM cores but 40-bit extended memory addressing, were just announced at the end of October last year and are expected to appear in systems during the first half of 2014.

Here is the statement that Calxeda put out about the shutdown:

Over the last few years, Calxeda has been a driving force in the industry for low power server processors and fabric-based computing. The concept of a fabric of ARM-based servers challenging the industry giants was not on anyone’s radar screen when we started this journey. Now it is a foregone conclusion that the industry will be transformed forever.

Now it's time to tackle the next challenge. Carrying the load of industry pioneer has exceeded our ability to continue to operate as we had envisioned. We wanted to let you know that Calxeda has begun a restructuring process. During this process, we remain committed to our customer’s success with ECX-2000 projects that are now underway.

Calxeda is proud of what we have accomplished, the partners who have collaborated with us, the investors who supported us, and the visionary customers who have encouraged us and inspired us along the way. We will update you as we conclude our restructuring process. In the meantime, we want to thank you personally for your interest and enthusiastic support. It's been an amazing journey.

Energy, matter, and innovation are never lost, just reassembled.  We look forward to the inevitable application of our ideas.

Calxeda got out in front on the ARM server wave with 32-bit chips, delivering its "High Bank" quad-core EXC-1000 processors and on-chip Fleet Fabric interconnect in November 2011 as the engine for Hewlett-Packard's first Moonshot hyperscale server platform. Since that time, Intel and AMD have been more aggressive about the microserver market in general and in delivering low-power X86 chips that compete against Calxeda's chips, and significantly, offer 64-bit processing. While Calxeda made the case that there were plenty of workloads where 32-bit processing was appropriate – media streaming and storage controllers come immediately to mind – IT shops want 64-bit processing and memory addressing.

Calxeda was a licensee of the ARMv8 designs from ARM Holdings, and was on deck to get its 64-bit ARM server chips, code-named "Lago" and "Sarita," taped out as 2013 came to a close. Samples of these two chips were expected to start shipping in the second of this year, with production quantities due at the end of 2014. That was later than originally expected, and these delays may be a part of the problem in getting more funding for Calxeda to continue.

Advanced Micro Devices expects to get its "Seattle" ARMv8 design, specifically based ARM Holdings' Cortex-A57 variant, back from Taiwan Semiconductor Manufacturing Corp, its foundry partner, during the first quarter of this this year. Andrew Feldman, general manager of AMD's Data Center Solutions Group, has confirmed to EnterpriseTech that the Seattle chip, which will sport eight cores using a 28 nanometer process, will ship in the second half of 2014.

The Seattle chips will include on-chip Ethernet ports running at 10 Gb/sec and on-chip PCI-Express 3.0 peripheral controllers – features that AMD's own Opteron X86 server chips are lacking. The Seattle chips will also have auxiliary circuits for encryption and data compression, and are expected to run at 2 GHz and higher and offer twice the performance of the quad-core Opteron X chips that AMD is currently shipping. And, at some point, the AMD ARM chips will also link into the Freedom fabric that is integral to the SeaMicro line of microservers. Given that AMD already has its own interconnect, probably doesn't need another one.

This Seattle chip would have been a significant competitive threat to those coming out of Calxeda, and so were the eight-core "X-Gene" server chips that are currently sampling from Applied Micro. The X-Gene chips are not based on the Cortex-A57 designs but rather on the more generic and broader ARMv8 license and are expected to have eight cores to start, just like the AMD Seattle chips. Cavium, Texas Instruments, and Marvell are also working on 64-bit ARM server variants, and the rumor is that Samsung, which has a chip design center and fabrication plant in Austin just down the road from AMD and Calxeda, is also widely expected to create an ARM server chip.

There will still be plenty of competition in the ARM server space, but with Intel having delivered its eight-core "Avoton" C2000 Atom server chips last September, which have 64-bit processing and memory addressing and which are well ahead of the ARMv8 pack, getting the attention of server buyers and server makers is going to be more difficult. Intel is also working on a system-on-chip variant of its future "Broadwell" Xeon E3 processor, due this year, and will also get a kicker to Avoton, called "Denverton," into the field this year. Both Broadwell and Denverton will be implemented using Intel's 14 nanometer technologies and will no doubt give any 64-bit ARM server chip a serious run for the money.

This is what happens when you give Intel three or four years to take on a competitive threat and when it has a significant chip fabrication process advantage.

Calxeda started out life in January 2008 as Smooth-Stone, and that was a reference to the rock that David put in his slingshot to slay Goliath with the strength of his arm. The company came out of stealth mode in August 2010, announcing that it had raised $48 million in two rounds of funding. The company closed its Series C in October 2012, bringing in another $55 million, and was in the process of peddling its Series D round when the investors said no dice on more dough. Evans and ARM Holdings kicked in angel funding to get the company started. Advanced Technology Investment Company, the investment arm of the government of Abu Dhabi that owns chip foundry GlobalFoundries, was an early investor in Calxeda, as was Battery Ventures, Flybridge Capital Partners, and Highland Capital Partners. Austin Ventures and Vulcan Capital were added during the Series C round.

Evans, who used to run Intel's low-powered X86 and XScale ARM server chip unit (the latter which it bought from Digital equipment and then sold to Marvell), was joined in founding Calxeda by chip engineer David Borland, who worked at Intel, AMD, and Marvell. Larry Wikelius, who managed alliances for Opteron server maker Newisys before it was acquired by contract manufacturer Sanmina-SCI in 2003, was also a co-founder.

The question now is what is the Fleet Fabric interconnect worth? Calxeda raised a total of $103 million, and a substantial portion of the investment was used to license ARM cores and to develop that interconnect. The price will no doubt depend on how well it performs at scale, and as far as we know, no one has bought enough servers based on the Calxeda chips to push it to the limits.