Pica8 Ports Network OS To Bare Metal Broadcom Trident II Switches
Servers based on X86 processors have Windows, Linux, and other options as operating systems, and an increasing number of switch buyers want to be able to deploy the network operating system of their choice on their switches, much as the largest hyperscale datacenter operators often do or are working to do. To that end, Pica8, one of the handful of upstart, open source network operating system providers that has emerged in the past several years, is expanding its PicOS operating system to cover more network ASICs.
Starting today, customers who buy switches based on Broadcom's Trident II ASIC can now use PicOS on those switches. There are some caveats to this support, of course. To get any open network operating system to work on any switch, you need three things. First, the network operating system has to be ported to the ASIC. Then the network OS has to be integrated with any other adjacent processors, coprocessors, FPGAs, and other features of the switch. And finally, you need a bootloader to actually get the network OS loaded onto the switch, just like you need a bootloader for a server. Just last week, Pica8 adopted the Open Network Install Environment (ONIE), an open source tool from rival Cumulus Networks that has been contributed to the Open Compute Project, as its official bootloader.
This latter bit, Steve Garrison, vice president of marketing at Pica8, explains to EnterpriseTech, is the hard bit for customers. Most incumbent switch makers bundle the hardware and the software as a single unit and if you hack a different operating system onto the switch you void the warranty. The network OS is not licensed separately from the switch in many cases, so restoring it becomes a problem as well.
That said, there certainly are a number of companies making whitebox Ethernet switches that use popular ASICs that can in fact run PicOS and other open source network operating systems such as Cumulus Networks, Vyatta, or On.Lab ONOS. Cumulus provides an open source OS for switches, Vyatta (now part of Brocade) provides an open OS for routers, and ONOS is a distributed network OS aimed at automating networks for service providers. This is a nascent market, and it stands to reason that if open networking takes off, as it certainly has among hyperscale datacenter operators like Google and Facebook, that there will be more entrants to the market. There is a good chance that some companies that have kept their network OS code closed will open it up and try to become the dominant supplier of such code running across lots of different hardware. Arista Networks and Juniper Networks, to name two, could gain some kind of leverage against Cisco Systems in this manner, just like Unix and Linux took on proprietary systems software on servers over the past two decades in a one-two punch.
PicOS already supported four Broadcom ASICs and one Marvell ASIC, according to Garrison. Broadcom is, of course, the market leading in supplying merchant silicon for Ethernet switch makers, with around 65 percent market share. Intel's Fulcrum unit and a few others like Marvell have carved out their slices, and new entrants such as XPliant are trying to carve their own niches with support for 25 Gb/sec Ethernet in a new line of ASICs due early next year.
The supported Broadcom ASICs include the Fireball-3, Triumph-2, and Helix chips, which are used in older gear, as well as the Trident+ and Trident II ASICs, which are more current and which are used to make 10 Gb/sec and 40 Gb/sec switches and which are sold under the Strata XGS brand. Garrison says that there are hundreds of thousands of Trident II top-of-rack switches running in the world, and that is a pretty decent installed base for a product that launched in August 2012 and started appearing inside switches in the summer of 2013. The ONIE bootloader works with Trident II ASICs and has the support of Broadcom and it also works with XPliant's XPA family of ASICs. It stands to reason that PicOS will find its way onto the XPAs as well.
Pica8 does not support Broadcom's Dune family of ASICs, but Garrison says that Pica8 is gearing up to port PicOS to the forthcoming "Tomahawk" Strata CNX ASICs from Broadcom, which were unveiled at the end of September and which will ship early next year. Significantly, the Tomahawk ASICs, like those from XPliant, will support 25 Gb/sec Ethernet – something that hyperscale datacenter operators basically compelled the IEEE to adopt as a standard. In addition to these Broadcom chips, PicOS can also run on the Lion ASICs from Marvell.
Interestingly, neither Cumulus Linux nor PicOS run on Intel's Fulcrum family of network ASICs, which is curious given how much Intel has benefitted from the availability of operating system alternatives for its Xeon server and now storage processors.
Support for the Trident II ASIC brings a number of goodies, says Garrison. First, a switch using the Trident II chip can combine the switching and routing memory in the switch, known as the Forwarding Information Base or FIB, with the Ternary Content Addressable Memory (TCAM), which among other things is used to store routing and packet data. (TCAM is a very fast and expensive form of memory found in switches and routers.) The significant thing is that by combining the FIB and TCAM memories, a Trident II switch can manage over 200,000 flows using OpenFlow protocols. Garrison says that companies with proprietary software-defined networking (SDN) approaches have tried to spread fear, uncertainty, and doubt about the amount of memory and the number of flows available to the OpenFlow protocol in switches. This large number of flows is important in multi-tenant cloud environments.
The Broadcom Trident II ASIC supports the two major Layer 2 overlays for Layer 3 networks – VMware's VXLAN and Microsoft's NVGRE – but also support two other tunneling protocols, called L2oGRE and L3oGRE, and PicOS is the first network OS to support these latter two protocols. You can see a hardware compatibility list at this link, for switches from Quanta Computer, Accton, Alpha Networks, Edge-Core Networks, Foxconn, and Pica8 itself.
The bare metal switching movement is just getting going among large cloud and hyperscale datacenter operators, but it stands to reason that the movement will gather momentum in the datacenter for all the same reason that industry standard servers did starting two decades ago.
In a busy October, Pica8 closed its second round of venture funding of $12.5 million with VantagePoint Capital Partners, Cross Head, and Pacific Venture Partners, and has raised over $20 million to date. Pica8 got its first reference architecture out the door in December 2012 and rolled out an open SDN kit a year later. Just this month, Pica8 adopted the ONIE loader and offered a PicOS free trial to customers to start building a base of tire kickers. The company has 320 paying customers at the moment, and 300 of them are doing trials of bare metal switching to figure out how it might fit into their datacenters. Of the 20 customers that are running bare metal switches running PicOS in production, they range in size from 10 to 2,000 top of rack switches. The latter is, of course, a very large installation, probably two per server rack.
