Better Together: Microsoft’s Storage Replica and Chelsio T5 iWARP RDMA for Disaster Recovery sponsored content by Chelsio Communications
Microsoft and Chelsio have teamed up to offer a comprehensive disaster recovery solution for mission-critical applications during power outages.
The solution includes Storage Replica (SR), a Windows Server 2016 Technical Preview 3 feature, which enables block-level replication between clusters or individual servers for disaster recovery, as well as stretching failover clusters to metropolitan (MAN) and wide area (WAN, US coast-to-coast) distances for high availability.
SR provides two modes of operation: synchronous and asynchronous replication. Synchronous replication enables mirroring of data with zero data loss guarantee at the volume level, whereas asynchronous replication trades off full data replication guarantees for reduced latency by locally completing I/O operations. The combination of the SR and Chelsio iWARP RDMA technologies offers high performance, long distance replication with the high efficiency provided by the zero copy and kernel/CPU bypass operation of the RDMA transport. The result is a reliable, scalable and robust disaster recovery solution for mission critical workloads.
Figure 1 – Microsoft Storage Replica Setup
Chelsio Terminator 5 ASIC
A key element of the solution is the Terminator 5 (T5), Chelsio’s fifth generation ASIC controller. This high-performance 2x40Gbps/4x10Gbps server adapter engine with Unified WireTM capability, allows offloaded storage, compute and networking traffic to run simultaneously.
Remote DMA (RDMA) is another essential part of the solution. It provides unprecedented levels of efficiency, thanks to direct system (or application) memory-to-memory communication, without CPU involvement or data copies. iWARP RDMA uses a hardware TCP/IP stack that runs in the adapter, completely bypassing the host software stack, thus eliminating any inefficiencies due to software processing. iWARP RDMA provides all the benefits of RDMA, including CPU bypass and zero copy, while operating over standard, plug-and-play Ethernet.
iWARP’s TCP/IP foundation supports standard Ethernet equipment, with no special configuration (like DCB and PFC) and without requiring a fabric overhaul or additional acquisition and management costs. iWARP stands out as the standards-based, mature, routable, scalable and robust plug-and-play option that is shipping today at 40Gbps from multiple vendors.
In contrast, alternative RDMA transports like RoCE and InfiniBand impose additional costs in equipment, such as DCB or Metro-X, and management complexity. In fact, Storage Replica does not support RoCE due to range limitations and lack of routability. The second, incompatible incarnation of the RoCE protocol – RoCEv2 – lacks critical congestion control capabilities to operate over long distance links. All versions of RoCE require dozens of steps to configure, more to debug and are therefore very hard to deploy and maintain. These attributes make T5 with iWARP the preferred solution for all storage networking needs: frontend, backend and high availability.
Making it All Happen
At its recent Ignite conference in Chicago, Microsoft showcased Windows Server Storage Replica operating over a 50Km fiber loop, in synchronous mode.
The demonstration setup consisted of a server connected back-to-back to a client using single 40Gbps link and standard MTU of 1500B. Both machines were configured with 2 Intel Xeon E5-2660 0 8-core processors running @ 2.20GHz, 256 GB of RAM, Windows Server 2016 Technical Preview 3 operating system (Build 10158) and 1 NVMe Intel P3700 1.45TB SSD each. 1 Chelsio T580-LP-CR adapter was installed in each system with inbox driver v5.3.22.140.
The command used was:
# diskspd.exe -c1G –b<IO Size> -t8 -o8 -r -h -L -w10 -d60 -W5 -C5 testfile.dat
The results?
- Storage line rate of 1.75GB/s (limited by Intel P3700 NVMe) - Remote SR performance identical to local disk.
- Consistent replication time of 857 secs with 1.45 TiB of data.
- Only ~25 to 30% network utilization leaving considerable bandwidth available for other applications.
Read the blog here.
In summary, long distance replication was shown to provide near local access performance levels, with low impact on I/O performance and latency. The results confirm iWARP’s native TCP/IP ability to operate beyond a single datacenter environment, and extend the RDMA transport over long distance. They demonstrate excellent performance and robustness under SR’s load pattern.
With Microsoft’s Storage Replica and Chelsio T5 iWARP RDMA administrators get an all-round disaster recovery solution for mission-critical applications.
