WekaIO, Penguin Computing Set Performance Record on STAC-M3 ‘Tick Analytics’ Benchmark
CAMPBELL, Calif., June 7, 2019 -- WekaIO, the innovation leader in high-performance, scalable file storage for data-intensive applications, today announced record-breaking performance on the STAC-M3 Benchmark from a new integrated solution with Penguin Computing—the FrostByte storage solution featuring the WekaIO File System. The STAC-M3 Antuco and Kanaga Benchmark Suites are the industry standard for testing solutions that enable high-speed analytics on time series data, such as tick database stacks. The WekaIO File System on Penguin Computing Relion servers and FrostByte storage system delivered record-breaking performance at scale for the STAC-M3 “Tick Analytics” Benchmark, affirming that the integrated solution is the ideal choice for algorithmic trading and quantitative analysis workloads common in financial services.
The STAC-M3 benchmark specifications are maintained by the STAC Benchmark Council, which consists of over 300 financial institutions and more than 50 vendor organizations whose purpose is to discuss technical challenges and solutions in financial services and to develop technology benchmark standards that are useful to financial organizations. User firms include the largest global banks, brokerage houses, exchanges, hedge funds, proprietary trading shops, and other market participants. The STAC-M3 results were audited by the Securities Technology Analysis Center (STAC), which facilitates the Council.
The system under test was a Kx kdb+ 3.6 database system distributed across 7 Penguin Computing Relion XO1132g servers with dual Intel Xeon Gold 6140 CPU @ 2.30 GHz and data stored on Penguin FrostByte with the WekaIO File System (v3.2.2) Software running on 8-1U RXE1112 server nodes with 9 NVMe SSDs per server, using Mellanox SB7780 36-port Non-blocking Managed EDR 100Gb/s InfiniBand Router and InfiniBand EDR/ Ethernet 100Gb 1-port 840QSFP28 Adapters. The tests were run on the baseline benchmark suite (“Antuco”) and the scaling suite (“Kanaga”), a test suite with five years of simulated stock market data. The test system had a combined performance density of up to 87.5 GB/sec available bandwidth to the client nodes, 40.5 GB/sec of actual throughput*, and 2.5M 4K IOPS in 8U, with the ability to fully distribute data, metadata and system services, as well as super low file system latency that leverages high-speed networks and NVMe-optimized storage (Figure).
Image courtesy of WekaIO
The test system delivered record-breaking performance, breaking 8 STAC-M3 world records for mean query-response times and 4 world records for throughput.
Kx’s time-series database kdb+ is commonly used for large scale complex analytics on streaming, real-time, and historical datasets in industries ranging from financial services to marketing analytics, and for Industrial IoT. In the financial industry, typical kdb+ applications are for algorithmic trading, back testing (trading algorithm validation), surveillance, regulatory and compliance reporting and research environments. These workloads can demand a lot from the file system.
Better than block-based solutions for Financial Services, the WekaIO File System is the only NVMe-optimized, shared, coherent, POSIX-compliant file system with a fully distributed architecture for data resiliency. The WekaIO Software eliminates islands of storage, providing all applications access to a shared data pool with record-breaking performance. The WekaIO File System delivers 10x better performance than a local file system by bending the rules on data locality, distributing the data across the cluster over low latency InfiniBand or Ethernet networks.
WekaIO is a firm believer in the value of benchmark testing as a means to guide and inform development in the industry and frame results in a manner that delivers actionable intelligence to customers. The WekaIO File System currently holds one of the top positions on the independent SPEC SFS 2014 benchmark and the #1 position on VI4IO’s 10 Node Challenge list.
For the full STAC Report, which contains an explanation of the benchmark specifications and the results for this solution, see www.STACresearch.com/news/2019/06/03/KDB190501.
“Traditionally, IT organizations have used block-based storage solutions for their trading databases, rather than compromise performance due to network and file system latencies. The result has been large investments in specialized hardware and software to run different applications,” said David Hiatt, Director of Business Development and Alliances at WekaIO. “Now with the STAC-M3 Benchmark results, we have proven that with the WekaIO File System on Penguin Computing Relion servers and FrostByte delivers better application performance for Kx kdb+ by running in distributed mode over a network than with traditional block storage, with the additional benefit of simultaneously supporting other applications on the same platform. That’s a huge leap in application acceleration and simplifying the storage infrastructure for financial services.”
*As per the benchmark STAC-M3.ß1.1T.5YRHIBID.MAX
About WekaIO
WekaIO helps companies manage, scale and futureproof their data center so they can solve big problems. WekaIO built the world’s fastest shared and parallel file system on new architecture optimized for flash that leapfrogs legacy storage infrastructures by delivering the best performance density at any scale, for a fraction of the cost. With the most flexible deployment models, the storage software is a modern file system designed for modern I/O intensive and extreme technical computing workloads. For more information go to, https://www.weka.io.
Source: WekaIO
