Covering Scientific & Technical AI | Thursday, November 28, 2024

Lightbits Links Kubernetes to Persistent Storage via NVMe 

Lightbits Labs, the “originator” of NVM Express over TCP block storage, said the latest version of its LightOS platform integrates cloud-native persistent storage for Kubernetes. The result is said to be improved scaling and availability via clustering.

San Jose-based Lightbits’ architecture separates storage and computing while leveraging NVMe/TCP (Transmission Control Protocol). Its software-defined storage framework now targets container-based applications orchestrated by Kubernetes and therefore requiring persistent storage. The new version is promoted as offering all-flash array efficiency along with direct-attach flash storage performance.

LightOS 2.0 also leverages the Container Storage Interface (CSI) for Kubernetes released last year. The plugin provides stateful container storage as more Kubernetes micro-services shift to stateful applications. The combination targets large-scale Kubernetes clusters requiring persistent storage for tasks like node migration and workload rebalancing.

In the case of a computer node failure, the platform also moves data virtually to another container in a network.

“At cloud scale, everything fails,” said Kam Eshghi, Chief Strategy Officer at Lightbits Labs’ chief strategy officer. Hence, the upgraded platform is designed to prevent data loss and service interruptions at scale if storage, servers or a network crash.

The approach allows container-based applications to leverage disaggregated storage via the Lightbits Labs’ CSI plugin for Kubernetes, “spinning up persistent storage volumes for containerized workloads just as easily as spinning up another container,” Eshghi claimed.

The persistent storage framework can be installed on commodity datacenter servers to run I/O-intensive computing clusters. Those clusters include Cassandra NoSQL, Kafka data streaming and MongoDB as well as time-series database workloads. Each storage server in a cluster can support 16K connections, the company said.

As more applications move to the network edge, datacenter equipment vendors say they are integrating storage platform supporting Kubernetes into their datacenter fabrics. The combination provides flash-style performance with greater reliability than server-based storage drives. Lightbits said its updated platform is geared toward less expensive quad-level cell flash drives, a technology that has received mixed reviews.

The company said its disaggregated storage architecture performs like attached NVMe SSDs with half the latency. The goal is to transform commodity servers into a storage pool linked via NVMe/TCP to the Kubernetes cluster orchestrator.

About the author: George Leopold

George Leopold has written about science and technology for more than 30 years, focusing on electronics and aerospace technology. He previously served as executive editor of Electronic Engineering Times. Leopold is the author of "Calculated Risk: The Supersonic Life and Times of Gus Grissom" (Purdue University Press, 2016).

AIwire