VMware To Complete Storage Virtualization With VVols 

VMware launched its Virtual SAN (VSAN) hyperconverged storage for its vSphere virtualized servers last fall and started shipping it this spring and has seen very good uptake of the technology, with 12,000 beta tests through the spring and more than 300 customers in production in the first three months of commercial availability. But that is not the end of the virtual storage story at VMware.

The next release of the hypervisor will include a feature called Virtual Volumes, or VVols for short, and this type of storage virtualization will be employed across all kinds of devices to virtualize the data plane between virtual machines and physical storage devices. In a way, VSAN offers a kind of preview of the VVols technology. In fact, Gaetan Castelein, senior director of product management and marketing for software-defined storage at VMware, explained to EnterpriseTech at the VMworld conference this week that the first release of VSAN has support for VVols in it already, and it is one of the reasons why VSAN can manage storage at the VM level instead of at the logical unit (LUN) level in a storage array as has been necessary in the past. (The "V" is capped with VSAN and VVols, but not with other VMware products – go figure.)

In a software-defined everything world, every element of compute, storage, and networking should be individually and programmatically managed with the option of then grouping elements together to be managed as a whole when it is appropriate. This is how Google, for instance, manages its massive, multi-million collection of software containers internally and this is also how Google's open source Kubernetes tool will be used to manage Docker containers and the pods that are collections of containers with similar attributes. (Docker plus Kubernetes takes inspiration from Google's internal software, but is not based on it.)

Up until now in the server virtualization world, said Castelein, the operational model for storage for virtual machines was a bottom up approach, where storage was provisioned for hypervisors in a static manner and bubbled up to the VMs. But this is not the correct way to do it, according to VMware. "The infrastructure needs to follow the virtual machines, not the other way around," Castelein contends. "The hypervisor ends up brokering storage between the applications and the underlying infrastructure."

This legacy method of providing storage for VMs has challenges for both IT shops and for those who make storage devices. The provisioning takes a lot of time and it does not have granularity down to the VM level as is clearly necessary if you want to have individual policies for each VM that control the quality and speed of the storage as well as its attributes such as whether or not encryption, compression, deduplication, replication, snapshotting, and other services are available for the storage underpinning the VMs. Storage array makers, Castelein said, are hampered by complex LUN and volume management and fragmented device management.

By abstracting the data plane between VMs and their apps and the underlying storage, which is what VVols does, the storage consumption model is common across all tiers, including VSAN and similar software that runs in a hyperconverged manner on compute farms as well as on outboard SAN and NAS arrays as well as on cloud storage platforms. In essence, the VMDK virtual disk that encapsulates and defines a virtual machine becomes the primary unit of data management. No more configuring LUNs and volumes on disk arrays, the hypervisor just reaches in through a set of APIs (called VASA, short for VMware APIs for Storage Awareness). Because all vendors will support the VASA APIs and the VVols abstraction layer that allows for VMDKs to be stored natively on their devices, the provisioning of storage for the ESXi hypervisor will be able to be automated to a large degree, as it is when using VMware's own VSAN, for instance.

The VASA APIs have been around since the ESXi 5.0 hypervisor and some storage array makers have made use of them to better integrate their devices and storage operating systems with the vSphere stack. But with VVols VMware is coming up with a consistent means of representing VMDKs across all kinds of storage devices.

VMware has announced 29 partners for VVols and it will support the linking of servers to storage over existing storage protocols such as Fibre Channel, iSCSI, and NFS. Tintri, one of the upstart server makers, had already come up with its own scheme for managing storage at the VM level, but Castelein said that this was done manually and now with VVols Tintri, like other array makers, would be able to manage VM storage programmatically. Dell, EMC Hewlett-Packard, Hitachi Data Systems, IBM, Fujitsu, NEC, NetApp, Nimble Storage, Pure Storage, and SolidFire were called out as the major adopters of VVols in VMware's presentations.

VVols doesn't just happen magically with the use of ESXi 6.0 and an external disk array. For those array makers who are going to map LUNs to VVols, the integration will be fairly easy, explained Castelein. For Tintri, it is more about exposing what it can do already. Others may choose to develop more sophisticated code to take full advantage of VVols. "All of these folks will have support very close to after we GA," said Castelein, adding that the work they had to do to support VVols was not monumental, in contrast to something like adding inline deduplication to an array, which certainly is.

NetApp is one of the early adopters of VVols, and Adam Fore, director of virtualization marketing solutions, told EnterpriseTech that NetApp was in fact a co-development partner for the technology in partnership with VMware.

While VSAN is good for certain workloads, VMware certainly has not pitched it as a product to replace actual NAS and SAN arrays for all workloads and all customers. It has scalability issues even though it is pretty impressive in its first release – 2 million IOPS across the 32 node limit of a vCenter cluster with 4.4 PB of capacity using 4 TB drives – VSAN does not have deduplication or compression yet and the data replication software that is in VSAN is not synchronous and is based on vSphere replication, which has a 15 minute recovery time objective and which is done asynchronously. The snapshotting is similarly done through vSphere at a much higher level.

For many customers already familiar with the data management software in their arrays, VVols is a blessing because it will allow them to treat a VM just like they would any other chunk of data, and use the replication, deduplication, snapshotting, and compression features of their arrays on the VMs, all transparent to the VMs and in conjunction with the ESXi hypervisor.

For NetApp, explains Fore, the use of VVols in conjunction with its FAS arrays and their Data ONTAP operating system, that means being able to do snapshots on a regular schedule and to implement retention policies for VMs as a company would for any data. (No more zombie VMs lurking in the storage.) The performance of the storage on each VM can be set as well, and VMs can be moved from all-disk to hybrid to all-flash FAS arrays as needed to boost their I/O capacity so long as customers have a mix of FAS arrays in their Clustered Data ONTAP machines. Recovery policies will similarly be the same for VMs as for other data in the NetApp arrays, and by creating policy groups for VMs, the FAS arrays will be able to manage thousands of VMs as a unit while giving them absolute individuality should one need to be altered.

NetApp has an open beta on the use of VVols on its FAS arrays right now, and it requires customers to be at the ONTAP 8.2.1 level. They also need a virtual storage controller for vCenter, which exposes the user interface for VVols and its integration with the FAS arrays.

VMware has obviously not forgotten that it is also majority owned by storage array juggernaut EMC, and here is conceptually how VVols will plug into EMC's ViPR object storage software:

The biggest change that customers might experience using VVols is an operational one. Now, the policies will be set through the hypervisors and cascade down to the VMs and their storage. This is different from the fairly manual processes and ticketing systems that storage admins use today. The storage admins control the capabilities that they want to make available to VMs, and the server admins set the policies that allow the hypervisor to subscribe to them for specific VMs.

VVols is in beta testing now along with the future ESXi 6.0 hypervisor, and it looks like this will in fact be the big new feature when ESXi 6.0 ships. VMware was mum on the subject of the timing of both VVols and ESXi 6.0 and would not even confirm that the two are tied together, but the odds favor that VVols is part of the ESXi 6.0 hypervisor and moreover that VMware will announce both at the VMworld Europe event in Barcelona, Spain in October. All that Castelein would say is that VVols "would be available soon."