Bringing Together the Features Of VVol and Traditional Data Stores

Bringing Together the Features of VVol and Traditional Data Stores

 

Introduction:

Virtualization plays a vital role in the modern enterprise IT infrastructure which had started off with software-defined compute. Later virtualization has become the de-facto technology used by many companies worldwide.  Virtualization has benefits like cost effectiveness, effective way of utilizing resources and so on, which leads Virtualization to move next level from compute to virtualized Data Center known as Software-Defined Data Center (SDDC) with three major building blocks namely Server virtualization, Storage Virtualization and Network virtualization.

In SDDC, Customer often points Storage as one of their top concerns on both cost and performance as well. Software-Defined storage (SDS) is the dynamic composition of storage services, aligned on application boundaries, driven by policy. Storage services are nothing but the combination of capacity, performance, deduplication, encryption, and so on which is provided by physical arrays. The SDS model will enable the granular adjustment of individual application components, independent of storage configuration boundaries (e.g. LUNs) which are not available with Traditional Datastore of vSphere environment. This can be achieved with Storage Policy-Based Management (SPBM) in SDS with Policy-Driven control plane layer.

Virtual Volume (VVol) is a new virtual machine disk management and integration framework which helps to leverage array based features for virtual machines. This innovation is not only for virtual infrastructure but also for IT architects & CTOs, as it allows individual applications to utilize the most appropriate IT resources available, with minimal administrative intervention. In this paper, we will explore the differences between Legacy storage and Software-Defined-Storage with Virtual Volumes, as well as some of the specific benefits achieved when using VVol enabled storage system from Dell EMC.

 

Before VVol:

In earlier days, Storage has been provisioned to ESXI host by Storage admin as LUN or NFS. In this article will discuss with respect to LUN, instead of both LUN and NFS. vSphere admin has to create a Datastore from LUN where the virtual machines sit. We all know that a Virtual machine is the set of files to create VM such as VMDK, VMSS, VMSD, VMSN and so on. In the traditional method, LUN is an entity which is used to store data in an array and act as an access point between ESXI host and Array. What an access point does? Access point helps ESXI host to send IO to the array. By this we can understand, every time when we create a LUN for ESXI host it creates access point as well. For example, if we have to create 100 LUNs the same number of access point also created within that LUNs which is not really required. VMware and Storage vendors developed Virtual Volume to avoid these challenges and will see briefly in following sections.

 

What is Virtual Volume?

In Storage aspects, Virtual Volume (VVol) is a virtual machine object stored natively on the storage with the help of Storage Container (SC). Storage Container (SC) is new term introduced with this release, we will discuss more briefly under Architecture section. Let us consider Storage Container as Physical array to understand the concept. Now we will see the definition of VVol given by VMWare,

“VMware Virtual Volumes is an integration and management framework for external storage that provides finer control at the VM-level streamlines storage operation and offers flexibility of choice.”

It can be called as Policy driven framework as well which enables per-VM storage management with set of policies. I agree you might have a question what are the changes with this new storage paradigm which is not with legacy datastores. I will elaborate those changes more in the upcoming section.

 

Virtual Volume Architecture:

To understand the Virtual Volume architecture will compare same with the existing Traditional Data store concepts. As we discussed earlier, In the traditional datastore LUN has to do two operations which store data in the storage array and act as an access point or mount point. In Virtual volume, we don’t have a concept called LUN and File System to create VM and manage it. This is achieved by Virtual Volume components such as vSphere API Storage Awareness (VASA) provider, Storage Container, and Protocol Endpoint and we see more like how data services are offloaded to the array and how we manage through Storage Policy-based management framework.

Figure 1: VMware Virtual Volume Architecture

 

VASA Provider (VP):

VASA stand for vSphere API Storage Awareness which is a set of APIs that ESXi exports to the underlying array. This bridges a gap of communication between ESXi and array. For example, we would like to take snapshot for a VM. VP actually transports into API and that API takes the snapshot in the storage array. VASA is a software component which has to be developed by Storage vendors like Dell EMC.

VP is a key which communicates between ESXi and Storage array. VPs are express to do thing two things. One is ESXIs expose fine granular details of files of VM. We will use APIs to actually have those individual files reside natively on the array. By this, our VM is a Virtual Volumes resides on the storage array. Single VASA manager can manage multiple arrays. VP can be implemented within arrays management server which is embedded in to storage array or firmware update or plugin for the storage array by the storage vendor. And the other important responsibility for VP is creating Virtual volumes.

Initially vSphere admin has an insight about which VM resides on which LUN or FS or Data Store. So by this development, Storage admin also have this information. Henceforth array is aware of these individual objects, now they can do direct services (Compression, De-duplication, Encryption, so on.) on these objects. Object here refers to virtual machine.

Example: During backup, the VM because of its fine granularity at array level, it translates backing up as an individual object. With this we can see efficient performance in storage. Earlier it takes an hour to backup, but now it will be reduced to matter of few minutes.

Figure 2: vSphere API Storage Awareness (VASA) Provider

 

Protocol Endpoint (PE):

VASA taken responsibility of creating VM and still one more action of LUN is not achieved that is an access point. Protocol Endpoint takes the responsibility of access point to send IO from ESXi to array. This is a brilliant idea that operation of a single LUN, like the creation of VM and access point divided into two entities named as VP and PE.

Initially it was just one entity. But how are we benefited from taking advantage of these two new entities? Why we need two instead of one? One or Two sounds difference. Before getting into advantage of this development, will discuss what are the challenges & limitations we faced on traditional storage paradigm.

In traditional storage, whenever we create a new LUN, access point also gets created along. As we know LUN has two tasks to do. So there will “n” number of access point which is not necessary. Definitely this issue has been fixed by creating required number of PE (Protocol Endpoint) as per the need. You may wonder if these are the only challenges in traditional storage. Then my answer will be No. I am trying to explain each component of VVol with its benefits compared to traditional storage paradigm.

Figure 3: Protocol Endpoint (PE)

 

Storage Container (SC):

From the beginning we were hearing the term Storage Container (SC), finally, we are there. Storage Container (SC) is a concept which helps to store the objects in an array. This is the other operation done by LUN which we discussed earlier. SC will be created by Storage Admin in the array with the desired capacity. As Sphere admin creates new VMs that will be provisioned in the storage container.

In traditional storage paradigm, LUN is used to store the VM files. In terms of storage, LUN has certain restrictions like the limitation of LUN capacity and the number of LUNs in the traditional method. Storage Container doesn’t have any those limitations regarding capacity. But all up to based on storage capacity, not more than that and SC doesn’t go across the array. A single SC can be accessed by multiple PE when you think about multiple workloads on an SC. You may ask if you can create SC in all arrays? No, this can be performed only on VVol enabled arrays such as Dell EMC VMAX, Unity, etc.

Figure 4: Storage Container (SC)

 

vSphere Virtual Volumes Requirements:

Software:

To utilize Virtual Volumes the following software components are required,

Ø  vCenter Server 6.0 Appliance (VCSA) or vCenter Server 6.0 for Windows

Ø  ESXi 6.0

Ø  vSphere Web Client

 

Hardware:

To utilize Virtual Volumes the following hardware components are required,

Ø  Any Server that is certified for vSphere 6.0 is listed on the VMware compatibility guide.

Ø  A storage array that supports vSphere Virtual Volumes will be able to integrate with vSphere through the VMware APIs for Storage Awareness.

License:

To utilize Virtual Volumes the following license is required,

Ø  Standard

Ø  Enterprise Plus

 

Managing VVol through Policies:

Virtual Volume is a policy driven framework which uses Storage Policy-based Management (SPBM) and VASA. SPBM exists already within vSphere framework and we also discussed a lot about VASA and the functions in the architecture section. SPBM is a VMware’s policy-driven control plane layer which defines the storage requirements of VM with certain set of rules (policies) in vSphere associated with the individual VM i.e. the virtual volumes and dynamically placing the VM on the right storage and making use of data services provided by physical array.

Figure 5: Manage Virtual Volume with set of rules

I have a snippet which helps to understand more about managing VM through policies. In the stage of creating new VM on storage container, vSphere 6.0 will ask for the rules to be aligned for the VM. SPBM & VASA API plays an important role to align specific VM with the storage services without manual intervention. With this new development data services has been offloaded to array.

Figure 6: Create New VM storage policy for Virtual Volume

 

Dell EMC implementation of VVol:

Dell EMC always does dynamic changes over their product especially on storage arrays which act as a benefit for customers in both cost effective and performance. After the announcement of Virtual volume by VMware in VMWorld 2014, Dell EMC engineers started working on the development of implementing Val option in the arrays which starts from the van to Unity. Dell EMC introduced VVol enabled arrays on both Enterprise and Midrange customers which include ALL-FLASH Array as well. Though Dell EMC has multiple products to support VVol, here we will see how VVol implemented on Dell EMC Unity.  

 

Integration of VVols with Dell EMC Unity:

Dell EMC Unity is the only storage system that successfully meets all the 4 requirements of today’s IT professionals namely Simplicity, Modern design, affordable price, and deployment flexibility. It is available in Hybrid array, All Flash array and Unity VSA (Software Defined Storage) where VASA Provider is embedded in the system, so there is no external plugins or add-ons are required. In order to make use of VASA, the Unity system must be added as a vendor provider in vSphere environment. Unity supports both Block and File for VVol.

In Unity, Storage Container formed by single or multiple storage pools, where each pools are configured with mixture of drives i.e. Flash, SAS, NL-SAS with supporting data services such as Tiering, Encryption, etc. Unity has a new entity named as Capability Profile which is the characteristics of a storage pool as part of SPBM. That includes service levels, Fast cache, drive type and so on.

Below are the capabilities Unity can advertise to vSphere environment,

Data services: Snapshots, Clones, Data Protection, Data Reduction, etc.,**

Drive types: Indicates the type of drives available in the storage pool

RAID types: Indicates RAID protection of the storage pool

FAST Cache: It’s a Boolean value to indicate whether FAST Cache is enabled for the storage pool

Tiering Policy: If FAST VP is enabled, storage tiering policy will be assigned to the storage pool

Usage Tag: Storage administrators can assign custom tags to the Capability Profile to indicate preferred uses of the storage pool (i.e. workloads, applications)

Service Level: Predefined descriptors to translate the Drive and RAID types of the storage pool into an indicator of service level, but not a service level objective.

**Note: Data services are offloaded to array

Figure 7: Dell EMC Unity Virtual Volume Implementation

In storage, where VVol environment being configured, the vSphere administrator can provision the VVol Data stores from the Storage Containers presented to the ESXi hosts.  From there, they can create storage policies using the advertised capabilities via Unity’s VASA Provider.  With SPBM, VMs can be created with the storage policies that will enforce appropriate storage is used to service the VM.  Compliance checks are available to ensure the underlying storage of the VM complies with the configured storage policy or policies.

We will see a demo how VVol can be provisioned to ESXi host for creating new VM on VVols. Before we start, have to create a pool in Unity array with Capability profile and the ESXi host has to be discovered from the array which defines a communication path through which a specific host or range of hosts can access storage resources.

Figure 8: Discover ESXi Host from Unity array

Here we will discuss only about Block and how it configured, even though File also supported. In Unity array, Protocol Endpoints will be created automatically when a host is granted access to a VVol datastore. For Block, iSCSI interface or FC connection are utilized for SCSI PE to send I/O. Two iSCSI PEs are created per ESXi host that has access to the storage system. The Block VVol will be bound to the associated SCSI PE every time that the VM is powered on. When the VM is powered off, the PE is unbound. SCSI protocol endpoints are like LUN mount points that allow I/O access to VVols from the ESXi host to the storage system.

Figure 9: Unity Array - SCSI Protocol Endpoint

For the ESXi host to communicate with the Unity array, add the Unity array as a storage provider in the vSphere client. It can be performed as below,

Figure 10: Register Storage VASA Provider with ESXi Host

**Note: It is recommended that you specify a user account with the VM Administrator role.

VVols reside in VVol datastores, also known as storage containers, which are comprised of storage allocations from one or more capability profiles. We can create VVol datastores based on one or more capability profiles and then allocate a specific amount of space from the capability profile to the VVol datastore. When a virtual volume is created in vSphere, it is assigned a storage policy profile. vSphere filters the compatible and incompatible available VVol datastores (from one or more storage systems) when the VVol is being created based on these profiles. Only VVol datastores that support the storage policy profile are considered compatible storage containers for deploying the VVol.

Figure 11: Unity Array - VMware Datastore

Once the VVol provisioned to ESXi host, process of creating VM in ESXi host exists the same as we do follow with the legacy storage paradigm.

 

Summary:

With the introduction of VMware Virtual volumes, we can reduce the administrative task of both Storage and vSphere administrator with Policy-driven framework. Though we still have Datastore entity to utilize the features of vSphere such as Fault Tolerance, DRS, HA, etc.,   This is just an overview of virtual volumes architecture and its components and how vvol used in Unity Systems. For further information regarding Virtual Volumes, Unity VVol integration and Unity VVol configuration guide please refer Dell EMC whitepaper.