There is little doubt that all-flash arrays can solve most performance problems in today's enterprise data center. But that efficiency comes at a cost. As a result, all-flash storage array suppliers are looking for ways to maximize the use of these systems with various storage efficiency techniques like thin provisioning, deduplication and compression.
In this series, we will look at each of these techniques so you can understand its pros and cons, as well as decide which features deserve the highest weight in your all-flash array selection process.
Thin provisioning is increasingly becoming a standard feature in storage systems of all types today, whether they are all-flash, hybrid or hard-drive-based. Thin provisioning allocates storage capacity on the storage system dynamically as an application needs it. Conversely, thick provisioning allocates capacity that application owners expect they will need upfront.
For example, let's say an application owner asks the storage administrator to allocate 1 terabyte (TB) of capacity to a new application they are bringing online. It is well understood that the application won't need that 1 TB of storage for quite some time, but in a thickly provisioned environment, that 1 TB is fully allocated to that application and no other application can use that capacity. To make matters worse, if that application grows beyond 1 TB, expanding a thickly provisioned volume is a complex problem that may need manual intervention that could potentially result in downtime.
In this same situation, a thin-provisioned volume could be set at 1 TB or more but the capacity of that volume could be allocated only as the application needs it. With thin provisioning, it doesn't matter if the application takes a year or more to reach its allocated capacity, because its capacity is available to any application in the meantime. Also, the storage administrator could double or triple the allocation request with no impact on physical capacity utilization.
In all-flash arrays, thin provisioning should be considered a required feature. The more commonly discussed storage efficiency features like compression and deduplication can't provide efficiency on the free space dedicated to a certain server or application.
But there are ramifications to using thin provisioning that users need to be aware of. First, thin provisioning can impact performance at the worst possible time: when an application is adding or modifying data. The reason is that as the application adds data, the thin provisioning process needs to allocate capacity to the volume that the application is using.
That process takes time, especially on flash, since an allocation is essentially a write process, which is the slowest and the most resource-intensive activity that a flash-based system has to perform. The good news is that flash's write performance is still faster than that of a hard disk, so the performance impact should be less noticeable.
Thin provisioning is not created equal
When you're evaluating all-flash arrays, it is important to understand that not all thin provisioning is created equal. Vendors use different techniques to deliver thin provisioning, and how well it works depends largely on their ability to virtualize the underlying storage.
Thin provisioning can be provided in one of two ways. First, it can be provided at the exact moment capacity is requested; so, if 2 MB of data is ready to be written by the application, the storage system first allocates 2 MB of capacity to the volume, and then the application writes the data. This method is extremely capacity-efficient, and most ideal for flash, except that it can impact performance if lots of these on-the-fly allocations must occur simultaneously across dozens of servers and hundreds of virtual machines. Whether the storage system is HDD-based or flash-based, this thin provisioning method requires well designed storage software and appropriate storage processing power to avoid a performance impact.
Some vendors provide thin provisioning-like experience by pre-allocating a larger chunk of capacity -- as much as a few hundred gigabytes -- in advance to each thin-provisioned volume. Sometimes this is called chubby provisioning because the method allocates capacity in advance. This thin provisioning method reduces the performance demand of constantly having to allocate space, but is not as capacity efficient, a bigger issue with premium-priced flash storage than with consumer-grade flash storage (think thumb drives that you can buy for a few dollars). That "few hundred gigabytes" across those dozens of physical and hundreds of virtual servers can add up to terabytes of unused capacity, which on flash can be quite costly.
The second method can also cause problems in flash-based storage systems beyond wasting capacity. This is because, when this threshold is reached, the allocation of another few hundred gigabytes is a large, all-at-once write operation that has to happen before the application is expecting to write its own net new data; even if that application write only consists of a few kilobytes of data. The result could be an increase in latency, which would manifest as erratic application performance. Performance is actually less affected when the volume is expanded in much smaller increments. The on-the-fly concerns are minimized by the high performance of flash that provides instant response times.
Thin provisioning is a critical feature for flash-based systems. Flash buyers should look for storage systems that provide thin provisioning as a built-in function, not an afterthought. These systems need to provide a fine-grained level of storage allocation so that no over-allocations of capacity need to be set aside, wasting capacity and causing performance problems.
Guide to enterprise all-flash storage arrays
All-flash arrays: Performance vs. function
All-flash storage arrays offer choice