Scale-out Storage and Falling Flash Prices

by Howard Marks, Deep Storage LLC

Even though every one of the 30 years I’ve been in the IT business has brought lower prices and higher quality than the year before, most organizations don’t factor this into many of their technology purchasing decisions. They buy systems fully populated to avoid the complications and disruption of in-use upgrades.

While this approach may have made sense in the past, it leads to over-buying of capacity, since users have to purchase sufficient capacity today to meet their requirements three to five years in the future. It also leads to overspending in dollars, as the capacity they will need two years from now will cost less in the future than it does today. At times like these, when the cost of all-flash storage is falling at a rate of 30% or more each year, the waste becomes even more significant.

In addition to Adam Smith’s “invisible hand” of supply and demand, the cost of flash memory has been driven by the use of ever-smaller cell geometries. The mainstream SSDs of just a few years ago, like Intel’s X25, used 25nm or even 33nm flash cells. In comparison, today’s planar flash uses cells just 16nm across. Smaller cells mean more bits per wafer, and since cost per wafer is a relative constant, lower costs per bit.

But the cost of flash in enterprise data centers is falling even faster than the cost of raw flash. A big part of this widening gap is the acceptance of lower grade flash in the enterprise. Just a few years ago, when flash was new and the concept of limited write endurance was scary, only SLC flash was good enough for enterprise flash drives.

Read about Data Centers:

View More Articles

Read about Server Virtualization:

View More Articles

Today, most enterprise storage systems use ordinary 2 bit per cell MLC flash rather than the much more expensive SLC enterprise users of yesteryear demanded. In part, this is due to the market's overall acceptance of flash and customers’ realization that their early concerns about flash wear-out were as overblown as worries that hash collisions on a deduplicating storage system would result in universal data corruption.

Just as significantly, the flash controller chips that turn raw flash into SSDs, and the modern flash storage systems that use them, have gotten much better at managing flash. Today’s SSD controllers use high-density parity coding for their error correction, which allows them to get several times more endurance out of the flash than the controllers of a few years ago that used less powerful error correcting codes.

Enterprise storage vendors, including SolidFire, have even started using 3 bit per cell TLC flash, which many of us had claimed just a few years ago would never be ready for prime time, or at least data center use. As the flash vendors change their production processes from making ever smaller planar cells to 3D NAND flash that stacks cells vertically on the chip, they are also making each cell bigger, allowing them to contain enough electrons to give TLC flash the endurance needed for enterprise use.

The road maps of all four flash foundries call for them to stack flash cells on chips higher and higher over the next several years, which should continue to drive flash prices down at a rate of 30% per year or so for the rest of the decade.

Taking advantage of the falling cost of flash with scale-out storage systems can be difficult. In the best case, when your storage vendor adds denser, cheaper SSDs to their product line, you can add a shelf or two of the new SSDs to your system. On many systems, especially those with roots in traditional spinning disk RAID systems, that shelf of 4TB or 8TB SSDs will have to be a separate storage pool because the system can't put 1TB and 4TB devices in the same RAID set. Multiple storage pools means more time managing the system and lower utilization as administrators reserve a bit of buffer space in each pool for unexpected data growth.

At some point the owners of scale-up systems will discover that their controllers don’t have enough RAM to manage the deduplication hash tables for significantly larger SSDs. Or they may find that they are unable to add new, less expensive flash to their system for reasons such as the vendor simply not supporting new SSDs on old controllers in order to incentivize customers to buy new storage systems.

Users of scale-out systems like Solidfire's, which supports nodes of different capacities in the same cluster, can simply add new nodes full of denser, cheaper flash to their clusters as they need added capacity. Since adding new nodes takes just a few non-disruptive minutes, users can postpone storage purchases and take advantage of the falling cost of flash.

Howard Marks is founder and chief scientist at Deep Storage LLC.