Outside the realm of PCs and laptops, traditional disk arrays are the most popular choice for solid-state storage among early corporate adopters. But research shows an uptick in server-based options, fueled in part by increasing interest in cards that plug directly into PCI Express (PCIe) slots.
While a number of solid-state storage companies are working on PCIe-based products, two companies that already have prominent offerings, Texas Memory Systems Inc. and Fusion Multisystems Inc. (which does business as Fusion-io), have been finding favor among IT organizations with especially I/O-intensive applications.
The main reason PCIe-based solid-state storage users say they're attracted to that approach over other server-based SATA/SAS/Fibre Channel (FC) solid-state drive (SSD) options is the faster performance they hope to achieve through a direct connection to the PCI bus. On the downside, PCIe-based solid-state cards are limited by the number of PCI slots in a server and, in many cases, subject to downtime during server maintenance or repair.
While the use cases for solid-state storage tend to be limited to mission-critical I/O-intensive applications because of the technology's higher cost vs. hard disk drives, the users of PCIe-based solid-state storage to whom we spoke say the cards have lived up to their expectations and delivered consistently high performance.
Market share for PCIe solid-state storage
NAND flash-based PCIe cards comprised just 6% of solid-state unit shipments last year, well behind 2.5- and 3.5-inch solid-state drives, according to Gartner Inc.'s form factor forecast. But Joseph Unsworth, a Gartner research director, predicted that all enterprise SSD suppliers will have PCIe products available in 2012, and that PCIe cards will account for 24% of solid-state unit shipments the following year.
"We're seeing a lot of attention right now on PCIe cards because this is going the direct-attached route in servers. If you look at who's coming with PCIe cards, it's just about everybody," Unsworth said, noting keen interest from vendors such as Dell Inc., Hewlett-Packard (HP) Co. and IBM, which all sell PCIe-based products made by Fusion-io.
Unsworth added that the leading supplier of SSDs, STEC Inc., is working on PCIe-based products, and he expects other major suppliers to join the fray, including Intel Corp., LSI Inc., Marvell Technology Group Ltd. and Seagate Technology LLC.
In the meantime, existing solid-state PCIe offerings are slowly starting to find a captive audience among IT shops that need to boost the performance of isolated I/O-intensive applications, such as Web 2.0, social networking, online transaction processing, data warehousing and especially active databases.
Ryan White, director of operations at Cloudmark Inc., which provides spam, fraud and virus protection services, said his company's automated message-filtering system is so specific to a fixed size of 160 GB to 200 GB with such a consistently high IOPS rate that its Fusion-io PCIe cards never get a break.
But before settling on the Fusion-io cards, White considered a variety of other solid-state storage options, including SSDs that plug into SAS or SATA interfaces and dedicated solid-state appliances. At "$50,000 to get my feet wet," White found the appliances too expensive, and was concerned that a SAS or SATA controller would be acting "as a middleman when it doesn't really need to be.
"It slows things down," White said, echoing the sentiments of other early adopters who favor the PCIe approach. "I was looking for something a little more direct."
While PCIe-based solid-state storage offers a more direct connection to the PCI bus, the serviceability challenges associated with it can lead to downtime. "In many cases, you have to take the server down if you want to upgrade or change over a failed card," said Jeff Janukowicz, a research manager in IDC's storage group.
Server-based PCIe-based options come in a variety of flavors. In addition to SATA-based SSDs, for instance, HP sells a StorageWorks IO Accelerator card (custom-built by Fusion-io) that plugs into the mezzanine slots of its BladeSystem c-Class servers. HP also offers a PCI Expansion Blade that accepts two PCIe or PCI-X solid-state storage cards.
Texas Memory vs. Fusion-io
Texas Memory Systems' PCIe-based RamSan-10 and RamSan-20 products differentiate themselves with an on-board flash management suite the company claims ensures flash SSDs have minimal impact on host server resources and prevents data loss during server crashes/power loss.
"Flash is a complicated chip, and making an enterprise system requires a lot of engineering," said Woody Hutsell, Texas Memory Systems' president, who claims his company's product is "much more complete" with its inclusion of flash, memory, CPU and an embedded CPU controller, in comparison with the "raw flash" of Fusion-io. "The core difference between us and them is [that] we do all of the flash management on the system itself, on the board, whereas they use the server resources to do that," he said.
Rick White, chief marketing officer at Fusion-io, said the company builds dual inline memory modules (DIMMs) that slot onto carrier cards, which fit into a server's PCIe slots. He claimed that his company's PCIe-based offerings are neither disk nor drive nor storage, but rather a "new type of memory tier" that uses direct memory access (DMA) "just like your RAM would." He said the products emulate disks but don't operate internally like disks.
"We're more like RAM than a disk," he said, adding in reference to Texas Memory Systems' approach with its RamSan-10 and RamSan-20, "They're more like a disk than RAM."
Fusion-io's White contended that Sun Microsystems Inc., now owned by Oracle Corp., is "one of the few companies that gets it," with internal projects in the works promising a technology approach similar to the one that Fusion-io takes.
"Everyone else seems to think it's about PCI Express and the throughput on PCI Express being higher than SATA or Fibre Channel. But that has nothing to do with it," he said. "The application acceleration customers are looking for comes from things like bandwidth [to each of the NAND flash chips] and latency, and not just I/Os and not just the bandwidth of the bus."
Fusion-io's customers tend toward system, application and database administrators who have empty PCIe slots and use Fusion-io technology similar to the way they use memory, White said.
"But the storage administrators, I'm sure they scratch their heads and can't figure it out," he said. "The other customers who don't really benefit from us are folks who are CPU-bound, not I/O-bound. We solve I/O bottlenecks where you just can't get enough I/O."
Applications that are I/O-bound, rather than CPU-bound, are the sweet spot for solid-state storage in general, whether the SSDs are deployed on servers, dedicated appliances or storage devices. Random reads and, to a lesser extent, random writes, gain the greatest performance benefit from SSDs vs. hard disk drives because the SSDs don't have to move an actuator to locate the data.
No matter the form factor, solid-state storage will continue to grow in popularity, as prices drop and performance improves, according to IDC's Janukowicz. IDC expects SSD revenue to increase by 87% this year, and by 2013 to break the $2 billion mark, he said.
This was first published in March 2010