Stock analysis software company VectorVest Inc. presented a classic use case for NAND flash -based solid-state storage via PCI Express (PCIe) cards in servers: an I/O-intensive application, crucial to its business, requiring the ultra-fast delivery of data to customers over the Internet. After scoping out various options, the company settled on Texas Memory Systems (TMS) Inc. RamSan-20 solid-state drives (SSDs).
Before deploying the RamSan storage, VectorVest's Cornelius, N.C.-based IT staff had tried to squeeze better performance out of its existing infrastructure, adding memory to its servers and testing different disk configurations with its Hewlett-Packard (HP) Co. StorageWorks Modular Smart Arrays (MSAs). But the group never saw the sort of gains they'd heard solid-state storage might bring, according to Matt Brooks, the company's network administrator.
Brooks dismissed the concept of solid-state drives in disk arrays out of concern that the SAS link between the array and the host servers would limit performance. He also briefly looked into a direct-attached DRAM appliance before realizing the technology would be too cost-prohibitive for the amount of storage it offered.
Figuring a direct PCIe connection would mean lower latency, VectorVest decided on PCI-based NAND flash drives for its HP ProLiant DL rack mount servers (each with 16 PCI slots) in hopes they would produce a dramatic performance boost for the SQL Server databases that drive the information feeds to investors who use the company's online real-time stock analysis software.
VectorVest initially tested first-generation PCIe-based solid-state cards from Fusion-io, but after encountering problems with the nascent Windows drivers, shifted to PCIe-based SSDs from Texas Memory Systems. VectorVest eventually purchased 16 TMS RamSan-20 solid-state drives to replace 250 SAS drives in its HP MSA storage systems.
One test showed the PCIe flash-based SSDs handling close to 3,000 transactions per minute -- processing complicated data queries from start to finish -- whereas the regular disk subsystem sustained 900 transactions per minute. "We worked significantly with TMS to try and optimize the drives to get to that number," Brooks said, noting that the performance boost was most pronounced for random access, on both reads and writes. The company didn't note any major improvement for sequential backups and restores, he said.
SSDs: Implementation challenges and pricing
The main implementation challenge was deciding on the drive sector size, or smallest addressable unit on the storage. VectorVest's database servers had been using 512 bytes with its 15,000 rpm SAS drives, but the native sector size on the new TMS SSDs was 4,096 bytes (4 KB, commonly referred to simply as 4 K). But after consulting with TMS, the IT staff opted to test the performance differential between 512 bytes running in compatibility mode and the SSD's native.
"SSDs work best at 4 K sector sizes, but not every application will work with it. So, the easiest thing to do is have the SSD present a 512 byte sector and then translate it to 4 K internally," said Jamon Bowen, director of sales engineering at TMS. "That way, you can just copy the data from the prior storage to the SSD and go. You don't have to do anything special. But if you don't do anything special, you might not get the best performance."
Running at 512 bytes in compatibility mode can require significant tweaking and tuning, and become somewhat difficult for users, Bowen said. But because Windows and SQL Server support storage devices that present 4 K sectors, users can avoid misconfiguration issues or tuning needs by converting their database to 4 K sectors, he noted.
"It's not terribly difficult, but you have to follow a specific procedure," Bowen said. "You detach the database, copy it from its existing storage to the 4 K sector SSD and reattach the database using new transaction logs."
VectorVest's testing showed that the native 4 K sector size resulted in better performance, so four staffers spent a couple of days doing the necessary conversions. "Learning to do exactly what we needed to do did take some time, but once we knew what to do, it was actually fairly easy," Brooks said.
The IT staff put the first RamSan-20 into production in September. Before the shift to solid-state storage, the raw disk throughput for copying files from disk to disk was roughly 400 MB per second. The figure has spiked as high as 2,500 MB per second when writing to multiple TMS cards at the same time, according to Brooks.
"All the SAS drives are going through 3 [Gbps] links. That was probably a bottleneck, because you can put only so many drives on one of those before you reach maximum capacity," Brooks said, noting that the TMS cards talk directly into the PCIe bus. "The drives themselves are capable of only 600 MB or 700 MB per second, but when you have multiple [cards] all taking in data at the same time, you get significantly higher throughput."
Brooks said any loss in capacity had been more than offset by the gains in performance. VectorVest had 700 GB of usable storage with each of its SAS disk arrays vs. 450 GB of usable storage with each of its RamSan-20s, "but that isn't a big concern of ours because 400 gigs is still a lot of space," he said.
Brooks also wasn't especially distressed over the price premium for solid-state drives, since the cost was comparable to hard disk drives "once you look at all the components that are needed to run a disk subsystem." The wear-out factor associated with the erase/program process in NAND flash wasn't overly troubling to Brooks, since each of the RamSan-20s builds in an extra 220 GB of raw capacity and other wear-leveling mechanisms.
"Just like anything, we know it's not going to last forever, but I can't even think of the number of times that I had to replace an actual physical hard drive," said Brooks. He added, "Honestly, things have been rather trouble-free since we implemented them. We really haven't had any problems."
This was first published in March 2010