jules - Fotolia
With storage threatening to become a chokepoint, LexisNexis Risk Solutions this year replaced traditional hard drives on its internally developed Data Analytics Supercomputer with more than 2,600 solid-state drives from Micron Technology.
LexisNexis, based in Alpharetta, Ga., uses high-performance computing to provide big data analytics to financial institutions, government agencies, healthcare firms and retailers. Its storage handles services for risk assessment, fraud detection, identity management and data aggregation, with a premium placed on making data secure yet highly available to end users.
LexisNexis servers comb through massive datasets of public records and other information sources to verify a user's identify. This is done anonymously by cross-checking the person's responses to specific security questions. Anyone who has lost or reset a banking password probably has used LexisNexis' services.
Application complexity drove need for flash storage
The Data Analytics Supercomputer (DAS) was designed about 15 years ago to quickly and accurately analyze petabytes of data. The system includes 1,200 Super Micro Computer Inc. 2U Twin servers, split into 12 100-node clusters. It provides parallel processing of batch-oriented data integration and real-time data analysis.
Each node ingests between 5 TB and 20 TB of new index data each day. The servers run as independent clusters, using N+1 mirroring to replicate data across clusters. Data that resides on one cluster also resides on the 11 other clusters.
LexisNexis historically relied on hard disk drive storage. Most of the time, drives are in read-only mode with high random access. As powerful as the DAS system is, storage performance started to degrade as data sets grew bigger and more complex, said Flavio Villanustre, LexisNexis' vice president of technology.
"For the real-time component, we reached a point where spinning drives no longer delivered the performance we need or expect. We tried solid-state drives from a number of vendors, with varying degrees of success," Villanustre said.
In one example, LexisNexis bought 75 enterprise SSDs from a leading enterprise drive vendor and installed them on its compute-intensive platform. After a month of heavy-duty testing, only 32 drives were still alive. LexisNexis also tested Micron P410m 400 GB enterprise SAS drives, which did not meet its capacity requirements.
"When we deploy data to the real-time iteration, we tend to deploy completely new data, and that happens in the database. The SSDs we were testing would very quickly become unusable. We would see a significant degradation of performance after a few cycles of wiping and rewriting the drives," Villanustre said.
Micron M500 drives supplant HDD cluster for nonregulated data
Following those trials, Burger and Villanustre investigated adding a flash-enabled SAN using the Hitachi Unified Storage VM system. Then, in December 2013, Burger received an envelope in the mail that changed that plan. Tucked inside the envelope, which Burger presumed was a Christmas card, were two 900 MB Micron M500 SSDs, courtesy of Micron senior advanced systems memory architect Richard Murphy.
Burger took the Micron 900 GB drives and replaced two 15,000 RPM hard drives, running the SSDs in full production for about two months. During that time, "we saw a 90% reduction in load though higher utilization on that particular node. We just wanted to know if we were going to burn the drives out."
Further testing convinced Burger and Villanustre the storage improvements were no fluke. That led LexisNexis to purchase 220 M500 drives from Micron and replace every HDD -- and the associated controller hardware -- in a single 100-node cluster to serve database queries. When none of the drives failed after running continuously for 45 days, they decided to purchase another 2,400 M500s and install them on a cluster that stores financial data which isn't subject to regulations of the federal Fair Credit Reporting Act.
The M500 SATA SSDs, originally intended as a client drive, are designed with Micron's 20-nanometer (nm) multilevel cell NAND flash technology and available in 120 GB, 240 GB, 480 GB and 960 GB capacities. The 960 GB drives that LexisNexis bought are rated for sequential read and sequential write performance of 500 megabytes per second (MBps) and 400 MBps, respectively, and 80,000 read/write IOPS. In September, Micron launched the higher-capacity M600 line made with 16-nm flash, which LexisNexis may move to in the future.
Burger lists improved load bearing capacity and accelerated service levels among the early benefits of integrating the M500 drives. The drives are self-encrypted for data at rest using the Opal Storage Specification developed by the Trusted Computing Group. That eliminates the need to encrypt data at the operating system level.
"With spinning disk, our utilization rate was approaching 70% to 72%, which made us uncomfortable," Burger said. "With the new Micron drives, we don't hit more than 6% capacity during peak, and we've been able to double our batch volume. We've gotten some big-name jobs down from 30 hours to eight hours, without having to actually increase the number of threads in a batch."
Better SLA performance, reduced query times
The SSDs have also contributed to better turnaround on service-level agreements and sped query results by 42%. Burger said the $5.5 million savings includes the difference between installing a SAN and purchasing the Micron M500 SSDs. LexisNexis budged $7 million for the Hitachi SAN, compared to $1.5 million spent on the Micron drives. The company plans to reallocate the savings to address other storage needs.
"Adding the Micron SSDs helps us to shore up other environments that we probably wouldn't have been able to this year," he said. "It enables us to bring flash to a larger array of servers. We're very likely going to roll out flash storage to our insurance environment, which is another 1,200 servers, sometime in 2015."
Micron unveils higher endurance SSDs