3D XPoint: Four Ways It Will Disrupt HPC

By in Industry Trends on November 30, 2016


3D XPoint is a dramatic new memory technology developed jointly by Intel and Micron. Intel claims that 3D XPoint is 1,000 times faster than NAND, has 1,000 times its endurance, and is 10 times denser than DRAM. Thus, this non-volatile computer storage medium could significantly disrupt current HPC technologies.

This technology is more than just a pending development. 3D XPoint is real, and nearing production delivery. Intel CEO Brian Krzanich stated in a recent conference call:

3D NAND production at the Dalian factory is ramping ahead of schedule with yields matching those of our other production facilities. We continue to see industry enthusiasm building for our groundbreaking new memory technology 3D XPoint, and we’re making steady progress towards bringing it into production.

3D XPoint in SSDs

3D XPoint will make its market debut inside SSDs (solid state drives) in late 2016. This is an astonishing new development because the last new memory technology to reach volume production – flash memory – happened almost 30 years ago. SSDs are used as write-through or write-back caches for hard disk drives (HDDs) to increase the HDD’s latency and bandwidth. But where NAND latency is measured in 10s of microseconds, 3D XPoint latency is measured in 10s of nanoseconds. When Intel’s OPTANE SSDs go to market as its first 3D XPoint products, they are bound to attract significant interest and attention from buyers at all levels, particularly in the enterprise and service provider segments.

Rick Coulson, Senior Fellow at Intel’s NVM Solutions Group, claims the 3D XPoint-based SSDs will deliver approximately seven times more IOPS than current SSDs. This represents an

rick-coulsonextremely welcome performance boost advance. Given a 7x multiplier is being claimed, why is 1,000x going unrealized?

Coulson noted that most of the block storage stack was designed with HDDs in mind. He said that while media latency is 1,000 times faster when 3D XPoint is included, numerous levels in the system architecture remain slow, primarily owing to HDD latency conceptually embedded into I/O software.

To achieve faster speeds, Coulson listed the following software elements in need of changes:

1. Controller ASIC, (ECC)
2. Controller Firmware (indirect system)
3. Platform link, transfer, and protocol components (SATA, SAS, PCIe)
4. Drivers
5. Storage Stack
6. File System

Coulson stated that system architectures would need modification to produce further speedups. Further, he advocates for a change from a block storage model to a persistent memory model. Given the scope of changes needed, he forecast that these system architecture modifications could take at least a decade to complete. To fully exploit the advantages of 3D XPoint, he said, it is best used as memory.

3D XPoint as DIMMs

DIMMs, or Dual In-line Memory Modules, are the logical next step in volume production for 3D XPoint. Such 3D XPoint DIMMs can replace current DIMMs without hardware or software changes. The driver guarantees that 3D XPoint DIMMs won’t wear out. DIMMs make up the current memory for supercomputer nodes (for example, Texas Advanced Computing Center’s Stampede supercomputer). But these DIMMs are volatile – that is, when the power goes off all information in those DIMMs vanishes.

Because 3D XPoint is not only 10 times denser than DRAM but also cheaper, we can expect a node on a supercomputer with 64GB of DIMMs to jump to 640GB. This will benefit applications that use large in-memory databases. Note also that while 3D XPoint is slightly slower than DRAM, DRAM may be built into next-generation DIMMs as a write-through or a write-back cache for 3D XPoint memory. Because 3D XPoint memory is non-volatile, the DIMM controller can turn off sections of unused memory, thereby reducing heat and power consumption.

3D XPoint as HDDs

Hard disk drives (HDDs) have always been at odds with HPC. For speed, moving electrons instead of spinning disks, and switching a transistor instead of moving a disk head, are much preferable.

But HDDs, with their rotating platters (for example, 7,200 revolutions per minute) and the actuator arm’s head which floats perilously close to the platter (10s of nanometers), are slow, big, and subject to head crashes and data loss. HDDs are clunky! Their only saving grace is large and growing storage capacity and low cost, as compared to other storage technologies.

If 3D XPoint can approximate the storage capacity and price of HDDs, this spells doom for HDDs. That’s because 3D XPoint offers more bandwidth and lower latency than HDDs, in a smaller, more energy-efficient package.

3D XPoint and Restart Files

Restart files have been a persistent and nagging problem in HPC. Consider an application running on hundreds of nodes on a supercomputer with working memory of several terabytes or more. Periodically, that application needs to dump a restart file to disk. Then, if one of the application’s nodes fails, it can start over from the latest restart file and proceed. The lower the frequency of restart files, the more work lost when a node fails. The higher that frequency, the harder the file system must work to read and store restart files. Worse still, an application needs to halt calculation to dump a restart file, which inevitably serializes its processing work.

How can 3D XPoint help? With the larger memory inherent to 3D XPoint, an application’s database may be stored in persistent form. Thus, calculations can proceed and restart files dumped simultaneously. Because this involves a speedy memory-to-memory transfer, the calculations and dumping of the restart file occurs in parallel and results in faster job completion overall.


Numerous significant changes to HPC should be coming, based on Intel and Micron’s new 3D XPoint persistent memory technology. This technology will make its debut late in 2016. The following formats for 3D XPoint are anticipated:

• SSDs, used as fast caches for HDDs, will have lower latency and higher bandwidth with 3D XPoint.

• DIMMs, currently the main memory for supercomputer nodes, will become larger and cheaper using 3D XPoint, and 3D XPoint DIMMs are drop-in replacements for current DIMMs.

• HDDs, which currently serve as persistent memory for supercomputer nodes, could become obsolete if 3D XPoint technology matches their storage capacity and price.

• Finally, 3D XPoint DIMMs’ larger and persistent memory offers a perfect solution to the HPC problem posed by restart files

For a conversation about how 3D XPoint could be deployed for the benefit of your own efforts, reach out.

Sign Up for Our Newsletter

Subscribe to Our Newsletter

Keep connected—subscribe to our blog by email.

Back to Top

All rights reserved. Copyright 2020 RedLine Performance Solutions, LLC.