Ravikanth Durgavajhala is an SD solutions architect focusing on Big Data & AI at Intel Corp. He’ll be speaking at the In-Memory Computing Summit Europe in London on June 25 at 11 a.m.
I connected with him recently about his session, “Expansion of System Memory using Intel Memory Drive Technology,” to share some insights in advance for those attending the conference – as well as for those who cannot make it there. If you’re in London or can make it there, I encourage you to enter our raffle for a £20 ticket (vs. £500+ regular price) Enter the Raffle! If you don't win, fear not. Just use my promo code "100pound" to save £100 (register here).
Allow me to set the stage for our discussion....
Intel Memory Drive Technology (IMDT) is a software-defined memory (SDM) product that allows for the expansion of system memory beyond DRAM by defining some of the PCIe-based Intel Optane SSD capacity as memory, instead of as storage. IMDT executes directly on the hardware and below the operating system, and allows for system memory to be assembled from DRAM and the 3D XPoint-based PCIe-based Intel Optane SSD. It leverages economic benefit of SSDs, and operates transparently as volatile system memory.
According to Intel, IMDT is optimized for up to 8x system memory expansion over installed DRAM capacity and provides ultra-low latencies and close-to DRAM performance for SDM operations. It is designed for high-concurrency and in-memory analytics workloads.
At runtime, part of the DRAM is used as cache. When there is a cache-miss, IMDT intercepts and brings the data into DRAM. IMDT avoids cache misses by using machine learning to continuously prefetch data from the Optane SSDs.
Ravikanth’s talk will be June 25 at 11 a.m. Now that you have a bit of background, let’s ask him a few questions (you are welcome to also ask questions in the comments section and I’ll make sure to send them to Ravikanth so he can address them here, too).
Tom: Combined with Intel Optane SSDs, what benefits does Intel Memory Drive Technology offer?
Ravikanth: Memory space is thought of as static from boot until shutdown; IMDT makes it transient and facilitates better cost for performance for many web/cloud workloads. IMDT provides two usage models – 1) Memory Replacement and 2) Memory Expansion
In case of Memory Replacement, IMDT can provide 25%-65% CAPEX savings by replacing up to 85% of system’s DRAM capacity.
In case of Memory Expansion, IMDT can provide up to 8x increased memory capacity above machine specification.
Both of those usage models could result in reduced overall footprint in the data center. With lower number of nodes and decreased power consumption, IMDT can significantly lower the Total Cost of Ownership.
Tom: What types of workloads benefit the most from Intel Memory Drive Technology?
Ravikanth: There are primarily 3 classes of workloads that can leverage IMDT:
1. Probability-based memory access pattern (using machine-learning for Pre-Fetch)
Workloads accessing structured datasets (such as sequential memory-access, but more complicated ones) can potentially benefit from IMDT. A SQL routine that accesses a database row which is of static size defined by the column data types can potentially be pre-fetched into DRAM from the Optane SSD (where it is actually residing) by IMDT. These are the type of workloads that IMDT can learn from the access pattern and offer closer to DRAM-like performance by leveraging the high bandwidth of Intel Optane SSDs to prefetch the entire data into DRAM and overshadowing the latency associated with SSD access.
2. High concurrency access pattern (using asynchronous memory load technology)
These are the type of workloads that differ from the pre-fetch workloads, because they are random and cannot be easily associated with an access pattern. However, these workloads tend to be highly parallelized, such as key-value web-caching engines. So IMDT can let the CPU work on the next task in the pipeline while it fetches the data for the previous task and have the CPU come back to the task at a later time. This could be beneficial to some workloads because typically in highly parallelized workloads the overall performance of the workload is what matters in the end and not the performance of the individual call, and Optane provides low latency per call.
3. CPU Intensive or memory-locality influenced workloads
These workloads can benefit from IMDT because it can bring the data closer to the right CPU. In multi-core CPUs where Non-Uniform Memory Access (NUMA) could be detrimental to application performance, IMDT can help bring the data closer to the CPU processing it, thus providing the performance advantage.
Tom: What are some examples of applications best suited for Intel Memory Drive Technology?
Ravikanth: Large sequential reads such as OLAP, columnar databases and key-value caches, OLTP workloads, HPC workloads with access patterns that IMDT can learn from, Containers and VMs, Multi-Tenant DBMS, high Throughput workloads
Tom: What about workloads (and applications) that that are not well-suited for Intel Memory Drive Technology?
Ravikanth: Applications that are memory intensive, where memory size is the bottle neck are the ones that can potentially leverage the benefits of IMDT. Applications that are memory bandwidth bound are not an ideal fit for IMDT. Applications that have random access patterns without a lot of parallelization are also not a good fit for IMDT. Having said that, IMDT does not require any changes at the application level. So if a system has Optane SSDs available, IMDT could be installed on them and run the workloads as-is to do performance evaluation. If the application proves not to be a good fit for IMDT, the Optane SSD could just be reused as a regular block device after simply a reboot.
Tom: Can you share a recommended configuration?
Ravikanth: IMDT targets configurations that are expensive or impossible with DRAM. For example, dual-socket Xeons needing 1TB of memory or more, or quad-socket Xeons needing 2TB and more. And by more, I mean a lot more. Sure, IMDT and Optane can give you a cost-effective 2TB of memory on a dual-socket Xeon, but we encourage customers to think outside the box. Consider a quad-socket Xeon with 24TB of memory for an in-memory database, or a dual socket Xeon with 6TB of memory that’s 50% higher than the Amazon and Google “fattest” instances – with amazing economics.