In this post I'll demonstrate how to improve a database's random write workloads by replacing regular SSDs with Intel® Optane™ SSDs powered by cutting-edge 3D XPoint™ technology.
What might be wrong with sustained random write workloads on good-old SSDs? Regular SSDs have to perform garbage collection routines endlessly by erasing blocks with stale data. Since any kind of garbage collection inevitably leads to performance degradation, SSDs manufactures came up with an over-provisioning technique that reserves some amount of space for cleaning needs. However, this space is limited and can be exhausted pretty quick depending on a workload.
Thus, it is common to get the top I/O numbers on regular SSDs in the first hours of operation and then suddenly hit a significant performance drop while keeping the workload the same. The picture below shows this exact tendency (click the image to enlarge the view):
Picture 1.
Indeed, the Apache® Ignite™ community reproduced a similar curve for its native persistence -- witnessing how SSDs speeded up to 100 MB/s in first 5 minutes of usage and slowed down to 20 MB/s just in a matter of 2 hours under a sustained random write workload.
Intel Optane SSDs (P4800X Series) to the Rescue
Once Intel Optane SSDs hit the market, Ignite performance geeks ran the same set of benchmarks that put a sustained write workload on the cluster throughout 10 hours. In these benchmarks, Ignite persistence used Intel Optane SSDs instead of regular SSDs. When the benchmarks were over, we were staring at the graph below in (almost) disbelief:
Picture 2. (Click image for enlarged view)
The graph shows that the random write throughput (red curve) did not fall dramatically even within 10 hours of sustained workload. To be more specific, ~20% decline was spotted after 6 hours of execution. That's negligible in comparison to the drops observed on regular SSDs under the same conditions that might be as big as 70%.
Furthermore, the benchmarks generated a sustained random read workload (blue curve) at 2 to 5 and at 7- to 10-hour intervals. The graph suggests that the reads do not notably affect the throughput of the writes.
Generally speaking, the benchmarks assured us that Optane SSDs do not suffer from the garbage collection routines that require spare memory to operate efficiently (over-provisioning). We were told that there is no spare area on an Intel Optane SSD, the media doesn’t require it. Also, Intel engineers shared with us that memory management is far simpler with this media, only a fraction as complex as with NAND-based NVM that everyone else uses. Thus, when it comes to writes, you can expect to see a greater balance of IOPS and overall quality of service performance.
It is worth mentioning that Intel Optane SSDs were mounted as standard disk drives during the benchmarking and accessed using generic Java File I/O APIs. I'm curious to see what the benchmarks will show once Apache Ignite supports Intel Optane SSDs more natively. By the way, there are even more upcoming performance improvements to the Java IO API’s in Java 10 (that is scheduled for March 2018).