Apache Ignite 3.0 Alpha 3 | Overview of MAJOR New Features

Hi there. My name is Val. I am Apache Ignite committer and a product manager at GridGain. I have really exciting news to share with you. Just a couple of days ago the Apache Ignite community released the third alpha version of Ignite 3.

If you haven't heard about Ignite 3 before, this is essentially the upcoming next generation of the Apache Ignite database. I've talked about the first two alpha releases in the previous videos and I highly recommend to check them out if you haven't seen them yet, you will find the links in the description.

Now let's talk about Alpha 3 and the features that has been included in this release. And the most prominent features are: new SQL engine based on Apache Calcite, new Unified Client Protocol, LSM tree storage which is based on RocksDB, and finally Raft-based data rebalancing mechanism. Let's go through all those features in a little bit more detail.

All right, so let's talk about the SQL engine first. The current version of Ignite, which is 2.x, uses the SQL engine which is based on H2 database. And while it's a very powerful and feature-rich engine, it's mostly optimized for operational workloads, which essentially forces the community to put extra effort into supporting various analytical capabilities.

And that's where Apache Calcite comes into play. Essentially, this is a framework that is specifically designed to act as a foundation for a high-performance database that can work with hybrid workloads, both transactional and analytical. And this makes Calcite the perfect fit for the architecture of Ignite.

So Apache Ignite 3 includes the first version of this SQL engine which is based on Apache Calcite. It is also coming with a JDBC driver which you can use to connect to an Ignite 3 cluster and execute some SQL queries.

The next version I wanted to talk about is the Unified Client Protocol. One of the goals of Ignite 3 is to unify client APIs and get rid of this ambiguity of thick and thin clients that currently exists in Ignite 2.x.

So Alpha 3 takes the first step towards this direction by providing the first version of the brand new client protocol along with implementations in Java and .NET. In the future, this protocol, of course, will provide all the APIs that we want to include in Ignite 3, and it will also be easy to implement this protocol for virtually any platform or any language.

All right, so the next feature is the LSM tree storage. As I'm sure you know, Ignite 2.x currently uses B+Tree data structures for persisting data on disks. And you don't have to worry, this implementation is not going anywhere. We have plans to port it into Ignite 3 in the next few months.

However, with Ignite 3, we can go even further. This project kind of rigorously follows the modular approach which basically allows us to make the storage pluggable and add the capability to plug in different implementation of the storage based on different requirements that you might have.

Just as an example, the B+Tree that I've mentioned before is the great fit for read-intensive workloads. But at the same time, you might prefer using LSM trees if your application is more write-intensive, as LSM trees are likely to provide better performance in that case. And Alpha 3 implements this pluggability in the first place and also adds an LSM tree implementation which is based on RocksDB.

This approach that we are taking here will essentially allow users to mix and match different types of storage for different data sets within a single cluster, which is yet another step towards a platform that is capable of processing hybrid workloads.

All right, so and the final feature that I would like to mention is Raft-based data rebalancing. Raft-based replication infrastructure is the backbone of Ignite 3 architecture. It is used or will be used for pretty much every component in the system: data synchronization, transactional processing, SQL query execution, computations, and so on.

Alpha 3 basically adds one more piece into this puzzle by introducing the rebalancing support which now also fully relies on Raft and specifically on the Raft snapshotting mechanisms.

So these are the most important advancements introduced by Alpha 3. I would certainly encourage you to download the release and try it on your own. The best way to do this would be to just go to the Apache Ignite documentation for the 3.0 Alpha version, go through the Getting Started Guide and also check out the built-in examples that are shipped with the release.

All right, so this is it for today. But Ignite 3 journey, of course, continues. The community is working on many different improvements in SQL engine, storage engine, rebalancing mechanisms, baseline topology management and many other parts of the system.

Just as a teaser, one of the biggest features coming up is the significantly improved transactional support which will introduce new fully asynchronous API for explicit transactions. Really exciting stuff. This will be released in the next alpha version which will be out most likely in the next three, four months. So stay tuned for updates. And for now, have a great day.