(Be sure to sign up for the upcoming virtual Ignite Community event on July 20, 2021, at 9:00 a.m. PST. Val will share a summary of changes planned in Apache Ignite 3 at this event. Community members will have an opportunity to share their feedback and discuss the proposed changes.)
Apache Ignite 3 is an ongoing, active project that focuses on creating the next generation of the Apache Ignite database. The goal is to retain the power and flexibility of the Ignite 2 platform, modernize the architecture, and significantly improve usability.
I described the motivation for the project and the initial set of updates in one of my previous blog posts: Ignite 3 Alpha: A Sneak Peek into the Future of Apache Ignite. Key enhancements planned in Ignite 3 include dynamic parameter updates, a schema-driven architecture, and a unified CLI tool.
The Apache Ignite community is approaching the development of Ignite 3 iteratively, providing sets of features step by step. At the beginning of 2021, the first alpha version was released. This version featured significant changes in the installation procedure, the new CLI tool, and the new configuration engine.
Meet Alpha 2!
At the end of June, Ignite 3 reached its second milestone—the release of the second alpha version. In this version, the following features were added.
Data Replication Infrastructure Based on Raft
Ignite 2 relies on multiple mechanisms and protocols to replicate data consistently. While this approach does not have an impact on Ignite application developers, Ignite contributors and committers have to put an extra effort to maintain the internal code over time. One of the goals of Ignite 3 is to create a unified data replication infrastructure that is capable of consistently replicating a state across the cluster while being easier to maintain. As in Ignite 2, replication in Ignite 3 is designed to work for both internal and user data. See the proposal for technical details: IEP-61: Common Replication Infrastructure
Alpha 2 introduces an infrastructure implementation that is based on Raft, an industry-standard consensus protocol. This component provides a unified mechanism for data replication that any module of the Ignite platform can use.
Modernized Cluster Discovery and Membership Protocols
The first component built with the new replication infrastructure is metastorage—an internal container responsible for distributing internal data across the nodes of a cluster. The data that is distributed via metastorage include configuration parameters, table schema descriptions, and so on.
Alpha 2 combines metastorage with the SWIM protocol to implement modern mechanisms for cluster discovery and membership. The new implementation in Ignite 3 is more resilient, and provides built-in support for split-brain protection at the discovery component level, while Ignite 2’s split-brain protection is implemented via a separate plugin.
Schema Management and Table API
Another goal of Ignite 3 is to simplify the way users work with data schemas. The goal is to have a strict one-to-one mapping between a table and its schema, thus guaranteeing consistency between the table and the schema. This Ignite proposal provides additional information: IEP-54: Schema-first Approach
With Alpha 2, you can create a table with a schema that you need and then use the Table API to insert and fetch records. This webpage identifies code examples that provide a good idea of how the process works: https://github.com/apache/ignite-3/tree/3.0.0-alpha2/examples.
Under the Table API is the basic in-memory atomic storage. For data distribution, the storage uses the replication infrastructure that was described earlier and the rendezvous hashing algorithm.
What’s Next?
Ignite 3 development continues. Currently under development are features like data rebalancing, persistence, and a SQL engine that is based on Apache Calcite. We expect to release Alpha 3 in a few months. So, stay tuned!
In the meantime, please feel free to play with Alpha 2. The best way to begin to play is to go to the Getting Started Guide. The guide describes all the steps required to install Ignite 3, use the CLI tool, create a cluster, and run examples.
For a quick demonstration of the Alpha 2 build, view the video: Ignite 3 Alpha 2: Introduction and Demo
If you have questions about or ideas or wishes for the Alpha 2 release or if you notice an issue, please do not hesitate to share your thoughts in the dedicated mailing list thread. The Apache Ignite community will contact you as soon as possible and incorporate your thoughts into their development process.