Apache Ignite Transactions Architecture: Transaction Handling at the Level of 3rd Party Persistence

This article is the last part of the Apache Ignite Transactions Architecture series. In the previous articles in this series, we discussed a range of topics associated with Apache Ignite's transactions handling in its Key-Value API.

In the first article, we briefly reviewed the two-phase commit protocol and described how it worked with various types of cluster nodes.

In the second article, we discussed locking modes and isolation levels and provided detailed message flows for the different isolation levels in both pessimistic and optimistic locking. We also briefly reviewed deadlock detection mechanisms.

In the third article, we looked at failover and recovery and discussed how Apache Ignite managed Backup Node failure, Primary Node failure and Transaction coordinator failure.

In the fourth article, we focussed on transaction handling with Ignite's native persistence. In particular, we discussed the Write-Ahead Log (WAL) and Checkpointing.

In this last part, we will focus on how Apache Ignite handles transactions with 3rd party persistence.

Read-Through and Write-Through

Two major benefits that can be achieved with Apache Ignite are scale and performance. And one of the underlying principles behind Ignite is no-rip-and-replace. In other words, it is highly likely that there are existing systems in organizations that serve very useful business purposes and cannot be easily replaced, but could be enhanced with superior scale and performance for many business queries, for example. In this scenario, Ignite can serve as an In-Memory Data Grid (IMDG) or distributed cache for a 3rd party database system enabling read-through (data are loaded from the database into the IMDG when not found in the cache) and write-through (data written to the IMDG are persisted to the database system), as shown in Figure 1.

Figure 1. Read-Through and Write-Through

Figure 1. Read-Through and Write-Through

However, transactions must be handled appropriately, since they could span both Ignite and the 3rd party storage system, and maintaining data consistency in the IMDG and in the database system is very important. To achieve this, Ignite provides a CacheStore interface that is fully transactional for read-through and write-through operations.

Two-Phase Commit

When Ignite uses a 3rd party database system for persistence, the Transaction coordinator will send updates to the 3rd party system first and then send a Commit message to its cluster nodes. Working with transactional database systems for persistence provides certain guarantees. Therefore, in the event of a transaction failure with the database system, the transaction can be rolled back and the cache and database system remain consistent.

Handling Failures

Since the database system is treated as the source of truth, managing failures is significantly easier than other scenarios we have discussed in this article series. Data can always be reloaded from the database system to ensure that the cache and the database are consistent, as shown in Figure 2. This works in all cases, including scenarios where the Transaction coordinator has crashed, potentially with other cluster nodes. Ignite’s CacheStore interface provides bulk loading of a cache from a database system, which can provide a speedy method to restore the cache.

Figure 2. 2-Phase Commit Recovery with Database

Figure 2. Two-Phase Commit Recovery with Database

Summary

Handling transaction failure with 3rd party persistence is actually easier than some of the other scenarios we have discussed in this article series, as updates and changes are applied to the 3rd party store first. This concludes this article series.