Data Consistency and Replication

GridGain provides strong consistency guarantees for transactions. When a write operation is confirmed, it is durable and immediately visible to subsequent reads. Take into account that the same data may be seen on concurrent clients after some delay due to replication lag and other reasons. However, those clients will always see consistent data without partial updates, and causally dependent transactions will always be consistent with their causal predecessors. Transactions provide isolation, ensuring you never see partial updates from other transactions. The RAFT consensus protocol ensures all replicas of a partition agree on the order of operations, maintaining consistency within partition groups while replication propagates changes across the cluster.

GridGain prioritizes consistency and partition tolerance, which means the system makes specific trade-offs during network issues or node failures. Understanding these trade-offs helps you design resilient applications and plan for failure scenarios.

GridGain’s design ensures you never lose committed data or see inconsistent results. The trade-off is that partitions become unavailable when they cannot maintain consistency guarantees. This design is appropriate when data correctness is critical, losing data is unacceptable, and brief unavailability during failures is acceptable.

If your workload prioritizes availability over consistency - meaning you prefer the system to remain operational even if it might mean losing recent writes - consider using high availability mode. This mode changes the behavior so that when a majority of replicas are lost, the system automatically reconfigures partitions to continue operations with the surviving replicas. See Cluster Fault Tolerance for detailed information about how different failure scenarios affect cluster availability.

GridGain replicates each partition to multiple nodes based on the REPLICAS setting you configure in the distribution zone. When a client sends updated data to a partition’s primary replica, that primary replica coordinates replication to the other replicas. Once a majority of replicas confirm they have received and stored updated data, it is considered committed. The primary replica then responds to the client, while remaining replicas receive updates asynchronously. This process ensures durability - committed data survives node failures as long as a majority of replicas survive.

To ensure only one node coordinates writes to a partition, GridGain uses a lease system managed by the placement driver component. A lease is a time-limited exclusive right to act as the partition’s primary replica. The placement driver runs as part of cluster management and grants time-limited leases to replication group leaders. Only the leaseholder can coordinate write operations for its partition, which prevents conflicting writes during failures.

The placement driver continuously monitors and renews leases for healthy primary replicas. Every 500 milliseconds, it checks which leases are approaching expiration and renews them for primaries that remain healthy. Leases are renewed when they have half their configured duration remaining - with the default 5-second expiration, renewal happens when 2.5 seconds remain. This continuous renewal means a healthy primary’s lease never actually expires during normal operation.

When a primary replica fails, its lease continues to be valid until it expires naturally. The placement driver cannot grant a new lease during this time because the old lease might still allow the failed primary to coordinate writes. Once the lease expires, the placement driver can grant a new lease to a different replica, typically the newly elected RAFT leader.

The time from primary failure to write availability depends on lease expiration plus leader election time. With default settings, lease expiration takes up to 5 seconds, and leader election typically takes between 1 and 11 seconds depending on cluster condition. As a result, write recovery typically completes within 6-16 seconds. Read-only transactions continue unaffected during this window since they can read from any available replica.

Leases have a configured duration (5 seconds by default) and cannot be revoked early - they must expire naturally. You can adjust lease timing through the replication configuration. Shorter lease durations provide faster failover when the primary fails but impose higher coordination overhead from more frequent renewals. Longer lease durations reduce coordination overhead but mean slower failover. The default values provide a good balance for most deployments:

For detailed information about how leases interact with RAFT groups and affect recovery times during different failure scenarios, see Cluster Fault Tolerance.

GridGain handles reads differently depending on the transaction type. Read-write transactions always read from the primary replica (the leaseholder) to ensure they see the latest committed data and that reads reflect writes made within the same transaction. This coordination with the primary replica is necessary because read-write transactions may make modifications that need to interact with other concurrent transactions.

Read-only transactions have more flexibility. They can read from any replica, whether primary or backup, because they read data as of their transaction’s read timestamp without modifying anything. This allows them to run without blocking or interfering with writes, and they can read from local replicas to reduce network traffic. For details on transaction behavior, see Transactions.

You can monitor partition health using the recovery partition states CLI command:

This shows the current state of each partition, helping you identify problems before they impact availability. See Partition States for descriptions of what each state means.

You can also query system views to inspect replication status. The system.zone_partitions view shows partition assignments across the cluster, while system.primary_replicas shows which nodes currently hold primary replica leases:

See System Views for the complete reference on available system views.

When designing your distribution zones and replica strategy, several considerations help ensure you get the consistency and availability characteristics you need.