GridGain Developers Hub

GridGain 8.9.19 Release Notes

Overview

GridGain 8.9.19 is a release bringing a number of bug fixes and improved management options.

New Features

New Partition Reconciliation Options

In this release, new reconciliation options are introduced. Two new options were added to handle specific cases during partition reconciliation:

  • latest_trust_missing_primary option uses the latest value from the primary partition, including any NULL values. This option may be useful when LATEST option cannot be used, as the field is nullable.

  • latest_skip_missing_primary - picks the latest value, skipping reconciliation for any keys that are missing in the primary partition.

You can use these options in the same way you would use other options:

control.sh --cache partition_reconciliation myCache --repair latest_trust_missing_primary

Improved Priority Management During FST

This release provides a new way to manage your full state transfer (FST) batch priority when using non-incremental DR, or when the GG_INCREMENTAL_DR_USE_REGULAR_STORE parameter is enabled. When using incremental DR, these options have no effect.

Priority management is required for high load scenarios, as when regular store is used, there are updates for both current load being synchronized between the data centers, and FST load sequentially copying data.

The following options are available:

  • FST_LAST - regular DR batches have priority over FST batches. This policy is used as default.

  • FST_FIRST FST batches have priority over regular DR batches.

  • ROUND_ROBIN - Regular and FST batches have the same priority and are resolved via round-robin scheduling.

  • FAIR - Regular and FST batches priority is resolved according to fair algorithm.

You can change the priority during runtime by setting the dr.sender.store_scan_policy option from the control script:

$GG_HOME/bin/control.sh --property set --name 'dr.sender.store_scan_policy' --val ROUND_ROBIN

Improvements and Fixed Issues

Community Edition Changes

Issue ID Category Description

GG-42613

Examples

Fixed NPE in log on node shutdown if TcpDiscoveryMulticastIpFinder was enabled.

GG-42312

General

The ignite-tensorflow and ignite-ml-tensorflow-model-parser modules have been removed.

GG-41917

Cluster Storage Engine

Fixed occasional binary recovery errors in long running clusters.

Enterprise Edition Changes

Issue ID Category Description

GG-42988

General

Updated Apache Hadoop’s transitive dependency Netty from version 4.1.100.Final to version 4.1.119.Final.

GG-42981

General

Updated Snowflake JDBC Driver dependency from version 3.22.0 to version 3.23.1.

GG-42980

General

Updated Apache Commons VFS dependency from version 2.7.0 to version 2.10.0.

GG-42905

General

Updated Apache Parquet dependency from version 1.13.1 to version 1.15.1.

GG-42718

General

Deprecated modules and some duplicates were removed from docker images, docker image size decreased as a result.

GG-42681

General

Docker images now include gcompat library.

GG-42605

General

Updated snowflake-jdbc from version 2.13.30 to version 3.22.0.

GG-42114

Cluster Data Replication

Improved a message about tombstones being cleared before they are replicated. This message is now an error.

GG-41944

Cluster Control Script

Added support for new algorithms for resolving conflicts that can be used by the partition reconciliation tool.

GG-40826

Cluster Data Replication

Introduced storage scan policies for DR senders.

Ultimate Edition Changes

Issue ID Category Description

GG-42952

Cluster Data Snapshots and Recovery

Fixed a NPE that could happen while restoring snapshot that contains empty cache while DR is enabled.

Installation and Upgrade Information

See the Rolling Upgrades page for information about how to perform automated upgrades and for details about version compatibility.

Migrating from GridGain 8.9.17 or Earlier With Java 8

Starting GridGain 8.9.18, GridGain uses Lucene 9 by default. As Lucene 9 requires Java 11, it will be impossible to run GridGain on Java 8 with default configuration.

To continue using Java 8 when migrating to GridGain 8.9.18 or later, replace the {GRIDGAIN_HOME}/libs/ignite-lucene-9 folder with {GRIDGAIN_HOME}/libs/optional/ignite-lucene-8.

Migrating From GridGain 8.9.0

GridGain 8.9.1 introduced a large number of changes in default configuration values. Your setup may be affected if you are using default configuration.

When migrating to GridGain 8.9.19, make sure to check Changed Default Values in GridGain 8.9.1 and Later section for any parameters you need to change.

Migrating From GridGain 8.X

When migrating from GridGain 8.8 to GridGain 8.9, no special actions are required for core functionality migration. By using rolling upgrades, you can update from any GridGain version listed below. You may need to perform minor configuration changes to ensure stability post migration.

  • If you are using GridGain Enterprise or Ultimate, the COPY command was reworked as described in the [COPY Command Changes] section. The syntax for working with CSV remains the same, but the path is now calculated on the node and the client. This change does not affect GridGain Community edition.

    To disable this behavior and use the old copy command, pass the disabledFeatures=SERVER_BULK_LOAD parameter in the JDBC connection command:

    jdbc:ignite:thin://127.0.0.1/?disabledFeatures=SERVER_BULK_LOAD
  • If you are using one of the optional modules listed in the [Removed Modules] section, they will continue to work, but compatibility and stability on GridGain 8.9 is not guaranteed. Consider using the alternatives:

  • If you are using default values in your configuration, you may need to set them manually to keep you cluster working the same way as in previous versions. Make sure to check Changed Default Values in GridGain 8.9.1 and Later section for any parameters you need to change.

Older GridGain Versions Compatibility

Below is a list of versions that are compatible with the current version. You can rolling-upgrade from any of those. Compatibility with other versions is not guaranteed. If you are on a version that is not listed, contact GridGain for information on upgrade options.

8.8.4, 8.8.13, 8.8.18, 8.8.22, 8.8.24, 8.8.27, 8.8.30, 8.8.37, 8.8.39, 8.8.43, 8.8.44, 8.9.0, 8.9.1, 8.9.5, 8.9.9, 8.9.12, 8.9.13, 8.9.14, 8.9.15, 8.9.16, 8.9.16-p1, 8.9.17, 8.9.18

Apache Ignite Versions Compatibility

Below is a list of versions that are tested for basic compatibility with the current version. If you are on a version that is not listed, contact GridGain for information on upgrade options.

2.11.1, 2.12.0, 2.13.0, 2.14.0, 2.15.0

Known Limitations

Java 8 Support in GridGain 8.9.18 and Later

Starting GridGain 8.9.18, GridGain uses Lucene 9 by default. As Lucene 9 requires Java 11, it will be impossible to run GridGain on Java 8 with default configuration.

We recommend switching to Java 11 or later before updating to GridGain 8.9.18 or later.

To continue using Java 8, replace the {GRIDGAIN_HOME}/libs/ignite-lucene-9 folder with {GRIDGAIN_HOME}/libs/optional/ignite-lucene-8.

Unsupported Features When Using Java 8 in GridGain 8.9.17

In GridGain 8.9.17, full text search is not supported when GridGain is running on Java 8. The new vector search feature requires using Java 11.

Additionally, GridGain 8.9.16 or earlier automatically created full text indexes for caches with string values, which may cause issues for working with the cache while using Java 8. To avoid this behavior, set the sql.disableCreateLuceneIndexForStringValueType property to true. If such index is already built for a cache with persistence, contact our support team for assistance with removing it.

We recommend switching to Java 11 or later before updating to GridGain 8.9.17 if you plan to use these features.

Changed Permissions for Control Center Agent in GridGain 8.9.4 and Later

GridGain 8.9.4 introduced stricter permissions checking for Control Center Agent. When running secured clusters, some actions that previously were available without permissions, will now require additional permissions.

Before updating, make sure to provide the required permissions, otherwise some actions may become unavailable.

New TRACING_CONFIGURATION_UPDATE Permission in GridGain 8.9.2 and Later

GridGain 8.9.2 introduced the TRACING_CONFIGURATION_UPDATE permission. If you were using security on a cluster before updating to this version, make sure that you provide the permissions to control utilities or users who need to update cluster permissions. Otherwise, they will not be able to update

Changed Default Values in GridGain 8.9.1 and Later

If you are updating from GridGain 8.8.X or 8.9.0, a large number of default values have been changed.

These changes may affect the stability or performance of your cluster if you are using default values.

We recommend checking the list below to make sure the changes do not have an adverse effect, and setting the value manually if necessary.

  • Service processor now use event-driven implementation by default. You can keep the old behavior by setting the IGNITE_EVENT_DRIVEN_SERVICE_PROCESSOR_ENABLE property to false. Make sure that all nodes in the cluster are set to the same value. See Services documentation for more information.

  • Data region metrics are now enabled by default. This may have a minor (within 2%) adverse effect on performance. You can keep the old behavior by setting the DataRegionConfiguration.metricsEnabled value to false.

  • Data storage metrics are now enabled by default. This may have a minor (within 2%) adverse effect on performance. You can keep the old behavior by setting the DataStorageConfiguration.metricsEnabled value to false.

  • TcpCommunicationSpi communication protocol now has a limit of 4096 messages for the outgoing messages queue by default.

  • Atomic operations are now not allowed in transactions by default. You can keep the old behavior by setting the IGNITE_ALLOW_ATOMIC_OPS_IN_TX value to true.

  • Logs are now in verbose mode by default. You can use the -q command line argument in the ignite.sh script to keep current log behavior.

  • SQL queries are now loaded lazily by default. This reduces memory consumption on medium and large queries and potentially improves garbage collection performance. You can keep the old behavior by setting the SqlFieldsQuery.setLazy(false).

  • Cache entries are now read from primary partitions by default, even if an entry is available on the node in a backup partition. This may have a minor performance impact, but significantly increases cluster stability. You can keep the old behavior by setting the readFromBackup property to true.

  • Partition map exchange transactions now time out after 1 minute instead of 0 (infinite) by default. You can keep the old timeout by setting the TX_TIMEOUT_ON_PARTITION_MAP_EXCHANGE setting to 0.

  • Data streamer now overwrites entries by default. You can keep the old behavior by setting the stmr.allowOverwrite property to false.

  • TCP discovery now uses static IP finder TcpDiscoveryVmIpFinder by default. To keep the old behavior, set the IP finder to use multicast IP finder.

  • Checkpointing process now starts upon reaching 75% of minWalArchiveSize instead of 25%. You can keep the old behavior by setting the IGNITE_CHECKPOINT_TRIGGER_ARCHIVE_SIZE_PERCENTAGE system variable to 0.25.