GridGain Developers Hub

Network Backups

Overview

GridGain provided a way to create network backups, allowing you to back up your cluster data to a network location such as NAS, NFS or SFTP for recovery purposes. Since the data is stored on a different physical device, network backups provide security against disk failures. The network location is used as a centralized storage where all cluster nodes can save data and then use it for recovery. Using centralized storage also lets you restore the cluster to a different physical location or even in the cloud.

Prerequisites

  • Snapshot functionality has to be enabled. Please refer to Snapshots & Recovery for instructions.

  • NAS or NFS storage: centralized storage has to be mounted to all machines hosting the cluster nodes and have the same path.

  • SFTP location: an SFTP server needs to be accessible from all GridGain cluster nodes. Follow this section to enable an SFTP location for GridGain backups.

Creating Network Backup

There are two ways to create a network backup:

  • Create a local (regular) snapshot and move it to a shared folder. This operation is unsupported for SFTP-based backups.

  • Create a snapshot directly into a shared folder.

These two approaches are discussed below.

Saving Snapshots Directly to Centralized Storage

You can tell GridGain to create a snapshot directly into the centralized storage by specifying a path:

snapshot-utility.sh snapshot -type=full -dest=/shared/folder
// Get a reference to the GridGain plugin.
GridGain gg = ignite.plugin(GridGain.PLUGIN_NAME);

// Get a reference to the Snapshots interface.
GridSnapshot snapshots = gg.snapshot();

// Create the snapshot. Data of all the caches will be added to the snapshot.
SnapshotFuture<Void> snapshotFuture = snapshots.createFullSnapshot(null,
        new File("/shared/folder/path/"), new SnapshotCreateParams(),
        "Snapshot has been created!");

// Wait until the snapshot is created.
snapshotFuture.get();
This API is not presently available for C++.

Moving Snapshots to Centralized Storage

If you do not specify a snapshot location when creating a snapshot, each node will save its data to a local folder. In this case, the snapshot is stored on local drives of the cluster nodes. To move the data to a centralized storage, use either the move command of the Snapshots Management Tool or the Java API.

# Example with shared folder such as NFS or NAS
snapshot-utility.sh move -id=123456 -dest=/shared/folder

# Example with an SFTP location that requires user credentials
snapshot-utility.sh move -id=123456 -dest=sftp://login:password@address:port/relativePath

# Example with an SFTP location that requires a key alias
snapshot-utility.sh move -id=123456 -dest=sftp://login@address:port/relativePath -key_alias=sftpKeyAlias
// Get a reference to GridGain plugin.
GridGain gg = ignite.plugin(GridGain.PLUGIN_NAME);

// Get a reference to the Snapshots.
GridSnapshot snapshot = gg.snapshot();

// Get the first snapshot from the list.
SnapshotInfo info = snapshot.list().get(0);

// Get the snapshot ID.
long snapshotId = info.snapshotId();

snapshot.moveSnapshot(snapshotId,
    new File("/shared/folder/path"), true, "Snapshot Moved");

SnapshotPath destination = sftpDestination
    ? SnapshotPath.sftp().uri(new URI("sftp://login@address:port/relativePath"))
    .keyAlias("keyAlias")
    .build()
    : SnapshotPath.file().path(new File("/shared/folder/path")).build();

// Move the snapshot with given ID to a shared folder.
SnapshotFuture<Void> future = snapshot.move(
    new MoveSnapshotParams()
        .snapshotId(snapshotId)
        .destinationPath(destination)
        .skipWalMove(true)
        .singleFileCopy(true)
).get()


    .move(snapshotId,
        new File("/shared/folder/path"), true, "Snapshot Moved");

// Wait for the operation to finish.
future.get();
var ignite = Ignition.Start(cfg);

// Get a reference to grid snapshot API.
var snapshot = ignite.GetSnapshot();

// Get the first snapshot from the list.
var en = snapshot.GetSnapshots(null).GetEnumerator();
en.MoveNext();
var info = en.Current;

// Get the snapshot ID.
long snapshotId = info.SnapshotId;

// Move the snapshot with given ID to a shared folder.
var task = snapshot.MoveSnapshotAsync(snapshotId, "/shared/folder/path", "Snapshot Moved");

// Wait for the operation to finish.
task.Task.Wait();
This API is not presently available for C++.

Restoring from Network Backup

To restore a cluster from a network backup, use the centralized storage as a snapshot location:

# Example with shared folder
snapshot-utility.sh restore -id=123456 -src=/shared/folder

# Example with sftp server with login-password auth
snapshot-utility.sh restore -id=123456 -src=sftp://login:password@address:port/relativePath

# Example with sftp server with login-key auth
snapshot-utility.sh restore -id=123456 -src=sftp://login@address:port/relativePath -key_alias=sftpKeyAlias
// Get a reference to GridGain plugin.
GridGain gg = ignite.plugin(GridGain.PLUGIN_NAME);

// Get a reference to GridSnapshot.
GridSnapshot storage = gg.snapshot();

// Get the first snapshot from the list.
SnapshotInfo info = storage.listSnapshots(null).get(0);

// Get the snapshot ID.
long snapshotId = info.snapshotId();

SnapshotPath location = sftpLocation
    ? SnapshotPath.sftp().uri(new URI("sftp://login@address:port/relativePath"))
    .keyAlias("keyAlias")
    .build()
    : SnapshotPath.file().path(new File("/shared/folder/path")).build();

// Replace content of all the caches with the content from the snapshot.
SnapshotFuture<Void> future = storage
    .restore(new RestoreSnapshotParams()
        .snapshotId(snapshotId)
        .optionalSearchPaths(Arrays.asList(location))
        .message("Cluster Restored!")
    );

// Wait until the operation finishes.
future.get();
var ignite = Ignition.Start(cfg);

// Get a reference to grid snapshot API.
var snapshot = ignite.GetSnapshot();

// Get the first snapshot from the list.
var en = snapshot.GetSnapshots(null).GetEnumerator();
en.MoveNext();
var info = en.Current;

// Get the snapshot ID.
long snapshotId = info.SnapshotId;

// Replace content of all the caches with the content from the snapshot.
var task = snapshot.RestoreSnapshotAsync(snapshotId, new[] { "/shared/folder/path" }, null, "Cluster Restored!");

// Wait for the operation to finish.
task.Task.Wait();
This API is not presently available for C++.

Each node will compute which partitions belong to the local node and restore corresponding partitions from the centralized storage to the local storage.

Configuring SFTP Location

To enable an SFTP location for GridGain backups, you need to provide org.gridgain.grid.persistentstore.snapshot.file.SftpConfiguration that includes several connectivity-related settings:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">

   <!-- Enabling the Ignite Native Persistence. -->
  <property name="dataStorageConfiguration">
    <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
        <property name="defaultDataRegionConfiguration">
        <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
          <property name="persistenceEnabled" value="true"/>
        </bean>
      </property>
    </bean>
  </property>

  <!-- Enabling the snapshots. -->
  <property name="pluginConfigurations">
    <bean class="org.gridgain.grid.configuration.GridGainConfiguration">
      <property name="snapshotConfiguration">
        <bean class="org.gridgain.grid.configuration.SnapshotConfiguration">
            <property name="sftpConfiguration">
                <!-- Enabling the SFTP location with keys access. -->
                <bean class="org.gridgain.grid.persistentstore.snapshot.file.SftpConfiguration">
                    <!-- Java KeyStore (JKS) path. -->
                    <property name="keyPath" value="path" />

                    <!-- JKS passphrase(optional). -->
                    <property name="passphrase" value="password" />
                </bean>
            </property>
        </bean>
      </property>
    </bean>
  </property>
</bean>

Follow the steps below to create a Java KeyStore (JKS) with the keys that Ignite can use to authenticate on your SFTP server:

  1. Generate certificates:

    openssl req -x509 -newkey rsa:4096 -keyout myKey.pem -out cert.pem -days 365 -nodes
  2. Convert the certificates to the PKCS12 format:

    openssl pkcs12 -export -out node.p12 -inkey myKey.pem -in cert.pem -name sftp
  3. Add a private key with its alias to the JKS:

    keytool -v -importkeystore -srckeystore node.p12 -srcstoretype PKCS12 -destkeystore example.jks -deststoretype JKS -alias example_alias
  4. (Optional) Create a public key for your SFTP server:

    ssh-keygen -y -f myKey.pem > myKey.pub
  5. Confirm the following files are generated:

    • example.jks - the JKS file with alias=example_alias

    • myKey.pub - a public key to authenticate on the SFTP server