org.apache.ignite.ml.dataset.impl.cache.util

Class ComputeUtils

java.lang.Object

org.apache.ignite.ml.dataset.impl.cache.util.ComputeUtils

public class ComputeUtils
extends Object

Util class that provides common methods to perform computations on top of the Ignite Compute Grid.

Constructor Summary

Constructors

Constructor and Description
ComputeUtils()

Method Summary

Modifier and Type	Method and Description
static <R> Collection<R>	affinityCallWithRetries(Ignite ignite, Collection<String> cacheNames, IgniteFunction<Integer,R> fun, int retries) Calls the specified fun function on all partitions so that is't guaranteed that partitions with the same index of all specified caches will be placed on the same node and will not be moved before computation is finished.
static <R> Collection<R>	affinityCallWithRetries(Ignite ignite, Collection<String> cacheNames, IgniteFunction<Integer,R> fun, int retries, int interval) Calls the specified fun function on all partitions so that is't guaranteed that partitions with the same index of all specified caches will be placed on the same node and will not be moved before computation is finished.
static <C extends Serializable> C	getContext(Ignite ignite, String datasetCacheName, int part) Extracts partition context from the Ignite Cache.
static <K,V,C extends Serializable,D extends AutoCloseable> D	getData(Ignite ignite, String upstreamCacheName, IgniteBiPredicate<K,V> filter, String datasetCacheName, UUID datasetId, int part, PartitionDataBuilder<K,V,C,D> partDataBuilder) Extracts partition data from the local storage, if it's not found in local storage recovers this data from a partition upstream and context.
static <K,V,C extends Serializable> void	initContext(Ignite ignite, String upstreamCacheName, IgniteBiPredicate<K,V> filter, String datasetCacheName, PartitionContextBuilder<K,V,C> ctxBuilder, int retries) Initializes partition context by loading it from a partition upstream.
static <K,V,C extends Serializable> void	initContext(Ignite ignite, String upstreamCacheName, IgniteBiPredicate<K,V> filter, String datasetCacheName, PartitionContextBuilder<K,V,C> ctxBuilder, int retries, int interval) Initializes partition context by loading it from a partition upstream.
static void	removeData(Ignite ignite, UUID datasetId) Remove data from local cache by Dataset ID.
static <C extends Serializable> void	saveContext(Ignite ignite, String datasetCacheName, int part, C ctx) Saves the specified partition context into the Ignite Cache.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ComputeUtils

public ComputeUtils()

Method Detail

affinityCallWithRetries

public static <R> Collection<R> affinityCallWithRetries(Ignite ignite,
                                                        Collection<String> cacheNames,
                                                        IgniteFunction<Integer,R> fun,
                                                        int retries,
                                                        int interval)

Calls the specified fun function on all partitions so that is't guaranteed that partitions with the same index of all specified caches will be placed on the same node and will not be moved before computation is finished. If partitions are placed on different nodes then call will be retried, but not more than retries times with interval interval specified in milliseconds.

Type Parameters:

R - Type of a result.

Parameters:

ignite - Ignite instance.

cacheNames - Collection of cache names.

fun - Function to be applied on all partitions.

retries - Number of retries for the case when one of partitions not found on the node.

interval - Interval of retries for the case when one of partitions not found on the node.

Returns:

Collection of results.

affinityCallWithRetries

public static <R> Collection<R> affinityCallWithRetries(Ignite ignite,
                                                        Collection<String> cacheNames,
                                                        IgniteFunction<Integer,R> fun,
                                                        int retries)

Calls the specified fun function on all partitions so that is't guaranteed that partitions with the same index of all specified caches will be placed on the same node and will not be moved before computation is finished. If partitions are placed on different nodes then call will be retried, but not more than retries times.

Type Parameters:

R - Type of a result.

Parameters:

ignite - Ignite instance.

cacheNames - Collection of cache names.

fun - Function to be applied on all partitions.

retries - Number of retries for the case when one of partitions not found on the node.

Returns:

Collection of results.

getData

public static <K,V,C extends Serializable,D extends AutoCloseable> D getData(Ignite ignite,
                                                                             String upstreamCacheName,
                                                                             IgniteBiPredicate<K,V> filter,
                                                                             String datasetCacheName,
                                                                             UUID datasetId,
                                                                             int part,
                                                                             PartitionDataBuilder<K,V,C,D> partDataBuilder)

Extracts partition data from the local storage, if it's not found in local storage recovers this data from a partition upstream and context. Be aware that this method should be called from the node where partition is placed.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

C - Type of a partition context.

D - Type of a partition data.

Parameters:

ignite - Ignite instance.

upstreamCacheName - Name of an upstream cache.

filter - Filter for upstream data.

datasetCacheName - Name of a partition context cache.

datasetId - Dataset ID.

part - Partition index.

partDataBuilder - Partition data builder.

Returns:

Partition data.

removeData

public static void removeData(Ignite ignite,
                              UUID datasetId)

Remove data from local cache by Dataset ID.

Parameters:

ignite - Ingnite instance.

datasetId - Dataset ID.

initContext

public static <K,V,C extends Serializable> void initContext(Ignite ignite,
                                                            String upstreamCacheName,
                                                            IgniteBiPredicate<K,V> filter,
                                                            String datasetCacheName,
                                                            PartitionContextBuilder<K,V,C> ctxBuilder,
                                                            int retries,
                                                            int interval)

Initializes partition context by loading it from a partition upstream.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

C - Type of a partition context.

Parameters:

ignite - Ignite instance.

upstreamCacheName - Name of an upstream cache.

filter - Filter for upstream data.

datasetCacheName - Name of a partition context cache.

ctxBuilder - Partition context builder.

initContext

public static <K,V,C extends Serializable> void initContext(Ignite ignite,
                                                            String upstreamCacheName,
                                                            IgniteBiPredicate<K,V> filter,
                                                            String datasetCacheName,
                                                            PartitionContextBuilder<K,V,C> ctxBuilder,
                                                            int retries)

Initializes partition context by loading it from a partition upstream.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

C - Type of a partition context.

Parameters:

ignite - Ignite instance.

upstreamCacheName - Name of an upstream cache.

filter - Filter for upstream data.

datasetCacheName - Name of a partition context cache.

ctxBuilder - Partition context builder.

retries - Number of retries for the case when one of partitions not found on the node.

getContext

public static <C extends Serializable> C getContext(Ignite ignite,
                                                    String datasetCacheName,
                                                    int part)

Extracts partition context from the Ignite Cache.

Type Parameters:

C - Type of a partition context.

Parameters:

ignite - Ignite instance.

datasetCacheName - Dataset cache names.

part - Partition index.

Returns:

Partition context.

saveContext

public static <C extends Serializable> void saveContext(Ignite ignite,
                                                        String datasetCacheName,
                                                        int part,
                                                        C ctx)

Saves the specified partition context into the Ignite Cache.

Type Parameters:

C - Type of a partition context.

Parameters:

ignite - Ignite instance.

datasetCacheName - Dataset cache name.

part - Partition index.