org.apache.ignite.ml.dataset.impl.cache

Class CacheBasedDatasetBuilder<K,V>

java.lang.Object

org.apache.ignite.ml.dataset.impl.cache.CacheBasedDatasetBuilder<K,V>

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

All Implemented Interfaces:

DatasetBuilder<K,V>

Direct Known Subclasses:

SqlDatasetBuilder

public class CacheBasedDatasetBuilder<K,V>
extends Object
implements DatasetBuilder<K,V>

A dataset builder that makes CacheBasedDataset. Encapsulate logic of building cache based dataset such as allocation required data structures and initialization of context part of partitions.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_NUMBER_OF_RETRIES Default number of retries for the case when one of partitions not found on the node where loading is performed.

Constructor Summary

Constructors

Constructor and Description
CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache) Constructs a new instance of cache based dataset builder that makes CacheBasedDataset with default predicate that passes all upstream entries to dataset.
CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter) Constructs a new instance of cache based dataset builder that makes CacheBasedDataset.
CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter, UpstreamTransformerBuilder transformerBuilder) Constructs a new instance of cache based dataset builder that makes CacheBasedDataset.
CacheBasedDatasetBuilder(Ignite ignite, IgniteCache<K,V> upstreamCache, IgniteBiPredicate<K,V> filter, UpstreamTransformerBuilder transformerBuilder, Boolean isKeepBinary, int retries) Constructs a new instance of cache based dataset builder that makes CacheBasedDataset.

Method Summary

Modifier and Type	Method and Description
<C extends Serializable,D extends AutoCloseable> CacheBasedDataset<K,V,C,D>	build(LearningEnvironmentBuilder envBuilder, PartitionContextBuilder<K,V,C> partCtxBuilder, PartitionDataBuilder<K,V,C,D> partDataBuilder, LearningEnvironment localLearningEnv) Constructs a new instance of Dataset that includes allocation required data structures and initialization of context part of partitions.
DatasetBuilder<K,V>	withFilter(IgniteBiPredicate<K,V> filterToAdd) Returns new instance of DatasetBuilder using conjunction of internal filter and filterToAdd.
CacheBasedDatasetBuilder<K,V>	withKeepBinary(boolean isKeepBinary) Add keepBinary policy.
CacheBasedDatasetBuilder<K,V>	withRetriesNumber(int retries) Sets number of retries. 15 * 60 by default.
DatasetBuilder<K,V>	withUpstreamTransformer(UpstreamTransformerBuilder builder) Returns new instance of DatasetBuilder with new UpstreamTransformerBuilder added to chain of upstream transformer builders.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_NUMBER_OF_RETRIES

public static final int DEFAULT_NUMBER_OF_RETRIES

Default number of retries for the case when one of partitions not found on the node where loading is performed.

See Also:

Constant Field Values

Constructor Detail

CacheBasedDatasetBuilder

public CacheBasedDatasetBuilder(Ignite ignite,
                                IgniteCache<K,V> upstreamCache)

Constructs a new instance of cache based dataset builder that makes CacheBasedDataset with default predicate that passes all upstream entries to dataset.

Parameters:

ignite - Ignite instance.

upstreamCache - Ignite Cache with upstream data.

CacheBasedDatasetBuilder

public CacheBasedDatasetBuilder(Ignite ignite,
                                IgniteCache<K,V> upstreamCache,
                                IgniteBiPredicate<K,V> filter)

Constructs a new instance of cache based dataset builder that makes CacheBasedDataset.

Parameters:

ignite - Ignite instance.

upstreamCache - Ignite Cache with upstream data.

filter - Filter for upstream data.

CacheBasedDatasetBuilder

public CacheBasedDatasetBuilder(Ignite ignite,
                                IgniteCache<K,V> upstreamCache,
                                IgniteBiPredicate<K,V> filter,
                                UpstreamTransformerBuilder transformerBuilder)

Constructs a new instance of cache based dataset builder that makes CacheBasedDataset.

Parameters:

ignite - Ignite instance.

upstreamCache - Ignite Cache with upstream data.

filter - Filter for upstream data.

CacheBasedDatasetBuilder

public CacheBasedDatasetBuilder(Ignite ignite,
                                IgniteCache<K,V> upstreamCache,
                                IgniteBiPredicate<K,V> filter,
                                UpstreamTransformerBuilder transformerBuilder,
                                Boolean isKeepBinary,
                                int retries)

Constructs a new instance of cache based dataset builder that makes CacheBasedDataset.

Parameters:

ignite - Ignite.

upstreamCache - Upstream cache.

filter - Filter.

transformerBuilder - Transformer builder.

isKeepBinary - Is keep binary for upstream cache.

retries - Number of retries for the case when one of partitions not found on the node where loading is performed.

Method Detail

build

public <C extends Serializable,D extends AutoCloseable> CacheBasedDataset<K,V,C,D> build(LearningEnvironmentBuilder envBuilder,
                                                                                         PartitionContextBuilder<K,V,C> partCtxBuilder,
                                                                                         PartitionDataBuilder<K,V,C,D> partDataBuilder,
                                                                                         LearningEnvironment localLearningEnv)

Constructs a new instance of Dataset that includes allocation required data structures and initialization of context part of partitions.

Specified by:

build in interface DatasetBuilder<K,V>

Type Parameters:

C - Type of a partition context.

D - Type of a partition data.

Parameters:

envBuilder - Learning environment builder.

partCtxBuilder - Partition context builder.

partDataBuilder - Partition data builder.

localLearningEnv - Local learning environment.

Returns:

Dataset.

withUpstreamTransformer

public DatasetBuilder<K,V> withUpstreamTransformer(UpstreamTransformerBuilder builder)

Returns new instance of DatasetBuilder with new UpstreamTransformerBuilder added to chain of upstream transformer builders. When needed, each builder in chain first transformed into UpstreamTransformer, those are in turn composed together one after another forming final UpstreamTransformer. This transformer is applied to upstream data before it is passed to PartitionDataBuilder and PartitionContextBuilder. This is needed to allow transformation to upstream data which are agnostic of any changes that happen after. Such transformations may be used for deriving meta-algorithms such as bagging (see BaggingUpstreamTransformer).

Specified by:

withUpstreamTransformer in interface DatasetBuilder<K,V>

Returns:

Returns new instance of DatasetBuilder with new UpstreamTransformerBuilder added to chain of upstream transformer builders.

withFilter

public DatasetBuilder<K,V> withFilter(IgniteBiPredicate<K,V> filterToAdd)

Returns new instance of DatasetBuilder using conjunction of internal filter and filterToAdd.

Specified by:

withFilter in interface DatasetBuilder<K,V>

Parameters:

filterToAdd - Additional filter.

withKeepBinary

public CacheBasedDatasetBuilder<K,V> withKeepBinary(boolean isKeepBinary)

Add keepBinary policy. False by default.

Parameters:

isKeepBinary - Is keep binary.

withRetriesNumber

public CacheBasedDatasetBuilder<K,V> withRetriesNumber(int retries)

Sets number of retries. 15 * 60 by default.

Parameters:

retries - Number of retries.

Returns:

CacheBasedDatasetBuilder instance.