org.apache.ignite.ml.dataset

Interface Dataset<C extends Serializable,D extends AutoCloseable>

Type Parameters:

C - Type of a partition context.

D - Type of a partition data.

All Superinterfaces:

AutoCloseable

All Known Implementing Classes:

CacheBasedDataset, DatasetWrapper, LocalDataset, SimpleDataset, SimpleLabeledDataset

public interface Dataset<C extends Serializable,D extends AutoCloseable>
extends AutoCloseable

A dataset providing an API that allows to perform generic computations on a distributed data represented as a set of partitions distributed across a cluster or placed locally. Every partition contains a context (reliably stored segment) and data (unreliably stored segment, which can be recovered from an upstream data and a context if needed). Computations are performed in a MapReduce manner, what allows to reduce a network traffic for most of the machine learning algorithms.

Dataset functionality allows to implement iterative machine learning algorithms via introducing computation context. In case iterative algorithm requires to maintain a state available and updatable on every iteration this state can be stored in the context of the partition and after that it will be available in further computations even if the Ignite Cache partition will be moved to another node because of node failure or rebalancing.

Partition context should be Serializable to be saved in Ignite Cache. Partition data should be AutoCloseable to allow system to clean up correspondent resources when partition data is not needed anymore.

See Also:

CacheBasedDataset, LocalDataset, DatasetFactory

Method Summary

Modifier and Type	Method and Description
default void	compute(IgniteBiConsumer<D,Integer> map) Applies the specified map function to every partition data in the dataset and partition index.
default <R> R	compute(IgniteBiFunction<D,Integer,R> map, IgniteBinaryOperator<R> reduce) Applies the specified map function to every partition data and partition index in the dataset and then reduces map results to final result by using the reduce function.
<R> R	compute(IgniteBiFunction<D,Integer,R> map, IgniteBinaryOperator<R> reduce, R identity) Applies the specified map function to every partition data and partition index in the dataset and then reduces map results to final result by using the reduce function.
default void	compute(IgniteConsumer<D> map) Applies the specified map function to every partition data in the dataset.
default <R> R	compute(IgniteFunction<D,R> map, IgniteBinaryOperator<R> reduce) Applies the specified map function to every partition data in the dataset and then reduces map results to final result by using the reduce function.
default <R> R	compute(IgniteFunction<D,R> map, IgniteBinaryOperator<R> reduce, R identity) Applies the specified map function to every partition data in the dataset and then reduces map results to final result by using the reduce function.
default void	computeWithCtx(IgniteBiConsumer<C,D> map) Applies the specified map function to every partition data and context in the dataset.
default <R> R	computeWithCtx(IgniteBiFunction<C,D,R> map, IgniteBinaryOperator<R> reduce) Applies the specified map function to every partition data and context in the dataset and then reduces map results to final result by using the reduce function.
default <R> R	computeWithCtx(IgniteBiFunction<C,D,R> map, IgniteBinaryOperator<R> reduce, R identity) Applies the specified map function to every partition data and context in the dataset and then reduces map results to final result by using the reduce function.
default void	computeWithCtx(IgniteTriConsumer<C,D,Integer> map) Applies the specified map function to every partition data, context and partition index in the dataset.
default <R> R	computeWithCtx(IgniteTriFunction<C,D,Integer,R> map, IgniteBinaryOperator<R> reduce) Applies the specified map function to every partition data, context and partition index in the dataset and then reduces map results to final result by using the reduce function.
<R> R	computeWithCtx(IgniteTriFunction<C,D,Integer,R> map, IgniteBinaryOperator<R> reduce, R identity) Applies the specified map function to every partition data, context and partition index in the dataset and then reduces map results to final result by using the reduce function.
default <I extends Dataset<C,D>> I	wrap(IgniteFunction<Dataset<C,D>,I> wrapper) Wraps this dataset into the specified wrapper to introduce new functionality based on compute and computeWithCtx methods.

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

computeWithCtx

<R> R computeWithCtx(IgniteTriFunction<C,D,Integer,R> map,
                     IgniteBinaryOperator<R> reduce,
                     R identity)

Applies the specified map function to every partition data, context and partition index in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data, context and partition index.

reduce - Function applied to results of map to get final result.

identity - Identity.

Returns:

Final result.

compute

<R> R compute(IgniteBiFunction<D,Integer,R> map,
              IgniteBinaryOperator<R> reduce,
              R identity)

Applies the specified map function to every partition data and partition index in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data and partition index.

reduce - Function applied to results of map to get final result.

identity - Identity.

Returns:

Final result.

computeWithCtx

default <R> R computeWithCtx(IgniteTriFunction<C,D,Integer,R> map,
                             IgniteBinaryOperator<R> reduce)

Applies the specified map function to every partition data, context and partition index in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data, context and partition index.

reduce - Function applied to results of map to get final result.

Returns:

Final result.

compute

default <R> R compute(IgniteBiFunction<D,Integer,R> map,
                      IgniteBinaryOperator<R> reduce)

Applies the specified map function to every partition data and partition index in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data and partition index.

reduce - Function applied to results of map to get final result.

Returns:

Final result.

computeWithCtx

default <R> R computeWithCtx(IgniteBiFunction<C,D,R> map,
                             IgniteBinaryOperator<R> reduce,
                             R identity)

Applies the specified map function to every partition data and context in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data and context.

reduce - Function applied to results of map to get final result.

identity - Identity.

Returns:

Final result.

compute

default <R> R compute(IgniteFunction<D,R> map,
                      IgniteBinaryOperator<R> reduce,
                      R identity)

Applies the specified map function to every partition data in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data.

reduce - Function applied to results of map to get final result.

identity - Identity.

Returns:

Final result.

computeWithCtx

default <R> R computeWithCtx(IgniteBiFunction<C,D,R> map,
                             IgniteBinaryOperator<R> reduce)

Applies the specified map function to every partition data and context in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data and context.

reduce - Function applied to results of map to get final result.

Returns:

Final result.

compute

default <R> R compute(IgniteFunction<D,R> map,
                      IgniteBinaryOperator<R> reduce)

Applies the specified map function to every partition data in the dataset and then reduces map results to final result by using the reduce function.

Type Parameters:

R - Type of a result.

Parameters:

map - Function applied to every partition data.

reduce - Function applied to results of map to get final result.

Returns:

Final result.

computeWithCtx

default void computeWithCtx(IgniteTriConsumer<C,D,Integer> map)

Applies the specified map function to every partition data, context and partition index in the dataset.

Parameters:

map - Function applied to every partition data, context and partition index.

compute

default void compute(IgniteBiConsumer<D,Integer> map)

Applies the specified map function to every partition data in the dataset and partition index.

Parameters:

map - Function applied to every partition data and partition index.

computeWithCtx

default void computeWithCtx(IgniteBiConsumer<C,D> map)

Applies the specified map function to every partition data and context in the dataset.

Parameters:

map - Function applied to every partition data and context.

compute

default void compute(IgniteConsumer<D> map)

Applies the specified map function to every partition data in the dataset.

Parameters:

map - Function applied to every partition data.

wrap

default <I extends Dataset<C,D>> I wrap(IgniteFunction<Dataset<C,D>,I> wrapper)

Wraps this dataset into the specified wrapper to introduce new functionality based on compute and computeWithCtx methods.

Type Parameters:

I - Type of a new wrapped dataset.

Parameters:

wrapper - Dataset wrapper.

Returns:

New wrapped dataset.