org.apache.ignite.ml.trainers

Class DatasetTrainer<M extends IgniteModel,L>

java.lang.Object

org.apache.ignite.ml.trainers.DatasetTrainer<M,L>

Type Parameters:

M - Type of a produced model.

L - Type of a label.

Direct Known Subclasses:

AdaptableDatasetTrainer, BaggedTrainer, GDBTrainer, MultiLabelDatasetTrainer, SingleLabelDatasetTrainer, StackedDatasetTrainer, TrainersParallelComposition, TrainersSequentialComposition

public abstract class DatasetTrainer<M extends IgniteModel,L>
extends Object

Interface for trainers. Trainer is just a function which produces model from the data.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DatasetTrainer.EmptyDatasetException EmptyDataset exception.

Field Summary

Fields

Modifier and Type	Field and Description
protected LearningEnvironmentBuilder	envBuilder Learning environment builder.
protected LearningEnvironment	environment Learning Environment.

Constructor Summary

Constructors

Constructor and Description
DatasetTrainer()

Method Summary

Modifier and Type	Method and Description
<K,V> M	fit(DatasetBuilder<K,V> datasetBuilder, Preprocessor<K,V> preprocessor) Trains model based on the specified data.
<K,V> M	fit(DatasetBuilder<K,V> datasetBuilder, Preprocessor<K,V> preprocessor, LearningEnvironment learningEnvironment) Trains model based on the specified data.
<K,V> M	fit(Ignite ignite, IgniteCache<K,V> cache, IgniteBiPredicate<K,V> filter, Preprocessor<K,V> preprocessor) Trains model based on the specified data.
<K,V> M	fit(Ignite ignite, IgniteCache<K,V> cache, Preprocessor<K,V> preprocessor) Trains model based on the specified data.
<K,V> M	fit(Map<K,V> data, IgniteBiPredicate<K,V> filter, int parts, Preprocessor<K,V> preprocessor) Trains model based on the specified data.
<K,V> M	fit(Map<K,V> data, int parts, Preprocessor<K,V> preprocessor) Trains model based on the specified data.
protected abstract <K,V> M	fitWithInitializedDeployingContext(DatasetBuilder<K,V> datasetBuilder, Preprocessor<K,V> preprocessor) Trains model based on the specified data.
protected M	getLastTrainedModelOrThrowEmptyDatasetException(M lastTrainedMdl) Used on update phase when given dataset is empty.
static <I,L> DatasetTrainer<IgniteModel<I,I>,L>	identityTrainer() Returns the trainer which returns identity model.
abstract boolean	isUpdateable(M mdl)
LearningEnvironment	learningEnvironment() Get learning environment.
<K,V> M	update(M mdl, DatasetBuilder<K,V> datasetBuilder, Preprocessor<K,V> preprocessor) Gets state of model in arguments, compare it with training parameters of trainer and if they are fit then trainer updates model in according to new data and return new model.
<K,V> M	update(M mdl, Ignite ignite, IgniteCache<K,V> cache, IgniteBiPredicate<K,V> filter, Preprocessor<K,V> preprocessor) Gets state of model in arguments, update in according to new data and return new model.
<K,V> M	update(M mdl, Ignite ignite, IgniteCache<K,V> cache, Preprocessor<K,V> preprocessor) Gets state of model in arguments, update in according to new data and return new model.
<K,V> M	update(M mdl, Map<K,V> data, IgniteBiPredicate<K,V> filter, int parts, Preprocessor<K,V> preprocessor) Gets state of model in arguments, update in according to new data and return new model.
<K,V> M	update(M mdl, Map<K,V> data, int parts, Preprocessor<K,V> preprocessor) Gets state of model in arguments, update in according to new data and return new model.
protected abstract <K,V> M	updateModel(M mdl, DatasetBuilder<K,V> datasetBuilder, Preprocessor<K,V> preprocessor) Trains new model taken previous one as a first approximation.
<L1> DatasetTrainer<M,L1>	withConvertedLabels(IgniteFunction<L1,L> new2Old) Creates DatasetTrainer with same training logic, but able to accept labels of given new type of labels.
DatasetTrainer<M,L>	withEnvironmentBuilder(LearningEnvironmentBuilder envBuilder) Changes learning Environment.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

envBuilder

protected LearningEnvironmentBuilder envBuilder

Learning environment builder.

environment

protected LearningEnvironment environment

Learning Environment.

Constructor Detail

DatasetTrainer

public DatasetTrainer()

Method Detail

identityTrainer

public static <I,L> DatasetTrainer<IgniteModel<I,I>,L> identityTrainer()

Returns the trainer which returns identity model.

Type Parameters:

I - Type of model input.

L - Type of labels in dataset.

Returns:

Trainer which returns identity model.

fit

public <K,V> M fit(DatasetBuilder<K,V> datasetBuilder,
                   Preprocessor<K,V> preprocessor)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

datasetBuilder - Dataset builder.

preprocessor - Extractor of UpstreamEntry into LabeledVector.

Returns:

Model.

fit

public <K,V> M fit(DatasetBuilder<K,V> datasetBuilder,
                   Preprocessor<K,V> preprocessor,
                   LearningEnvironment learningEnvironment)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

datasetBuilder - Dataset builder.

preprocessor - Extractor of UpstreamEntry into LabeledVector.

learningEnvironment - Local learning environment.

Returns:

Model.

fitWithInitializedDeployingContext

protected abstract <K,V> M fitWithInitializedDeployingContext(DatasetBuilder<K,V> datasetBuilder,
                                                              Preprocessor<K,V> preprocessor)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

datasetBuilder - Dataset builder.

preprocessor - Extractor of UpstreamEntry into LabeledVector.

Returns:

Model.

update

public <K,V> M update(M mdl,
                      DatasetBuilder<K,V> datasetBuilder,
                      Preprocessor<K,V> preprocessor)

Gets state of model in arguments, compare it with training parameters of trainer and if they are fit then trainer updates model in according to new data and return new model. In other case trains new model.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

mdl - Learned model.

datasetBuilder - Dataset builder.

preprocessor - Extractor of UpstreamEntry into LabeledVector.

Returns:

Updated model.

isUpdateable

public abstract boolean isUpdateable(M mdl)

Parameters:

mdl - Model.

Returns:

True if current critical for training parameters correspond to parameters from last training.

getLastTrainedModelOrThrowEmptyDatasetException

@NotNull
protected M getLastTrainedModelOrThrowEmptyDatasetException(M lastTrainedMdl)

Used on update phase when given dataset is empty. If last trained model exist then method returns it. In other case throws IllegalArgumentException.

Parameters:

lastTrainedMdl - Model.

fit

public <K,V> M fit(Ignite ignite,
                   IgniteCache<K,V> cache,
                   Preprocessor<K,V> preprocessor)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

ignite - Ignite instance.

cache - Ignite cache.

preprocessor - Upstream preprocessor.

Returns:

Model.

update

public <K,V> M update(M mdl,
                      Ignite ignite,
                      IgniteCache<K,V> cache,
                      Preprocessor<K,V> preprocessor)

Gets state of model in arguments, update in according to new data and return new model.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

mdl - Learned model.

ignite - Ignite instance.

cache - Ignite cache.

preprocessor - Upstream preprocessor.

Returns:

Updated model.

fit

public <K,V> M fit(Ignite ignite,
                   IgniteCache<K,V> cache,
                   IgniteBiPredicate<K,V> filter,
                   Preprocessor<K,V> preprocessor)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

ignite - Ignite instance.

cache - Ignite cache.

filter - Filter for upstream data.

preprocessor - Upstream preprocessor.

Returns:

Model.

update

public <K,V> M update(M mdl,
                      Ignite ignite,
                      IgniteCache<K,V> cache,
                      IgniteBiPredicate<K,V> filter,
                      Preprocessor<K,V> preprocessor)

Gets state of model in arguments, update in according to new data and return new model.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

mdl - Learned model.

ignite - Ignite instance.

cache - Ignite cache.

filter - Filter for upstream data.

preprocessor - Upstream preprocessor.

Returns:

Updated model.

fit

public <K,V> M fit(Map<K,V> data,
                   int parts,
                   Preprocessor<K,V> preprocessor)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

data - Data.

parts - Number of partitions.

preprocessor - Upstream preprocessor.

Returns:

Model.

update

public <K,V> M update(M mdl,
                      Map<K,V> data,
                      int parts,
                      Preprocessor<K,V> preprocessor)

Gets state of model in arguments, update in according to new data and return new model.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

mdl - Learned model.

data - Data.

parts - Number of partitions.

preprocessor - Upstream preprocessor.

Returns:

Updated model.

fit

public <K,V> M fit(Map<K,V> data,
                   IgniteBiPredicate<K,V> filter,
                   int parts,
                   Preprocessor<K,V> preprocessor)

Trains model based on the specified data.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

data - Data.

filter - Filter for upstream data.

parts - Number of partitions.

preprocessor - Upstream preprocessor.

Returns:

Model.

update

public <K,V> M update(M mdl,
                      Map<K,V> data,
                      IgniteBiPredicate<K,V> filter,
                      int parts,
                      Preprocessor<K,V> preprocessor)

Gets state of model in arguments, update in according to new data and return new model.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

data - Data.

filter - Filter for upstream data.

parts - Number of partitions.

preprocessor - Upstream preprocessor.

Returns:

Updated model.

withEnvironmentBuilder

public DatasetTrainer<M,L> withEnvironmentBuilder(LearningEnvironmentBuilder envBuilder)

Changes learning Environment.

Parameters:

envBuilder - Learning environment builder.

updateModel

protected abstract <K,V> M updateModel(M mdl,
                                       DatasetBuilder<K,V> datasetBuilder,
                                       Preprocessor<K,V> preprocessor)

Trains new model taken previous one as a first approximation.

Type Parameters:

K - Type of a key in upstream data.

V - Type of a value in upstream data.

Parameters:

mdl - Learned model.

datasetBuilder - Dataset builder.

preprocessor - Extractor of UpstreamEntry into LabeledVector.

Returns:

Updated model.

learningEnvironment

public LearningEnvironment learningEnvironment()

Get learning environment.

Returns:

Learning environment.

withConvertedLabels

public <L1> DatasetTrainer<M,L1> withConvertedLabels(IgniteFunction<L1,L> new2Old)

Creates DatasetTrainer with same training logic, but able to accept labels of given new type of labels.

Type Parameters:

L1 - New labels type.

Parameters:

new2Old - Converter of new labels to old labels.

Returns:

DatasetTrainer with same training logic, but able to accept labels of given new type of labels.