pygridgain package

Submodules

pygridgain.binary module

GenericObjectMeta is a metaclass used to create classes, which objects serve as a native Python values for GridGain Complex object data type. You can use this metaclass with your existing classes to save and restore their selected attributes and properties to/from GridGain caches. It is also used internally by pygridgain to create simple data classes “on the fly” when retrieving arbitrary Complex objects.

You can get the examples of using Complex objects in the Complex objects section of pygridgain documentation.

class pygridgain.binary.GenericObjectMeta(name: str, base_classes: tuple, namespace: dict, type_name: str = None, schema: collections.OrderedDict = None, **kwargs)

Bases: pygridgain.binary.GenericObjectPropsMeta

Complex (or Binary) Object metaclass. It is aimed to help user create classes, which objects could serve as a pythonic representation of the BinaryObject GridGain data type.

__init__(name: str, base_classes: tuple, namespace: dict, type_name: str = None, schema: collections.OrderedDict = None, **kwargs)

Initializes binary object class.

Parameters:
  • type_name – (optional) binary object name. Defaults to class name,
  • schema – (optional) a dict of field names: field types,
Raise:

ParseError if one or more binary field types did not recognized.

version = None
class pygridgain.binary.GenericObjectProps

Bases: pygridgain.datatypes.base.GridGainDataTypeProps

This class is mixed both to metaclass and to resulting class to make class properties universally available. You should not subclass it directly.

schema

Binary object schema.

schema_id

Binary object schema ID.

class pygridgain.binary.GenericObjectPropsMeta

Bases: type, pygridgain.binary.GenericObjectProps

pygridgain.cache module

class pygridgain.cache.Cache(client: Client, settings: Union[str, dict] = None, with_get: bool = False, get_only: bool = False)

Bases: object

GridGain cache abstraction. Users should never use this class directly, but construct its instances with create_cache(), get_or_create_cache() or get_cache() methods instead. See this example on how to do it.

__init__(client: Client, settings: Union[str, dict] = None, with_get: bool = False, get_only: bool = False)

Initialize cache object.

Parameters:
  • client – GridGain client,
  • settings – cache settings. Can be a string (cache name) or a dict of cache properties and their values. In this case PROP_NAME is mandatory,
  • with_get – (optional) do not raise exception, if the cache is already exists. Defaults to False,
  • get_only – (optional) do not communicate with GridGain server at all, only create Cache instance. Defaults to False.
affinity = None
cache_id

Cache ID.

Returns:integer value of the cache ID.
clear(keys: Union[list, NoneType] = None)

Clears the cache without notifying listeners or cache writers.

Parameters:keys – (optional) list of cache keys or (key, key type hint) tuples to clear (default: clear all).
clear_key(key, key_hint: object = None)

Clears the cache key without notifying listeners or cache writers.

Parameters:
  • key – key for the cache entry,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
client

GridGain Client object.

Returns:Client object, through which the cache is accessed.
contains_key(key, key_hint=None) → bool

Returns a value indicating whether given key is present in cache.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
Returns:

boolean True when key is present, False otherwise.

contains_keys(keys: Iterable[T_co]) → bool

Returns a value indicating whether all given keys are present in cache.

Parameters:keys – a list of keys or (key, type hint) tuples,
Returns:boolean True when all keys are present, False otherwise.
destroy()

Destroys cache with a given name.

get(key, key_hint: object = None) → Any

Retrieves a value from cache by key.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
Returns:

value retrieved.

get_all(keys: list) → list

Retrieves multiple key-value pairs from cache.

Parameters:keys – list of keys or tuples of (key, key_hint),
Returns:a dict of key-value pairs.
get_and_put(key, value, key_hint=None, value_hint=None) → Any

Puts a value with a given key to cache, and returns the previous value for that key, or null value if there was not such key.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • value – value for the key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • value_hint – (optional) GridGain data type, for which the given value should be converted.
Returns:

old value or None.

get_and_put_if_absent(key, value, key_hint=None, value_hint=None)

Puts a value with a given key to cache only if the key does not already exist.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • value – value for the key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • value_hint – (optional) GridGain data type, for which the given value should be converted,
Returns:

old value or None.

get_and_remove(key, key_hint=None) → Any

Removes the cache entry with specified key, returning the value.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
Returns:

old value or None.

get_and_replace(key, value, key_hint=None, value_hint=None) → Any

Puts a value with a given key to cache, returning previous value for that key, if and only if there is a value currently mapped for that key.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • value – value for the key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • value_hint – (optional) GridGain data type, for which the given value should be converted.
Returns:

old value or None.

get_best_node(*args, **kwargs) → Callable
get_best_node_120(*args, **kwargs)
get_best_node_130(*args, **kwargs)
get_protocol_version() → Union[Tuple, NoneType]

Returns the tuple of major, minor, and revision numbers of the used thin protocol version, or None, if no connection to the GridGain cluster was not yet established.

This method is not a part of the public API. Unless you wish to extend the pygridgain capabilities (with additional testing, logging, examining connections, et c.) you probably should not use it.

get_size(peek_modes=0)

Gets the number of entries in cache.

Parameters:peek_modes – (optional) limit count to near cache partition (PeekModes.NEAR), primary cache (PeekModes.PRIMARY), or backup cache (PeekModes.BACKUP). Defaults to all cache partitions (PeekModes.ALL),
Returns:integer number of cache entries.
name

Lazy cache name.

Returns:cache name string.
put(key, value, key_hint: object = None, value_hint: object = None)

Puts a value with a given key to cache (overwriting existing value if any).

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • value – value for the key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • value_hint – (optional) GridGain data type, for which the given value should be converted.
put_all(pairs: dict)

Puts multiple key-value pairs to cache (overwriting existing associations if any).

Parameters:pairs – dictionary type parameters, contains key-value pairs to save. Each key or value can be an item of representable Python type or a tuple of (item, hint),
put_if_absent(key, value, key_hint=None, value_hint=None)

Puts a value with a given key to cache only if the key does not already exist.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • value – value for the key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • value_hint – (optional) GridGain data type, for which the given value should be converted.
remove_all()

Removes all cache entries, notifying listeners and cache writers.

remove_if_equals(key, sample, key_hint=None, sample_hint=None)

Removes an entry with a given key if provided value is equal to actual value, notifying listeners and cache writers.

Parameters:
  • key – key for the cache entry,
  • sample – a sample to compare the stored value with,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • sample_hint – (optional) GridGain data type, for whic the given sample should be converted.
remove_key(key, key_hint=None)

Clears the cache key without notifying listeners or cache writers.

Parameters:
  • key – key for the cache entry,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
remove_keys(keys: list)

Removes cache entries by given list of keys, notifying listeners and cache writers.

Parameters:keys – list of keys or tuples of (key, key_hint) to remove.
replace(key, value, key_hint: object = None, value_hint: object = None)

Puts a value with a given key to cache only if the key already exist.

Parameters:
  • key – key for the cache entry. Can be of any supported type,
  • value – value for the key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • value_hint – (optional) GridGain data type, for which the given value should be converted.
replace_if_equals(key, sample, value, key_hint=None, sample_hint=None, value_hint=None) → Any

Puts a value with a given key to cache only if the key already exists and value equals provided sample.

Parameters:
  • key – key for the cache entry,
  • sample – a sample to compare the stored value with,
  • value – new value for the given key,
  • key_hint – (optional) GridGain data type, for which the given key should be converted,
  • sample_hint – (optional) GridGain data type, for which the given sample should be converted
  • value_hint – (optional) GridGain data type, for which the given value should be converted,
Returns:

boolean True when key is present, False otherwise.

scan(page_size: int = 1, partitions: int = -1, local: bool = False)

Returns all key-value pairs from the cache, similar to get_all, but with internal pagination, which is slower, but safer.

Parameters:
  • page_size – (optional) page size. Default size is 1 (slowest and safest),
  • partitions – (optional) number of partitions to query (negative to query entire cache),
  • local – (optional) pass True if this query should be executed on local node only. Defaults to False,
Returns:

generator with key-value pairs.

select_row(query_str: str, page_size: int = 1, query_args: Union[list, NoneType] = None, distributed_joins: bool = False, replicated_only: bool = False, local: bool = False, timeout: int = 0)

Executes a simplified SQL SELECT query over data stored in the cache. The query returns the whole record (key and value).

Parameters:
  • query_str – SQL query string,
  • page_size – (optional) cursor page size. Default is 1, which means that client makes one server call per row,
  • query_args – (optional) query arguments,
  • distributed_joins – (optional) distributed joins. Defaults to False,
  • replicated_only – (optional) whether query contains only replicated tables or not. Defaults to False,
  • local – (optional) pass True if this query should be executed on local node only. Defaults to False,
  • timeout – (optional) non-negative timeout value in ms. Zero disables timeout (default),
Returns:

generator with key-value pairs.

settings

Lazy Cache settings. See the example of reading this property.

All cache properties are documented here: Cache Properties.

Returns:dict of cache properties and their values.

pygridgain.client module

This module contains Client class, that lets you communicate with GridGain cluster node by the means of GridGain binary client protocol.

To start the communication, you may connect to the node of their choice by instantiating the Client object and calling connect() method with proper parameters.

The whole storage room of GridGain cluster is split up into named structures, called caches. For accessing the particular cache in key-value style (a-la Redis or memcached) you should first create the Cache object by calling create_cache() or get_or_create_cache() methods, than call Cache methods. If you wish to create a cache object without communicating with server, there is also a get_cache() method that does just that.

For using GridGain SQL, call sql() method. It returns a generator with result rows.

register_binary_type() and query_binary_type() methods operates the local (class-wise) registry for GridGain Complex objects.

class pygridgain.client.Client(compact_footer: bool = None, partition_aware: bool = False, **kwargs)

Bases: object

This is a main pygridgain class, that is build upon the Connection. In addition to the attributes, properties and methods of its parent class, Client implements the following features:

  • cache factory. Cache objects are used for key-value operations,
  • GridGain SQL endpoint,
  • binary types registration endpoint.
__init__(compact_footer: bool = None, partition_aware: bool = False, **kwargs)

Initialize client.

Parameters:
affinity_version = None
close()

This property remembers Complex object schema encoding approach when decoding any Complex object, to use the same approach on Complex object encoding.

Returns:True if compact schema was used by server or no Complex object decoding has yet taken place, False if full schema was used.
connect(*args)

Connect to GridGain cluster node(s).

Parameters:args – (optional) host(s) and port(s) to connect to.
create_cache(settings: Union[str, dict]) → pygridgain.cache.Cache

Creates GridGain cache by name. Raises CacheError if such a cache is already exists.

Parameters:settings – cache name or dict of cache properties’ codes and values. All cache properties are documented here: Cache Properties. See also the cache creation example,
Returns:Cache object.
get_binary_type(binary_type: Union[str, int]) → dict

Gets the binary type information from the GridGain server. This is quite a low-level implementation of GridGain thin client protocol’s OP_GET_BINARY_TYPE operation. You would probably want to use query_binary_type() instead.

Parameters:binary_type – binary type name or ID,
Returns:binary type description − a dict with the following fields:
  • type_exists: True if the type is registered, False otherwise. In the latter case all the following fields are omitted,
  • type_id: Complex object type ID,
  • type_name: Complex object type name,
  • affinity_key_field: string value or None,
  • is_enum: False in case of Complex object registration,
  • schemas: a list, containing the Complex object schemas in format: OrderedDict[field name: field type hint]. A schema can be empty.
get_cache(settings: Union[str, dict]) → pygridgain.cache.Cache

Creates Cache object with a given cache name without checking it up on server. If such a cache does not exist, some kind of exception (most probably CacheError) may be raised later.

Parameters:settings – cache name or cache properties (but only PROP_NAME property is allowed),
Returns:Cache object.
get_cache_names() → list

Gets existing cache names.

Returns:list of cache names.
get_or_create_cache(settings: Union[str, dict]) → pygridgain.cache.Cache

Creates GridGain cache, if not exist.

Parameters:settings – cache name or dict of cache properties’ codes and values. All cache properties are documented here: Cache Properties. See also the cache creation example,
Returns:Cache object.
get_protocol_version() → Union[Tuple, NoneType]

Returns the tuple of major, minor, and revision numbers of the used thin protocol version, or None, if no connection to the GridGain cluster was not yet established.

This method is not a part of the public API. Unless you wish to extend the pygridgain capabilities (with additional testing, logging, examining connections, et c.) you probably should not use it.

partition_aware
protocol_version = None
put_binary_type(type_name: str, affinity_key_field: str = None, is_enum=False, schema: dict = None)

Registers binary type information in cluster. Do not update binary registry. This is a literal implementation of GridGain thin client protocol’s OP_PUT_BINARY_TYPE operation. You would probably want to use register_binary_type() instead.

Parameters:
  • type_name – name of the data type being registered,
  • affinity_key_field – (optional) name of the affinity key field,
  • is_enum – (optional) register enum if True, binary object otherwise. Defaults to False,
  • schema – (optional) when register enum, pass a dict of enumerated parameter names as keys and an integers as values. When register binary type, pass a dict of field names: field types. Binary type with no fields is OK.
query_binary_type(binary_type: Union[int, str], schema: Union[int, dict] = None, sync: bool = True)

Queries the registry of Complex object classes.

Parameters:
  • binary_type – Complex object type name or ID,
  • schema – (optional) Complex object schema or schema ID,
  • sync – (optional) look up the GridGain server for registered Complex objects and create data classes for them if needed,
Returns:

found dataclass or None, if schema parameter is provided, a dict of {schema ID: dataclass} format otherwise.

random_node
random_node_120()
random_node_130()
register_binary_type(data_class: Type[CT_co], affinity_key_field: str = None)

Register the given class as a representation of a certain Complex object type. Discards autogenerated or previously registered class.

Parameters:
  • data_class – Complex object class,
  • affinity_key_field – (optional) affinity parameter.
sql(query_str: str, page_size: int = 1, query_args: Iterable[T_co] = None, schema: Union[int, str] = 'PUBLIC', statement_type: int = 0, distributed_joins: bool = False, local: bool = False, replicated_only: bool = False, enforce_join_order: bool = False, collocated: bool = False, lazy: bool = False, include_field_names: bool = False, max_rows: int = -1, timeout: int = 0)

Runs an SQL query and returns its result.

Parameters:
  • query_str – SQL query string,
  • page_size – (optional) cursor page size. Default is 1, which means that client makes one server call per row,
  • query_args – (optional) query arguments. List of values or (value, type hint) tuples,
  • schema – (optional) schema for the query. Defaults to PUBLIC,
  • statement_type

    (optional) statement type. Can be:

    • StatementType.ALL − any type (default),
    • StatementType.SELECT − select,
    • StatementType.UPDATE − update.
  • distributed_joins – (optional) distributed joins. Defaults to False,
  • local – (optional) pass True if this query should be executed on local node only. Defaults to False,
  • replicated_only – (optional) whether query contains only replicated tables or not. Defaults to False,
  • enforce_join_order – (optional) enforce join order. Defaults to False,
  • collocated – (optional) whether your data is co-located or not. Defaults to False,
  • lazy – (optional) lazy query execution. Defaults to False,
  • include_field_names – (optional) include field names in result. Defaults to False,
  • max_rows – (optional) query-wide maximum of rows. Defaults to -1 (all rows),
  • timeout – (optional) non-negative timeout value in ms. Zero disables timeout (default),
Returns:

generator with result rows as a lists. If include_field_names was set, the first row will hold field names.

pygridgain.constants module

This module contains some constants, used internally throughout the API.

pygridgain.exceptions module

exception pygridgain.exceptions.BinaryTypeError

Bases: pygridgain.exceptions.CacheError

A remote error in operation with Complex Object registry.

exception pygridgain.exceptions.CacheCreationError

Bases: pygridgain.exceptions.CacheError

This exception is raised, when any complex operation failed on cache creation phase.

exception pygridgain.exceptions.CacheError

Bases: Exception

This exception is raised, whenever any remote Thin client operation returns an error.

exception pygridgain.exceptions.HandshakeError(expected_version: Tuple[int, int, int], message: str)

Bases: OSError

This exception is raised on GridGain binary protocol handshake failure, as defined in https://apacheignite.readme.io/docs/binary-client-protocol#section-handshake

__init__(expected_version: Tuple[int, int, int], message: str)

Initialize self. See help(type(self)) for accurate signature.

exception pygridgain.exceptions.ParameterError

Bases: Exception

This exception represents the parameter validation error in any pygridgain method.

exception pygridgain.exceptions.ParseError

Bases: Exception

This exception is raised, when pygridgain is unable to build a query to, or parse a response from, GridGain node.

exception pygridgain.exceptions.ReconnectError

Bases: Exception

This exception is raised by Client.reconnect method, when no more nodes are left to connect to. It is not meant to be an error, but rather a flow control tool, similar to StopIteration.

exception pygridgain.exceptions.SQLError

Bases: pygridgain.exceptions.CacheError

An error in SQL query.

pygridgain.utils module

class pygridgain.utils.DaemonicTimer(interval, function, args=None, kwargs=None)

Bases: threading.Thread

Same as normal threading.Timer, but do not delay the program exit.

__init__(interval, function, args=None, kwargs=None)

This constructor should always be called with keyword arguments. Arguments are:

group should be None; reserved for future extension when a ThreadGroup class is implemented.

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.

args is the argument tuple for the target invocation. Defaults to ().

kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

cancel()

Stop the timer if it hasn’t finished yet.

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

pygridgain.utils.cache_id(cache: Union[str, int]) → int

Create a cache ID from cache name.

Parameters:cache – cache name or ID,
Returns:cache ID.
pygridgain.utils.capitalize(string: str) → str

Capitalizing the string, assuming the first character is a letter. Does not touch any other character, unlike the string.capitalize().

pygridgain.utils.datetime_hashcode(value: int) → int

Calculates hashcode from UNIX epoch.

Parameters:value – UNIX time,
Returns:Java hashcode.
pygridgain.utils.decimal_hashcode(value: decimal.Decimal) → int

This is a translation of java.math.BigDecimal class hashCode() method to Python.

Parameters:value – pythonic decimal value,
Returns:hashcode.
pygridgain.utils.entity_id(cache: Union[str, int]) → Union[int, NoneType]

Create a type ID from type name or field ID from field name.

Parameters:cache – entity name or ID,
Returns:entity ID.
pygridgain.utils.get_field_by_id(obj: GenericObjectMeta, field_id: int) → Tuple[Any, pygridgain.datatypes.base.GridGainDataType]

Returns a complex object’s field value, given the field’s entity ID.

Parameters:
  • obj – complex object,
  • field_id – field ID,
Returns:

complex object field’s value and type.

pygridgain.utils.hashcode(string: Union[str, bytes]) → int

Calculate hash code used for identifying objects in GridGain binary API.

Parameters:string – UTF-8-encoded string identifier of binary buffer,
Returns:hash code.
pygridgain.utils.int_overflow(value: int) → int

Simulates 32bit integer overflow.

pygridgain.utils.is_binary(value)

Check if a value is a pythonic representation of a Complex object.

pygridgain.utils.is_hinted(value)

Check if a value is a tuple of data item and its type hint.

pygridgain.utils.is_iterable(value: Any) → bool

Check if value is iterable.

pygridgain.utils.is_pow2(value: int) → bool

Check if value is power of two.

pygridgain.utils.is_wrapped(value: Any) → bool

Check if a value is of WrappedDataObject type.

pygridgain.utils.process_delimiter(name: str, delimiter: str) → str

Splits the name by delimiter, capitalize each part, merge.

pygridgain.utils.schema_id(schema: Union[int, dict]) → int

Calculate Complex Object schema ID.

Parameters:schema – a dict of field names: field types,
Returns:schema ID.
pygridgain.utils.select_version(func)

This decorator tries to find a method more suitable for a current protocol version, before calling the decorated method. The object which method is being decorated must have get_protocol_version() method.

Parameters:func – decorated method,
Returns:wrapper.
pygridgain.utils.status_to_exception(exc: Type[Exception])

Converts erroneous status code with error message to an exception of the given class.

Parameters:exc – the class of exception to raise,
Returns:decorator.
pygridgain.utils.unsigned(value: int, c_type: _ctypes._SimpleCData = <class 'ctypes.c_uint'>) → int

Convert signed integer value to unsigned.

pygridgain.utils.unwrap_binary(client: Client, wrapped: tuple) → object

Unwrap wrapped BinaryObject and convert it to Python data.

Parameters:
  • client – connection to GridGain cluster,
  • wrappedWrappedDataObject value,
Returns:

dict representing wrapped BinaryObject.

Module contents