pygridgain package¶
Subpackages¶
- pygridgain.api package
- pygridgain.connection package
- pygridgain.datatypes package
- Submodules
- pygridgain.datatypes.base module
- pygridgain.datatypes.binary module
- pygridgain.datatypes.cache_config module
- pygridgain.datatypes.cache_properties module
- pygridgain.datatypes.complex module
- pygridgain.datatypes.internal module
- pygridgain.datatypes.key_value module
- pygridgain.datatypes.null_object module
- pygridgain.datatypes.primitive module
- pygridgain.datatypes.primitive_arrays module
- pygridgain.datatypes.primitive_objects module
- pygridgain.datatypes.prop_codes module
- pygridgain.datatypes.sql module
- pygridgain.datatypes.standard module
- pygridgain.datatypes.type_codes module
- pygridgain.datatypes.type_ids module
- pygridgain.datatypes.type_names module
- Module contents
- pygridgain.queries package
Submodules¶
pygridgain.binary module¶
GenericObjectMeta
is a metaclass used to create
classes, which objects serve as a native Python values for GridGain Complex
object data type. You can use this metaclass with your existing classes
to save and restore their selected attributes and properties to/from
GridGain caches. It is also used internally by pygridgain to create simple
data classes “on the fly” when retrieving arbitrary Complex objects.
You can get the examples of using Complex objects in the Complex objects section of pygridgain documentation.
-
class
pygridgain.binary.
GenericObjectMeta
(name: str, base_classes: tuple, namespace: dict, type_name: str = None, schema: collections.OrderedDict = None, **kwargs)¶ Bases:
pygridgain.binary.GenericObjectPropsMeta
Complex (or Binary) Object metaclass. It is aimed to help user create classes, which objects could serve as a pythonic representation of the
BinaryObject
GridGain data type.-
__init__
(name: str, base_classes: tuple, namespace: dict, type_name: str = None, schema: collections.OrderedDict = None, **kwargs)¶ Initializes binary object class.
Parameters: - type_name – (optional) binary object name. Defaults to class name,
- schema – (optional) a dict of field names: field types,
Raise: ParseError if one or more binary field types did not recognized.
-
version
= None¶
-
-
class
pygridgain.binary.
GenericObjectProps
¶ Bases:
pygridgain.datatypes.base.GridGainDataTypeProps
This class is mixed both to metaclass and to resulting class to make class properties universally available. You should not subclass it directly.
-
schema
¶ Binary object schema.
-
schema_id
¶ Binary object schema ID.
-
-
class
pygridgain.binary.
GenericObjectPropsMeta
¶ Bases:
type
,pygridgain.binary.GenericObjectProps
pygridgain.cache module¶
-
class
pygridgain.cache.
Cache
(client: Client, settings: Union[str, dict] = None, with_get: bool = False, get_only: bool = False)¶ Bases:
object
GridGain cache abstraction. Users should never use this class directly, but construct its instances with
create_cache()
,get_or_create_cache()
orget_cache()
methods instead. See this example on how to do it.-
__init__
(client: Client, settings: Union[str, dict] = None, with_get: bool = False, get_only: bool = False)¶ Initialize cache object.
Parameters: - client – GridGain client,
- settings – cache settings. Can be a string (cache name) or a dict of cache properties and their values. In this case PROP_NAME is mandatory,
- with_get – (optional) do not raise exception, if the cache is already exists. Defaults to False,
- get_only – (optional) do not communicate with GridGain server at all, only create Cache instance. Defaults to False.
-
affinity
= None¶
-
cache_id
¶ Cache ID.
Returns: integer value of the cache ID.
-
clear
(keys: Union[list, NoneType] = None)¶ Clears the cache without notifying listeners or cache writers.
Parameters: keys – (optional) list of cache keys or (key, key type hint) tuples to clear (default: clear all).
-
clear_key
(key, key_hint: object = None)¶ Clears the cache key without notifying listeners or cache writers.
Parameters: - key – key for the cache entry,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
-
contains_key
(key, key_hint=None) → bool¶ Returns a value indicating whether given key is present in cache.
Parameters: - key – key for the cache entry. Can be of any supported type,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
Returns: boolean True when key is present, False otherwise.
-
contains_keys
(keys: Iterable[T_co]) → bool¶ Returns a value indicating whether all given keys are present in cache.
Parameters: keys – a list of keys or (key, type hint) tuples, Returns: boolean True when all keys are present, False otherwise.
-
destroy
()¶ Destroys cache with a given name.
-
get
(key, key_hint: object = None) → Any¶ Retrieves a value from cache by key.
Parameters: - key – key for the cache entry. Can be of any supported type,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
Returns: value retrieved.
-
get_all
(keys: list) → list¶ Retrieves multiple key-value pairs from cache.
Parameters: keys – list of keys or tuples of (key, key_hint), Returns: a dict of key-value pairs.
-
get_and_put
(key, value, key_hint=None, value_hint=None) → Any¶ Puts a value with a given key to cache, and returns the previous value for that key, or null value if there was not such key.
Parameters: - key – key for the cache entry. Can be of any supported type,
- value – value for the key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- value_hint – (optional) GridGain data type, for which the given value should be converted.
Returns: old value or None.
-
get_and_put_if_absent
(key, value, key_hint=None, value_hint=None)¶ Puts a value with a given key to cache only if the key does not already exist.
Parameters: - key – key for the cache entry. Can be of any supported type,
- value – value for the key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- value_hint – (optional) GridGain data type, for which the given value should be converted,
Returns: old value or None.
-
get_and_remove
(key, key_hint=None) → Any¶ Removes the cache entry with specified key, returning the value.
Parameters: - key – key for the cache entry. Can be of any supported type,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
Returns: old value or None.
-
get_and_replace
(key, value, key_hint=None, value_hint=None) → Any¶ Puts a value with a given key to cache, returning previous value for that key, if and only if there is a value currently mapped for that key.
Parameters: - key – key for the cache entry. Can be of any supported type,
- value – value for the key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- value_hint – (optional) GridGain data type, for which the given value should be converted.
Returns: old value or None.
-
get_best_node
(*args, **kwargs) → Callable¶
-
get_best_node_120
(*args, **kwargs)¶
-
get_best_node_130
(*args, **kwargs)¶
-
get_protocol_version
() → Union[Tuple, NoneType]¶ Returns the tuple of major, minor, and revision numbers of the used thin protocol version, or None, if no connection to the GridGain cluster was not yet established.
This method is not a part of the public API. Unless you wish to extend the pygridgain capabilities (with additional testing, logging, examining connections, et c.) you probably should not use it.
-
get_size
(peek_modes=0)¶ Gets the number of entries in cache.
Parameters: peek_modes – (optional) limit count to near cache partition (PeekModes.NEAR), primary cache (PeekModes.PRIMARY), or backup cache (PeekModes.BACKUP). Defaults to all cache partitions (PeekModes.ALL), Returns: integer number of cache entries.
-
name
¶ Lazy cache name.
Returns: cache name string.
-
put
(key, value, key_hint: object = None, value_hint: object = None)¶ Puts a value with a given key to cache (overwriting existing value if any).
Parameters: - key – key for the cache entry. Can be of any supported type,
- value – value for the key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- value_hint – (optional) GridGain data type, for which the given value should be converted.
-
put_all
(pairs: dict)¶ Puts multiple key-value pairs to cache (overwriting existing associations if any).
Parameters: pairs – dictionary type parameters, contains key-value pairs to save. Each key or value can be an item of representable Python type or a tuple of (item, hint),
-
put_if_absent
(key, value, key_hint=None, value_hint=None)¶ Puts a value with a given key to cache only if the key does not already exist.
Parameters: - key – key for the cache entry. Can be of any supported type,
- value – value for the key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- value_hint – (optional) GridGain data type, for which the given value should be converted.
-
remove_all
()¶ Removes all cache entries, notifying listeners and cache writers.
-
remove_if_equals
(key, sample, key_hint=None, sample_hint=None)¶ Removes an entry with a given key if provided value is equal to actual value, notifying listeners and cache writers.
Parameters: - key – key for the cache entry,
- sample – a sample to compare the stored value with,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- sample_hint – (optional) GridGain data type, for whic the given sample should be converted.
-
remove_key
(key, key_hint=None)¶ Clears the cache key without notifying listeners or cache writers.
Parameters: - key – key for the cache entry,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
-
remove_keys
(keys: list)¶ Removes cache entries by given list of keys, notifying listeners and cache writers.
Parameters: keys – list of keys or tuples of (key, key_hint) to remove.
-
replace
(key, value, key_hint: object = None, value_hint: object = None)¶ Puts a value with a given key to cache only if the key already exist.
Parameters: - key – key for the cache entry. Can be of any supported type,
- value – value for the key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- value_hint – (optional) GridGain data type, for which the given value should be converted.
-
replace_if_equals
(key, sample, value, key_hint=None, sample_hint=None, value_hint=None) → Any¶ Puts a value with a given key to cache only if the key already exists and value equals provided sample.
Parameters: - key – key for the cache entry,
- sample – a sample to compare the stored value with,
- value – new value for the given key,
- key_hint – (optional) GridGain data type, for which the given key should be converted,
- sample_hint – (optional) GridGain data type, for which the given sample should be converted
- value_hint – (optional) GridGain data type, for which the given value should be converted,
Returns: boolean True when key is present, False otherwise.
-
scan
(page_size: int = 1, partitions: int = -1, local: bool = False)¶ Returns all key-value pairs from the cache, similar to get_all, but with internal pagination, which is slower, but safer.
Parameters: - page_size – (optional) page size. Default size is 1 (slowest and safest),
- partitions – (optional) number of partitions to query (negative to query entire cache),
- local – (optional) pass True if this query should be executed on local node only. Defaults to False,
Returns: generator with key-value pairs.
-
select_row
(query_str: str, page_size: int = 1, query_args: Union[list, NoneType] = None, distributed_joins: bool = False, replicated_only: bool = False, local: bool = False, timeout: int = 0)¶ Executes a simplified SQL SELECT query over data stored in the cache. The query returns the whole record (key and value).
Parameters: - query_str – SQL query string,
- page_size – (optional) cursor page size. Default is 1, which means that client makes one server call per row,
- query_args – (optional) query arguments,
- distributed_joins – (optional) distributed joins. Defaults to False,
- replicated_only – (optional) whether query contains only replicated tables or not. Defaults to False,
- local – (optional) pass True if this query should be executed on local node only. Defaults to False,
- timeout – (optional) non-negative timeout value in ms. Zero disables timeout (default),
Returns: generator with key-value pairs.
-
settings
¶ Lazy Cache settings. See the example of reading this property.
All cache properties are documented here: Cache Properties.
Returns: dict of cache properties and their values.
-
pygridgain.client module¶
This module contains Client class, that lets you communicate with GridGain cluster node by the means of GridGain binary client protocol.
To start the communication, you may connect to the node of their choice
by instantiating the Client object and calling
connect()
method with proper
parameters.
The whole storage room of GridGain cluster is split up into named structures,
called caches. For accessing the particular cache in key-value style
(a-la Redis or memcached) you should first create
the Cache
object by calling
create_cache()
or
get_or_create_cache()
methods, than call
Cache
methods. If you wish to create a cache object
without communicating with server, there is also a
get_cache()
method that does just that.
For using GridGain SQL, call sql()
method.
It returns a generator with result rows.
register_binary_type()
and
query_binary_type()
methods operates
the local (class-wise) registry for GridGain Complex objects.
-
class
pygridgain.client.
Client
(compact_footer: bool = None, partition_aware: bool = False, **kwargs)¶ Bases:
object
This is a main pygridgain class, that is build upon the
Connection
. In addition to the attributes, properties and methods of its parent class, Client implements the following features:- cache factory. Cache objects are used for key-value operations,
- GridGain SQL endpoint,
- binary types registration endpoint.
-
__init__
(compact_footer: bool = None, partition_aware: bool = False, **kwargs)¶ Initialize client.
Parameters: - compact_footer – (optional) use compact (True, recommended) or full (False) schema approach when serializing Complex objects. Default is to use the same approach the server is using (None). GridGain binary protocol documentation on this topic: https://apacheignite.readme.io/docs/binary-client-protocol-data-format#section-schema
- partition_aware – (optional) try to calculate the exact data placement from the key before to issue the key operation to the server node: https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients The feature is in experimental status, so the parameter is False by default. This will be changed later.
-
affinity_version
= None¶
-
close
()¶
This property remembers Complex object schema encoding approach when decoding any Complex object, to use the same approach on Complex object encoding.
Returns: True if compact schema was used by server or no Complex object decoding has yet taken place, False if full schema was used.
-
connect
(*args)¶ Connect to GridGain cluster node(s).
Parameters: args – (optional) host(s) and port(s) to connect to.
-
create_cache
(settings: Union[str, dict]) → pygridgain.cache.Cache¶ Creates GridGain cache by name. Raises CacheError if such a cache is already exists.
Parameters: settings – cache name or dict of cache properties’ codes and values. All cache properties are documented here: Cache Properties. See also the cache creation example, Returns: Cache
object.
-
get_binary_type
(binary_type: Union[str, int]) → dict¶ Gets the binary type information from the GridGain server. This is quite a low-level implementation of GridGain thin client protocol’s OP_GET_BINARY_TYPE operation. You would probably want to use
query_binary_type()
instead.Parameters: binary_type – binary type name or ID, Returns: binary type description − a dict with the following fields: - type_exists: True if the type is registered, False otherwise. In the latter case all the following fields are omitted,
- type_id: Complex object type ID,
- type_name: Complex object type name,
- affinity_key_field: string value or None,
- is_enum: False in case of Complex object registration,
- schemas: a list, containing the Complex object schemas in format: OrderedDict[field name: field type hint]. A schema can be empty.
-
get_cache
(settings: Union[str, dict]) → pygridgain.cache.Cache¶ Creates Cache object with a given cache name without checking it up on server. If such a cache does not exist, some kind of exception (most probably CacheError) may be raised later.
Parameters: settings – cache name or cache properties (but only PROP_NAME property is allowed), Returns: Cache
object.
-
get_cache_names
() → list¶ Gets existing cache names.
Returns: list of cache names.
-
get_or_create_cache
(settings: Union[str, dict]) → pygridgain.cache.Cache¶ Creates GridGain cache, if not exist.
Parameters: settings – cache name or dict of cache properties’ codes and values. All cache properties are documented here: Cache Properties. See also the cache creation example, Returns: Cache
object.
-
get_protocol_version
() → Union[Tuple, NoneType]¶ Returns the tuple of major, minor, and revision numbers of the used thin protocol version, or None, if no connection to the GridGain cluster was not yet established.
This method is not a part of the public API. Unless you wish to extend the pygridgain capabilities (with additional testing, logging, examining connections, et c.) you probably should not use it.
-
partition_aware
¶
-
protocol_version
= None¶
-
put_binary_type
(type_name: str, affinity_key_field: str = None, is_enum=False, schema: dict = None)¶ Registers binary type information in cluster. Do not update binary registry. This is a literal implementation of GridGain thin client protocol’s OP_PUT_BINARY_TYPE operation. You would probably want to use
register_binary_type()
instead.Parameters: - type_name – name of the data type being registered,
- affinity_key_field – (optional) name of the affinity key field,
- is_enum – (optional) register enum if True, binary object otherwise. Defaults to False,
- schema – (optional) when register enum, pass a dict of enumerated parameter names as keys and an integers as values. When register binary type, pass a dict of field names: field types. Binary type with no fields is OK.
-
query_binary_type
(binary_type: Union[int, str], schema: Union[int, dict] = None, sync: bool = True)¶ Queries the registry of Complex object classes.
Parameters: - binary_type – Complex object type name or ID,
- schema – (optional) Complex object schema or schema ID,
- sync – (optional) look up the GridGain server for registered Complex objects and create data classes for them if needed,
Returns: found dataclass or None, if schema parameter is provided, a dict of {schema ID: dataclass} format otherwise.
-
random_node
¶
-
random_node_120
()¶
-
random_node_130
()¶
-
register_binary_type
(data_class: Type[CT_co], affinity_key_field: str = None)¶ Register the given class as a representation of a certain Complex object type. Discards autogenerated or previously registered class.
Parameters: - data_class – Complex object class,
- affinity_key_field – (optional) affinity parameter.
-
sql
(query_str: str, page_size: int = 1, query_args: Iterable[T_co] = None, schema: Union[int, str] = 'PUBLIC', statement_type: int = 0, distributed_joins: bool = False, local: bool = False, replicated_only: bool = False, enforce_join_order: bool = False, collocated: bool = False, lazy: bool = False, include_field_names: bool = False, max_rows: int = -1, timeout: int = 0)¶ Runs an SQL query and returns its result.
Parameters: - query_str – SQL query string,
- page_size – (optional) cursor page size. Default is 1, which means that client makes one server call per row,
- query_args – (optional) query arguments. List of values or (value, type hint) tuples,
- schema – (optional) schema for the query. Defaults to PUBLIC,
- statement_type –
(optional) statement type. Can be:
- StatementType.ALL − any type (default),
- StatementType.SELECT − select,
- StatementType.UPDATE − update.
- distributed_joins – (optional) distributed joins. Defaults to False,
- local – (optional) pass True if this query should be executed on local node only. Defaults to False,
- replicated_only – (optional) whether query contains only replicated tables or not. Defaults to False,
- enforce_join_order – (optional) enforce join order. Defaults to False,
- collocated – (optional) whether your data is co-located or not. Defaults to False,
- lazy – (optional) lazy query execution. Defaults to False,
- include_field_names – (optional) include field names in result. Defaults to False,
- max_rows – (optional) query-wide maximum of rows. Defaults to -1 (all rows),
- timeout – (optional) non-negative timeout value in ms. Zero disables timeout (default),
Returns: generator with result rows as a lists. If include_field_names was set, the first row will hold field names.
pygridgain.constants module¶
This module contains some constants, used internally throughout the API.
pygridgain.exceptions module¶
-
exception
pygridgain.exceptions.
BinaryTypeError
¶ Bases:
pygridgain.exceptions.CacheError
A remote error in operation with Complex Object registry.
-
exception
pygridgain.exceptions.
CacheCreationError
¶ Bases:
pygridgain.exceptions.CacheError
This exception is raised, when any complex operation failed on cache creation phase.
-
exception
pygridgain.exceptions.
CacheError
¶ Bases:
Exception
This exception is raised, whenever any remote Thin client operation returns an error.
-
exception
pygridgain.exceptions.
HandshakeError
(expected_version: Tuple[int, int, int], message: str)¶ Bases:
OSError
This exception is raised on GridGain binary protocol handshake failure, as defined in https://apacheignite.readme.io/docs/binary-client-protocol#section-handshake
-
__init__
(expected_version: Tuple[int, int, int], message: str)¶ Initialize self. See help(type(self)) for accurate signature.
-
-
exception
pygridgain.exceptions.
ParameterError
¶ Bases:
Exception
This exception represents the parameter validation error in any pygridgain method.
-
exception
pygridgain.exceptions.
ParseError
¶ Bases:
Exception
This exception is raised, when pygridgain is unable to build a query to, or parse a response from, GridGain node.
-
exception
pygridgain.exceptions.
ReconnectError
¶ Bases:
Exception
This exception is raised by Client.reconnect method, when no more nodes are left to connect to. It is not meant to be an error, but rather a flow control tool, similar to StopIteration.
-
exception
pygridgain.exceptions.
SQLError
¶ Bases:
pygridgain.exceptions.CacheError
An error in SQL query.
pygridgain.utils module¶
-
class
pygridgain.utils.
DaemonicTimer
(interval, function, args=None, kwargs=None)¶ Bases:
threading.Thread
Same as normal threading.Timer, but do not delay the program exit.
-
__init__
(interval, function, args=None, kwargs=None)¶ This constructor should always be called with keyword arguments. Arguments are:
group should be None; reserved for future extension when a ThreadGroup class is implemented.
target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.
name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.
args is the argument tuple for the target invocation. Defaults to ().
kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.
If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.
-
cancel
()¶ Stop the timer if it hasn’t finished yet.
-
run
()¶ Method representing the thread’s activity.
You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.
-
-
pygridgain.utils.
cache_id
(cache: Union[str, int]) → int¶ Create a cache ID from cache name.
Parameters: cache – cache name or ID, Returns: cache ID.
-
pygridgain.utils.
capitalize
(string: str) → str¶ Capitalizing the string, assuming the first character is a letter. Does not touch any other character, unlike the string.capitalize().
-
pygridgain.utils.
datetime_hashcode
(value: int) → int¶ Calculates hashcode from UNIX epoch.
Parameters: value – UNIX time, Returns: Java hashcode.
-
pygridgain.utils.
decimal_hashcode
(value: decimal.Decimal) → int¶ This is a translation of java.math.BigDecimal class hashCode() method to Python.
Parameters: value – pythonic decimal value, Returns: hashcode.
-
pygridgain.utils.
entity_id
(cache: Union[str, int]) → Union[int, NoneType]¶ Create a type ID from type name or field ID from field name.
Parameters: cache – entity name or ID, Returns: entity ID.
-
pygridgain.utils.
get_field_by_id
(obj: GenericObjectMeta, field_id: int) → Tuple[Any, pygridgain.datatypes.base.GridGainDataType]¶ Returns a complex object’s field value, given the field’s entity ID.
Parameters: - obj – complex object,
- field_id – field ID,
Returns: complex object field’s value and type.
-
pygridgain.utils.
hashcode
(string: Union[str, bytes]) → int¶ Calculate hash code used for identifying objects in GridGain binary API.
Parameters: string – UTF-8-encoded string identifier of binary buffer, Returns: hash code.
-
pygridgain.utils.
int_overflow
(value: int) → int¶ Simulates 32bit integer overflow.
-
pygridgain.utils.
is_binary
(value)¶ Check if a value is a pythonic representation of a Complex object.
-
pygridgain.utils.
is_hinted
(value)¶ Check if a value is a tuple of data item and its type hint.
-
pygridgain.utils.
is_iterable
(value: Any) → bool¶ Check if value is iterable.
-
pygridgain.utils.
is_pow2
(value: int) → bool¶ Check if value is power of two.
-
pygridgain.utils.
is_wrapped
(value: Any) → bool¶ Check if a value is of WrappedDataObject type.
-
pygridgain.utils.
process_delimiter
(name: str, delimiter: str) → str¶ Splits the name by delimiter, capitalize each part, merge.
-
pygridgain.utils.
schema_id
(schema: Union[int, dict]) → int¶ Calculate Complex Object schema ID.
Parameters: schema – a dict of field names: field types, Returns: schema ID.
-
pygridgain.utils.
select_version
(func)¶ This decorator tries to find a method more suitable for a current protocol version, before calling the decorated method. The object which method is being decorated must have get_protocol_version() method.
Parameters: func – decorated method, Returns: wrapper.
-
pygridgain.utils.
status_to_exception
(exc: Type[Exception])¶ Converts erroneous status code with error message to an exception of the given class.
Parameters: exc – the class of exception to raise, Returns: decorator.
-
pygridgain.utils.
unsigned
(value: int, c_type: _ctypes._SimpleCData = <class 'ctypes.c_uint'>) → int¶ Convert signed integer value to unsigned.
-
pygridgain.utils.
unwrap_binary
(client: Client, wrapped: tuple) → object¶ Unwrap wrapped BinaryObject and convert it to Python data.
Parameters: - client – connection to GridGain cluster,
- wrapped – WrappedDataObject value,
Returns: dict representing wrapped BinaryObject.