org.gridgain.grid.spi.deployment.uri
Class GridUriDeploymentSpi

java.lang.Object
  org.gridgain.grid.spi.GridSpiAdapter
      org.gridgain.grid.spi.deployment.uri.GridUriDeploymentSpi

All Implemented Interfaces:

GridDeploymentSpi, GridUriDeploymentSpiMBean, GridSpi, GridSpiJsonConfigurable, GridSpiManagementMBean

@GridSpiInfo(author="GridGain Systems",
             url="www.gridgain.com",
             email="support@gridgain.com",
             version="3.6.0e.13012012")
@GridSpiMultipleInstancesSupport(value=true)
public class GridUriDeploymentSpi
extends GridSpiAdapter
implements GridDeploymentSpi, GridUriDeploymentSpiMBean

Implementation of GridDeploymentSpi which can deploy tasks from different sources like file system folders, FTP, email and HTTP. There are different ways to deploy tasks in grid and every deploy method depends on selected source protocol. This SPI is configured to work with a list of URI's. Every URI contains all data about protocol/transport plus configuration parameters like credentials, scan frequency, and others.

When SPI establishes a connection with a URI, it downloads deployable units to the temporary directory in order to prevent it from any changes while scanning. Use method setTemporaryDirectoryPath(String)) to set custom temporary folder for downloaded deployment units.

SPI tracks all changes of every given URI. This means that if any file is changed or deleted, SPI will re-deploy or delete corresponding tasks. Note that the very first apply to GridUriDeploymentSpi.findResource(String) findClassLoader(String)} is blocked until SPI finishes scanning all URI's at least once.

There are several deployable unit types supported:

• GAR file.
• Local disk folder with structure of unpacked GAR file.
• Local disk folder containing only compiled Java classes.

GAR file

GAR file is a deployable unit. GAR file is based on ZLIB compression format like simple JAR file and its structure is similar to WAR archive. GAR file has '.gar' extension.

GAR file structure (file or directory ending with '.gar'):

      META-INF/
              |
               - gridgain.xml
               - ...
      lib/
         |
          -some-lib.jar
          - ...
      xyz.class
      ...

• META-INF/ entry may contain gridgain.xml file which is a task descriptor file. The purpose of task descriptor XML file is to specify all tasks to be deployed. This file is a regular Spring XML definition file. META-INF/ entry may also contain any other file specified by JAR format.
• lib/ entry contains all library dependencies.
• Compiled Java classes must be placed in the root of a GAR file.

GAR file may be deployed without descriptor file. If there is no descriptor file, SPI will scan all classes in archive and instantiate those that implement GridTask interface. In that case, all grid task classes must have a public no-argument constructor. Use GridTaskAdapter adapter for convenience when creating grid tasks.

By default, all downloaded GAR files that have digital signature in META-INF folder will be verified and deployed only if signature is valid.

URI

This SPI uses a hierarchical URI definition. For more information about standard URI syntax refer to java.net.URI documentation.

[scheme:][//authority][path][?query][#fragment]

Every URI defines its own deployment repository which will be scanned for any changes. URI itself has all information about protocol, connectivity, scan intervals and other parameters.

URI's may contain special characters, like spaces. If encodeUri flag is set to true (see GridUriDeploymentSpi.setEncodeUri(boolean)), then URI 'path' field will be automatically encoded. By default this flag is set to true.

Configuration

GridUriDeploymentSpi has the following optional configuration parameters (there are no mandatory parameters):

• Temporary directory path where scanned GAR files and directories are copied to (see setTemporaryDirectoryPath(String)).
• List of URIs to scan (see GridUriDeploymentSpi.setUriList(List)). If not specified, then URI specified by DFLT_DEPLOY_DIR is used.
• Flag to control encoding of the 'path' portion of URI (see setEncodeUri(boolean)).

Protocols

Following protocols are supported in SPI:

• file:// - File protocol
• classes:// - Custom File protocol.
• ftp:// - File transfer protocol
• http:// - HTTP protocol
• https:// - Secure HTTP protocol

In addition to SPI configuration parameters, all necessary configuration parameters for selected URI should be defined in URI. Different protocols have different configuration parameters described below. Parameters are separated by ';' character.

File

For this protocol SPI will scan folder specified by URI on file system and download any GAR files or directories that end with .gar from source directory defined in URI. For file system URI must have scheme equal to file.

Following parameters are supported for FILE protocol:

Parameter	Description	Optional	Default
freq	File directory scan frequency in milliseconds.	Yes	5000 ms specified in DFLT_DISK_SCAN_FREQUENCY.

File URI Example

The following example will scan 'c:/Program files/gridgain/deployment' folder on local box every '5000' milliseconds. Note that since path has spaces, setEncodeUri(boolean) parameter must be set to true (which is default behavior).

file://freq=5000@localhost/c:/Program files/gridgain/deployment

Classes

For this protocol SPI will scan folder specified by URI on file system looking for compiled classes that implement GridTask interface. This protocol comes very handy during development, as it allows developer to specify IDE compilation output folder as URI and all task classes in that folder will be deployed automatically.

Following parameters are supported for CLASSES protocol:

Parameter	Description	Optional	Default
freq	File directory scan frequency in milliseconds.	Yes	5000 ms specified in DFLT_DISK_SCAN_FREQUENCY.

Classes URI Example

The following example will scan 'c:/Program files/gridgain/deployment' folder on local box every '5000' milliseconds. Note that since path has spaces, setEncodeUri(boolean) parameter must be set to true (which is default behavior).

classes://freq=5000@localhost/c:/Program files/gridgain/deployment

FTP

For FTP protocol SPI will scan and download only GAR files from source directory defined in URI. SPI doesn't scan FTP folders recursively. The following parameters are supported for FTP protocol:

Parameter	Description	Optional	Default
freq	FTP location scan frequency in milliseconds.	Yes	300000 ms specified in DFLT_FTP_SCAN_FREQUENCY.
username:password	FTP username and password specified in standard URI server-based authority format.	No	---

FTP URI Example

Here is an example of an FTP URI that connects identified as username:password to 'localhost' on port '21', with initial path set to 'gridgain/deployment'

ftp://username:password;freq=10000@localhost:21/gridgain/deployment

HTTP URI Example

The following example will scan 'gridgain/deployment' folder with on site 'www.mysite.com' using authentication 'username:password' every '10000' milliseconds.

http://username:password;freq=10000@www.mysite.com:110/gridgain/deployment

Java Example

GridUriDeploymentSpi needs to be explicitly configured to override default local deployment SPI.

 GridUriDeploymentSpi deploySpi = new GridUriDeploymentSpi();

 GridConfigurationAdapter cfg = new GridConfigurationAdapter();

 List<String> uris = new ArrayList<String>(5);

 uris.add("http://www.site.com/tasks");
 uris.add("ftp://ftpuser:password;freq=10000@localhost:21/gg-test/deployment");
 uris.add("file://freq=20000@localhost/c:/Program files/gg-deployment");
 uris.add("classes:///c:/Java_Projects/myproject/out");

 // Set URIs.
 deploySpi.setUriList(uris);

 // Override temporary directory path.
 deploySpi.setTemporaryDirectoryPath("c:/tmp/grid");

 //  Override default deployment SPI.
 cfg.setDeploymentSpi(deploySpi);

 //  Start grid.
 G.start(cfg);
 

Spring Example

GridUriDeploymentSpi can be configured from Spring XML configuration file:

 <bean id="grid.custom.cfg" class="org.gridgain.grid.GridConfigurationAdapter" singleton="true">
         ...
         <property name="deploymentSpi">
             <bean class="org.gridgain.grid.spi.deployment.uri.GridUriDeploymentSpi">
                 <property name="temporaryDirectoryPath" value="c:/tmp/grid"/>
                 <property name="uriList">
                     <list>
                         <value>http://www.site.com/tasks</value>
                         <value>ftp://ftpuser:password;freq=10000@localhost:21/gg-test/deployment</value>
                         <value>file://freq=20000@localhost/c:/Program files/gg-deployment</value>
                         <value>classes:///c:/Java_Projects/myproject/out</value>
                     </list>
                 </property>
             </bean>
         </property>
         ...
 </bean>
 

For information about Spring framework visit www.springframework.org

Wiki & Forum:

Wiki
Forum

See Also:

GridDeploymentSpi

Field Summary

static String	DEPLOY_TMP_ROOT_NAME Default temporary directory name relative to file path GridUriDeploymentSpi.setTemporaryDirectoryPath(String)} (value is gg.uri.deployment.tmp).
static String	DFLT_DEPLOY_DIR Default deployment directory where SPI will pick up GAR files (value is work/deployment/file).
static int	DFLT_DISK_SCAN_FREQUENCY Default scan frequency for file:// and classes:// protocols (value is 5000).
static int	DFLT_FTP_SCAN_FREQUENCY Default scan frequency for ftp:// protocol (value is 300000).
static int	DFLT_HTTP_SCAN_FREQUENCY Default scan frequency for http:// protocol (value is 300000).
static String	XML_DESCRIPTOR_PATH Default task description file path and name (value is META-INF/gridgain.xml).

Constructor Summary

GridUriDeploymentSpi()

Method Summary

GridDeploymentResource	findResource(String rsrcName) Finds class loader for the given class.
protected List<String>	getConsistentAttributeNames() Returns back a list of attributes that should be consistent for this SPI.
String	getTemporaryDirectoryPath() Gets temporary directory path.
List<String>	getUriList() Gets list of URIs that are processed by SPI.
boolean	register(ClassLoader ldr, Class<?> rsrc) Registers a class loader with this SPI.
void	setEncodeUri(boolean encodeUri) Indicates that URI must be encoded before usage.
void	setListener(GridDeploymentListener lsnr) Sets or unsets deployment event listener.
void	setTemporaryDirectoryPath(String tmpDirPath) Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.
void	setUriList(List<String> uriList) Sets list of URI which point to GAR file and which should be scanned by SPI for the new tasks.
void	spiStart(String gridName) This method is called to start SPI.
void	spiStop() This method is called to stop SPI.
String	toString()
boolean	unregister(String rsrcName) Unregisters all class loaders that have a class with given name or have a class with give GridTaskName value.

Methods inherited from class org.gridgain.grid.spi.GridSpiAdapter

assertParameter, checkConfigurationConsistency, configInfo, createSpiAttributeName, getAuthor, getGridGainHome, getLocalNodeId, getName, getNodeAttributes, getSpiContext, getStartTimestamp, getStartTimestampFormatted, getUpTime, getUpTimeFormatted, getVendorEmail, getVendorUrl, getVersion, injectables, onContextDestroyed, onContextInitialized, registerMBean, setJson, setName, startInfo, startStopwatch, stopInfo, unregisterMBean

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.gridgain.grid.spi.GridSpi

getName, getNodeAttributes, onContextDestroyed, onContextInitialized

Methods inherited from interface org.gridgain.grid.spi.GridSpiJsonConfigurable

setJson

Methods inherited from interface org.gridgain.grid.spi.GridSpiManagementMBean

getAuthor, getGridGainHome, getLocalNodeId, getName, getStartTimestamp, getStartTimestampFormatted, getUpTime, getUpTimeFormatted, getVendorEmail, getVendorUrl, getVersion

Field Detail

DFLT_DEPLOY_DIR

public static final String DFLT_DEPLOY_DIR

Default deployment directory where SPI will pick up GAR files (value is work/deployment/file). Note that this path relative to GRIDGAIN_HOME folder.

See Also:

Constant Field Values

DFLT_DISK_SCAN_FREQUENCY

public static final int DFLT_DISK_SCAN_FREQUENCY

Default scan frequency for file:// and classes:// protocols (value is 5000).

See Also:

Constant Field Values

DFLT_FTP_SCAN_FREQUENCY

public static final int DFLT_FTP_SCAN_FREQUENCY

Default scan frequency for ftp:// protocol (value is 300000).

See Also:

Constant Field Values

DFLT_HTTP_SCAN_FREQUENCY

public static final int DFLT_HTTP_SCAN_FREQUENCY

Default scan frequency for http:// protocol (value is 300000).

See Also:

Constant Field Values

XML_DESCRIPTOR_PATH

public static final String XML_DESCRIPTOR_PATH

Default task description file path and name (value is META-INF/gridgain.xml).

See Also:

Constant Field Values

DEPLOY_TMP_ROOT_NAME

public static final String DEPLOY_TMP_ROOT_NAME

Default temporary directory name relative to file path GridUriDeploymentSpi.setTemporaryDirectoryPath(String)} (value is gg.uri.deployment.tmp).

See Also:

Constant Field Values

Constructor Detail

GridUriDeploymentSpi

public GridUriDeploymentSpi()

Method Detail

setTemporaryDirectoryPath

@GridSpiConfiguration(optional=true)
public void setTemporaryDirectoryPath(String tmpDirPath)

Sets absolute path to temporary directory which will be used by deployment SPI to keep all deployed classes in.

If not provided, default value is java.io.tmpdir system property value.

Parameters:

tmpDirPath - Temporary directory path.

setUriList

@GridSpiConfiguration(optional=true)
public void setUriList(List<String> uriList)

Sets list of URI which point to GAR file and which should be scanned by SPI for the new tasks.

If not provided, default value is list with file://${GRIDGAIN_HOME}/work/deployment/file element. Note that system property GRIDGAIN_HOME must be set. For unknown GRIDGAIN_HOME list of URI must be provided explicitly.

Parameters:

uriList - GAR file URIs.

setEncodeUri

@GridSpiConfiguration(optional=true)
public void setEncodeUri(boolean encodeUri)

Indicates that URI must be encoded before usage. Encoding means replacing all occurrences of space with '%20', percent sign with '%25' and semicolon with '%3B'.

If not provided, default value is true.

Parameters:

encodeUri - true if every URI should be encoded and false otherwise.

getTemporaryDirectoryPath

public String getTemporaryDirectoryPath()

Gets temporary directory path.

Specified by:

getTemporaryDirectoryPath in interface GridUriDeploymentSpiMBean

Returns:

Temporary directory path.

getUriList

public List<String> getUriList()

Gets list of URIs that are processed by SPI.

Specified by:

getUriList in interface GridUriDeploymentSpiMBean

Returns:

List of URIs.

setListener

public void setListener(GridDeploymentListener lsnr)

Sets or unsets deployment event listener. Grid implementation will use this listener to properly add and remove various deployments.

Specified by:

setListener in interface GridDeploymentSpi

Parameters:

lsnr - Listener for deployment events. null to unset the listener.

spiStop

public void spiStop()
             throws GridSpiException

This method is called to stop SPI. After this method returns kernel assumes that this SPI is finished and all resources acquired by it are released.

Note that this method can be called at any point including during recovery of failed start. It should make no assumptions on what state SPI will be in when this method is called.

Specified by:

spiStop in interface GridSpi

Throws:

GridSpiException - Thrown in case of any error during SPI stop.

spiStart

public void spiStart(String gridName)
              throws GridSpiException

This method is called to start SPI. After this method returns successfully kernel assumes that SPI is fully operational.

Specified by:

spiStart in interface GridSpi

Throws:

GridSpiException - Throws in case of any error during SPI start.

Parameters:

gridName - Name of grid instance this SPI is being started for (null for default grid).

findResource

@Nullable
public GridDeploymentResource findResource(String rsrcName)

Finds class loader for the given class.

Specified by:

findResource in interface GridDeploymentSpi

Parameters:

rsrcName - Class name or class alias to find class loader for.

Returns:

Deployed class loader, or null if not deployed.

register

public boolean register(ClassLoader ldr,
                        Class<?> rsrc)
                 throws GridSpiException

Registers a class loader with this SPI. This method exists to be able to add external class loaders to deployment SPI. Deployment SPI may also have its own class loaders. For example, in case of GAR deployment, every GAR file is loaded and deployed with a separate class loader maintained internally by the SPI.

The array of classes passed in should be checked for presence of GridTaskName annotations. The classes that have this annotation should be accessible by this name from GridDeploymentSpi.findResource(String) method.

Specified by:

register in interface GridDeploymentSpi

Throws:

GridSpiException - If registration failed.

Parameters:

ldr - Class loader to register.

rsrc - Class that should be checked for aliases. Currently the only alias in the system is GridTaskName for task classes; in future, there may be others.

Returns:

True if resource was registered.

unregister

public boolean unregister(String rsrcName)

Unregisters all class loaders that have a class with given name or have a class with give GridTaskName value.

Specified by:

unregister in interface GridDeploymentSpi

Parameters:

rsrcName - Either class name or GridTaskName value for a class whose class loader needs to be unregistered.

Returns:

True if resource was unregistered.

getConsistentAttributeNames

protected List<String> getConsistentAttributeNames()

Returns back a list of attributes that should be consistent for this SPI. Consistency means that remote node has to have the same attribute with the same value.

Overrides:

getConsistentAttributeNames in class GridSpiAdapter

Returns:

List or attribute names.

toString

public String toString()

Overrides:

toString in class Object