Apache Ignite is like a software version of a Swiss Army Knife. With so many features and uses, deploying and configuring it can confuse even an experienced developer. Until now.
GridGain Nebula was created to simplify Ignite. It’s a public cloud service designed to help developers deploy, run, monitor, manage, and troubleshoot in-memory applications running any flavor of Apache Ignite or GridGain in any environment.
Because so many different software editions and configurations are possible across Apache Ignite and GridGain, this article will simplify by using “Ignite” as a shorthand for any version of Ignite or GridGain deployed in any environment.
The Two Modes of GridGain Nebula
GridGain Nebula provides developers with two complementary approaches for working with Ignite clusters: create and monitor (Figure 1).
Figure 1. Two Modes for Working with Ignite Clusters in GridGain Nebula
- Create New Cluster: Create a fully-managed cluster in GridGain’s cloud. In this approach, you specify and provision the cluster which the GridGain team manages for you. This frees up your team to develop and run custom applications.
- Monitor Existing Cluster: Attach to a GridGain or Apache Ignite cluster running in your private or public environment. In this approach, Nebula provides Ignite developers and administrators with a visual interface to monitor, troubleshoot, maintain and scale clusters.
This blog is about features in GridGain Nebula that simplify the experience of working with Ignite in your environment throughout the application lifecycle.
Deploy Ignite
Deploying Ignite involves provisioning hardware or cloud instances, downloading and installing Ignite, and configuring Ignite according to the latest best practices. This can be done manually and you can use other features of GridGain Nebula later, to monitor and manage an application. Or you can use the wizard in Nebula (Figure 2) to simplify the entire Ignite lifecycle with automated provisioning of hardware in a cloud provider of your choice, installing Ignite, configuring it with best practices (e.g., SSL, security, and backups). Connect to the provisioned cluster as usual with all the standard Ignite clients, including Java, .Net, C++, Python, JDBC, and more.
Figure 2. GridGain Nebula Deployment Wizard
Experienced Ignite users are familiar with setting up a cluster in their private environments via the Ignite command line, without the simplicity offered by GridGain Nebula. Newer Ignite users will enjoy Nebula’s “Create New Cluster” functionality which provides a visual interface for setting up an optimized cloud-hosted cluster in as few as 7 minutes.
The process of setting up an Ignite cluster begins with a few important decisions, including the on premises or cloud data center where the cluster will reside, the node resource capacity, and the cluster size. Cluster configurations are based on best practices to help ensure that each cluster is configured properly from the start.
Optional configuration settings, such as storage data region allocation, enable developers to decide whether the cluster nodes should be persistent, in-memory, or a combination of the two. Security access lists, SSL, and authentication are also configurable.
The cluster will be provisioned by Nebula in less than 7 minutes, and then applications can be connected through a wide variety of client APIs and languages, including JDBC, Java thin client, Java thick client, Python, REST, .NET, and more. Connection templates provide sample snippets that can be tailored to the application code.
Run Ignite
After an Ignite cluster is deployed, someone needs to keep it running. It requires constant monitoring, maintaining the hardware, applying software upgrades and security patches, and resolving incidents. Nebula can help with all of that.
The clusters deployed with Nebula are actively managed by the GridGain support team. The team monitors cluster health and is responsible for maintenance, restarts, and recovery to maintain SLAs. Managed GridGain clusters include automated daily backups of cluster data.
Monitor and Visualize Processes in Ignite
GridGain Nebula can be used as an all-inclusive monitoring solution for Ignite clusters or as a complement to existing visualization tools like Grafana, Zabbix, and DataDog. The power of Nebula is that it knows Ignite like no other tool, and can collect and visualize the information other tools can’t easily access.
GridGain Nebula dashboards can be set up and customized to monitor key application performance metrics, (Figure 3), observe the current cluster state and trends, track processes such as rolling upgrades and data rebalancing. Customized alerts can be set up to watch hundreds of cluster, node, and storage metrics, and to send a notification to you and your team members if something is wrong.
Figure 3. GridGain Nebula Dashboard
A major value of GridGain Nebula is most clearly demonstrated by its ability to provide monitoring and visualization of nodes and a variety of metrics. Before Nebula, it was hard to troubleshoot or optimize clusters holistically. With Nebula, developers can quickly ascertain if their Ignite application is running efficiently (e.g., if all compute hardware is being used optimally).
The Compute screen is used to monitor compute tasks executed on the cluster (Figure 4). GridGain Nebula constantly checks for new tasks that were added and provides a list of the new tasks since the last update.
Figure 4. GridGain Nebula Compute Grid Screen
Manage Ignite
Nebula provides several tools to comprehensively manage your cluster right from the user interface.
To access the cluster data from UI, SQL screen can be used. It allows you to run SQL queries, view and insert data, manipulate the tables with SQL Data Definition Language (DDL). It can also show the status of running SQL queries, their execution plans, and allows you to cancel them if required
Figure 5. SQL Query Results and Statistics
To manage cluster caches and tables, the Caches screen can be used (Figure 6). A Cache List tab displays a table containing all active caches, including the name of the cache, its mode, and atomicity. The table supports filtering caches by name, mode, atomicity, or cache ID. You can also use this screen to recover the caches in case of a partition loss.
Figure 6. Caches Screen Allows Viewing and Managing Caches in a Cluster
If an application requires custom libraries for Ignite compute tasks, services, or any other reason, a feature called the Code Deployment in Nebula can be used to manage the lifecycle of the libraries and their dependencies (Figure 7). The code can be uploaded right in the Nebula UI or you can use Maven libraries or artifacts available on a Web server. Once the code source is uploaded, it can be deployed and redeployed to the cluster. Nebula will make sure all cluster nodes have the required version of the code, and will also take care of the versioning.
Figure 7. Code Deployment with GridGain Nebula
Troubleshoot Ignite
When problems arise, Nebula can help Ignite developers and cluster administrators to more easily find the root cause and fix the issue quickly. All tools used for monitoring and management, which were already mentioned, contribute to that. Another very powerful feature of Nebula that helps with troubleshooting is tracing.
The performance of operations under load can be traced during stress tests or in production to identify operations that do not meet SLAs. Traces and spans of underperforming operations can be examined and the root causes and bottlenecks can be pinpointed (Figure 8).
Figure 8. Transaction Tracing in GridGain Nebula to Find Performance Bottlenecks
A Tracing screen displays information about traces of a specific event recorded during cluster operation. The recorded information can include informational messages, error messages, duration of specific operations, stack traces, etc. This information is used to troubleshoot cluster operation or to identify latency issues. The screen displays a table containing all recorded traces, including the name of the event, its start time, duration, and number of spans.
Each trace consists of a tree of spans, which are individual units of work performed by the system to process events. A trace can include spans from multiple nodes. For most troubleshooting, only a small percentage of traces need to be collected to provide an operational overview without adversely affecting performance.
Simplifying Your Ignite Journey
Developers are at different stages in their Ignite journeys. Some have years of in-house expertise with both Ignite development and operations, and need minimal support and resources. Others are looking to partner with GridGain to augment their team’s Ignite expertise as they deploy enterprise-grade Ignite applications.
GridGain Nebula was designed to be used by both. Nebula can be used simply to observe what’s going on in an Ignite cluster. Observe, learn, and then optimize for better performance and utilization. Other organizations will rely on GridGain Nebula as their managed infrastructure during the entire Ignite application life cycle. Still others will start on their own and then embrace some or all of Nebula as they’re ready, scale their Ignite applications. Regardless of the path you choose, your applications utilize the public Ignite APIs and offer maximum choice and control over your in-memory computing investment.
Get Started with GridGain Nebula
Whether you are looking to monitor your in-production cluster or seeking to get a new application live quickly, Nebula provides substantial value to everyone on your development and operations team. Sign up for Nebula now and start using its Ignite management, monitoring and provisioning capabilities in just minutes. Your new GridGain Nebula account includes a $500 credit so you can test it out with your workloads at no charge to you.