GridGain Metrics With Prometheus and Grafana

Senior Software Engineer

GridGain supports a wide range of integrations and tools that can be used to monitor its status. Among them, a popular tool is Grafana, a popular open source software for monitoring metrics and building dashboards. This tutorial shows how you can export GridGain metrics to Prometheus, which can then be connected to Grafana.

What You Need

This tutorial uses the following program versions:

GridGain Community Edition 8.9.1
Grafana OSS 10.3.1
Prometheus server 2.49.1

When using a different versions, required dependencies may be different.

GridGain Configuration

First, we need to configure GridGain to use a Prometheus exporter and create an http server for Prometheus to collect data from. In this example, we will launch GridGain from Java code, so we will be updating Maven dependencies to include Prometheus:

Add the following dependencies to your project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>test</groupId>
    <artifactId>test</artifactId>
    <version>1.0-SNAPSHOT</version>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>GridGain External Repository</id>
            <url>http://www.gridgainsystems.com/nexus/content/repositories/external</url>
        </repository>
    </repositories>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
        <ignite.version>8.9.1</ignite.version>
        <prometheus.version>0.16.0</prometheus.version>
        <opencensus.version>0.31.1</opencensus.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.gridgain</groupId>
            <artifactId>ignite-indexing</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.gridgain</groupId>
            <artifactId>ignite-core</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.gridgain</groupId>
            <artifactId>ignite-spring</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.gridgain</groupId>
            <artifactId>ignite-opencensus</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>io.opencensus</groupId>
            <artifactId>opencensus-exporter-stats-prometheus</artifactId>
            <version>${opencensus.version}</version>
        </dependency>
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient</artifactId>
            <version>${prometheus.version}</version>
        </dependency>
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_httpserver</artifactId>
            <version>${prometheus.version}</version>
        </dependency>
    </dependencies>
</project>

We use the following versions:

The latest version of Prometheus from the maven repository. We recommend updating it to the latest version if it is available.
The OpenCensus version included in GridGain 8.9.1. The included opencensus library is updated regularly and can be different for different GridGain verisons. You can check the version of the library for your version in the {IGNITE_HOME}/libs/ignite-opencensus folder. The opencensus-impl-X.XX.X file denotes the version you should use in the configuration.

Enable exporting of data. You can do it by configuring OpenCensusMetricExporterSpi in your configuration or from Java code:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation=" http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
    <bean id="httpServer" class="io.prometheus.client.exporter.HTTPServer">
        <constructor-arg type="java.lang.String" value="127.0.0.1"/>
        <constructor-arg type="int" value="1234"/>
        <constructor-arg type="boolean" value="true"/>
    </bean>

    <bean id="opencensusWrapper" class="org.springframework.beans.factory.config.MethodInvokingBean">
        <property name="staticMethod"
                  value="io.opencensus.exporter.stats.prometheus.PrometheusStatsCollector.createAndRegister"/>
    </bean>

    <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="metricsUpdateFrequency" value="1000"/>
        <property name="cacheConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="mycache"/>
                    <!-- Enable statistics for the cache. -->
                    <property name="statisticsEnabled" value="true"/>
                    <property name="backups" value="1"/>
                </bean>
            </list>
        </property>
        <property name="metricExporterSpi">
            <list>
                <bean class="org.apache.ignite.spi.metric.opencensus.OpenCensusMetricExporterSpi">
                    <property name="period" value="1000"/>
                </bean>
            </list>
        </property>
        <property name="dataStorageConfiguration">
            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
                <property name="metricsEnabled" value="true"/>
                <property name="defaultDataRegionConfiguration">
                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="name" value="Default_Region"/>
                        <property name="initialSize" value="#{4L * 1024 * 1024 * 1024}"/>
                        <property name="maxSize" value="#{4L * 1024 * 1024 * 1024}"/>
                        <property name="persistenceEnabled" value="true"/>
                        <property name="metricsEnabled" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="localPort" value="47500"/>
                <property name="localPortRange" value="10"/>
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <value>127.0.0.1:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

import io.opencensus.exporter.stats.prometheus.PrometheusStatsCollector;
import io.prometheus.client.exporter.HTTPServer;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteException;
import org.apache.ignite.lifecycle.LifecycleBean;
import org.apache.ignite.lifecycle.LifecycleEventType;
import org.apache.ignite.resources.IgniteInstanceResource;

import java.io.IOException;

public class Exporter implements LifecycleBean {
    private int port;
    public Exporter(int port) {
        this.port = port;
    }
    private HTTPServer server;

    @IgniteInstanceResource
    Ignite ignite;

    // Creates an exporter for Prometheus.
    // This method will be executed when the node starts.
    @Override
    public void onLifecycleEvent(LifecycleEventType evt) throws IgniteException {
        if (evt == LifecycleEventType.BEFORE_NODE_START) {

            ignite.log().info("Starting Prometheus collector");

            PrometheusStatsCollector.createAndRegister();
            try {
                server = new HTTPServer("localhost", port, true);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        else if (evt == LifecycleEventType.BEFORE_NODE_STOP) {
            if (server != null) {
                server.stop();
                server = null;
            }
        }
    }
}

Start GridGain nodes.

You should be able to access the metrics on the http server URL: http://localhost:1234.

Prometheus Server Configuration

To produce metrics ready for Grafana, we need to collect them on the Prometheus server. You can find instructions on using Prometheus in the Prometheus documentation. In this example we will modify the default prometheus.yml file to also collect data from GridGain:

Add the ignite-metrics job to make Prometheus collect GridGain metrics:
```
scrape_configs:
- job_name: ignite-metrics
```

List the addresses of all OpenCensus exporters for each node in the targets block:

static_configs:
- targets: ['localhost:1234', 'localhost:1235', 'localhost:1236']

Start your Prometheus Server:

prometheus.exe --config.file=./prometheus.yml

By default, the Prometheus server will start on port 9090. Open the http://localhost:9090 URL in your browser and use Prometheus to make sure the metrics are available:

Grafana Configuration

To proceed with the tutorial, you will need Grafana. Download a standalone executable from the website.

When you start Grafana, it will be available on the http://localhost:3000/login URL by default. Default credentials will be admin/admin. You must change them after the first login.

Go to Connections/Data sources section and select Add data source.
Add the Prometheus data source to Grafana. The image below shows the example parameters for this document:
Make sure the connection is configured correctly:

Grafana can now receive metrics from GridGain.

Importing a Sample Dashboard

For this tutorial, we have prepared a sample dashboard that displays a number of metrics for the cluster.

Open your data source in Grafana by going to the Connections/Data sources section and selecting a newly created data source.
Copy the data source ID from the URL:
Download the sample dashboard. Grafana imports metrics in a JSON format.

Open the downloaded dashboard JSON and edit the datasource.uid field to match your data source ID:

"datasource": {
  "type": "datasource",
  "uid": "e105d3fd-a91b-462e-b2e9-891dc66b8d78"
}

In Grafana, go to Dashboards/Create dashboard and click Import dashboard. Load the prepared dashboard file.

Grafana will import the dashboard and display the metrics on it:

© 2024 GridGain Systems, Inc. All Rights Reserved. Privacy Policy | Legal Notices. GridGain® is a registered trademark of GridGain Systems, Inc.
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.

Last updated on Apr 10, 2024