GridGain Developers Hub

Starting the Apache Ignite Streaming Demo

Head of Developer Relations at GridGain
Apache Ignite Committer and PMC Member

The Apache Ignite Streaming Demo starts a simple application that simulates the trading activities of a stock exchange. The application receives executed market deals in real time from the PubNub Market Orders data stream and forwards the deals to a locally running Ignite cluster. The demo is designed for Docker environments, enabling you to deploy the application in minutes on your personal laptop. With this first step of the tutorial, you deploy an Ignite cluster, connect the cluster with the hosted version of GridGain Control Center, and launch the application that streams market orders.

What You Need

  • Docker 19 or later, or Docker Desktop Community 2.3 or later

  • Docker Compose 1.25.5 or later

  • Java Developer Kit 8 or later

  • Apache Maven 3.3 or later

  • Git 2.24 or later

  • Your favorite IDE, such as IntelliJ IDEA or Eclipse

Download the Streaming Demo

First, you need to download the demo project to your local laptop:

  • Download and unzip the source repository or use Git to clone the repository:

    git clone https://github.com/GridGain-Demos/ignite-streaming-monitoring-demo.git
  • Go into the project root directory:

    cd ignite-streaming-monitoring-demo

Easy, right? Let’s move forward.

Start the Apache Ignite Cluster

Start a two-node Apache Ignite cluster, with each node running as a separate container:

docker-compose -f docker/ignite-cluster.yaml up -d --scale ignite-server-node=2

The cluster nodes will persist all the records on disk. For caching, the nodes are allowed to use no more than 150MB of memory. The on-disk records are stored in the ignite-streaming-monitoring-demo\work\db folder that is automatically created when you start the cluster. Check ignite-streaming-monitoring-demo\config\ignite-config-node.xml and ignite-streaming-monitoring-demo\docker\ignite-cluster.yaml files for additional configuration settings.

Register the Cluster With GridGain Control Center

Through the rest of the tutorial, you use GridGain Control Center as your primary management and monitoring tool for Ignite clusters. You create an account with Control Center and interconnect your locally running Ignite cluster with the tool.

First, generate a new permanent token that you use to register the cluster with Control Center (the default token expires in 5 minutes after the cluster startup time):

  1. Connect to the container of the first cluster node:

    docker exec -it docker_ignite-server-node_1 bash
  2. Go into the /opt/gridgain/bin/ folder of the container:

    cd /opt/gridgain/bin/
  3. Generate and copy a new token to register the cluster with the tool:

    ./management.sh --token
  4. Quit the node’s container:

    exit

Second, sign up with the hosted version of Control Center and interconnect it with the cluster:

  1. Go to https://control.gridgain.com.

  2. Create an account to sign in into Control Center.

  3. Register the cluster with Control Center using the previously generated token.

  4. Activate the cluster.

Now, you should see a dashboard similar to the following:

Control Center Dashboard

Launch Market Orders Application

In this final step of this part of the tutorial, you launch the application that streams executed market orders into the Ignite cluster. Make sure that you have an Internet connection so that the application can connect to the PubNub Market Order data stream.

Now, you compile, build, and start the application:

  1. Use Maven to generate an application executable file from the source code of the project:

    mvn clean package
  2. Dockerize the application:

    docker build -f docker/StreamingAppDockerfile -t ignite-streaming-app .
  3. Deploy the application in Docker:

    docker-compose -f docker/ignite-streaming-app.yaml up -d

Finally, you need to confirm that the application is, in fact, streaming data into the cluster by querying the records from the Control Center SQL screen:

Control Center SQL Screen
  • Get the last 10 executed trades (as indicated in the example screenshot):

    SELECT * FROM Trade ORDER BY order_date DESC LIMIT 10;
  • Get the most popular stocks among the buyers:

    SELECT symbol, SUM(bid_price) as total FROM Trade GROUP BY(symbol) ORDER BY total DESC;
  • Get a list of the buyers that completed the most transactions:

    SELECT buyer.id, first_name, last_name, sum(bid_price) as total FROM Trade
    JOIN Buyer ON trade.buyer_id = buyer.id
    GROUP BY (buyer.id) ORDER BY total DESC;

What’s Next

You set up a Control Center dashboard that displays storage-related metrics: