Starting the Apache Ignite Streaming Demo
The Apache Ignite Streaming Demo starts a simple application that simulates the trading activities of a stock exchange. The application receives executed market deals in real time from the PubNub Market Orders data stream and forwards the deals to a locally running Ignite cluster. The demo is designed for Docker environments, enabling you to deploy the application in minutes on your personal laptop. With this first step of the tutorial, you deploy an Ignite cluster, connect the cluster with the GridGain Nebula, and launch the application that streams market orders.
What You Need
Docker 19 or later, or Docker Desktop Community 2.3 or later
Docker Compose 1.25.5 or later
Java Developer Kit 8 or later
Apache Maven 3.3 or later
Git 2.24 or later
Your favorite IDE, such as IntelliJ IDEA or Eclipse
Download the Streaming Demo
First, you need to download the demo project to your local laptop:
Download and unzip the source repository or use Git to clone the repository:
git clone https://github.com/GridGain-Demos/ignite-streaming-monitoring-demo.git
Go into the project root directory:
Easy, right? Let’s move forward.
Start the Apache Ignite Cluster
Start a two-node Apache Ignite cluster, with each node running as a separate container:
docker-compose -f docker/ignite-cluster.yaml up -d --scale ignite-server-node=2
The cluster nodes will persist all the records on disk. For caching, the nodes are allowed to use no more than 150MB of memory.
The on-disk records are stored in the
ignite-streaming-monitoring-demo\work\db folder that is automatically created
when you start the cluster. Check
ignite-streaming-monitoring-demo\docker\ignite-cluster.yaml files for additional configuration settings.
Register the Cluster With GridGain Nebula
Through the rest of the tutorial, you use GridGain Nebula as your primary management and monitoring tool for Ignite clusters. You create an account with GridGain Nebula and interconnect your locally running Ignite cluster with the tool.
First, generate a new permanent token that you use to register the cluster with Nebula (the default token expires in 5 minutes after the cluster startup time):
Connect to the container of the first cluster node:
docker exec -it docker_ignite-server-node_1 bash
Go into the
/opt/gridgain/bin/folder of the container:
Generate and copy a new token to register the cluster with the tool:
Quit the node’s container:
Second, sign up with Nebula and interconnect it with the cluster:
Go to https://portal.gridgain.com.
Create an account to sign in into Nebula.
Register the cluster with Nebula by using the previously generated token.
Activate the cluster.
Now, you should see a dashboard similar to the following:
Launch Market Orders Application
In this final step of this part of the tutorial, you launch the application that streams executed market orders into the Ignite cluster. Make sure that you have an Internet connection so that the application can connect to the PubNub Market Order data stream.
Now, you compile, build, and start the application:
Use Maven to generate an application executable file from the source code of the project:
mvn clean package
Dockerize the application:
docker build -f docker/StreamingAppDockerfile -t ignite-streaming-app .
Deploy the application in Docker:
docker-compose -f docker/ignite-streaming-app.yaml up -d
Finally, you need to confirm that the application is, in fact, streaming data into the cluster by querying the records from the Nebula SQL screen:
Get the last 10 executed trades (as indicated in the example screenshot):
SELECT * FROM Trade ORDER BY order_date DESC LIMIT 10;
Get the most popular stocks among the buyers:
SELECT symbol, SUM(bid_price) as total FROM Trade GROUP BY(symbol) ORDER BY total DESC;
Get a list of the buyers that completed the most transactions:
SELECT buyer.id, first_name, last_name, sum(bid_price) as total FROM Trade JOIN Buyer ON trade.buyer_id = buyer.id GROUP BY (buyer.id) ORDER BY total DESC;
You set up a Nebula dashboard that displays storage-related metrics:
Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are either registered trademarks or trademarks of The Apache Software Foundation.