GridGain vs. Hazelcast® Benchmarks

Introduction

GridGain Professional Edition 1.7.1 and Hazelcast® 3.7 (Open Source) were benchmarked on AWS EC2 using Yardstick configurations. Results for throughput, latency, and percentile probes measuring various metrics during benchmark execution were collected.

GridGain vs. Hazelcast Performance Summary

Deadlock Free Transactions: 61% – 129% higher operations/sec throughput
Transactional Operations: 11% – 83% higher operations/sec throughput
Atomic Operations: Put Benchmark: 13% – 24% higher operations/sec throughput
Put and Get Benchmark: Depending on configuration, either GridGain or Hazelcast performed slightly better
SQL Query: 13% – 27% higher operations/sec throughput

GridGain outperformed Hazelcast on all atomic and transactional cache operations and SQL-based cache queries except the Atomic Put and Get benchmark, which provided mixed results. The testing results are compiled below to help you make an informed decision about the best in-memory solution for your company.

AWS EC2 Configuration
Hardware c4.2xlarge
CPU 8
RAM 15 Gib
OS Ubuntu 14.04
Yardstick Configuration
Nodes 1 Client, 4 Servers
Threads 64
Backup 1, Synchronous
JDK Java 1.7.0_80

Deadlock Free Transactions (Optimistic, Serializable)

Hazelcast does not have support for deadlock-free transactions, so we are comparing Ignite deadlock-free transactions to Hazelcast pessimistic transactions.

Put Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 63,230 37,642 68%
2 30,325 17,270 75%
6 15,768 9,064 73%
10 12,575 6,268 101%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.01 1.72
2.11 3.70
4.06 7.06
5.09 10.21

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 27,801 17,228 61%
2 24,705 14,692 68%
6 13,866 6,881 101%
10 10,629 4,634 129%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
2.30 3.71
2.59 4.36
4.61 9.34
6.02 13.89

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Transactional Operations

Put Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 42,149 37,642 11%
2 26,836 17,270 55%
6 11,623 9,064 28%
10 7,922 6,268 26%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.51 1.72
2.38 3.70
5.50 7.06
8.08 10.21

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Pessimistic Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 21,895 17,228 27%
2 22,809 14,692 55%
6 11,169 6,881 62%
10 7,944 4,634 71%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
2.92 3.71
2.80 4.36
5.73 9.34
8.05 13.89

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Optimistic Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 46,672 28,662 62%
2 22,439 13,314 68%
6 10,220 6,154 66%
10 7,387 4,030 83%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
1.37 2.24
2.85 4.80
6.26 10.40
8.66 15.89

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Atomic Operations

Put Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 120,609 104,131 15%
2 91,388 80,090 14%
6 61,342 54,194 13%
10 51,118 40,975 24%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
0.53 0.61
0.70 0.80
1.04 1.18
1.25 1.56

Graphs: 1 key, 2 keys, 6 keys, 10 keys

Atomic Put and Get Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
# of keys Ignite Hazelcast Delta
1 71,146 67,756 5%
2 48,949 52,018 -5%
6 31,294 32,950 -5%
10 25,748 25,250 1%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
0.89 0.95
1.30 1.22
2.04 1.94
2.48 2.53

Graphs: 1 key, 2 keys, 6 keys, 10 keys

SQL Query Benchmarks

Throughput, operations/sec
Bigger number represents better throughput
Benchmark Ignite Hazelcast Delta
SQL Query 74,447 58,605 27%
SQL Query Put 69,807 61,646 13%
Latency, milliseconds
Smaller number represents better latency
Ignite Hazelcast
0.85 1.09
0.91 1.03

Graphs: SQL query, SQL query put