High-Performance Data Architectures for IoT – White Paper

The number of smart devices making up the Internet of Things (IoT) has soared. From traditional printers and scanners to refrigerators, from smart watches and clothing to self-driving cars and drones, Gartner expects the IoT to include over 20 billion connected things by 2020. With all these connected devices transmitting information that needs to be analyzed, companies will need to deploy data architectures that meet several critical challenges:

  • Scale – The growth in the number of devices will be relentless
  • Speed – Many IoT use cases will involve responding to the collected data in real-time
  • Distribution – Devices must remain connected at all times, from any location
  • Security – IoT platforms must protect the private information they collect and transmit
  • Omni-channel – IoT platforms must be able to use the same pathways and backends for a variety of devices, whether phones, watches, laptops or cars
  • Variable workloads – Perhaps the biggest challenge in IoT use cases, platforms must be able to execute both analytical and transactional workloads in real time

In-Memory Computing HTAP: The High-Performance Data Architecture for IoT

One of the most promising technologies for solving the challenge of variable workloads in IoT is a hybrid transactional and analytical processing (HTAP) layer that is capable of event-stream processing, fast analytics, and storing data for advanced and long-term historical analysis.

Similar to the way the Lambda architecture works for Big Data applications, IoT devices communicate either batch or streaming data through a message bus to an HTAP layer for transactional and analytical workloads. The HTAP layer then provides fast analytics and transactional processing to backend users, third-party clients, and other devices.

Creating an HTAP architecture today is possible through expensive proprietary technologies or by cobbling together a number of different open source technologies. For example, the SMACK stack combines Apache Spark, Mesos, Akka, Cassandra, and Kafka.

A simpler strategy, however, and one that places far less stress on budgets or IT staff to learn several new applications, is the GridGain in-memory computing platform, which handles the needs of both transactional and analytical processing, and provides persistency and event processing — all in a high-speed, linearly scalable platform. The key modules of the GridGain in-memory computing platform are that relevant to IoT use cases are:

  • Data grid – Essentially an in-memory key value store that can be queried
  • Compute grid - A stateless grid that provides high-performance computation in memory using clusters of computers and parallel processing
  • Service grid - A service grid in which grid service instances are deployed across the distributed data and compute grids
  • Streaming – The ability to consume an endless stream of information and process it in real-time
  • Advanced clustering – The ability to automatically discover nodes, eliminating the need to restart the entire cluster when adding new nodes

IoT platforms must be capable of providing real-time insights into massive amounts of streaming data to power useful actions at the device level. The GridGain in-memory computing platform can simplify the deployment of a Lambda architecture with a full featured HTAP architecture. And with GridGain, there is just one core technology to learn.

High-Performance Data Architectures for the Internet of Things White Paper

If your organization is developing or trying to improve an IoT infrastructure, please download High-Performance Data Architectures for the Internet of Things, a new GridGain Systems white paper that takes a detailed look at IoT platform requirements and how in-memory computing can deliver the performance and scale IoT use cases demand.

As always, if you have questions or comments, please let us know!