Internet of Things (IoT) Databases and Analytics Powered by an In-Memory Computing Platform
Gartner expects the Internet of Things (IoT) to have over 20 billion connected things by 2020. This many connected devices will transmit information and data that will require an enormous amount of processing to derive value. To cope with this rapid growth, successful IoT database platforms will need a data architecture which leverages an in-memory computing platform. These architectures will address the significant challenges in terms of speed, scalability, variable workloads, and other issues created by IoT applications.
The Architecture of HTAP for IoT
The architecture of the Hybrid Transactional/Analytical Processing (HTAP) component of IoT analytics and transactions is similar to the Lambda architecture defined by Nathan Marz for Big Data applications. It takes advantage of both stream- and batch-processing methods. The Lambda architecture includes the following layers:
- A high-speed layer — a real-time processing and transactional engine (typically something like a caching system and a compute grid, such as Redis and Spark)
- A batch/storage layer— data storage with an analytical or historical processing engine, such as Hadoop with Hive
The Lambda architecture of the HTAP component of the Internet of Things deals with event-stream processing, fast analytics, and storing data for advanced and long-term historical analysis, when necessary.
To get started with IoT, combining multiple technologies can require a significant investment in terms of skill set. While it is possible to find people who know each of a variety of technologies, it is not easy to find people who know all of them. There is a lot of complexity involved.
The GridGain in-memory computing platform provides a way to simplify the HTAP architecture for IoT databases and analytics. It addresses the needs of both transactional and analytical processing and also provides persistency and event processing — all in a high-speed, linearly scalable platform. And GridGain is just one core technology with one skill set to learn.