GridGain/Apache Ignite community update

GridGain's Denis Magda, left, and Confluent's Viktor Gamov pose for a selfie before their joint talk at the NYC In-Memory Computing Meetup on June 26 in Manhattan.After a long holiday weekend, our GridGain experts are gearing up for more events next week. We’re scheduling the next Bay Area In-Memory Computing Meetup for July 17 and the NYC In-Memory Computing Meetup for July 25.

I'll share details on those later this week. In the meantime, here's a recap of a very busy last week of June.

Monday, June 24: Bay Area In-Memory Computing Meetup. GridGain’s Valentin (Val) Kulichenko delivered a talk titled, "Best Practices for Native Persistence and Data Recovery."

As an in-memory computing platform, Val explained that GridGain and Apache® Ignite™ support native persistence that stores data and indexes transparently on non-volatile memory, SSD or disk. When persistence is enabled, memory becomes a cache for the most frequently used data and indexes. Native persistence is ACID-compliant, durable and enables immediate availability on a restart of each node. Data is never lost; GridGain supports full and incremental snapshots along with continuous archiving, and provides Point-in-Time recovery to an individual transaction.

In the video of his talk, Val shares insights into the underlying architecture and best practices for implementing native persistence in production. He covers:

- An architectural overview of native persistence, and centralized backup and recovery
- Tips and tricks for for configuring and managing persistence
- Best practices for checkpointing, using the Write-Ahead Log (WAL) and restoring from a failure
- Performance tuning recommendations to balance durability and performance, including how to create snapshots under load

Tuesday, June 25: Data Natives Oslo Meetup. GridGain's Yury Babak was in Olso, where he delivered a talk titled, "Distributed Machine and Deep Learning at Scale with Apache Ignite." Yury is the head of ML/DL framework development at GridGain and an Apache Ignite committer. In his presentation, he explained how Apache Ignite and GridGain address limitations like ETL costs, scaling issues and Time-To-Market for the new models and help achieve near-real-time, continuous learning.

Wednesday, June 26: NYC In-Memory Computing Meetup. Denis Magda, vice president of the Apache Ignite PMC and director of product management at GridGain Systems, was one of the featured speakers at the June 26 IMC Meetup in Manhattan.

Denis teamed up with Viktor Gamov, developer advocate at Confluent, to deliver a “how-to” presentation for building a real-time alerting, analytics and reporting system (at scale).

We also have the video of their talk. In it, you’ll see how integrating Apache Kafka with Apache Ignite speeds and simplifies projects associated with building a system (or upgrading one) capable of:

- handling unbound streams of data
- offer real-time alerting,
- capable of storing terabytes and petabytes of data
- capable of acting on the and data within milliseconds.

Denis explained that integrating Apache Kafka with Apache Ignite solves most of your Big Data and Fast Data requirements faster and easier. A battle-tested recipe is simple -- take Kafka Connect and have your data stream through Kafka pipelines, add a pinch of KSQL to act on the streams with SQL in real-time with zero delays, rinse and flush the pre-processed data in Ignite as in-memory databases and get further insights by analyzing your hot and cold datasets.

Denis and Viktor also demonstrate how to implement the solution in practice. They explain architectural reasoning and the benefits of real-time integration and share common usage patterns.

If you’d like Denis, Val or Yury to speak at your meetup, just let me know in the comment section below. Better yet, if YOU have a talk around distributed systems for one of our meetups, please share it with me and the community on the GridGain Forums.