The New Digital Experience thanks to In-Memory Computing, Big Data & HTAP

How Digital Business, Big Data, HTAP and In-Memory Computing Came Together to Improve the Customer Experience:

The New Digital Experience – How Digital Business, Big Data, HTAP and In-Memory Computing Came Together to Improve the Customer ExperienceI’ve had two long-standing professional interests for half my life; middleware and customer experience management.  I am happy to say that not only is there a focus at the executive level on improving the customer experience. The technologies needed have evolved to a point that make it possible to actually improve the experience.  But in working with and observing many companies, I’ve come to realize there still isn’t a great understanding of how to use these technologies together, and of how IT needs to supports this transformation.

Few companies have pieced together all the technologies needed to improve the customer experience. For example, most organizations by now understand the importance of API management for digital business, and the importance of Big Data for understanding the customer. But these two technologies are separate in most companies. They’re handled by different digital business and Big Data analytics teams. In general, operations and analytics are separate.

Keeping operations and analytics separate is a problem. If you’re going to improve the customer experience the way Amazon, Expedia and other “Internet Economy” companies do it, you need to be using Big Data Analytics during an interaction, in real-time. This is what some people refer to as hybrid transactional/analytical processing (HTAP). Companies have shown that best way to succeed with real-time HTAP is with in-memory computing.

How do you know if you have this problem? Try and find a project using API Management with a technology like Apache Spark and in-memory computing platform like GridGain or Apache Ignite for a customer-facing application. If you can’t, you have a technology and architecture problem.

Part of the root cause of this problem is that there are too many separate groups. If we learned anything from SOA, data governance or more recently digital business, we learned that it’s important to bring together the people, processes and technologies together into a group like a competency center whose primary job is to focus on and solve the problem. Look for the teams, and then how they work together. Who lays out a roadmap and chooses the right projects and technology investments at the right time for your company? If you can’t find executive sponsors, IT heads and architects all focused on this, you have an organizational problem.

This series of blogs on “The New Digital Experience” is meant to share how companies have succeeded with various stages of their transformations both in terms of how they adopted the right technologies and the best practices around them. It’s meant for the chief customer officers, chief digital officers, CTOs and chief architects involved in putting together the people, processes and technologies needed to improve the customer experience.

The first set of blogs are meant to explain how these companies set themselves up for success:

  • The Math and Science of Improving the Customer Experience – The companies that succeeded analyzed the customer experience, understood how much the customer was worth and mapped out the better experience
  • The Architectural Foundation of Customer Experience Management – They also identified how operational technologies like API management and transactional applications fit together with big data analytics, streaming analytics and machine learning using in-memory computing to improve the customer experience
  • The Roadmap for HTAP and In-Memory Computing – They successfully laid out a roadmap on how to start with in-memory computing to handle the load of digital business (Part 1), build new transactional and operational applications (Part 2), and then add intelligence and automate decisions to improve the customer experience (Part 3)

The second set of blogs are meant to detail the best practices across the different stages of projects, and how different companies built a new in-memory computing layer that helped transition from separate operations and analytics to HTAP across their customer-facing channels. These blogs will cover three stages:

  • Part 1 – Adding Speed and Scalability to Existing Applications (Handing the Reads)
  • Part 2 – Build new transactional and operational applications (Handing the Writes)
  • Part 3 – Automating Your Way to a Better Experience (Automating Decisions)

Along the way the blogs will try to highlight the insights from some of the companies about their lessons learned around people, processes and technologies; the good, the bad and the ugly.

But before we begin the journey, we do need to explain one key assumption: that one way to improve the customer experience is with HTAP, and the proven way to implement real-time HTAP is using in-memory computing.  We’ll kick that discussion off with a blog and a Webinar.

Rob Meyer
Rob Meyer is senior director of outbound product management at GridGain Systems.