Nikita Ivanov, GridGain co-founder and CTO, provides insight on choosing the best in-memory solution for your apps in a recent Meetup in the Bay Area. He discusses different types of in-memory systems, including RDBMS in-memory options, in-memory databases, data grids and data fabrics, while taking several questions from the audience. You will learn about the key features to zero in on when selecting the right solution for your environment.


Okay. So, what I’m going to do, I’ll probably slate between me and Dmitriy and Konstantin we’ll probably about 30 minutes and we’re going to have a five minute break, then I think Demetri will go up, and we’ll take a five-minute break after that, then Konstantin will finish up.

So, we have three talks today about in-memory computing in general. So, what I’m going to do first is – it’s vital how we choose in-memory computing solution for you apps, but I’m going to spend a little bit more time in the beginning about what’s a in-memory computing. I assume most of you know. So, I’m not going to be giving many marketing speeches here.

We’ll kind of quickly go technical overview quite a few slides about some of the myths as far as what we see in the in-memory computing field. I’ve been working about ten years in this technology. We’ll talk about some of the product categories, and we’ll finish up with some of the IMC features.

So, before I jump into – when we talk about how to choose something best, I am like pretty biased, I founded this company, I work on the projects and whatnot. So, when I compare the features of Product A and Product B and saying who’s better and which works, every vendor has a website. You can find all the comparisons right there. We have comparisons with some of the competing products, open source and commercial.

You can go to any of the competing projects of other companies to find the same thing. So, that’s not the purpose. I’m not going to be comparing X and Y. But, what I’m going to be talking today is some of the auxiliary thinking about it, how to choose, which directions to concentrate, which are some of the features without comparing it are important for you.

So, what is a memory config? And probably most of you probably have a kind of, you know, subconscious understanding of that, right? You boost up the RAM and things go fast. Typically that’s exactly how you use RAM, but over here, so if you want to get down technically, in-memory computing systems basically use high performance distributed memory systems for compute and transact on live data sets in real-time orders of magnitude faster than is possible with traditional flash disk-based structure.

So, almost every word here is kind of important. We’re talking about a high-performance distributed RAM, or high performance goes without saying. RAM is pretty much high performance. Why are we talking about distribution? It’s kind of an interesting point. we’re talking about distributed memory because typically, even today, the single server doesn’t have enough RAM to do anything interesting.

So, most of the systems, most of the in-memory systems historically and today have been distributed in nature. It’s kind of an interesting fact that in-memory goes with distribution hand in hand. In fact, ideally that modern in-memory systems are probably one of the most advanced distributed systems in existence.

Just to give you an example, literally six years ago our system was running in production I’m talking about GridGain and Apache Ignite, and over 2000 nodes in the transaction cluster. I would challenge you to find any other systems with 2000 nodes in a fully transactional cluster in financial real-time systems. They can do that.

And it’s, for in-memory systems it’s the necessity. Not something we brag about or do for some reason. It’s necessity. Typically a single server doesn’t enough RAM, we have to cluster very effectively with a lot of brains behind it, really combine RAM from all the quadrants. We’re talking about the transacting computer. We’re talking about this, but that’s a very important point.

We’re not talking about only a computational thing or basically data storage, we’re talking about both data storage, which is processing in the memory system. It’s very important because there’s a lot of drive for a convergent storage systems where we not only store data in something, we’re also processing data in the same place where we store data.

But, most importantly, if there’s anything you want to know about binary computing it’s speed. Nothing else matters. We’re only using in-memory computer for one fundamental reason, to get speed and performance. Nothing else can give you orders of magnitude performance. Think about this. It’s not two times faster or three times or five times. We’re talking about hundreds and thousands time faster compared to flash and disk. That’s the key.

I always like to give you a circle and kind of a look back, you know, why – not only me by the way, you talk to any analyst, they’ll tell the same story that in-memory coputing is a paradigm shift. Think about a story very quickly, 50s, right? Before 50s in the late 40s or early 50s we didn’t have any external storage.

You were tied to your program, whatever pulled the strings on the computers, and it goes into RAM, you power off, the program is gone. So, there was a very radical idea in the early 50s, what about external storage? So, RAM needs to find a way – in our old companies, if you’re hunting RAM, basically it’s a weapons manufacturer.

Back then they developed the first magnetic tape device for storing data outside of computers. That’s where we started the era of external storage. Fast forward to the 60s and late 60s, early 70s, disks become a big deal, and again, this IBM 340 systems came around probably the 70s and that revolutionized the whole data storage because all of a sudden you get a blazingly fast disks back them, we called them HS multiple disks, the same kind of device, and what else happened? What else happened in the mid to late 70s?

The SQL publishing happened. There was a tiny change in how you stored data and how to process data. Disks came around and were very affordable, relatively affordable, and SQL won the kind of war between multiple pro disk hard process data. So, we’ve got the affordable disks, and we’ve got an extended base processing for the data.

2000s role in — the 80s was nothing happening, 90s nothing happening — but in the late 90s Toshiba developed a new technology. It took a lot of time to really get the economics right, but somewhere by mid-2000s live storage pretty much two-fold storage for a lot of devices, look at your iPhones, look at your tablets, look at your iPads and tablets and laptops, predominantly they have today live storage, you’re not even thinking about this.

The flash is there inside. Somewhere around 2010, give or take, most analysts believe we entered the era in-memory computing. And that’s kind of interesting. There are multiple reasons for that transition. We can probably spend another hour talking about it, but I usually define them in two reasons, through economics and demand. The economics of 64 basically use became available, and 64 basically allows you to use a lot of RAM in a single plug.

The price of RAM also dropped down. It’s dropping down 20 percent year over year, so around that time all of a sudden the RAM, traditional RAM become available. But, also the demand of things, how is the old RAM [unintelligible] a lot of data, why now in the real-time.

So, in-memory was the focal point. What also happening in the same time, 2010, give our take is the NoSQL revolution, right? Although some people kind of realized the majority of data we’re generating today will be unstructured, and therefore the world will need unstructured data is upon us.

That’s why I do believe, and not only me, and basically I’ll give you one more idea, that in-memory computing as evolutional point, it is also the last frontier of the data storage now. It may sound a little bit rough. Think about this. Where else can you store data that will be faster? Think about a tradition from the 50s to today, storing from nano tapes to magnetic disks, to disks and the flash and other mechanical devices.

We went from flash to the DRAM, where else? We can put it on the CPU cab, but that’s very small. It’s only about two times faster than the ER4. Think about this, there is nowhere else to put the data from a high level perspective in the current architecture of computers to make it faster or closer to application.

So, that’s why a lot of people think that flash is traditional point natural thing, but the byte-addressable storage which is the data you ran without the new batch of RAM coming up, both hybrid DR and flash, but byte-addressable storage on mother board on the memory controller is the fundamental last frontier where to store data.

Critical thing, Gartner has a very nice way of saying this: basically, RAM is the new disk and disk is the new tape. You know, mostly analysts predict we’re going to use disks only for the backup, but we use data tapes for long storage backups, disks are rapidly becoming a backup device. Another take on this, and think about this, I’m giving kind of different directions and different angles to look at the memory. Another way to look at in-memory– why all of sudden memory, only memory can be so fast is this.

Not only memory’s fast, right, memory’s fast, everybody kind of naturally understands this, but where does application performance come from? This is pretty good indication of that. Think about the disk-first of architecture. Now, you always keep saying in memory, disk-first memory. Memory systems are never memory only, there’s always a disk somewhere — flash, spinning disk, for backup. But think about a disk first architecture, imagine you have an application that needs to have access to a piece of data that’s on the disk? What happens? The application calls the operating system, the operating system calls the I/O subsystem, the I/O subsystem makes a call to an I/O controller, and the I/O controller makes a call to a device. Device has a seek time you find on the block, then what? Then you basically have to move this block up the chain all the way back to application, the application has to demarshall this product from bytes to the object.

And voila, you’ve got the object all the way through the device and all the way back, pretty long. Now, think about this, what happens when you have data in RAM? About the only thing that happens is a pointer [unintelligible]. You just point to a different set, different location in RAM, and most importantly you don’t do anything at all, and most importantly, say it twice, you don’t have any demarshalling happening. It’s right there in exactly the format you need.

This is the difference, this is the fundamental difference between basically having data on the disk on a flash software, which is essential to the block network device, and versus having the data right there in RAM. It is the same space where your application is. This is the fundamental reason you get those two, three, sometimes four orders of magnitude when you basically go into the memory.

Let me talk about some of the myth of memory computing. The first one, absolutely too expensive. Every time we are Apache Ignite project introducing somebody to our GridGain company, almost immediate reaction is, god damn, it’s too expensive. It’s probably a myth by now. Think about this, yes, RAM is still relatively more expensive than a flash device given the same capacity, but think about this.

I have it on a slide, but I always keep asking out, I basically pose the question, how do you think a terabyte of RAM costs today? You can see it there, but you’d be surprised if I talk to an audience and ask them, a terabyte of RAM, quarter million dollars, about 10, 20 hands in the audience, and that’s roughly 10 times wrong.

The terabyte of RAM today, I’m talking about the full class, I’m not just talking about the RAM sheet, I’m talking about the entire cluster. Let’s say ten blades, each blade has probably about a hundred gigabyte of RAM, the power of a network indication, everything, is about $20, $25K. Think about this, a terabyte DDR-3 with all the processing capacity, all the disk, all the power, everything else, $20K.

Just to give you a perspective, a number of years ago, I think two or three years ago, maybe four years ago, the entire working set of Twitter, which is the whole global tweets for a week, was about seven terabytes. Now, think about this, for the price of a decent Tesla you can buy enough RAM and processing capacity to have literally a sub-second SLA on any analytics on an entire Twitter work itself. The economics are there. Yes, it’s not free. I also keep saying that speeding disks up real literally and figuratively are free today. You can get whatever.

It’s actually a bit more expensive, DRAM is still relatively more expensive, but in a larger terms the too expensive kind of medium is no longer true, it’s literally a myth. And honestly we have new types of memory coming out Toshiba, Micron, any of those guys that are dealing with Hitachi memory. There is a a lot of memory coming out, anywhere from Intel to explore and provide different types of memory all trying to optimize cross-performance ratio.

So, the price of RAM will be going dramatically down and down and down. We’re going to have new types of RAM fully compatible with the DDR, DDR post. But, it’s a little bit different performance if you need an extra fast memory you can pay a lot less literally this year or next year too. Not durable.

That’s not basically made by – you know, all of us thinking it’s a RAM, so if I unplug the computer, it’s gone. Technically, unless you use non-volatile RAM, which is available today by the way, not really supported by its software, but technically you’re right. If you unplug the DDR stick from power it loses the power, it loses the content.

The point is that almost any even rudimentary in-memory system, let along something like Apache Ignite, which is extremely advanced in a sense, it has a latitude ability in persistence building, and you can configure as much as you like of durability options in the products and the projects. You can basically keep the copy of the same data on multiple nodes. You can see them as they go as increments, they flashing through the disk in times of storage disk databases.

You can do whatever you want. You can actually make a memory configured system as durable as traditional disk-based database. You’re not going to get much performance because of that because it’s basically going to reduce itself into the disk space systems. But, you have all these extremes.

What we do find a lot in use cases is that there’s plenty of use cases and we’ll ask the customers, lots of data sets don’t have to be persistent at all. They can be completely lost and they’re meant for disaster or something, and they can be re-created, a lot of financial stuff is like this. Obviously, keeping transaction on disk and whatnot, but a lot of intermittent stuff doesn’t have to be linking in dramatic performance because of that.

Flash is fast enough. Another interesting – by the way, this is in my opinion not really as much as a myth, but it’s a judgement problem. And I always keep saying when I’m talking to customers, talking to users, it’s a very simple thing. If you’re looking for a two, three, maybe four time performance increase, don’t look at -inmemory.

Buy a SSD, plug it in, you get it right there. Things will go a little faster, writes doesn’t stack much faster, but the reads will go much faster. If you’re looking for a 10X, 100X, 1000X performance increase, flash will not help, it was never designed for it, it will never do it for you.

The only option you have is to really go with in-memory computing, and the difference between – you know, people are constantly asking, who needs a 10X, think about this. If a process takes 40 seconds, what does 10X performance increase? Three? You know, in real-time. Think about this, the current processing takes a half a minute, if you want to go in real-time you have to be somewhere a sub-second. It’s actually more than 10X performance you have to realize.

Nothing will give it to you. You can derive your apps in assembly, it’s not going to help. You cannot buy a 10X more powerful CPU unless you spend literally tens of millions of dollars. There’s nothing else available to you, you as an engineer or developer other than that go with in-memory processor. That can give you 10X sometimes. We’ve seen up to almost tens of thousands increase performance.

So, Flash is fast enough sometimes. If you need a small moderate improvement flash is perfect. If you need to break out 10X or more, there’s nothing else available. Another interesting myth about it is, actually in-memory computer is only for caching. And actually it’s been true if you look back at history, even like 15 years ago, predominant use case for everything was caching. It’s very natural. Databases have caching since the day one, it’s a very natural idea.

I’d have to go to the disk to keep something that used elements right there in RAM. But, today caching actually is kind of used, obviously, but because there’s so many apps already have cached Americans becoming really moving away from caching, just caching, it becomes a system of records where you transacted that, where you store data.

For example, a project like Apache Ignite supports both transaction, key-value, license key, file system, and memory. Full SQL support, full data capability, full NDP, NDI integration HDC title processing. It’s a lot of acronyms now you can do, a lot of use cases you can process on the same data in RAM, and the cache is just one of them. Of course you could cache. Anything that can be erased can be used for caching, which is basically intuitive there into some of the projects we’re discussing today.

So, we see a lot of usage, and most of our usage for the last two to three years has been away from cache, and for a variety of different system record use cases where people transact, run the SQL, run the transactions from computations complex computations.

So, let me talk about some of the categories. We talk a lot about, what is in-memory computing, some of the basic [unintelligible] still may be using that. So, what are some of the objects? Keep in mind, I’m not talking right now about channel marketing view of in-memory because there’s in-memory analytics, in memory BI, anything else.

I’m just talking about middleware. So, I’m talking about in-memory middleware. I know that those are four options that most of us can basically defense. First one’s a memory, literally in-memory options. It typically builds a just a buying options to existing databases.

In a typical corporation it will be Oracle, Microsoft server, I think IBM has an option for some kind of in-memory [unintelligible]. Okay. We’ll talk about them each individually. In-memory RDB events, a typical example is MemSQL, those are typical databases that’s been either rewritten or upgraded with in-memory storage, but those are traditional signal databases.

The next one in-memory datagrids. This is kind of minimal bona fide to build architecture from scratch, in-memory configured middleware. It’s specifically designed for in-memory first. It’s no database, datagrid it’s not an option, it’s from the ground up middleware developed specifically for in-memory processing.

The last one in-memory data fabric, which is an extension of in-memory data grid it adds stuff to it and make it a little bit more strategic on what it can do. So, we’ll talk about each one of them. In-memory option, so first one it’s literally an option. If you cache Oracle if you just want to spend a couple hundred thousand dollars or more, you can go to Oracle, buy the option, enlabel that and some tables can store data in RAM things will go faster in some use cases.

The key here it’s a feature in the existing databases. There’s absolutely zero integration, there is literally no need to do anything, no API changes, no code changes, no data migration. The accounts of that is extremely limited capabilities. It’s literally – I would never call it in-memory computing option, but it exists today. But, literally it’s more or less just an optimization with an existing database that you can basically get.

But, from a marketing lingo, it’s there, so we’ll talk briefly about it but that’s about it. So, in-memory databases, in-memory [unintelligible]. This is kind of from me the first where it started making any kind of sense. So, if you guys for example look at mySQL, and you’re going to have a mySQL with in-memory storage there’s a company like MemSQL that did that for you.

So, if you want to stay with SQL, if you don’t want to move anywhere, if you want to just basically just have SQL, nothing else, just SQL but you want to have a different database with in-memory storage, there are a couple companies, VoltDB and MemSQL are pretty cool projects you guys can look at.

What’s the downside? Well, the downside is pretty massive. You have to move to a different database. So, if any of you work in an organization and try to replace Oracle with something else, try to replace maybe two Microsoft databases. They’re pretty big deal to replace it.

So, it could be multiple years of projects. So, this is the biggest [unintelligible]. And you’re not getting a lot in return because you’re just getting the same database that can work faster in certain use cases, but you have to migrate the entire data, not the entire back office application potentially incompatible with your database.

Another very significant problem with in-memory databases, and actually removing in-memory from databases is scalability. SQL doesn’t scale. It was never designed for it. It doesn’t scale in a true scale. You can shard it, but it’s a very primitive way and best to say look, a portion of my data is here, a portion of my data is there, but it’s a very primitive way.

SQL doesn’t scale, it doesn’t scale because of joints. If your tables are pretty big and joints are pretty big, you basically face the same fundamental problem of keys exploding, you have to exchange keys everywhere. So, yeah, if your tables are pretty small, everything works. If your tables go into the millions you forget about some of the joints. You’re going to kill either the network or RAM in both.

So, one of the problems with a SQL databases doesn’t matter. Disk or memory. They do not scale. Definitely not. Take a look at Oracle, anybody using Oracle REX or Oracle partition Oracle databases. That’s the reason, typically REX can go in [unintelligible] but not [unintelligible].

You’ve never seen a database that can go scale out to 200 or 300 nodes. It’ll be pretty nice because it’ll give you access parallelization but you can’t in SQL, SQL doesn’t scale.

Audience: But that has nothing to do with SQL. It’s an implementation artifact. We’ve been implementing SQL in a stupid and naïve way for 30, 40 years. That’s not inherently the relational model and that’s not inherently the language. SQL is a language and it’s an interface. It’s not the engine.

Interviewer: I agree with you. It’s a good point. So, I’m talking about SQL implementation, not SQL the language.

Audience: Fair enough.

Interviewer: But, guess what, if Oracle cannot make it work effectively, the problem is not trivial. There are quite a few companies that are trying to make it – we for example, do distribute SQL, but bound by the same limitations as well.

Anyways, in every day use this is the area where it starts a little bit become more easy because in-memory data grids is the first kind of what I call bona fide in-memory middleware. It was designed from scratch to be specific for in-memory processing. The big plus of in-memory data grids is that they optimize the performance and they provide all the specific APIs and paradigms to effectively in-memory processing. It has again a key-value access to the RAM, which is pretty close [unintelligible], it typically has some transaction that may or may not have some seek capabilities.

But, it is the systems that are designed from scratch to be a perfect fit for in-memory computing. The typical negative side of that is that all of a sudden at this point it will require a potentially significant integration effort. Because the difference between the key-value and proprietary APIs between that and SQL for example is pretty significant.

Some of the risks may support a subset of SQL, some other query language, but the integration effort is no longer just an hour or a day. It could be weeks and it could be months sometimes. So, this is a major kind of negative side, if you will, of data grids and essentially data fabric as well.

They do require a substantial integration effort. In return you can get those 10X, 100X, 1000X performance increase from the applications. And finally, in-memory fabric. So, the in-memory data fabrics are built on top of data rates, and the difference there, they provide more functionality, more capabilities than just data rates. Typical data topics would have a full SQL support, a full streaming and processing.

They will have things like file system integration, Hadoop integration or Spark integration, a variety of different things. So, fabrics really treat in-memory as kind of a strategic layer of performance. It’s based on RAM. It can put a lot of data into it, then you can do whatever practically you want with this data, right.

You load up in RAM across multiple computers, you can do key value access, you can do SQL, you can do transactions, non-transactions, you can do NPP processing, high performance parallelization, you can do Spark integrations, you can do stuff like this. So, whenever we talk about in data fabrics, and I’m limited by a [unintelligible] because Apache Ignite projects, which I’m involved with is a memory data fabric.

The idea is there is we just move away from just data grid use case and treat RAM as a strategic storage layer with all the different types of processing. You can do on top of all of that. So, this is the kind of a summary of what – you know, four options, and those again, despite the fact that I’m kind of being negative on the first one, it’s always a judgment call you have to make.

What really works for you? For your project? Sometimes database options are perfectly fine, it’s not often that I’ve seen that, and it’s working, and obviously companies like VoltDB, MemSQL, they’re pretty successful, they do something right, there’s plenty of customers for that. We also basically see a lot of use of data grids and data fabrics. But I might be bragging here.

I have a last slide here, and if I can borrow five more minutes. Some of the key figures in data integration fabrics and I’m not going to be telling you who is doing the best and worst, you guys can decide for yourself. But some of the interesting capabilities you have to be able to look at.

First of all, scale out and scale up. Basically, horizontal then vertical scale. It’s very, very important because depending on what your application is doing sometimes when you have a clear and a horizontal scale out, scale out to tens, sometimes hundreds of nodes, but sometimes you want to have your app or your middleware supporting a vertical scale.

What if having one of those producer boxes with five terabyte of RAM on –

Audience: Sixty-four terabytes.

Interviewer: Sixty-four terabytes, yeah. There’s plenty of computers available today with that kind of architecture. You have to be able to work on that box as well, so it may sound obvious, why wouldn’t you, right? But, believe me, there is a fundamental difference in how you treat a [unintelligible] scale system or a [unintelligible] scale system.

One example, on a box there is no network overhang, everything is encased in this box, so you have to really test this and really have your bender or the project you guys are using to really tell me if it’s working or not. We really learned the pretty hard way that it’s absolutely not automatic.

We thought of this, it’s far away from being automatic, if you work in one environment, would you work in this environment? You have to design your system, you have to fix a lot of stuff. So, that’s really important consideration to really see if you are aware of choice to do that.

Well, you’ve got some memory data grids, things like SQL support, things like transactions, things like SQL support, those are pretty obvious, but in most of the projects to support this in some way shape or form. But, let me give you an example about SQL, always ask what kind of SQL are you supporting? There’s a pretty big deal between a full SQL 99 support where your transition to this systems can be very painless, versus some kind of sublanguage that looks like SQL with a long, long list of exceptions. If you’re taking out your Oracle SQL, you’re going to spend days and days and days really trying to bend over to make it work with some kind of proprietary language.

So, supporting, for example, instead of SQL 99 something like this can significantly shorten any kind of transaction, any kind of integration. Cross language interrupt, I mean, most of our stuff today’s written in a bunch of language, right. Your front-end Javascript, your back in Scala/Java, C++, .NET structure.

I don’t think you can find either where it works everywhere, but the projects that actually have either clients on the language. Look at MongoDB, why it’s so wonderful? Of course use cases, it’s the ecosystem of drivers with every god damn language that exists on earth. That’s the beauty of MongoDB — you can basically use it everywhere.

So, in a [unintelligible] project, in a [unintelligible] with in-memory computing systems, your backend can be .NET, it can be C++ straight, it can be Java, it can be Scala. Your front end can be no [unintelligible]. Always look out what’s available ’cause if it’s not available you’re in a world of pain basically develop yourself, and it sometimes can be pretty hard.

When it comes to compute grids, look at what’s available, and typically people just make it some kind of random interface available to you. But, it’s not enough. Compute grids just as complex as data grids in terms of computation semantics. Based on all the indications that came to be [unintelligible]. All those different types of processing, how they fail over, how do you balance, a variety of different questions.

Obviously, if you guys into the Enterprise usage, things like security and it goes without saying manageability. Look at what’s needed, things like Spark and Hadoop. It’s hard to find a project today that’s not touched by Hadoop ecosystem or Spark ecosystem, somewhere, somewhere you guys are doing something with it.

It’s nice when middleware provides us support with that. Because those are very key elements in typical big data ecosystems, and you have to have some type of integrations with it. I’m out of time, so this is my last slide. I’m not going to be taking questions, but I’m going to be around here because I’m out of time, and I think Dmitriy will be next, but we’ll take a five minute break or not?

Audience: Yeah.

Interviewer: All right. Five minute break and then the next one’s coming, thank you guys.
Talk Date