InetSoft Webinar: How Much Data Can You Visualize In-Memory

This is the continuation of the transcript of a Webinar hosted by InetSoft in May 2017 on the topic of "How Data Discovery Software Uncovers Stories in the Data." The speaker is Abhishek Gupta, Product Manager at InetSoft.

That concludes the formal part of the webinar. We committed to stay over a little bit to answer questions. So we are going to pick up questions now, and let’s take a look at what we have? And, yeah, feel free to email additional questions to Here we have got one question about how much data can you visualize in-memory, which comes up every time.

In-memory technology has expanded tremendously. Our compression algorithms are pretty good. We are fitting tens of millions of rows on a normal windows class machine whether it's a client or server, and we’re often pulling in 60, 80, 100 tables from something like an Oracle database on a nightly basis and linking and joining them and doing roll ups and whatever else we have to do.

We are not going to put you of Wal-Mart transactions in-memory. But if you’re doing analysis you probably don’t want to look at that anyway you probably roll up the transaction level to the day level if you’re looking at three years of data. And if you are looking at products, probably you could roll up to the class or the sub class instead of the SKU.

dashboard demo
View a 2-minute demonstration of InetSoft's easy, agile, and robust BI software.

Even for the very large data sets when you get down to solving business problems and doing analysis, there are ways of handling large data sets in a way that’s appropriate to solve the problem. And we really know the size you can put in-memory really has been constrained, and these examples I’m running with have a couple of million claims. I am using a laptop and performances are sub second on an interaction which is what we design for. Let’s look another question. What kind of problems are you involved in?

We are working in all kinds of industries both direct and via partners. We tried to share examples in this Webinar from four different industries. The common theme is people who need to drill into explore the data, to answer questions that can be anything from fundraising or sales. For example, a frontline business development manager who is going out on a trip to Los Angles and wants to know who he or she should visit.

You want to take a list of 100,000 and cut it first geographically by Los Angeles. Then maybe filter for if they gave last year but haven’t this year. There are a common set of cuts where you’re doing a list reduction that’s why fairly it’s a common application that spans a lot of industries.

Another one is the more metrics based kinds of problems like with the call center where you have got a lot of calls. You have got a lot of callers. You have got different segments of customers. In that case of calling alumni, there were different segments of customers. In another setting, you’re trying to see who is doing well and who is not. It is not as simple.

I am under on pledges maybe I am under on pledges, but I have got a tougher group to call. Maybe I have been calling during the bad time of day for these customers. My shift should be shifted. So it's those kinds of problems to summarize. Risk reduction where you take a massive amount of data and cut it down to get a list to do action on. It's not just one metric, but you eventually have to get all the way to details to take action. And I think the airline example of visual display is good, where you can’t find the problems by looking at a list. You need to see the patterns, and the patterns are a complex set of metric that with color and shape they jump out, and you say I’ve got a problem in the north east.

Okay, here is another question. How many levels can you observe in a theoretical visualization that you show?

We can take the raw data instead of hierarchies. When you saw the example of the bar chart on giving, I don’t think I still have it open. Where we had the gift brackets like 10,000 to 50,000, now back in the hierarchy where you have gift buckets at one level. You can right click and adjust the gift amounts so the buckets would explode into all detail.

The cross hierarchy selection also works because we are doing that off the detail data in the raw table. We are not pre summarizing it, and we’re not working on the cube. The beauty of the in-memory analysis is you’re working on the raw data. So you have got this incredible ability of select subsets of one part of one hierarchy and mash it up with another.

Some of the charts also you will show hierarchy. A bubble chart can show what they are. A heat map is actually a chart that shows hierarchy. We saw the groups of fund categories, sub groups in a fund family. Then we are coloring individual funds with the selection of fund levels. You could see how the funds fit into the families and the categories, and it's fairly quick visual way to say hey a good part of my international funds is where the problem is.

Read why choosing InetSoft's cloud-flexible BI provides advantages over other BI options.