What Are Some of the Problems with Traditional Business Intelligence?

Below is the continuation of the transcript of a Webinar hosted by InetSoft on the topic of Traditional BI vs Agile BI. The presenter is Mark Flaherty, Vice President of Marketing at InetSoft.

Telcos and Mobile Data

Telcos, for example, increasingly are getting more and more data especially now that not just associated with phone calls, but for mobile devices all the kinds of data activities are tracked. Now have the ability to locate where people are with their mobile devices given that most of those devices have a location sensor in them, and so this additional data that can be taken advantage of from mapping applications.

Other industries like retail and pharmaceuticals are well known for generating large data volumes. If you just take one of those Web data examples, social networks, as you can see here the world is most definitely obsessed with Facebook. Here’s a snapshot of some of the activities going on in Facebook today. One in thirteen people in the world is a member of Facebook. 500 million active monthly users for 500 billion pages viewed every month. That makes 25 billion new pieces of data every month. So this kind of site is a unique site, and it has quite enormous volumes of data.

You could think of a few others as well, for example eBay or Google or Yahoo. They would probably be up there to, and Amazon of course. But in addition, we also have the emergence of sensors. Sensors perhaps started out with RFIDs, which are now in abundance across to the world. But now we have sensors able to detect a whole range of things from light, vibration, or airflow pressure.

In addition to these structured types of data, we have also got the world of unstructured content. I have read that as much as 80% of our data is in unstructured form. Some examples are news feeds, blogs, Twitter, of course, Facebook I already mentioned, e-mail, RSS feeds, all of that, all representing unstructured content. And yes we have rich media as well which is video and sound, but text in particular is now a hot area for a lot of organizations, particularly trying to understand things like sentiment and what’s going on with respect to their competitors.

Companies are also trying to combine that kind of volume of click stream data with customer data in order to understand behaviors. In ecommerce people are trying to understand what’s selling together online in a basket analysis and also trying to understand whether any of the promotions or recommendations are effective.

Expanding Data Landscapes

As organizations confront these expanding data landscapes, one of the most practical challenges is simply getting decision-makers to the right information at the right time. Traditional BI often assumes that IT can anticipate every question in advance and pre-build the necessary reports and cubes. In a big data world, that assumption breaks down quickly. Business users need the freedom to explore, pivot, and refine their questions on the fly, without waiting weeks for a new data mart or report iteration. Agile BI platforms address this by enabling rapid data mashups, self-service visualization, and iterative refinement of analytics assets.

Another key dimension of big data is the diversity of sources involved. It is no longer enough to analyze a single transactional system or a single marketing platform. Competitive insight often comes from blending operational data with external feeds, social signals, sensor readings, and third-party benchmarks. Agile BI tools make this multi-source integration more feasible by providing flexible data connectors, semantic layers that can be adapted quickly, and visual interfaces that help users understand how disparate datasets relate to each other. This reduces the friction between experimentation and production, allowing promising insights to be operationalized faster.

Performance and scalability also become central concerns when dealing with big data. Traditional BI architectures may rely heavily on pre-aggregated structures and batch processing windows that cannot keep pace with real-time or near-real-time demands. Agile BI solutions typically embrace in-memory processing, columnar storage, and optimized query engines that can handle large volumes without forcing users to compromise on interactivity. When analysts can slice billions of rows and still receive sub-second responses, they are far more likely to iterate, test hypotheses, and uncover patterns that would otherwise remain hidden.

Governance and data quality, however, cannot be ignored in the rush toward agility. Big data environments often introduce new risks: inconsistent definitions, duplicate records, and unclear lineage. Agile BI does not mean abandoning control; it means rethinking how governance is implemented. Modern platforms allow organizations to define reusable data models, apply role-based security, and track usage patterns, while still giving business users room to experiment within trusted boundaries. This balance between freedom and control is essential for scaling analytics beyond a small group of specialists and embedding it into everyday decision-making.

Finally, the cultural shift required for agile BI in a big data context may be as significant as the technology shift. Teams that are accustomed to static reports and annual planning cycles must learn to embrace continuous discovery and rapid feedback loops. Successful organizations invest in data literacy, encourage cross-functional collaboration, and treat analytics as an ongoing conversation rather than a one-time deliverable. When business stakeholders, IT, and data specialists work together in this way, big data stops being an overwhelming challenge and becomes a strategic asset—fuel for innovation, differentiation, and more responsive customer experiences.