Using InetSoft's BI Front-end to Create Materialized OLAP Overlays

There are many options available to transform an OLAP overlay into a dedicated business intelligence data store, including materialized views, star schema/snowflake schema, and multi-dimension databases.

The Star Schema is an obvious choice because of the conceptual connection. However, a separate physical structure will introduce other challenges that do not exist in an OLAP overlay. For example, changing dimensions and measures requires careful consideration of the trade-offs involved in using different types of update methods. Care must also be taken when choosing from the many detail attributes which ones should be copied into the new schema. The blurred line between the snow flake schema and the star schema can also be challenging. Refer to the following books for more data warehouse design ideas:

• The Data Warehouse Toolkit, by Ralph Kimball
• Building the Data Warehouse, by William Inmon

Multidimensional databases have similar challenges that typically require a more knowledgeable data warehouse specialist to address.

InetSoft’s patent-pending Data Block architecture allows a materialized OLAP overlay to work seamlessly with the underlying data. After this step, business users can create interactive monitoring dashboards to view and track their desired key metrics.

Architecture Articles on InetSoft

• Cloud Native Microservice

  Cloud-native microservice architecture combines cloud computing principles with the microservices style, enabling applications to be scalable, resilient, and portable across public, private, and hybrid clouds through containerization using tools like Docker and dynamic orchestration with platforms such as Kubernetes. Microservices are designed as small, independent, and loosely coupled services, each with its own database, development team, and lifecycle, communicating via APIs like REST, gRPC, or messaging queues such as Kafka, and deployable independently without affecting the entire system. Key characteristics include horizontal scalability of individual services based on demand, resilience through isolation and fault tolerance patterns like circuit breakers, and portability across environments due to containerization. Agility is enhanced by parallel team development and CI/CD pipelines for quick deployments, while observability is achieved with monitoring tools like Prometheus and Grafana for logs, metrics, and traces. In a real-world example, an e-commerce application decomposes into services like catalog, cart, payment, and notification, each running in containers managed by Kubernetes and scaled independently.

• Operational Data Organization

  Operational data architecture is defined as the design and organization of data used to run an organization's daily operations, focusing on how operational data is captured, stored, managed, processed, and accessed to support business activities. It encompasses various operational systems, including transactional systems that record business interactions in real-time or with latency for historical auditing, workflow automation systems that handle document routing and task management, and manufacturing automation systems that integrate machines and software for low-latency data exchange. Commercial IoT systems automate building management through real-time monitoring of environmental conditions, while industrial IoT systems interconnect sensors and devices for productivity gains, all requiring architectural decisions on data flow to data lakes or warehouses. Challenges in operational data architecture arise from legacy systems, purchased applications like ERP and SaaS with unique semantics, and interoperability issues between operational and analytic architectures, often leading to data silos and suboptimal integration via data warehouses or lakes. A new approach to operational data architecture involves developing tailored data models, structures, and definitions, implementing specific management processes, ensuring consistent data quality and governance, and integrating operational data with analytic data to preserve integrity.

• Data Architecture Service

  Data Architecture as a Service (DAaaS) is a peer-based methodology designed for decentralized organizations, combining SaaS and PaaS concepts to address data governance challenges by assisting application projects with data-related issues. It supports data modeling through tools that ensure products are attractive and useful, while also aiding business data owners in simplifying data access restrictions and collaborating with implementers to enforce enterprise-level information access guidelines. The core argument for DAaaS emphasizes data sharing across applications, requiring agreement on data format and meaning to avoid costly IT transformations, delays in data processing, and risks of system fragility. InetSoft promotes openness by making artifacts like a high-level enterprise data model (limited to 20 entity types highlighting key organizational linkages) and detailed data models publicly accessible, treating them as dynamic, crowd-supported resources. Additionally, the enterprise data concepts model serves as a structured glossary intertwined with the corporate data model, defining high-level entity types and essential business concepts to facilitate structured data exchange.

• Single BI Tool

  The single BI tool architecture replaces the old modular setup involving multiple proprietary tools like Informatica, Oracle, and Business Objects with a unified approach that integrates, stores, and presents data within one efficient platform, often managed by a single team or individual. This architecture eliminates the need to pre-build data models with predefined dimensions, measures, and hierarchies, allowing users to simply point to various data sources, mash them together as columns, and create analytical views directly. It supports rapid prototyping as the primary use case for data discovery, enabling quick data blending and analysis. Once datasets are mashed, the in-memory performance layer facilitates unfettered drilling without hierarchical paths, supporting diagnostics and root cause analysis through intuitive navigation and interactive visualization. The architecture combines visualization and memory for a strong framework, with search-based data discovery emerging as a variant using search indexes to blend structured and unstructured data, though it has not yet achieved mainstream adoption.

• Microservice BI Environment

  StyleBI is architected as microservices that can be deployed into the orchestration services of any cloud platform, enabling a cloud-native BI environment. It integrates directly with orchestration services from AWS, Azure, and Google Cloud Platform, which are simpler to use than Kubernetes, and provides pre-configured deployment plans for users with limited technical expertise. StyleBI's Kubernetes integration allows for platform-neutral deployment, making it seamless for enterprises already using Kubernetes to implement it as microservices. The architecture supports provisioning, autoscaling, and high availability by integrating with a wide range of cloud platform services, requiring minimal effort and expertise. Additionally, it includes data source integration as a key architectural service to deliver a powerful BI environment.

• Business Intelligence Platform

  A business intelligence platform is a software solution that enables developers and end-users to build reusable applications such as dashboards, analyses, data visualizations, production reports, and interactive reports, with essential components including data access, integration across various sources, on-the-fly data transformation, KPI calculations, real-time access, and caching capabilities. In contrast to limited BI tools that only generate SQL statements or static PDF reports from reporting systems, a true platform supports comprehensive information display and application development. Older BI approaches relied on data warehouses for batch aggregation of historical data, which suffered from latency, scalability issues with transaction-level granularity, and the need for retuning queries or re-architecting models, while rules engines and case management systems offered flexibility but required prebuilt rules and suffered from siloed data capture. Modern BI architecture must address diverse user needs—such as exception-based dashboards for executives, OLAP tools for analysts, operational reports for staff, and functionality for external stakeholders—by combining traditional data warehousing with operational BI components and embedded analytics within business processes for intelligent process automation. InetSoft's BI platform, based on Java and featuring patent-pending Data Block technology, provides a secure, scalable, and collaborative operational framework that aligns intelligence flow with business processes, allowing Data Blocks to build incrementally for aggregation, comparison, and visualization across users, with deployment flexibility for resource efficiency.

• BI Architecture Slide

  The webinar transcript emphasizes that a BI architecture slide, which outlines tools and their uses, is beneficial but represents only one component of a comprehensive BI strategy and not the strategy itself. It critiques the approach of IT soliciting specific report requirements from business users, noting that business users often lack sufficient understanding of data structures, leading to inadequate or mismatched deliverables that fail to meet needs. Building a data warehouse without prior assessment of business needs is highlighted as a potential pitfall, as it may not align with the questions or answers required for effective analysis. Deploying a data warehouse involves complex technical challenges, including inadequate data quality and consistency from disparate sources, which can undermine the reliability of reporting and decision-making if not addressed early. Additionally, scalability and performance issues, such as inefficient data modeling or suboptimal query optimization, can cause bottlenecks and downtime, while organizational resistance and lack of user training hinder adoption and the realization of the warehouse's full potential.

• Pervasive BI Architecture

  Pervasive BI architecture enables the delivery of integrated, right-time data warehousing information to all users, including front-line employees, suppliers, customers, and partners, by liberating existing infrastructure and connecting it to multiple operational business processes. Data integration plays a key role by simplifying the process through InetSoft's unique data mashup engine, which combines various data sources into a common BI infrastructure for creating dashboards and reports. In the traditional approach, data environments were highly fragmented, making enterprise-wide integration challenging, but recent advancements in operational integration and the new information supply chain environment have made pervasive BI feasible. The architecture includes transactional services such as OLTP for enterprise bookkeeping functions, encompassing call center automation, operational CRM, enterprise resource planning, supply chain management, and legacy applications. Data integration services bridge multiple domains by extracting, discovering, cleansing, transforming, and delivering data from transactional repositories to decision repositories like enterprise data warehouses, data marts, and operational data stores for high-speed access.

• BI Infrastructure Facets

  Business intelligence software combines executive information systems with IT functionality to provide a framework for business operations, emphasizing the need for proper architecture in BI initiatives. A key best practice involves outlining architecture for various facets of the BI infrastructure, including standards for design, definitions, processes, tools, and technologies required for implementation. Alignment between IT and business is critical, starting with identifying the intent and expectations of the BI initiative, ensuring they align with business objectives and strategies while understanding and prioritizing information needs across stakeholder groups. Developing, executing, and appraising a plan is essential, which includes creating metrics to measure both the implementation and ongoing success of the BI architecture. Failing to follow these practices can lead to issues such as slow performance and scalability challenges due to inadequate infrastructure planning, as well as silos and fragmented data from lack of integration between sources and departments.

• Serverless Dashboard Solution

  A serverless dashboard is a cloud-native solution for data visualization and analysis that eliminates the need for managing infrastructure, relying instead on a scalable backend provided by the cloud vendor, with usage-based pricing and no fixed costs. Key architectural advantages include instant deployment without server provisioning, automatic scaling to handle high user loads, zero maintenance for patching or tuning, global availability across cloud regions for low latency, and built-in security features like data encryption and role-based access. The platform features a no-code drag-and-drop builder for visualizations, real-time data connections to databases and APIs, fully customizable UI with white-labeling, API-driven integrations for automation, and multi-tenant support in a single environment. This architecture ensures consistent performance from day one, whether serving few or many users, through a globally distributed network with event-driven, on-demand scaling. Notable vendors offering serverless or similar architectures include StyleBI with its microservice-based BI, Google Looker Studio's fully hosted design, Amazon QuickSight's auto-scalable compute on AWS, and Microsoft Power BI's semi-serverless Fabric integration.