Data Mashups in an Ideal World

The DM Radio Webcast, “The Last Mile: Data Visualization in a Mashed-Up” from Information Management continues. The transcript of that Webcast, which was hosted by Eric Kavanagh and included InetSoft's Product Manager Tibby Xu and BI consultants William Laurent and Malcolm Chisholm resumes below:

EK: Let’s bring in Malcolm and William to discuss the evolving role of IT and the usefulness of mashups. William, what do you think? Do you think that is a red herring or we are on to something?

William Laurent (WL): No, any evolution in IT is not a red herring. What I find interesting, is the federated approach and the ad hoc approach that we have talked about is diametrically opposed to the SOA approach that we see being used in systems everywhere these days, and I think it’s important to keep the concepts of mashup more nimble, and more agile. Especially as thing gravitate towards cloud computing, and we’re storing our data in the cloud type format.

How is the Federated, Ad Hoc Approach to Data Services Diametrically Opposed to the SOA Approach?

The federated, ad hoc approach to data services and the Service-Oriented Architecture (SOA) approach represent two fundamentally different paradigms in how organizations structure, manage, and deliver data and services. Their differences reflect opposing philosophies in terms of control, design, governance, and flexibility.

1. Architectural Philosophy

• SOA Approach:
  • Centralized and Structured: SOA is characterized by a centralized, structured approach to service design and deployment. It is built on the principle that services should be reusable, loosely coupled, and adhere to well-defined standards. SOA emphasizes the creation of a cohesive and integrated architecture where services are carefully designed and orchestrated to work together seamlessly across an organization.
  • Rigorous Design and Planning: In SOA, services are planned and designed with a long-term perspective. Each service is typically developed according to predefined specifications and standards, ensuring consistency and interoperability. This approach often involves significant upfront planning and a clear governance model to maintain control over service interactions.
• Federated, Ad Hoc Approach:
  • Decentralized and Flexible: The federated, ad hoc approach is decentralized, allowing different teams or departments within an organization to create and manage their own data services independently. This approach is inherently more flexible, enabling rapid development and deployment of services to meet specific, immediate needs without requiring alignment with a central architecture or strict standards.
  • Emergent and Opportunistic: Services in a federated, ad hoc environment are often developed on an as-needed basis, with less emphasis on long-term planning or adherence to a centralized design. This allows for greater adaptability but can lead to inconsistencies and integration challenges.

2. Control and Governance

• SOA Approach:
  • Strong Governance: SOA typically operates under a strong governance model that enforces rules, standards, and policies across the entire organization. This governance ensures that all services are aligned with the overall architecture, promoting consistency, reliability, and security. Centralized control is key in SOA, with a focus on managing service lifecycle, ensuring compliance with organizational standards, and maintaining service quality.
  • High Coordination: Given the centralized nature of SOA, significant coordination is required across teams to ensure that services are developed and deployed in a way that aligns with the organization's architectural goals. This often involves oversight by a centralized IT or architecture team.
• Federated, Ad Hoc Approach:
  • Minimal Governance: In contrast, the federated, ad hoc approach minimizes centralized governance, allowing individual teams to operate more autonomously. Governance is typically lighter and more localized, giving teams the freedom to create and manage services in ways that best suit their immediate needs. This can lead to a more agile and responsive environment but at the expense of control and consistency.
  • Decentralized Decision-Making: Decision-making is decentralized, with teams having the authority to determine how services are built, deployed, and maintained. This approach can accelerate innovation and time-to-market for new services but may result in fragmented and siloed service ecosystems.

3. Service Interoperability and Standardization

• SOA Approach:
  • Standardization and Interoperability: SOA places a strong emphasis on standardization to ensure that services can interoperate seamlessly across the organization. Services are designed to be reusable components that can be easily integrated into different workflows and applications. This standardization is crucial for maintaining a coherent system where services can communicate and work together effectively.
  • Unified Service Model: SOA often involves the use of standardized protocols (such as SOAP or REST) and a unified service model that promotes consistent interaction patterns and data exchange formats across all services.
• Federated, Ad Hoc Approach:
  • Lack of Standardization: In a federated, ad hoc environment, services are often developed independently without a strong emphasis on standardization. This can lead to a diversity of service interfaces, protocols, and data formats, making interoperability more challenging. Each team might use different technologies or frameworks, which can create barriers to integration.
  • Diverse Service Models: The ad hoc nature of service creation in this approach means that different teams might implement similar services in different ways, leading to redundancy and complexity. This diversity can be both a strength, in terms of innovation, and a weakness, in terms of system coherence.

4. Scalability and Evolution

• SOA Approach:
  • Scalable and Long-Term: SOA is designed with scalability in mind, allowing services to evolve and scale as organizational needs grow. The centralized and standardized nature of SOA supports the creation of a scalable architecture that can handle increasing service demand while maintaining consistency and reliability.
  • Planned Evolution: Changes and enhancements to services within an SOA are typically planned and coordinated to ensure that they align with the overall architectural vision. This approach promotes stability and minimizes the risk of disruption when scaling services or adding new capabilities.
• Federated, Ad Hoc Approach:
  • Scalability Challenges: While the federated, ad hoc approach offers greater flexibility in the short term, it can face challenges in scaling effectively. The lack of standardization and centralized control can lead to inefficiencies and difficulties in managing a large and complex service ecosystem as it grows.
  • Organic Evolution: Services in a federated, ad hoc environment evolve more organically, driven by immediate needs rather than long-term planning. While this can foster rapid innovation, it can also lead to technical debt, where the accumulation of ad hoc solutions becomes difficult to manage and integrate over time.

5. Use Cases and Suitability

• SOA Approach:
  • Best for Large, Complex Organizations: SOA is well-suited for large, complex organizations where consistency, reliability, and long-term scalability are critical. It is ideal for environments where services need to be reused across multiple departments or applications, and where a high level of control and governance is required.
  • Enterprise-Level Applications: SOA is often employed in enterprise-level applications where the integration of multiple services and systems is necessary. It supports complex workflows and business processes that require a high degree of orchestration and coordination.
• Federated, Ad Hoc Approach:
  • Best for Agile, Innovative Environments: The federated, ad hoc approach is more appropriate for environments where agility, innovation, and rapid response to changing needs are prioritized. It is well-suited for startups, R&D departments, or any organization where the speed of development and deployment is more important than strict adherence to architectural standards.
  • Experimental and Niche Applications: This approach is often used in experimental projects, niche applications, or in situations where the organization is exploring new markets or technologies and needs the flexibility to iterate quickly.