c24 Application Hosting Specialists

Some notes on the ACM Webinar on 2012 – Big Data: End of the World or End of BI? by Dr. Barry Devlin of 9sight Consulting

The original Data Warehouse Architecture was conceived in 1988 as a single logical storehouse. This changed in the early 1990′s into a layered model of an Enterprise Data Warehouse with Data Marts.

There are four (4) ancient postulates of data warehousing:

• Postulate 1 (1970s): Operational and informational environments should be separated for both business and technical reasons.
• Postulate 2 (1980s): A data warehouse is the only way to obtain a dependable, integrated view of the business.
• Postulate 3 (1980s): The data warehouse is the only possible instantiation of the full enterprise data model.
• Postulate 4 (1990s): A layered data warehouse is necessary for speedy and reliable query performance.

See: Devlin, B. “Business Integrated Insight (BI²): Reinventing enterprise information management”, (2009), www.9sight.com/resources.htm

Slide #8

This explication of the underlying assumptions (or postulates) helps to explain the evolution of the data warehouse architecture. It seems now that these decisions were made based on the available computing power at the time. The operational data stores were straining under the load at the time, and BI was seen as a luxury compared to the real business of making money. Now with the large computing resources of CPU, disk space, and networks, this constraint is no longer a barrier to integration of front-end and back-end business processes.

Devlin says that the explosion in the number of DW components from the mid 1990s onwards suggests that the data warehouse architecture is failing. From my perspective, this mess came about because some enterprises tried to do data warehousing on the cheap. Requirements were usually vague and the implemented solutions were ad-hoc. I think Devlin is saying that this mess was inevitable given the ancient posulates given above.

After reflecting on this mess, Devlin came up with five (5) modern postulates for highly evolved business:

1. Modern business processes seamlessly combine action-taking and decision-making, and require an integrated continuum of consistent information.
2. The new information architecture must be based on a comprehensive enterprise information model, spanning all types of information used in the business.
3. The business information resource is best maintained as a single copy of each data item, with only the most minimal resort to transient layers or copies of specific subsets of data for specialized needs.
4. An integrated, model-based and closed-loop process environment is needed to create, maintain and use both the business information an activities.
5. An integrated, flexible and role-based user interface provides access to the entire business information.

Slide #10

What is a comprehensive enterprise information model? How is it different from a data model? Data model was mentioned in postulate #3 above. So, are we moving up the knowledge hierarchy from data to information? If so, I think the analysis is confused by the ambivalent meaning of data model—see my earlier notes at On the Logical Difference Between Model and Implementation.

Devlin goes on to propose a new architecture Business Integrated Insight (BI²)…covering all information and process:

• People Personal Action Domain
• Process Business Function Assembly
• Information Business Information Resource

See: Devlin, B. “Business Integrated Insight (BI²): Reinventing enterprise information management”, (2009), http://bit.ly/BI2_White_Paper

Slide #11

Devlin introduces Biz-Tech ecosystem. He does not think IT is dead despite what many analysts say. He says that IT has evolved into a Biz-Tech ecosystem which is the fully symbiotic existence and IT. This has the following three (3) characteristics:

1. Interdependence
   New technology enables business possibilities;
   new business opportunities drive technology advances
2. Reintegration
   Silos in business and IT are obvious to Web-savvy customers;
   coherence becomes mandatory
3. Cross-over
   Business people need IT skills to see how to recreate the business with new technoology;
   IT people need business acumen to see how to satisfy business needs in new ways with emerging technology

Slide #14

This view flies in the face of the idea of computing (or IT) as a commodity. IT people need to be integrated into the business as much as sales, marketing, HR, production, and design. All of these people has to come together to create a coherent product for the customer. IT people are no longer resources simply to be brought on the open market. And IT people need to stop thinking of themselves as simply Java programmers or Oracle DBAs.

He gives three (3) examples of Biz-Tech ecosystems:

1. Business Intelligence reinvents Retail (cf Walmart)
2. The web recreates the library (cf Wikipedia)
3. Big data redefines automobile insurance—Pay as you drive

Devlin sees evolution of BI² occurring in three (3) parts:

1. Removal of layers in BI².
   • Introduction of the advanced information warehouse which has pillars rather layers. Data, metadata, and models are shared across the pillars. EDW has evolved into Core Business Data. (See slide #21)
   • Data virtualisation becomes more important by enabling queries to be constructed across differing data stores.
2. Dealing with new information types:
   • Big data challenges our fundamental beliefs about the relationship between data and knowledge.
   • The DIKW pyramid is no longer valid. (Date -> Information -> Knowledge -> Wisdom) (See slide #23)
3. Introducing m3 – the modern meaning model (see slide #24)
   • Decision making moves from individual to collaboration
   • Decisions are not rational

Devlin gave the following picture of the Modern Meaning Model:

I have not absorbed this model yet, but it does appear to be sensible. Whether or not it is useful remains to be seen.

Devlin sees mobile computing as important as the producer and consumer of information, and decisive in team-based decision making (the iSight Model—see slide #29). He sees the informal interactions being recorded for future analysis.

Devlin’s conclusions are:

1. Overall—simplify the BI environment
   • Less layers, less copies, less ETL
   • Recognise the emerging biz-tech ecosystem
2. Big Data—forget the hype, but do evaluate
   • Business opportunities may exist in unexpected places
   • Recall that big data has very different characteristics
3. Enable innovation through team working
   • Collaborative decisioning vs. collaborative BI
   • The emerging role of informal information

Slide #31