Reflections on #ACM Webinar “2012 – #BigData: End of the World or End of #BI?”


Some notes on the ACM Webinar on 2012 – Big Data: End of the World or End of BI? by Dr. Barry Devlin of 9sight Consulting

The original Data Warehouse Architecture was conceived in 1988 as a single logical storehouse. This changed in the early 1990′s into a layered model of an Enterprise Data Warehouse with Data Marts.

There are four (4) ancient postulates of data warehousing:

  • Postulate 1 (1970s): Operational and informational environments should be separated for both business and technical reasons.
  • Postulate 2 (1980s): A data warehouse is the only way to obtain a dependable, integrated view of the business.
  • Postulate 3 (1980s): The data warehouse is the only possible instantiation of the full enterprise data model.
  • Postulate 4 (1990s): A layered data warehouse is necessary for speedy and reliable query performance.

See: Devlin, B. “Business Integrated Insight (BI2): Reinventing enterprise information management”, (2009), www.9sight.com/resources.htm

Slide #8

This explication of the underlying assumptions (or postulates) helps to explain the evolution of the data warehouse architecture. It seems now that these decisions were made based on the available computing power at the time. The operational data stores were straining under the load at the time, and BI was seen as a luxury compared to the real business of making money. Now with the large computing resources of CPU, disk space, and networks, this constraint is no longer a barrier to integration of front-end and back-end business processes.

Devlin says that the explosion in the number of DW components from the mid 1990s onwards suggests that the data warehouse architecture is failing. From my perspective, this mess came about because some enterprises tried to do data warehousing on the cheap. Requirements were usually vague and the implemented solutions were ad-hoc. I think Devlin is saying that this mess was inevitable given the ancient posulates given above.

After reflecting on this mess, Devlin came up with five (5) modern postulates for highly evolved business:

  1. Modern business processes seamlessly combine action-taking and decision-making, and require an integrated continuum of consistent information.
  2. The new information architecture must be based on a comprehensive enterprise information model, spanning all types of information used in the business.
  3. The business information resource is best maintained as a single copy of each data item, with only the most minimal resort to transient layers or copies of specific subsets of data for specialized needs.
  4. An integrated, model-based and closed-loop process environment is needed to create, maintain and use both the business information an activities.
  5. An integrated, flexible and role-based user interface provides access to the entire business information.

Slide #10

What is a comprehensive enterprise information model? How is it different from a data model? Data model was mentioned in postulate #3 above. So, are we moving up the knowledge hierarchy from data to information? If so, I think the analysis is confused by the ambivalent meaning of data model—see my earlier notes at On the Logical Difference Between Model and Implementation.

Devlin goes on to propose a new architecture Business Integrated Insight (BI2)…covering all information and process:

  • People Personal Action Domain
  • Process Business Function Assembly
  • Information Business Information Resource

See: Devlin, B. “Business Integrated Insight (BI2): Reinventing enterprise information management”, (2009), http://bit.ly/BI2_White_Paper

Slide #11

Devlin introduces Biz-Tech ecosystem. He does not think IT is dead despite what many analysts say. He says that IT has evolved into a Biz-Tech ecosystem which is the fully symbiotic existence and IT. This has the following three (3) characteristics:

  1. Interdependence
    New technology enables business possibilities;
    new business opportunities drive technology advances
  2. Reintegration
    Silos in business and IT are obvious to Web-savvy customers;
    coherence becomes mandatory
  3. Cross-over
    Business people need IT skills to see how to recreate the business with new technoology;
    IT people need business acumen to see how to satisfy business needs in new ways with emerging technology

Slide #14

This view flies in the face of the idea of computing (or IT) as a commodity. IT people need to be integrated into the business as much as sales, marketing, HR, production, and design. All of these people has to come together to create a coherent product for the customer. IT people are no longer resources simply to be brought on the open market. And IT people need to stop thinking of themselves as simply Java programmers or Oracle DBAs.

He gives three (3) examples of Biz-Tech ecosystems:

  1. Business Intelligence reinvents Retail (cf Walmart)
  2. The web recreates the library (cf Wikipedia)
  3. Big data redefines automobile insurance—Pay as you drive

Devlin sees evolution of BI2 occurring in three (3) parts:

  1. Removal of layers in BI2.
    • Introduction of the advanced information warehouse which has pillars rather layers. Data, metadata, and models are shared across the pillars. EDW has evolved into Core Business Data. (See slide #21)
    • Data virtualisation becomes more important by enabling queries to be constructed across differing data stores.
  2. Dealing with new information types:
    • Big data challenges our fundamental beliefs about the relationship between data and knowledge.
    • The DIKW pyramid is no longer valid. (Date -> Information -> Knowledge -> Wisdom) (See slide #23)
  3. Introducing m3 – the modern meaning model (see slide #24)
    • Decision making moves from individual to collaboration
    • Decisions are not rational

Devlin gave the following picture of the Modern Meaning Model:

Devlin's Modern Meaning Model (m3)

I have not absorbed this model yet, but it does appear to be sensible. Whether or not it is useful remains to be seen.

Devlin sees mobile computing as important as the producer and consumer of information, and decisive in team-based decision making (the iSight Model—see slide #29). He sees the informal interactions being recorded for future analysis.

Devlin’s conclusions are:

  1. Overall—simplify the BI environment
    • Less layers, less copies, less ETL
    • Recognise the emerging biz-tech ecosystem
  2. Big Data—forget the hype, but do evaluate
    • Business opportunities may exist in unexpected places
    • Recall that big data has very different characteristics
  3. Enable innovation through team working
    • Collaborative decisioning vs. collaborative BI
    • The emerging role of informal information

Slide #31

Advertisements

2 thoughts on “Reflections on #ACM Webinar “2012 – #BigData: End of the World or End of #BI?”

  1. Wonderful blog! I found it while surfing around on Yahoo News.
    Do you have any suggestions on how to get listed in Yahoo News?
    I’ve been trying for a while but I never seem to get there! Thank you

  2. Good day! This is my first visit to your blog! We are a group of volunteers and
    starting a new initiative in a community in the same
    niche. Your blog provided us beneficial information to work on.
    You have done a wonderful job!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s