One of the under-appreciated points about Big Data is that as consumers we also act as producers. In each transaction, we contribute bits of analyzable data to the corporate information stockpile. Data hungry companies then draw non-obvious connections by mining zillions of data points. For example, if you are Walmart, you’ve learned that stores in the path of a hurricane often see a spike in demand for Pop-Tarts.
Walmart’s response after spotting this correlation is to now always make sure affected outlets are well stocked in advance with this easy-to-prepare food item. Data mining has been a powerful business decision tool for big box stores, but what about everyone else: can we as consumers directly benefit from all the data we’ve helped to create?
Online retailers and social media sites have led in this area. They have returned to the consumer some of their insights by providing customer-specific recommendations that are based on a global analysis of behaviors. Special collaborative filtering algorithms hunt through the data to find similarities between your own purchasing patterns and larger groups or clusters. These statistically-based recommendations are at the heart of Amazon’s book and Netflix’s movie suggestions.
But outside of e-commerce, companies have generally been reluctant to share their Big Data.
This lack of transparency was taken up in an article recently in The New York Times, If My Data Is an Open Book, Why Can’t I Read It? The writer tells about the frustrations in getting detailed data about cell phone and electric usage from each of her respective providers. She was hoping to see the geo-location data her carrier records (and, by the way, does make available to third-party marketers), but was told that the company doesn’t share customers’ own location logs with them without a subpoena. Her energy utility had similar reservations.
One of the stumbling blocks mentioned in the Times article is that old-economy companies feel they play the role of a benevolent data owner that shares just enough data to be a little helpful. It turns out that consumers are also uncomfortable with the idea that their long-time vendors might be analyzing, categorizing, and sharing conclusions from their personal data.
But attitudes are changing for both consumers and corporate data collectors.
For example, many of us have probably engaged in on-line banking through third-party applications, using desktop software to pay bills and analyze spending trends. Recently my stodgy bank began to offer direct online bill paying—yours probably has done the same long before mine– and so I transitioned to their cloud-based software.
I lost some of the convenience of instant analysis that I had when I was accessing my data on mydesktop computer. But then I noticed the bank was adding modest features—alerts that could be configured when my balance reached certain limits. I suspect there’ll be more features and reporting capabilities in the near future in their cloud-based service.
And in the equally conservative credit card space, start-ups have emerged to analyze millions of transactions for fraudulent charges. The key innovation here was to borrow a cue from Amazon’s book reviews: crowdsource vendor evaluation based on feedback from the service’s subscribers. I count myself as a customer of one of these credit card fraud detection services. It was clear in the terms of service that I was allowing my credit card data to be used in a collective fashion to help spot fraudsters.
The key mindset change for companies is that they have to recognize that consumers own their data, and consumers must realize that they are granting access to their data with (hopefully) suitable guarantees of privacy.
Once these data ownership understandings are formalized and accepted by both parties, it won’t be long before consumers have their own Pop-Tarts realizations as they reap benefits from Big Data.
Posted by david ricketts 
