This post originally appeared on the Forbes website on August 8, 2013.
Big Data has been declared the “sexiest job of the 21st century” and it’s already reshaping corporate decisionmaking and even talent recruitment. As Phil Simon, author of Too Big to Ignore: The Business Case for Big Data, points out, “In a recent report, McKinsey estimated that the U.S. will soon face a shortage of approximately 175,000 data scientists.” Seemingly everyone wants in on Big Data, a blanket term for the massive amounts of information we generate today. (Simon notes that roughly 80 percent of that information is unstructured – such as tweets, blog posts, Facebook likes, or YouTube videos – as compared to the smaller and easier-to-manage “structured data” that populates spreadsheets.) With the impending rise of wearable technology and “The Internet of Things,” it appears the pace of data creation will only increase. Here are four things you need to know about the Era of Big Data.
It’s Not Just for Large Companies. “Companies like Amazon, Apple, Facebook, Google, Twitter, and others would not be nearly as effective without Big Data,” says Simon. But it’s not exclusively the province of billion-dollar tastemakers. “In the book, I write about a few small companies that have taken advantage of Big Data. Quantcast is one of them. There’s no shortage of myths around Big Data, and one of the most pernicious is that an organization needs thousands of employees and billions in revenue to take advantage of it. Simply not true.”
Data Visualization may be the Next Big Thing. You have massive amounts of data – how can you possibly comprehend it? Enter data visualization, which Simon believes is “absolutely essential for organizations…[These tools] are helping employees make sense of the never-ending stream of data hitting them faster than ever. Our brains respond much better to visuals than rows on a spreadsheet. This goes way beyond Excel charts, graphs, or even pivot tables. Companies like Tableau Software have allowed non-technical users to create very interactive and imaginative ways to visually represent information.”
Intuition isn’t Dead. It turns out Big Data and the certainty of numbers hasn’t killed intuition, after all. “Contrary to what some people believe,” says Simon, “intuition is as important as ever. When looking at massive, unprecedented datasets, you need some place to start. Intuition is more important than ever precisely because there’s so much data now. We are entering an era in which more and more things can be tested.
Big Data has, at least not yet, replaced intuition; the latter merely complements the former. The relationship between the two is a continuum, not a binary.”
It Isn’t a Panacea. “When used right,” says Simon, “Big Data can reduce uncertainty, not eliminate it. We can know more about previously unknowable things. This helps us make better predictions and better business decisions.” But it doesn’t mean we can rely solely on the numbers without injecting our knowledge, perspective, and (yes) intuition.
Is your company using Big Data? How do you expect it to impact your business or industry?
Dorie Clark is CEO of Clark Strategic Communications and the author of the newly-released Reinventing You: Define Your Brand, Imagine Your Future (Harvard Business Review Press, 2013). She is a strategy consultant who has worked with clients including Google, Yale University, and the Ford Foundation. Listen to her podcasts or follow her onTwitter.