The Fallacy of Being Data-Driven


The unprecedented pace of evolution of technology during past decade or so, has led to data democratization at a scale never witnessed before. Unequivocally, as techno pundits say, “data is the new oil”. There is absolutely no ambiguity on the fact that data had been one of the most primary elements in bolstering business performance and creating competitive advantage for businesses. The burgeoning scope of capturing customer data because of availability of tools in abundance has ostensibly made the enterprises and marketers to believe that consumption is largely predictable.

Today, we are living in a world of algorithmic surveillance, which has monumentally amplified the capabilities of marketers to design persuasive architecture to influence customer decisions. Every customer touch point today, is meant to play a dual role, the role of capturing invaluable data from interactions and in-turn, mining those to generate insights that can help in creating and delivering exceptional customer experience, what is known as moments of truth in business.

In 1950s, Detroit based auto companies were the most valuable enterprises in the world. Today, hardly any of those feature in the list of top ten most valuable companies. All those great auto makers have been dislodged from their positions by new age tech companies. We all are aware of the fact that the most valuable enterprises in today’s time are the ones which own and control data.  To give you a bit of perspective, six out of top ten performing brands in 2020, featuring in Forbes list, are the brands which belongs to bay area’s tech companies. Many of us know them by their abbreviated name, FAANG (Facebook, Apple, Amazon, Netflix and Google).

So, the reason for data obsession of today’s enterprises is obvious and seemingly rational. However, there are certain facts which draws attention towards business outcomes which contradicts or ditches the popular matrices. For example, though Click through Rate (CTR) is widely used as a matrix to measure effectiveness of digital ads., but many brands with low CTR have been found to perform exceptionally well with relation to repeat purchase and customer loyalty.

A recent study conducted by Bart de Lange, a marketing professor and behavioral scientist at University of Ramon Llull, ESADE and Stefano Puntoni, also a marketing professor at the Rotterdam School of Management, Erasmus University, revealed facts which are quite counterintuitive in this context. The eight long years of research by these two academicians to measure corporate data literacy disinterred that only 24% of companies have achieved something they term as data- driven nirvana. Their research paper published in MIT Sloan Management Review with the title, “Leading with decision- driven data analytics”, is an eye-opener for many, who are uncritically attached to data-all belief and have disenfranchised themselves from all other school of thoughts.

Today, Companies have more data than ever, despite that many executives say that their data analytics initiatives do not provide actionable insights and produce disappointing overall results, as highlighted by Lange and Puntoni. The inability to generate desired outcome, even after being data rich, is the central concern of most corporate executives.

Harvard Business School Professor, Clayton Christensen, insightfully narrated about data-trap in his book, competing against luck. While elaborating on the fallacy of conforming data, Clayton shares that data has an annoying way of conforming itself to support whatever point of view we want it to support. Nate Silver, a well-known statistician and founder of the New York Times political blog, FiveThirtyEight, noted, “The most calamitous failures of prediction usually have a lot in common. We focus on those signals that tell a story about the world as we would like it to be, not how it really is.”

As neuro-marketing expert and Professor of Harvard Business School, Gerald Zaltman observed and stated, “We pick and choose the data that suits us. Decisions do not get made, they happen.” Zaltman has spent years studying how executives and managers represent their ideas and apply those in business context. One of the most common and damaging mistake that managers and executives are found to make, is to treat facts as insights and leap directly from data to action. In the journal of advertising research, he wrote that it is common when research is used to prove points rather than being used as fuel for imaginative insights.

Corporate executives are often found to be defending themselves on objectivity of their decisions. Zaltman says, “It might look like a leader has made a big decision- A versus B-when in fact, in all the layers that led up to that decision, the data has been increasingly skewed toward A. A leader may think that he has made a leap of faith based on clear data, but in reality, it is already been kind of pre-ordained.” No wonder, why ninety percent of new products and offerings are unable produce results, anything close to what expected.

Sometime back, H. Thomas Johnson and Robert S. Kaplan convincingly proved that in cases of many companies, numbers representing revenues, costs and profits in their financial statements are the result of processes of estimation, negotiation, debate and politics in allocating overhead costs that can produce grossly inaccurate reflections of true cost and profit. For all the time that senior leaders spend analyzing data, they should be making equal investment to determine what data should be created in the first place.

All this market cacophony and ambiguity, even after achieving a state of data abundance, probably needs to be viewed through a different lens. The solution is simple, as suggested by Lange and Puntoni; Instead of finding a purpose for data, find data for a purpose. They call this approach as decision-driven data analytics.

Mere ‘Data-Driven’ often means answering the wrong question. A recent research conducted by professors at Wharton school in collaboration with the customer experience consultancy, the Verde group, reveals an important data related downside of customer loyalty programs. The researchers surveyed more than 5000 American retail consumers in February 2020 and an additional 2500 in the month of May, after the COVID-19 pandemic took hold. The findings of the research drew attention of the retailers towards facts which were unwillingly ignored due to blind data dependency. It suggested that, as the retailers audit their customers’ journey, they should avoid the temptation to blindly focus on the most frequent service failures depicted by captured data. As, the most common sources of friction aren’t necessarily the ones that are most damaging and causing the most loyal customers to depart. The survey revealed no overlap between the ten most frequent service failures and the ten that caused the biggest erosion or dent in loyalty.

To be data-driven or decision-driven, the distinction might seem trivial, but it’s not. As, Stefano Puntoni says, “decision-making culture isn’t as cut-and-dry as saying Boston is a better sports town than New York”. Decision-driven thinking approaches organizational opportunities and challenges in distinctly different way than data-driven approach. It starts with questions, not data. We must remember that it the decision-maker who lead any project, not the data scientists. Decision-driven approach contemplate more on unknowns than knowns. It looks wide first, then deep dives, unlike data-driven approach which often first dives into the pool of data already captured. Starting with questions would accelerate fathoming of missing data. Decision-led teams tend to be more diverse, which naturally helps to root out bias by questioning assumptions.

And as Professor Puntoni asserts, “Data-driven thinking starts with historical data that accounts for what already happened. While the past might be prologue, pre-pandemic patterns might not still apply.



Dr. Mrinmoy Bhattacharjee
Associate Professor of Marketing
Alliance Business School