Never Fall in Love with Data, at Least, Not at First Sight!

"It is very lonely up there." When they reach the top, company directors face the enormous challenge of making tough decisions. They obviously make them based on their own knowledge of the business, but they also take into consideration the perspectives and opinions of their team. So, what happens when team members have different views and/or interests? What happens when the team doesn't dare to challenge the leader?

Even when interests are aligned, the ways of reaching a conclusion often differ. I cannot recall a single planning meeting or review of results where there were no discrepancies in the interpretation of the data. For example, the sales leader at the country level has a different expectation in his sales forecast from the product leader at the regional level. Who should be believed if the data for both come saturated with graphs and supported by bulletproof Excel sheets?

This example, which I like to share with my EGADE Business School MBA students, illustrates a leader's dilemma. Suppose a report shows that product returns from the previous month represented 1% of total billing. The business leader may consider this to be a reasonable revenue loss. However, curiosity leads him/her to ask a question that will provide new insight: which product was returned most? This question leads to another report that shows the defective products customers returned ordered by revenue (negative, of course). Despite being nothing out of the ordinary, intuition leads him/her to also order the list by number of units returned. The product with the highest number of returns (top offender) is a product that did not appear in the report ordered by revenue. Why? It turns out that the most returned product was a gift (of poor quality) that was added to many purchases and that was assumed as an advertising campaign cost.

The real issue here was not the impact of the returns on revenue, but the dissatisfied customers who went to the trouble of returning the product. The leader should then ask whether there is a correlation between returns and subsequent purchases from those customers. The answer is obvious: the vast majority of the customers who returned products did not purchase from the company again.

This case can also illustrate a conflict of interest or complicity between the marketing manager who approved the campaign, the sourcing manager who bought the gifts, and even the operations manager who wants to show better planning and control. Regardless of the cause, the leader should define the actions required to win back the lost clients. Even more importantly, he/she should think about how to move from the description to the prescription, and determine whether the company’s system can act automatically when it detects that a customer does not buy again, by, for example, sending an aggressive discount on the next purchase.

Cases like these happen every day on both small and large scales. For example, on the basis of a report showing that the best results on the SAT academic aptitude test came from graduates of small schools, the Gates Foundation invested millions of dollars in transforming and creating schools to ensure personalized education. Several months and millions later, the data revealed a contradiction: the worst SAT scores came from small schools as well. Someone fell in love with the first part of the initial report but did not look into the second.

Another example. You wouldn't doubt a statistic that confirms that apple pie is America's favorite dessert. The economist Walter Oi published a study showing that this preference is real, but only when it is offered in family size. When offered in an individual serving, the consumer chooses any flavor other than apple. Why? The family version has to be shared and it is a flavor (after the individual favorite) that everyone usually likes.

Many leaders expect someone else to detect data inconsistencies. But in reality, regardless of whether or not they have a Rasputin who can order their lives, it will be their curiosity and knowledge about how the data were organized and presented that will make them believe in it or not, and use it to change course or transform their organization.

Are you the type of leader who uses data extensively to make decisions? Just one more tip: make sure your team has the confidence to challenge you with data. Leaders who do not admit questioning to their way of thinking will only receive data models that support what they have in mind. Ronald Coase, the great British economist, used to say: "Torture the data enough and they will tell you what you want to hear."

If your team members cannot challenge you with data, or if they have tried and failed, they will bend over backwards to make data models and PowerPoint charts to match your expectations. This apparent harmony will probably make everyone enjoy the apple pie you brought to the meeting, but it will distance your organization from the innovation and change you are looking for and, most likely, make you fall into the networks of a wrong statistic, a repeated idea... a love from the past.

Originally published in Forbes.