Big data has been a hotly discussed topic for a while now, but 2014 will see the big data debate move from hype to application. In 2013 one of the thorniest issues has been the practical problem of extracting information from complex data sets.
In 2014 we expect some of these logistical issues to be resolved as the investment agencies and brands have made in processes and technology comes to fruition. However, the increasing volume of data could bring dangers if it is not handled properly.
An increase in available data raises dangers of marketers being distracted by irrelevant data, of seeking out data that backs up their existing beliefs and becoming irrationally over-confident in their predictive abilities.
The first issue is that with so much data available marketers can be led to focus on unimportant, irrelevant data rather than the key metrics. This problem has been demonstrated in a number of behavioural experiments. The psychologist Andreassen has shown, in stock-picking experiments with MIT students, that increased levels of data can actually reduce performance.
A similar result has been found by Groves amongst teachers – more data leads to worse predictions about students’ performance. In both cases the additional information, even when participants know it’s irrelevant, is distracting.
A second, more pernicious, problem may be that marketers, and people in general, don’t get distracted by irrelevant data in a random fashion. The data that we seek out tends to fit our existing beliefs. An Ohio State study in 2009, for example, found that people spend over 36% more time reading an essay if it aligns with their opinions. Furthermore, we tend to interpret that evidence in a way that fits our beliefs.
In a classic experiment by Hastorf and Cantril Princeton and Dartmouth students were shown footage of a controversial football match between the two universities. When asked to count the number of fouls both sides “saw” many more fouls by the opposing side. The same data had markedly different interpreations.
With a greater abundance of data the issue is exacerbated as it forces a greater degree of selectiveness upon us. As we become more selective it becomes easier to unwittingly pick the data that we like and ignore the rest. So greater access to data on a real time basis may not move us closer to business or brand truths.
It may just make us more certain that our existing approach is the correct one. These concerns have remarkably old roots. In the 16th century Shakespeare’s Caesar summed our situation up: “men may construe things after their fashion / clean from the purpose of the things themselves.”
Whilst it is unclear if our insight automatically grows with an increasing volume of data it seems our confidence certainly does. Over-confidence in our abilities has plagued predictions for a while. Philip Teltlock ran one of the longest prediction experiments ever when he collected the predictions of 284 professional and economic forecasters.
Across 20 years he collated 82,361 forecasts. The results were fascinating. Despite the profusion of data available to these experts their forecasts were barely better than those of laymen. Events which the scientists said had a zero probability of occurring had a 15% chance of happening.
Yet despite the evidence that data doesn’t necessarily boost our predictive ability our confidence grows in line with the volume of data. This is creating a dangerous mismatch between our predictive abilities and our expectations.
We should not, however, be overly pessimistic. The promise of Big Data in enhancing our insight and predictive abilities may well come to fruition. However, we must overcome not just technological hurdles but also more fundamental psychological ones. This is not an impossible task.
We need to be aware of the biases that afflict us and have a structured plan in place to remove distracting data and force ourselves to consider contradictory information. For the brands and agencies that commit to these principles there is a marked competitive advantage.
