It’s naive to assume that quantitative analysis is inherently a more reliable or scientific than human judgment. Quantitative data, too, is the product of bias.
Choices must be made about why, how, and what data is selected, collected, and weighed. Human beings are responsible for establishing the criteria for what “counts” from among the data available, and may be biased toward data that is easy to find and measure. As well, each analyst brings their own perspective to their interpretation of the data. Two analysts may well interpret the same date set in very different ways.
In some cases, data is collected not because it is meaningful, but because it is consistent with other data that has been collected in the past. Moreover, quantified data is shallow: it strips knowledge of rich context. When used to evaluate human behaviour, for instance, there will always remain a gap between what actually happened and what can be measured. It's not effective at explaining human behaviour—it tells us what, not why. In other words, it's a kind of lossy compression. Finally, data can only ever measure what's happened in the past. It can't predict the future, and even the most elegant predictive model can be derailed by a "black swan" event.
According to Madsbjerg, too narrow a focus on quantitative research can inhibit our ability to have a strong intuition about people and culture.
This isn't to say that quantitative analysis is not without merit; however, it must be undertaken with the understanding that its emphasis on scale and causation may come at the cost of coherence and focus.
Related
- quant-data-is-lossy
- qualitative-and-quantitative-research-represent-different-philosophies-of-knowledge
