Today, we arguably collect too much data. It's all around us. Our every movement, every click, is tracked and measured; we quantify the quality of our sleep, how much exercise we get, and how much time we spend looking at our phones. Never mind the copious data being collected by various devices connected to the internet of things. Data is cheap. It's now the detritus of our digital lives, or, as Davis and Davidson put it, "data exhaust." As Sam Ladner argues, we spend more time collecting data than asking good questions of it.
This surfeit of information convinces us that all our answers lie within the numbers. But, this information often conceals as well as creates knowledge. It's not as clear cut as we might like to think. And so, we run into problems, trying to apply "narrow" data to "wide" problems. We look to the numbers for revelation, but we risk overestimate the meaning in the measures. Data gathered from analytics can't, for instance, tell us about intention. That takes interpretation—and interpretation can vary between analysts.
Data that are easiest to come by—and likely more abundant—are more likely to be lagging indicators; basing strategic decisions around lagging indicators is risky as it forces us into a reactive mode and means we may respond to change too late.
- Metric fixation is a symptom of a decline in social trust: metrics become a substitute for meaningful interpersonal relationships within organizations and in society more general
- Quant data is lossy suggests that quantitative data is highly compresses and loses information in the process of abstraction
- Quantitative analysis is not inherently more reliable than qualitative data argues that in spite of feeling more concrete quantitative data isn't inherently "better" than qualitative data
Kay, John, and Mervyn King. Radical Uncertainty: Decision-Making Beyond the Numbers. New York, NY: WW Norton, 2020.
Ladner, Sam. Mixed Methods: A Short Guide to Applied Mixed Methods Research. Sam Ladner, 2019.
Sutherland, Rory. Alchemy: The Dark Art and Curious Science of Creating Magic in Brands, Business, and Life. Illustrated edition. William Morrow, 2019.
Adapted from David Drysdale's Moby Diction notes.