There are three kinds of lies: lies, damned lies, and statistics.
How To Lie With Statistics was first published in 1954. I picked it up after seeing the book on Bill Gates’ summer reading list. Weighing in at only 142 pages, it allowed me to quickly inch a bit closer to my (perhaps overly ambitious) goal of reading 1-2 books per week.
The author, Darrell Huff, reminds us how easily math can be manipulated. In many ways, it is more relevant than ever given the volume of statistics and infographics thrown our way via the likes of Facebook, Twitter, etc.
The secret language of statistics, so appealing in today’s fast, fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify.
While not all of the statistical information that we come across can be tested with the sureness of scientific analysis, we can dig a bit deeper by asking five simple questions, outlined by Huff, and summarized below:
- Who Says So? About the first thing to look for is bias—the laboratory with something to prove for the sake of a theory, a reputation, or a fee; the newspaper whose aim is a good story; labor or management with a wage level at stake.
- How Does He Know? Watch out for evidence of a biased sample, one that has been selected improperly or has selected itself. Ask the question: Is the sample large enough to permit any reliable conclusion? Similarly with a reported correlation: Is it big enough to mean anything? Are there enough cases to add up to any significance?
- What’s Missing? You won’t always be told how many cases. The absence of such a figure, particularly when the source is an interested one, is enough to throw suspicion on the whole thing. Similarly a correlation given without a measure of reliability is not to be taken very seriously.
- Did Somebody Change the Subject? When assaying a statistic, watch out for a switch somewhere between the raw figure and the conclusion. One thing is all too often reported as another.
- Does It Make Sense? “Does it make sense?” will often cut a statistic down to size when the whole rigmarole is based on an unproved assumption. Many a statistic is false on its face. It gets by only because the magic of numbers brings about a suspension of common sense.
Image courtesy of: www.xkcd.com.
