Sometimes it’s hard to know what statistics are worthy of trust. But we shouldn’t count out stats altogether … instead, we should learn to look behind them. In this delightful, hilarious talk, data journalist Mona Chalabi shares handy tips to help question, interpret and truly understand what the numbers are saying. (Ted official site)
Who is she?:
Chalabi is currently the Data Editor of the Guardian US, where she writes articles, produces documentaries and turns data into illustrations and animations. In 2016, her data illustrations were commended by the Royal Statistical Society.(Op cit)
Why you should view:
After working for a humanitarian organization, Mona Chalabi saw how important data was, but also how easily it could be used by people with their own specific agendas. Since then, her work for organizations like Transparency International and The Guardian has had one goal: to make sure as many people as possible can find and question the data they need to make informed decisions about their lives.(Op cit)
- About 4 out of 10 Americans distrust the economic data that gets reported by government. Among supporters of President Trump it’s even higher; it’s about 7 out of 10. What does she think this means?
- Statistics come from the state; that’s where they got their name. The point was to better measure the population to better serve it. We need these government numbers, but we also have to move beyond either blindly accepting or blindly rejecting them. We need to learn the skills to be able to spot bad statistics. What are her suggestions?
- So, question number one is: Can you see uncertainty? the second question that you guys should be asking yourselves to spot bad numbers is: Can I see myself in the data? This question is also about averages in a way, because part of the reason why people are so frustrated with these national statistics, is they don’t really tell the story of who’s winning and who’s losing from national policy. It’s easy to understand why people are frustrated with global averages when they don’t match up with their personal experiences. the third and final question that I want you guys to think about when you’re looking at statistics is: How was the data collected? So far, I’ve only talked about the way data is communicated, but the way it’s collected matters just as much. I know this is tough, because methodologies can be opaque and actually kind of boring, but there are some simple steps you can take to check this. Explain the steps she outlines.
- Private companies don’t have a huge interest in getting the numbers right, they just need the right numbers. Government statisticians aren’t like that. In theory, at least, they’re totally impartial, not least because most of them do their jobs regardless of who’s in power. They’re civil servants. And to do their jobs properly, they don’t just speak to a couple hundred people. Those unemployment numbers I keep on referencing come from the Bureau of Labor Statistics, and to make their estimates, they speak to over 140,000 businesses in this country. Do you believe her case for government impartiality?
- As a student of psychology you will see lots of statistics. Do you feel your training thus far has given you sufficient training in analyzing a given data set? Explain your answer to someone.
The U.S. unemployment rate: I refer to this statistics from the Bureau of Labor Statistics throughout. You can find their data here.
“Four out of ten Americans now distrust the economic data that gets reported by the government.”
On page 30 of this pdf, you’ll see how poll results typically get reported. There’s a lot to dig into! To find out how many distrust the data, I added together those that say they somewhat distrust it (18.6%) and those that said they don’t trust it at all (25.0%). That adds up to 43.6% which I rounded down to say “four out of ten”. You can see how easily numbers can be manipulated can’t you? I could have chosen only the more extreme of those groups and rounded down to say “two in ten Americans now distrust the economic data that gets reported by the government.”
Here’s the link to the bill I mention that’s in Congress right now. It’s known as the Local Zoning Decisions Protection Act of 2017.
I mention 230 million eligible voters. That estimate comes from here, from the United States Elections Project – the voting age population is about 250 million people but citizens can be excluded for various reasons (you can find out more about that here.
Reader questions about data:
Here’s a list of some of the questions I get.
Looking beyond the average:
The animated chart on the unemployment rate used data about education status from here, on sex here. There are also huge racial differences in unemployment. I collected the data on that – and you can view it in this online spreadsheet. You can find out about the full BLS methodology here (including those businesses I said they interview).
L’Oréal was the company which only spoke to 48 women to make their claims about their skin care products.
Speaker’s reading list
This article provides some great historical context – about why governments first started collecting statistics and why they’re still so important today.
Although it doesn’t use data, Martin’s work is inspiring because it’s so meditative and peaceful. Her art uses lines and dots to explore themes like friendship and sadness – things we don’t necessarily associate with structure but still provide order to our lives.
This is a timeless classic – the lovely little illustrations are a great reminder that every concept can be visualized one way or another.
If you feel totally confused by a chart, it’s probably the chats fault, not yours. This piece (which was also a great talk you can see here) provides some great context about why some charts make sense and others don’t.