Forums

[SlaveNoMore]

Oh, joy!:

Media map gives wrong directions on incomes
Stuart Buck and Megan McArdle:

WASHINGTON - It often gets dicey for readers when journalists, who are rarely math majors, play with numbers and then publish misleading or mistaken conclusions. It happened Labor Day when the Detroit Free Press published a horrifying map showing huge losses in household income across America. Horrifying and totally wrong, that is.

According to the map, between 1999 and 2005 median household income had fallen in 46 states, sometimes by double digits, plunging by 6 percent in the U.S. as a whole.

We knew incomes had fallen slightly since the peak of the technology bubble. Labor markets have remained soft, and the cost of non-wage compensation such as health care, which is not included in income calculations, has been soaring. But the declines shown on the map were shocking. The Free Press claimed that nine states, with a total of 75 million citizens, had seen median incomes plummet by roughly 10 percent.

More surprisingly, these figures didn’t match those in the Census Bureau’s Current Population Survey, or CPS, which showed that median household income in the US had fallen only 2.8 percent — and had risen in around 20 states, not four. Where, we wondered, had they gotten their figures?

An e-mail exchange with the journalists gave us the answer: They had taken their 2005 numbers not from the CPS, but from the American Community Survey, a new research product that is scheduled to replace the detailed “long form” census collected every decade. But they hadn’t taken the 1999 figures from the ACS — in fact, the ACS is so new that it didn’t even publish nationwide data for 1999. Instead, the journalists had taken the 1999 income figures from the official 2000 census.

Some statisticians already will be shaking their heads in dismay; different surveys, taken at different times and asking slightly different questions, often produce very different pictures of the economy. If the journalists had checked the helpful section of the Census Bureau Web site called “Using the Data”, they would have discovered this warning: “Users should exercise care when comparing income figures from the American Community Survey with those of Census 2000.”

They might also have found another Census Web page warning that “[E]stimates from any one survey will almost never exactly match the estimates from any other (unless explicitly controlled), because of differences such as in questionnaires, data collection methodology, reference period, and edit procedures.” Or had they Googled “Comparing the ACS and the Census,” they’d have discovered a helpful document on the comparison problems, available from multiple state governments. It calls the two income numbers “not comparable.”

With good reason. A 2003 report by census staff indicated that median incomes from the ACS were much lower than those from the 2000 census: 4.4 percent lower for the United States.

What does this mean? Simple: If you start with income from the 2000 census, and then compare it to income from the 2005 ACS, which we know tends to be much lower because of survey differences — you’ll find a much greater decline than really was the case. Much of the reported 6 percent drop — probably more than half — comes from comparing apples to oranges.

When we wrote this up on our blogs, many conservative readers attributed the misleading figures to liberal media bias. But it is more likely ignorance than malice. Every year, scores of fledgling journalists pour out of liberal arts programs. Though many will need to pick through mountains of statistics in search of the truth, few have been taught the skills to do it.

They quickly become victims of advocacy groups pushing skewed statistics. Through ignorance, they may also start manufacturing their own flawed numbers. Since number-crunching beats (such as business and finance) are generally viewed as a tedious waystation en route to more interesting beats, few are enthusiastic about developing these skills. And their editors may not be in any position to help them.

The problem is compounded by the fact that journalists who do know how to read a balance sheet, run a regression, or analyze economic data, can generally get a job that pays a lot more than journalism. Some stay in the field out of love for their work (journalism is a really great job), but in our experience some of the best flee to greener pastures.

Even worse, as mathematician John Allen Paulos is fond of pointing out, Americans are often too innumerate to analyze statistics printed in the newspaper. America’s schools haven’t given its citizens any more ability than its journalists to analyze the information that floods our lives. We would call it a case of the blind leading the blind, but the comparison is inappropriate. Blind people know they can’t see.

Stuart Buck, an attorney whose articles have appeared in the Harvard Law Review, who blogs at The Buck Stops Here.

Megan Mcardle is a journalist who covers economics and blogs at Asymmetrical Information

link