The new announcement that the Census Bureau is completely changing its Current Population Survey (CPS) questions about health insurance coverage (see previous post here) is devastating for those of us who do health research.
We have all known for years — decades — that the CPS count of the insured isn’t especially accurate. The questions it asks are about full-year coverage but people tend to answer based on their current status. It chronically under-reports Medicaid enrollment — the actual head count from Medicaid programs is always higher than indicated in the survey. The same is probably true for employment-based coverage. It has often been criticized for being weak on foreign language questions. Massachusetts, for example, has a significant population of people who speak Portuguese and that state thought the CPS failed to capture those people.
The Census Bureau recently (in 2007) revised its numbers because the software was misallocating people who reported that everyone in their family was covered. More on this below.
But despite all this, the CPS numbers were very useful. It is an enormous survey of 78,000 households, and since it also asks about employment and income, it is possible to look at very detailed demographic categories. The error rate appears to occur throughout the data, it is not concentrated in any one demographic group, so it is not a factor in comparing sub-groups.
And most importantly, it has been going on since 1987, so it is possible to measure changes over time — during, before, and after recessions, before and after new initiatives like the SCHIP program.
And the state-by-state numbers are invaluable. Want to know how Massachusetts fared under RomneyCare? The CPS is the only place to go.
To drop everything that has gone before in favor of a brand new set of questions is unprecedented. And to do so at the very moment of the biggest revision of health care in American history is completely irresponsible.
We simply will not be able to compare before and after ObamaCare, at least not based on the CPS. This is a tragedy.
It’s not like the CPS hasn’t been revised before. A short paper explaining the 2007 revision lists the more significant changes over the years, including-
- Converting from paper to digital questionnaires in 1994.
- Adding child-specific questions in 1995.
- Adding “verification” questions for people who claimed no insurance in 2000.
The 2007 revision explains how the Bureau did not just stop using one data set and switch to another. Instead, it went back two years and revised the numbers for 2005 and 2007 and provided instruments to allow, “advanced users the ability to approximate the correction for 1997 through 2004.” Both sets of numbers were available for these years to maintain the integrity of the trend lines.
If the Bureau were being responsible it would run the old questionnaire alongside the new one for at least three years, so we could measure the effect of ObamaCare independently of the effect of the new survey.
That it has chosen not to do this only heightens the suspicion of a political agenda in play and further degrades the reliability and trustworthiness of anything that comes out of the federal bureaucracy in the Obama era.